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Foreword 


My first introduction to robotics came via a phone call 
in 1964. The caller was Fred Terman, the author of 
the world-famous Radio Engineer’s Handbook, who 
was at the time Provost of Stanford University. Dr. 
Terman informed me that a computer science profes¬ 
sor, John McCarthy, had just been awarded a large 
research grant, part of which required the develop¬ 
ment of computer-controlled manipulators. Someone 
had suggested to Terman that it would be prudent if 
the mathematically oriented McCarthy had some con¬ 
tact with mechanical designers. Since I was the only 
one on the Stanford faculty whose specialty was mecha¬ 
nism design, Terman decided to phone me, even though 
we had never met and I was a young assistant profes¬ 
sor fresh out of graduate school with only 2 years at 
Stanford. 

Dr. Terman’s phone call led me to a close associa¬ 
tion with John McCarthy and the Stanford Artificial In¬ 
telligence Laboratory (SAIL) that he founded. Robotics 
became one of the pillars of my entire academic career, 
and I have maintained my interest in teaching and re¬ 
searching the subject through to the present day. 

The modern history of robotic manipulation dates 
from the late 1940s when servoed arms were developed 
in connection with master-slave manipulator systems 
used to protect technicians handling nuclear materials. 
Developments in this area have continued to the present 
day. However, in the early 1960s there was very little 
academic or commercial activity in robotics. The first 
academic activity was the thesis of H. A. Ernst, in 1961, 
at MIT. He used a slave arm equipped with touch sen¬ 
sors, and ran it under computer control. The idea in his 
study was to use the information from the touch sensors 
to guide the arm. 

This was followed by the SAIL project and a simi¬ 
lar project started by Professor Marvin Minsky at MIT, 
which were the only sizeable academic ventures into 
robotics at that time. There were a few attempts at com¬ 
mercial manipulators, primarily in connection with part 
production in the automotive industry. In the USA there 
were two different manipulator designs that were being 
experimented with in the auto industry; one came from 
American Machine and Foundry (AMF) and the other 
from Unimation, Inc. 

There were also a few mechanical devices devel¬ 
oped as hand, leg, and arm prosthetics, and, a bit later, 
some exoskeletal devices to enhance human perfor¬ 
mance. In those days there were no microprocessors. 
So, these devices were either without computer control. 


or tethered to a remote so-called 
minicomputer, or even a mainframe 
computer. 

Initially, some in the computer 
science community felt that comput¬ 
ers were powerful enough to con¬ 
trol any mechanical device and make 
it perform satisfactorily. We quickly 
learned that this was not to be 
the case. We started on a twofold 
track. One was to develop particu¬ 
lar devices for SAIL, so that hard¬ 
ware demonstrations and proof-of- Bernard Roth 
concept systems were available for Professor of 
the fledgling robotics community to Mechanical Engineering 
... . , Stanford University 

experiment with, lhe other track, 1 

which was more or less moonlighted from the work at 
SAIL, was the development of a basic mechanical sci¬ 
ence of robotics. I had a strong feeling that a meaningful 
science could be developed, and that it would be best to 
think in terms of general concepts rather than concen¬ 
trate exclusively on particular devices. 

Fortuitously, it turned out that the two tracks sup¬ 
ported each other very naturally and, most importantly, 
the right students were interested in doing their re¬ 
search in this area. Hardware developments proved to 
be specific examples of more general concepts, and the 
students were able to develop both the hardware and the 
theory. 

Originally, we purchased an arm in order to get 
started quickly. A group at Rancho Los Amigos Hos¬ 
pital, in Los Angeles, was selling a tongue-switch- 
controlled motor-driven exoskeleton arm to assist pa¬ 
tients without muscular control of their arms. We pur¬ 
chased one of these, and connected it to a time-shared 
PDP-6 computer. The device was named Butterfin¬ 
gers', it was our first experimental robot. Several films 
demonstrating visual feedback control, block stacking 
tasks, and obstacle avoidance were made with Butterfin¬ 
gers as the star performer. 

The first manipulator that we designed on our own 
was known simply as the Hydraulic Arm. As its name 
implies, it was powered by hydraulics. The idea was 
to build a very fast arm. We designed special rotary 
actuators, and the arm worked well. It became the ex¬ 
perimental platform for testing the first ever dynamic 
analysis and time-optimal control of a robotic arm. 
However, its use was limited since the design speeds 
were much faster than required due to the limitations 


of the computational, planning, and sensing capabilities 
that were common at that time. 

We made an attempt to develop a truly digital 
arm. This led to a snake-like structure named the Orm 
(the Norwegian word for snake.) The Orm had several 
stages, each with an array of inflatable pneumatic actu¬ 
ators that were either fully extended or fully contracted. 
The basic idea was that, even though only a finite num¬ 
ber of positions in the workspace could be reached, 
these would be sufficient if there were a large number 
of positions. A small prototype proof-of-concept Orm 
was developed. It led to the realization that this type of 
arm would not really serve the SAIL community. 

The first truly functional arm from our group was 
designed by Victor Scheinman, who was a graduate stu¬ 
dent at the time. It was the very successful Stanford 
Arm, of which over ten copies were made as research 
tools to be used in various university, government, and 
industrial laboratories. The arm had six independently 
driven joints; all driven by computer-controlled ser- 
voed, DC electric motors. One joint was telescoping 
(prismatic) and the other five were rotary (revolute). 

Whereas the geometry of Butterfingers required an 
iterative solution of the inverse kinematics, the geo¬ 
metric configuration of the Stanford Arm was chosen 
so that the inverse kinematics could be programmed 
in any easy-to-use time-efficient closed form. Further¬ 
more, the mechanical design was specifically made to 
be compatible with the limitations inherent in time- 
share computer control. Various end-effectors could be 
attached to act as hands. On our version, the hand 
was in the form of a vise-grip jaw, with two slid¬ 
ing fingers driven by a servoed actuator (hence, a true 
seventh degree of freedom). It also had a specially 
designed six-axis wrist force sensor. Victor Schein¬ 
man went on to develop other important robots: the 
first was a small humanoid arm with six revolute 
joints. The original design was paid for by Marvin 
Minsky at the MIT AI Lab. Scheinman founded Vi- 
carm, a small company, and produced copies of this 
arm and the Stanford Arm for other labs. Vicarm later 
became the West Coast Division of Unimation, Inc., 
where Scheinman designed the PUMA manipulator 
under General Motors sponsorship through Unima¬ 
tion. Later, for a company called Automatix, Schein¬ 
man developed the novel Robot World multirobot sys¬ 
tem. After Scheinman left Unimation, his colleagues 
Brian Carlisle and Bruce Shimano reorganized Unima- 
tion’s West Coast Division into Adept, Inc., which to 
this day is the largest US manufacturer of assembly 
robots. 

Quickly, the modern trend of carefully detailed me¬ 
chanical and electronic design, optimized software, and 


complete system integration became the norm; to this 
day, this combination represents the hallmark of most 
highly regarded robotic devices. This is the basic con¬ 
cept behind mechatronic, a word conied in Japan as 
a concatenation of the words mechanics and electronics. 
Mechatronics that relies on computation is the essence 
of the technology inherent in robotics as we know it 
today. 

As robotics developed around the world, a large 
number of people started working on various aspects, 
and specific subspecialties developed. The first big di¬ 
vision was between people working on manipulators 
and those working on vision systems. Early on, vi¬ 
sion systems seemed to hold more promise than any 
other method for giving robots information about their 
environment. 

The idea was to have a television camera capture 
pictures of objects in the environment, and then use 
algorithms that allowed the computer images of the pic¬ 
tures to be analyzed, so as to infer required information 
about location, orientation, and other properties of ob¬ 
jects. The initial successes with image systems were 
in problems dealing with positioning blocks, solving 
object manipulation problems, and reading assembly 
drawings. It was felt that vision held potential for use 
in robotic systems in connection with factory automa¬ 
tion and space exploration. This led to research into 
software that would allow vision systems to recognize 
machine parts (particularly partially occluded parts, as 
occurred in the so-called bin-picking problems) and 
ragged-shaped rocks. 

After the ability to see and move objects became 
established, the next logical need had to do with plan¬ 
ning a sequence of events to accomplish a complex task. 
This led to the development of planning as an impor¬ 
tant branch in robotics. Making fixed plans for a known 
fixed environment is relatively straightforward. How¬ 
ever, in robotics, one of the challenges is to let the 
robot discover its environment, and to modify its ac¬ 
tions when the environment changes unexpectedly due 
to errors or unplanned events. Some early landmark 
studies in this area were carried out using a vehicle 
named Shakey, which, starting in 1966, was developed 
by Charlie Rosen’s group at the Stanford Research In¬ 
stitute (now called SRI). Shakey had a TV camera, 
a triangulating range finder, bump sensors, and was 
connected to DEC PDP-10 and PDP-15 computers via 
radio and video links. 

Shakey was the first mobile robot to reason about 
its actions. It used programs that gave it the ability 
for independent perception, world modeling, and action 
generation. Low-level action routines took care of sim¬ 
ple moving, turning, and route planning. Intermediate- 


level actions combined the low-level ones in ways that 
accomplished more complex tasks. The highest level 
programs could make and execute plans to achieve 
high-level goals supplied by a user. 

Vision is very useful for navigation, locating ob¬ 
jects, and determining their relative positions and ori¬ 
entation. However, it is usually not sufficient for as¬ 
sembling parts or working with robots where there are 
environmental constraining forces. This led to the need 
to measure the forces and torques generated by the en¬ 
vironment, on a robot, and to use these measurements 
to control the robot’s actions. For many years, force- 
controlled manipulation became one of the main topics 
of study at SAIL, and several other labs around the 
world. The use of force control in industrial practice has 
always lagged the research developments in this area. 
This seems to be due to the fact that, while a high level 
of force control is very useful for general manipulation 
issues, specific problems in very restricted industrial en¬ 
vironments can often be handled with limited, or no, 
force control. 

In the 1970s, specialized areas of study such as 
walking machines, hands, automated vehicles, sensor 
integration, and design for hostile environments be¬ 
gan to develop rapidly. Today there are a large number 
of different specialties studied under the heading of 
robotics. Some of these specialties tire classical engi¬ 
neering subject areas within which results have been 
developed that have been particularized to the types 
of machines called robots. Examples here are kine¬ 
matics, dynamics, controls, machine design, topology, 
and trajectory planning. Each of these subjects has 
a long history predating the study of robotics; yet 
each has been an area of in-depth robotics research 
in order to develop its special character in regard to 
robotic-type systems and applications. In doing this 
specialized development, researchers have enriched the 
classical subjects by increasing both their content and 
scope. 

At the same time that the theory was being devel¬ 
oped, there was a parallel, although somewhat separate, 
growth of industrial robotics. Strong commercial de¬ 
velopment occurred in Japan and Europe, and there 
was also continued growth in the USA. Industrial as¬ 
sociations were formed (the Japan Robot Association 
was formed in March 1971, and the Robotic Industries 
Association (RIA) was founded in 1974 in the USA) 
and trade shows, together with application-oriented 
technical sessions, were introduced and held on a reg¬ 
ular basis. The most important were the International 
Symposium on Industrial Robots, the Conference on In¬ 
dustrial Robot Technology (now called the International 
Conference on Industrial Robot Technology), and the 


RIA annual trade show, which is now called the Inter¬ 
national Robots and Vision Show and Conference. 

The first regular series of conferences emphasizing 
research, rather than the industrial, aspects of robotics, 
was inaugurated in 1973. It was sponsored jointly 
by the International Center for Mechanical Sciences 
(CISM), based in Udine, Italy, and the International 
Federation for the Theory of Mechanisms and Ma¬ 
chines (IFToMM). (Although IFToMM is still used, its 
meaning has been changed to the International Feder¬ 
ation for the Promotion of Mechanism and Machine 
Science.) It was named the Symposium on Theory and 
Practice of Robots and Manipulators (RoManSy). Its 
trademark was an emphasis on the mechanical sciences 
and the active participation of researchers from East¬ 
ern and Western Europe as well as North America and 
Japan. It is still held biannually. On a personal note, it 
is at RoManSy where I first met each of the editors of 
this Handbook: Dr. Khatib in 1978 and Dr. Siciliano 
in 1984. They were both students: Bruno Siciliano had 
been working on his PhD for about one year, and Ous- 
sama Khatib had just completed his PhD research. In 
both cases, it was love at first sight! 

RoManSy was quickly joined by a host of other 
new conferences and workshops; today there are a large 
number of research oriented robotics meetings that take 
place through the year in many countries. Currently, 
the largest conference is the International Conference 
on Robotics and Automation (ICRA), which regularly 
draws well over 1000 participants. 

In the beginning of the 1980s, the first real text¬ 
book on robotic manipulation in the USA was written 
by Richard Lou Paul (Richard P. Paul, Robot Manipu¬ 
lators: Mathematics, Programming, and Control, The 
MIT Press, Cambridge, MA, 1981). It used the idea 
of taking classical subjects in mechanics and applying 
them to robotics. In addition there were several topics 
developed directly from his thesis research at SAIL. (In 
the book, many examples are based on Scheinman’s 
Stanford Arm.) Paul’s book was a landmark event in 
the USA; it created a pattern for several influential 
future textbooks and also encouraged the creation of 
specialized robotics courses at a host of colleges and 
universities. 

At about this same time, new journals were created 
to deal primarily with research papers in the areas re¬ 
lated to robotics. The International Journal of Robotics 
Research was founded in the spring of 1982, and three 
years later the IEEE Journal of Robotics and Automa¬ 
tion (now the IEEE Transactions on Robotics) was 
founded. 

As microprocessors became ubiquitous, the ques¬ 
tion of what is or is not a robot came more into 


play. This issue has, in my mind, never been success¬ 
fully resolved. I do not think a definition will ever be 
universally agreed upon. There are of course the sci¬ 
ence fiction creatures-from-outer-space varieties, and 
the robots of the theater, literature, and the movies. 
There are examples of imaginary robot-like beings that 
predate the industrial revolution, but how about more 
down-to-Earth robots? In my view the definition is es¬ 
sentially a moving target that changes its character with 
technological progress. For example, when it was first 
developed, a ship’s gyro auto-compass was considered 
a robot. Today, it is not generally included when we list 
the robots in our world. It has been demoted and is now 
considered an automatic control device. 

For many, the idea of a robot includes the concept 
of multifunctionality, meaning the device is designed 
and built with the ability to be easily adapted or re¬ 
programmed to do different tasks. In theory this idea 
is valid, but in practice it turns out that most robotic de¬ 
vices are multifunctional in only a very limited arena. 
In industry it was quickly discovered that a specialized 
machine, in general, performs much better than a gen¬ 
eral purpose machine. Furthermore, when the volume 
of production is high enough, a specialized machine can 
cost less to manufacture than a generalized one. So, spe¬ 
cialized robots were developed for painting, riveting, 
quasiplanar parts assembly, press loading, circuit board 
stuffing, etc. In some cases robots are used in such spe¬ 
cialized ways that it becomes difficult to draw the line 
between a so-called robot and an adjustable piece of 
fixed automation. Much of this practical unfolding is 
contrary to the dream of the pioneers in robotics, who 
had hoped for the development of general purpose ma¬ 
chines that would do everything, and hence sell in great 
enough volume to be relatively inexpensive. 

My view is that the notion of a robot has to do with 
which activities are, at a given time, associated with 
people and which are associated with machines. If a ma¬ 
chine suddenly becomes able to do what we normally 
associate with people, the machine can be upgraded in 
classification and classified as a robot. After a while, 
people get used to the activity being done by machines, 
and the devices get downgraded from robot to machine. 
Machines that do not have fixed bases, and those that 
have arm- or leg-like appendages have the advantage of 
being more likely called robots, but it is hard to think of 
a consistent set of criteria that fits all the current naming 
conventions. 

In actuality any machines, including familiar house¬ 
hold appliances, which have microprocessors directing 
their actions can be considered as robots. In addition to 
vacuum cleaners, there are washing machines, refrig¬ 
erators, and dishwashers that could be easily marketed 
as robotic devices. There are of course a wide range 


of possibilities, including those machines that have 
sensory environmental feedback and decision-making 
capabilities. In actual practice, in devices considered to 
be robotic, the amount of sensory and decision making 
capability may vary from a great deal to none. 

In recent decades the study of robotics has expanded 
from a discipline centered on the study of mechatronic 
devices to a much broader interdisciplinary subject. 
An example of this is the area called human-centered 
robotics. Here one deals with the interactions between 
humans and intelligent machines. This is a growing area 
where the study of the interactions between robots and 
humans has enlisted expertise from outside the clas¬ 
sical robotics domain. Concepts such as emotions in 
both robots and people are being studied, and older ar¬ 
eas such as human physiology and biology are being 
incorporated into the mainstream of robotics research. 
These activities enrich the field of robotics, as they in¬ 
troduce new engineering and science dimensions into 
the research discourse. 

Originally, the nascent robotics community was fo¬ 
cused on getting things to work. Many early devices 
were remarkable in that they worked at all, and little 
notice was taken of their limited performance. Today, 
we have sophisticated, reliable devices as part of the 
modern array of robotic systems. This progress is the 
result of the work of thousands of people throughout 
the world. A lot of this work took place in universi¬ 
ties, government research laboratories, and companies. 
It is a tribute to the worldwide engineering and scien¬ 
tific community that it has been able to create the vast 
amount of information that is contained in the 64 chap¬ 
ters of this Handbook. Clearly these results did not arise 
by any central planning or by an overall orderly scheme. 
So the editors of this handbook were faced with the dif¬ 
ficult task of organizing the material into a logical and 
coherent whole. 

The editors have accomplished this by organiz¬ 
ing the contributions into a three-layer structure. The 
first layer deals with the foundations of the subject. 
This layer consists of a single part of nine chapters 
in which the authors lay out the root subjects: kine¬ 
matics, dynamics, control, mechanisms, architecture, 
programming, reasoning, and sensing. These are the ba¬ 
sic technological building blocks for robotics study and 
development. 

The second layer has four parts. The first of these 
deals with robot structures', these are the arms, legs, 
hands, and other parts that most robots are made up of. 
At first blush, the hardware of legs, arms, and hands 
may look quite different from each other, yet they share 
a common set of attributes that allows them to all be 
treated with the same, or closely related, aspects of the 
fundamentals described in the first layer. 


The second part of this layer deals with sensing 
and perception, which are basic abilities any truly au¬ 
tonomous robotic system must have. As was pointed 
out earlier, in practice, many so-called robotic devices 
have little of these abilities, but clearly the more ad¬ 
vanced robots cannot exist without them, and the trend 
is very much toward incorporating such capabilities into 
robotic devices. The third part of this layer treats the 
subject areas associated with the technology of manip¬ 
ulation and the interfacing of devices. The fourth part of 
this layer is made up of eight chapters that treat mobile 
robots and various forms of distributed robotics. 

The third layer consists of two separate parts (a to¬ 
tal of 22 chapters) that deal with advanced applications 
at the forefront of today’s research and development. 
There are two parts to this layer; one deals with field 
and service robots, and the other deals with human- 
centered and lifelike robots. To the uninitiated observer, 
these chapters are what advanced robotics is all about. 
However, it is important to realize that many of these 


extraordinary accomplishments would probably not ex¬ 
ist without the previous developments introduced in the 
first two layers of this Handbook. 

It is this intimate connection between theory and 
practice that has nurtured the growth of robotics and 
become a hallmark of modem robotics. These two com¬ 
plementary aspects have been a source of great personal 
satisfaction to those of us who have had the opportu¬ 
nity to both research and develop robotic devices. The 
contents of this Handbook admirably reflect this com¬ 
plementary aspect of the subject, and present a very 
useful bringing together of the vast accomplishments 
which have taken place in the last 50 years. Certainly, 
the contents of this Handbook will serve as a valuable 
tool and guide to those who will produce the even more 
capable and diverse next generations of robotic devices. 
The editors and authors have my congratulations and 
admiration. 

Stanford, August 2007 Bernard Roth 


Foreword 


To open this Handbook and unfold the richness of its 
64 chapters, we here attempt a brief personal overview 
to sketch the evolution of robotics in its many aspects, 
concepts, trends, and central issues. 

The modern story of Robotics began about half 
a century ago with developments in two different di¬ 
rections. 

First, let us acknowledge the domain of me¬ 
chanical arms, ranging from teleoperated tasks on 
radiation-contaminated products to industrial arms, 
with the landmark machine UNIMATE - standing for 
uni(versal)mate. The industrial development of prod¬ 
ucts, mostly around the six-degree-of-freedom serial 
links paradigm and active research and development, 
associating mechanical engineering to the control spe¬ 
cialism, was the main driving force here. Of particular 
note nowadays is the successfully pursued effort to 
design novel application-optimized structures, using 
powerful sophisticated mathematical tools. In a simi¬ 
lar way, an important issue concerns the design and 
the actual building of arms and hands in the context of 
human-friendly robots for tomorrow’s cognitive robot. 

Second, and less well recognized, we should ac¬ 
knowledge the stream of work concerned with themes 
in artificial intelligence. A landmark project in this area 
was the mobile robot Shakey developed at Stanford In¬ 
ternational. This work, which aimed to bring together 
computer science, artificial intelligence, and applied 
mathematics to develop intelligent machines, remained 
a secondary area for quite some time. During the 1980s, 
building strength from many study cases encompassing 
a spectacular spectrum ranging from rovers for extreme 
environments (planet exploration, Antarctica, etc.), to 
service robots (hospitals, museum guides, etc.), a broad 
research domain arose in which machines could claim 
the status of intelligent robots. 

Hence robotics researches could bring together 
these two different branches, with intelligent robots cat¬ 
egorized in a solely computational way as bounded 
rationality machines, expanding on the 1980s third- 
generation robot definition: 

( robot)... operating in the three-dimensional world 
as a machine endowed with the capacity to interpret 
and to reason about a task and about its execution , 
by intelligently relating perception to action. 

The field of autonomous robots, a widely rec¬ 
ognized test-bed, has recently benefited from salient 
contributions in robot planning using the results of 


algorithmic geometry as well as of 
a stochastic framework approach ap¬ 
plied both to environmental mod¬ 
eling and robot localization prob¬ 
lems (SLAM, simultaneous localiza¬ 
tion and modeling), and further from 
the development of decisional pro¬ 
cedures via Bayesian estimation and 
decision approaches. 

For the last decade of the millen¬ 
nium, robotics largely dealt with the 
intelligent robot paradigm, blend¬ 
ing together robots and machine- 
intelligence generic research within 
themes covering advanced sensing 
and perception, task reasoning and 
planning, operational and decisional 
autonomy, functional integration architectures, intelli¬ 
gent human-machine interfaces, safety, and depend¬ 
ability. 

The second branch, for years referred to as non¬ 
manufacturing robotics, concerns a wide spectrum of 
research-driven real-world cases pertaining to field, 
service, assistive, and, later, personal robotics. Here, 
machine intelligence is, in its various themes, the cen¬ 
tral research direction, enabling the robot to act: 

1. Asa human surrogate, in particular for intervention 
tasks in remote and/or hostile environments 

2. In close interaction with humans and operating 
in human environments in all applications encom¬ 
passed by human-friendly robotics, also referred to 
as human-centered robotics 

3. In tight synergy with the user, expanding from 
mechanical exoskeleton assistance, surgery, health 
care, and rehabilitation into human augmentation. 

Consequently, at the turn of the millennium, 
robotics appears as a broad spectrum of research 
themes both supporting market products for well-en¬ 
gineered industrial workplaces, and a large number 
of domain-oriented application cases operating in haz¬ 
ardous and/or harsh environments (underwater robotics, 
rough-terrain rovers, health/rehabilitation care robotics, 
etc.) where robots exhibit meaningful levels of shared 
autonomy. 

The evolution levels for robotics stress the role of 
theoretical aspects, moving from application domains 
to the technical and scientific area. The organization of 
this Handbook illustrates very well these different lev- 
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els. Furthermore, it rightly considers, besides a body of 
software systems, front-line matters on physical appear¬ 
ance and novel appendages, including legs, arms, and 
hands design in the context of human-friendly robots 
for tomorrow’s cognitive robot. 

Forefront robotics in the first decade of the cur¬ 
rent millennium is making outstanding progress, com¬ 
pounding the strength of two general directions: 

• Short/mid-term application-oriented study cases 

• Mid/long-term generic situated research. 

For completeness, we should mention the large 
number of peripheral, robotics-inspired subjects, quite 
often concerning entertainment, advertising, and so¬ 
phisticated toys. 

The salient field of human-friendly robotics encom¬ 
passes several front-line application domains where the 
robots operate in a human environment and in close 
interaction with humans (entertainment and education, 
public-oriented services, assistive and personal robots, 
etc.), which introduces the critical issue of human- 
robot interaction. 

Right at the core of the field, emerges the forefront 
topic of personal robots for which three general charac¬ 
teristics should be emphasized: 

1. They may be operated by a nonprofessional user; 

2. They may be designed to share high-level decision 
making with the human user; 

3. They may include a link to environment devices and 
machine appendages, remote systems, and opera¬ 
tors; the shared decisional autonomy concept (co¬ 
autonomy) implied here unfolds into a large set of 
cutting-edge research issues and ethical problems. 

The concept of the personal robot, expanding to 
robot assistant and universal companion, is a truly great 
challenge for robotics as a scientific and technical field, 
offering the mid/long-term perspective of achieving 
a paramount societal and economical impact. This in¬ 
troduces, and questions, front-line topics encompassing 
cognitive aspects: user-tunable human-machine intel¬ 


ligent interfaces, perception (scene analysis, category 
identification), open-ended learning (understanding the 
universe of action), skills acquisition, extensive robot- 
world data processing, decisional autonomy, and de¬ 
pendability (safety, reliability, communication, and op¬ 
erating robustness). 

There is an obvious synergistic effort between the 
two aforementioned approaches, in spite of the neces¬ 
sary framework time differences. The scientific link not 
only brings together the problems and obtained results 
but also creates a synergistic exchange between the two 
sides and the benefits of technological progress. 

Indeed, the corresponding research trends and ap¬ 
plication developments are supported by an explosive 
evolution of enabling technologies: computer process¬ 
ing power, telecommunications, networking, sensing 
devices, knowledge retrieval, new materials, micro- and 
nanotechnologies. 

Today, looking to the mid- and long-term future, we 
are faced with very positive issues and perspectives but 
also having to respond to critical comments and loom¬ 
ing dangers for machines that are in physical contact 
with the user and may also be capable of unwanted, 
unsafe behavior. Therefore, there is a clear need to in¬ 
clude at the research level safety issues and the topic of 
multifaced dependability and the corresponding system 
constraints. 

The Handbook of Robotics is an ambitious and 
timely endeavor. It summarizes a large number of prob¬ 
lems, questions, and facets considered by 164 authors 
in 64 chapters. As such it not only provides an effi¬ 
cient display of basic topics and results obtained by 
researches around the world, but furthermore gives ac¬ 
cess to this variety of viewpoints and approaches to 
everyone. This is indeed an important tool for progress 
but, much more, is the central factor that will establish 
the two first decades of this millennium as the dawn of 
robotics, lifted to a scientific discipline at the core of 
machine intelligence. 

Toulouse, December 2007 Georges Giralt 


Foreword 


The field of robotics was born in the middle of the 
last century when emerging computers were altering 
every field of science and engineering. Having gone 
through fast yet steady growth via a procession of stages 
from infancy, childhood, and adolescence to adulthood, 
robotics is now mature and is expected to enhance the 
quality of people’s lives in society in the future. 

In its infancy, the core of robotics consisted of pat¬ 
tern recognition, automatic control, and artificial intel¬ 
ligence. Taking on these new challenge, scientists and 
engineers in these fields gathered to investigate novel 
robotic sensors and actuators, planning and program¬ 
ming algorithms, and architectures to connect these 
components intelligently. In so doing, they created arti¬ 
facts that could interact with humans in the real world. 
An integration of these early robotics studies yielded 
hand-eye systems, the test-bed of artificial intelligence 
research. 

The playground for childhood robotics was the 
factory floor. Industrial robots were invented and in¬ 
troduced into the factory for automating spraying, spot 
welding, grinding, materials handling, and parts assem¬ 
bly. Machines with sensors and memories made the 
factory floor smarter, and its operations more flexible, 
reliable, and precise. Such robotic automation freed hu¬ 
mans from heavy and tedious labor. The automobile, 
electric appliance, and semiconductor industries rapidly 
retooled their manufacturing lines into robot-integrated 
systems. In the late 1970s, the word mechatronics, orig¬ 
inally coined by the Japanese, defined a new concept 
of machinery, one in which electronics was fused with 
mechanical systems, making a wide range of indus¬ 
trial products simpler, more functional, programmable, 
and intelligent. Robotics and mechatronics exerted an 
evolutionary impact on the design and operation of 
manufacturing processes as well as on manufactured 
products. 

As robotics entered its adolescence, researchers 
were ambitious to explore new horizons. Kinematics, 
dynamics, and control system theory were refined and 
applied to real complex robot mechanisms. To plan and 
carry out real tasks, robots had to be made cognizant 
of their surroundings. Vision, the primary channel for 
external sensing, was exploited as the most general, ef¬ 
fective, and efficient means for robots to understand 
their external situation. Advanced algorithms and pow¬ 
erful devices were developed to improve the speed 
and robustness of robot vision systems. Tactile and 
force sensing systems also needed to be developed for 


robots to manipulate objects. Stud¬ 
ies on modeling, planning, knowl¬ 
edge, reasoning, and memorization 
expanded their intelligent proper¬ 
ties. Robotics became defined as 
the study of intelligent connection 
of sensing to actuation. This defini¬ 
tion covered all aspects of robotics: 
three scientific cores and one syn¬ 
thetic approach to integrate them. 

Indeed, system integration became 
a key aspect of robotic engineering 
as it allows the creation of lifelike Hirochika Inoue 
machines. The fun of creating such Professor Emeritus 
robots attracted many students to the The u n iversity of Tokyo 
robotics field. 

In advancing robotics further, scientific interest 
was directed at understanding humans. Comparative 
studies of humans and robots led to new approaches 
in scientific modeling of human functions. Cognitive 
robotics, lifelike behavior, biologically inspired robots, 
and a psychophysiological approach to robotic ma¬ 
chines culminated in expanding the horizons of robotic 
potential. Generally speaking, an immature field is 
sparse in scientific understanding. Robotics in the 1980s 
and 1990s was in such a youthful stage, attracting 
a great many inquisitive researchers to this new frontier. 

Their continuous explorations into new realms form the 
rich scientific contents of this comprehensive volume. 

Further challenges, along with expertise acquired 
on the cutting edge of robotics, opened the way to 
real-world applications for mature robotics. The early- 
stage playground gave way to a workshop for industrial 
robotics. Medical robotics, robot surgery, and in vivo 
imaging save patients from pain while providing doc¬ 
tors with powerful tools for conducting operations. New 
robots in such areas as rehabilitation, health care, and 
welfare are expected to improve quality of life in an ag¬ 
ing society. It is the destiny of robots to go everywhere, 
in the air, under water, and into space. They are ex¬ 
pected to work hand in hand with humans in such areas 
as agriculture, forestry, mining, construction, and haz¬ 
ardous environments and rescue operations, and to find 
utility both in domestic work and in providing services 
in shops, stores, restaurants, and hospitals. In a myr¬ 
iad of ways, robotic devices are expected to support 
our daily lives. At this point, however, robot appli¬ 
cations are largely limited to structured environments, 
where they are separated from humans for safety sake. 


In the next stage, their environment will be expanded 
to an unstructured world, one in which humans, as ser¬ 
vice takers, will always live and work beside robots. 
Improved sensing, more intelligence, enhanced safety, 
and better human understanding will be needed to pre¬ 
pare robots to function in such an environment. Not 
only technical but also social matters must be con¬ 
sidered in finding solutions to issues impeding this 
progress. 

Since my initial research to make a robot turn 
a crank, four decades have passed. I feel both lucky and 
happy to have witnessed the growth of robotics from its 
early beginnings. To give birth to robotics, fundamental 
technologies were imported from other disciplines. Nei¬ 
ther textbooks nor handbooks were available. To reach 
the present stage, a great many scientists and engineers 
have challenged new frontiers; advancing robotics, they 
have enriched this body of knowledge from a variety 
of perspectives. The fruits of their endeavors are com¬ 
piled in this Handbook of Robotics. More than 100 of 
the world’s leading experts have collaborated in produc¬ 
ing this publication. Now, people who wish to commit 
themselves to robotics research can find a firm founda¬ 


tion to build upon. This Handbook is sure to be used to 
further advance robotics science, reinforce engineering 
education, and systematically compile knowledge that 
will innovate both society and industry. 

The roles of humans and robots in an aging soci¬ 
ety pose an important issue for scientists and engineers 
to consider. Can robotics contribute to securing peace, 
prosperity, and a greater quality of life? This is still an 
open question. However, recent advances in personal 
robots, robotic home appliances, and humanoids sug¬ 
gest a paradigm shift from the industrial to the service 
sector. To realize this, robotics must be addressed from 
such viewpoints as the working infrastructure within 
society, psychophysiology, law, economy, insurance, 
ethics, art, design, drama, and sports science. Future 
robotics should be studied as a subject that envelops 
both humanity and technology. This Handbook offers 
a selected technical foundation upon which to advance 
such newly emerging fields of robotics. I look forward 
to continuing progress adding page after page of robot- 
based prosperity to future society. 

Tokyo, September 2007 Hirochika Inoue 


Foreword 


Robots have fascinated people for thousands of years. 
Those automatons that were built before the 20th cen¬ 
tury did not connect sensing to action but rather oper¬ 
ated through human agency or as repetitive machines. 
However, by the 1920s electronics had gotten to the 
stage that the first true robots that sensed the world 
and acted in it appropriately could be built. By 1950 
we started to see descriptions of real robots appearing 
in popular magazines. By the 1960s industrial robots 
came onto the scene. Commercial pressures made them 
less and less responsive to their environments but faster 
and faster in what they did in their carefully engi¬ 
neered world. Then in the mid 1970s in France, Japan, 
and the USA we started to see robots rising again in 
a handful of research laboratories, and now we have 
arrived at a world-wide frenzy in research and the be¬ 
ginnings of large-scale deployment of intelligent robots 
throughout our world. This Handbook brings together 
the current state of robotics research in one place. It 
ranges from the mechanism of robots through sens¬ 
ing and perceptual processing, intelligence, action, and 
many application areas. 

I have been more than fortunate to have lived 
with this revolution in robotics research over the last 
30 years. As a teenager in Australia I built robots 
inspired by the tortoises of Walter described in the Sci¬ 
entific American in 1949 and 1950. When I arrived in 
Silicon Valley in 1977, just as the revolution in the 
personalization of computation was really coming into 
being, I instead turned to the much more obscure world 
of robots. In 1979 I was able to assist Hans Moravec 
at the Stanford Artificial Intelligence Lab (SAIL) as he 
coaxed his robot The Cart to navigate 20 m in 6 hours. 
Just 26 years later, in 2005, at the same laboratory, 
SAIL, Sebastian Thrun and his team coaxed their robot 
to autonomously drive 200000 m in 6 hours: four or¬ 
ders of magnitude improvement in a mere 26 years, 
which is slightly better than a doubling every 2 years. 
However, robots have not just improved in speed, they 
have also increased in number. When I arrived at SAIL 
in 1977 we knew of three mobile robots operating in 
the world. Recently a company that I founded manu¬ 
factured its 3 000 000th mobile robot, and the pace is 
increasing. Other aspects of robots have had similarly 
spectacular advances, although it is harder to provide 
such crisp numeric characterizations. In recent years we 
have gone from robots being too unaware of their sur¬ 
roundings that it was unsafe for people to share their 
workspace to robots that people can work with in close 


contact, and from robots that were 
totally unaware of people to robots 
that pick up on natural social cues 
from facial expressions to prosody 
in people’s voices. Recently robotics 
has crossed the divide between flesh 
and machines so that now we are 
seeing neurorobotics ranging from 
prosthetic robotic extensions to reha¬ 
bilitative robots for the disabled. And 
very recently robotics has become 
a respected contributor to research in 
cognitive science and neuroscience. 

The research results chronicled 
in this volume give the key ideas that 
have enabled these spectacular ad¬ 
vances. The editors, the part editors, 
and all the contributors have done a stellar job in bring 
this knowledge together in one place. Their efforts have 
produced a work that will provide a basis for much 
further research and development. Thank you, and con¬ 
gratulations to all who have labored on this pivotal 
book. 

Some of the future robotics research will be in¬ 
cremental in nature, taking the state of the art and 
improving upon it. Other parts of future research will be 
more revolutionary, based on ideas that are antithetical 
to some of the ideas and current state of the art pre¬ 
sented in this book. 

As you study this volume and look for places to con¬ 
tribute to research through your own talents and hard 
work I want to alert you to capabilities or aspirations 
that I believe will make robots even more useful, more 
productive, and more accepted. I describe these capabil¬ 
ities in terms of the age at which a child has equivalent 
capabilities: 

• The object-recognition capabilities of a 2-year-old 

child 

• The language capabilities of a 4-year-old child 

• The manual dexterity of a 6-year-old child 

• The social understanding of an 8-year-old child. 

Each of these is a very difficult goal. However even 
small amounts of progress towards any one of these 
goals will have immediate applications to robots out in 
the world. Good reading and best wishes as you con¬ 
tribute further to robotkind. 

Cambridge, October 2007 Rodney Brooks 


Rodney Brooks 
Panasonic Professor 
of Robotics 

Massachusetts Institute 
of Technology 


Preface to the Second Edition 


The Springer Handbook of Robotics was a challeng¬ 
ing six-year endeavour from 2002 to 2008. It mobilized 
a large number of active scientists and researchers to 
produce this unique comprehensive reference source 
combining basic and advanced developments. The 
handbook has been very successful and extremely 
well received in our community. New researchers have 
been attracted to robotics which in turn have con¬ 
tributed to further progress in this trans-disciplinary 
field. 

The handbook soon established itself as a land¬ 
mark in robotics publishing and beyond. It has been 
the bestseller of all Springer engineering books during 
the last seven years, the number one in chapter down¬ 
loads (nearly forty thousand a year), and the fourth 
most downloaded over all Springer books in 2011. 
In February 2009, the handbook was recognized as 
the Winner of the American Association of Publish¬ 
ers (AAP) PROSE Award for Excellence in Physical 
Sciences & Mathematics as well as the Award for En¬ 
gineering & Technology. 

The rapid growth of our field as well as the birth 
of new research areas motivated us in 2011 to start 
pursuing a second edition with the intent to provide 
not only an update but also an expansion of the hand¬ 
book’s contents. Our editorial board (with David Orin, 
Frank Park, Henrik Christensen, Makoto Kaneko, Raja 
Chatila, Alex Zelinsky, and Daniela Rus) has been 
enthusiastically engaged during the last four years 
to coordinate the contributions of the authors to the 
seven parts of the handbook in its three-layer struc¬ 
ture. The contents have been restructured to achieve 
four main objectives: the enlargement of foundational 
topics for robotics, the enlightenment of design of var¬ 
ious types of robotic systems, the extension of the 
treatment on robots moving in the environment, and 
the enrichment of advanced robotics applications. Most 
previous chapters have been revised, fifteen new chap¬ 
ters have been introduced on emerging topics, and 
a new generation of authors have joined the hand¬ 
book’s team. The contents were finalized by the spring 
of 2015 after extensive review and feedback, and the 
project was completed by the fall of 2015 - gen¬ 
erating, by that time, a record of over 12 000 ad¬ 
ditional emails in our folders to the 10000 of the 
first edition. The result is an impressive collection of 
80 chapters over the 7 parts, contributed by 229 authors, 


with more than 2300 pages, 1375 illustrations and 9411 
references. 

One of the major additions of the second edition 
of the handbook is the inclusion of multimedia ma¬ 
terial. An editorial team has been established under 
the leadership of Torsten Kroger and the contribu¬ 
tions of Gianluca Antonelli, Dongjun Lee, Dezhen 
Song and Stefano Stramigioli. With the commitment 
of such a group of energetic young scholars, the mul¬ 
timedia project has been pursued in parallel to the 
handbook project. The multimedia editorial team has 
selected for each chapter video contributions, from 
those suggested by the authors, based on their qual¬ 
ity and relevance to the chapter’s contents. In addition, 
the handbook editors have produced tutorial videos that 
can be accessed directly from each part of the hand¬ 
book. An openly accessible multimedia website, http:// 
handbookofrobotics.org, has been established to host 
these videos with the sponsorship of IEEE Robotics and 
Automation Society and Google. The website has been 
conceived as a live dissemination project bringing the 
latest robotics contributions to the world community. 

We are deeply grateful for the continuous commit¬ 
ment of our handbook extended team, particularly the 
newcomers to the project. We would like to express 
our gratitude and appreciation to Judith Hinterberg, 
Werner Skolaut and Thomas Ditzinger from Springer 
for their strong support, as well as to Anne Strohbach 
and the le-tex staff for their highly professional typeset¬ 
ting work in the production. 

Eight years after the first appearance of the hand¬ 
book, the second edition comes to light. Beyond its 
tutorial value for our community, it is our conviction 
that the handbook will continue to serve as a useful 
source to attract new researchers to robotics and inspire 
decades of vibrant progress in this fascinating field. The 
cooperative spirit inspiring our team since the inception 
of the first edition is amusingly illustrated in the video 
The Handbook - A Short History (|43>M!ihliiilJ). The 
completion of the second edition has been inspired by 
that same spirit and the gradient has been kept :-) Our 
fellows in the robotics community are reminded now to 
... keep the Hessian ;-) 

January 2016 

Bruno Siciliano Naples 

Oussama Khatib Stanford 



Preface to the Multimedia Extension 


Scientific and technical advancements in the domain of robotics have accelerated 
significantly over the past decade. Since the inception of the Second Edition of the 
Springer Handbook of Robotics in 2011, the Editors Bruno Siciliano and Oussama 
Khatib decided to add multimedia content and appointed an editorial team: Gianluca 
Antonelli, Dongjun Lee, Dezhen Song, Stefano Stramigioli, and myself as the Multi- 
media Editor. 

Over the five years of the project, everyone on the team worked with all of the 
229 authors, the Part Editors, and the Editors. Besides communicating with all 80 
Authors’ teams and reviewing, selecting, and improving all video contributions, we 
also scanned all the videos published at robotics conferences organized by the IEEE 
Robotics and Automation Society since 1991. A total of more than 5500 e-mails were 
sent back and forth to coordinate the project and to ensure the quality of the content. 
We implemented a video management system that allows authors to upload videos, 
editors to review videos, and readers to access videos. Videos were selected with the 
goal of helping convey content to all readers of the Second Edition. They may be 
relevant from a technical, scientific, educational, or historical perspective. All chapter 
and part videos are publicly accessible and can be found at 

http://handbookofrobotics.org 

In addition to the videos referenced in the chapters, each of the seven parts is 
accompanied by a part video giving an overview of each part. The storyboards of these 
videos were created by the Paid Editors and then professionally produced. 

The video content provided in the Multimedia Extension makes understanding the 
written content easier and was designed to be a comprehensive addition to the Hand¬ 
book. Concepts, methods, experiments, and applications described in the book were 
animated, visually illustrated, or paired with sound and narration - giving readers a fur¬ 
ther dimension to comprehend the written content of the book. 

Coordinating the work with more than 200 contributors cannot just be done by 
a small team, and we are deeply grateful for the support of many people and organiza¬ 
tions. Judith Hinterberg and Thomas Ditzinger from the Springer Team in Heidelberg 
helped us tremendously with professional support during the entire production phase. 
The app for smartphones and tablets was implemented by Rob Baldwin from Studio 
Orb and allows readers easy access to multimedia content. The IEEE Robotics and 
Automation Society granted permissions to use all videos that have been published in 
the proceedings of conferences sponsored by the society. Google and X supported us 
by donating funds for the implementation of the website backend. 

Following the Editors’ inspiration, let us keep working and communicating as one 
community - and let us keep the Hessian all together ...! 


March 2016 
Torsten Kroger 


Mountain View 


Accessing Multimedia Contents 


Multimedia contents are an integral part of the Second Edition of the Springer Hand¬ 
book of Robotics. 69 chapters contain video icons like this one: 

Each icon indicates a video ID that can be used to access individual videos in 
various simple and intuitive ways. 

Using the Multimedia App 

We recommend using the multimedia app for smartphone and tablet PCs. You can 
install the app on iOS and Android devices using the QR code below. The app allows 
you to simply scan the pages of the book and automatically play all videos on your 
device while reading the book. 


Multimedia Contents 



Using the Website: http://handbookofrobotics.org 
All chapter videos and part videos can be accessed directly from the website of the 
multimedia extension. Just enter a video ID in the search held in the top right corner of 
the website. You may also use the website to browse through chapter and part videos. 

Using PDF Files 

If you read an electronic copy of the Handbook, each video icon contains a hyper link. 
Just click on the link to watch the corresponding video. 

Using QR Codes 

Each chapter starts with a QR code that contains a link to all videos of the chapter. Part 
videos can be accessed through the QR code at the beginning of each part. 
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active chord mechanism 

aRDnet 

agile robot development network 

ACM 

active cord mechanism 

ARM 

Acorn RISC machine architecture 

ACT 

anatomically correct testbed 

ARM 

assistive !robot service manipulator 

ADAS 

advanced driving assistance system 

ARX 

auto regressive estimator 

ADC 

analog digital conveter 

ASAP 

adaptive sampling and prediction 

ADCP 

acoustic Doppler current profiler 

ASCII 

American standard code for information 

ADL 

activities for daily living 


interchange 

ADSL 

asymmetric digital subscriber line 

ASD 

autism spectrum disorder 

AFC 

alkaline fuel cell 

ASIC 

application-specific integrated circuit 

AFC 

armoured (or articulated) face conveyor 

ASIC 

application-specific feature transform 

AFM 

atomic force microscope 

ASIMO 

advanced step in innovative mobility 

AFV 

autonomous !flying vehicle 

ASK 

amplitude shift keying 

AGV 

autonomous guided vehicle 

ASL 

autonomous systems laboratory 

AGV 

automated!guided vehicle 

ASM 

advanced servomanipulator 




ASN 

active sensor network 

c 


ASR 

automatic! spoken-language recognition 



ASR 

automatic [speech recognition 

c 

cylindrical joint 

ASTRO 

autonomous [space transport robotic 

C/A 

coarse-acquisition 


operations 

C/S 

client/server 

ASV 

adaptive suspension vehicle 

CA 

collision avoidance 

ASyMTRc 

automated [synthesis of multirobot task 

CACC 

cooperative adaptive cruise control 


solutions through software 

CAD 

computer-aided drafting 


reconfiguration 

CAD 

computer-aided design 

AT 

anti-tank mine 

CAE 

computer-aided engineering 

ATHLETE 

all-terrain hex-legged extra-terrestrial 

CALM 

communication access for land mobiles 


explorer 

CAM 

computer-aided manufacturing 

ATLANTIS 

a three layer architecture for navigating 

CAN 

controller area network 


through intricate situations 

CARD 

computer-aided remote driving 

ATLSS 

advanced technology for large structural 

CARE 

coordination action for robotics in 


systems 


Europe 

ATR 

automatic [target recognition 

CASA 

Civil Aviation Safety Authority 

AuRA 

autonomous robot architecture 

CASALA 

Centre for Affective Solutions for 

AUV 

autonomous underwater vehicle 


Ambient Living Awareness 

AUV 

autonomous aquatic vehicle 

CASPER 

continuous activity scheduling. 

AUVAC 

Autonomous Undersea Vehicles 


planning, execution and replanning 


Application Center 

CAT 

collision avoidance technology 

AUVSI 

Association for Unmanned Vehicle 

CAT 

computer-aided tomography 


Systems International 

CB 

computional brain 

AV 

anti-vehicle 

CB 

cluster bomb 



CBRNE 

chemical, biological, nuclear. 

B 



radiological, or explosive 



CC 

compression criterion 

B/S 

browser/server 

CCD 

charge-coupled device 

B2B 

business to business 

CCD 

charge-coupled detector 

BCI 

brain-computer interface 

CCI 

control command interpreter 

BE 

body extender 

CCP 

coverage configuration protocol 

BEMT 

blade element momentum theory 

CCT 

conservative congruence transformation 

BEST 

boosting [engineering science and 

CCW 

counterclockwise 


technology 

CC&D 

camouflage, concealment, and deception 

BET 

blade element theory 

CD 

collision detection 

BFA 

bending fluidic actuator 

CD 

committee draft 

BFP 

best-first-planner 

CD 

compact disc 

BI 

brain imaging 

CDC 

cardinal direction calculus 

BIP 

behavior-interaction-priority 

CDOM 

colored dissolved organic matter 

BLE 

broadcast of local eligibility 

CE 

computer ethic 

BLEEX 

Berkely exoskeleton 

CEA 

Commissariat a l’Energie Atomique 

BLUE 

best linear unbiased estimator 

CEA 

Atomic Energy Commission 

BML 

behavior [mark-up language 

CEBOT 

cellular robotic system 

BMS 

battery management system 

CEC 

Congress on Evolutionary Computation 

BN 

Bayesian network 

CEPE 

Computer Ethics Philosophical Enquiry 

BOM 

bill of material 

CES 

Consumer Electronics Show 

BOw 

bag-of-word 

CF 

carbon fiber 

BP 

behavior primitive 

CF 

contact formation 

BP 

base plate 

CF 

climbing fiber 

BRICS 

best practice in robotics 

CFD 

computational fluid dynamics 

BRT 

bus rapid transit 

CFRP 

carbon fiber reinforced prepreg 

BWSTT 

body-weight supported treadmill 

CFRP 

carbon fiber reinforced plastic 


training 

CG 

center of gravity 




CG 

computer graphics 

CP 

continuous path 

CGI 

common gateway interface 

CP 

cerebral palsy 

CHMM 

coupled Ihidden Markov model 

CPC 

central pattern generation 

CHMM 

continuous hidden Markov model 

CPC 

central pattern generator 

CIC 

computer integrated construction 

CPS 

cyber physical system 

CIE 

International Commission on 

CPSR 

Computer Professional for Social 


Illumination 


Responsibility 

CIP 

Children’s Innovation Project 

CPU 

central processing unit 

CIRCA 

cooperative intelligent real-time control 

CRASAR 

Center for Robot-Assisted Search and 


architecture 


Rescue 

CIS 

computer-integrated surgery 

CRBA 

composite-rigid-body algorithm 

CLARAty 

coupled layered architecture for robot 

CRF 

conditional random field 


autonomy 

CRLB 

Cramer-Rao lower bound 

CLEaR 

closed-loop execution and recovery 

CSAIL 

Computer Science and Artificial 

CLIK 

closed-loop inverse kinematics 


Intelligence Laboratory 

CMAC 

cerebellar model articulation controller 

CSIRO 

Commonwealth Scientific and Industrial 

CMCs 

ceramic matrix composite 


Research Organisation 

CML 

concurrent!mapping and localization 

CSMA 

carrier-sense multiple-access 

CMM 

coordinate measurement machine 

CSP 

constraint satisfaction problem 

CMOMMT 

cooperative multirobot observation of 

CSSF 

Canadian Scientific Submersile Facility 


multiple moving target 

CT 

computed tomography 

CMOS 

complementary 

CTFM 

continuous-transmission frequency 


metal-oxide-semiconductor 


modulation 

CMP 

centroid moment pivot 

CU 

control unit 

CMTE 

Cooperative Research Centre for Mining 

cv-SLAM 

ceiling vision SLAM 


Technology and Equipment 

CVD 

chemical vapor deposition 

CMU 

Carnegie Mellon University 

CVIS 

cooperative vehicle infrastructure 

CNC 

computer numerical control 


system 

CNN 

convolutional neural network 

CVT 

continuous variable transmission 

CNP 

contract net protocol 

CW 

clockwise 

CNRS 

Centre National de la Recherche 
Scientifique 

CWS 

contact!wrench sum 

CNT 

carbon nanotube 

D 


COCO 

common objects in context 



COG 

center of gravity 

D 

distal 

COM 

center of mass 

D/A 

digital-to-analog 

COMAN 

compliant humanoid platform 

DAC 

digital analog converter 

COMEST 

Commission mondiale d’ethique des 

DARPA 

Defense Advanced Research Projects 


connaissances scientifiques et des 


Agency 


technologies 

DARS 

distributed!autonomous robotic systems 

COMINT 

communication intelligence 

DBN 

dynamic Bayesian network 

CONE 

Collaborative Observatory for Nature 

DBN 

deep belief network 


Environments 

DC 

disconnected 

COP 

center of pressure 

DC 

direct current 

CoP 

center of pressure 

DC 

dynamic Constrained 

COR 

center of rotation 

DCS 

dynamic covariance scaling 

CORBA 

common object request broker 

DCT 

discrete!cosine transform 


architecture 

DD 

differentially driven 

CORS 

continuous operating reference station 

DDD 

dangerous, dirty, and dreary 

COT 

cost!of transport 

DDF 

decentralized data fusion 

COTS 

commercial off-the-shelf 

DDP 

differential dynamic programming 

COV 

characteristic output vector 

DDS 

data distribution service 

CP 

complementarity problem 

DEA 

differential elastic actuator 

CP 

capture point 

DEM 

discrete!element method 



DFA 

design !for assembly 

EDM 

electrical discharge machining 

DFRA 

distributed field robot architecture 

EE 

end-effector 

DFT 

discrete Fourier transform 

EEC 

electroencephalography 

DGPS 

differential global positioning system 

EGNOS 

European Geostationary Navigation 

DH 

Denavit-Flartenberg 


Overlay Service 

DHMM 

discrete Ihidden Markov model 

EHC 

enhanced horizon control 

DHS 

US Department of Homeland Security 

EHPA 

exoskeleton!for human performance 

DIRA 

distributed!robot architecture 


augmentation 

DIST 

Dipartmento di Informatica Sistemica e 

EKF 

extended Kalman filter 


Telematica 

ELS 

ethical, legal and societal 

DL 

description logic 

EM 

expectation maximization 

DLR 

Deutsches Zentrum fur Luft- und 

emf 

electromotive force 


Raumfahrt 

EMG 

electromyography 

DLR 

German Aerospace Center 

EMIB 

emotion, motivation and intentional 

DMFC 

direct methanol fuel cell 


behavior 

DMP 

dynamic movement primitive 

EMS 

electrical !master-slave manipulator 

DNA 

deoxyribonucleic acid 

EO 

electrooptical 

DNF 

dynamic!neural field 

EO 

elementary operator 

DOD 

Department of Defense 

EOA 

end of arm 

DOF 

degree of freedom 

EOD 

explosive !ordnance disposal 

DOG 

difference of Gaussian 

EP 

exploratory procedure 

DOP 

dilution of precision 

EP 

energy packet 

DPLL 

Davis-Putnam algorithm 

EPFL 

Ecole Polytechnique Federate de 

DPM 

deformable part model 


Lausanne 

DPN 

dip-pen nanolithography 

EPP 

extended!physiological proprioception 

DPSK 

differential phase shift keying 

EPS 

expandable polystyrene 

DRIE 

deep reactive ion etching 

ER 

electrorheological 

DSM 

dynamic!state machine 

ER 

evolutionary Robotics 

DSO 

Defense Sciences Office 

ERA 

European robotic arm 

DSP 

digital signal processor 

ERP 

enterprise resource planning 

DSRC 

dedicated short-range communications 

ERSP 

evolution robotics software platform 

DU 

dynamic !unconstrained 

ES 

electrical! stimulation 

DVL 

Doppler velocity log 

ESA 

European Space Agency 

DWA 

dynamic window approach 

ESC 

electronic speed controller 

DWDM 

dense wave division multiplex 

ESL 

execution support language 

D&D 

deactivation and decommissioning 

ESM 

energy!stability margin 



ESM 

electric support measure 

E 


ETL 

Electro-Technical Laboratory 



ETS-VII 

Engineering Test Satellite VII 

e-beam 

electron-beam 

EU 

European Union 

EAP 

electroactive polymer 

EURON 

European Robotics Research Network 

EBA 

energy bounding algorithm 

EVA 

extravehicular activity 

EBA 

extrastriate body part area 

EVRYON 

evolving morphologies for human-robot 

EBID 

electron-beam induced deposition 


symbiotic interaction 

EC 

externally connected 



EC 

exteroception 



ECAI 

European Conference on Artificial 

F 



Intelligence 



ECD 

eddy current damper 

F5 

frontal area 5 

ECEF 

earth-centred, earth-fixed 

FAA 

Federal Aviation Administration 

ECER 

European Conference on Educational 

FAO 

Food and Agriculture Organization 


Robotics 

FARS 

Fagg-Arbib-Rizzolatti-Sakata 

ECG 

electrocardiogram 

FARSA 

framework for autonomous robotics 

ECU 

electronics controller unit 


simulation and analysis 




fastSLAM 

fast simultaneous localization and 

GBAS 

ground based augmentation system 


mapping 

GCDC 

Grand Cooperative Driving Challenge 

FB-EHPA 

full-body EHPA 

GCER 

Global Conference on Educational 

FCU 

flight control-unit 


Robotics 

FD 

friction damper 

GCR 

goal-contact relaxation 

FDA 

US Food and Drug Association 

GCS 

ground Icontrol station 

FDM 

fused deposition modeling 

GDP 

grossldomestic product 

FE 

finite element 

GenoM 

generator of modules 

FEA 

finite element analysis 

GEO 

geostationary Earth orbit 

FEM 

finite element method 

GF 

grapple fixture 

FESEM 

field-emission SEM 

GFRP 

glass-fiber reinforced plastic 

FF 

fast forward 

GI 

gastrointestinal 

FFI 

Norwegian defense research 

GIB 

GPS intelligent buoys 


establishment 

GICHD 

Geneva International Centre for 

FFT 

fast Fourier transform 


Humanitarian Demining 

FIFO 

first-in first-out 

GID 

geometric [intersection data 

FIRA 

Federation of International Robot-soccer 

GIE 

generalized-inertia ellipsoid 


Association 

GIS 

geographic information system 

FIRRE 

family of integrated rapid response 

GJM 

generalizedljacobian matrix 


equipment 

GLONASS 

globalnaya navigatsionnaya 

FIRST 

For Inspiration and Recognition of 


sputnikovaya sistema 


Science and Technology 

GLS 

global navigation satellite system 

Fl-UAS 

flapping wing unmanned aerial system 

GMAW 

gas-shielded metal arc welding 

FLIR 

forward Hooking infrared 

GMM 

Gaussian mixture model 

FMBT 

feasible minimum buffering time 

OMSK 

Gaussian minimum shift keying 

FMCW 

frequency modulation continuous wave 

GMTI 

ground [moving target indicator 

fMRI 

functional!magnetic resonance imaging 

GNC 

guidance, navigation, and control 

FMS 

flexible [manufacturing system 

GO 

golgi! tendon organ 

FNS 

functional Ineural stimulation 

GP 

Gaussian process 

FOA 

focus of attention 

GPCA 

generalized principal component 

FOG 

fiber-optic gyro 


analysis 

FOPEN 

foliage penetration 

GPRS 

general [packet radio service 

FOPL 

first-order predicate logic 

GPS 

global positioning system 

FOV 

field of view 

GPU 

graphics processing unit 

FP 

fusion primitive 

GRAB 

guaranteed recursive adaptive bounding 

FPGA 

field-programmable gate array 

GRACE 

graduate robot attending conference 

FR 

false range 

GraWoLF 

gradient-based win or learn fast 

FRI 

foot rotation indicator 

GSD 

geon structural description 

FRP 

fiber-reinforced plastics 

GSN 

gait sensitivity norm 

FRP 

fiber-reinforced prepreg 

GSP 

Gough-Stewart platform 

fs 

force! sensor 

GUI 

graphical user interface 

FSA 

finite-state acceptor 

GV 

ground vehicle 

FSK 

frequency shift keying 

GVA 

grosslvalue added 

FSR 

force sensing resistor 

GZMP 

generalized! ZMP 

FSW 

friction!stir welding 



FTTH 

fiber to the home 

H 


FW 

fixed-wing 

H 

helical joint 

G 


HAL 

hybrid [assistive limb 



HAMMER 

hierarchical [attentive multiple models 

GA 

genetic algorithm 


for execution and recognition 

GAPP 

goal as parallel programs 

HASY 

hand [arm system 

GARNICS 

gardening with a cognitive system 

HBBA 

hybrid behavior-based architecture 

GAS 

global asymptotic stability 

HCI 

human-computer interaction 




HD 

high definition 

IARC 

International Aerial Robotics 

HD 

haptic device 


Competition 

HD-SDI 

high-definition serial digital interface 

IAS 

intelligent!autonomous system 

HDSL 

high data rate digital subscriber line 

IBVS 

image-based visual servo control 

HE 

hand lexoskeleton 

IC 

integrated chip 

HF 

hard finger 

IC 

integrated circuit 

HF 

histogram filter 

ICA 

independent!component analysis 

HFAC 

high frequency alternating current 

ICAPS 

International Conference on Automated 

HHMM 

hierarchical'hidden Markov model 


Planning and Scheduling 

HIC 

head injury criterion 

ICAR 

International Conference on Advanced 

Hill 

Hybrid III dummy 


Robotics 

HIP 

haptic interaction point 

ICBL 

International Campaign to Ban 

HIB 

Hamilton-Jacobi-Bellman 


Landmines 

HJI 

Hamilton-Jacobi-Isaac 

ICC 

instantaneous center of curvature 

HMCS 

human-machine! cooperative system 

ICE 

internet communications engine 

HMD 

head-mounted display 

ICP 

iterative closest point 

HMDS 

hexamethyldisilazane 

ICR 

instantaneous center of rotation 

HMI 

human-machi ne! interaction 

ICRA 

International Conference on Robotics 

HMI 

human-machine linterface 


and Automation 

HMM 

hidden Markov model 

ICT 

information!and communication 

HO 

human operator 


technology 

HOG 

histogram of oriented gradient 

ID 

inside diameter 

HOG 

histogram of oriented features 

ID 

identifier 

HPC 

high-performance computing 

IDE 

integrated! development environment 

HRI 

human-robot interaction 

IDL 

interface definition language 

HRI/OS 

HRI operating system 

IE 

information! ethics 

HRP 

humanoid robotics project 

IED 

improvised explosive device 

HRR 

high resolution radar 

IEEE 

Institute of Electrical and Electronics 

HRTEM 

high-resolution transmission electron 


Engineers 


microscope 

IEKF 

iterated extended Kalman filter 

HSGR 

high safety goal 

IETF 

internet!engineering task force 

HST 

Hubble space telescope 

IFA 

Internationale Funk Ausstellung 

HSTAMIDS 

handheld standoff mine detection system 

IFOG 

interferometric fiber-optic gyro 

HSWR 

high safety wide region 

IFR 

International Federation of Robotics 

HTAS 

high tech automotive system 

IFREMER 

Institut frangais de recherche pour 

HTML 

hypertext markup language 


T exploitation de la mer 

HTN 

hierarchical task network 

IFRR 

International Foundation of Robotics 

HTTP 

hypertext transmission protocol 


Research 

HW/SW 

hardware/software 

IFSAR 

interferometric SAR 



IHIP 

intermediate haptic interaction point 



HR 

infinite impulse response 

1 


IIS 

Internet Information Services 



IIT 

Istituto Italiano di Tecnologia 

I/O 

input/output 

IJCAI 

International Joint Conference on 

I3CON 

industrialized, integrated, intelligent, 


Artificial Intelligence 


construction 

IK 

inverse kinematics 

IA 

interval algebra 

ILLS 

instrumented logical sensor system 

IA 

instantaneous lallocation 

ILO 

International Labor Organization 

IAA 

interaction! agent 

ILQR 

iterative linear quadratic regulator 

IAB 

International Association of Bioethics 

IM 

injury measure 

IACAP 

International Association for Computing 

IMAV 

International Micro Air Vehicles 


and Philosophy 

IMTS 

intelligent !multimode transit system 

IAD 

interaural amplitude difference 

IMU 

inertial measurement unit 

IAD 

intelligentlassisting device 

INS 

inertia navigation system 






INS 

inertial navigation system 

IAEA 

Japan Atomic Energy Agency 

10 

input output 

JAMSTEC 

Japan Agency for Marine-Earth Science 

10 

inferior olive 


and Technology 

IOSS 

input-output-to-state stability 

JAMSTEC 

Japan Marine Science and Technology 

IP 

internet protocol 


Center 

IP 

interphalangeal 

JAUS 

joint architecture for unmanned systems 

IPA 

Institute for Manufacturing Engineering 

JAXA 

Japan Aerospace Exploration Agency 


and Automation 

JDL 

joint directors of laboratories 

IPC 

interprocess communication 

JEM 

Japan Experiment Module 

IPC 

international AI planning competition 

JEMRMS 

Japanese experiment module remote 

IPMC 

ionic polymer-metal composite 


manipulator system 

IPR 

intellectual property right 

JHU 

Johns Hopkins University 

IR 

infrared 

JND 

just noticeable difference 

IRB 

Institutional Review Board 

JPL 

Jet Propulsion Laboratory 

IREDES 

International Rock Excavation Data 

JPS 

jigsaw positioning system 


Exchange Standard 

JSC 

Johnson Space Center 

IRL 

in real life 

JSIM 

joint-space inertia matrix 

IRL 

inverselreinforcement learning 

JSP 

Java server pages 

IRLS 

iteratively reweighted least square 



IRNSS 

Indian regional navigational satellite 

K 



system 



IROS 

Intelligent Robots and Systems 

KAIST 

Korea Advanced Institute of Science 

IS 

importance sampling 


and Technology 

ISA 

industrial standard architecture 

KERS 

kinetic energy recovery system 

ISA 

international standard atmosphere 

KIPR 

KISS Institute for Practical Robotics 

ISAR 

inverse SAR 

KLD 

Kullback-Leibler divergence 

ISDN 

integrated services digital network 

KNN 

k-nearest neighbor 

ISE 

international submarine engineering 

KR 

knowledge representation 

ISER 

International Symposium on 

KRISO 

Korea Research Institute of Ships and 


Experimental Robotics 


Ocean Engineering 

ISM 

implicit shape model 



ISO 

International Organization for 

L 



Standardization 



ISP 

Internet service provider 

L/D 

lift-to-drag 

ISR 

intelligence, surveillance and 

LAAS 

Laboratory for Analysis and 


reconnaissance 


Architecture of Systems 

ISRR 

International Symposium of Robotics 

LADAR 

laser radar 


Research 

LAGR 

learning!applied to ground robots 

ISS 

international space station 

LARC 

Lie algebra rank condition 

ISS 

input-to-state stability 

LARS 

Laparoscopic Assistant Robotic System 

1ST 

Instituto Superior Tecnico 

LASC 

Longwall Automation Steering 

1ST 

Information Society Technologies 


Committee 

IT 

intrinsic tactile 

LBL 

long-baseline system 

IT 

information! technology 

LCAUV 

long-range cruising AUV 

IT 

inferotemporal cortex 

LCC 

life-cycle-costing 

ITD 

interaural time difference 

LCD 

liquid-crystal display 

IU 

interaction! unit 

LCM 

light-weight communications and 

IV 

instrumental variable 


marshalling 

IvP 

interval programming 

LCP 

linear complementarity problem 

IWS 

intelligent!wheelchair system 

LCSP 

linear constraint satisfaction program 

IxTeT 

indexed time table 

LDA 

latent Dirichlet allocation 

J 


LED 

light-emitting diode 


LENAR 

lower! extremity nonanthropomorphic 




robot 









LEO 

low!Earth orbit 

MDARS 

mobile detection assessment and 

LEV 

leading edge vortex 


response system 

LfD 

learning !from demonstration 

MDL 

minimum description length 

LGN 

lateral!geniculate nucleus 

MDP 

Markov decision process 

LHD 

load!haul-dump 

ME 

mechanical !engeneering 

LIDAR 

light detection and ranging 

MEG 

magnetoencephalography 

LIGA 

Lithographie, Galvanoumformung, 

MEL 

Mechanical Engineering Laboratory 


Abformung 

MEMS 

microelectromechanical system 

LIP 

linear inverted pendulum 

MEP 

motor!evoked potential 

LIP 

lateral Lntraparietal sulcus 

MESSIE 

multi expert system for scene 

LiPo 

lithium polymer 


interpretation and evaluation 

LLC 

locality constrained linear coding 

MESUR 

Mars environmental survey 

LMedS 

least median of squares 

MF 

mossy fiber 

LMS 

laser measurement system 

MFI 

micromechanical flying insect 

LOG 

Laplacian of Gaussian 

MFSK 

multiple FSK 

LOPES 

lower!extremity powered exoskeleton 

MHS 

International Symposium on Micro 

LOS 

line-of-sight 


Mechatronics and Human Science 

LP 

linear program 

MHT 

multihypothesis tracking 

LQG 

linear quadratic Gaussian 

MIA 

mechanical impedance adjuster 

LQR 

linear quadratic regulator 

MIME 

mirror!image movement enhancer 

LSS 

logical sensor system 

MIMICS 

multimodal immersive motion 

LSVM 

latent support vector machine 


rehabilitation with interactive cognitive 

LtA 

lighter-than-air 


system 

LtA-UAS 

lighter-than-air system 

MIMO 

multiple-input-multiple-output 

LTL 

linear temporal logic 

MIP 

medial intraparietal sulcus 

LVDT 

linear variable differential transformer 

MIPS 

microprocessor without interlocked 

LWR 

light-weight robot 


pipeline stages 



MIR 

mode identification and recovery 



MIRO 

middleware for robot 

M 


MIS 

minimally invasive surgery 



MIT 

Massachusetts Institute of Technology 

MACA 

Afghanistan Mine Action Center 

MITI 

Ministry of International Trade and 

MACCEPA 

mechanically adjustable compliance and 


Industry 


controllable equilibrium position 

MKL 

multiple kernel learning 


actuator 

ML 

machine! learning 

MAP 

maximum a posteriori 

MLE 

maximum likelihood estimate 

MARS 

multiappendage robotic system 

MLR 

mesencephalic locomotor region 

MARUM 

Zentrum fur Marine 

MLS 

multilevel surface map 


Umweltwissenschaften 

MMC 

metal matrix composite 

MASE 

Marine Autonomous Systems 

MMMS 

multiple master multiple-slave 


Engineering 

MMSAE 

multiple model switching adaptive 

MASINT 

measurement!and signatures intelligence 


estimator 

MAV 

micro aerial vehicles 

MMSE 

minimum mean-square error 

MAZE 

Micro robot maze contest 

MMSS 

multiple master single-slave 

MBA 

motivated behavioral architecture 

MNS 

mirror!neuron system 

MBARI 

Monterey Bay Aquarium Research 

MOCVD 

metallo-organic chemical vapor 


Institute 


deposition 

MBE 

molecular-beam epitaxy 

MOMR 

multiple operator multiple robot 

MBS 

mobile !base system 

MOOS 

mission oriented operating suite 

MC 

Monte Carlo 

MOOS 

motion-oriented operating system 

MCFC 

molten carbonate fuel cell 

MORO 

mobile robot 

MCP 

magazining, cleaning, plotting 

MOSR 

multiple operator single robot 

MCP 

metacarpophalangeal 

MP 

moving plate 

MCS 

mission !control system 

MPC 

model predictive control 



MPF 

manifold particle filter 

NIDRR 

National Institute on Disability and 

MPFIM 

multiple!paired forward-inverse model 


Rehabilitation Research 

MPHE 

multiphalanx hand exoskeleton 

NiMH 

nickel metal hydride battery 

MPSK 

Mary phase shift keying 

NIMS 

networked !infomechanical systems 

MQAM 

Mary quadrature amplitude modulation 

NIOSH 

United States National Institute for 

MR 

magnetorheological 


Occupational Safety and Health 

MR 

multiple reflection 

NIRS 

near infrared spectroscopy 

MR 

multirobot! task 

NIST 

National Institute of Standards and 

MRAC 

model reference adaptive control 


Technology 

MRDS 

Microsoft robotics developers studio 

NLIS 

national livestock identification scheme 

MRF 

Markov random field 

NLP 

nonlinear!programming problem 

MRHA 

multiple [resource host architecture 

NMEA 

National Marine Electronics Association 

MRI 

magnetic resonance imaging 

NMF 

nonnegative matrix factorization 

MRSR 

Mars rover sample return 

NMMI 

natural machine motion initiative 

MRTA 

multirobot!task allocation 

NMR 

nuclear [magnetic resonance 

MSAS 

multifunctional satellite augmentation 

NN 

neural network 


system 

NO A A 

National Oceanic and Atmospheric 

MSER 

maximally stable extremal region 


Administration 

MSHA 

US Mine Safety and Flealth 

NOAH 

navigation!and obstacle avoidance help 


Administration 

NOC 

National Oceanography Centre 

MSK 

minimum shift keying 

NOTES 

natural [orifice transluminal surgery 

MSL 

middle-size league 

NPO 

nonprofit organization 

MSM 

master-slave! manipulator 

NPS 

Naval Postgraduate School 

MST 

microsystem technology 

NQE 

national qualifying event 

MT 

momentum theory 

NRI 

national robotics initiative 

MT 

multitask 

NRM 

nanorobotic manipulator 

MT 

medial temporal area 

NRTK 

network real-time kinematic 

MTBF 
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US Office of Naval Research 

PDMS 

polydimethylsiloxane 

OOF 

out of field 

PDOP 

positional dilution of precision 

OOTL 

human!out of the loop control 

PDT 

proximity [detection technology 

OPRoS 

open platform for robotic service 

PEAS 

probing environment and adaptive 

ORCA 

open robot control architecture 


sleeping protocol 

ORCCAD 

open robot controller computer aided 

PEFC 

polymer electrolyte fuel cell 


design 

PEMFC 

proton exchange membrane fuel cell 

ORI 

openlroboethics initiative 

PerceptOR 

perception Tor off-road robotics 

ORM 

obstacle restriction method 

PET 
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1. Robotics and the Handbook 


Robots! Robots on Mars and in oceans, in hospitals 
and homes, in factories and schools; robots fight¬ 
ing fires, making goods and products, saving time 
and lives. Robots today are making a consider¬ 
able impact on many aspects of modern life, from 
industrial manufacturing to healthcare, trans¬ 
portation, and exploration of the deep space and 
sea. Tomorrow, robots will be as pervasive and per¬ 
sonal as today's personal computers. This chapter 
retraces the evolution of this fascinating field from 
the ancient to the modern times through a number 
of milestones: from the first automated mechani¬ 
cal artifact (1400 BC) through the establishment of 
the robot concept in the 1920s, the realization of 
the first industrial robots in the 1960s, the defini¬ 
tion of robotics science and the birth of an active 
research community in the 1980s, and the expan- 


1.1 A Brief History of Robotics 

The dream to create machines that are skilled and 
intelligent has been part of humanity from the begin¬ 
ning of time. This dream is now becoming part of 
our world’s striking reality. Since the early civiliza¬ 
tions, one of man’s greatest ambitions has been to 
create artifacts in their image. The legend of the Ti¬ 
tan Prometheus, who molded humankind from clay, 
or that of the giant Talus, the bronze slave forged 
by Hephaestus (3500 BC), testify to this quest in 
Greek mythology. The Egyptians’ oracle statues hid¬ 
ing priests inside (2500 BC) were perhaps the pre¬ 
cursor of our modern thinking machines. The clepsy¬ 
dra water clock introduced by the Babylonians (1400 
BC) was one of the first automated mechanical ar¬ 
tifacts. In the following centuries, human creativity 
has given rise to a host of devices such as the au¬ 
tomaton theatre of Hero of Alexandria (100 AD), the 
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sion towards the challenges of the human world of 
the twenty-first century. Robotics in its long jour¬ 
ney has inspired this handbook which is organized 
in three layers: the foundations of robotics science; 
the consolidated methodologies and technologies 
of robot design, sensing and perception, manip¬ 
ulation and interfaces, mobile and distributed 
robotics; the advanced applications of field and 
service robotics, as well as of human-centered 
and life-like robotics. 


hydro-powered water-raising and humanoid machines 
of Al-Jazari (1200), and Leonardo da Vinci’s numer¬ 
ous ingenious designs (1500). The development of 
automata continued to flourish in the eighteen century 
both in Europe and Asia, with creations such as Jacquet- 
Droz’s family of androids (drawer, musician and writer) 
and the karakuri-ningyo mechanical dolls (tea server 
and archer). 

The robot concept was clearly established by those 
many creative historical realizations. Nonetheless, the 
emergence of the physical robot had to await the ad¬ 
vent of its underlying technologies during the course 
of the twentieth century. In 1920, the term robot - de¬ 
rived from robota which means subordinate labour in 
Slav languages - was first introduced by the Czech 
playwright Karel Capek in his play Rossum’s Universal 
Robots (R.U.R.). In 1940, the ethics of the interaction 
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2 Introduction 


between robots and humans was envisioned to be gov¬ 
erned by the well-known three fundamental laws of 
Isaac Asimov, the Russian science-fiction writer in his 
novel Runaround. 

The middle of the twentieth century brought the first 
explorations of the connection between human intelli¬ 
gence and machines, marking the beginning of an era 
of fertile research in the field of artificial intelligence 
(AI). Around that time, the first robots were realized. 
They benefited from advances in the different technolo¬ 
gies of mechanics, controls, computers and electronics. 
As always, new designs motivate new research and dis¬ 
coveries, which, in turn, lead to enhanced solutions 
and thus to novel concepts. This virtuous circle over 
time produced that knowledge and understanding which 
gave birth to the field of robotics , properly referred to 
as: the science and technology of robots. 

The early robots built in the 1960s stemmed from 
the confluence of two technologies: numerical control 
machines for precise manufacturing, and teleoperators 
for remote radioactive material handling. These master- 
slave arms were designed to duplicate one-to-one the 
mechanics of the human arm, and had rudimental con¬ 
trol and little perception about the environment. Then, 
during the mid-to-late twentieth century, the devel¬ 
opment of integrated circuits, digital computers and 
miniaturized components enabled computer-controlled 
robots to be designed and programmed. These robots, 
termed industrial robots, became essential components 
in the automation of flexible manufacturing systems 
in the late 1970s. Further to their wide application in 
the automotive industry, industrial robots were success¬ 
fully employed in general industry, such as the metal 
products, the chemical, the electronics and the food 
industries. More recently, robots have found new appli¬ 
cations outside the factories, in areas such as cleaning, 
search and rescue, underwater, space, and medical ap¬ 
plications. 

In the 1980s robotics was defined as the science 
which studies the intelligent connection between per¬ 
ception and action. With reference to this definition, 
the action of a robotic system is entrusted to a loco¬ 
motion apparatus to move in the environment (wheels, 
crawlers, legs, propellers) and/or to a manipulation ap¬ 
paratus to operate on objects present in the environment 
(arms, end effectors, artificial hands), where suitable 
actuators animate the mechanical components of the 
robot. The perception is extracted from the sensors pro¬ 
viding information on the state of the robot (position 


and speed) and its surrounding environment (force and 
tactile, range and vision). The intelligent connection is 
entrusted to a programming, planning and control ar¬ 
chitecture which relies on the perception and available 
models of the robot and environment and exploits learn¬ 
ing and skill acquisition. 

In the 1990s research was boosted by the need to 
resort to robots to address human safety in hazardous 
environments (field robotics), or to enhance the human 
operator ability and reduce his/her fatigue ( human aug¬ 
mentation), or else by the desire to develop products 
with wide potential markets aimed at improving the 
quality of life ( service robotics). A common denomina¬ 
tor of such application scenarios was the need to operate 
in a scarcely structured environment which ultimately 
requires increased abilities and a higher degree of au¬ 
tonomy. 

The first video attached to this chapter Robots - 
A 50 Year Journey by Oussama Khatib (2000) shows the 
development of robotics through the first five decades 

By the dawn of the new millennium, robotics has 
undergone a major transformation in scope and di¬ 
mensions. This expansion has been brought about by 
the maturity of the field and the advances in its re¬ 
lated technologies. From a largely dominant industrial 
focus, robotics has been rapidly expanding into the 
challenges of the human world ( human-centered and 
life-like robotics). The new generation of robots is ex¬ 
pected to safely and dependably co-habitat with humans 
in homes, workplaces, and communities, providing sup¬ 
port in services, entertainment, education, healthcare, 
manufacturing, and assistance. 

Beyond its impact on physical robots, the body of 
knowledge robotics has produced is revealing a much 
wider range of applications reaching across diverse 
research areas and scientific disciplines, such as: biome¬ 
chanics, haptics, neurosciences, virtual simulation, ani¬ 
mation, surgery, and sensor networks among others. In 
return, the challenges of the new emerging areas are 
proving an abundant source of stimulation and insights 
for the field of robotics. It is indeed at the intersection of 
disciplines that the most striking advances are expected 
to happen. 

The second video attached to this chapter Robots - 
The Journey Continues by Bruno Siciliano, Oussama 
Khatib and Torsten Kroger (2015) shows the intensive 
and vibrating evolution of robotics through the last fif¬ 
teen years tl-s &MWhl if ) 
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1.2 The Robotics Community 

The dissemination of research results and findings in 
archival publications and conference presentations has 
played an important role in the advancement of robotics 
in the past three decades. The extent of scientific ac¬ 
tivities in robotics has led to the establishment of 
professional societies and research networks devoted to 
the field, which have ultimately contributed to the build¬ 
ing of an international robotics community. 

In the early 1980s, robotics was entering a new 
era in its development following the earlier successes 
in industrial applications. With much energy and ex¬ 
citement, the pioneers of robotics launched a number 
of initiatives, which came to shape what became our 
field. The first robotics textbook by Richard Paul Robot 
Manipulators: Mathematics, Programming, and Con¬ 
trol appeared in 1981 (MIT Press). A year later, Mike 
Brady, Richard Paul, and their colleagues established 
the first journal entirely devoted to robotics, the Inter¬ 
national Journal of Robotics Research (IJRR), which 
was followed by the organization in 1983 of the first In¬ 
ternational Symposium of Robotics Research (ISRR). 
The intent was to foster the notion of robotics as 
a distinct scientific field and to create a broad com¬ 
munity of robotics. It was in the pursuit of these 
aims that the International Foundation of Robotics Re¬ 
search (IFRR) was later formally founded in 1986 
with the aim of promoting the development of robotics 
as a science, establishing the theoretical foundations 
and technology basis for its ever-expanding applica¬ 
tions, with emphasis on its potential role to benefit 
humans. 

The year 1984 brought a host of other major de¬ 
velopments in robotics with the establishment of the 
IEEE Robotics and Automation Council (RAC), led by 
George Saridis, the organization of the first IEEE Con¬ 
ference on Robotics (John Jarvis and Richard Paul) - 
which became the IEEE International Conference on 
Robotics and Automation (ICRA) the year after, the 
launching of the IEEE Journal of Robotics and Au¬ 
tomation (George Bekey), and RAC Newsletter (Wesley 
Snyder). The transformation of the council into the 
IEEE Robotics and Automation Society (RAS) was for¬ 
mally accomplished in 1989. With the birth of RAS, 
the IEEE Journal of Robotics and Automation became 
the IEEE Transactions on Robotics and Automation - 
which split into the IEEE Transactions on Robotics 
and the IEEE Transactions on Automation Science and 
Engineering later in 2004; then, the RAS Newsletter be¬ 
came the IEEE Robotics and Automation Magazine in 
1994. 

Over the following years, a number of other activi¬ 
ties were launched. New conferences such as IEEE/RSJ 


International Conference on Intelligent Robots and Sys¬ 
tems (IROS 1983), International Conference on Ad¬ 
vanced Robotics (ICAR 1983), International Sympo¬ 
sium on Experimental Robotics (ISER 1989), Robotics: 
Science and Systems (RSS 2005), as well as new 
journals such as Robotica (1983), Journal of Robotic 
Systems (1984, subsequently Journal of Field Robotics 
2006), Robotics and Autonomous Systems (1985), Ad¬ 
vanced Robotics (1986), Journal of Intelligent and 
Robotic Systems (1988), Autonomous Robots (1994), 
contributed to a further substantial growth of the 
robotics community. 

The year 1991 marked an important novelty into 
the ICRA flagship conference series with the creation 
of video proceedings, introduced earlier by Oussama 
Khatib and Vincent Hayward in ISER 1989. Since then 
video proceedings attained an important role for dis¬ 
semination of research work in our robotics community. 
This media has become an integral part of most archival 
publications in recent years. 

On the other hand, the introduction of graduate pro¬ 
grams in robotics in many academic institutions around 
the world in the last two decades is a clear illustration of 
the level of maturity reached by robotics as a scientific 
field. Further to Paul’s seminal book, other textbooks 
were published between the middle 1980s and middle 
1990s, such as Introduction to Robotics: Mechanics and 
Control by John Craig (1985), Robot Analysis and Con¬ 
trol by Harry Asada and Jean-Jacques Slotine (1986), 
Robotics: Control, Sensing, Vision, and Intelligence by 
King Sun Fu, Rafael Gonzalez and George Lee (1987), 
Robot Dynamics and Control by Mark Spong and M. 
Vidyasagar (1989), Foundation of Robotics by Tsu- 
neo Yoshikawa (1990), A Mathematical Introduction to 
Robotic Manipulation by Richard Murray, Zexiang Li 
and Shankar Sastry (1994), and Modelling and Control 
of Robot Manipulators by Lorenzo Sciavicco and Bruno 
Siciliano (1995). 

Engagement with students is essential for the future 
of the robotics field. Through a joint initiative between 
IEEE RAS and IFRR, the School of Robotics Science 
was founded in 2004. Summer schools have rapidly ex¬ 
panded in scope and frequency, covering a wide range 
of robotics research areas. Such programs are providing 
students and young researchers with an educational op¬ 
portunity of the highest scientific level, enabling them 
to feed and grow their potential to become the top 
robotics scientists in our community. 

Both IEEE RAS and IFRR have constantly pro¬ 
moted research, education, dissemination, and in¬ 
dustrial realizations through their own forums and 
through collaborations with many other organizations 
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in the robotics community world-wide, including Eu¬ 
ropean Robotics Research Network (EURON) - which 
has recently merged into the European Association 
of Robotics (euRobotics), International Federation of 
Robotics (IFR), and Robotics Society of Japan (RSJ). 

As our field continues to expand and embrace the 
challenges of operation in the human world, robotics 
is revealing a much wider range of applications and 
involving diverse scientific disciplines such as biome- 

1.3 This Handbook 

The intensive stream of robotics research documented 
in the literature culminates into this unique reference, 
which aims at collecting in one self-contained vol¬ 
ume the most significant achievements of our inter¬ 
national robotics community. The Springer Handbook 
of Robotics presents a full coverage of the field from 
its foundations, through the research areas, up to the 
new emerging applications of robotics. Accordingly, 
the material is organized in three logical layers reflect¬ 
ing the historical development of the field, as illustrated 
in Fig. 1.1. 

The foundations of robotics science, laid down in 
the first layer (Part A with its 14 chapters), address the 
theory of robot mechanics, sensing, planning, and con¬ 
trol. The consolidated methodologies and technologies 
of robot design (Part B with its 12 chapters), sensing 
and perception (Part C with its 8 chapters), manipu¬ 
lation and interfaces (Part D with its 9 chapters) and 
mobile and distributed robotics (Part E with its 9 chap¬ 
ters) are presented in the second layer. The third layer 


chanics, neuroscience, haptics, animation, surgery, and 
sensor networks among others. Today, new communi¬ 
ties of researchers and developers are forming, with 
growing connections to the core of robotics research. 
A strategic goal for our community is one of outreach 
and scientific cooperation across all regions and organi¬ 
zations. Our common endeavor as roboticists to reach 
this goal certainly holds the promise of exciting ad¬ 
vances toward new frontiers. 


is devoted to advanced applications, such as in field 
and service robotics (Part F with its 13 chapters) and 
human-centered and life-like robotics (Part G with its 
14 chapters). 

Part A presents the fundamental principles and 
methods that are used to model, design, and control 
a robotic system. All of the foundational topics are 
included in this part: kinematics, dynamics, mecha¬ 
nisms and actuation, sensing and estimation, model 
identification, motion planning, motion control, force 
control, redundant robots, robots with flexible ele¬ 
ments, robotic systems architectures and programming, 
behaviour-based systems, AI reasoning methods for 
robotics, robot learning. A chapter is devoted to each of 
these topics. The topics are expanded and applied to spe¬ 
cific robotic structures and systems in subsequent parts. 

Part B is concerned with the design of various 
robotic systems. Some of the obvious mechanical struc¬ 
tures that come to mind are arms, legs, wheels and 
hands; to this list can be added robots in the air 
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and subsea, as well as robot structures at the micro 
and nano scales. With a separate chapter devoted to 
design and evaluation performance, the chapters in 
this part successively examine limbed systems, paral¬ 
lel mechanisms, robot hands, snake-like and contin¬ 
uum robots, novel actuators for soft robotics, modular 
robots, biomimetic robots, wheeled robots, underwater 
robots, flying robots, and micro/nano robots. 

Part C covers different sensory modalities and in¬ 
tegration of sensor data across space and time to gen¬ 
erate models of robots and the external environment. 
Robotics is the intelligent coupling of perception and 
action and as such Part C complements Part B to build 
systems. This part of the handbook covers sensing 
across contact, proprioception and exteroception. The 
main sensing modalities such as force and tactile sens¬ 
ing, inertial sensing, global positioning system (GPS) 
and odometry, sonar sensing, range sensing, three- 
dimensional (3-D) vision for navigation and grasping 
visual object class recognition, and visual servoing 
are presented. Both basic sensor models, sensor data 
processing and associated representations are covered. 
Finally, a chapter on multisensor data fusion introduces 
the mathematical tools needed for integration of sensor 
information across space and time. 

Part D is concerned with interaction between robots 
and objects, or between humans and robots. Manipu¬ 
lation is supposed to handle an object through direct 
contact by arms or fingers or just a pusher, while in¬ 
terfaces are supposed to make either direct or indirect 
interaction between humans and robots. For enhancing 
dexterity in robot manipulation, motion for manipula¬ 
tion tasks, contact modelling and manipulation, grasp¬ 
ing, cooperative manipulation, mobility and manipula¬ 
tion, active manipulation for perception are addressed 
in the first half of this part. For achieving a skilful 
manipulation or power increase in a human-robot sys¬ 
tem, haptics, telerobotics, and networked robots are 
discussed in the second half of Part D. 

Part E covers a wide span of topics concerning the 
motion of robots in the environment. At first this part 
addresses world modelling, simultaneous localization 
and mapping, motion planning and obstacle avoidance. 
Then the modelling and control issues of legged robots, 
wheeled mobile robots, robots on rough terrain, under¬ 
water robots, and aerial robots are dealt with. This part 
completes Part A on foundations in the context of mo¬ 
bile robotics, and given the role of perception, is closely 
related to Part C on sensing. In addition, multiple mo¬ 
bile robot systems are discussed. 

Part F covers topics related to creating field and ser¬ 
vice application-based robots that operate in all types of 
environments. This includes applications ranging from 
industrial robots to robotics competitions and chal¬ 


lenges, through a diverse array of robotics applications: 
space, agriculture and forestry, construction, hazardous 
applications, mining, disaster scenarios, intelligent ve¬ 
hicles, medical and computer-integrated surgery, reha¬ 
bilitation and health-care, domestic environments. This 
part of the handbook, which draws on Parts A, B, C, D 
and E, describes how robots can be put to work. 

Part G covers topics related to creating robots 
that operate in human-centered environments. These 
include humanoids, human motion reconstruction, 
physical human-robot interaction, human robot aug¬ 
mentation, cognitive human-robot interaction, social 
robotics, socially assistive robotics, learning from 
humans, biologically-inspired robotics, evolutionary 
robotics, neurobotics, and perceptual robotics. The part 
concludes with the use of robotics for education, and 
the socio-ethical implications of robots. 

Each part is accompanied by a video which illus¬ 
trates the main concepts and the underlying challenges 
in common to the chapters in the part, while topical 
videos related to the specific contents of each chapter 
can be accessed through the web site of the multi- 
media contents associated with the handbook http:// 
handbookofrobotics.org/. 

The handbook has been conceived to provide a valu¬ 
able resource not only for robotics experts, but also 
for newcomers to this expanding held, e.g., engineers, 
medical doctors, computer scientists, and designers. In 
particular, it is important to underline the tutorial value 
of Part A to graduate students and post-docs, the re¬ 
search value of Parts B to E for a wider coverage of 
research in robotics, and the added value of Parts F and 
G to engineers and scientists interested in new applica¬ 
tions. 

The contents of the various chapters have been in¬ 
spired by a classic cut, i. e., avoiding the inclusion of 
on-going or not well-established methods. An objec¬ 
tive perspective has been taken, while covering multiple 
approaches, with the goal of ensuring a high archival 
value to the handbook. Each chapter is preceded by 
a short summary, and an introductory section provid¬ 
ing the state of the art in the area. The core sections 
are developed at a tutorial level. Lengthy mathemati¬ 
cal derivations have been avoided whenever possible, 
while the equations, tables, and algorithms are illus¬ 
trated in ready-to-use form. The final section of the 
chapter provides conclusions and topics for further 
reading. From the foundations to the social and ethi¬ 
cal implications of robotics, the eighty chapters of the 
handbook provide a comprehensive collection of five 
decades of progress in robotics. This sum is a testi¬ 
mony to the level of accomplishments in our field, and 
a premise of further advances towards new frontiers of 
robotics. 
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Robots - A 50 year journey 

available from http://handbookofrobotics.org/view-chapter/01/videodetails/805 
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The chapters contained in Part A, Robotics Founda¬ 
tions, present the fundamental principles and methods 
that are used to develop a robotic system. In order to 
perform the tasks that are envisioned for robots, many 
challenging problems have been uncovered in kine¬ 
matics, dynamics, design, actuation, sensing, model¬ 
ing, motion planning, control, programming, decision¬ 
making, task planning, and learning. Robots with re¬ 
dundant kinematic degrees of freedom or with flexible 
elements add to the complexity of these systems. The 
chapters in this part address the basic issues in each of 
these areas. 

Some of the basic problems in robotics are outlined 
as follows. Robots often consist of a large number of 
degrees of freedom so that they can provide the rich 
set of three-dimensional (3-D) motions that may be re¬ 
quired for a range of tasks. The kinematic and dynamic 
relationships between the joint actuators’ motion and 
torques, and the desired motion and force for a task can 
be very complex. The design of the link and joint struc¬ 
tures, as well as the actuation, to achieve the desired 
performance is also challenging. 

The robot is a nonlinear, coupled system which is 
difficult to model and control because of its complex 
dynamics. Kinematic redundancy and flexible elements 
in the robots increase this complexity. The problem is 
exacerbated when the environment is unstructured, and 
often sophisticated sensing and estimation techniques 
are required. 

In addition to control of the motion, control of the 
interaction forces between the robot and environment 
is needed when manipulating objects or interacting 
with humans. A fundamental robotics task is to plan 
collision-free motion for complex bodies from a start 
to a goal position among a collection of obstacles, and 
this can become an intractable computational problem. 

In order to achieve some of the intelligence ascribed 
to humans, robots will need to be equipped with sophis¬ 
ticated action planners that employ symbolic reasoning 
to move in dynamic, partially known environments. In 
other scenarios, robots need to execute behaviors that 
take advantage of dynamic interactions with the envi¬ 
ronment rather than rely solely on explicit reasoning 
and planning. Robot software architectures also have 
special needs because of all of these requirements. 
Robot learning will be necessary to generate actions 
and control for ever-changing task requirements and en¬ 
vironments, in order to achieve the level of autonomy 
envisioned. 

While the basic issues outlined in the previous para¬ 
graphs are addressed in this part, more depth can be 


found in other parts of the handbook. The kinematics, 
dynamics, mechanical design, and control principles 
and methods introduced in this part can be applied to 
robotic structures made up of arms, hands, and legs 
(Part B) as well as manipulators (Part D), wheeled 
and other mobile robots (Part E), and field and service 
robots (Part F). Force control is especially important for 
manipulators and their interfaces (Part D). The basic 
sensing and estimation techniques presented here are 
expanded and applied to specific sensing modalities in 
Part C. Motion planning is an important aspect of ma¬ 
nipulation (Part D) and mobile robots moving in the 
environment (Part E). Robotic systems architectures, 
behavior-based systems, artificial intelligence (AI) rea¬ 
soning methods, and robot learning are particularly 
important in mobile robots moving in the environment 
(Part E) and robots interacting with humans (Part G). 

With this overview of Part A, we now provide 
a brief synopsis of each chapter: 

Chapter 2, Kinematics, provides a number of rep¬ 
resentations and conventions to describe the motion 
of the bodies in a robotic mechanism. These include 
rotation matrices, Euler angles, quaternions, homoge¬ 
neous transformations, screw transformations, matrix 
exponential parameterization, and Pliicker coordinates. 
Representations of the kinematics of all common joint 
types are provided, along with a modified form of 
the Denavit-Hartenberg convention. These representa¬ 
tional tools are applied to compute the workspace, the 
forward and inverse kinematics, the forward and in¬ 
verse instantaneous kinematics, Jacobian, and the static 
wrench transmission. 

Chapter 3, Dynamics, presents the dynamic equa¬ 
tions of motion which provide the relationships be¬ 
tween actuation and contact forces acting on robot 
mechanisms, and the acceleration and motion trajec¬ 
tories that result. Efficient algorithms are provided 
for important dynamics computations which include 
inverse dynamics, forward dynamics, the joint-space in¬ 
ertia matrix, and the operational-space inertia matrix. 
The algorithms may be applied to fixed-base robots, 
mobile robots, and parallel robot mechanisms. Com¬ 
pact formulation of the algorithms results from using 
six-dimensional (6-D) spatial notation to describe rigid- 
body velocity, acceleration, inertia, etc. 

Chapter 4, Mechanisms and Actuation, focuses on 
the principles that guide the design and construction of 
robotic systems. The kinematic equations and Jacobian 
are used to characterize the work envelope and mechan¬ 
ical advantage, and guide the selection of the robot’s 
size and joint arrangement. The design of both serial 
and parallel robots is addressed. Practical consideration 
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is given to the design of the link and joint structures 
along with selection of the actuators and transmission 
drives to power the movement. Robot performance in 
terms of speed, acceleration, repeatability, and other 
measures is also addressed. 

Chapter 5, Sensing and Estimation, provides a brief 
overview of common sensing methods and estima¬ 
tion techniques that have found broad applicability in 
robotics. These provide information about the state of 
the environment and robot system. The presentation is 
structured according to a perception process model that 
includes sensing, feature extraction, data association, 
parameter estimation and model integration. Several 
common sensing modalities are introduced and charac¬ 
terized. Methods for estimation in linear and nonlinear 
systems are discussed, including statistical estimation, 
the Kalman filter, and sample-based methods. The sev¬ 
eral common representations for estimation are also 
introduced. 

Chapter 6, Model Identification, discusses methods 
for determining the kinematic and inertial parame¬ 
ters of robot manipulators. For kinematic calibration, 
the primary aim is to identify the geometric Denavit- 
Hartenberg parameters or equivalent, typically by sens¬ 
ing a combination of joint and endpoint positions. 
Inertial parameters in turn are estimated through the ex¬ 
ecution of a trajectory while additionally sensing one or 
more components of joint forces or torques. The chap¬ 
ter is organized such that both kinematic and inertial 
parameter identification are cast into a common frame¬ 
work of least-squares parameter estimation: common 
features relating to the identifiability of the parame¬ 
ters, adequacy of the measurement sets, and numerical 
robustness are identified and emphasized for both the 
kinematic and inertial parameters. 

Chapter 7, Motion Planning, addresses the funda¬ 
mental robotics task to plan collision-free motion for 
complex bodies from a start to a goal position among 
a collection of obstacles. The basic geometric path 
planning problem (Piano Mover’s problem) is intro¬ 
duced, but the focus of the chapter is on sampling-based 
planning methods because of their generally wider 
applicability. Planning with differential constraints is 
considered and is important for wheeled mobile robots. 
Extensions and variations to the basic motion planning 
problem, as well as advanced issues, are discussed at 
the end of the chapter. 

Chapter 8, Motion Control, focuses on the con¬ 
trol of the motion of a rigid manipulator. The main 
challenges addressed are the complexity of the non¬ 
linear, coupled dynamics and the model structural 
uncertainties. The chapter discusses topics ranging 


from independent-joint control and PID (proportional- 
integral-derivative) control to computed-torque control 
to manage the complex dynamics. Operational space (or 
task space) control is presented, and this is needed for 
control of tasks such as those associated with the end- 
effector. Adaptive and robust control are discussed to 
address the problems related to the uncertainties in the 
system. Other topics presented include trajectory gen¬ 
eration, digital implementation, and learning control. 

Chapter 9, Force Control, focuses on the control of 
the interaction forces between a robotic system and its 
environment. The chapter groups interaction control in 
two categories: indirect and direct force control, which 
achieve the control without (indirect) or with (direct) 
explicit closure of a force feedback loop. Impedance 
control and hybrid force/motion control are examples of 
each, respectively. The fundamental problem of interac¬ 
tion tasks modeling is presented to provide the basis for 
the force control schemes. 

Chapter 10, Redundant Robots, addresses the mo¬ 
tion generation and control of manipulators with re¬ 
dundant kinematic degrees of freedom. Kinematic re¬ 
dundancy affords a robot with an increased level of 
dexterity that may be used to, e.g., avoid singulari¬ 
ties, joint limits and workspace obstacles, and also to 
minimize joint torques, energy, or other suitable perfor¬ 
mance criteria. The chapter discusses inverse kinematic 
redundancy resolution schemes which are arranged in 
two main categories, namely those based on the op¬ 
timization of suitable performance criteria and those 
relying on the augmentation of the task space. The 
use of kinematic redundancy for fault tolerance is also 
analyzed. 

Chapter 11, Robots with Flexible Elements, ad¬ 
dresses the dynamic modeling and control of robots 
with flexibility in the joints and links. Because the con¬ 
trol methods developed to compensate for joint versus 
link flexibility are structurally different, the chapter is 
organized such that these two types of flexibility are ex¬ 
amined independently. The methods, however, can be 
extended to the case when both joint and link flexibil¬ 
ities are present, possibly even dynamically interacting 
at the same time. The chapter also examines the typical 
sources of flexibility in industrial robots. 

Chapter 12, Robotic Systems Architectures and 
Programming, presents the software architectures and 
supporting programming tools and environments that 
have been developed for robotic systems. Robot archi¬ 
tectures have special needs because of the requirements 
of the robot to interact asynchronously, in real time, 
with an uncertain, often dynamic, environment. The 
chapter discusses the major types of architectural com- 
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ponents for a layered robot control architecture - be¬ 
havioral control, executives, and task planners - along 
with the commonly used techniques for interconnecting 
these components. 

Chapter IB, Behavior-Based Systems, describes 
a control methodology aimed at situated robots oper¬ 
ating in unconstrained, challenging, and dynamic con¬ 
ditions in the real world. Distributed behaviors are used 
as the underlying building blocks, allowing behavior- 
based systems to take advantage of dynamic interac¬ 
tions with the environment rather than rely solely on 
explicit reasoning and planning. The focus of the chap¬ 
ter is to provide the basic principles of behavior-based 
systems and their use in autonomous control problems 
and applications. The chapter presents several different 
classes of learning methods, but in all cases behaviors 
are used as the underlying building blocks for the learn¬ 
ing process. 

Chapter 14, AI Reasoning Methods for Robotics, 
sketches the main robotics-relevant topics of symbol- 
based AI reasoning. Reasoning on a mobile robot is 
especially challenging because of its dynamic, par¬ 
tially known environment. This chapter describes basic 
methods of knowledge representation and inference, 
covering both logic- and probability-based approaches. 
Results for several types of practical, robotics-related 
reasoning tasks are given, with an emphasis on tem¬ 
poral and spatial reasoning. Plan-based robot control is 


described in some detail, as it is a particularly obvious 
form of employing reasoning to the end of improving 
robot action. 

Chapter 15, Robot Learning, surveys work in ma¬ 
chine learning for techniques that are used for learning 
control and behavior generation in robots. In the future, 
robots will no longer be used to only execute the same 
task thousands of times, but rather they will be faced 
with thousands of different tasks that rarely repeat in 
an ever-changing environment. Robot learning will be 
necessary to achieve the high degree of autonomy envi¬ 
sioned. This chapter focuses on the core robot learning 
approaches capable of learning action generation and 
control, and in particular, techniques for model learn¬ 
ing and reinforcement learning. 

Part A presents the fundamental principles and 
methods that are used to model, design, and con¬ 
trol a robotic system. All of the foundational top¬ 
ics are included in this part: kinematics, dynamics, 
mechanisms and actuation, sensing and estimation, 
model identification, motion planning, motion control, 
force control, redundant robots, robots with flexible 
elements, robotic systems architectures and program¬ 
ming, behavior-based systems, AI reasoning methods 
for robotics, and robot learning. A chapter is devoted 
to each of these topics. The topics are expanded and 
applied to specific robotic structures and systems in 
subsequent parts. 
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2. Kinematics 


NJ 


Kenneth J. Waldron, James Schmiedeler 


Kinematics pertains to the motion of bodies 
in a robotic mechanism without regard to the 
forces/torques that cause the motion. Since robotic 
mechanisms are by their very essence designed 
for motion, kinematics is the most fundamen¬ 
tal aspect of robot design, analysis, control, and 
simulation. The robotics community has focused 
on efficiently applying different representations of 
position and orientation and their derivatives with 
respect to time to solve foundational kinematics 
problems. 

This chapter will present the most useful rep¬ 
resentations of the position and orientation of 
a body in space, the kinematics of the joints 
most commonly found in robotic mechanisms, 
and a convenient convention for representing 
the geometry of robotic mechanisms. These rep¬ 
resentational tools will be applied to compute 
the workspace, the forward and inverse kine¬ 
matics, the forward and inverse instantaneous 
kinematics, and the static wrench transmission 
of a robotic mechanism. For brevity, the focus 
will be on algorithms applicable to open-chain 
mechanisms. 

The goal of this chapter is to provide the reader 
with general tools in tabulated form and a broader 
overview of algorithms that can be applied to¬ 
gether to solve kinematics problems pertaining to 
a particular robotic mechanism. 
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Robotics Foundations 


2.1 Overview 

Unless explicitly stated otherwise, robotic mechanisms 
are systems of rigid bodies connected by joints. The 
position and orientation of a rigid body in space are 
collectively termed the pose. Therefore, robot kine¬ 
matics describes the pose, velocity, acceleration, and 
all higher-order derivatives of the pose of the bod¬ 
ies that comprise a mechanism. Since kinematics does 
not address the forces/torques that induce motion, this 
chapter focuses on describing pose and velocity. These 
descriptions are foundational elements of dynamics 
(Chap. 3), motion planning (Chap. 7), and motion con¬ 
trol (Chap. 8) algorithms. 

Among the many possible topologies in which 
systems of bodies can be connected, two of particu¬ 
lar importance in robotics are serial chains and fully 
parallel mechanisms. A serial chain is a system of 
rigid bodies in which each member is connected to 


two others, except for the first and last members 
that are each connected to only one other member. 
A fully parallel mechanism is one in which there are 
two members that are connected together by multi¬ 
ple chains of other members and joints. In practice, 
each of these chains is often itself a serial chain. 
This chapter focuses almost exclusively on algorithms 
applicable to serial chains. Parallel mechanisms are 
dealt with in more detail in Chap. 18. Another im¬ 
portant topology is the tree structure, which is sim¬ 
ilar to a serial chain in that it has no closed loops, 
but differs from a serial chain in that each mem¬ 
ber might have multiple members connected to it, 
forming multiple branches. A serial chain is actu¬ 
ally just a special case of a tree structure with no 
branches. Tree structures are addressed in greater depth 
in Chap. 3. 


2.2 Position and Orientation Representation 


Spatial, rigid-body kinematics can be viewed as a com¬ 
parative study of different ways of representing the pose 
of a body. Translations and rotations, referred to in 
combination as rigid-body displacements, are also ex¬ 
pressed with these representations. No one approach is 
optimal for all purposes, but the advantages of each can 
be leveraged appropriately to facilitate the solution of 
different problems. 

The minimum number of coordinates required to 
locate a body in Euclidean space is six. Many rep¬ 
resentations of spatial pose employ sets with super¬ 
abundant coordinates in which auxiliary relationships 
exist among the coordinates. The number of indepen¬ 
dent auxiliary relationships is the difference between 
the number of coordinates in the set and six. 

This chapter and those that follow it make frequent 
use of coordinate frames or simply frames. A coor¬ 
dinate frame i consists of an origin, denoted O,, and 
a triad of mutually orthogonal basis vectors, denoted 
(Xifi Zi), that are all fixed within a particular body. 
The pose of a body will always be expressed rela¬ 
tive to some other body, so it can be expressed as the 
pose of one coordinate frame relative to another. Sim¬ 
ilarly, rigid-body displacements can be expressed as 
displacements between two coordinate frames, one of 
which may be referred to as moving, while the other 
may be referred to as fixed. This indicates that the ob¬ 
server is located in a stationary position within the 
fixed frame, not that there exists any absolutely fixed 
frame. 


2.2.1 Position and Displacement 

The position of the origin of coordinate frame i relative 
to coordinate frame j can be denoted by the 3x1 vector 



The components of this vector are the Cartesian co¬ 
ordinates of Oi in the j frame, which are the projections 
of the vector J p, onto the corresponding axes. The vec¬ 
tor components could also be expressed as the spherical 
or cylindrical coordinates of 0, in the j frame. Such 
representations have advantages for analysis of robotic 
mechanisms including spherical and cylindrical joints. 

A translation is a displacement in which no point 
in the rigid body remains in its initial position and all 
straight lines in the rigid body remain parallel to their 
initial orientations. (The points and lines are not neces¬ 
sarily contained within the boundaries of the finite rigid 
body, but rather, any point or line in space can be taken 
to be rigidly fixed in a body.) The translation of a body 
in space can be represented by the combination of its 
positions prior to and following the translation. Con¬ 
versely, the position of a body can be represented as 
a translation that takes the body from a starting position 
(in which the coordinate frame fixed to the body co¬ 
incides with the fixed coordinate frame) to the current 
position (in which the two frames are not coincident). 
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Thus, any representation of position can be used to cre¬ 
ate a representation of displacement, and vice versa. 

2.2.2 Orientation and Rotation 

There is significantly greater breadth in the representa¬ 
tion of orientation than in that of position. This section 
does not include an exhaustive summary, but focuses on 
the representations most commonly applied to robotic 
mechanisms. 

A rotation is a displacement in which at least one 
point in the rigid body remains in its initial position 
and not all lines in the body remain parallel to their ini¬ 
tial orientations. For example, a body in a circular orbit 
rotates about an axis through the center of its circular 
path, and every point on the axis of rotation is a point 
in the body that remains in its initial position. As in the 
case of position and translation, any representation of 
orientation can be used to create a representation of ro¬ 
tation, and vice versa. 

Rotation Matrices 

The orientation of coordinate frame i relative to coor¬ 
dinate frame j can be denoted by expressing the basis 
vectors (£, y,- z,) in terms of the basis vectors ( XjyjZj ). 
This yields which when written together as 

a 3 x 3 matrix is known as the rotation matrix. The com¬ 
ponents of iR,- are the dot products of the basis vectors 
of the two coordinate frames. 


X 1 ' Xj 

yrXj 

Zi-Xj\ 

X, ■yj 

9i-y.i 

Zi ■yj 

xr% 

yrzj 

ZrZj) 


Because the basis vectors are unit vectors and the dot 
product of any two unit vectors is the cosine of the angle 
between them, the components are commonly referred 
to as direction cosines. 

An elementary rotation of frame i about the Zj axis 
through an angle 8 is 

/ cos 8 — sin 8 0\ 

Rz(0)=jsin0 cos 8 Oj, (2.2) 

Vo 0 1/ 

while the same rotation about the y 7 - axis is 

( cos (9 0 sin(9\ 

0 1 0 , (2.3) 

— sin 8 0 cos 8 J 

and about the xj axis is 

/! 0 0 \ 

Rr(0) = I 0 cos0 — sin# I . (2.4) 

Vo sin (9 cos@ / 


Rotation matrices are combined through simple matrix 
multiplication such that the orientation of frame i rela¬ 
tive to frame k can be expressed as 

R = R, R . 

The rotation matrix 'R, contains nine elements, 
while only three parameters are required to define the 
orientation of a body in space. Therefore, six aux¬ 
iliary relationships exist among the elements of the 
matrix. Because the basis vectors of coordinate frame i 
are mutually orthonormal, as are the basis vectors of 
coordinate frame j, the columns of 'R, formed from 
the dot products of these vectors are also mutually 
orthonormal. A matrix composed of mutually orthonor¬ 
mal vectors is known as an orthogonal matrix and has 
the property that its inverse is simply its transpose. This 
property provides the six auxiliary relationships. Three 
require the column vectors to have unit length, and three 
require the column vectors to be mutually orthogonal. 
Alternatively, the orthogonality of the rotation matrix 
can be seen by considering the frames in reverse order. 
The orientation of coordinate frame j relative to coordi¬ 
nate frame i is the rotation matrix R ( whose rows are 
clearly the columns of the matrix 'R, . 

In summary, R, is the rotation matrix that trans¬ 
forms a vector expressed in coordinate frame i to 
a vector expressed in coordinate frame j. It provides 
a representation of the orientation of frame i relative 
to j and thus, can be a representation of rotation from 
frame i to frame j. Table 2.1 lists the equivalent rota¬ 
tion matrices for the other representations of orientation 
listed in this section. Table 2.2 contains the conversions 
from a known rotation matrix to these other representa¬ 
tions. 

Euler Angles 

For a minimal representation, the orientation of coor¬ 
dinate frame i relative to coordinate frame j can be 
denoted as a vector of three angles (a,f},y) T . These 
angles are known as Euler angles when each represents 
a rotation about an axis of a moving coordinate frame. 
In this way, the location of the axis of each succes¬ 
sive rotation depends upon the preceding rotation(s), so 
the order of the rotations must accompany the three an¬ 
gles to define the orientation. For example, the symbols 
(a, yS, y) T are used throughout this handbook to indicate 
Z-Y-X Euler angles. Taking the moving frame i and the 
fixed frame j to be initially coincident, a is the rotation 
about the z axis of frame i, fi is the rotation about the 
rotated y axis of frame i, and finally, y is the rotation 
about the twice rotated x axis of frame i. The equiv¬ 
alent rotation matrix R, is given in Table 2.1. Z-T-Z 
and Z-X-Z Euler angles are other commonly used con- 
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Table 2.1 Equivalent rotation matrices for various representations of orientation, with abbreviations eg '■= cos 9, sg : = 
sin 9, and vg := 1 — cos 9 


/eaCp c ot s gSy s aCy Ca s pCy Sa s y 

Z-Y-X Euler angles (a, /?, y) T - , r j . = I s a cp s a sps y + c a c y s a spc y - c a s v 

\ -sp cps y cpc y 

/C<t>CQ e^SgS^x C([)SQC\j/ T SffiSi// 

X-Y-Z fixed angles (f, 0, </>) T J Rj = I s^cg s^sgs^ + s^sgc-^ — c^s-^ 

\—sg cgs■,), ege^ 

( w x v 0 + c 8 WxWyVg — w z sg w x w z vg + w y sg 

w x VJyVg + w z sg WyVg + cg WyW z vg — w x sg 

w x w z vg — WySg WyW z vg + w x sg w z vg + eg 

A—2(e| + ef) 2(eiC2 — eoQ) 2 (fie 3 + eo£2)\ 

Unit quaternions (e 0 ei €2 € 3 ) T J R; = 2 (ei€ 2 + £oq) 1-2 (ef + e 3 ) 2 (e 2 e 3 — eod) 

\2 (eie 3 — 6oe 2 ) 2(f 2 € 3 + eofi) 1 —2 (ef + ef) / 


ventions from among the 12 different possible orders of 
rotations. 

Regardless of the order of rotations, an Euler angle 
representation of orientation always exhibits a singular¬ 
ity when the first and last rotations both occur about the 

Table 2.2 Conversions from a rotation matrix to various 
representations of orientation 
Rotation matrix: 


same axis. This can be readily seen in the second block 
of Table 2.2 wherein the angles a and y are undefined 
when p = ±90°. (For Z-Y-Z and Z-X-Z Euler angles, 
the singularity occurs when the second rotation is 0° 
or 180°.) This creates a problem in relating the angu¬ 
lar velocity vector of a body to the time derivatives of 
Euler angles, which somewhat limits their usefulness in 
modeling robotic systems. This velocity relationship for 
Z-Y-X Euler angles is 


m 

n 2 

7*13 

7*21 

7*22 

7*23 

/31 

7*32 

7*33. 


Z-Y-X Euler angles (a , fi, y) T : 

P = Atan2 r 3 i, yjt\ x + rj l ^ 

“= Atan2 (327’ eSV) 

y = Aran2 ( 

X-Y-Z fixed angles ( 1 jr, 6, < p) T \ 

0 = Atan2 f-r 3i , Jr 2 n + 

x l / = Man2 (^e-^e) 

<P = Atan2 (^e-^rg) 

Angle axis 6 w: 

9 = cos" 1 


1 


( r 32 — r 23 
w = 2^8 I '13 r 31 

\r 2 i - r i2 

Unit quaternions (ro < 1 £ 2 € 3 ) T : 
Co = I V 1 + r ll + r 22 + r 33 


4eo 
m—m 


4<Fo 
21 — n: 

4<?o 



sin y 

cos y \ . 

°>*\ 

cos y cos P 

— sin y cos /J 1 


sin y sin /3 

cos y sin /J / 

w 



(2.5) 

j 1 = is given in moving frame i. 


In some circumstances, the inverse of this relationship 
may be required. 


/VA / -sin/8 0 1\ /a\ 

I (Oy I = I cos/lsiny cosy 0 II ft . (2.6) 

\ 0 ) z / \cos/? cos y — siny 0/ \y J 

Fixed Angles 

A vector of three angles can also denote the orienta¬ 
tion of coordinate frame i relative to coordinate frame j 
when each angle represents a rotation about an axis of 
a fixed frame. Appropriately, such angles are referred to 
as fixed angles, and the order of the rotations must again 
accompany the angles to define the orientation. X-Y-Z 
fixed angles, denoted here as (ifr, 9, (/>) T , are a common 
convention from among the, again, 12 different possi¬ 
ble orders of rotations. Taking the moving frame i and 
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the fixed frame j to be initially coincident, \[/ is the yaw 
rotation about the fixed Xj axis, 9 is the pitch rotation 
about the fixed y ; - axis, and ip is the roll rotation about 
the fixed Zj axis. As can be seen by comparing the re¬ 
spective equivalent rotation matrices in Table 2.1 and 
the respective conversions in Table 2.2, a set of X-Y-Z 
fixed angles is exactly equivalent to the same set of Z-Y- 
X Euler angles (a = cp, = 9, and y = i/r). This result 
holds in general such that three rotations about the three 
axes of a fixed frame define the same orientation as 
the same three rotations taken in the opposite order 
about the three axes of a moving frame. Likewise, all 
fixed-angle representations of orientations suffer from 
the singularity discussed for Euler angles. Also, the re¬ 
lationship between the time derivatives of fixed angles 
and the angular velocity vector is similar to the relation¬ 
ship for Euler angles. 

Angle-Axis 

A single angle 9 in combination with a unit vector w 
can also denote the orientation of coordinate frame i 
relative to coordinate frame j. In this case, frame i is 
rotated through the angle 9 about an axis defined by 
the vector w = (w x Wy w z ) r relative to frame j. The 
vector w is sometimes referred to as the equivalent 
axis of a finite rotation. The angle-axis representation, 
typically written as either 9w or (6w x Owy 9w z ) T , is 
superabundant by one because it contains four param¬ 
eters. The auxiliary relationship that resolves this is 
the unit magnitude of vector w. Even with this aux¬ 
iliary relationship, the angle-axis representation is not 
unique because rotation through an angle of —9 about 
— w is equivalent to a rotation through 9 about w. Ta¬ 
ble 2.3 contains the conversions from the angle-axis 
representation to unit quaternions and vice versa. The 
conversions from these two representations to Euler 
angles or fixed angles can be easily found by using 

Table 2.3 Conversions from angle-axis to unit quaternion 
representations of orientation and vice versa 

Angle-axis 9 w to unit quaternion (eo €1 f2 £a) T : 

{[) = cos J 

£1 = w x sin j 
€2 = Wy sin ® 

£3 = w z sin §■ 

Unit quaternion (eo £1 £2 £3) T to angle-axis 6 w: 

8 = 2 cos~* £0 



the conversions in Table 2.2 in conjunction with the 
equivalent rotation matrices in Table 2.1. Velocity re¬ 
lationships are more easily dealt with using the closely 
related quaternion representation. 

Quaternions 

The quaternion representation of orientation due to 
Hamilton [2.1], while largely superseded by the sim¬ 
pler vector representations of Gibbs [2.2] and Grafi- 
mann [2.3], is extremely useful for problems in robotics 
that result in representational singularities in the vec¬ 
tor/matrix notation [2.4]. Quaternions do not suffer 
from singularities as Euler and fixed angles do. 

A quaternion e is defined to have the form 

e = e 0 + eii + 62 j + e 3 k, 

where the components eo, 61 , 62 , and 63 are scalars, 
sometimes referred to as Euler parameters, and i, j, 
and k are operators. The operators are defined to satisfy 
the following combinatory rules 

ii = jj = kk = — 1 , 
ij = k ,jk = i , ki = j , 
ji = —k , kj = —i , ik = —j . 

Two quaternions are added by adding the respective 
components separately, so the operators act as sepa¬ 
rators. The null element for addition is the quaternion 
0 = 0 + Oi + Oj + Ok, and quaternion sums are associa¬ 
tive, commutative, and distributive. The null element 
for multiplication is I = 1 + Oi + Oj + Ok, as can be seen 
using Ie = e for any quaternion e. Quaternion products 
are associative and distributive, but not commutative, 
and following the conventions of the operators and ad¬ 
dition, have the form 

ab = aobo — a\b\— 02^2 — < 33^3 

+ (flo^i +aifco + a 2 b^-a 3 b2)i 
+ (flo ^2 + a 2 bo + a 2 b\ —a\b 2 )j 
+ (uo ^3 T Q 2 bo T a\b 2 — a 2 b\)k . (2.7) 

It is convenient to define the conjugate of a quaternion 

e = €o~eii — eij — e 2 k , 

so that 

ee = ee = 6 q + €) + + 63 . 

A unit quaternion can then be defined such that ee = 1. 
Often, eo is referred to as the scalar part of the quater¬ 
nion, and (ei 62 63 ) T is referred to as the vector part. 
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Unit quaternions are used to describe orientation, 
and the unit magnitude provides the auxiliary relation¬ 
ship to resolve the use of superabundant (four) coor¬ 
dinates. A vector is defined in quaternion notation as 
a quaternion with eo = 0. Thus, a vector/? = (p x p y p z ) J 
can be expressed as a quaternion p = p x i + p y j + p z k. 
For any unit quaternion e, the operation e/?e performs 
a rotation of the vector p about the direction (e \ e 2 63 ) T . 
This is clearly seen by expanding the operation epe and 
comparing the results with the equivalent rotation ma¬ 
trix listed in Table 2.1. Also, as shown in Table 2.3, unit 
quaternions are closely related to the angle-axis repre¬ 
sentation. eo corresponds (but is not equal) to the angle 
of rotation, while 61 , 62 , and €3 define the axis of rota¬ 
tion. 

For velocity analysis, the time derivative of the 
quaternion can be related to the angular velocity vec¬ 
tor as 


matrices are combined together in a compact notation. 
Any vector l r expressed relative to the i coordinate 
frame can be expressed relative to the j coordinate 
frame if the position and orientation of the i frame 
are known relative to the j frame. Using the notation 
of Sect. 2.2.1, the position of the origin of coordinate 
frame i relative to coordinate frame j can be denoted 
by the vector 7 /?, = ( , p^ 7 /?J 7 /?;) T . Using the notation of 
Sect. 2.2.2, the orientation of frame i relative to frame j 
can be denoted by the rotation matrix 7 !?,. Thus, 

j r= J %‘r+jpi. (2.9) 

This equation can be written 



where 


( 2 . 8 ) 


( e°\ /—ei 62 —e A 

61 _ £ €q €3 —€2 

<?2 2 —63 6o 61 

Y<b/ \ e 2 -€i eo / 

where ( a> x ,a> y ,a > z )' = 7 ft?,- is given in fixed frame j. 
Defining e \-3 = (ei, 62 , 63 ) T , it is straightforward to ver¬ 
ify that 


1 ; T 

60 = - -■'ft?; 61:3 

61:3 = 7 (e 0 7 6?;-6i:3 X J (O l ) . 

In the case that the angular velocity is given in the 
moving coordinates, these rate equations take a similar 
form. 


1 i T 

60 = - - ft?; 61:3 
61:3 = ^ (fo'ft?l+ 61:3 X 'ft?/) 



( 2 . 11 ) 


is the 4x4 homogeneous transformation matrix and 
( 7 r 1) T and ('r 1) T are the homogeneous representations 
of the position vectors 7 r and 'r. The matrix 7 T, trans¬ 
forms vectors from coordinate frame i to coordinate 
frame j. Its inverse 7 T~ 1 transforms vectors from co¬ 
ordinate frame j to coordinate frame i. 


?Tr 1 = 'T; = 




( 2 . 12 ) 


Composition of 4 x 4 homogeneous transformation ma¬ 
trices is accomplished through simple matrix multipli¬ 
cation, just as in the case of 3 x 3 rotation matrices. 
Therefore, A ’T ; = A ’T/T;. Since matrix multiplications 
do not commute, the order or sequence is important. 

The homogeneous transformation of a simple rota¬ 
tion about an axis is sometimes denoted Rot such that 
a rotation of 6 about an axis z is 


The corresponding rate matrix for this case is the same 
as ( 2 . 8 ) with the exception that the off-diagonal ele¬ 
ments in the bottom 3x3 submatrix have opposite sign. 

While a unit quaternion represents only the orienta¬ 
tion of a body, quaternions may be dualized [2.5-7] to 
create an algebra that provides a description of the posi¬ 
tion and orientation of a body in space. Other combined 
representations are discussed in the following sections. 

2.2.3 Homogeneous Transformations 

The preceding sections have addressed representations 
of position and orientation separately. With homoge¬ 
neous transformations, position vectors and rotation 


Rot(z, 8 ) 


(cos 8 
sin 8 
0 

V 0 


— sin 8 
cos 8 
0 
0 


0 0 \ 

0 0 
1 0 ' 

0 \) 

(2.13) 


Similarly, the homogeneous transformation of a simple 
translation along an axis is sometimes denoted Trans 
such that a translation of d along an axis x is 


Transti, d) 


(\ 

0 

0 

\0 


0 

1 

0 

0 


0 

0 

1 

0 


d\ 

0 

0 

V 


(2.14) 
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Homogeneous transformations are particularly at¬ 
tractive when compact notation is desired and/or when 
ease of programming is the most important considera¬ 
tion. This is not, however, a computationally efficient 
representation since it introduces a large number of ad¬ 
ditional multiplications by ones and zeros. Although 
homogeneous transformation matrices technically con¬ 
tain 16 elements, four are defined to be zero or one, 
and the remaining elements are composed of a rotation 
matrix and a position vector. Therefore, the only truly 
superabundant coordinates come from the rotation ma¬ 
trix component, so the relevant auxiliary relationships 
are those associated with the rotation matrix. 

2.2.4 Screw Transformations 

The transformation in (2.9) can be viewed as composed 
of a rotation between coordinate frames i and j and 
a separate displacement between those frames. To get 
from frame i to frame j, one could perform the rotation 
first, followed by the displacement, or vice versa. Alter¬ 
natively, the spatial displacement between the frames 
can be expressed, except in the case of a pure transla¬ 
tion, as a rotation about a unique line combined with 
a translation parallel to that line. 

Chasles' Theorem 

Chasles’ theorem, in the form stated by Chirikjian and 
Kyatkin [2.8], has two parts. The first states that: 

Any displacement of a body in space can be accom¬ 
plished by means of a translation of a designated 
point from its initial to its final position, followed 
by a rotation of the whole body about that point to 
bring it into its final orientation. 

The second part states that: 

Any displacement of a body in space can be accom¬ 
plished by means of a rotation of the body about 
a unique line in space accompanied by a transla¬ 
tion of the body parallel to that line. 

Such a line is called a screw axis, and it is this second 
result that is usually thought of as Chasles’ theorem. 

The first part of the theorem is almost axiomatic. 
A designated point in a body anywhere in Euclidean 
space can be displaced from a given initial position 
to a given final position. By further requiring that all 
points in the body traverse the same displacement, the 
body translates so that the designated point moves from 
its initial position to its final position. The body can then 
be rotated about that point into any given final orienta¬ 
tion. 

The second part of the theorem depends on this 
representation of a spatial displacement and requires 


a more complex argument. A preliminary theorem due 
to Euler allows greater specificity about the rotation of 
the body: Any displacement of a body in which one 
point remains fixed is equivalent to a rotation of that 
body about a unique axis passing through that point. 
Geometrically, embedding three points in the moving 
body and letting one be the fixed point about which 
rotation occurs, each of the other two will have ini¬ 
tial and final positions. The right bisector planes of the 
lines joining the initial and final positions in each case 
necessarily contain the fixed point. Any line in the bi¬ 
sector plane can be the axis of a rotation that carries 
the corresponding point from its initial to its final po¬ 
sition. Therefore, the unique line common to the two 
bisector planes is such that rotation about it will carry 
any point in the body from its initial to its final posi¬ 
tion. The rigidity condition requires that all planes in 
the body that contain that line rotate through the same 
angle. 

For any rotation of a rigid body described by a ro¬ 
tation matrix J R,, Euler’s theorem states that there is 
a unique eigenvector w such that 

'RjW = w , (2.15) 

where w is a unit vector parallel to the axis of rotation. 
This expression requires a unit eigenvalue of 7 R, cor¬ 
responding to the eigenvector w. The remaining two 
eigenvalues are cos 9 ± i sin 9, where i is the complex 
operator and 9 is the angle of rotation of the body about 
the axis. 

Combining the first part of Chasles’ theorem with 
Euler’s theorem, a general spatial displacement can be 
expressed as a translation taking a point from its ini¬ 
tial to its final position, followed by a unique rotation 
about a unique axis through that point that carries the 
body from its initial to its final orientation. Resolv¬ 
ing the translation into components in the direction of 
and orthogonal to the rotation axis, every point in the 
body has the same displacement component in the di¬ 
rection of the axis because rotation about it does not 
affect that component. Projected onto a plane normal 
to the rotation axis, the kinematic geometry of the dis¬ 
placement is identical to that of planar motion. Just as 
there is a unique point in the plane about which a body 
could be rotated between two given positions, there is 
a unique point in the projection plane. If the rotation 
axis is moved to pass through that point, the spatial dis¬ 
placement can be accomplished by a rotation about that 
axis combined with a translation along it, as stated by 
the theorem. 

The line about which rotation takes place is called 
the screw axis of the displacement. The ratio of the lin¬ 
ear displacement d to the rotation 9 is referred to as the 
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pitch h of the screw axis [2.4]. Thus, 

d = h6 . (2.16) 

The screw axis of a pure translation is not unique. Any 
line parallel to the translation can be regarded as the 
screw axis, and since the rotation 9 is zero, the axis of 
a translation is said to have infinite pitch. 

A screw axis is most conveniently represented in 
any coordinate frame by means of a unit vector w par¬ 
allel to it and the position vector p of any point lying on 
it. Additional specification of the pitch h and the angle 
of rotation 9 completely defines the location of a second 
coordinate frame relative to the first frame. Thus, a to¬ 
tal of eight coordinates define a screw transformation, 
which is superabundant by two. The unit magnitude 
of vector w provides one auxiliary relationship, but in 
general, there is no second auxiliary relationship be¬ 
cause the same screw axis is defined by all points lying 
on it, which is to say that the vector p contains one free 
coordinate. 

Algebraically, a screw displacement is represented 
by 

J r = J R;(‘r — p) + dw + p . (2.17) 

Comparing this expression to (2.9) yields 

J Pi = dw + (l 3x3 —'ll;) P ■ (2.18) 

1 . 3 x 3 denotes the 3x3 identity matrix. An expression 
for d is easily obtained by taking the inner product of 
both sides of the equation with w . 

d = w Tj pt . (2.19) 

The matrix I 3 X 3 — J R i is singular, so (2.18) cannot be 
solved to give a unique value of p, but since p can rep¬ 
resent any point on the screw axis, this would not be 
appropriate. One component of p can be arbitrarily cho¬ 
sen, and any two of the component equations can then 
be solved to find the two other components of p. All 
other points on the screw axis are then given by p + kw , 
where k can take any value. 

Table 2.4 contains the conversions between screw 
transformations and homogeneous transformations. 
Note that the equivalent rotation matrix for a screw 
transformation has the same form as the equivalent ro¬ 
tation matrix for an angle-axis representation of orien¬ 
tation in Table 2.1. Also, the auxiliary relationship that 
the vector p be orthogonal to the screw axis (w J p = 0 ) 
is used in Table 2.4 to provide a unique conversion to 
the screw transformation. The inverse result, that of 
finding the rotation matrix 'R, and the translation 'p, 
corresponding to a given screw displacement, is found 
from Rodrigues’ equation. 


Rodrigues' Equation 

Given a screw axis, the angular displacement of a body 
about it, and the translation of the body along it, the 
displacement of an arbitrary point in that body can 
be found. Viewing a matrix transformation as describ¬ 
ing the displacement of the body, this is equivalent to 
finding the matrix transformation equivalent to a given 
screw displacement. 

Referring to Fig. 2.1, the position vectors of a point 
before and after a screw displacement can be geometri¬ 
cally related as 

■'r = 'r + dw + sin 9w x ('r — p) 

— (1 — cos 9)(‘r — p) — ('r — p) ■ ww , ( 2 . 20 ) 

where 'r and 'r denote the initial and final positions of 
the point, w and p specify the screw axis, and 9 and d 
give the displacement about it. This result is usually re¬ 
ferred to as Rodrigues’ equation [2.9], which can be 
written as a matrix transformation [ 2 . 10 ], 

j r= j R‘r + j Pi , (2.21) 

since, when expanded, it gives three linear equations for 
the components of >r in terms of those of 'r. 

( w x vg + Cg w x WyVg — w z Sg w x w z vg + WySg\ 
w x WyVg + w z sg WyVg + cg u>yW z vg — w x sg I 
w x w z vg — WySg WyW z vg + w x sg w z vg+cg / 

J Pi = ( 13 X 3 - ; R) P+hd-W , 



Fig. 2.1 Initial and final positions of an arbitrary point in 
a body undergoing a screw displacement; 'r is the position 
of the point relative to the moving frame, which is coinci¬ 
dent with the fixed frame j in its initial position; 'r is the 
position of the point relative to the fixed frame after the 
screw displacement of the moving body 
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Table 2.4 Conversions from a screw transformation to a homogeneous transformation and vice versa, with abbreviations 
eg := cos 9, sg := sin 8 , and vg := 1 — cos 9 



I w x vg+cg 

w x WyVg- ui z sg 

Screw transformation to homogeneous transformation 

J R , = W x WyVQ + W Z SQ 

wfvg +cg 


\w x VJ z V0 — WySQ 

WyW z vg + w x sg 


J P; = (13X3 - J Ri)P + hd 

w 


T 

/ f32 ~ P23\ 


Homogeneous transformation to screw transformation 

l = 1 '13-^31 



c 

1 



ui x w z vg + WySg 
WyW z vg-w x sg 
w z v e + eg 


9 = sign (l T ipi) cos 


-1 fill 


+ f "22 + ^33 — 1 N 


h = 

" ~ 20 sin 0 

_(l3x3 —' R jyPi 

P ~ 2(1—cos 0) 


W = 


l 

2 sin 0 


where the abbreviations are eg := cos 9, sg := sin$, 
and vg = 1 — cos 9. The rotation matrix 7 R, expressed 
in this form is also called the screw matrix, and these 
equations give the elements of 'R, and J p, in terms of 
the screw parameters. 

An exception arises in the case of a pure translation, 
for which 9=0 and Rodrigues’ equation becomes 

i r = i r + dw. (2.22) 

Substituting for this case,'R; = I 3 X 3 and J p; = dw. 

Additional information on screw theory can be 
found in [2.11-15], 

2.2.5 Matrix Exponential Parameterization 

The position and orientation of a body can also be 
expressed in a unified fashion with an exponential 
mapping. This approach is introduced first with its ap¬ 
plication to pure rotation and expanded to rigid-body 
motion. More details on the approach can be found 
in [2.16] and [2.17]. 

Exponential Coordinates for Rotation 
The set of all orthogonal matrices with determinant 1, 
which is the set of all rotation matrices R, is a group 
under the operation of matrix multiplication denoted 
as SO( 3) C R 3x3 [2.18]. This stands for special or¬ 
thogonal wherein special alludes to the det R being + 1 
instead of ±1. The set of rotation matrices satisfies the 
four axioms of a group: 

• Closure: RiR 2 eSO(3) VRi,R 2 e50(3) 

• Identity: I 3 X 3 R = RI 3 X 3 = R VR e 50(3) 

• Inverse: R T e 50(3) is the unique inverse of R 
VR e 50(3) 


• Associativity: (RiR 2 )R 3 = Ri(R 2 R 3 ) VRi, R 2 , 
R 3 e 50(3). 

In the angle-axis representation presented in 
Sect. 2.2.2, orientation is expressed as an angle 9 of 
rotation about an axis defined by the unit vector w. The 
equivalent rotation matrix found in Table 2.1 can be ex¬ 
pressed as the exponential map 

R = e s{{b)9 

o 2 

= l 3 x3 + #S(m) + ^yS(U)) 2 
Q 3 

+ — S(w ) 3 + ... , (2.23) 

where S(u>) is the unit skew-symmetric matrix 

( 0 — W Z Wy \ 

w z 0 — w x I . (2.24) 

-uiy w x 0 / 

Thus, the exponential map transforms a skew-sym¬ 
metric matrix S(w) that corresponds to an axis of 
rotation w into an orthogonal matrix R that corre¬ 
sponds to a rotation about the axis w through an angle 
of 9. It can be shown that the closed-form expression 
for e s(ul)0 , which can be efficiently computed, is 

e s(iu) 6 ‘ _ _|_ S({y) si n 0 - 1 - S(tb) 2 (l — cos 6 ). 

(2.25) 

The components of (9w x 9wy 9w z ) T , which are related 
to the elements of the rotation matrix R in Table 2.2, are 
referred to as the exponential coordinates for R. 
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Exponential Coordinates 
for Rigid-Body Motion 

As indicated in Sect. 2.2.3, the position and orientation 
of a body can be expressed by the combination of a po¬ 
sition vector p e R 3 and a rotation matrix R e SO(3). 
The product space of K 3 with SO( 3) is the group known 
as SE( 3), which stands for special Euclidean. 

SE( 3) = {(p,R):peR\ReSO(3)} 

= M 3 x 50(3). 

The set of homogeneous transformations satisfies the 
four axioms of a group: 


• Closure: T,T 2 e5£(3) VT U T 2 e SE(3) 

• Identity. I 4 X 4 T = T 1 4X 4 = T VTe SE( 3) 

• Inverse: the unique inverse of T VT e SE( 3) is 
given in ( 2 . 12 ) 

• Associativity: (TiT2)T3 = Ti(T2T3) VTi,T2, 
T 3 e SE( 3). 

In the screw transformation representation in 
Sect. 2.2.4, position and orientation are expressed by 
the angle 9 of rotation about a screw axis defined by the 
unit vector w , the point p on the axis such that w r p = 
0, and the pitch h of the screw axis. The equivalent ho¬ 
mogeneous transformation found in Table 2.4 can be 
expressed as the exponential map 


^ ie * , , (i^) 2 , (i 0 ) 3 , 

T = e ? = l 4 x 4 + 50 4-—-1-—-1- . 


2 ! 


3! 


(2.26) 


where 


§ 


(S(w) 

V o T 


:) 


(2.27) 


is the generalization of the unit skew-symmetric ma¬ 
trix S (w) known as a twist. The twist coordinates of i- 
are given by £ := (■ut T t; T ) T . It can be shown that the 
closed-form expression for is 


e* e = 

/ e s (w)0 (i 3x3 — e s ^ e )(w x»)f w T v9w\ 

l « T 1 )' 

(2.28) 

Comparison of this result with the conversion between 
homogeneous and screw transformations in Table 2.4 
yields 

v = pxw (2.29) 

and 

(2.30) 


Thus, the exponential map for a twist transforms the ini¬ 
tial pose of a body into its final pose. It gives the relative 
rigid-body motion. The vector £ 6 contains the exponen¬ 
tial coordinates for the rigid-body transformation. 

As for screw transformations, the case of pure trans¬ 
lation is unique. In this case, w = 0, so 



2.2.6 Pliicker Coordinates 

A minimum of four coordinates are needed to de¬ 
fine a line in space. The Pliicker coordinates of a line 
form a six-dimensional ( 6 -D) vector, so they are su¬ 
perabundant by two. They can be viewed as a pair of 
three-dimensional (3-D) vectors; one is parallel to the 
line, and the other is the moment of that vector about 
the origin. Thus, if u is any vector parallel to the line 
and p is the position of any point on the line relative to 
the origin, the Pliicker coordinates (L, M. N, P. Q, and 
R ) are given by 

(L,M,N) = h t ; (P.Q.R) = (pxu) T . (2.32) 

For simply defining a line, the magnitude of u is not 
unique, nor is the component of p parallel to u. Two 
auxiliary relationships are imposed to reduce the set to 
just four independent coordinates. One is that the scalar 
product of the two three-dimensional vectors is identi¬ 
cally zero. 

LP + MQ + NR = 0. (2.33) 

The other is the invariance of the line designated when 
the coordinates are all multiplied by the same scaling 
factor. 


(L. M, N, P. Q. R) = (kL. kM. kN, kP , kQ , kR) . 

(2.34) 

This relationship may take the form of constraining u 
to have unit magnitude so that L, M, and N are the di¬ 
rection cosines. 

In this handbook, it is often useful to express 
velocities in Pliicker coordinates, wherein unlike the 
definition of lines, the magnitudes of the two three- 
dimensional vectors are not arbitrary. This leads to 
the motor notation of von Mises [2.9, 19] and Ev¬ 
erett [2.20]. For instantaneously coincident coordinate 
frames, one fixed and the other embedded in the mov¬ 
ing body, a) is the angular velocity of the body and v 0 
is the velocity of the origin O of the body-fixed frame 
when both are expressed relative to the fixed frame. 
This provides a Pliicker coordinate system for the spa¬ 
tial velocity v of the body. The Pliicker coordinates of v 


h = w T v . 
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are simply the Cartesian coordinates of u> and v 0 , such that 


v = 



(2.35) 


The transformation from Pliicker coordinate sys¬ 
tem i to Pliicker coordinate system j for spatial veloc¬ 
ities is achieved with the spatial transform J X,. If v, 
and \j denote the spatial velocities of a body relative 
to the i and j frames, respectively, and ■' Pi and J R, de¬ 
note the position and orientation of frame i relative to 
frame j. 


X = ; X,v, 


where 


J X: = 


R, 

s( j Pi yRi 


03x3^ 

'R, ) 


(2.36) 


(2.37) 



and 


'X; 4 X/X;. (2.39) 

and S (ipi) is the skew-symmetric matrix 
/ 0 -Jpil \ 

j p z 0 -Jp? . (2.40) 

\ w j P- 0 ) 

Spatial vector notation, which includes the spatial 
velocities and transforms briefly mentioned here, is 
treated in greater depth in Sect. 3.2. Specifically, Ta¬ 
ble 3.1 gives a computationally efficient algorithm for 
applying a spatial transform. 


2.3 Joint Kinematics 

Unless explicitly stated otherwise, the kinematic de¬ 
scription of robotic mechanisms typically employs 
a number of idealizations. The links that compose the 
robotic mechanism are assumed to be perfectly rigid 
bodies having surfaces that are geometrically perfect in 
both position and shape. Accordingly, these rigid bod¬ 
ies are connected together at joints where their idealized 
surfaces are in ideal contact without any clearance be¬ 
tween them. The respective geometries of these surfaces 
in contact determine the freedom of motion between the 
two links, or the joint kinematics. 

A kinematic joint is a connection between two bod¬ 
ies that constrains their relative motion. Two bodies that 
are in contact with one another create a simple kine¬ 
matic joint. The surfaces of the two bodies that are 
in contact are able to move over one another, thereby 
permitting relative motion of the two bodies. Simple 
kinematic joints are classified as lower pair joints if con¬ 
tact occurs over surfaces [2.21] and as higher pair joints 
if contact occurs only at points or along lines. 

A joint model describes the motion of a frame fixed 
in one body of a joint relative to a frame fixed in the 
other body. The motion is expressed as a function of the 
joint’s motion variables, and other elements of a joint 
model include the rotation matrix, position vector, free 
modes, and constrained modes. The free modes of 
a joint define the directions in which motion is allowed. 
They are represented by the 6 x n, matrix <J>, whose 
columns are the Pliicker coordinates of the allowable 


motion. This matrix relates the spatial velocity vector 
across the joint t; re y to the joint velocity vector (];, 

Wrel.i = $i<li • (2.41) 

In contrast, the constrained modes of a joint define 
the directions in which motion is not allowed. They 
are represented by the 6 x (6 — nj) matrix 4>j that 
is complementary to Tables 2.5 and 2.6 contain 
the formulas of the joint models for all of the joints 
described in this section. They are used extensively for 
the dynamic analysis presented in Chap. 3. Additional 
information on joints can be found in Chap. 4. 

2.3.1 Lower Pair Joints 

Lower pair joints are mechanically attractive since wear 
is spread over the whole surface and lubricant is trapped 
in the small clearance space (in nonidealized systems) 
between the surfaces, resulting in relatively good lubri¬ 
cation. As can be proved [2.23] from the requirement 
for surface contact, there are only six possible forms of 
lower pair joints: revolute, prismatic, helical, cylindri¬ 
cal, spherical, and planar joints. 

Revolute 

The most general form of a revolute joint, often abbre¬ 
viated as R and sometimes referred to colloquially as 
a hinge or pin joint, is a lower pair composed of two 


Part A | 2.3 



Part A | 2.3 


22 Part A 


Robotics Foundations 


Table 2.5 Joint model formulas for one-degree-of-freedom lower pair joints, with abbreviations ce i := cost?, and s$ j := sin#,- 
(adapted in part from Table 4.1 in [2.22]) 


Joint type Joint rotation matrix J R; Position vector '/), Free modes <P; Constrained modes 4>‘ Pose state vars. f 


Revolute R 


Helical H 
(pitch h ) 


-sg, 0 

C8t 0 

0 1 


Prismatic P I 3 X 3 


-sg, 0 

C0i 0 

0 1 


/o\ 

0 

1 

0 

0 

Vo/ 

/0\ 

0 

0 

0 

0 

Vi/ 

/o\ 

0 

1 

0 

0 

w 


/I 

0 10 0 
0 0 0 0 


0 0 0 0 \ 
0 


0 0 
0 0 


1 

0 1 
Vo 0 0 0 

/I 

0 1 
0 0 


0 

0 0 


0 

1 / 

0 0 0 0 \ 
0 0 0 


1 

0 0 0 1 
0 0 0 0 
Vo 0 0 0 

(\ 0 0 0 
0 10 0 
0 0 0 0 


0 0 
0 
1 

0 / 


0 0 
0 0 


Vo 0 0 0 


0 \ 

0 

—h 

0 

0 

1 / 


di 


congruent surfaces of revolution. The surfaces are the 
same except one of them is an external surface, convex 
in any plane normal to the axis of revolution, and one 
is an internal surface, concave in any plane normal to 
the axis. The surfaces may not be solely in the form of 
right circular cylinders, since surfaces of that form do 
not provide any constraint on axial sliding. A revolute 
joint permits only rotation of one of the bodies joined 
relative to the other. The position of one body relative 
to the other may be expressed as the angle between two 
lines normal to the joint axis, one fixed in each body. 
Thus, the joint has one degree of freedom (DOF). When 
the z axis of coordinate frame i is aligned with a revolute 
joint axis, the formulas in Table 2.5 define the revolute 
joint model. 

Prismatic 

The most general form of a prismatic joint, often ab¬ 
breviated as P and sometimes referred to colloquially 
as a sliding joint, is a lower pair formed from two con¬ 
gruent general cylindrical surfaces. These may not be 
right circular cylindrical surfaces. A general cylindrical 
surface is obtained by extruding any curve in a constant 
direction. Again, one surface is internal and the other is 
an external surface. A prismatic joint permits only slid¬ 
ing of one of the members joined relative to the other 
along the direction of extrusion. The position of one 
body relative to the other is determined by the distance 
between two points on a line parallel to the direction 
of sliding, with one point fixed in each body. Thus, this 
joint also has one degree of freedom. When the z axis 


of coordinate frame i is aligned with a prismatic joint 
axis, the formulas in Table 2.5 define the prismatic joint 
model. 

Helical 

The most general form of a helical joint, often abbre¬ 
viated as H and sometimes referred to colloquially as 
a screw joint, is a lower pair formed from two helicoidal 
surfaces formed by extruding any curve along a helical 
path. The simple example is a bolt and nut wherein the 
basic generating curve is a pair of straight lines. The 
angle 9 of rotation about the axis of the helical joint is 
directly related to the distance d of displacement of one 
body relative to the other along that axis by the expres¬ 
sion d = h9, where the constant h is called the pitch of 
the helical joint. When the z axis of coordinate frame 1 
is aligned with a helical joint axis, the formulas in Ta¬ 
ble 2.5 define the helical joint model. 

Cylindrical 

A cylindrical joint, often abbreviated as C, is a lower 
pair formed by contact of two congruent right circu¬ 
lar cylinders, one an internal surface and the other 
an external surface. It permits both rotation about the 
cylinder axis and sliding parallel to it. Therefore, it is 
a joint with two degrees of freedom. Lower pair joints 
with more than one degree of freedom are easily re¬ 
placed by kinematically equivalent compound joints 
(Sect. 2.3.3) that are serial chains of one-degree-of- 
freedom lower pairs. In the present case, the cylindrical 
joint can be replaced by a revolute in series with a pris- 
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Table 2.6 Joint model formulas for higher-degree-of-freedom lower pair joints, universal joint, rolling contact joint, and 6 -DOF 
joint, with abbreviations cg j := cos (9; and sg i := sin 9i (adapted in part from Table 4.1 in [2.22]) *The Euler angles a,-, /?,, and y,- 
could be used in place of the unit quaternion to represent orientation 


Joint type 

Joint rotation matrix 7 /?, 

Position vector 7 /7, 

Free modes <1>, 

Constrained modes $ j 

Pose 

Velocity 






Variables 

Variables 

ki 


Cylindrical 

C 


Spherical* 

S 


Planar 


Flat planar 
rolling 
contact 
(fixed 
radius r) 


Univer¬ 
sal U 


6-DOF* 


Cdi 

s 8 i 

0 


-s Si O' 
0 

0 1 


Table 2.1 


eg ,■ 

S0i 

0 


-s ei O' 

effi 0 

0 1 


c oti c fit s a.i c oci s fit 

SoiiCPi Coa s di s fii 

~ S 0i 0 C Pi 


see 
Table 2.1 


cg t d X i SQjdy, 

sgjdxi + cg t dy 
0 


( c 8i 

-sg. 

°\ 

/ r9 i cg i - 

rS8i \ 

s 8i 

c 6i 

0 

~e0iSg i - 

- reg, 

Vo 

0 

1/ 

V 0 

) 


/0 0 \ 

0 0 
1 0 
0 0 
0 0 
VO 1/ 

/1 0 0 \ 
0 1 0 
0 0 1 
0 0 0 
0 0 0 
\o 0 0 / 

/o 0 o\ 
0 0 0 
1 0 0 
0 1 0 
0 0 1 
\0 0 0 / 

/ 0 \ 

0 
1 
r 
0 

VO/ 


t-SPi 
0 
ept 

0 
0 
V 0 

16 x6 


0 \ 

1 

0 

0 

0 

0 / 


(\ 0 0 o\ 
0 10 0 
0 0 0 0 
0 0 10 
0 0 0 1 
\0 0 0 0 / 

/o 0 o\ 

0 0 0 
0 0 0 
1 0 0 
0 1 0 
Vo 0 1 } 

( 1 0 0 \ 

0 1 0 
0 0 0 
0 0 0 
0 0 0 
VO 0 1, 

/l 0 c 
0 1 c 
0 0- 
0 0 1 
0 0 c 
\o 0 c 


( c Pi 

0 

S P> 

0 

0 

V0 


0 0 \ 
0 0 
0 0 
0 0 
1 0 
0 1 


0 0 0 \ 
0 0 0 
0 0 0 
1 0 0 


0 1 
0 0 




dxi 


e, 


6) i rel 


matic joint whose direction of sliding is parallel to the 
revolute axis. While simpler to implement using the 
geometric representation discussed in Sect. 2.4, this 
approach has disadvantages for dynamic simulation. 
Modeling a single cylindrical joint as a combination 
of a prismatic and a revolute joint requires the ad¬ 
dition of a virtual link between the two with zero 
mass and zero length. The massless link can create 
computational problems. When the z axis of coordi¬ 
nate frame i is aligned with a cylindrical joint axis, 
the formulas in Table 2.6 define the cylindrical joint 
model. 


Spherical 

A spherical joint, often abbreviated as S, is a lower pair 
formed by contact of two congruent spherical surfaces. 
Once again, one is an internal surface, and the other is 
an external surface. A spherical joint permits rotation 
about any line through the center of the sphere. Thus, it 
permits independent rotation about axes in up to three 
different directions and has three degrees of freedom. 
A spherical joint is easily replaced by a kinematically 
equivalent compound joint consisting of three revolutes 
that have axes that all intersect in a single point - the 
center of the spherical joint. The revolute joint axes 


Part A | 2.3 






































Part A | 2.3 


24 Part A I Robotics Foundations 


do not need to be successively orthogonal, but often 
they are implemented that way. The arrangement is, in 
general, kinematically equivalent to a spherical joint, 
but it does exhibit a singularity when the revolute joint 
axes become coplanar. This is as compared to the na¬ 
tive spherical joint that never has such a singularity. 
Likewise, if a spherical joint is modeled in simulation 
as three revolutes, computational difficulties again can 
arise from the necessary inclusion of massless virtual 
links having zero length. The joint model formulas of 
a spherical joint are given in Table 2.6. 

Planar 

A planar joint is formed by planar contacting surfaces. 
Like the spherical joint, it is a lower pair joint with 
three degrees of freedom. A kinematically equivalent 
compound joint consisting of a serial chain of three 
revolutes with parallel axes can replace a planar joint. 
As was the case with the spherical joint, the compound 
joint exhibits a singularity when the revolute axes be¬ 
come coplanar. When the z axis of coordinate frame i 
is aligned with the normal to the plane of contact, the 
formulas in Table 2.6 define the planar joint model. 

2.3.2 Higher Pair Joints 

Some higher pair joints also have attractive proper¬ 
ties, particularly rolling pairs in which one body rolls 
without slipping over the surface of the other. This 
is mechanically attractive since the absence of sliding 
means the absence of abrasive wear. However, since 
ideal contact occurs at a point, or along a line, appli¬ 
cation of a load across the joint may lead to very high 
local stresses resulting in other forms of material fail¬ 
ure and, hence, wear. Higher pair joints can be used to 
create kinematic joints with special geometric proper¬ 
ties, as in the case of a gear pair or a cam and follower 
pair. 

Rolling Contact 

Rolling contact actually encompasses several different 
geometries. Rolling contact in planar motion permits 
one degree of freedom of relative motion as in the case 
of a roller bearing, for example. Planar rolling contact 
can take place along a line, thereby spreading the load 
and wear somewhat. Three-dimensional rolling con¬ 
tact allows rotation about any axis through the point 
of contact that is, in principle, unique. Hence, a three- 
dimensional rolling contact pair permits relative motion 
with three degrees of freedom. When the z axis of co¬ 
ordinate frame i is aligned with the axis of rotation and 
passes through the center of the roller of fixed radius r, 
the formulas in Table 2.6 define the planar rolling con¬ 
tact joint model for a roller on a flat surface. 


Regardless of whether the joint is planar or three- 
dimensional, the no-slip condition associated with 
a rolling contact joint requires that the instantaneous 
relative velocity between the points on the two bodies 
in contact be zero. If P is the point of rolling contact 
between bodies i and j, 

v Pi/Pj = 0 • (2.42) 

Likewise, relative acceleration is in the direction of the 
common normal to the two surfaces at the point of con¬ 
tact. Because the constraint associated with the joint is 
expressed in terms of velocity and cannot be expressed 
in terms of position alone, it is nonholonomic, as dis¬ 
cussed in Sect. 2.3.6. A more detailed discussion of 
the kinematic constraints for rolling contact is found 
in Chap. 24. 

2.3.3 Compound Joints 

Compound kinematic joints are connections between 
two bodies formed by chains of other members and sim¬ 
ple kinematic joints. A compound joint may constrain 
the relative motion of the two bodies joined in the same 
way as a simple joint. In such a case, the two joints are 
said to be kinematically equivalent. 

Universal 

A universal joint, often abbreviated as U and referred 
to as a Cardan or Hooke joint, is a compound joint 
with two degrees of freedom. It consists of a serial 
chain of two revolutes whose axes intersect orthogo¬ 
nally. The joint model for a universal joint, in which, 
from Euler angle notation, 04 is the first rotation about 
the Z-axis and then /l, is the rotation about the 7-axis, 
is given in Table 2.6. This is a joint for which the matri¬ 
ces <b, and <t> j are not constant, so in general, 0 , 0 

• C 

and <1>, / 0. As seen in Table 2.6, the orientation of 
the first joint axis (expressed in the outboard coordinate 
frame) varies with /J;. 

2.3.4 6-DOF Joint 

The motion of two bodies not jointed together can 
be modeled as a six-degree-of-freedom joint that in¬ 
troduces no constraints. This is particularly useful 
for mobile robots, such as aircraft, that make at 
most intermittent contact with the ground, and thus, 
a body in free motion relative to the fixed frame 
is termed a floating base. Such a free motion joint 
model enables the position and orientation of a float¬ 
ing base in space to be expressed with six joint 
variables. The 6 -DOF joint model is included in Ta¬ 
ble 2 . 6 . 
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2.3.5 Physical Realization 

In an actual robotic mechanism, the joints may have 
physical limits beyond which motion is prohibited. 
The workspace (Sect. 2.5) of a robotic manipulator 
is determined by considering the combined limits and 
freedom of motion of all the joints within the mech¬ 
anism. Revolute joints are easily actuated by rotating 
motors and are, therefore, extremely common in robotic 
systems. They may also be present as passive, un¬ 
actuated joints. Also common, although less so than 
revolutes, prismatic joints are relatively easily actu¬ 
ated by means of linear actuators such as hydraulic or 
pneumatic cylinders, ball screws, or screw jacks. They 
always have motion limits since unidirectional slid¬ 
ing can, in principle, produce infinite displacements. 
Helical joints tire most often found in robotic mecha¬ 
nisms as constituents of linear actuators such as screw 
jacks and ball screws and are seldom used as primary 
kinematic joints. Joints with more than one degree of 
freedom are generally used passively in robotic mecha¬ 
nisms because each degree of freedom of an active joint 
must be separately actuated. Passive spherical joints 
are quite often found in robotic mechanisms, while 
passive planar joints are only occasionally found. The 
effect of an actuated spherical joint is achieved by em¬ 
ploying the kinematically equivalent combination of 
three revolutes and actuating each. Universal joints are 
used in robotic mechanisms in both active and passive 
forms. 

Serial chains are commonly denoted by the abbre¬ 
viations for the joints they contain in the order in which 
they appear in the chain. For example, an RPR chain 
contains three links, the first jointed to the base with 
a revolute and to the second with a prismatic, while 
the second and third are jointed together with another 
revolute. If all of the joints are identical, the notation 
consists of the number of joints preceding the joint 
abbreviation, such as 6R for a six-axis serial-chain ma¬ 
nipulator containing only revolute joints. 

Joints are realized with hardware that is more com¬ 
plex than the idealizations presented in Sects. 2.3.1 
and 2.3.2. For example, a revolute joint may be 
achieved with a ball bearing composed of a set of 

2.4 Geometric Representation 

The geometry of a robotic mechanism is conveniently 
defined by attaching coordinate frames to each link. 
While these frames could be located arbitrarily, it is ad¬ 
vantageous both for consistency and computational effi¬ 
ciency to adhere to a convention for locating the frames 
on the links. Denavit and Hartenberg [2.26] introduced 


bearing balls trapped between two journals. The balls 
ideally roll without slipping on the journals, thereby 
taking advantage of the special properties of rolling 
contact joints. A prismatic joint may be realized by 
means of a roller-rail assembly. 

2.3.6 Holonomic 

and Nonholonomic Constraints 

With the exception of rolling contact, all of the con¬ 
straints associated with the joints discussed in the 
preceding sections can be expressed mathematically by 
equations containing only the joint position variables. 
These are called holonomic constraints. The number of 
equations, and hence the number of constraints, is 6 — 11 , 
where n is the number of degrees of freedom of the 
joint. The constraints are intrinsically part of the axial 
joint model. 

A nonholonomic constraint is one that cannot be 
expressed in terms of the position variables alone, but 
includes the time derivative of one or more of those 
variables. These constraint equations cannot be inte¬ 
grated to obtain relationships solely between the joint 
variables. The most common example in robotic sys¬ 
tems arises from the use of a wheel or roller that 
rolls without slipping on another member. Nonholo¬ 
nomic constraints, particularly as they apply to wheeled 
robots, are discussed in more detail in Chap. 24. 

2.3.7 Generalized Coordinates 

In a robotic mechanism consisting of N bodies, 6 N 
coordinates are required to specify the position and ori¬ 
entation of all the bodies relative to a coordinate frame. 
Since some of those bodies are jointed together, a num¬ 
ber of constraint equations will establish relationships 
among some of these coordinates. In this case, the 6 N 
coordinates can be expressed as functions of a smaller 
set of coordinates q that are all independent. The coor¬ 
dinates in this set are known as generalized coordinates, 
and motions associated with these coordinates are con¬ 
sistent with all of the constraints. The joint variables q 
of a robotic mechanism form a set of generalized coor¬ 
dinates [2.24, 25]. 


the foundational convention that has been adapted in 
a number of different ways, one of which is the con¬ 
vention introduced by Khalil and Dombre [2.27] used 
throughout this handbook. In all of its forms, the con¬ 
vention requires only four rather than six parameters 
to locate one coordinate frame relative to another. The 
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four parameters consist of two link parameters, the link 
length cii and the link twist a,-, and two joint parameters, 
the joint offset d, and the joint angle This parsimony 
is achieved through judicious placement of the coordi¬ 
nate frame origins and axes such that the x axis of one 
frame both intersects and is perpendicular to the z axis of 
the following coordinate frame. The convention is appli¬ 
cable to robotic mechanisms consisting of revolute and 
prismatic joints, so when multiple-degree-of-freedom 
joints are present, they are modeled as combinations of 
revolute and prismatic joints, as discussed in Sect. 2.3. 

There are essentially four different forms of the 
convention for locating coordinate frames in a robotic 
mechanism. Each exhibits its own advantages by man¬ 
aging trade-offs of intuitive presentation. In the original 
Denavit and Hartenberg [2.26] convention, joint i is lo¬ 
cated between links i and 2 + 1, so it is on the outboard 
side of link i. Also, the joint offset dj and joint angle 0,- 
are measured along and about the i — 1 joint axis, so 
the subscripts of the joint parameters do not match that 
of the joint axis. Waldron [2.28] and Paul [2.29] modi¬ 
fied the labeling of axes in the original convention such 
that joint i is located between links i— 1 and i in order 
to make it consistent with the base member of a se¬ 
rial chain being member 0. This places joint i at the 
inboard side of link 2 and is the convention used in all 
of the other modified versions. Furthermore, Waldron 
and Paul addressed the mismatch between subscripts 
of the joint parameters and joint axes by placing the z,- 



Fig. 2.2 Schematic of the numbering of bodies and joints 
in a robotic mechanism, the convention for attaching coor¬ 
dinate frames to the bodies, and the definitions of the four 
parameters, a;, a;, d,-, and 0,, that locate one frame relative 
to another 


axis along the i + 1 joint axis. This, of course, relocates 
the subscript mismatch to the correspondence between 
the joint axis and the z axis of the coordinate frame. 
Craig [2.30] eliminated all of the subscript mismatches 
by placing the z,- axis along joint 2 , but at the ex¬ 
pense of the homogeneous transformation ' 1 T,- being 
formed with a mixture of joint parameters with sub¬ 
script i and link parameters with subscript i — 1. Khalil 
and Dombre [2.27] introduced another variation similar 
to Craig’s except that it defines the link parameters a j 
and a,- along and about the i;_i axis. In this case, the 
homogeneous transformation *T, is formed only by 
parameters with subscript i, and the subscript mismatch 
is such that a, and a, indicate the length and twist of link 
i— 1 rather than link 2 . Thus, in summary, the advantages 
of the convention used throughout this handbook com¬ 
pared to the alternative conventions are that the z axes 
of the coordinate frames share the common subscript of 
the joint axes and the four parameters that define the 
spatial transform from coordinate frame i to coordinate 
frame 2 — 1 all share the common subscript 2 . 

In this handbook, the convention for serial chain 
mechanisms is shown in Fig. 2.2 and summarized as 
follows. The numbering of bodies and joints follows the 
convention: 

• The N moving bodies of the robotic mechanism are 
numbered from 1 to N. The number of the base is 0. 

• The N joints of the robotic mechanism are num¬ 
bered from 1 to N , with joint 2 located between 
members 2 — 1 and 2 . 



Fig. 2.3 Example six-degree-of-freedom serial chain ma¬ 
nipulator composed of an articulated arm with no joint 
offsets and a spherical wrist 
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Table 2.7 Geometric parameters of the example serial 
chain manipulator in Fig. 2.3 
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With this numbering scheme, the attachment of co¬ 
ordinate frames follows the convention: 

• The Zi axis is located along the axis of joint i, 

• The i;_i axis is located along the common normal 

between the z,-_ 1 and z,- axes. 

Using the attached frames, the four parameters that 
locate one frame relative to another are defined as: 

• a, is the distance from z,—i to z,- along x,_ 1 , 

• ofj is the angle from z,_i to Zi about Jc,_i, 

• di is the distance from x,—\ to i,- along 

• 6 t is the angle from £,_i to x, about 

The geometric parameters for the example manip¬ 
ulator shown in Fig. 2.3 are listed in Table 2.7. All of 
the joints of this manipulator are revolutes, and joint 1 


2.5 Workspace 

Most generally, the workspace of a robotic manipula¬ 
tor is the total volume swept out by the end-effector 
as the manipulator executes all possible motions. The 
workspace is determined by the geometry of the ma¬ 
nipulator and the limits of the joint motions. It is 
more specific to define the reachable workspace as 
the total locus of points at which the end-effector 
can be placed and the dextrous workspace [2.31] as 
the subset of those points at which the end-effector 
can be placed while having an arbitrary orientation. 
Dexterous workspaces exist only for certain ideal¬ 
ized geometries, so real industrial manipulators with 
joint motion limits almost never possess dexterous 
workspaces. 

Many serial-chain robotic manipulators are de¬ 
signed such that their joints can be partitioned into 
a regional structure and an orientation structure. The 
joints in the regional structure accomplish the posi¬ 
tioning of the end-effector in space, and the joints in 
the orientation structure accomplish the orientation of 
the end-effector. Typically, the inboard joints of a se¬ 


has a vertical orientation. Joint 2 is perpendicular to 
joint 1 and intersects it. Joint 3 is parallel to joint 2, 
and the length of link 2 is <73. Joint 4 is perpendicular to 
joint 3 and intersects it. Joint 5 likewise intersects joint 
4 perpendicularly at an offset of d 4 from joint 3. Finally, 
joint 6 intersects joint 5 perpendicularly. 

With this convention, coordinate frame i can be 
located relative to coordinate frame i— 1 by execut¬ 
ing a rotation through an angle a, about the x l —\ 
axis, a translation of distance a, along x t —\, a rotation 
through an angle 0 , about the z, axis, and a translation 
of distance d, along z,. Through concatenation of these 
individual transformations, 

Rot(i,_i, a,jTrans(x,_i, a,)Rot(z,, 0,) 

Transfz,■,£/,), (2.43) 

the equivalent homogeneous transformation is, 

,-t T . = 

( cos 6 j — sin 9j 0 ^ 

sin di cos a,- cos 0 ; cos a; —sin ct; — sinc^r/; 
sin dj sin cos d ; sin a, cos a ,■ cos ct/dj 

v 0 0 0 1 / 

(2.44) 

The identification of geometric parameters is addressed 
in Chap. 8 . 


rial chain manipulator comprise the regional structure, 
while the outboard joints comprise the orientation struc¬ 
ture. Also, since prismatic joints provide no capability 
for rotation, they are generally not employed within the 
orientation structure. 

The regional workspace volume can be calculated 
from the known geometry of the serial-chain manipula¬ 
tor and motion limits of the joints. With three inboard 
joints comprising the regional structure, the area of 
workspace for the outer two (joints 2 and 3) is computed 
first, and then the volume is calculated by integrating 
over the joint variable of the remaining inboard joint 
(joint 1). In the case of a prismatic joint, this simply in¬ 
volves multiplying the area by the total length of travel 
of the prismatic joint. In the more common case of 
a revolute joint, it involves rotating the area about the 
joint axis through the full range of motion of the rev¬ 
olute [2.32]. By the theorem of Pappus, the associated 
volume V is 

V = Ary , 


(2.45) 
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where A is the area, r is the distance from the area’s 
centroid to the axis, and y is the angle through which 
the area is rotated. The boundaries of the area are de¬ 
termined by tracing the motion of a reference point in 
the end-effector, typically the center of rotation of the 
wrist that serves as the orientation structure. Starting 
with each of the two joints at motion limits and with 
joint 2 locked, joint 3 is moved until its second mo¬ 
tion limit is reached. Joint 3 is then locked, and joint 2 

2.6 Forward Kinematics 

The forward kinematics problem for a serial-chain ma¬ 
nipulator is to find the position and orientation of the 
end-effector relative to the base given the positions of 
all of the joints and the values of all of the geometric 
link parameters. Often, a frame fixed in the end-effector 
is referred to as the tool frame, and while fixed in the fi¬ 
nal link N, it in general has a constant offset in both 
position and orientation from frame N. Likewise, a sta¬ 
tion frame is often located in the base to establish the 
location of the task to be performed. This frame gener¬ 
ally has a constant offset in its pose relative to frame 0, 
which is also fixed in the base. 

A more general expression of the forward kine¬ 
matics problem is to find the relative position and 
orientation of any two designated members given the 
geometric structure of the robotic mechanism and 
the values of a number of joint positions equal to 
the number of degrees of freedom of the mecha¬ 
nism. The forward kinematics problem is critical for 
developing manipulator coordination algorithms be¬ 
cause joint positions are typically measured by sensors 
mounted on the joints and it is necessary to calcu¬ 
late the positions of the joint axes relative to the fixed 
frame. 

In practice, the forward kinematics problem is 
solved by calculating the transformation between a co¬ 
ordinate frame fixed in the end-effector and another 
coordinate frame fixed in the base, i. e., between the tool 
and station frames. This is straightforward for a serial 
chain since the transformation describing the position 
of the end-effector relative to the base is obtained by 
simply concatenating transformations between frames 
fixed in adjacent links of the chain. The convention 
for the geometric representation of a manipulator pre¬ 
sented in Sect. 2.4 reduces this to finding an equivalent 
4x4 homogeneous transformation matrix that relates 
the spatial displacement of the end-effector coordinate 
frame to the base frame. 

For the example serial-chain manipulator shown in 
Fig. 2.3 and neglecting the addition of tool and station 


is freed to move to its second motion limit. Joint 2 is 
again locked, while joint 3 is freed to move back to 
its original motion limit. Finally, joint 3 is locked, and 
joint 2 freed to move likewise to its original motion 
limit. In this way, the trace of the reference point is 
a closed curve whose area and centroid can be calcu¬ 
lated mathematically. 

More details on manipulator workspace can be 
found in Chaps. 4 and 16. 


Table 2.8 Forward kinematics of the example serial chain 
manipulator in Fig. 2.3, with abbreviations c$ ; := cos 0, 
and SQ j := sin 6, 
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v« 
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Ml (c$ 2 C$3 C$2 C$3 )(c$ 4 C$4 C$ 4 C$ 5 C$ 6 ) 

C$ f C$ 5 C$ fi (C$2 C$ 3 T C$ 2 C$3 ) 

+ C$2 (c$ 4 C$, C$4 T C$4 C$5 ) , 

Ml C$2(C$ 2 C$ 3 C$2 C$4 )(c$ 4 C$4 C$ 4 C$ g C$g) 

C$2 C$5 C$4 (c$2 C$4 T C$ 2 C$3 ) 

-c$, (c$ 4 c$ 5 c$ 6 + c$ 4 c$ 6 ) , 

Ml (c$2C$4 F C$2C$4 )(.S$ 4 .S$4 C$ 4 C$gC$ 6 ) 

F C$4 C$4 f.V$2 C$3 C$2 C$3 ) , 

M 2 = c$, (c$ 2 c$ 3 — C$ 2 c$ 3 )(c$ 4 c$ 5 s$ 6 + c$ 4 c$ 6 ) 

"F C$2 C$3 C$4 (C$2C$3 “t - C$2 C$3 ) 

+ c$, (c$ 4 c$ 6 - c$ 4 c$ 5 c$ 6 ) , 

M 2 C$2 (C$ 2 C$3 C$2 C$4 ) (c$ 4 C$4 C$4 “f-C$ 4 C$ 6 ) 

"F C$2 C$5 C$4 (c$2 C$3 T C$2 C$ 3 ) 

-C$, (C$ 4 C$4 C$ 4 C$ 5 C$4 ) , 

M2 (c$2C$3 + C$2C$3 )(c$ 4 C$5 C$ 6 f C$ 4 C$ 6 ) 

C$5 C$4 (c$2 C$3 C$2 C$3) , 

M 3 C$2 C$ 4 C$5 (c$ 2 C$ 3 C$ 2 C$ 3 ) 

C$2 C$5 (C$2 C$3 T C$2 C$3 ) 

C$2C$ 4 C$5 , 

M 3 C$2 C$ 4 C$5 (c$2C$3 C$2 C$4 ) 

C$2 C$5 (C$2 C$3 “t” C$ 2 C$3 ) “t“ C$j C$ 4 C$5 , 

M 3 C$ 4 C$5 (c$2C$3 Tc$2C$3) 

"F C$5 (c$ 2 C$3 C$2 C$3) , 

P(, C3 C$ | C$2 1/4 C$ | (c$2 C$3 T C$2 C$3 ) , 

°pl = a 3 c$,c$ 2 - 4 4 c$i(c$ 2 c$ 3 +c$ 2 c$ 3 ) , 

°P Z 6 = $3 C$ 2 + 4 4 (c$ 2 C$3 C$2 C$3 ) . 
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frames, the transformation is 

°T 6 = °T 1 1 T2 2 T3 3 T4 4 T5 5 T 6 . ( 2 . 46 ) 

Table 2.8 contains the elements of °Tg that are calcu¬ 
lated using Table 2.7 and (2.44). 

Once again, homogeneous transformations provide 
a compact notation, but are computationally inefficient 
for solving the forward kinematics problem. A reduc¬ 
tion in computation can be achieved by separating the 
position and orientation portions of the transformation 
to eliminate all multiplications by the 0 and 1 elements 
of the matrices. In Chap. 3, calculations are made us¬ 
ing the spatial vector notation briefly introduced here 


in Sect. 2.2.6 and explained in detail in Sect. 3.2. This 
approach does not employ homogeneous transforma¬ 
tions, but rather separates out the rotation matrices and 
positions to achieve computation efficiency. Table 3.1 
provides the detailed formulas, with the product of 
spatial transforms particularly relevant to the forward 
kinematics problem. 

Kinematic trees are the general structure of robotic 
mechanisms that do not contain closed loops, and the 
forward kinematics of tree structures are addressed 
in Chap. 3. The forward kinematics problem for closed 
chains is much more complicated because of the addi¬ 
tional constraints present. Solution methods for closed 
chains are included in Chap. 18. 


2.7 Inverse Kinematics 

The inverse kinematics problem for a serial-chain ma¬ 
nipulator is to find the values of the joint positions given 
the position and orientation of the end-effector relative 
to the base and the values of all of the geometric link 
parameters. Once again, this is a simplified statement 
applying only to serial chains. A more general state¬ 
ment is: given the relative positions and orientations of 
two members of a mechanism, find the values of all of 
the joint positions. This amounts to finding all of the 
joint positions given the homogeneous transformation 
between the two members of interest. 

In the common case of a six-degree-of-freedom se¬ 
rial chain manipulator, the known transformation is °Tg. 
Reviewing the formulation of this transformation in 
Sect. 2.6, it is clear that the inverse kinematics problem 
for serial-chain manipulators requires the solution of 
sets of nonlinear equations. In the case of a six-degree- 
of-freedom manipulator, three of these equations relate 
to the position vector within the homogeneous transfor¬ 
mation, and the other three relate to the rotation matrix. 
In the latter case, these three equations cannot come 
from the same row or column because of the depen¬ 
dency within the rotation matrix. With these nonlinear 
equations, it is possible that no solutions exist or mul¬ 
tiple solutions exist [2.33]. For a solution to exist, the 
desired position and orientation of the end-effector must 
lie in the workspace of the manipulator. In cases where 
solutions do exist, they often cannot be presented in 
closed form, so numerical methods are required. 

2.7.1 Closed-Form Solutions 

Closed-form solutions are desirable because they are 
faster than numerical solutions and readily identify all 
possible solutions. The disadvantage of closed-form so¬ 


lutions is that they are not general, but robot dependent. 
The most effective methods for finding closed-form so¬ 
lutions are ad hoc techniques that take advantage of 
particular geometric features of specific mechanisms. 
In general, closed-form solutions can only be obtained 
for six-degree-of-freedom systems with special kine¬ 
matic structure characterized by a large number of 
the geometric parameters defined in Sect. 2.4 being 
zero-valued. Most industrial manipulators have such 
structure because it permits more efficient coordina¬ 
tion software. Sufficient conditions for a six-degree- 
of-freedom manipulator to have closed-form inverse 
kinematics solutions are [2.34-36]: 

1. Three consecutive revolute joint axes intersect at 
a common point, as in a spherical wrist. 

2. Three consecutive revolute joint axes are parallel. 

Closed-form solution approaches are generally di¬ 
vided into algebraic and geometric methods. 

Algebraic Methods 

Algebraic methods involve identifying the significant 
equations containing the joint variables and manipulat¬ 
ing them into a soluble form. A common strategy is 
reduction to a transcendental equation in a single vari¬ 
able such as, 


Ci cos 9j + C 2 sind; + C 3 = 0 , (2.47) 

where Ci, C 2 , and C 3 are constants. The solution to such 
an equation is 


% = 2 tan 


{ C 2 ± yJcj-Cj + C* 


C 1 -C 3 


( 2 . 48 ) 
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Special cases in which one or more of the constants are 
zero are also common. 

Reduction to a pair of equations having the form, 

C 1 cos Qj + C 2 sin0,- + C 3 = 0 , (2.49) 

Ci sin 6 j — C 2 cos #,■ + C 4 = 0 , (2.50) 

is another particularly useful strategy because only one 
solution results, 

9, = Atan2(—C 1 C 4 — C 2 C 3 , C 2 C 4 — C 1 C 3 ) . (2.51) 

Geometric Methods 

Geometric methods involve identifying points on the 
manipulator relative to which position and/or orienta¬ 
tion can be expressed as a function of a reduced set of 
the joint variables. This often amounts to decompos¬ 
ing the spatial problem into separate planar problems. 
The resulting equations are solved using algebraic ma¬ 
nipulation. The two sufficient conditions for existence 
of a closed-form solution for a six-degree-of-freedom 
manipulator that are listed above enable the decomposi¬ 
tion of the problem into inverse position kinematics and 
inverse orientation kinematics. This is the decomposi¬ 
tion into regional and orientation structures discussed in 
Sect. 2.5, and the solution is found by rewriting (2.46), 

°T 6 6 t 5 5 t 4 4 T3 = °Ti 1 T 2 2 T 3 . (2.52) 

The example manipulator in Fig. 2.3 has this structure, 
and its regional structure is commonly known as an 
articulated or anthropomorphic arm or an elbow manip¬ 
ulator. The solution to the inverse position kinematics 
problem for such a structure is summarized in Table 2.9. 
Because there are two solutions for 9\ and likewise two 
solutions for both 62 and 63 corresponding to each 6 1 
solution, there are a total of four solutions to the inverse 
position kinematics problem of the articulated arm ma¬ 
nipulator. The orientation structure is simply a spherical 
wrist, and the corresponding solution to the inverse 
orientation kinematics problem is summarized in Ta¬ 
ble 2.10. Two solutions for 9s are given in Table 2.10, 
but only one solution for both 9 4 and 9g corresponds 
to each. Thus, the inverse orientation kinematics prob¬ 
lem of a spherical wrist has two solutions. Combining 
the regional and orientation structures, the total number 
of inverse kinematics solutions for the manipulator in 
Fig. 2.3 is eight. 

2.7.2 Numerical Methods 

Unlike the algebraic and geometric methods used to 
find closed-form solutions, numerical methods are not 
robot dependent, so they can be applied to any kine¬ 


matic structure. The disadvantages of numerical meth¬ 
ods are that they can be slower and in some cases, they 
do not allow computation of all possible solutions. For 
a six-degree-of-freedom serial-chain manipulator with 
only revolute and prismatic joints, the translation and 
rotation equations can always be reduced to a poly¬ 
nomial in a single variable of degree not greater than 
16 [2.37]. Thus, such a manipulator can have as many 
as 16 real solutions to the inverse kinematics prob¬ 
lem [2.38], Since closed-form solution of a polynomial 
equation is only possible if the polynomial is of degree 
four or less, it follows that many manipulator geome¬ 
tries are not soluble in closed form. In general, a greater 
number of nonzero geometric parameters corresponds 
to a polynomial of higher degree in the reduction. For 
such manipulator structures, the most common numeri¬ 
cal methods can be divided into categories of symbolic 
elimination methods, continuation methods, and itera¬ 
tive methods. 

Symbolic Elimination Methods 
Symbolic elimination methods involve analytical ma¬ 
nipulations to eliminate variables from the system of 
nonlinear equations to reduce it to a smaller set of 
equations. Raghavan and Roth [2.39] used dialytic 
elimination to reduce the inverse kinematics problem 
of a general 6R serial-chain manipulator to a polyno- 

Table 2.9 Inverse position kinematics of the articulated 
arm within the example serial chain manipulator in Fig. 2.3 

0 1 =Atan2(V 6 , 0 ^) 
orAtan2(-V 6 ,- 0 ^) 

83 = —Atan 2 (d, ± Vl -D 2 ) , 

where D := W+W+f# 1 -*-* , 

Za3a4 

0 2 = Atan 2 (% ^(OpJ ) 2 + (V*) 2 ) 

—Atan 2 (da cos 83,03— da sin 83) 

Table 2.10 Inverse orientation kinematics of the spheri¬ 
cal wrist within the example serial chain manipulator in 
Fig. 2.3, with abbreviations cg l '■= cos 0, and sg t := sin #,■ 

8 5 =Atan 2 | ± \fl - (03^1 - mcg 1 f ,n3Sg l -r 2 3Cg^j 

da, =Atan 2 (=F (ri3Cg t + r 2 3^9i) s (9 2 +9 3 ) -F r 33 c ( 9 2 + 8 3 ) , 

± {rnCQi + r 2 3Sg l )c(g 2 +g 3 ) T r 2 3S(g 2 +g 3 )) 

8 6 =Atan 2 (± (n 2 sg 1 + r 22 cg 1 ) . - r 21 cg 2 )) , 

where the d= choice for #5 dictates all of the subsequent d= 
and T for 64 and 65 . 
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mial of degree 16 and to find all possible solutions. The 
roots provide solutions for one of the joint variables, 
while the other variables are computed by solving lin¬ 
ear systems. Manocha and Canny [2.40] improved the 
numerical properties of this technique by reformulating 
the problem as a generalized eigenvalue problem. An 
alternative approach to elimination makes use of Grob- 
ner bases [2.41,42]. 

Continuation Methods 

Continuation methods involve tracking a solution path 
from a start system with known solutions to a tar¬ 
get system whose solutions are sought as the start 
system is transformed into the target system. These 
techniques have been applied to inverse kinematics 
problems [2.43], and special properties of polynomial 
systems can be exploited to find all possible solu¬ 
tions [2.44]. 

Iterative Methods 

A number of different iterative methods can be em¬ 
ployed to solve the inverse kinematics problem. Most 


of them converge to a single solution based on an ini¬ 
tial guess, so the quality of that guess greatly impacts 
the solution time. Newton-Raphson methods provide 
a fundamental approach that uses a first-order approx¬ 
imation of the original equations. Pieper [2.34] was 
among the first to apply the method to inverse kinemat¬ 
ics, and others have followed [2.45,46]. Optimization 
approaches formulate the problem as a nonlinear op¬ 
timization problem and employ search techniques to 
move from an initial guess to a solution [2.47,48]. 
Resolved motion rate control converts the problem to 
a differential equation [2.49], and a modified predictor- 
corrector algorithm can be used to perform the joint 
velocity integration [2.50]. Control-theory-based meth¬ 
ods cast the differential equation into a control prob¬ 
lem [2.51], Interval analysis [2.52] is perhaps one of 
the most promising iterative methods because it of¬ 
fers rapid convergence to a solution and can be used 
to find all possible solutions. For complex mecha¬ 
nisms, the damped least-squares approach [2.53] is 
particularly attractive, and more detail is provided 
in Chap. 10. 


2.8 Forward Instantaneous Kinematics 


The forward instantaneous kinematics problem for 
a serial-chain manipulator is: given the positions of 
all members of the chain and the rates of motion 
about all the joints, find the total velocity of the end- 
effector. Here the rate of motion about the joint is 
the angular velocity of rotation about a revolute joint 
or the translational velocity of sliding along a pris¬ 
matic joint. The total velocity of a member is the 
velocity of the origin of the coordinate frame fixed 
to it combined with its angular velocity. That is, the 
total velocity has six independent components and 
therefore, completely represents the velocity field of 
the member. It is important to note that this prob¬ 
lem definition includes an assumption that the pose 
of the mechanism is completely known. In most sit¬ 
uations, this means that either the forward or in¬ 
verse position kinematics problem must be solved 
before the forward instantaneous kinematics problem 
can be addressed. The same is true of the inverse 
instantaneous kinematics problem discussed in the fol¬ 
lowing section. The forward instantaneous kinematics 
problem is important when doing acceleration anal¬ 
ysis for the purpose of studying dynamics. The to¬ 
tal velocities of the members are needed for the 
computation of Coriolis and centripetal acceleration 
components. 


2.8.1 Jacobian 

Differentiation with respect to time of the forward po¬ 
sition kinematics equations yields a set of equations of 
the form 

k v N =i(q)q, (2.53) 

where k v N is the spatial velocity of the end-effector 
expressed in any frame k, q is an /(-dimensional vec¬ 
tor composed of the joint rates, and J(qr) is a 6 x 
n matrix whose elements are, in general, nonlinear 
functions of q. i(q) is called the Jacobian matrix 
of this algebraic system and is expressed relative 
to the same coordinate frame as the spatial veloc¬ 
ity k v N [2.54]. Alternately, (2.53) can be expressed 
as 

k v N = [ J 1 J 2 ••• J w ] q , (2.54) 

where N is the number of joints (each with possibly 
more than 1 degree-of-freedom) and J, provides the col- 
umn(s) of J(<jr) which correspond(s) to q r If the joint 
positions are known, (2.53) yields six linear algebraic 
equations in the joint rates. If the joint rates are given, 
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a solution of (2.53) is a solution of the forward in¬ 
stantaneous kinematics problem. Note that J(^) can be 
regarded as a known matrix for this purpose provided 
all the joint positions are known. 


Algorithm 2.1 Jacobian Computation Algorithm for 
a Serial-Chain Mechanism 
inputs: k X N , n X n - U ..., 2 Xj 

output: J 

X = ‘X* 

for i = N to 1 do 
J, = X *,■ 
if i > I then 
X = X 'X,_1 
end if 
end for 


Using the spatial vector notation briefly introduced 
in Sect. 2.2.6 and explained in detail in Sect. 3.2, the Ja¬ 
cobian can be easily computed from the free modes <h, 


of the joints and the associated spatial transforms 'X,-. 

J i = k X i * i . (2.55) 

To understand why this holds, note that 0, describes 
the spatial velocities created by joint i in local coordi¬ 
nates and A X, transforms spatial velocities expressed in 
frame i to frame k. The quantities J,- can be efficiently 
computed from k X N and the link-to-link transforms 
l Xj — 1 . Algorithm 2.1 contains an algorithm for com¬ 
puting the columns of the Jacobian in this manner. 
Table 3.1 provides efficient methods to carry out the 
multiplications required in this algorithm which ex¬ 
ploit the structure of spatial transformation matrices. 
Note that the quantity k X N can be computed with 
forward kinematics and each 'X,_i can be computed 
with simple joint kinematics. Thus, use of the algo¬ 
rithm in Alg. 2.1 simplifies the problem of forward 
instantaneous kinematics to one of forward standard 
kinematics. Additional information about the Jacobian 
can be found in Chap. 10. 


2.9 Inverse Instantaneous Kinematics 


The important problem from the point of view of 
robotic coordination is the inverse instantaneous kine¬ 
matics problem. More information on robot coordi¬ 
nation can be found in Chaps. 7 and 8. The inverse 
instantaneous kinematics problem for a serial chain 
manipulator is: given the positions of all members of 
the chain and the total velocity of the end-effector, 
find the rates of motion of all joints. When controlling 
a movement of an industrial robot that operates in the 
point-to-point mode, it is not only necessary to compute 
the final joint positions needed to assume the desired 
final hand position. It is also necessary to generate 
a smooth trajectory for motion between the initial and 
final positions. There are, of course, an infinite number 
of possible trajectories for this purpose. However, the 
most straightforward and successful approach employs 
algorithms based on the solution of the inverse instan¬ 
taneous kinematics problem. This technique originated 
in the work of Whitney [2.55] and of Pieper [2.34]. 

2.9.1 Inverse Jacobian 

In order to solve the linear system of equations in the 
joint rates obtained by decomposing (2.53) into its com¬ 


ponent equations when \ N is known, it is necessary to 
invert the Jacobian matrix. The equation becomes 

« = J -1 (9)vat. (2.56) 

Since J is a 6x6 matrix, numerical inversion is 
not very attractive. It is quite possible for J to be¬ 
come singular (|J| = 0), in which case the inverse 
does not exist. More information on singularities can 
be found in Chaps. 4 and 18. Even when the Jaco¬ 
bian matrix does not become singular, it may become 
ill-conditioned, leading to degraded performance in 
significant portions of the manipulator’s workspace. 
Most industrial robot geometries are simple enough 
that the Jacobian matrix can be inverted analyti¬ 
cally, leading to a set of explicit equations for the 
joint rates [2.56-58]. This greatly reduces the num¬ 
ber of operations needed as compared to numerical 
inversion. For more complex manipulator geometries, 
though, numerical inversion is the only solution op¬ 
tion. The Jacobian of a redundant manipulator is not 
square, so it cannot be inverted. Chapter 10 dis¬ 
cusses how various pseudoinverses can be used in such 
cases. 
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2.10 Static Wrench Transmission 

A general force system can be shown to be equiva¬ 
lent to a single force together with a moment acting 
about its line of action. This is called a wrench. There 
is a deep isometry between the geometries of systems 
of wrench axes and that of systems of instantaneous 
screw axes [2.59]. Static wrench analysis of a manip¬ 
ulator establishes the relationship between wrenches 
applied to the end-effector and forces/torques applied 
to the joints. This is essential for controlling a ma¬ 
nipulator’s interactions with its environment. Examples 
include tasks involving fixed or quasi-fixed workpieces 
such as inserting a component in place with a speci¬ 
fied force and tightening a nut to a prescribed torque. 
More information can be found in Chaps. 9 and 37. 
Through the principle of virtual work, the relation¬ 
ship between wrenches applied to the end-effector 
and forces/torques applied to the joints can be shown 
to be 


r = J T f, (2.57) 

where r is the n-dimensional vector of applied joint 
forces/torques for an n-degree-of-freedom manipulator 
and/ is the spatial force vector 



in which n and / are the vectors of torques and forces, 
respectively, applied to the end-effector, both expressed 
in the coordinate frame relative to which the Jacobian 
is also expressed. Thus, in the same way the Jacobian 
maps the joint rates to the spatial velocity of the end- 
effector, its transpose maps the wrenches applied to the 
end-effector to the equivalent joint forces/torques. As in 
the velocity case, when the Jacobian is not square, the 
inverse relationship is not uniquely defined. 


2.11 Conclusions and Further Reading 

This chapter presents an overview of how the funda¬ 
mentals of kinematics can be applied to robotic mech- 1 
anisms. The topics include various representations of 1 
the position and orientation of a rigid body in space, ( 
the freedom of motion and accompanying mathemat- : 
ical models of joints, a geometric representation that : 
describes the bodies and joints of a robotic mechanism, i 
the workspace of a manipulator, the problems of for- I 
ward and inverse kinematics, the problems of forward 1 
and inverse instantaneous kinematics including the def- i 
inition of the Jacobian, and finally the transmission of 1 
static wrenches. This chapter is certainly not a com- ] 
prehensive account of robot kinematics. Fortunately, 1 
a number of excellent texts provide a broad introduction 1 
to robotics with significant focus on kinematics [2.17, 
27,29,30,51,60-64], 1 


From a historical perspective, robotics fundamen¬ 
tally changed the nature of the field of mecha¬ 
nism kinematics. Before the first work on the gen¬ 
eration of coordination equations for robots [2.34, 
55], the focus of the field was almost entirely on 
single-degree-of-freedom mechanisms. This is why 
robotics, following on from the advent of digi¬ 
tal computing, led to a renaissance of work in 
mechanism kinematics. More details can be found 
in Chap. 4. The evolution of the field has contin¬ 
ued as it has broadened from the study of sim¬ 
ple serial chains for industrial robots, the focus of 
the analysis in this chapter, to parallel machines 
(Chap. 18), human-like grippers (Chap. 19), robotic 
vehicles (Chaps. 17 and 24-26), and even small-scale 
robots (Chap. 27). 
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3. Dynamics 


UJ 


The dynamic equations of motion provide the re¬ 
lationships between actuation and contact forces 
acting on robot mechanisms, and the accelera¬ 
tion and motion trajectories that result. Dynamics 
is important for mechanical design, control, and 
simulation. A number of algorithms are important 
in these applications, and include computation 
of the following: inverse dynamics, forward dy¬ 
namics, the joint-space inertia matrix, and the 
operational-space inertia matrix. This chapter 
provides efficient algorithms to perform each of 
these calculationson a rigid-body model ofa robot 
mechanism. The algorithms are presented in their 
most general form and are applicable to robot 
mechanisms with general connectivity, geometry, 
and joint types. Such mechanisms include fixed- 
base robots, mobile robots, and parallel robot 
mechanisms. 

In addition to the need for computational ef¬ 
ficiency, algorithms should be formulated with 
a compact set of equations for ease of development 
and implementation. The use of spatial notation 
has been very effective in this regard, and is used in 
presenting the dynamics algorithms. Spatial vector 
algebra is a concise vector notation for describing 
rigid-body velocity, acceleration, inertia, etc., us¬ 
ing six-dimensional (6- D) vectors and tensors. 

The goal of this chapter is to introduce the 
reader to the subject of robot dynamics and to 
provide the reader with a rich set of algorithms, 
in a compact form, that they may apply to their 
particular robot mechanism. These algorithms are 
presented in tables for ready access. 
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3.1 Overview 

Robot dynamics provides the relationships between ac¬ 
tuation and contact forces, and the acceleration and 
motion trajectories that result. The dynamic equations 
of motion provide the basis for a number of computa¬ 
tional algorithms that are useful in mechanical design, 
control, and simulation. A growing area of their use is 
in computer animation of mobile systems, especially 
using human and humanoid models. In this Chapter, 
the fundamental dynamic relationships for robot mech¬ 
anisms are presented, along with efficient algorithms 
for the most common computations. Spatial vector no¬ 
tation, a concise representation which makes use of 6-D 
vectors and tensors, is used in the algorithms. 

This chapter presents efficient low-order algorithms 
for four major computations: 

1. Inverse dynamics, in which the required joint actua¬ 
tor torques/forces are computed from a specification 
of the robot’s trajectory (position, velocity, and ac¬ 
celeration), 

2. Forward dynamics in which the applied joint ac¬ 
tuator torques/forces are specified and the joint 
accelerations are to be determined, 

3. The joint-space inertia matrix, which maps the joint 
accelerations to the joint torques/forces, and 

4. The operational-space inertia matrix, which maps 
task accelerations to task forces in operational or 
Cartesian space. 

Inverse dynamics is used in feedforward control and 
trajectory planning. Forward dynamics is required for 
simulation. The joint-space inertia (mass) matrix is used 
in analysis, in feedback control to linearize the dynam¬ 
ics, and is an integral part of many forward dynamics 
formulations. The operational-space inertia matrix is 
used in control at the task or end-effector level. 

3.1.1 Spatial Vector Notation 

Section 3.2 presents the spatial vector notation, which 
is used to express the algorithms in this chapter in 
a clear and concise manner. It was originally developed 
by Featherstone [3.1] to provide a concise vector no¬ 
tation for describing rigid-body velocity, acceleration, 
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inertia, etc., using 6-D vectors and tensors. Section 3.2 
explains the meanings of spatial vectors and operators, 
and provides a detailed tabulation of the correspon¬ 
dence between spatial and standard three-dimensional 
(3-D) quantities and operators, so that the algorithms in 
the later sections can be understood. Formulae for effi¬ 
cient computer implementation of spatial arithmetic are 
also provided. Effort is taken in the discussion of spatial 
vectors to distinguish between the coordinate vectors 
and the quantities they represent. This illuminates some 
of the important characteristics of spatial vectors. 

3.1.2 Canonical Equations 

The dynamic equations of motion are provided in 
Sect. 3.3 in two fundamental forms: the joint-space for¬ 
mulation and the operational-space formulation. The 
terms in the joint-space formulation have traditionally 
been derived using a Lagrangian approach in which 
they are developed independently of any reference co¬ 
ordinate frame. The Lagrange formulation provides 
a description of the relationship between the joint ac¬ 
tuator forces and the motion of the mechanism, and 
fundamentally operates on the kinetic and potential 
energy in the system. The resulting joint-space formula¬ 
tion has a number of notable properties that have proven 
useful for developing control algorithms. The equations 
to relate the terms in the joint-space and operational- 
space formulations, along with an impact model, are 
also provided in this section. 

3.1.3 Dynamic Models 

of Rigid-Body Systems 

The algorithms in this chapter are model-based and re¬ 
quire a data structure describing a robot mechanism 
as one of their input arguments. Section 3.4 gives 
a description of the components of this model: a con¬ 
nectivity graph, link geometry parameters, link inertia 
parameters, and a set of joint models. The description 
of the connectivity is general so that it covers both kine¬ 
matic trees and closed-loop mechanisms. Kinematic 
trees and the spanning tree for a closed-loop mecha¬ 
nism share a common notation. In order to describe 
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the link and joint geometry, two coordinate frames are 
associated with each joint, one each attached to the pre¬ 
decessor and successor links. These frames are defined 
to be compatible with the modified Denavit-Hartenberg 
convention of Craig [3.2] for serial mechanisms con¬ 
taining single-degree-of-freedom (DOF) joints, but are 
nevertheless applicable to general rigid-body systems 
containing general multi-DOF joints. The relationship 
between connected links is described using the general 
joint model of Roberson and Schwertassek [3.3]. A hu¬ 
manoid robot is given as an example to illustrate the link 
and joint numbering scheme, as well as the assignment 
of coordinate frames to describe the links and joints. 
The example includes a floating base, and revolute, uni¬ 
versal, and spherical joints. 

3.1.4 Kinematic Trees 

The algorithms presented in Sect. 3.5 calculate the in¬ 
verse dynamics, forward dynamics, joint-space inertia 
matrix, and operational-space inertia matrix for any 
robot mechanism that is a kinematic tree. An O(n) al¬ 
gorithm for inverse dynamics is provided, where n is 
the number of degrees of freedom in the mechanism. It 
uses a Newton-Euler formulation of the problem, and is 
based on the very efficient recursive Newton-Euler al¬ 
gorithm (RNEA) of Luh et al. [3.4]. Two algorithms are 
provided for forward dynamics. The first is the 0(n) 
articulated-body algorithm (ABA) which was devel¬ 
oped by Featherstone [3.1]. The second is the 0(n 2 ) 
composite-rigid-body algorithm (CRBA), developed by 
Walker and Or in [3.5], to compute the joint-space iner¬ 
tia matrix (JSIM). This matrix, together with a vector 
computed using the RNEA, provide the coefficients of 
the equation of motion, which can then be solved di¬ 
rectly for the accelerations [3.5]. The operational-space 
inertia matrix (OSIM) is a kind of articulated-body in¬ 
ertia, and two algorithms are given to calculate it. The 


3.2 Spatial Vector Notation 

There is no single standard notation for robot dy¬ 
namics. The notations currently in use include 3-D 
vectors, 4x4 matrices, and several types of 6-D vec¬ 
tor: screws, motors, Lie algebra elements, and spatial 
vectors. Six-dimensional vector notations are gener¬ 
ally the best, being more compact than 3-D vectors, 
and more powerful than 4x4 matrices. We therefore 
use 6-D vectors throughout this chapter. In particu¬ 
lar, we shall use the spatial vector algebra described 
in [3.8], This section provides a brief summary of 
spatial vectors. Descriptions of 4x4 matrix nota¬ 


first uses the basic definition of the OSIM, and the sec¬ 
ond is a straightforward O(n) algorithm which is based 
on efficient solution of the forward dynamics prob¬ 
lem. The inputs, outputs, model data, and pseudocode 
for each algorithm are summarized in tables for ready 
access. 

3.1.5 Kinematic Loops 

The above algorithms apply only to mechanisms hav¬ 
ing the connectivity of kinematic trees, including un¬ 
branched kinematic chains. A final algorithm is pro¬ 
vided in Sect. 3.6 for the forward dynamics of closed- 
loop systems, including parallel robot mechanisms. The 
algorithm makes use of the dynamic equations of mo¬ 
tion for a spanning tree of the closed-loop system, and 
supplements these with loop-closure constraint equa¬ 
tions. Three different methods are outlined to solve 
the resulting linear system of equations. Method 2 is 
particularly useful if n n c , where n c is the num¬ 
ber of constraints due to the loop-closing joints. This 
method offers the opportunity to use O(n) algorithms 
on the spanning tree [3.6]. The section ends with an 
efficient algorithm to compute the loop-closure con¬ 
straints by transforming them to a single coordinate 
system. Since the loop-closure constraint equations are 
applied at the acceleration level, standard Baumgarte 
stabilization [3.7] is used to prevent the accumulation 
of position and velocity errors in the loop-closure con¬ 
straints. 

The final section in this chapter provides a conclu¬ 
sion and suggestions for further reading. The area of 
robot dynamics has been, and continues to be, a very 
rich area of investigation. This section outlines the ma¬ 
jor contributions that have been made in the area and 
the work most often cited. Unfortunately, space does 
not permit us to provide a comprehensive review of the 
extensive literature in the area. 


tions can be found in [3.2,9], and descriptions of 
other 6-D vector notations can be found in [3. 10— 
12 ]. 

In this handbook, vectors are usually denoted by 
bold italic letters (e.g.,/, v). However, to avoid a few 
name clashes, we shall use upright bold letters to denote 
spatial vectors (e.g., f, v). Note that this applies only to 
vectors, not tensors. Also, in this section only, we will 
underline coordinate vectors to distinguish them from 
the vectors that they represent (e.g., v_ and v, represent¬ 
ing v and v). 
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3.2.1 Motion and Force 

For mathematical reasons, it is useful to distinguish 
between those vectors that describe the motions of 
rigid bodies, and those that describe the forces act¬ 
ing upon them. We therefore place motion vectors in 
a vector space called M 6 , and force vectors in a space 
called F 6 . (The superscripts indicate the dimension.) 
Motion vectors describe quantities like velocity, accel¬ 
eration, infinitesimal displacement, and directions of 
motion freedom; force vectors describe force, momen¬ 
tum, contact normals, and so on. 

3.2.2 Basis Vectors 

Suppose that v is a 3-D vector, and that v. = 
( v x , Vy, v z ) J is the Cartesian coordinate vector that rep¬ 
resents v in the orthonormal basis \x,y.z). The rela¬ 
tionship between v and t; is then given by the formula 

v = xv x +yvy + zv z . 

This same idea applies also to spatial vectors, except 
that we use Pliicker coordinates instead of Cartesian co¬ 
ordinates, and a Pliicker basis instead of an orthonormal 
basis. 

Pliicker coordinates were introduced in Sect. 2.2.6, 
but the basis vectors are shown in Fig. 3.1. There are 
12 basis vectors in total: six for motion vectors and six 
for forces. Given a Cartesian coordinate frame, O xyz , the 
Pliicker basis vectors are defined as follows: three unit 
rotations about the directed lines Ox, Oy, and Oz, de¬ 
noted by do. v , d oy, and d 0; , three unit translations in the 
directions x, y, and z, denoted by d x , d v , and d z , three 
unit couples about the x, y, and z directions, denoted 
by e A , e,,, and e z , and three unit forces along the lines 
Ox, Oy, and Oz, denoted by Co x , eo y . and e^-. 

3.2.3 Spatial Velocity and Force 

Given any point O, the velocity of a rigid body can be 
described by a pair of 3-D vectors, co and v 0 , which 


specify the body’s angular velocity and the linear veloc¬ 
ity of the body-fixed point currently at O. Note that v 0 
is not the velocity of O itself, but the velocity of the 
body-fixed point that happens to coincide with O at the 
current instant. 

The velocity of this same rigid body can also be de¬ 
scribed by a single spatial motion vector, ve M 6 . To 
obtain v from co and Vo, we first introduce a Carte¬ 
sian frame, O xyz , with its origin at O. This frame defines 
a Cartesian coordinate system for co and v 0 , and also 
a Pliicker coordinate system for v. Given these coordi¬ 
nate systems, it can be shown that 


v — do x oj x + doyCOy + d oz M z + d x vo x 

+ dyVoy + d z vo z , (3.1) 

where u> x ,..., Vq z are the Cartesian coordinates of co 
and v 0 in O xvz . Thus, the Pliicker coordinates of v are 
the Cartesian coordinates of co and v 0 . The coordinate 
vector representing v in O xyz can be written 


< (O x > 
\VOzJ 



(3.2) 


The notation on the far right of this equation is simply 
a convenient abbreviation of the list of Pliicker coordi¬ 
nates. 

The definition of spatial force is very similar. Given 
any point O, any system of forces acting on a single 
rigid body is equivalent to a single force / acting on 
a line passing through O, together with a pure couple, 
n 0 , which is the moment of the force system about 
O. Thus, the two vectors / and n 0 describe the force 
acting on a rigid body in much the same way that co 
and v 0 describe its velocity. This same force can also 
be described by a single spatial force vector, f e F 6 . In¬ 
troducing the frame O xxz , as before, it can be shown that 

f = e x nox + Cyfioy + £ z noz + e oxfx 

+ e-Oyfy + ZOzfz i (3-3) 

where n 0x , ... ,f z are the Cartesian coordinates of n 0 
and / in O xyz . The coordinate vector representing f in 
O xyz can then be written 



Again, these are the Pliicker coordinates of f in 0„ z , 
and the notation on the far right is simply a convenient 
abbreviation of the list of Pliicker coordinates. 
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3.2.4 Addition and Scalar Multiplication and its inverse is 


Spatial vectors behave in the obvious way under addi¬ 
tion and scalar multiplication. For example, if fi and f 2 
both act on the same rigid body, then their resultant 
is fi +f 2 ; if two different bodies have velocities of vi 
and V 2 , then the velocity of the second body relative to 
the first is v 2 — Vi; and if f denotes a force of 1 N acting 
along a particular line in space, then a f denotes a force 
of a N acting along the same line. 

3.2.5 Scalar Product 




(3.10) 


The quantity S(p) is the skew-symmetric matrix that 
satisfies S(p)p = p x v_ for any 3-D vector v_. It is de¬ 
fined by the equation 

/ 0 -p- p y \ 

S(P) = I P Z 0 - Px . (3.11) 

\~Py Px 0 ) 


A scalar product is defined between any two spatial vec¬ 
tors, provided that one of them is a motion and the other 
a force. Given any me M 6 and f e F 6 , the scalar prod¬ 
uct can be written either f • m or m ■ f, and expresses the 
work done by a force f acting on a body with motion m. 
Expressions like f-f and m m are not defined. If m and f 
are coordinate vectors representing m and f in the same 
coordinate system, then 

m-f=m T f. (3.5) 

3.2.6 Coordinate Transforms 


3.2.7 Vector Products 

There are two vector (cross) products defined on spatial 
vectors. The first takes two motion-vector arguments, 
and produces a motion-vector result. It is defined by the 
formula 



Motion and force vectors obey different transformation 
rules. Let A and fi be two coordinate frames, each defin¬ 
ing a coordinate system of the same name; and let m,, 
nig, I, . and fg be coordinate vectors representing the 
spatial vectors m e M 6 and f e F 6 in A and B coordi¬ 
nates, respectively. The transformation rules are then 

m B = B X A m A (3.6) 


and 


The second takes a motion vector as left-hand argument 
and a force vector as right-hand argument, and produces 
a force-vector result. It is defined by the formula 



(3.13) 


= ( 3 - 7 ) 

where B X A and B X B are the coordinate transformation 
matrices from A to fi for motion and force vectors, re¬ 
spectively. These matrices are related by the identity 

X=(Xr r =(X) T . (3.8) 


These products arise in differentiation formulae. 

It is possible to define a spatial cross-product oper¬ 
ator, in analogy with (3.11), as follows 


S(m) 


( S(tn) 0 \ 
\S(HLo) S(m))' 


(3.14) 


Suppose that the position and orientation of frame A 
relative to frame fi is described by a position vector B p■ 
and a 3 x 3 rotation matrix B R A , as described in Sect. 2.2. 
The formula for B X A is then 



in which case 

nijxm, =S(m 1 )m 2 , (3.15) 

but 

mxf=—S(m) T f. (3.16) 

Observe that ,S'(m) maps motion vectors to motion vec¬ 
tors, but S(m) T maps force vectors to force vectors. 
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3.2.8 Differentiation 


The derivative of a spatial vector is defined by 


d s(x + &c) — s(x) 

—s(x) = lim --- 

Ax Sx-t-o ox 


(3.17) 


where s here stands for any spatial vector. The deriva¬ 
tive is a spatial vector of the same kind (motion or force) 
as that being differentiated. 

The formula for differentiating a spatial coordinate 
vector in a moving coordinate system is 

(^ s \ = ^ +yAXS -" <31SI 


where s is any spatial vector, ds/dr is the time deriva¬ 
tive of s, A is the moving coordinate system, (ds/ At)& 
is the coordinate vector that represents ds/ At in A coor¬ 
dinates, §4 is the coordinate vector that represents s in A 
coordinates, d^/df is the time derivative of §4 (which 
is the componentwise derivative, since §4 is a coordi¬ 
nate vector), and V 4 is the velocity of the A coordinate 
frame, expressed in A coordinates. 

The time derivative of a spatial vector that changes 
only because it is moving is given by 

d 

—s = vxs, (3.19) 

At 


where v is the velocity of s. This formula is useful for 
differentiating quantities that do not change in their own 
right, but are attached to moving rigid bodies (e.g., joint 
axis vectors). 


3.2.9 Acceleration 


Spatial acceleration is defined as the rate of change of 
spatial velocity. Unfortunately, this means that spatial 
acceleration differs from the classical textbook defi¬ 
nition of rigid-body acceleration, which we shall call 
classical acceleration. Essentially, the difference can be 
summarized as follows 

and a' = J , (3.20) 

where a is the spatial acceleration, a' is the classical 
acceleration, v_ 0 is the derivative of v_ Q taking O to be 
fixed in space, and v' 0 is the derivative of v_ Q taking O 
to be fixed in the body. The two accelerations are related 
by 



a' = a + 


It/ A U Q 


If r is a position vector giving the position of the body- 
fixed point at O relative to any fixed point, then 

v 0 =r. 
i>o=r, 

Vo = f — (0 x Vo ■ (3.22) 

The practical difference is that spatial accelerations are 
easier to use. For example, if the bodies B\ and Bi have 
velocities of Vi and V 2 , respectively, and v re i is the rela¬ 
tive velocity of lh with respect to Si, then 


V 2 = Vi + Vrel . 

The relationship between their spatial accelerations is 
obtained simply by differentiating the velocity formula 

d 

— (v 2 = v, + v re i) =>■ a 2 = a[ + a re i. 

At 

Observe that spatial accelerations are composed by ad¬ 
dition, exactly like velocities. There are no Coriolis or 
centrifugal terms to worry about. This is a significant 
improvement on the formulae for composing classical 
accelerations, such as those in [3.2, 13, 14], 

3.2.10 Spatial Momentum 


Suppose that a rigid body has a mass of m, a center 
of mass at C, and a rotational inertia of I about C 
(Fig. 3.2). If this body is moving with a spatial veloc¬ 
ity of v c = (« T u£) T , then its linear momentum is h = 
mvc, and its intrinsic angular momentum is h c = 
I 00 . Its moment of momentum about a general point, 

0,ish 0 = h c + cxh, where c = OC. We can assemble 
these vectors into a spatial momentum vector as follows 



(3.23) 


(3.24) 


Spatial momentum is a force vector, and transforms ac¬ 
cordingly. 

3.2.11 Spatial Inertia 


The spatial momentum of a rigid body is the product of 
its spatial inertia and velocity 


(3.21) 


h = /v, 


(3.25) 
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where 7 is the spatial inertia. Expressed in Pliicker co¬ 
ordinates at C, we have 


he — Ic'fc » 
which implies 



(3.26) 


(3.27) 


This is the general formula for the spatial inertia of 
a rigid body expressed at its center of mass. To ex¬ 
press it at another point, O, we proceed as follows. 
From (3.24), (3.26), and (3.27) 


-G T)c; :,)u, T 

_ t / + mS(c)S(c) T mS(c)\ 

~ V mS(c) T ml ) ’ 

but we also have that h 0 = /oV 0 , so 

. _ fi™ + mS(c)S(c) T inS(c) \ 

° mS(c) T ml ) ’ 


This equation can also be written 
Io = 


I 

mS(c) 


o m, 

T 


5(c) \ 
ml ) 


(3.28) 


(3.29) 


where 


Io=r +mS(c)S(cy 


(3.30) 


is the rotational inertia of the rigid body about O. 

Spatial inertia matrices are symmetric and positive- 
definite. In the general case, 21 numbers are required 



Momentum: 

Linear /1 = m vc 
Angular he = / an 0 

Spatial ^ l,c ^ 


to specify a spatial inertia (e.g., for an articulated-body 
or operational-space inertia); but a rigid-body inertia 
needs only 10 parameters: the mass, the coordinates of 
the center of mass, and the six independent elements of 
either 7 or 7 0 . 

The transformation rule for spatial inertias is 

I b = b X f a I a a X b , (3.31) 

where A and B are any two coordinate systems. In prac¬ 
tice, we often need to calculate I A from I B , given only 
b X a . The formula for this transformation is 

I a = ( b X a ) t I b b X a . (3.32) 

If two bodies, having inertias I\ and 1 2 , are rigidly con¬ 
nected to form a single composite body, then the inertia 
of the composite, 7 tot , is the sum of the inertias of its 
parts 

/tot = /| +h - (3.33) 

This single equation takes the place of three equations 
in the traditional 3-D vector approach: one to com¬ 
pute the composite mass, one to compute the composite 
center of mass, and one to compute the composite ro¬ 
tational inertia. If a rigid body with inertia I is moving 
with a velocity of v, then its kinetic energy is 

1 

T = — v • /v . (3.34) 

If a rigid body, B, is part of a larger system, then 
it is possible to define an apparent-inertia matrix for B, 
which describes the relationship between a force acting 
on B and its resulting acceleration, taking into account 
the effects of the other bodies in the system. Such quan¬ 
tities are called articulated-body inertias. If B happens 
to be the end-effector of a robot, then its apparent inertia 
is called an operational-space inertia. 

3.2.12 Equation of Motion 

The spatial equation of motion states that the net force 
acting on a rigid body equals its rate of change of mo¬ 
mentum 

f= —(Jv) = /a + /v. 
df 

It can be shown that the expression 7v evaluates to (v x 
7v) [3.8, 15], so the equation of motion can be written 

f=7a + vx7v. (3.35) 

This single equation incorporates both Newton’s 
and Euler’s equations of motion for a rigid body. To 
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Table 3.1 Summary of spatial vector notation 


Spatial quantities: 

V 

Velocity of a rigid body 

a 

Spatial acceleration of a rigid body (a = v) 

a' 

Classical description of rigid-body acceleration 
expressed as a 6-D vector 

f 

Force acting on a rigid body 

/ 

Inertia of a rigid body 

X 

Pliicker coordinate transform for motion vectors 

X F 

Pliicker coordinate transform for force vectors 
(. X F = X- T ) 

b Xa 

Pliicker transform from A coordinates to B coordinates 

m 

A generic motion vector (any element of M 6 ) 

3-D quantities: 

O 

Coordinate system origin 

r 

Position of the body-fixed point at 0 relative to any 
fixed point in space 

CO 

Angular velocity of a rigid body 

Vo 

Linear velocity of the body-fixed point at 0 (vo = r) 

60 

Angular acceleration of a rigid body 

Vo 

The derivative of vo taking 0 to be fixed in space 

v'o 

The derivative of vo taking 0 to be fixed in the body; 
the classical acceleration of the body-fixed point at 
0(v 0 =r) 

f 

Lineal* force acting on a rigid body, or the resultant of 
a system of linear forces 

n 0 

Moment about 0 of a linear force or system of linear 
forces 

m 

Mass of a rigid body 

c 

Position of a rigid body’s center of mass, measured 
relative to 0 

h 

First moment of mass of a rigid body, h = mc\ can also 
denote linear momentum 

-cm 

Moment of inertia about a body’s centre of mass 

7 

Moment of inertia about 0 

b Ra 

Orthonormal rotation matrix transforming 
from A coordinates to B coordinates 

A Pb 

Location of the origin of B coordinates relative to the 
origin of A coordinates, expressed in A coordinates 


verify this, we can recover them as follows. Express¬ 
ing (3.35) at the body’s center of mass, and using (3.16), 
(3.14), and (3.22), we have 


\ rr o 

\ (S(coj T 

S(v c ) T \(T m co\ 

) V 0 m\)\v c 

J { o 

S{(o)' J\mv c J 


. \ 

/ -cm \ 

(I o \ 

CO \ 

(O XI G)\ 

= .. , .. 

I + 

_ “ 

\ 0 mlj\c — 

60 X V_ c ) 

\ m o)_ x y_ c J 

/-cm . -cm \ 


_ 11 co_-tco_ x/ 

- 


y me 





(3.36) 


Table 3.1 (continued) 



3.2.13 Computer Implementation 

The easiest way to implement spatial vector arithmetic 
on a computer is to start with an existing matrix arith¬ 
metic tool, like MATLAB and write (or download from 
the Web) routines to do the following: 
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1. Calculate S(m) from m according to (3.14). 

2. Compose X from R and p according to (3.9). 

3. Compose / from m, c, and I according to (3.28). 

All other spatial arithmetic operations can be per¬ 
formed using standard matrix arithmetic routines. How¬ 
ever, some additional routines could usefully be added 
to this list, such as: 

• Routines to calculate R from various other represen¬ 
tations of rotation. 

• Routines to convert between spatial and 4x4 matrix 
quantities. 

This is the recommended approach whenever hu¬ 
man productivity is more important than computational 
efficiency. A software package along these lines can be 
found at [3.16]. 

If greater efficiency is required, then a more elabo¬ 
rate spatial arithmetic library must be used, in which 

1. A dedicated data structure is defined for each kind 
of spatial quantity, and 

2. A suite of calculation routines are provided, each 
implementing a spatial arithmetic operation by 
means of an efficient formula. 

Some examples of suitable data structures and ef¬ 
ficient formulae are shown in Table 3.1. Observe that 
the suggested data structures for rigid-body inertias 
and Pliicker transforms contain only a third as many 


3.3 Canonical Equations 

The equations of motion of a robot mechanism are 
usually presented in one of two canonical forms: the 
joint-space formulation, 

(q)q + C(q, q)q + r g (q) = r , (3.37) 

or the operational-space formulation, 

A(x)v + /t(x, \) + p(x) =f. (3.38) 

These equations show the functional dependencies ex¬ 
plicitly: H is a function of q, A is a function of x, and 
so on. Once these dependencies are understood, they are 
usually omitted. In (3.38), x is a vector of operational- 
space coordinates, while v and f are spatial vectors 
denoting the velocity of the end-effector and the exter¬ 
nal force acting on it. If the robot is redundant, then the 
coefficients of this equation must be defined as func¬ 
tions of q and q rather than x and v. 

These two equations are further explained below, 
along with a description of the Lagrange formulation 


numbers as the 6x6 matrices they represent. The ef¬ 
ficient arithmetic formulae listed in this table offer cost 
savings ranging from a factor of 1.5 to a factor of 6 
relative to the use of general 6x6 and 6x1 matrix 
arithmetic. Even more efficient formulae can be found 
in [3.17]. 

3.2.14 Summary 

Spatial vectors are 6-D vectors that combine the linear 
and angular aspects of rigid-body motion, resulting in 
a compact notation that is very suitable for describing 
dynamics algorithms. To avoid a few name clashes with 
3-D vectors, we have used bold upright letters to denote 
spatial vectors, while tensors are still denoted by ital¬ 
ics. In the sections that follow, upright letters will be 
used to denote both spatial vectors and vectors that are 
concatenations of other vectors, like q. 

Table 3.1 presents a summary of the spatial quanti¬ 
ties and operators introduced in this section, together 
with the formulae that define them in terms of 3-D 
quantities and operators. It also presents data structures 
and formulae for efficient computer implementation of 
spatial arithmetic. This table should be read in con¬ 
junction with Tables 2.5 and 2.6, which show how to 
compute the orientation, position, and spatial velocities 
for a variety of joint types. Note that (R, and 'p, in these 
tables correspond to and A p B , respectively, when 
read in conjunction with Table 3.1. 


of (3.37), and the impulsive equations of motion for im¬ 
pact. 

3.3.1 Joint-Space Formulation 

The symbols q, q, q, and r denote /7-dimensional vec¬ 
tors of joint position, velocity, acceleration and force 
variables, respectively, where n is the number of de¬ 
grees of motion freedom of the robot mechanism. H 
is an n x n symmetric, positive-definite matrix, and is 
called the generalized, or joint-space, inertia matrix 
(JSIM). C is an n x n matrix such that C q is the vector 
of Coriolis and centrifugal terms (collectively known as 
velocity product terms); and r g is the vector of grav¬ 
ity terms. More terms can be added to this equation, as 
required, to account for other dynamical effects (e.g., 
viscous friction). The effects of a force f exerted on the 
mechanism at the end-effector can be accounted for by 
adding the term J T f to the right side of (3.37), where J 
is the Jacobian of the end-effector (Sect. 2.8.1). 
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q specifies the coordinates of a point in the mechan¬ 
ism’s configuration space. If the mechanism is a kine¬ 
matic tree (Sect. 3.4), then q contains every joint vari¬ 
able in the mechanism, otherwise it contains only an 
independent subset. The elements of q are generalized 
coordinates. Likewise, the elements of q, q, and r are 
generalized velocities, accelerations, and forces. 

3.3.2 Lagrange Formulation 

Various methods exist for deriving the terms in (3.37). 
The two that are most commonly used in robotics are 
the Newton-Euler formulation and the Lagrange for¬ 
mulation. The former works directly with Newton’s and 
Euler’s equations for a rigid body, which are contained 
within the spatial equation of motion, (3.35). This for¬ 
mulation is especially amenable to the development of 
efficient recursive algorithms for dynamics computa¬ 
tions, such as those described in Sects. 3.5 and 3.6. 

The Lagrange formulation proceeds via the La- 
grangian of the robot mechanism, 

L=T—U, (3.39) 


where T and U are the total kinetic and potential energy, 
respectively, of the mechanism. The kinetic energy is 
given by 

T = -q r Hq . (3.40) 

The dynamic equations of motion can then be devel¬ 
oped using Lagrange’s equation for each generalized 
coordinate 


d dL dL 
d t dcji dq t 


(3.41) 


The resulting equation can be written in scalar form 


'y ' Hij'qj + y y ' Cijkqfqk + 7gi — L > (3.42) 

j= 1 j= 1 k= 1 


which shows the structure of the velocity-product 
terms. C,j k are known as Christoffel symbols of the first 
type, and are given by 


_ 1 (dHjj i 9 H ik m jk \ 

,jk 2 v dqk dqt ) 


(3.43) 


They are functions of only the position variables, q t . 
The elements of C in (3.37) can be defined as 


Cij = Y Cjjkqk • (3.44) 

k= 1 

However, C is not unique, and other definitions are pos¬ 
sible. 


With the choice of C given in (3.44), it is possible 
to show that the matrix N, given by 

(V(q,q)=Lf(q)-2C(q,q), (3.45) 

is skew-symmetric [3.18]. Thus, for any nxl vector a, 

a T (V(q, q) a = 0 . (3.46) 

This property is quite useful in control, especially when 
considering a = q, which gives 

q T (V(q, q) q = 0 . (3.47) 

By applying the principle of conservation of energy, it 
can be shown that (3.47) holds for any choice of the 
matrix C [3.18, 19]. 

3.3.3 Operational-Space Formulation 

In (3.38), x is a 6-D vector of operational-space coordi¬ 
nates giving the position and orientation of the robot’s 
end-effector; v is the velocity of the end-effector; and f 
is the force exerted on the end-effector, x is typically 
a list of Cartesian coordinates, and Euler angles or 
quaternion components, and is related to v via a dif¬ 
ferential equation of the form 

x = E{x)\. (3.48) 

A is the operational-space inertia matrix, which is the 
apparent inertia of the end-effector taking into account 
the effect of the rest of the robot’s mechanism (i. e., it 
is an articulated-body inertia), fi and p are vectors of 
velocity-product and gravity terms, respectively. 

Operational space (also known as task space) is 
the space in which high-level motion and force com¬ 
mands are issued and executed. The operational-space 
formulation is therefore particularly useful in the con¬ 
text of motion and force control systems (Sects. 8.2 and 
9.2). Equation (3.38) can be generalized to operational 
spaces with dimensions other than six, and to opera¬ 
tional spaces that incorporate the motions of more than 
one end-effector [3.20]. 

The terms in (3.37) and (3.38) are related by the 
following formulae 


v = Jq. 

(3.49) 

\ = Jq + jq, 

(3.50) 

r =/ T f, 

(3.51) 

§ 

II 

(3.52) 

,1 = A(JH 'Cq Jq) , 

(3.53) 

p = AJH~ l r g . 

(3.54) 
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These equations assume that m < n (m is the dimension 
of operational-space coordinates), and that the Jaco¬ 
bian/ has full rank. More details can be found in [3.21]. 

3.3.4 Impact Model 

If a robot strikes a rigid body in its environment, then 
an impulsive force arises at the moment of impact and 
causes a step change in the robot’s velocity. Let us as¬ 
sume that the impact occurs between the end effector 
and a rigid body in the environment, and that a spatial 
impulse of f / is exerted on the end-effector. This im¬ 
pulse causes a step change of Av in the end-effector’s 
velocity; and the two are related by the operational- 
space equation of impulsive motion [3.22], 

A Av = f' . (3.55) 

In joint space, the equation of impulsive motion for 
a robot mechanism is 

HAq = T f , (3.56) 

where r' and Aq denote the joint-space impulse and 
velocity change, respectively. In the case of a collision 


involving the robot’s end-effector, we have 

t' =/ T f' (3.57) 

and 

Av = /Aq, (3.58) 

which follow from (3.51) and (3.49). Equations (3.55)- 
(3.57) imply that 

Aq = /Av, (3.59) 

where / is the inertia-weighted pseudoinverse of / and 
is given by 

J = H~ 1 J t A. (3.60) 

/ is also known as the dynamically consistent general¬ 
ized inverse of the Jacobian matrix [3.21]. Note that the 
expression AJH- 1 , which appears in (3.53) and (3.54), 
is equal to / T since H and A are both symmetric. Al¬ 
though we have introduced / in the context of impulsive 
dynamics, it is more typically used in normal (i. e., non- 
impulsive) dynamics equations. 


3.4 Dynamic Models of Rigid-Body Systems 


A basic rigid-body model of a robot mechanism has 
four components: a connectivity graph, link and joint 
geometry parameters, link inertia parameters, and a set 
of joint models. To this model, one can add various 
force-producing elements, such as springs, dampers, 
joint friction, actuators, and drives. The actuators and 
drives, in particular, may have quite elaborate dynamic 
models of their own. It is also possible to add extra mo¬ 
tion freedoms to model elasticity in the joint bearings or 
links (Chap. 11). This section describes a basic model. 
More on this topic can be found in books such as [3.3, 
8,23]. 

3.4.1 Connectivity 

A connectivity graph is an undirected graph in which 
each node represents a rigid body and each arc repre¬ 
sents a joint. The graph must be connected; and exactly 
one node represents a fixed base or reference frame. If 
the graph represents a mobile robot (i. e., a robot that 
is not connected to a fixed base), then it is necessary 
to introduce a fictitious 6-DOF joint between the fixed 
base and any one body in the mobile robot. The chosen 
body is then known as a floating base. If a single graph 


is to represent a collection of mobile robots, then each 
robot has its own floating base, and each floating base 
has its own 6-DOF joint. Note that a 6-DOF joint im¬ 
poses no constraints on the two bodies it connects, so 
the introduction of a 6-DOF joint alters the connectiv¬ 
ity of the graph without altering the physical properties 
of the system it represents. 

In graph-theory terminology, a loop is an arc that 
connects a node to itself, and a cycle is a closed path 
that does not traverse any arc more than once. In the 
connectivity graph of a robot mechanism, loops are 
not allowed, and cycles are called kinematic loops. 
A mechanism that contains kinematic loops is called 
a closed-loop mechanism; and a mechanism that does 
not is called an open-loop mechanism or a kinematic 
tree. Every closed-loop mechanism has a spanning tree, 
which defines an open-loop mechanism, and every joint 
that is not in the spanning tree is called a loop-closing 
joint. The joints in the tree are called tree joints. 

The fixed base serves as the root node of a kine¬ 
matic tree, and the root node of any spanning tree on 
a closed-loop mechanism. A kinematic tree is said to be 
branched if at least one node has at least two children, 
and unbranched otherwise. An unbranched kinematic 
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tree is also called a kinematic chain, and a branched 
tree can be called a branched kinematic chain. A typical 
industrial robot arm, without a gripper, is a kinematic 
chain, while a typical humanoid robot is a kinematic 
tree with a floating base. 

In a system containing N B moving bodies and Nj 
joints, where Nj includes the 6-DOF joints mentioned 
above, the bodies and joints are numbered as follows. 
First, the fixed base is numbered body 0. The other bod¬ 
ies are then numbered from 1 to N B in any order such 
that each body has a higher number than its parent. 
If the system contains kinematic loops then one must 
first choose a spanning tree, and commit to that choice, 
since the identity of a body’s parent is determined by the 
spanning tree. This style of numbering is called a regu¬ 
lar numbering scheme. 

Having numbered the bodies, we number the tree 
joints from 1 to N B such that joint i connects body i to 
its parent. The loop-closing joints, if any, are then num¬ 
bered from N B + 1 to Ay in any order. Each loop-closing 
joint k closes one independent kinematic loop, and we 
number the loops from 1 to N B (where Nl = Nj — N B is 
the number of independent loops) such that loop / is the 
one closed by joint k = N B + l. Kinematic loop / is the 
unique cycle in the graph that traverses joint k , but does 
not traverse any other loop-closing joint. 

For an unbranched kinematic tree, these rules pro¬ 
duce a unique numbering in which the bodies are 
numbered consecutively from base to tip, and the joints 
are numbered such that joint i connects bodies i and 
i— 1. In all other cases, regular numberings are not 
unique. 

Although the connectivity graph is undirected, it 
is necessary to assign a direction to each joint for the 
purpose of defining joint velocity and force. This is 
necessary for both tree joints and loop-closing joints. 
Specifically, a joint is said to connect from one body to 
another. We may call them the predecessor p(i ) and suc¬ 
cessor s(i) for joint i, respectively. Joint velocity is then 
defined as the velocity of the successor relative to the 
predecessor; and joint force is defined as a force acting 
on the successor. It is standard practice (but not a ne¬ 
cessity) for all tree joints to connect from the parent to 
the child. 

The connectivity of a kinematic tree, or the span¬ 
ning tree on a closed-loop mechanism, is described by 
an lV B -element array of parent body numbers, where the 
i-th element p(i) is the parent of body i. Note that the 
parent p(i) for body i is also the predecessor p{i) for 
joint i, and thus the common notation. Many algorithms 
rely on the property p(i) < i to perform their calcula¬ 
tions in the correct order. The set of all body numbers 
for the children of body i, c(i), is also useful in many 
recursive algorithms. 


The connectivity data for kinematic loops may be 
described in a variety of ways. A representation that 
facilitates use in recursive algorithms includes the fol¬ 
lowing conventions. Loop-closing joint k joins bodies 
p{k) (the predecessor) and s(k ) (the successor). The set 
LR{i) for body i gives the numbers of the loops for 
which body i is the root. Using the property p(i) < i for 
bodies in the spanning tree, the root of a loop is chosen 
as the body with the lowest number. In addition, the set 
LB{i) for body i gives the numbers of the loops to which 
body i belongs but is not the root. 

An example of a closed-loop system is given in 
Fig. 3.3. The system consists of a humanoid mechanism 
with topologically varying contacts, with the environ¬ 
ment and within the mechanism, which form closed 
loops. The system has N B = 16 moving bodies and 
Nj = 19 joints with N B = Nj — N B = 3 loops. The main 
body (1) is considered to be a. floating base for this 
mobile robot system. It is connected to the fixed base 
(0) through a fictitious 6-DOF joint (1). To complete 
the example, the loop-closing joint and the body num¬ 
bers p(k) and s(k) as well as the root body for each loop 
are given in Table 3.2. The body-based sets c(i ) and 



Fig. 3.3 Humanoid robot example. Note: to distinguish 
between body numbers and joint numbers in this figure, 
body numbers are preceded by a B for clarity 
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LB(i ) are given in Table 3.3. Note that LR( 0) = {1,3} 
and LR( 1) = {2} and all other LR sets are null for this 
example. 

3.4.2 Link Geometry 

When two bodies are connected by a joint, a complete 
description of the connection consists of a description 
of the joint itself, and the locations of two coordinate 
frames, one in each body, which specify where in each 
body the joint is located. If there are Nj joints in the 
system, then there are a total of 2 Nj joint-attachment 
frames. One half of these frames are identified with the 
numbers 1 to Nj, and the remainder with the labels /I 
to JNj. Each joint i connects from frame Ji to frame i. 

For joints 1 to Nb (i. e., the tree joints), frame i 
is rigidly attached to body i. For joints + 1 to Nj, 
frame k for loop-closing joint k will be rigidly attached 
to body s(k). The second coordinate frame Ji is attached 
to the predecessor p(i) for each joint i, whether it is 
a tree joint or a loop-closing joint. Coordinate frame Ji 
provides a base frame for joint i in that the joint rotation 
and/or translation is defined relative to this frame. 

Figure 3.4 shows the coordinate frames and trans¬ 
forms associated with each joint in the system. The 
overall transform from frame p(f) coordinates to frame i 
coordinates for a tree joint is given by 


a ) b ) x L2 (k) 



Fig.3.4a,b Coordinate frames and transforms associated with 
(a) a tree joint and (b) a loop-closing joint 


transform which completes the transformation across 
joint i from Ji to i coordinates. 

Similarly, the overall transform from frame p(k ) co¬ 
ordinates to frame k coordinates for a loop-closing joint 
is given by 

%(k) = k Xjk Jk X m = Xj(k)X L1 (k) . (3.62) 

An additional transform X L i (k) is defined from 
frame s(k) coordinates to frame k coordinates and is 
given by 


Xli (k) = k X m . 


(3.63) 


% ( o = i X Ji M X p(i) = Xj(i)X L (i) . (3.61) 

The transform X L (i) is a fixed link transform which 
sets the base frame Ji of joint i relative to p(i). It may 
be used to transform spatial motion vectors from p(i) 
to Ji coordinates. The transform Xj(i) is a variable joint 


Table 3.2 Loop-closing joints and roots of loops for the 
humanoid example 


Loop / 

Loop-closing 
joint k 

p(k) 

s(k) 

Root 

1 

17 

0 

5 

0 

2 

18 

16 

1 

1 

3 

19 

0 

13 

0 


Link and joint geometry data can be specified in 
a variety of different ways. The most common method 
is to use Denavit-Hartenberg parameters [3.24]. How¬ 
ever, standard Denavit-Hartenberg parameters are not 
completely general, and are insufficient for describ¬ 
ing the geometry for a branched kinematic tree, or for 
a mechanism containing certain kinds of multi-DOF 
joints. A modified form of Denavit-Hartenberg param¬ 
eters [3.2] is used for single-DOF joints in this Hand¬ 
book (Sect. 2.4). The parameters have been extended 
for branched kinematic trees [3.23] and closed-loop 
mechanisms. 

3.4.3 Link Inertias 


Table 3.3 Body-based sets for the humanoid example 


Body i 

e(i) LB(i) 

Body i 

c(i) 

LB(i) 

0 

1 

9 

10, 11, 14 

2,3 

1 

2,9 1,3 

10 



2 

3,6 1 

11 

12 

3 

3 

4 1 

12 

13 

3 

4 

5 1 

13 


3 

5 

1 

14 

15 

2 

6 

7 

15 

16 

2 

7 

8 

16 


2 

8 






The link inertia data consists of the masses, positions 
of centers of mass, and rotational inertias of each link 
in the mechanism. The inertia parameters for link i are 
expressed in coordinate frame i, and are therefore con¬ 
stants. 

3.4.4 Joint Models 

The relationship between connected links is described 
using the general joint model of Roberson and Schwer- 
tassek [3.3]. For a kinematic tree or spanning tree on 
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a closed-loop mechanism, an n, x 1 vector, </,, relates the 
velocity of link i to the velocity of its parent, link p(i), 
where «, is the number of degrees of freedom at the 
joint connecting the two links. For a loop-closing joint 
in a closed-loop mechanism, the relationship is between 
the velocity of link s(i) (the successor) and the velocity 
of link p(i) (the predecessor). In either case, the rela¬ 
tionship is between the velocity of coordinate frames i 
and Ji. 

Let v re i and a re i denote the velocity and acceleration 
across joint i, that is, the velocity and acceleration of 
link s(i) relative to p(i). The free modes of the joint are 
represented by the 6 x n ,■ matrix 0,, such that v re i and a re i 


are given as follows 


Vrei = 0,<7, 

(3.64) 

and 


a re i = 0,7/, + 0,<7, , 

(3.65) 


where 0, and 0, depend on the type of joint [3.3]. The 
matrix 0, has full column rank, so we can define a com¬ 
plementary matrix, 0 ), such that the 6 x 6 matrix ( 0 , 0 )) 
is invertible. We can regard the columns of this matrix 
as forming a basis on M 6 such that the first n, basis vec¬ 
tors define the directions in which motion is allowed, 
and the remaining 6 — «, = n) vectors define directions 
in which motion is not allowed. Thus, 0) represents the 
constrained modes of joint i. 

The force transmitted across joint i from its prede¬ 
cessor to its successor, f,, is given as follows 

f, = (0,0)) j , (3.66) 

where r,- is the n t x 1 vector of applied forces along the 
free modes. A, is the (6 — n,) x 1 vector of constraint 
forces, and 0 , and 0 ) are computed as follows 

(00;) = (0,0)' y T . (3.67) 

For most common joint types, it is possible to choose 0, 
and 0 ) such that the matrix ( 0 , 0 )) is numerically 
orthonormal, so that ( 0 , 0 )) is numerically equal 
to (0,0)). Note that (3.67) implies the following 
relationships: (0,) T 0, = (0,) T 0) = 0 „,. x(6 _„,.), 

(0)) T 0, = 0( 6 _„, and (0) ) T 0) = 1(6—ni)x(6—«,•) • 
When applied to (3.66), the following useful relation¬ 
ship results 

T, = 0)f,-. (3.68) 

The value of 0, in (3.65) depends on the type of 
joint. The general formula is 


where v, is the velocity of link i, and 0 , is the apparent 
derivative of 0,, as seen by an observer moving with 
link i, and is given by 

o 90, 

0/ = - 5 - q, ■ (3.70) 

dqi 

O 

For most common joint types, 0, = 0. 

Single-DOF joints (n,- = 1) are especially straight¬ 
forward to work with when using the Denavit- 
Hartenberg convention. Motion is chosen along (pris¬ 
matic) or about (revolute) the z,- coordinate axis. In this 
case, 0, = (000001 ) T for a prismatic joint and 0, = 
( 0 0 1 0 0 0 ) T for a revolute joint. Also, 0, =0. 

The fictitious 6 -DOF joint for a floating base for 
a mobile robot is also handled relatively easily. For this 

O 

case, 0, = 1 (6 x 6 identity matrix) and 0, = 0. 

The revolute joint and floating-base joint, as well as 
the universal joint (n, = 2) and spherical joint («,- = 3) 
are illustrated in the example in the next section. For 
additional details on joint kinematics, see Sect. 2.3. 

3.4.5 Example System 

In order to illustrate the conventions used for the link 
and joint models, coordinate frames are attached to the 
first five links (bodies) and fixed base of the humanoid 
robot as shown in Fig. 3.5. Note that frame Ji is attached 
to link p(i ) = i — 1 for each of the five joints. For this 
example, the origin of frame /I is set coincident with 
the origin of frame 0, and the origins of frames J2, J 3, 
J4, and J 5 are coincident with the origins of frames 2, 
3, 4, and 5, respectively. 

Note that 71 could be set at any position/orientation 
on the fixed base (B0) to permit the most convenient 

Table 3.4 Number of degrees of freedom («,), fixed rota¬ 
tion ( J 'R p a )) and position ( P ^pji) from frame p(i) to base 
frame Ji for joint i of the example system. Note that 2/,■ is 
the nominal length of link i along its long axis 



0 , = 0 , + v, x 0 , , 


(3.69) 
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Fig. 3.5 Coordinate frames for the first five links and 
joints of the humanoid robot example 


representation for the motion of the floating base (Bl) 
relative to the fixed base. Also, the origin of 72 could 
be set anywhere along ii- 

The number of degrees of freedom, and fixed rota¬ 
tion and position of the base frames Ji for each joint 
of the example system, are given in Table 3.4. The 

3.5 Kinematic Trees 

The dynamics of a kinematic tree is simpler, and eas¬ 
ier to calculate, than the dynamics of a closed-loop 
mechanism. Indeed, many algorithms for closed-loop 
mechanisms work by first calculating the dynamics of 
a spanning tree, and then subjecting it to the loop- 
closure constraints. 

This section describes the following dynamics al¬ 
gorithms for kinematic trees: the recursive Newton- 
Euler algorithm (RNEA) for inverse dynamics, the 
articulated-body algorithm (ABA) for forward dynam¬ 
ics, the composite-rigid-body algorithm (CRBA) for 


rotation Jl R p u) transforms 3-D vectors in p(i) coor¬ 
dinates to Ji coordinates. The position P ^pu is the 
vector giving the position of the origin Oj, relative to 
O p d), expressed in p(i) coordinates. The spatial trans¬ 
form X L (i) = Jl X p (i) may be composed from these 3-D 
quantities through the equation for B X A in Table 3.1. 
The humanoid has a floating base, the torso, a revolute 
joint between the torso and pelvis (about ii), a spher¬ 
ical joint at the hip, a revolute joint at the knee, and 
a universal joint at the ankle. As shown in Fig. 3.5, the 
leg is slightly bent and the foot is turned out to the side 
(f« 90° rotation abouty 3 at the hip). 

The free modes, velocity variables, and position 
variables for all of the joint types in the humanoid 
are given in Tables 2.5 and 2.6. The expressions for 
J R, and in these tables give l R J n and J 'pi, respec¬ 
tively, through which the joint transform Xj(i) = ‘Xjj 
may be composed. Revolute joints follow the Denavit- 
Hartenberg convention with rotation about the z, axis. 
The ankle has a pitch rotation of as about the Zjs axis 
followed by a roll rotation of ft 5 about the y 5 axis (see 
the Z-Y-X Euler angle definitions in Table 2.1). The 
hip is modeled as a ball-and-socket, spherical joint. 
To avoid the singularities that are associated with Eu¬ 
ler angles, the quaternion e,- may be used to represent 
the orientation at the hip. The relationship between the 
quaternion rate e, and the relative rotation rate w, re 1 is 
given in (2.8) of the Handbook. 

The floating base uses the position of the torso °p 1 
and quaternion e 1 for its position and orientation state 
variables, respectively. The position of the torso may 
be computed by integrating the velocity for the link, 
as expressed in fixed base coordinates: = °RjVi, 

where v 1 is the velocity of the torso in moving coordi¬ 
nates. 

O 

Note that 0, = 0 for all joints except the universal 
joint. Since the components of Zjs in link 5 coordinates 
vary with /J 5 ,Zj 5 i=- 0. See Sect. 2.3 of the Handbook for 
further details of the joint kinematics. 


calculating the joint-space inertia matrix (JSIM), and 
two algorithms to calculate the operational-space iner¬ 
tia matrix (OSIM). Implementations of the first three 
can be found in [3.16]. 

3.5.1 The Recursive Newton-Euler 
Algorithm 

This is an O(n) algorithm for calculating the inverse dy¬ 
namics of a fixed-base kinematic tree, and is based on 
the very efficient RNEA of Luh et al. [3.4]. A floating- 
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base version can be found in [3.8, 15]. Given the joint 
position and velocity variables, this algorithm calcu¬ 
lates the applied joint torque/force variables required to 
produce a given set of joint accelerations. 

The link velocities and accelerations are first com¬ 
puted through an outward recursion from the fixed base 
to the leaf links of the tree. The required forces on 
each link are computed using the Newton-Euler equa¬ 
tions (3.35) during this recursion. A second, inward 
recursion uses the force balance equations at each link 
to compute the spatial force across each joint and the 
value of each joint torque/force variable. The key step 
for computational efficiency is to refer most quantities 
to local link coordinates. Also, the effects of gravity on 
each link are efficiently included in the equations by ac¬ 
celerating the base of the mechanism upward. 

The calculation proceeds in four steps, as follows, 
with two steps in each of the two recursions. 

Step 1 

Calculate the velocity and acceleration of each link in 
turn, starting with the known velocity and acceleration 
of the fixed base, and working towards the tips (i. e., the 
leaf nodes in the connectivity graph). 

The velocity of each link in a kinematic tree is given 
by the recursive formula 

V; = \ p (i) + , (v 0 = 0) , (3.71) 

where v, is the velocity of link i, 0 , is the motion matrix 
of joint i, and q j is the vector of joint velocity variables 
for joint i. 

The equivalent formula for accelerations is obtained 
by differentiating (3.71), giving 

a, = a, ;( i) + 0,<j, + <t>,q, , (a 0 = 0) , (3.72) 

where a, is the acceleration of link i, and q t is the vector 
of joint acceleration variables. 

The effect of a uniform gravitational field on the 
mechanism can be simulated by initializing ao to —a g 
instead of zero, where a,, is the gravitational accelera¬ 
tion vector. In this case, a,- is not the true acceleration of 
link i, but the sum of its true acceleration and —a a . 

Step 2 

Calculate the equation of motion for each link. This step 
computes the forces required to cause the accelerations 
calculated in step 1. The equation of motion for link i is 

ff = 7, a, + v, x /,v,, (3.73) 

where 7, is the spatial inertia of link i, and ff is the net 
force acting on link i. 



Step 3 

Calculate the spatial force across each joint. Referring 
to Fig. 3.6, the net force acting on link i is 

*?=?+*-£«-. 

jSc(i) 

where f, is the force transmitted across joint i, ff is the 
sum of all relevant external forces acting on link i, and 
c(i) is the set of children of link i. Rearranging this 
equation gives the following recursive formula for cal¬ 
culating the joint forces 

jec(i) 

where i iterates from Nb to 1. 

ff may include contributions from springs, dampers, 
force fields, contact with the environment, and so on, 
but its value is assumed to be known, or at least to be 
calculable from known quantities. 

If gravity has not been simulated by a fictitious base 
acceleration, then the gravitational force acting on link i 
must be included in ff. 

Step 4 

Calculate the joint force variables, r,-. By definition, 
they are given by the equation 

Ti = 0jf/ • (3.75) 

Coordinate-Free Algorithm 
Equations (3.71)—(3.75) imply the algorithm shown in 
Algorithm 3.1, which is the coordinate-free version of 
the RNEA. This is the simplest form of the algorithm, 
and it is suitable for mathematical analysis and re¬ 
lated purposes. However, it is not suitable for numerical 
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computation because a numerical version must use co¬ 
ordinate vectors. 


Algorithm 3.1 Coordinate-free recursive Newton- 
Euler algorithm (RNEA) for inverse dynamics 
v 0 = 0 
30 ~ «tg 
for i = 1 to N b do 
V; = \ p (i) + 0, q, 
a, = a,,(,, + <t>, q, + 0, q, 

f i = h a, + V; X /, V; - ff 

end for 

for i = N b to 1 do 

T, = f, 

if p(i) / 0 then 

^(o = Uo + f ' 

end if 
end for 


Link-Coordinates Algorithm 
In general, we say that an algorithm is implemented 
in link coordinates if a coordinate system is defined 
for each link, and the calculations pertaining to link i 
are performed in the coordinate system associated with 
link i. The alternative is to implement the algorithm 
in absolute coordinates, in which case all calculations 
are performed in a single coordinate system, typically 
that of the base link. In practice, the RNEA is more 
computationally efficient when implemented in link co¬ 
ordinates, and the same is true of most other dynamics 
algorithms. 

To convert the RNEA to link coordinates, we 
first examine the equations to see which ones involve 
quantities from more than one link. Equations (3.73) 
and (3.75) each involve quantities pertaining to link i 
only, and therefore need no modification. Such equa¬ 
tions are said to be local to link i. The remaining 
equations involve quantities from more than one link, 
and therefore require the insertion of coordinate trans¬ 
formation matrices. The modified versions of (3.71), 
(3.72), and (3.74) are 

V; = X p(i) \ p(i) + 0 i q i , (3.76) 

a , = X p(l) a p(;) + 0^ + 0^ , (3.77) 

and 

f, = ff - X' a °ff + £ Xft,. (3.78) 

jSc(i) 

Equation (3.78) assumes that external forces are ex¬ 
pressed in absolute (i. e., link 0 ) coordinates. 


Algorithm 3.2 Recursive Newton-Euler algorithm 
using spatial vectors 
inputs: q, q, q, model , °f® 

output: r 

model data : N B , jtype(i), p(i), X L (i), I , 

Vo = 0 

a 0 — — a g 

for i = 1 to Nb do 

Xj(i) = xjcalc(jtype(0,?;) 

X pU) =Xj(i)X L (i) 
if p(i) / 0 then 

i x 0 = i x pU) p (i) x 0 

end if 

4>i =pcalc(jtype( 0 ,?,) 

O 

4>i = pdcalc(jtype(i), qi.q,) 

Vi Xp(i) \p(l) + 1 

= <£,<?, + Vi X 

ai Xp(i) a ; ,(i) 1 , + t(i 

f, = /,a,- + V; x /,-Vi — l Xf r °ff 

end for 

for i = N b to 1 do 

r ,■ = 0jfi 

if p(i) ^ 0 then 

f P (0 = f p(0 + ' X pU) fi¬ 
end if 
end for 


The complete algorithm is shown in Algorithm 3.2. 
The function j type returns the type code for joint i ; 
the function xjcalc calculates the joint transforma¬ 
tion matrix for the specified type of joint; and the 

O 

functions pcalc and pdcalc calculate 0 , and The 
formulae used by these functions for a variety of joint 
types can be found in Tables 2.5 and 2.6, bearing in 
mind that the rotation matrices that must be used are the 
transposes of those listed in these tables. In the general 
case, both pcalc and pdcalc are needed. However, 
for most common joint types, 0 , is a known constant 

O 

in link coordinates, and is therefore zero. If it is 
known in advance that all joints will have this property, 
then the algorithm can be simplified accordingly. The 
quantities J, and X L (i) are known constants in link co¬ 
ordinates, and are part of the data structure describing 
the robot mechanism. 

The last assignment in the first loop initializes 
each f, to the expression ff — Xq °f f (using the iden¬ 
tity Xq = Xjf T ). The summation on the right-hand side 
of (3.78) is then performed in the second loop. This al¬ 
gorithm includes code to calculate Xq, which is used 
to transform the external forces to link coordinates. If 
there are no external forces, then this code can be omit¬ 
ted. If there is only a single external force (e.g., a force 
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at the end-effector of a robot arm) then this code can 
be replaced with code that transforms the external force 
vector successively from one link coordinate system to 
the next, using ’X p ^). 

Note: although the phrase link coordinates suggests 
that we are using moving coordinate frames, the algo¬ 
rithm is in fact implemented in stationary coordinates 
that happen to coincide with the moving coordinates at 
the current instant. 

3-D Vector RNEA 

The original version of the RNEA was developed 
and expressed using 3-D vectors (e.g. [3.2,4]). Algo¬ 
rithm 3.3 shows a special case of this algorithm, in 
which the joints are assumed to be revolute, and the 
joint axes are assumed to coincide with the z axes of the 
link coordinate systems. (Without these assumptions, 
the equations would be a lot longer.) It also assumes 
that the external forces are zero. 


Algorithm 3.3 Recursive Newton-Euler algorithm 
in 3-D vectors, for revolute joints only 
inputs: q, q, q, model 
output: r 

model data : N B , p(i). Rid), pi,) Pi, m;, c,-, 7™ 

«o = 0 
Wo = o 

K = -v'g 

for / = 1 to N b do 

'Rp(i) = rotz (qi)R L (i) 

(V i — Rp(i) r^p(i) T zfli 
di — Rp(i) d>p(i) T (Rp(i) (ttpU )) x z.i 'tji T zfc[i 

v'^'RpU) («'o)+«f(o xMi V 

+«,,(,) X (Op (,) x pMpi) 
fi = mfv'i + (bj x Cj + a>i x x c,) 
n, = 1 1 (o, + (o, x 7 ; coj + c, x/, 

end for 

for / = N b to 1 do 

-T 

Ci = z,- rii 
if p(i) 0 then 
fp(i) = fp(i ) + R p (i)fi 

n pU) = n pd ) + iR p U ) n i+ p 0) P- x iR p(i)fi 

end if 
end for 


In this algorithm, vf is the linear acceleration due 
to gravity, expressed in base (link 0) coordinates; rotz 
computes the transpose of the matrix shown in (2.2); 
R[,(i) is the rotational component of X L {i)\ 'R p (i) is the 
rotational component of l X p Q ); pcalc and pdcalc 
are not used because is the known constant (z T 0 T ) T ; 


v\ is the linear acceleration of the origin of link i coor¬ 
dinates (£>,), and is the linear component of the classical 
acceleration of link i\ P ^’pi is the position of O, relative 
to Opd) expressed in p(i) coordinates; and 774 , c,-, and7 ; 
are the inertia parameters of link i. (See Table 3.1 for 
the equations relating these 3-D quantities to the corre¬ 
sponding spatial quantities.) 

At first sight, the 3-D vector algorithm looks sig¬ 
nificantly different from the spatial vector algorithm. 
Nevertheless, it can be obtained directly from the spa¬ 
tial vector algorithm simply by expanding the spatial 
vectors to their 3-D components, restricting the joint 
type to revolute, converting spatial accelerations to clas¬ 
sical accelerations (i. e., replacing each instance of v ,■ 
with v' — (OjXVi as per (3.22)), and applying some 
3-D vector identities to bring the equations into the 
form shown in the table. The conversion from spatial 
to classical acceleration has one interesting side-effect: 
v i cancels out of the equation of motion, and therefore 
does not need to be calculated. As a result, the 3-D ver¬ 
sion of the algorithm has a slight speed advantage over 
the spatial version. 

3.5.2 The Articulated-Body Algorithm 

The ABA is an O(Np) algorithm for calculating the 
forward dynamics of a kinematic tree. However, un¬ 
der normal circumstances, 0(N B ) = O(n), so we shall 
refer to it as an 0(n) algorithm. The ABA was de¬ 
veloped by Featherstone [3.1] and is an example of 
a constraint-propagation algorithm. Given the joint po¬ 
sition, velocity, and applied torque/force variables, this 
algorithm calculates the joint accelerations. With the 
joint accelerations determined, numerical integration 
may be used to provide a simulation of the mechanism. 

The key concept in the ABA is illustrated in 
Fig. 3.7. The subtree rooted at link i interacts with the 
rest of the kinematic tree only through a force f,- that is 
transmitted across joint i. Suppose we break the tree at 
this point, and consider only the motion of the subtree 
subject to an unknown force, f,-, acting on link i. It is 
possible to show that the acceleration of link i is related 
to the applied force according to the equation 

f, = /fa, + p? , (3.79) 

where if is called the articulated-body inertia of link i in 
the subtree (which we can now call an articulated body), 
and pj 1 is the associated bias force, which is the force 
required to produce zero acceleration in link i. Note 
that pj 1 depends on the velocities of the individual bod¬ 
ies in the articulated body. Equation (3.79) takes into 
account the complete dynamics of the subtree. Thus, if 
we happened to know the correct value of f,, then (3.79) 
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would immediately give us the correct acceleration of 
link i. 

The reason we are interested in the quantities if 
and P; is that they allow us to calculate q : from a p (,-), 
which in turn allows us to calculate a,-, which then al¬ 
lows us to calculate more joint accelerations, and so on. 
Combining (3.79) with (3.75) and (3.72) gives 

r, = 4>Jt) = 0] {lf( Mi) + 0{q t + #,■«,.) + p?) , 

which can be solved for q i to give 

9 i=D i (u i -Uja p(i) ) , (3.80) 

where 

Vi = if @1 , 

Di=(0fUi)-' = wif^r 1 , 

«i = T i -U]S l -*Jpf 

and 

Ci = 0,q, = 0 ,(1, + Vi x 0,q t . 

a, can then be calculated via (3.72). 

It turns out that the articulated-body inertias and 
bias forces can be calculated efficiently via the recur¬ 
sive formulae 


Xj(i) = xjcalc(jtype(i), qi) 
%(0 =Xj(i)X L (i) 
if p(i) / 0 then 
i X 0 = i X p(i) P^X 0 

end if 

0, = pcalc(jtype(0.<7,) 

O 

0, = pdcalc( jtype(i). qj. q t ) 
C Xp(j) Vp(i) T 0, qj 

= 0, q, + v, x 0, q t 

If=h 

Pf = v ; x /, v,- — ‘Xf 1 °ff 

end for 

for i = Ay; to 1 do 


V, = If 0/ 

Di = (0j Ui)~ l 

U, = T,-U](,-0] P f 

if p(i) / 0 then 

Kih=K(h+'K(0 W~V,DiU])‘X p(l) 

4) = pAo + *Ao (pf + 7 f<'■ + U < D < U <) 


end if 
end for 

for 1 = 1 to N b do 


a i = 'Xp (0 a p(i) 

q, = Di (m - Vj a,) 
a, = a, + 0, q, + <, 


end for 


I i= I ‘+ J 2 (!?- u Pi u ]) < 3 - 81 > 

ysc(o 

and 

Pf = P- + J2 (p/ + 7 f& + V i D i u j) ■ < 3 - 82 ) 

ySc(;) 

where 

Pi = Vi x 7,-Vi - ff . 

These formulae are obtained by examining the relation¬ 
ship between f, and a, in Fig. 3.7 and assuming that if 
and pj 1 are already known for every j e c(i). For more 
details see [3.1,8, 1 5, 25] . 

The complete algorithm is shown in Algorithm 3.4. 

Algorithm 3.A Articulated-body algorithm for for¬ 
ward dynamics 
inputs: q, q, r, model, °ff 

output: q 

model data : N B , jtype(i), p(i), X L (i), /, 

vo = 0 
3o = — 3g 

for i = 1 to N b do 


It is expressed in link coordinates, as per the RNEA 
in Algorithm 3.2. It makes a total of three passes 
through the kinematic tree. The first pass iterates from 
the base out to the tips; it calculates the link veloci¬ 
ties using (3.76), the velocity-product term = 0,q,, 
and it initializes the variables If and pj 1 to the values /, 
and p,(= V,- x/ ; v; — 'Xq °ff), respectively. The second 
pass iterates from the tips back to the base; it calculates 
the articulated-body inertia and bias force for each link 
using (3.81) and (3.82). The third pass iterates from the 
base to the tips; it calculates the link and joint accelera¬ 
tions using (3.80) and (3.77). 
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3.5.3 The Composite-Rigid-Body Algorithm 

The CRB A is an algorithm for calculating the joint- 
space inertia matrix (JSIM) of a kinematic tree. The 
most common use for the CRBA is as part of a for¬ 
ward dynamics algorithm. It first appeared as method 3 
in [3.5]. 

Forward dynamics, in joint space, is the task of 
calculating q from q, q, and r. Starting from (3.37), 
the most obvious way to proceed is to calculate H 
and Cq + r g , and then solve the linear equation 

Hq = t — (Cq + T g ) (3.83) 

for q. If the mechanism is a kinematic tree, then H 
and Cq+Tg can be computed in 0(n 2 ) and O(n) 
operations, respectively, and (3.83) can be solved in 
0(n 3 ) operations. Algorithms that take this approach 
are therefore known collectively as 0(n 3 ) algorithms. 
However, this figure of 0(n 3 ) should be regarded as the 
worst-case complexity, since the actual complexity de¬ 
pends on the amount of branching in the tree [3.26]. 
Furthermore, even in the worst case, the n 3 term has 
a small coefficient, and does not dominate until approx¬ 
imately n = 60. 

Cq + T g can be calculated using an inverse dynam¬ 
ics algorithm. If ID( q. q, q) is the result of an inverse 
dynamics calculation with arguments q, q and q, then 

ID(q, q, q) = r = Hq + Cq + r g , 


where c* (i) is the set of links in the subtree rooted at 
link i, including link i itself, and 

1 ? = J2 h ■ (3 - 86) 

J€c*(0 

See [3.8, 15]. In fact, if is the inertia of the composite 
rigid body formed by the rigid assembly of all the links 
in and this is where the algorithm gets its name. 

Equations (3.85) and (3.86) are the basis of the al¬ 
gorithm shown in Algorithm 3.5, which is the CRBA in 
link coordinates. 


Algorithm 3.5 Composite-rigid-body algorithm 
for calculating the JSIM 

inputs: mode1 , RNEA partial results 

output: H 

model data : Nb, p(i), h 
RNEA data : l X p U) 

H = 0 

for i = 1 to N b do 
7 C = / 

ij — 1 , 

end for 

for i = N b to 1 do 
F = If 0, 

H u = 0jF 
if p(i) / 0 then 

rC _ rC I i vT jC i v 

p(i) ~ p(i) + p(0 l i A Pd) 

end if 


so 


Cq+ T g = ID(q. q, 0) . 


(3.84) 


Thus, the value of Cq + r g for a kinematic tree can be 
calculated efficiently using the RNEA with q = 0. 

The key concept in the CRBA is to note that the 
JSIM only depends on the joint positions, and not their 
rates. The CRBA makes the simplifying assumption 
that the rate at each joint is zero. By also assuming 
that gravity is zero, Cq + r g is eliminated from (3.83). 
Furthermore, for a revolute joint, a unit of joint acceler¬ 
ation applied at the y-th joint produces the y th column 
of the JSIM. This partitions the mechanism into two 
composite rigid bodies connected by the y-th joint, and 
simplifies the dynamics considerably. This concept has 
been generalized so that the CRBA may be applied to 
any joint type within a kinematic tree structure. 

It can be shown that the general form of the JSIM 
for a kinematic tree is 


Htj = 


0flf^ 
0Jlf<Pj 


0 


if iec*{j) 
if 7 e c* (i) 
otherwise, 


(3.85) 


j = i 

while p(j) f 0 do 

F = JX Pdi F 
j = Pij) 

Hij = F T 0j 

Hji=Hfj 

end while 
end for 


This algorithm assumes that the matrices 'X p a) 
and 4>j have already been calculated, e.g., during the 
calculation of Cq + r g . If this is not the case, then the 
relevant lines from Algorithm 3.2 can be inserted into 
the first loop. The matrix F is a local variable. The first 
step, H = 0, can be omitted if there are no branches in 
the tree. 

Having calculated Cq + r g and H. the final step is 
to solve (3.83) for q. This can be done using a standard 
Cholesky or LDL J factorization. Note that H can be 
highly ill-conditioned [3.27], reflecting an underlying 
ill-conditioning of the kinematic tree itself, so it is rec¬ 
ommended to use double-precision arithmetic for every 
step in the forward dynamics calculation. (This advice 
applies also to the ABA.) 
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Exploiting Sparsity 

Equation (3.85) implies that some elements of H will 
automatically be zero if there are branches in the kine¬ 
matic tree. An example of this effect is shown in 
Fig. 3.8. Observe that nearly half of the elements are 
zero. It is possible to exploit this sparsity using the 
factorization algorithms described in [3.26]. Depend¬ 
ing on the amount of branching in the tree, the sparse 
algorithms can run many times faster than the standard 
algorithms. 

3.5.4 Operational-Space Inertia Matrix 

Two different algorithms are presented to calculate the 
OSIM. The first is an 0(n 3 ) algorithm that uses the 
basic definition of the OSIM along with efficient fac¬ 
torization of the JSIM. The second is an O( n ) algorithm 
that is based on efficient solution of the forward dynam¬ 
ics problem. 

Algorithm Using Basic Definition 
If a robot has a relatively small number of freedoms 
(e.g., six), then the most efficient method for calculating 
the OSIM is via (3.52). The procedure is as follows: 

1. Calculate H via the CRBA. 

2. Factorize H into H = LL T (Cholesky factorization). 

3. Use back-substitution to calculate Y = LT X J 1 . 

4. A~' = Y t Y. 

5. Factorize A 1 (optional). 

The final step is only possible if the end-effector 
has a full six DOFs, and is only necessary if the appli¬ 
cation requires A rather than A~K In the second step, 
an LDL T factorization can be used instead of Cholesky, 
or one can use one of the efficient factorizations de¬ 
scribed in [3.26] for branched kinematic trees. 

The other terms in (3.38) can be calculated via 
(3.53) and (3.54). In particular, (3.38) can be rewritten 
in the form 

v + A~ 1 (x)[p(x, v) + /o(x)] = A _1 (x) f. (3.87) 



Fig. 3.8 Branch-induced sparsity: branches in the kine¬ 
matic tree cause certain elements in the JSIM to be zero 


and the quantity A ] (p, + p) can be calculated from the 
formula 

A~ l (p, + p) = JH~ l (Cq + T g ) — jq . (3.88) 

The term /q is the velocity-product acceleration of the 
end-effector (3.50). It is calculated as a by-product of 
calculating Cq + r g via the RNEA (3.84). Specifically, 
J 4 = a ee — ao, where a ee is the calculated acceleration 
of the end-effector (expressed in the same coordinates 
as v) and ao is the acceleration of the base (—a g ). 

0(n) Algorithm 

For a sufficiently large value of n, it becomes more 
efficient to use an 0(n) algorithm. Several such algo¬ 
rithms can be found in [3.28-30]. In this section, a more 
straightforward algorithm is given, which is based on 
an 0(n) calculation of the joint-space forward dynam¬ 
ics problem, e.g., via the ABA. It is a variation of the 
unit force method [3.29] and computes the inverse of 
the OSIM. 

Starting with (3.87), observe that A^ 1 is a func¬ 
tion of position only, and certain terms in the dynamic 
equations can be neglected without affecting its value. 
Specifically, if the joint rates, q, joint forces, r, and 
gravitational forces are all set to zero, the value of A 
will remain unchanged. Under these conditions, 

v = zt _1 f. (3.89) 

Fet us define e, to be a 6-D coordinate vector with 
a 1 in the i-th coordinate and zeros elsewhere. If we set 
f = e,- in (3.89), then v will equal column i of A 1 . Let 
us also define the function FD(i,j, q, q, ao, r, f), which 
performs a forward-dynamics calculation and returns 
the true acceleration of link i (i. e., a, — ao), expressed 
in the same coordinates as f (typically the base coor¬ 
dinates). The arguments q, q and r set the values of 
the joint position, velocity, and force variables, while j 
and f specify that an external force of f is to be applied 
to link j. The argument ao specifies a fictitious base ac¬ 
celeration to include gravitational effects, and is set to 
either 0 or —a g . 

With these definitions, we have 

(A~ 1 )' = FD(ee, ee, q. 0,0,0, e,) (3.90) 

and 

A~ l (p + p) = —FD(ee, ee, q. q, —a g , r,0) , (3.91) 

where (A -1 )' is column i of A~ 1 , and ee is the body 
number of the end-effector. It therefore follows that the 
coefficients of (3.87) can be calculated using the algo¬ 
rithm shown in Algorithm 3.6. This algorithm is O(n). 
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Algorithm 3.6 Algorithm to compute the inverse 
of the operational-space inertia matrix and other 
terms 

for j = I to 6 do 

x' = FD(ee, ee. q, 0,0,0, e,) 

end for 

A~‘ = [v * 1 v 2 * * * v 6 ] 

A~ x (fi + p) = —FD(ee, ee, q, q, —a g , t, 0) 

The efficiency of the algorithm may be increased 
significantly when computing A~ 1 by noting that: 

(1) v,-, and x j in the ABA calculation (Algorithm 3.4) 

may be set to zero, and (2) if and the quantities that 

depend upon it ([/, and Df) need only be computed 

once, since they do not vary with the applied force. 

Also, note that the algorithm may be applied to multi¬ 


ple end-effectors by modifying FD to accept a list of 
end-effector body numbers in its first argument, and 
return a composite vector containing the accelerations 
of all the specified bodies. Algorithm 3.6 is then en¬ 
closed in a for loop that controls the second argument 
to FD, and iterates over all of the end-effector body 
numbers [3.20]. 

However, for the case of branched mechanisms with 
multiple end-effectors, several published algorithms 
achieve even better efficiency, and should be used in¬ 
stead [3.20, 29, 31-33]. The best achievable complexity 
is 0(n + md + m 2 ), where m is the number of end- 
effectors and cl is the depth of the system’s connec¬ 
tivity tree [3.33]; but the fastest algorithm for a typi¬ 
cal humanoid exploits branch-induced sparsity [3.32, 
33]. 


3.6 Kinematic Loops 

All of the algorithms in the last section were for kine¬ 
matic trees. In this section, a final algorithm is provided 
for the forward dynamics of closed-loop systems. The 
algorithm supplements the dynamic equations of mo¬ 
tion for a spanning tree of the closed-loop system with 
the loop-closure constraint equations. Three different 
methods are given to solve the resulting linear system 
of equations. An efficient algorithm is given to compute 
the loop-closure constraints. 

Systems with closed kinematic loops exhibit 
more complicated dynamics than kinematic trees. For 
example: 

1. The degree of motion freedom of a kinematic tree is 
fixed, but that of a closed-loop system can vary. 

2. The degree of instantaneous motion freedom is 
always the same as the degree of finite motion free¬ 
dom in a kinematic tree, but they can be different in 
a closed-loop system. 

3. Every force in a kinematic tree can be determined, 
but some forces in a closed-loop system can be in¬ 
determinate. This occurs whenever a closed-loop 
system is overconstrained. 

Two examples of these phenomena are shown in 
Fig. 3.9. The mechanism in Fig. 3.9a has no finite mo¬ 
tion freedom, but it has two degrees of infinitesimal 
motion freedom. The mechanism in Fig. 3.9b has one 
degree of freedom when 6 0, but if 9 = 0 then the 

two arms, A and B, are able to move independently, and 
the mechanism has two degrees of freedom. Moreover, 
at the boundary between these two motion regimes, the 
mechanism has three degrees of infinitesimal motion 
freedom. Both these mechanisms are planar, and are 


therefore overconstrained. As a result, the out-of-plane 
components of the joint constraint forces are indetermi¬ 
nate. This kind of indeterminacy has no effect on the 
motions of these mechanisms, but it does complicate 
the calculation of their dynamics. 

3.6.1 Formulation of Closed-Loop Algorithm 

A closed-loop system can be modeled as a spanning tree 
subject to a set of loop-closure constraint forces. If 

H'q + Cq+ r g = r 

is the equation of motion of the spanning tree on its 
own, then the equation of motion for the closed-loop 
system is 

Hq + Cq + r g = r + T a + r c , (3.92) 



Fig.3.9a r b Pathological closed-loop systems, (a) Example 
of varying motion freedom, (b) Example with differing fi¬ 
nite and infinitesimal motion freedoms 
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where r a and r c are vectors of loop-closure active and 
constraint forces, respectively, expressed in the gen¬ 
eralized force coordinates of the spanning tree. r a is 
a known quantity, and r c is unknown. r a comes from 
the force elements acting at the loop-closing joints 
(springs, dampers and actuators). If there are no such 
force elements, then r a = 0. 

The loop-closure constraints restrict the motion of 
the spanning tree. At the acceleration level, these con¬ 
straints can be expressed in the form of a linear equa¬ 
tion, 

Lq = 1, (3.9B) 

where L is an n c x n matrix, rf is the number of con¬ 
straints due to the loop-closing joints, and is given by 
the formula 

Nj 

rf = n\, (3.94) 

k=N B +1 

where rf k is the number of constraints imposed by 
joint k. If rank(L) < if then the loop-closure constraints 
are linearly dependent, and the closed-loop mechanism 
is overconstrained. The mobility of a closed-loop sys¬ 
tem (i. e., its degree of motion freedom) is given by the 
formula 

mobility = n — rank(L) . (3.95) 

Given a constraint equation in the form of (3.93), 
it follows that the constraint forces can be expressed in 
the form 

t c =L t X, (3.96) 

where X = ••• Xj/^ is an rf x 1 vector of 

unknown constraint-force variables (or Lagrange mul¬ 
tipliers). If the mechanism is overconstrained, then JL T 
will have a null space, and the component of X lying in 
this null space will be indeterminate. 

It is often possible to identify redundant constraints 
in advance. For example, if a kinematic loop is known 
to be planar, then the out-of-plane loop-closure con¬ 
straints are redundant. In these circumstances, it is 
advantageous to remove the corresponding rows of L 
and elements of 1 and X. The removed elements of X 
can be assigned a value of zero. 

Combining (3.92), (3.93), and (3.96) produces the 
following equation of motion for a closed-loop system 

H q j = ^r + r a -(Cq+r g )^ 

(3.97) 

The system matrix is symmetric, but indefinite. If L has 
full rank, then the system matrix will be nonsingular. 


otherwise it will be singular, and one or more elements 
of X will be indeterminate. 

Equation (3.97) can be solved in any of the follow¬ 
ing ways: 

1. Solve it directly for q and X. 

2. Solve for X first, and then use the result to solve 
for q. 

3. Solve (3.93) for q, substitute the result into (3.92), 
eliminate the unknown constraint forces, and solve 
for the remaining unknowns. 

Method 1 is the simplest, but generally also the least 
efficient. This method is appropriate when the system 
matrix is nonsingular. As the size of the system matrix 
is (11 + rf) x (n + rf), this method is 0((n + rf) 3 ). 

Method 2 is particularly useful if n» rf, and offers 
the opportunity to use 0(n) algorithms on the spanning 
tree [3.6]. From (3.97), 

LH~ x lJX = \ -LH~ 1 [r + r a — (Cq + r g )] . 

(3.98) 

This equation can be formulated in 0(n (rf) 2 ) oper¬ 
ations via 0(n) algorithms, and solved in 0((rf) 3 ). 
Once X is known, r c can be calculated via (3.96) in 
0(nrf) operations, and (3.92) solved by an 0(n) al¬ 
gorithm; so the total complexity is 0(n (rf) 2 + (rf) 3 ). 
If L is rank deficient, then LH '/ * will be singu¬ 
lar; but it is still a positive-semidefinite matrix, and 
presents a slightly easier factorization problem than 
a singular instance of the indefinite system matrix 
in (3.97). 

Method 3 is useful if n — rf is small, or if L is ex¬ 
pected to be rank deficient. Equation (3.93) is solved 
using a special version of Gaussian elimination (or sim¬ 
ilar procedure), which is equipped with a numerical 
rank test, and which is designed to solve underdeter¬ 
mined systems. The solution is an equation of the form 

q = Ky + q 0 , 

where qo is any particular solution to (3.93), K is an 
n x (n— rank(L)) matrix with the property LK = 0, andy 
is a vector of —rank(L) unknowns. (Typically, y is 
a linearly independent subset of the elements of q.) 
Substituting this expression for q into (3.92), and pre¬ 
multiplying both sides by K T to eliminate r c , produces 

K 7 HKy = K t (t + r a - (Cq + r g ) - Hq 0 ) . 

(3.99) 

This method also has cubic complexity, but it can be the 
most efficient if n — n c is small. It is also reported to be 
more stable than method 1 [3.34]. 
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3.6.2 Closed-Loop Algorithm 


where 


Algorithms for calculating H and Cq+r g can be 
found in Sect. 3.5.3 and 3.5.1, respectively, which just 
leaves L, 1 and T a . To keep things simple, we will as¬ 
sume that all loop-closing joints are zero-DOF joints. 

There is no loss of generality with this assumption: 
one simply breaks open the loops by cutting links in¬ 
stead of joints (Fig. 3.10). Flowever, there may be some 
loss of efficiency. With this assumption, we only need 
to calculate L and 1, since r a = 0. 


+ 1 


&ik — 


-1 

0 


if s{k) G c*{i ) and p(k) £ c*(i) , 
if p(k) G c*(i) and s(k) <£ c*(i) , 
otherwise. 


In other words, = +1 if joint i lies on the path to s(k) 
but not the path to p(k)\ e& = — 1 if joint i lies on the 
path to p(k ) but not the path to s(k)\ and e& = 0 if joint i 
lies on both paths or on neither. 

The loop acceleration constraint can now be written 


Loop Constraints 

In the general case, the velocity constraint equation for 
loop k is 

i'K) 1 (v.stt) — V P «)) = 0 , (3.100) 

and the acceleration constraint is 

{*kf (a,® - Mk)) + (**) (W® - Mk)) = 0 ■ 

(3.101) 

However, if every loop-closing joint has zero DOF, then 
these equations simplify to 

v,y«) - 'fp(k) = 0 (3.102) 


and 


0 — a s(k) ap® 

= JkM + JkM 

= Jk q + a^-a^), (3.106) 

where aj 1 ’ is the velocity-product acceleration of link i, 
which is the acceleration it would have if q were 
zero. The velocity-product acceleration of every link 
is calculated during the calculation of the vector Cq + 
r g (3.84). If the RNEA is used to calculate Cq+ r g , 
then aj p will be the value of a,- calculated by the RNEA 
with its acceleration argument set to zero. 

The matrices L and 1 can now be expressed as fol¬ 
lows 


L = 

^Ln b - l-t' 

and 1 = 

'^Nb +1 ^ 


V l Nj ) 


V Inj j 


(3.107) 


<ks(k) *kp(k) 0 ■ (3.103) 

Let us define a loop Jacobian, Jk, with the property 


where 

Lk = Jk 


and 


(3.108) 


v.\(k) Vp(k) Jk q - 


I V P V P 

(3.104) h — - a s(lt ) . 


(3.109) 


Jk is a 6 x n matrix defined by the formula 


Jk — (e\k@ 1 ■■■ SN B k^N B ) . (3.105) 




For dynamic equivalence, /i + h = I 

Fig. 3.10 Inserting a zero-DOF joint in preparation for cut¬ 
ting the loop open at that joint 


Constraint Stabilization 

In practice, it is necessary to stabilize loop-closure con¬ 
straints, or they will simply fly apart during simulation 
because of numerical integration errors. The standard 
technique is due to Bcmmgarte [3.3, 7, 35], and consists 
of replacing each constraint equation of the form 

fl e = 0 , 

with one of the form 

de T Ky Vq T KpP e — 0 . 

where a e , v e , and p t are the acceleration, velocity, and 
position errors, respectively, and K v and K p are posi¬ 
tive constants. Typically, one chooses a time constant, 
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4 , according to how quickly one wants the position and 
velocity errors to decay. K p and K v are then given by the 
formulae K v = 2/t c and K p = 1 /t\. However, there is 
no good rule for choosing 4 . If 4 is too long, then loop- 
constraint errors accumulate faster than they decay; if 
4 is too short, then the equations of motion become ex¬ 
cessively stiff, causing a loss of numerical integration 
accuracy. A reasonable value for a large, slow indus¬ 
trial robot is 4 = 0 . 1 , while a smaller, faster robot might 
need 4 = 0 . 01 . 

To incorporate stabilization terms into the loop con¬ 
straint equation, we replace (3.109) with 

4 = a'aj - a X) -K v (\ s (k) -Vp(k))~ K P Pe* , (3.110) 

where p e A is a vector representing the position error in 
loop k. In absolute coordinates (i. e., link 0 coordinates), 
PeA is given by 


PeA = x_to_vec(% ( A> X~' (k) X L 2 (k) sW X 0 ) , 

(3.111) 

where the X L \(k) and X L 2 (k) transforms are defined 
in (3.62) and (3.63), and shown in Fig. 3.4 for joint k, 
and x to vec ( B X A ) computes a vector approximating 
the displacement from frame A to frame B, assuming 
this displacement to be infinitesimal, x to vec can be de¬ 
fined as 


x to vec(Z) 


1 

2 


^23 — -^32^ 

X 3 i-X 13 
X12-X21 
X 53 — ^62 
Xet-Xa 
\X42-X 5l J 


(3.112) 


Algorithm 

Algorithm 3.7 shows an algorithm for calculating L 
and 1 for the special case when all the loop-closing 
joints have 0-DOF. It combines simplicity with good 
performance by transforming every quantity that is 
needed to formulate the loop-closure constraints into 
a single coordinate system, in this case absolute (link 0 ) 
coordinates, so that no further transforms are needed. 

The first loop calculates the transforms from abso¬ 
lute to link coordinates, and uses them to transform 0, 


to absolute coordinates. Only the 0, that are needed in 
the loop-closure constraints are transformed. 

The second loop calculates the nonzero elements 
of L (which can be sparse), according to (3.105). The 
inner while loop terminates on the root of the loop, 
which is the highest-numbered common ancestor of 
links p(k) and s(k). It could be the fixed base if they 
have no other common ancestor. The second loop ends 
with the calculation of 1 . in absolute coordinates, ac¬ 
cording to (3.110). 


Algorithm 3.7 Algorithm to calculate loop-closure 
constraints 

inputs: model , RNEA partial results 

outputs: L, 1 

model data : Nb, p{i), Nj, p{k), s(k), LB(i), X Li (k), 
X L2 (k),K p ,K v 

RNEA data . ^, Xp^ , (a), v v (a), 

for i = 1 to /V/j do 
if/hi) / 0 then 

% = %« p(i) X 0 

end if 

if LB(i) / null then 

°0i = ' V 0 ,- 

end if 
end for 
JL = 0 

for k = N b + 1 to Nj do 

i = p(k ) 

j = s(k) 

while ( / j do 
if i > j then 

Lk.i = —°0; 

i = p{i ) 

else 

L kJ = ° 0 j 

j=p(j ) 

end if 
end while 

„ - Hk)y-1 „ V P _p(.k)y- 1 „ V P 
a e - A 0 a jW A 0 H p(k) 

Ve = ^X, 1 V J( A) -? W X 0 - ! \p(k) 

Pe = x_to vec (r (k) X~' X~' ( k ) X L2 (k) s(k) X 0 ) 

4 = —a e — K v v e — K p p e 

end for 


3.7 Conclusions and Further Reading 

This chapter has presented the fundamentals of rigid- gebra, which provides a concise notation for describing 
body dynamics as they apply to robot mechanisms. It and implementing dynamics equations and algorithms; 
has covered the following topics: the spatial vector al- the canonical equations of motion that are most fre- 
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quently used in robotics; how to construct a dynamic 
model of a robot; and several efficient model-based 
algorithms for calculating inverse dynamics, forward 
dynamics, and the joint-space and operational-space in¬ 
ertia matrices. 

There are many topics in dynamics that have not 
been mentioned in this chapter, but can be found in later 
chapters of this handbook. The dynamics of robots with 
elastic links and joints is covered in Chap. 11 ; the prob¬ 
lem of identifying the parameters of a dynamic model 
is covered in Chap. 6; the dynamics of physical contact 
between a robot and the objects in its environment is 
described in Chap. 37; and the dynamics of robots with 
floating bases is described in Chap. 55. 

We conclude this chapter by noting that a brief his¬ 
tory of robot dynamics can be found in [3.36], and that 
a more extensive treatment of robot dynamics can be 
found in books such as [3.8, 10,15,29, 37-40]. Finally, 
some suggestions for further reading are listed below. 

3.7.1 Multibody Dynamics 

Robot dynamics can be regarded as a subset (or a spe¬ 
cific application) of the broader discipline of multibody 
dynamics. Books on multibody dynamics include [3.3, 
14,35,41-46]. Of course, multibody dynamics is, in 
turn, a subset of classical mechanics; and the mathe¬ 
matical foundations of the subject can be found in any 
good book on classical mechanics, such as [3.13]. 

3.7.2 Alternative Representations 

This chapter has used spatial vectors to express the 
equations of motion. There are various alternatives to 
the use of spatial vectors: other kinds of 6-D vector, 
3-D vectors, 4x4 matrices, and the spatial operator al¬ 
gebra. All 6-D vector formalisms are similar, but are 
not exactly the same. The main alternatives to spatial 
vectors are: screws [3.10-12], motors [3.47], Lie al¬ 
gebras [3.12,48], and ad hoc notations. (An ad hoc 
notation is one in which 3-D vectors are grouped into 
pairs for the purpose of reducing the volume of alge¬ 
bra.) Three-dimensional vectors are the formalism used 
in most classical mechanics and multibody texts, and 
are also a precursor to 6-D vector and 4x4 matrix for¬ 
malisms. 4x4 matrices are popular in robotics because 
they are very useful for kinematics. However, they are 
not so useful for dynamics. 4x4 matrix formulations 
of dynamics can be found in [3.37,49,50]. The spatial 
operator algebra was developed at the Jet Propulsion 
Laboratory (JPL) by Rodriguez, Jain, and others. It uses 
6 A-dimensional vectors and 6 N x 6 N matrices, the latter 
regarded as linear operators. Examples of this notation 
can be found in [3.38, 51-53]. 


3.7.3 Alternative Formulations 

This chapter used a vectorial formulation of the equa¬ 
tions of motion that is usually called the Newtonian or 
Newton-Euler formulation. The main alternative is the 
Lagrangian formulation, in which the equations of mo¬ 
tion are obtained via Lagrange’s equation. Examples 
of the Lagrangian formulation can be found in [3.9, 
10,18,54, 55]. Kane's method has also been applied in 
robotics [3.56,57], 

3.7.4 Efficiency 

Because of the need for real-time implementation, es¬ 
pecially in control, the robotics community has focused 
on the problem of computational efficiency. For inverse 
dynamics, the O(n) recursive Newton-Euler algorithm 
(RNEA) of Luh et al. [3.4] remains the most impor¬ 
tant algorithm. Further improvements to the algorithm 
are given in [3.58,59]. For forward dynamics, the two 
algorithms presented in this chapter remain the most 
important for computational considerations: the 0 (n) 
articulated-body algorithm (ABA) developed by Feath- 
erstone [3.1] and the 0(rc 3 ) algorithm based on the 
composite-rigid-body algorithm (CRBA) of Walker and 
Orin [3.5]. Improvements were made in the ABA over 
the years [3.15, 17, 25] so that it was more efficient than 
the CRBA-based algorithm for decreasingly smaller 
values of n. However, more recent application of the 
CRBA to branched kinematic trees [3.26] and robotic 
systems with motion-controlled appendages [3.60] con¬ 
tinue to show the viability of the CRBA approach. 

For the joint-space inertia matrix, the CRBA [3.5] 
is the most important algorithm. A number of improve¬ 
ments and modifications have been made over the years 
to increase its computational efficiency [3.15,61-63]. 
For the operational-space inertia matrix, efficient 0(n) 
algorithms have been developed [3.28-30] and applied 
to increasingly complex systems [3.20, 31, 33]. The ex¬ 
ploitation of branch-induced sparsity also results in an 
efficient algorithm [3.32]. 

3.7.5 Accuracy 

Concerns can arise over the numerical accuracy of a dy¬ 
namics algorithm, the accuracy of a simulation (i. e., 
numerical integration accuracy), or the accuracy of 
a dynamic model. The numerical accuracy of dynamics 
algorithms has received relatively little attention com¬ 
pared with efficiency. The RNEA, CRBA, and ABA 
have all been tested for accuracy on a large variety 
of rigid-body systems, but the same cannot be said 
of most other algorithms. Rigid-body systems are of¬ 
ten ill-conditioned, in the sense that a small change 
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in the applied force (or a model parameter) can pro¬ 
duce a large change in the resulting acceleration. This 
phenomenon was studied by Featherstone [3.27], who 
discovered that the ill-conditioning gets worse with in¬ 
creasing body count, and that it can grow in proportion 
to 0(n 4 ) in the worst case. Other publications on this 
topic include [3.8, 34,64, 65]. 

3.7.6 Software Packages 

A number of software packages have been devel¬ 
oped to provide dynamic simulation capabilities for 
multibody systems, and in particular, robotic systems. 
Several have been written in MATLAB for ease of 
integration with other analysis, control, and simula¬ 
tion programs. Many packages are open source, and 
some are offered at a relatively low cost to the user. 
They differ in their capabilities in a variety of ways 
including: speed, topologies and joint models sup¬ 
ported, accuracy, underlying dynamic formulation and 
associated order of complexity, user interface, graph¬ 
ics support, numerical integration routines, integra¬ 
tion with other code, application support, and cost. 
Among those commonly cited are: Adams [3.66], 
Autolev [3.67], Bullet [3.68], DART [3.69], Dy- 
naMechs [3.70], Gazebo [3.71], Open Dynamics En¬ 
gine [3.72], Robotics Studio [3.73], Robotics Tool¬ 
box [3.74], Robotran [3.75,76], SD/FAST [3.77], Sim- 
body [3.78], SimMechanics [3.79], SYMORO [3.80, 
81] and Webots [3.82], 

3.7.7 Symbolic Simplification 

The technique of symbolic simplification takes a gen¬ 
eral-purpose dynamics algorithm, and applies it sym¬ 
bolically to a specific dynamic model. The result is a list 
of assignment statements detailing what the algorithm 
would have done if it had been executed for real. This 
list is then inspected and pruned of all unnecessary cal¬ 
culations, and the remainder is output to a text file in 
the form of computer source code. This code can run 
as much as ten times faster than the original general- 
purpose algorithm, but it is specific to one dynamic 
model. Both Autolev [3.67] and SD/FAST [3.77] use 


this technique. Other publications on symbolic simpli¬ 
fication for dynamics include [3.76, 80, 81, 83-88]. 

3.7.8 Algorithms for Parallel Computers 

In order to speed up the common dynamics compu¬ 
tations, a number of algorithms have been developed 
for parallel and pipelined computers. For inverse dy¬ 
namics, early work focused on speeding up the O(n) 
RNEA on up to n processors [3.89,90] while sub¬ 
sequent work resulted in 0(log 2 n) algorithms [3.91, 
92]. For the 0(n 2 ) CRBA to compute the joint-space 
inertia matrix, early work resulted in 0(log 2 n) al¬ 
gorithms for n processors to compute the composite- 
rigid-body inertias and diagonal elements of the ma¬ 
trix [3.93,94]. Subsequent work resulted in 0( log 2 ft) 
algorithms for 0 (n 2 ) processors to compute the entire 
matrix [3.95,96]. For forward dynamics, speedup was 
obtained for a multiple manipulator system on a paral¬ 
lel/pipelined supercomputer [3.97]. The first 0(log 2 n) 
algorithm for n processors was developed for an un¬ 
branched serial chain [3.98]. More recent work has 
focused on 0 { log 2 n) algorithms for more complex 
structures [3.65,99, 100]. 

3.7.9 Topologically-Varying Systems 

There are many robot mechanisms whose topology 
varies over time because of a change of contact con¬ 
ditions, especially with the environment. In legged 
vehicles, use of a compliant ground-contact model 
to compute the contact forces reduced the closed- 
loop structure to a tree structure [3.101]. However, 
for cases in which the contacts are very stiff, numer¬ 
ical integration problems may result. In more recent 
work [3.40,102] in which hard contact constraints 
are assumed, an efficient method was used to reduce 
the large number of coordinate variables from that 
which may be necessary in general-purpose motion 
analysis systems [3.43], Also, they were able to auto¬ 
matically identify the variables as the structure varied 
and developed a method for computing the velocity 
boundary conditions after configuration changes [3.40, 
102 ], 
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This chapter focuses on the principles that guide 
the design and construction of robotic systems. The 
kinematics equations and Jacobian of the robot 
characterize its range of motion and mechanical 
advantage, and guide the selection of its size and 
joint arrangement. The tasks a robot is to per¬ 
form and the associated precision of its movement 
determine detailed features such as mechanical 
structure, transmission, and actuator selection. 
Here we discuss in detail both the mathematical 
tools and practical considerations that guide the 
design of mechanisms and actuation for a robot 
system. 

The following sections (Sect. 4.1) discuss char¬ 
acteristics of the mechanisms and actuation that 
affect the performance of a robot. Sections 4.2-4. 6 
discuss the basic features of a robot manipula¬ 
tor and their relationship to the mathematical 
model that is used to characterize its performance. 
Sections 4.7 and 4.8 focus on the details of the 
structure and actuation of the robot and how they 
combine to yield various types of robots. The final 
Sect. 4.9 relates these design features to various 
performance metrics. 
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Part A 


Robotics Foundations 


4.1 Overview 

The physical structure such as the beams, links, cast¬ 
ings, shafts, slides, and bearings of a robot that create 
its movable skeleton is termed the mechanical struc¬ 
ture or mechanism of the robot. The electric, hydraulic 
and pneumatic motors and other elements that cause the 
links of the mechanism to move are called actuators. In 
this chapter we consider the variety of designs for the 
mechanisms and actuators that result in a machine sys¬ 
tem that transforms computer commands into versatile 
physical movement. 

Early robots were designed with general motion ca¬ 
pability under the assumption that they would find the 
largest market if they could perform the widest variety 
of tasks; this emphasis on flexibility proved to be ex¬ 
pensive in both cost and performance. Robots are now 
often designed for specific applications and to perform 
limited sets of tasks. 

Robot design focuses on the number of joints, 
physical size, payload capacity, and the movement re¬ 


4.2 System Features 

The primary features that characterize a robot are its 
work envelope and load capacity. 

4.2.1 Work Envelope 

The space in which a robot can operate is its work 
envelope, which encloses its workspace. While the 
workspace of the robot defines positions and orienta¬ 
tions that it can achieve to accomplish a task, the work 
envelope also includes the volume of space the robot it¬ 
self occupies as it moves. This envelope is defined by 
the types of joints, their range of movement and the 
lengths of the links that connect them. The physical size 
of this envelope and the loads on the robot within this 
envelope are of primary consideration in the design of 
the mechanical structure of a robot. 

Robot work envelope layouts must include consid¬ 
erations of regions of limited accessibility where the 
mechanical structure may experience movement limi¬ 
tations. These constraints arise from limited joint travel 
range, link lengths, the angles between axes, or a com¬ 
bination of these. Revolute joint manipulators generally 
work better in the middle of their work envelopes than 
at extremes (Fig. 4.1). Manipulator link lengths and 
joint travel should be chosen to leave margins for vari¬ 
able sensor-guided controlled path motions and for tool 
or end-effector changes, as offsets and length differ¬ 
ences will often alter the work envelope. 


quirements of the end-effector. The configuration of the 
movable skeleton and the overall size of the robot are 
determined by task requirements for reach, workspace, 
and reorientation ability. These features affect the preci¬ 
sion of end-effector path control needed for applications 
such as for arc-welding and for the smooth movement 
of paint spraying. They also define the absolute posi¬ 
tioning capability necessary for small part assembly, the 
repeatability needed for material and package handling, 
and the fine resolution that allows precise, real-time 
sensor-based motions. 

A critical concern in robotic system design is the 
range of tasks the robot is expected to perform. The 
robot should be designed to have the flexibility it needs 
to perform the range of tasks for which it is intended. 
This determines the topology of the robot mechanism 
and the actuator system. The choices of geometry, mate¬ 
rial, sensors, and cable routing follow from these basic 
decisions. 


4.2.2 Load Capacity 

Load capacity, a primary robot specification, is closely 
coupled with acceleration and speed. For assembly 
robots, mechanism acceleration and stiffness (structure 
and drive stiffness) are often more important design pa¬ 
rameters than peak velocity or maximum load capacity, 
as minimizing pick-and-place motion cycle time, while 
maintaining placement precision, is generally a top pri¬ 
ority in small part assembly. In the case of arc-welding, 
where slow controlled-path motion is required, velocity 



Fig. 4.1 The PUMA 560 robot 
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jitter and weld path-following accuracy are important. 
Load capacity should be seen as a variable. It is wise 
to design and specify a manipulator in terms of useful 
payload as a function of performance rather than just in 
terms of maximum capacity. 

Load capacity specifications must take into account 
gravity and inertial loading seen at the end-effector. 
These factors strongly affect wrist, end-effector design 
and drive selection. In general, load capacity is more 
a function of manipulator acceleration and wrist torque 
than any other factor. The load rating also affects ma¬ 
nipulator static structural deflection, steady-state motor 
torque, system natural frequency, damping, and the 
choice of control system parameters for best perfor¬ 
mance and stability. 

4.2.3 Kinematic Skeleton 

Manipulator shape and size is determined by require¬ 
ments on its workspace shape and layout, the precision 
of its movement, its acceleration and speed, and its 
construction. Cartesian manipulators have the simplest 
transform and control equation solutions. Their pris¬ 
matic (straight-line motion) and orthogonal (perpendic¬ 
ular) axes make motion planning and computation rela¬ 
tively straightforward. Because their major motion axes 
do not couple dynamically (to a first order), their con¬ 
trol equations are also simplified. Manipulators with all 
revolute (rotary) joints are generally harder to control, 
but they feature a more compact and efficient structure 
for a given working volume. It is generally easier to de- 

4.3 Kinematics and Kinetics 

The dynamics of a robot can be separated into the 
properties of the movement that depend upon the ge¬ 
ometry of its mechanical structure, termed kinematics, 
and those that depend on forces that act on the system, 
known as kinetics. It is a law of dynamics that the differ¬ 
ence between the change in energy of the moving robot 
and the work performed by the forces acting on it does 
not change over small variations of its trajectory. This is 
called the principle of virtual work and states that vari¬ 
ations in work and energy must cancel for all virtual 
displacements [4.1,2]. 

Because machines such as robots are designed to 
minimize energy losses, often due to joint friction and 
material strain losses, we can assume that the variation 
in energy is small. This means that the work input of 
the actuators is nearly equal to the work of the output 
forces. 

If we consider this relationship over a small dura¬ 
tion of time, we have that the time rate of input work. 


sign and build a good revolute joint than a long motion 
prismatic joint. The workspaces of revolute joint manip¬ 
ulators can easily overlap for coordinated multi-robot 
installations, in contrast to the more exclusive useful 
workspace of gantry style robots. 

Final selection of the robot configuration should 
capitalize on specific kinematic, structural or perfor¬ 
mance requirements. For example, a requirement for 
a very precise vertical straight-line motion may dictate 
the choice of a simple prismatic vertical axis rather than 
two or three revolute joints requiring coordinated con¬ 
trol. 

Six degrees of freedom (DOFs) are the minimum 
required to place the end-effector or tool of a robotic 
manipulator at any arbitrary location (position and ori¬ 
entation) within its accessible workspace. Most simple 
or preplanned tasks can be performed with fewer than 
six DOFs by careful task setup to eliminate certain axis 
motions, or because the tool or task does not require full 
specification of location. An example of this is vertical 
assembly using a powered screwdriver, where all oper¬ 
ations can be achieved with three degrees of freedom. 

Some applications require the use of manipulators 
with more than six DOFs, in particular when mobil¬ 
ity or obstacle avoidance are necessary. For example, 
a pipe-crawling maintenance robot requires control of 
the robot shape as well as precise positioning of its 
end-effector. Generally, adding degrees of freedom in¬ 
creases cycle time and reduces load capacity and ac¬ 
curacy for a given manipulator configuration and drive 
system. 


or input power is nearly equal to the associated output 
power. Because power is force times velocity, we ob¬ 
tain the fundamental relationship that the ratio of input 
to output forces is the reciprocal of the ratio of input to 
output speeds. Another way of saying this is that, in the 
ideal machine, the mechanical advantage of a machine 
is the inverse of its speed ratio. 

4.3.1 Robot Topology 

The kinematic skeleton of a robot is modeled as a se¬ 
ries of links connected by either hinged or sliding 
joints forming a serial chain. This skeleton has two ba¬ 
sic forms, that of a single serial chain called a serial 
robot (Fig. 4.1) and as a set of serial chains sup¬ 
porting a single end-effector, called a parallel robot, 
such as the platform shown in Fig. 4.2. Robots can be 
configured to work in parallel such as the individual 
legs of walking machines (Figs. 4.3 and 4.4) [4.3], as 
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Platform 


Fig. 4.2 A parallel robot can have as many as six se¬ 
rial chains that connect a platform to the base frame 

ri<m agn»rcirEia ) 


Fig. 4.3 A photograph of the adaptive suspension vehicle 
(ASV)walking machine 


Fig. 4.4 The adaptive suspension vehicle walking ma¬ 
chine 

well as the fingers of mechanical hands (Figs. 4.5^1.7 
and lo-MEHiHa) [4.4], 

The robot end-effector is the preferred tool for inter¬ 
action with the environment, and the ability to position 
and orient this end-effector is defined by the skeleton of 


y 

Fig. 4.6 The Salisbury hand as the end-effector of 
a PUMA robot (the drive system is not shown) 


Fig. 4.7 A photograph of the Salisbury three-fingered 
hand grasping a block 

the robot. In a general serial robot, a chain of six joints 
provides full control over the end-effector. In a general 
parallel robot, there are more than six joints and the six 
actuators may be applied to these joints in a variety of 
ways to control the movement of the end-effector. 


Fig. 4.5 The Salisbury three-fingered robot hand with its 
cable drive system 
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Fig. 4.8 The kinematic skeleton defined by the join axes 
and their common normals 

4.3.2 Kinematics Equations 

A robot is designed so that specifying the values of lo¬ 
cal joint parameters, such as the angle of rotary joints 
and the travel of sliding joints, specifies the position 
of every component of a machine using its kinematics 
equations. To do this the robot is described by a se¬ 
quence of lines representing the axes Zj of equivalent 
revolute or prismatic joints and the common normal 
lines Xj which form the kinematic skeleton of the chain 
(Fig. 4.8). This construction allows the specification of 
the location of each link of the robot relative to the base 
by the matrix equation 

T = Z(e l ,di)X(a l ,ai)Z{9 2 , d 2 ) ... 

xX(a,„— 1 , ci m — i)Z(0 m ,d m ) , (4.1) 

known as the kinematics equations of the chain [4.5, 6] 
(Chap. 2; (2.46)). The set of all positions T that the end- 
effector can reach for all values of the joint parameters 
is called the workspace of the robot. 

The matrices Z(6j, dj) and X(aj, a,) are 4 x 4 matri¬ 
ces that define screw displacements around and along 
the joint axes zj and xj, respectively [4.7]. The parame¬ 
ters a,j and aj define the dimensions of the links in the 
chain. The parameter Qj is the joint variable for revo¬ 
lute joints and dj is the variable for prismatic joints. The 
trajectory F p(t) of a point M p in the end-effector is ob¬ 
tained from the joint trajectory, 

q(t) = ( qi (t),...,q,„(t)) J , 


If the end-effector is connected to the base frame by 
more than one serial chain (Fig. 4.2) then we have a set 
of kinematics equations for each chain, 

T = B,T(^)E„ j=l,...,n, (4.3) 

where B ; locates the base of the /-th chain and F, defines 
the position of its attachment to the part, or end-effector. 
The set of positions T that simultaneously satisfy all 
of these equations is the workspace of the part. This 
imposes constraints on the joint variables that must be 
determined to define its workspace completely [4.8, 
9]. 

4.3.3 Configuration Space 

The kinematics equations of the robot relate the range 
of values available for the joint parameters, called the 
configuration space of the robot, to the workspace of 
the end-effector. This configuration space is a funda¬ 
mental tool in robot path planning for obstacle avoid¬ 
ance [4.10]. Though any link in the chain forming 
a robot may hit an obstacle, it is the end-effector that is 
intended to approach and move around obstacles such 
as the table supporting the robot and the fixtures for 
parts it is to pick up. Obstacles define forbidden posi¬ 
tions and orientations in the workspace which map back 
to forbidden joint angles in the configuration space of 
the robot. Robot path planners seek trajectories to a goal 
position through the free space around these joint space 
obstacles [4.11]. 

4.3.4 Speed Ratios 

The speed ratios of a robot relate the velocity F p of 
a point F p in the end-effector to the joint rates 

q = (q\ .<?m) T , 

that is 

F p = v +(o x ( F p — d) , (4.4) 

where d and v are the position and velocity of a refer¬ 
ence point, respectively, and co is the angular velocity 
of the end-effector. 

The vectors v and oj depend on the joint rates qj 
through the formula 


where q , is either or dj depending on the joint, given 
by 

F p(t) = T (q(t)fp . (4.2) 


dv 

dv 

dv \ 

( q x \ 

dqi 

dqi 

dqw \ 

da) 

da) 

da> 1 


dqi 

dqi 

dq„,/ 

\hm) 


(4.5) 
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or 


v = iq . 


, dv , F d(t> 

= f ^ + \P~ d ) x f ^' J =l .' 

o qj dqj 


(4.6) 


(4.8) 


The coefficient matrix J in this equation is called the 
Jacobian and is a matrix of speed ratios relating the ve¬ 
locity of the tool to the input joint rotation rates [4.6, 

9]. 


We have arranged this equation to introduce the force- 
torque vector 

f=(fA F p~d)xf) T 


4.3.5 Mechanical Advantage 


If the end-effector of the robot exerts a force / at the 
point F p, then the power output is 


/•'/> 1> 


jt r L34 


dv do) , F , 




(4.7) 


at the reference point d. 

The equations (4.8) can be assembled into the ma¬ 
trix equation 

T = J T f , (4.9) 

where J is the Jacobian defined above in (4.5). For 
a chain with six joints this equation can be solved for 
the output force-torque vector/, 


Each term in this sum is the portion of the output power 
that can be associated with an actuator at joint Sj, if one 
exists. 

The power input at joint Sj is the product iof the 
torque r ; and joint angular velocity Using the princi¬ 
ple of virtual work for each joint we can compute 


/=(J T ) ' r . (4.10) 

Thus, the matrix that defines the mechanical advantage 
for this system is the inverse of the matrix of speed 
ratios. 


4.4 Serial Robots 

A serial chain robot is a sequence of links and joints 
that begins at a base and ends with an end-effector 
(Fig. 4.9). The links and joints of a robot are of¬ 
ten configured to provide separate translation and 
orientation structures. Usually, the first three joints 
are used to position a reference point in space and 
the last three form the wrist which orients the end- 
effector around this point [4.12,13]. This reference 
point is called the wrist center. The volume of space 
in which the wrist center can be placed is called the 
reachable workspace of the robot. The rotations avail- 



Fig. 4.9 A single finger of the Salisbury hand is a serial 
chain robot 


able at each of these points is called the dexterous 
workspace. 

The design of a robot is often based on the symme¬ 
try of its reachable workspace. From this point of view 
there are three basic shapes: rectangular, cylindrical, and 
spherical [4. 6]. A rectangular workspace is provided by 
three mutually perpendicular prismatic (P) joints which 
form a PPPS chain called a Cartesian robot - S de¬ 
notes a spherical wrist which allows all rotations about 
its center point. A rotary base joint combined with two 
prismatic joints forms a CPS chain with a cylindrical 
workspace - C denotes a rotary (R) and sliding (P) joint 
with the same axis. The P-joint can be replaced by a rev¬ 
olute (R) joint that acts as an elbow in order to provide 
the same radial movement. Finally, two rotary joints at 
right angles form a T-joint at the base of the robot that 
supports rotations about a vertical and horizontal axes. 
Radial movement is provided either by a P-joint, or by 
an R-joint configured as an elbow. The result is a TPS 
or TRS chain with a spherical workspace. 

It is rare that the workspace is completely symmet¬ 
rical because joint axes are often offset to avoid link 
collisions and there are limits to joint travel which com¬ 
bine to distort the shape of the workspace. 
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4.4.1 Design Optimization 

Another approach to robot design uses a direct speci¬ 
fication of the workspace as a set of positions for the 
end-effector of a robotic system [4.14-17] which we 
call the task space. A general serial robot arm has two 
design parameters, link offset and twist, for each of five 
links combined with four parameters each that locate 
the base of the robot and the workpiece in its end- 
effector, making a total of 18 design variables. The 
link parameters are often specified so the chain has 
a spherical wrist and specific workspace shape. The de¬ 
sign goal is usually to determine the workspace volume 
and locate the base and workpiece frames so that the 
workspace encloses the specified task space. 

The task space is defined by a set of 4 x 4 trans¬ 
formations D, , i = l,... ,k. The problem is solved iter¬ 
atively by selecting a design and using the associated 
kinematics equations T(<jr) to evaluate relative displace¬ 
ments in the objective function 

k 

/M = £ ll D '- T — : ‘fa)| • (4.11) 

;=i 

Optimization techniques yield the design parameter 
vector r that minimizes this objective function. 

This optimization depends on the definition of the 
distance measure between the positions reached by the 
end-effector and the desired workspace. Park [4.18], 
Martinez and Duffy [4.19], Zefran et al. [4.20], Lin and 
Burdick [4.21], and others have shown that there is no 
distance metric that is coordinate frame invariant. This 
means that, unless this objective function can be forced 
to zero so the workspace completely contains the task 
space, the resulting design will not be geometric in the 
sense that the design is not independent of the choice of 
coordinates. 

4.4.2 Speed Ratios 

A six-axis robot has a 6x6 Jacobian J obtained 
from (4.5) that is an array of speed ratios relating the 


4.5 Parallel Robots 

A robotic system in which two or more serial chain 
robots support an end-effector is called a parallel robot. 
For example, the adaptive suspension vehicle (ASV) 
leg (Fig. 4.10), is a pantograph mechanism driven by 
parallel actuation. Each supporting chain of a paral¬ 
lel robot may have as manyas six degrees of freedom, 
however, in general only a total of six joints in the en¬ 


components of the velocity v of the wrist center and 
the angular velocity a> of the end-effector to each of the 
joint velocities. Equation (4.9) shows that this Jacobian 
defines the force-torque vector / exerted at the wrist 
center in terms of the torque applied by each of the ac¬ 
tuators. The link parameters of the robot can be selected 
to provide a Jacobian J with specific properties. 

The sum of the squares of the actuator torques of 
a robot is often used as a measure of effort [4.22,23]. 
From (4.9) we have 

t t t=/ t JJ t /. (4.12) 

The matrix JJ T is square and positive definite. There¬ 
fore, it can be viewed as defining a hyperellipsoid in 
six-dimensional space [4.24]. The lengths of the semi¬ 
diameters of this ellipsoid are the inverse of the absolute 
value of the eigenvalues of the Jacobian J. These eigen¬ 
values may be viewed as modal speed ratios that define 
the amplification associated with each joint velocity. 
Their reciprocals are the associated modal mechanical 
advantages, so the shape of this ellipsoid illustrates the 
force amplification properties of the robot. 

The ratio of the largest of these eigenvalues to the 
smallest, called the condition number, gives a measure 
of the anisotropy or out-of-roundness of the ellipsoid. 
A sphere has a condition number of one and is termed 
isotropic. When the end-effector of a robot is in a posi¬ 
tion with an isotropic Jacobian there is no amplification 
of the speed ratios or mechanical advantage. This is 
considered to provide high-fidelity coupling between 
the input and output because errors are not ampli¬ 
fied [4.25,26]. Thus, the condition number is used as 
a criterion in a robot design [4.27], 

In this case, it is assumed that the basic design 
of the robot provides a workspace that includes the 
task space. Parameter optimization finds the internal 
link parameters that yield the desired properties for 
the Jacobian. As in minimizing the distance to a de¬ 
sired workspace, optimization based on the Jacobian 
depends on a careful formulation to avoid coordinate 
dependency. 


tire system are actuated. A good example is the Stewart 
platform formed from six TPS robots in which usually 
only the prismatic joint (P-joint) in each chain is actu¬ 
ated (Fig. 4.2) [4.9,28,29]. 

The kinematics equations of the TPS legs are 

T = B / T(0 / )E,., j = l,... ,6 , 


(4.13) 
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where By locates the base of the leg and E ; defines the 
position of its attachment to the end-effector. The set 
of positions T that simultaneously satisfy all of these 
equations is the workspace of the parallel robot. 

Often the workspace of an individual chain of a par¬ 
allel robot can be defined by geometric constraints, for 
example, a position T is in the workspace of the y-th 
supporting TPS chain if it satisfies the constraint equa¬ 
tion 

(T xj -pj) • (T xj -pj) = pf . (4.14) 

This equation defines the distance between the base 
joint pj and the point of attachment F Xj = T.r ; to the 
platform as the length p ; - is controlled by the actuated 
prismatic joint. In this case the workspace is the set of 
positions T that satisfy all six equations, one for each 
leg. 

4.5.1 Workspace 

The workspace of a parallel robot is the intersection 
of the workspaces of the individual supporting chains. 
However, it is not the intersection of the reachable and 
dexterous workspaces separately. These workspaces are 
intimately combined in parallel robots. The dexterous 
workspace is usually largest near the center of the 
reachable workspace and shrinks as the reference point 
moves toward the edge. A focus on the symmetry of 



Fig. 4.10 One leg of the ASV walking machine is a paral¬ 
lel robot 


movement allowed by supporting leg designs has been 
an important design tool resulting in many novel paral¬ 
lel designs [4.30,31]. Simulation of the system is used 
to evaluate its workspace in terms of design parameters. 

Another approach is to specify directly the posi¬ 
tions and orientations that are to lie in the workspace 
and solve the algebraic equations that define the leg 
constraints to determine the design parameters [4.32, 
33], This is known as kinematic synthesis and yields 
parallel robots that are asymmetric but have specified 
reachable and dexterous workspaces, see McCarthy and 
Soh [4.34], 

4.5.2 Mechanical Advantage 

The force amplification properties of a parallel robot are 
obtained by considering the Jacobians of the individual 
supporting chains. Let the linear and angular velocity of 
the platform be defined by the six-vector v = ( v , &>)', 
then from the kinematics equations of each of the sup¬ 
port legs we have 

v = Ji/i), = J2P2 = ••• = JePe • ( 4 . 15 ) 

Here we assume that the platform is supported by six 
chains, but it can be fewer, such as when the fingers of 
a mechanical hand grasp an object [4.4]. 

The force on the platform applied by each chain is 
obtained from the principle of virtual work as 

fj = (J , T ) _1 Tj, j = 1,.... 6 . (4.16) 

There are only six actuated joints in the system so we 
assemble the associated joint torques into the vector 
r = (n,.... T6) T . If /,- is the force-torque vector ob¬ 
tained from (4.16) for r; = 1 and the remaining torques 
to zero, then the resultant force-torque w applied to the 
platform is 

w= (fxji,--- ,f6)r , (4.17) 

or 

w = Tt. (4.18) 

The elements of the coefficient matrix T define the 
mechanical advantage for each of the actuated joints. In 
the case of a Stewart platform the columns of this ma¬ 
trix are the Pliicker coordinates of the lines along each 
leg [4.29], 

The principle of virtual work yields the velocity of 
the platform in terms of the joints rates p as 


r T v = p. 


(4.19) 
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Thus, the inverse of T defines the speed ratios between 
the actuated joints and the end-effector. The same equa¬ 
tion can be obtained by computing the derivative of the 
geometric constraint equations (4.14), and T is the Ja¬ 
cobian of the parallel robot system [4.35]. 

The Jacobian T is used in parameter optimiza¬ 
tion algorithms to design parallel robots [4.36] with 
isotropic mechanical advantage. The square root of 
the determinant | T T T | measures the six-dimensional 
volume spanned by the column vectors f r The dis¬ 
tribution of the percentage of this volume compared 
to its maximum within the workspace is also used as 
a measure of the overall performance [4.37,38]. A sim¬ 
ilar performance measure normalizes this Jacobian by 

4.6 Mechanical Structure 

For the purposes of dynamic modeling, the links of 
a robot are generally considered to be rigid. However, 
a robot is not a rigid structure. Like all structures it de¬ 
flects under applied loads, such as its own weight and 
the weight of the payload, termed gravity loading; see 
Figs. 4.11 and 4.12. The issue is a matter of degree. 
The more force that is needed to cause a deflection in 
the links, the more the robot moves like a connected set 
of rigid bodies. Rigid robots have links designed to be 
stiff so the deflections under load are less than the pos¬ 
itioning accuracy required for their range of tasks. This 
allows the dynamic model and control algorithms to ig¬ 
nore link deflection. Most commercially available robot 
arms are of this type (Rivin [4.44]). 

It is possible to improve the positioning accuracy 
of a rigid robot by augmenting a control algorithm that 
includes a model of link deflection resulting from grav¬ 
ity loading. It is also possible to use strain sensors to 
measure loads and deflections. These semirigid robots 
assume small structural deflections that are linearly re¬ 
lated to known applied loads. 

Flexible robots require that the dynamic model in¬ 
clude the deflection of its various links under gravity 
loading as well as under the forces associated with link 
acceleration, called inertia loading. The robot control 
algorithms must control the vibration of the system as 
well its gross motion. Management of vibration may 
be required even in rigid robots to reduce cycle time 
when subjected to high dynamic loading at high speeds 
or when manipulating large payloads. 

4.6.1 Links 

For industrial robots a critical concern is the link stiff¬ 
ness in bending and in torsion. To provide this stiffness, 
robot links are designed either as beams or shell (mono- 


the maximum joint torques available and the maximum 
component of force and torque desired, and then seeks 
an isotropic design [4.39]. 


Another approach to the design of parallel robots has 
been to separate their functionality into orientation and 
translational platforms. Tsai and Joshi [4.40] and Jin 
and Yang [4.41] survey designs for a class of par¬ 
allel chains that generate pure translation. Kong and 
Gosselin [4.42] and Hess-Coelho [4.43] do the same 
for parallel chains that provide rotational movement in 
space. 


coque) structures. Monocoque structures have lower 
weight or higher strength-to-weight ratios, but are more 
costly and generally more difficult to manufacture. 
Cast, extruded, or machined beam-based links are of¬ 
ten more cost effective; Juvinall and Marshek [4.45], 
and Shigley and Mischke [4.46,47] 

Another important consideration is whether the link 
structure includes bolted, welded, or adhesive bonded 
assemblies of cast, machined, and fabricated elements. 
Screw and bolted connections may seem straightfor¬ 
ward, inexpensive, and easily maintained, but the in¬ 
evitable deflection of a link even in the manufacturing 
process introduces creep in these multiple element as¬ 
semblies that changes the dimensions and performance 
of the robot. Welded and cast structures are much less 
susceptible to creep and the associated hysteresis defor¬ 
mation, though in many cases they require secondary 
manufacturing operations such as thermal stress reliev¬ 
ing and finish machining. 

The minimum practical wall or web thickness for 
castings may be thicker than necessary for stiffness. 
Thin walls can be achieved with structural skin (mono- 
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Fig. 4.11 The hydraulic Skywash aircraft cleaning robot 


4.5.3 Specialized Parallel Robots 
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Fig. 4.12 DeLaval Cow Milking System features hy¬ 
draulic robot with machine vision guided positioning 

(l^MMi) 


coque) structures but this is offset by the potential for 
denting, permanent deformation, and damage in the 
event of slight collisions. Therefore, both the perfor¬ 
mance and application requirements must be considered 
when selecting the construction and fabrication details 
of the robot. 


4.7 Joint Mechanisms 

A robot joint mechanism consists of at least four ma¬ 
jor components: the joint axis structure, an actuator, 
transmission, and state sensor (usually for position and 
velocity feedback, but force sensors are also com¬ 
mon). 

For low-performance manipulators that accelerate 
the payload at less than a peak of 0.5 g, system inertia 
is not as important as gravity forces and torques. This 
means the actuators can be placed near the joints, and 
their suspended weight compensated by using counter¬ 
balancing masses, springs, or gas pressure. 

In high-performance robots where peak payload ac¬ 
celerations reach 3—10 g or more, minimizing system 
inertia is important. The actuators are placed near the 
first joint axis of a serial link manipulator to minimize 
its inertial contribution, and drive linkages, belts, cables 
or spaced gear stages are used to drive the joints. 

While a longer transmission distance can reduce 
mass and gravity moments and inertia, it introduces 


Performance- and application-specific materials and 
geometry are used to reduce the weight of the links 
and therefore the associated gravity and inertial loading. 
For structures that move in a straight line, aluminum 
or magnesium alloy extrusions of constant cross sec¬ 
tion are convenient. Carbon and glass-fiber composites 
provide lower mass for robots that require high ac¬ 
celeration (painting robots). Thermoplastic materials 
provide low-cost link structures though at reduced load 
capacity. Stainless steel is often used in robots for med¬ 
ical and food service applications. The longer links on 
serial chain robots often are designed to taper in cross 
section or wall thickness to reduce the associated iner¬ 
tial loading. 

4.6.2 Joints 

Joints for most robots allow either rotary or linear 
movement, termed revolute and prismatic joints. Other 
joints that are available are the ball-in-socket, or spher¬ 
ical joint, and the Hooke-type universal joint. 

Integration of the mechanical structure of the robot 
with its joint mechanism, which includes the actuator 
and joint motion sensor, is a source of structural flexibil¬ 
ity. Precision is reduced when deformation in the joint 
at its bearings can reduce gear and shaft preloads, al¬ 
lowing undesirable backlash and free play. Structural 
flexibility can also introduce changes in gear center 
spacing, introducing forces and torques and associated 
deflection, binding jamming, and wear. 


flexibility and thus reduces the system stiffness. The 
design of the actuator placement and transmission for 
each joint is a trade-off between weight, inertia, stiff¬ 
ness, and complexity. This choice dictates the major 
physical characteristics of a manipulator design. To il¬ 
lustrate this point, consider the Adept 1 assembly robot 
with four degrees of freedom, each a different structure. 
The first axis has a direct motor drive. The second axis 
is driven by a steel band drive, and the third by a syn¬ 
chronous belt drive. Finally, the fourth axis uses a linear 
ball-screw drive. For a variety of useful joint mecha¬ 
nisms Sclater and Chironis [4.48]. 

4.7.1 Joint Axis Structures 

Revolute Joints 

Revolute or rotary motion joints are designed to per¬ 
form pure rotation while minimizing other displace¬ 
ments and motions. The most important measure of the 
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quality of a revolute joint is its stiffness or resistance 
to all undesired motion. Key factors to be considered in 
design for stiffness are shaft diameter, mounting con¬ 
figuration and preloading of the bearings and proper 
clearances and tolerances. Shaft diameter and bearing 
size are not always based on load-carrying capacity; 
rather, they often will be selected to be compatible with 
a rigid mounting configuration and also have a bore 
large enough to pass cables, hoses, and even drive 
elements for other joints. Because joint shafts will fre¬ 
quently be torque-transmitting members, they and their 
supporting structure must be designed both for bending 
and torsional stiffness. The first axis of the PUMA robot 
is an example of such a joint with its large-diameter 
tubular configuration. 

An important factor in maintaining stiffness in a rev¬ 
olute joint is the choice of bearing-mounting configura¬ 
tion. The mounting arrangement and mount must be de¬ 
signed to accommodate manufacturing tolerances, ther¬ 
mal expansion and bearing preload. Axial preloading of 
ball or tapered roller bearings improves system accu¬ 
racy and stiffness by minimizing both axial and radial 
play. Preloads can be achieved through selective assem¬ 
bly or elastic (spring) elements, shim spacers, threaded 
collars, four-point contact bearings, duplex bearing ar¬ 
rangements, and tight manufacturing tolerances. 

Prismatic Joints 

There are two basic types of prismatic or linear mo¬ 
tion joints: single-stage and telescoping joints. Often 
a prismatic joint and its associated link and actuator are 
combined as a linear actuator. Single-stage joints are 
made up of a moving surface that slides linearly along 
a fixed surface. Telescoping joints are essentially sets 
of nested or stacked single-stage joints. Single-stage 
joints feature simplicity and high stiffness, whereas 
the primary advantage of telescoping joints is their 
retracted-state compactness and large extension ratio. 
Telescoping joints have a lower effective joint inertia 
for some configurations and motions because part of 
the joint may remain stationary or move with reduced 
acceleration. 

The primary functions of bearings in prismatic 
joints are to facilitate motion in a single direction and to 
prevent motion in all other directions. Preventing these 
unwanted motions poses the more challenging design 
problem. Deformations in the structure can distort the 
bearing. In severe cases, ball or roller deflection under 
load may cause binding, which precludes motion. For 
high-precision prismatic joints, ways must be straight 
and precise along their entire length which may be sev¬ 
eral meters. The required precision grinding on multiple 
surfaces can be costly. Bulky covers may be required to 
shield and seal a prismatic bearing and way assembly. 


The primary criterion for evaluating higher num¬ 
ber (in or near the wrist or end-effector) linear motion 
joints or axes is the stiffness-to-weight ratio. Achiev¬ 
ing a good stiffness-to-weight ratio requires the use of 
a hollow or thin-walled structure rather than solid mem¬ 
bers for the moving elements. 

Bearing spacing is extremely important in design 
for stiffness. If this spacing is too short, system stiff¬ 
ness will be inadequate no matter how great the bearing 
stiffness. A major cause for failure in prismatic joints 
is surface fatigue wear (brinelling) of the ways caused 
by excessive ball loading due to high preload, moment 
loads and shock loads. 

The large exposed precision surfaces in most pris¬ 
matic joints make them much more sensitive than rev¬ 
olute joints to contamination, improper handling and 
environmental effects. They are also significantly more 
difficult to manufacture, properly assemble, and align. 

Common types of sliding elements for prismatic 
motion are bronze or thermoplastic impregnated bush¬ 
ings. These bushings have the advantage of being low 
in cost, of having relatively high load capacity, and 
of working with unhardened or superficially hardened 
(i. e., plated or anodized) surfaces. Because the local or 
contact stress on the moving element is distributed and 
is low this element may be made of thin tube or an ex¬ 
truded shape. Another type of bushing in common use 
is the ball bushing. Ball bushings have the advantages of 
lower friction and greater precision than plain bushings. 
However, they require that the contacting surface of the 
joint be heat treated or hardened (generally to Rc 55 
or greater) and of sufficient case and wall thickness to 
support the ball point contact loads and resulting high 
stresses. 

Ball and roller slide assemblies are commonly used 
in robot prismatic joints. There are two basic categories 
of these slides; recirculating and non-recirculating. 
Non-recirculating ball and roller slides are used pri¬ 
marily for short-travel applications. They feature high 
precision and very low friction at the expense of being 
more sensitive to shock and relatively poor at accom¬ 
modating moment loading. Recirculating ball slides are 
somewhat less precise but can carry higher loads than 
non-recirculating ball slides. They can also be set up to 
handle relatively large moment loads. Travel range can 
be up to several meters. Commercial recirculating ball 
slides and ways have greatly simplified the design and 
construction of linear axes, particularly in gantry and 
track mounted manipulators. 

Another common type of prismatic robot joint is 
made up of cam followers, rollers, or wheels rolling on 
extruded, drawn, machined, or ground surfaces. In high- 
load applications the surfaces must be hardened before 
they are finish ground. Cam followers can be purchased 
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Fig. 4.14 Wrist 
for surgery has 
no singularities 
within range of 
motion. Fea¬ 
tures tungsten 
cable actuation 
(after [4.49]) 


Fig. 4.13 RobotWorld is an integrated workcell with mul¬ 
tiple robot modules that move on air bearings 

with eccentric mounting shafts to facilitate setup and 
adjustment. Elastomer rollers provide quiet, smooth op¬ 
eration. 

Two less common types of linear or prismatic joints 
feature flexures and air bearings. Flexure-based joints, 
whose motion results from elastic bending deforma¬ 
tions of beam support elements, are used primarily for 
small, high-resolution, quasilinear motions. Air bear¬ 
ings require smooth surfaces and close control of tol¬ 
erances as well as a constant supply of filtered, oil-free 
compressed air. Two- and three-degree-of-freedom air 
bearings ( x , y, 9) can enable multiaxis motion with few 
moving parts (Fig. 4.13). 

Joint Travel 

For revolute joint configurations, the shoulder and el¬ 
bow joints and links determine the gross volume of the 
work envelope (reachable workspace) of a robot ma¬ 
nipulator arm. The wrist joints generally determine the 
orientation range (dexterous workspace) about a point 
within this work envelope. Larger joint travel may 
increase the number of possible manipulator configura¬ 
tions that will reach a particular location (increased task 
space). Wrist joint travel in excess of 360° and up to 


720° can be useful for situations requiring controlled- 
path (e.g., straight-line) motion, synchronized motion 
such as small part assembly on a moving conveyor 
belt, or sensor-modified motions as in using machine 
vision to select and guide picking jumbled items from 
a bin. Continuous last-joint rotation is desirable in cer¬ 
tain cases like loading or unloading a rotating machine 
or mating threaded parts. 

Additional joints and links, sometimes in the 
robot but more often in the end-effector, and spe¬ 
cialized tooling also serve to increase the task space 
of a robot. Continuous and controlled-path robot mo¬ 
tion requires planning to avoid singularities (regions 
where two or more joints may become aligned or 
nearly aligned) and the resulting unstable end-effector 
motion in these regions. Manipulator design coordi¬ 
nated with a well-planned workcell layout can im¬ 
prove the useful task space by placing critical mo¬ 
tions well away from singularity regions. For exam¬ 
ple, a standard three-axis robot wrist has singularities 
180° apart, which can be increased to 360° by im¬ 
plementing a somewhat more complex reduced singu¬ 
larity wrist. Such a wrist is used in a sheep-shearing 
robot to achieve multiple, long, continuous, smooth, 
constant-velocity, sensor-guided passes over the con¬ 
toured body of a sheep to shear its wool [4.50,51]. 
Figure 4.14 shows a wrist used for minimally-invasive 
surgery that has no singularities within its range of mo¬ 
tion [4.49]. 



4.8 Actuators 

Actuators supply the motive power for robots. Most 
robot actuators are commercially available components, 
which are adapted or modified, as necessary, for a spe¬ 
cific robot application. Three commonly used actuator 
types are electromagnetic, hydraulic and pneumatic. 

4.8.1 Electromagnetic Actuators 

The most common types of actuators in robots today are 
electromagnetic actuators. 


Electric Servomotors 

Most robot manipulators use servomotors as a power 
source. Servomotors are designed to accurately follow 
the desired position, velocity and torque which change 
frequently and sometimes abruptly. They have struc¬ 
tures similar to ordinary electric motors, but with low 
inertia and large torque capably for high accelerations. 
Typical servomotors used for robotic applications are 
permanent magnet (PM) DC motors and brushless DC 
(BLDC) motors. 
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PM DC motors are widely used as a servomotor be¬ 
cause of high torque, speed controllability over a wide 
range, well-behaved torque-speed characteristics, and 
adaptability to various types of control methods. The 
DC motor converts electrical energy into rotational or 
linear mechanical energy. It comes in many different 
types and configurations. The lowest-cost PM motors 
use ceramic (ferrite) magnets and robot toys and hobby 
robots often use this type of motor. A PM motor with 
a rare-earth (neodymium-iron-boron), NEO magnet sta¬ 
tor produces the most torque and power for its size. 

Brushless motors, also called AC servomotors or 
brushless DC motors, are widely used in industrial 
robots (Figs. 4.15 and 4.16). They substitute magnetic 
or optical sensors and electronic switching circuitry 
for the graphite brushes and copper bar commutator 
used in brush-type DC motors, thus eliminating the 
friction, spark-ing, and wear of commutating parts. 
Brushless motors generally have good performance at 
low cost because of the decreased complexity of the 
motor. However, the controllers for these motors are 
more complex and expensive than brush-type motor 
controllers. A passive multi-pole neodymium magnet 
rotor and a wire-wound iron stator of a brushless motor 
provide good heat dissipation and excellent reliability. 
Linear brushless motors function like unrolled rotary 
motors. They typically have a long, heavy, multiple 



Fig. 4.15 The Baldor AC servomotor 



Fig. 4.16 The Anorad brushless linear motor 


magnet passive stator and a short, lightweight, electron¬ 
ically commutated wire-wound forcer (slider). 

Modeling of DC Motors and BLDC Motors. Both DC 
and BLDC motors can be described by almost identi¬ 
cal equations although they have different structures, de 
Silva [4.52]. Fig. 4.17. The motor torque r,„ generated 
at the motor is given by, 

x m =K t i a , ( 4 . 20 ) 

where K, is the torque constant [N m/A] and i a is the 
armature current [A]. 

When the rotor is rotating, the back electromotive 
force (emf) Vb is induced as follows, 

Vb=Kb&> m , ( 4 . 21 ) 

where Kb is the back emf constant [V/(rad/s)] and tu„, 
is the angular speed of the motor [rad/s]. 

The circuit equation is given by 

v a — Ra ia + L a — - 1 -Vb, ( 4 . 22 ) 

dr 

where v a is the armature voltage (supply voltage to the 
armature) [V], R a and L a are the resistance [12] and in¬ 
ductance [H] of the armature winding, respectively. 
Lastly, the mechanical equation is described by 

d a) m x 1 

J—-+Bco m = x m ~-, ( 4 . 23 ) 

dr r 

where J is the equivalent moment of inertia re¬ 
ferred to the motor shaft [kgm 2 ], B is the equivalent 
viscous-friction coefficient referred to the motor shaft 
[N m/rad/s], r/ is the load torque [Nm], and r is the 
gear ratio. The quantities J and B are given by 

J, B, 

J — Jm 4 — B — B,„ 4 —^ > ( 4 . 24 ) 

where J m and B m are related to the motor including the 
gear connected to the motor shaft and J/ and B/ are re¬ 
ferred to the load including the gear connected to the 
load shaft. 



Fig. 4.17 Servomotor models 
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The four equations above can be represented as the 
block diagram in Fig. 4.18. 

In the servomotors, the electrical time constant x e = 
LJR a and mechanical time constant r,„ = J/B have ef¬ 
fects on their performance. The electrical time constant 
is much smaller than mechanical time constant because 
of the armature inductance is usually negligible. Con¬ 
sidering this factor, the angular speed of the motor is 
described in terms of the armature voltage v a and load 
torque 17 as follows 


co m (s) 


1 

JR a s + (R a B + K,K h ) 


iyK l v a {s)-R a 



( 4 . 25 ) 


The above equation is frequently used in simula¬ 
tions of both DC motor and BLDC motors. 


Stepper Motors. Small, simple robots, such as bench- 
top adhesive dispensing robots, frequently use stepper 
or pulse motors of the permanent magnet (PM) hybrid 
type or sometimes the variable reluctance (VR) type 
(Fig. 4.19). These robots use open-loop position and 
velocity control. They are relatively low in cost and 
interface easily to electronic drive circuits. Microstep 
control can produce 10000 or more discrete robot joint 
positions. In open-loop step mode the motors and robot 
motions have a significant settling time, which can be 


^9 

Vb 


1 


L a s + R a 


K, 


t,/r 


"*k> 


1 


J s + B 


Fig. 4.18 Block diagram for servomotors 



Fig. 4.19 The Sony robot uses open-loop permanent- 
magnet stepper motors 


damped either mechanically or through the application 
of control algorithms. Power-to-weight ratios are lower 
for stepper motors than for other types of electric mo¬ 
tors. Stepper motors operated with closed-loop control 
function similarly to direct-current (DC) or alternating- 
current (AC) servomotors (Fig. 4.20). 

Permanent-Magnet DC Motor. The permanent- 
magnet, direct-current, brush-commutated motor is 
widely available and comes in many different types 
and configurations. The lowest-cost permanent-magnet 
motors use ceramic (ferrite) magnets. Robot toys and 
hobby robots often use this type of motor. Neodymium 
(NEO) magnet motors have the highest energy-product 
magnets, and in general produce the most torque and 
power for their size. 

Ironless rotor motors, often used in small robots, 
typically have copper wire conductors molded into 
epoxy or composite cup or disk rotor structures. The ad¬ 
vantages of these motors include low inductance, low 
friction, and no cogging torque. Disk armature mo¬ 
tors have several advantages. They have short overall 
lengths, and because their rotors have many commu¬ 
tation segments, they produce a smooth output with 
low torque ripple. A disadvantage of ironless armature 
motors is that they have a low thermal capacity due 
to low mass and limited thermal paths to their case. 
As a result, when driven at high power levels they 
have rigid duty-cycle limitations or require forced-air 
cooling. 

4.8.2 Hydraulic Actuators 

Hydraulic actuators, chosen as power sources for the 
earliest industrial robots, offer very large force capa¬ 
bility and high power-to-weight ratios. They convert 
hydraulic power into useful mechanical energy. The 
mechanical motion produced may be linear for hy- 



Fig. 4.20 The Adept robot uses closed-loop control and 
variable-reluctance motors (I'gfcM'll'H'lRl) 
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Fig. 4.21 General hydraulic circuit 


draulic cylinders or rotary for hydraulic motors and 
vane actuators. 

Figure 4.21 shows the basic hydraulic circuit com¬ 
posed of oil tank, hydraulic pump, check valve, relief 
valve, directional control valve and hydraulic actuator. 
Hydraulic fluid pressurized by the hydraulic pump is 
conveyed to the hydraulic machine to perform the spec¬ 
ified task. To this end, the pressure, flow, and direction 
of hydraulic fluid is changed by the hydraulic control 
valves such as a pressure control valve, flow control 
valve, and directional control valve, respectively. The 
directional control valve changes the flow direction of 
the hydraulic fluid which is pressurized in the hydraulic 
pump and thus the direction of actuator motion. The 
flow control valve changes the flow rate of hydraulic 
fluid flow and thus the speed of the hydraulic actuator. 
For safety, the highest pressure is limited by the pres¬ 
sure control valve such as a relief valve. 

Hydraulic actuators have several advantages and 
disadvantages due to the use of high pressure fluid. 
They can offer very large force or torque and high 
power-to-weight ratios. Both linear and rotary motions 
are readily available with small inertia of the moving 
part. However, the hydraulic power supply is bulky and 
the cost of the proportional, fast-response servovalves 
are high. Leaks and maintenance issues have limited the 
use and applica-tion of hydraulically powered robots. 


There have been many applications of hydraulic 
actuators to robotics. For high forces or torques and 
speeds, hydraulic servo actuators out-perform current 
electromagnetic actuators. With proper design, leakage 
can be virtually eliminated, Hollerbach et al. [4.53]. 
Figure 4.22 shows some typical applications of hy¬ 
draulic actuators, [4.54-56]. 

4.8.3 Pneumatic Actuators 

Pneumatic actuators are similar to hydraulic actuators. 
Pneumatic actuators convert energy (in the form of 
compressed air) into mechanical motion, which may 
be linear or rotary. Pneumatic actuators are primarily 
found in simple manipulators. Typically they provide 
uncontrolled motion between mechanical limit stops. 
These actuators provide good performance in point- 
to-point motion. They are simple to control and are 
low in cost. Pneumatic motors have several advantages 
over electric motors. They are relatively safe in the 
explosive environment. They are also less affected by 
ambient temperature and humidity than electric mo¬ 
tors. Although a few small actuators may be run with 
typical factory air supplies, extensive use of pneumatic- 
actuated robots requires the purchase and installation 
of a costly dedicated compressed-air source. Pneumatic 
actuators have low energy efficiency. 

Pneumatic systems consist of pneumatic genera¬ 
tor, pneumatic valves, pneumatic actuator and pipes. 
A pneumatic generator produces compressed air using 
an air compressor. Pneumatic valves are used to control 
the pressure, flow rate and direction. The mechanical 
motion produced may be linear for pneumatic cylinders 
or rotary for pneumatic motors. 

Pneumatic actuators are not used for applications 
requiring large forces or torques since they produce 
less power than hydraulic actuators or electric actua¬ 
tors. However, they are used in robot hands or artificial 
muscles, which require high power-to-weight ratios. 
Pneumatic artificial muscles are contractile or exten- 
sional devices operated by pressurized air filling a pneu¬ 
matic bladder. They are usually grouped in the agonist 



Fig.4.22a-c Applications of 
hydraulic actuators to robot: 

(a) BigDog (Boston dynamics) 

< I daMUtlTTlHI 1 (b) Sarcos exoscele- 
ton (Raytheon) (c) 

Magnum 7 (International Submarine 
Engineering) 
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Fig.4.23a,b Applications of pneu¬ 
matic actuator: (a) robot hand and 
arm with artificial muscle (Shadow 
robot) and (b) pneumatic step motor 
and MrBot (Urobotics, Johns Hop¬ 
kins) 




and antagonist pairs. Furthermore, they can be used 
for medical robots since they are not affected by mag¬ 
netic field, and for robots in explosive environments 
because there is no electrical arcing that exists in elec¬ 
tromagnetic actuators. Figure 4.23 shows some robotic 
applications of pneumatic actuators [4.57,58]. 

4.8.4 Other Actuators 

A wide variety of other types of actuators have been 
applied to robots. A sampling of these include, ther¬ 
mal, shape-memory alloy (SMA), bimetallic, chemical, 
piezoelectric, magnetostrictive, electroactive polymer 
(EAP), bladder, and micro-electromechanical system 
(MEMS) actuators (see Figs. 4.24 and 4.25). Most of 
these actuators have been applied to research and spe¬ 
cial application robots rather than volume production 
industrial robots. An example of a piezoelectric actuator 
powered robot is the six-axis PI (Physik Instrumente) 
piezo hexapod with sub-nanometer resolution shown 
in Fig. 4.26. 



Fig. 4.24 The artificial muscle EAP motor 



Fig. 4.25 The Elliptec piezoelectric motor 


4.8.5 Transmissions 

The purpose of a transmission or drive mechanism is 
to transfer mechanical power from a source to a load. 
The design and selection of a robot drive mechanism re¬ 
quires consideration of motion, load, and power require¬ 
ments and the placement of the actuator with respect 
to the joint. The primary considerations in transmission 
design are stiffness, efficiency, and cost. Backlash and 
windup impact drive stiffness especially in robot appli¬ 
cations where motion is constantly reversing and load¬ 
ing is highly variable. High transmission stiffness and 
low or no backlash result in increased friction losses. 
Most robot transmission elements have good efficien¬ 
cies when they are operating at or near their rated power 
levels but not necessarily when lightly loaded. Larger 
than necessary drives add weight, inertia and friction 
loss to the system. Underdesigned drives have lower 
stiffness, can wear rapidly in continuous or in high duty 
cycle operation or fail due to accidental overloads. 

Joint actuation in robots is generally performed by 
drive mechanisms which interface the actuator (mech¬ 
anical work source) to the robot links through the joints 
in an energy-efficient manner. A variety of drive mech¬ 
anisms are incorporated in practical robots. The trans¬ 
mission ratio of the drive mechanism sets the torque, 
speed, and inertia relationship of the actuator to the link. 
Proper placement, sizing, and design of the drive mech¬ 
anisms set the stiffness, mass, and overall operational 
performance of the robot. Most modern robots incorpo¬ 
rate efficient, overload damage resistant, back-driveable 
drives. 



Fig. 4.26 A six-axis Physik Instrumente (PI) piezo hexa¬ 
pod with sub-nanometer resolution (|o>*!lliliIilSl) 
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Direct Drives 

The direct drive is kinematically the simplest drive 
mechanism. In the case of pneumatic or hydraulic 
actuated robots, the actuator is directly connected be¬ 
tween the links. Electric direct-drive robots employ 
high-torque, low-speed rotary or linear motors directly 
interfaced to the links. The complete elimination of free 
play and smooth torque transmission are features of 
a direct drive. However, there is often a poor dynamic 
(inertia ratio) match of the actuator to the link requiring 
a larger, less energy efficient, actuator. 

Band Drives 

A variant of direct drive is band drive. A thin alloy steel 
or titanium band is fixed between the actuator shaft 
and the driven link to produce limited rotary or linear 
motion. Drive ratios in the order of up to 10 : 1 (10 ac¬ 
tuator revolutions for 1 revolution of the joint) can be 
obtained. Actuator mass is also moved away from the 
joint - usually toward the base, to reduce robot iner¬ 
tia and gravity loading. It is a smoother and generally 
stiffer drive than a cable or belt drive. 

Belt Drives 

Synchronous (toothed) belts are often employed in 
drive mechanisms of smaller robots and some axes of 
larger robots. These function much the same as band 
drives, but have the ability to drive continuously. Mul¬ 
tiple stages (two or three) are occasionally used to 
produce large drive ratios (up to 100: 1). Belt tension 
is controlled with idlers or center adjustment. The elas¬ 
ticity and mass of long belts can cause drive instability 
and thus increased robot settling time. 

Gear Drives 

Spur or helical gear drives provide reliable, sealed, 
low-maintenance power transmission in robots. They 
are used in robot wrists where multiple axes inter¬ 
sect and compact drive arrangements are required. 
Large-diameter gears are used in the base joints of 
larger robots to handle high torques with high stiff¬ 
ness. Gears are often used in stages and often with 
long drive shafts, enabling large physical separation 
between actuator and driven joint. For example, the 
actuator and one stage of reduction may be located 
near the elbow driving another stage of gearing or dif¬ 
ferential in a wrist through a long hollow drive shaft 
(Fig. 4.1). 

Planetary gear drives are often integrated into com¬ 
pact gearmotors (Fig. 4.27). Minimizing backlash (free 
play) in a joint gear drive requires careful design, 
high-precision and rigid support to produce a drive 
mechanism which does not sacrifice stiffness, efficiency 
and accuracy for low backlash. Backlash in robots is 


controlled by a number of methods including selective 
assembly, gear center adjustment, and proprietary anti¬ 
backlash designs. 

Worm Gear Drives 

Worm gear drives are occasionally used in low-speed 
robot manipulator applications. They feature right- 
angle and offset drive capability, high ratios, simplicity, 
good stiffness and load capacity. They have poor effi¬ 
ciency, which makes them non-back-driveable at high 
ratios. This causes the joints to hold their position when 
unpowered but also makes them prone to damage by 
attempts to manually reposition the robot. 

Proprietary Drives 

Proprietary drives are widely used in standard industrial 
manipulators. The harmonic drive and the rotary vector 
(RV) drive are two examples of compact, low-backlash, 
high-torque-capability drives using special gears, cams, 
and bearings (Figs. 4.28 and 4.29). 

Harmonic drives are frequently used in very small to 
medium-sized robots. These drives have low backlash, 
but its flexspline allows elastic windup and low stiff¬ 
ness during small reversing movements. RV drives are 



Fig. 4.27 The Space Shuttle robot arm has planetary gear 
joint drives 



Fig. 4.28 The harmonic drive il-aaiM'Jl'lhlff i 
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usually used in larger robots, especially those subject to 
overloads and shock loading. 

Linear Drives 

Direct-drive linear actuators incorporate a linear motor 
with a linkage to a linear axis. This linkage is often 
merely a rigid or flexure connection between the actua¬ 
tor forcer and the robot link. Alternatively, a packaged 
linear motor with its own guideways is mechanically 
connected directly to a linear axis. Direct linear elec¬ 
tromagnetic drives feature zero backlash, high stiffness, 
high speeds, and excellent performance but are heavy, 
have poor energy efficiency, and cost more than other 
types of linear drives. 

Ball Screws 

Ball-screw-based linear drives efficiently and smoothly 
convert rotary actuator motion into linear motion. Typ¬ 
ically, a recirculating ball nut mates with a ground and 
hardened alloy steel screw to convert rotary motion into 
linear motion. Ball screws can be easily integrated into 
linear axes. Compact actuator/drive packages are avail¬ 
able, as well as components for custom integration. 
Stiffness is good for short and medium travel, however 
it is lower for long motions because the screw can only 
be supported at its ends. Low or zero backlash can be 
obtained with precision-ground screws. Speeds are lim¬ 



Fig. 4.29 The Nabtesco RV drive 



ited by screw dynamic stability so rotating nuts enable 
higher speeds. Low-cost robots may employ plain screw 
drives featuring thermoplastic nuts on smooth rolled 
thread screws. 

Rack-and-Pinion Drives 

These traditional components are useful for long mo¬ 
tions where the guideways are straight or even curved. 
Stiffness is determined by the gear/rack interface and 
independent of length of travel. Backlash can be diffi- 



Fig. 4.31 Two-arm collaborative robot with Series Elastic 
Actuators (after [4.59]) 



Fig. 4.32 Series Elastic Actuator assembly in knee-ankle- 
foot gait rehabilitation robot (after [4.60]) 
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cult to control as rack-to-pinion center tolerances must 
be held over the entire length of travel. Dual pinion 
drives are sometimes employed to deal with backlash 
by providing active preload. Forces are generally lower 
than with screws due to lower ratios. Small-diameter 
(low teeth count) pinions have poor contact ratios, 
resulting in vibration. Sliding involute tooth contact 
requires lubrication to minimize wear. These catalog 
stock drive components are often used on large gantry 
robots and track-mounted manipulators (Fig. 4.30). 

Compliant Actuators. Compliance in the form of 
elasticity in a drive can be an asset or a defect in 
a robot. High overall robot stiffness traditionally pro¬ 
duces fast response, high positioning accuracy and 
simplified control. However the contact and interaction 
forces developed due to unexpected errors in dealing 
with the workpieces, tools, workplace and the environ¬ 
ment can cause damage to the robot, the surroundings, 
and injury to persons. By adding controlled and mea- 
sureable compliance to actuators, the elasticity of the 
robot can be usefully increased. 

One class of reduced stiffness actuator is the Series 
Elastic Actuator (SEA). It features an elastic output el¬ 
ement (spring) with a displacement sensor (measures 


4.9 Robot Performance 

Industrial robot performance is often specified in terms 
of functional operations and cycle time. For assembly 
robots the specification is often the number of typical 
pick-and-place cycles per minute. Arc-welding robots 
are specified with a slow weld pattern and weave speed 
as well as by a fast repositioning speed. For painting 
robots, the deposition or coverage rate and spray pattern 
speed are important. Peak robot velocity and accelera¬ 
tion catalog data are generally just calculated numbers 
and will vary due to dynamic (inertia) and static (grav¬ 
ity) coupling between robot joints due to configuration 
changes as a robot moves. 

4.9.1 Robot Speed 

Maximum joint velocity (angular or linear) is not an 
independent value. For longer motions it is often lim¬ 
ited by servomotor bus voltage or maximum allowable 
motor speed. For manipulators with high accelera¬ 
tions, even short point-to-point motions may be velocity 
limited. For low-acceleration robots, only gross mo¬ 
tions will be velocity limited. Typical peak end-effector 
speeds can range up to 20 m/s for large or fast manipu¬ 
lators. 


spring stretch) in series with a stiff actuator and trans¬ 
mission component. With an appropriate controller, the 
stiff classical position control actuator acts as a force 
actuator (F = —kx), effectively isolating the drive iner¬ 
tia from the load inertia. This limits the forces resulting 
from collisions or forced compliance often encoun¬ 
tered when working in unstructured environments and 
around persons. SEAs and similar compliant actuators 
and compliant actuator configurations are discussed in 
Chap. 69. Collaborative robots are designed to safely 
work next to or in contact with persons, Fig. 4.31 [4.59], 
and are an example of robots featuring compliant actu¬ 
ators. Figure 4.32 shows compliant actuators used in an 
exoskeleton for gait rehabilitation [4.60], 

Other Drive Components 

Splined shafts, kinematic linkages (four-bar, slider- 
crank mechanisms, etc.) chains, cables, flex couplings, 
clutches, brakes, and limit stops are some examples of 
other mechanical components used in robot drive mech¬ 
anisms (Fig. 4.8). The Yaskawa RobotWorld assembly 
and process automation robot system features a mag¬ 
netically suspended direct electromagnetic drive planar 
(2 axis) motor with no internal moving parts floating on 
a virtually frictionless planar air bearing. (Fig. 4.12). 


4.9.2 Robot Acceleration 

In most modern manipulators, because the payload 
mass is small when compared with the manipulator 
mass, more power is spent accelerating the manipulator 
than the load. Acceleration affects gross motion time 
as well as cycle time (gross motion time plus settling 
time). Manipulators capable of greater acceleration tend 
to be stiffer manipulators. In high-performance robot 
manipulators, acceleration and settling time are more 
important design parameters than velocity or load ca¬ 
pacity. Maximum acceleration for some assembly and 
material handling robots is in excess of 10 g with light 
payloads. 

4.9.3 Repeatability 

This specification represents the ability of the ma¬ 
nipulator to return repeatedly to the same location. 
Depending on the method of teaching or programming 
the manipulator, most manufacturers intend this fig¬ 
ure to indicate the radius of a sphere enclosing the 
set of locations to which the arm returns when sent 
from the same origin by the same program with the 
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same load and setup conditions. This sphere may not 
include the target point because calculation round-off 
errors, simplified calibration, precision limitations, and 
differences during the teaching and execution modes 
can cause significantly larger errors than those just due 
to friction, unresolved joint and drive backlash, servo 
system gain, and structural and mechanical assembly 
clearances and play. The designer must seriously con¬ 
sider the real meaning of the required repeatability 
specification. Repeatability is important when perform¬ 
ing repetitive tasks such as blind assembly or machine 
loading. Typical repeatability specifications range from 
1—2 mm for large spot-welding robots to 0.005 mm 
(5 |xm) for precise micropositioning manipulators. 

4.9.4 Resolution 

This specification represents the smallest incremental 
motion that can be produced by the manipulator. Res¬ 
olution is important in sensor-controlled robot motion 
and in fine positioning. Although most manufacturers 
calculate system resolution from the resolution of the 
joint position encoders, or from servomotor and drive 
step size, this calculation is misleading because system 
friction, windup, backlash, and kinematic configura¬ 
tion affect the system resolution. Typical encoder or 
resolver resolution is 10 14 —10 25 counts for full-axis or 
joint travel, but actual physical resolution may be in the 
range 0.001—0.5 mm. The useful resolution of a multi¬ 
joint serial-link manipulator is worse than that of its 
individual joints. 

4.9.5 Accuracy 

This specification covers the ability of a robot to po¬ 
sition its end-effector at a preprogrammed location 
in space. Robot accuracy is important in the perfor¬ 
mance of nonrepetitive types of tasks programmed 
from a database, or for taught tasks that have been 
remapped or offset owing to measured changes in the 
installation. 

Accuracy is a function of the precision of the 
arm kinematic model (joint type, link lengths, angles 
between joints, any accounting for link or joint de¬ 
flections under load, etc.), the precision of the world, 
tool, and fixture models, and the completeness and 
accuracy of the arm solution routine. Although most 
higher-level robot programming languages support arm 
solutions, these solutions usually model only simplified 
rigid-body kinematic configurations. Thus, manipulator 
accuracy becomes a matter of matching the robot geom¬ 
etry to the robot solution in use by precisely measuring 
and calibrating link lengths, joint angles, and mounting 
positions. 


Typical accuracies for industrial manipulators range 
from ±10 mm for uncalibrated manipulators that have 
poor computer models to ±0.01 mm for machine-tool- 
like manipulators that have controllers with accurate 
kinematic models and solutions and precisely manufac¬ 
tured and measured kinematic elements. 

4.9.6 Component Life and Duty Cycle 

The three subassemblies in an electrically powered 
robot with the greatest failure problems are the actua¬ 
tors (servomotors), transmissions, and power and sig¬ 
nal cables. Mean time between failures (MTBF) should 
be a minimum of 5000 h on line, and ideally at least 
10 000 operating hours should pass between major com¬ 
ponent preventive maintenance replacement schedules. 

Worst-case motion cycles must be assumed as most 
current robot installations are used in generally repet¬ 
itive tasks. Small-motion design-cycle life (less than 
5% of joint travel range) for assembly robots should be 
20—100 million full bidirectional cycles. Large-motion 
cycle life (greater than 50% of full joint range) should 
typically be 5—40 million cycles. 

Short-term peak performance is frequently lim¬ 
ited by maximum drive loading, whereas long-term, 
continuous, performance is limited by motor heating. 
Rather than design for equal levels of short- and long¬ 
term performance, cost savings and performance im¬ 
provements can result from designing for an anticipated 
duty cycle. This allows the use of smaller, lower-inertia, 
lighter motors. Industrial robots usually become ob¬ 
solete and are replaced before they reach their design 
cycle life. 

4.9.7 Collisions 

In the course of operation, unforeseen or unexpected 
situations may occasionally result in a collision involv¬ 
ing the manipulator, its tools, the workpiece, or other 
objects in the workplace. These accidents may result in 
no, little, or extensive damage, depending in large part 
on the design of the manipulator. Crash-resistant design 
options should be considered early in the design process 
if the time lost or cost of such accidents could be signifi¬ 
cant. Typical damage due to accidents include fracture 
or shear failures of gear teeth or shafts, dented or bent 
link structures, slipping of gears or pulleys on shafts, 
cut or severely abraded or deformed wires, cables or 
hoses, and broken connectors, fittings, limit stops or 
switches. Compliant elements such as overload (slip) 
clutches, elastic members, and padded surfaces can be 
incorporated to reduce shock loads and help decouple or 
isolate the actuators and drive components in the event 
of such collisions. 
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4.10 Conclusions and Further Reading 


The mechanical design of a robot is an iterative pro¬ 
cess involving engineering, technical, and application- 
specific considerations evaluations, and choices. The 
final design should reflect consideration of detailed task 
requirements rather than simply broad specifications. 
Proper identification and understanding of these re¬ 
quirements is a key to achieving the design goals. 

Design and choice of specific components involves 
tradeoffs. A purely static, rigid-body approach to ma¬ 
nipulator design is often used, but is not always suffi¬ 
cient. Mechanical system stiffness, natural frequencies, 
control system compatibility, and intended robot appli¬ 
cations and installation requirements must be consid¬ 
ered. 

There are many opportunities for further reading on 
the design of the mechanisms and actuation that form 
the core of a robotic system. A well-known and useful 
reference for robot design is Rivin [4.44]. 


Craig [4.6] and Tsai [4.9] provide the mathematical 
relations between the mechanical structure of a robot 
and its workspace and mechanical advantage. Sclater 
and Chironis [4.48] is a reprint of a valuable com¬ 
pendium of devices useful for a variety of applications, 
such as joint drives and transmissions. See McCarthy 
and Soh [4.34] for geometric techniques to design spe¬ 
cialized mechanisms. 

Juvinall and Marshek [4.45] and Shigley and Mis- 
chke [4.46,47] are important references for the design 
of the components such as the link structure, bearings, 
and transmissions that are central to the effective me¬ 
chanical performance of robotic systems. 

Although many design decisions can be made 
through the application of straightforward algorithms 
and equations, a multitude of other important consider¬ 
ations transform the challenge of robot design into one 
requiring good engineering judgment. 
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A parallel robot 

available from http://handbookofrobotics.org/view-chapter/04/videodetails/640 
Three-fingered robot hand 

available from http://handbookofrobotics.org/view-chapter/04/videodetails/642 
Robotics milking system 

available from http://handbookofrobotics.org/view-chapter/04/videodetails/643 
SCARA robots 

available from http://handbookofrobotics.org/view-chapter/04/videodetails/644 
Big Dog -Applications of hydraulic actuators 

available from http://handbookofrobotics.org/view-chapter/04/videodetails/645 
Raytheon Sarcos exoskeleton 

available from http://handbookofrobotics.org/view-chapter/04/videodetails/646 
PI piezo hexapod 

available from http://handbookofrobotics.org/view-chapter/04/videodetails/648 
Harmonic drive 

available from http://handbookofrobotics.org/view-chapter/04/videodetails/649 
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common representations for estimation are 
introduced. 



Sensing and estimation are essential aspects of the 
design of any robotic system. At a very basic level, 
the state of the robot itself must be estimated for 
feedback control. At a higher level, perception, 
which is defined here to be task-oriented inter¬ 
pretation of sensor data, allows the integration 
of sensor information across space and time to 
facilitate planning. 

This chapter provides a brief overview of 
common sensing methods and estimation tech¬ 
niques that have found broad applicability in 
robotics. The presentation is structured accord¬ 
ing to a process model that includes sensing, 
feature extraction, data association, parameter es¬ 
timation, and model integration. Several common 
sensing modalities are introduced and character¬ 
ized. Common methods for estimation in linear 
and nonlinear systems are discussed, includ¬ 
ing statistical estimation, the Kalman filter, and 
sample-based methods. Strategies for robust esti¬ 
mation are also briefly described. Finally, several 


5.1 Introduction 

Controlling a robotic system would be relatively simple 
if a complete model of the environment was available, 
and if the robot actuators could execute motion com¬ 
mands perfectly relative to this model. Unfortunately, 
in most cases of interest, a complete world model is not 
available, and perfect control of mechanical structures 
is never a realistic assumption. Sensing and estimation 
are a means of compensating for this lack of complete 
information. Their role is to provide information about 
the state of the environment and the state of the robot 
system as a basis for control, decision making, and in¬ 
teraction with other agents in the environment, such as 
humans. 


For the purposes of discussion, we will differenti¬ 
ate between sensing and estimation to recover the state 
of the robot itself, referred to as proprioception, versus 
sensing and estimation to recover the state of the ex¬ 
ternal world, referred to as exteroception. In practice, 
most robot systems are designed to have the propri¬ 
oception necessary to estimate and control their own 
physical state. On the other hand, recovering the state 
of the world from sensor data is usually a much larger 
and more complex problem. 

Early work on computational perception for 
robotics assumed that one could recover a complete 
general-purpose model of the environment, use such 
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a model to make decisions, and subsequently act on 
them, as for example presented by [5.1]. More re¬ 
cently it has become apparent that such an approach 
is not realistic. Indeed, considering that sensor-based 
robots now appear in diverse applications such as mo¬ 
bile surveillance, high-performance manipulation, and 
medical interventions, it is clear that appropriate sens¬ 
ing and estimation for a given system must be highly 
task dependent. Consequently, the discussion here is 
organized along the lines of task-oriented sensing and 
estimation of the external world. 

Sensing and estimation together can be viewed as 
the process of transforming a physical quantity into 
a computer representation that can be used for further 
processing. Sensing is thus closely tied to transducers 
that transform some physical entity into a signal that 
can be processed by a computer. Sensing is also inti¬ 
mately tied to perception, the process of representing 
the sensory information in an task-oriented model of the 
world. However, sensor data is usually corrupted in var- 


5.2 The Perception Process 

The input to the perception process is typically twofold: 
(1) digital data from a number of sensors/transducers, 
and (2) a partial model of the environment (a world 
model) that includes information about the state of the 
robot and other relevant entities in the external world. 
The sensor data itself can take on a number of dif¬ 
ferent forms such as a scalar or vector value x(ot, (1) 
acquired over a time series x(t), a scan x r (0j), a vec¬ 
tor field x or a three-dimensional volume x(p, 6, <p). In 
many cases, a system must integrate data from several 
disparate sensors, for example, an estimate of the po¬ 
sition of a mobile robot may integrate data from axis 
encoders, vision, global positioning system (GPS) data, 
and inertial sensors. 

To further structure the discussion in this chapter, 
we adopt a general model of the perception process 


ious ways that complicate this process. Statistical noise 
arises from the transducer, discretization is introduced 
in the digitization process, and ambiguity is introduced 
by poor sensor selectivity to name a few examples. 
Estimation methods are thus introduced to support ap¬ 
propriate integration of information into models of the 
environment and for improvement of the signal-to-noise 
ratio. 

In this chapter the general characteristics of sensing 
and estimation are introduced, while more in-depth pre¬ 
sentations of select topics are provided in Part C of the 
handbook. In Sect. 5.2 the overall sensing/perception 
process is introduced. In Sect. 5.3 different kinds of 
sensors are introduced and some key characteristics 
are presented. Estimation of world representations can 
utilize a number of different methods involving both 
parametric and nonparametric techniques as discussed 
in Sect. 5.4. For model-based integration a variety of 
different representations can be used, as described in 
Sect. 5.5. 


as shown in Fig. 5.1. In this model, we have included 
the most common operations applied to integrate sen¬ 
sor data with a world model. Depending on the task in 
question, some of the included modules may be miss¬ 
ing, and others may themselves take on a complicated 
structure. However, the supplied model suffices to illus¬ 
trate many of the issues in sensing and estimation. In the 
remainder of this section, we discuss an example from 
mobile localization to illustrate this model. 

The initial problem in sensory processing is data 
preprocessing and feature extraction. The role of pre¬ 
processing is to reduce noise from the transducer, to 
remove any systematic errors, and to enhance relevant 
aspects of the data. In some cases, sensory information 
might also have to be temporally or spatially aligned 
for subsequent integration. There are innumerable ways 



Fig. 5.1 Example of a perception process as discussed in this chapter 
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Fig. 5.2 An example of feature extraction from a laser 
scan (after [5.2]) 


that data can be preprocessed to enhance or extract 
features that are used in the integration. One common 
approach is model fitting, as illustrated for a laser scan¬ 
ner in Fig. 5.2. Once sensor information is available, 
it is often necessary to match the data with an exist¬ 
ing model (Fig. 5.3). This model may be based on 
a priori known structure (e.g., a computer-aided de¬ 
sign (CAD) model of the environment), or may have 
been built up from previously acquired data. Data as- 



Fig. 5.3 An example environmental model for mobile 
robot localization (after [5.2]) 



Fig. 5.4 Estimation of position and orientation for the ex¬ 
ample mobile robot (after [5.2]) 


sociation methods are commonly employed to estimate 
the relationship between sensor data and the model of 
the environment. In our mobile robot localization ex¬ 
ample, the extracted line features are matched against 
a polygonal world model. This matching process can 
be performed in several different ways, but in general 
it is an optimization that maximizes the alignment of 
features to the model. 

Once sensory data has been matched against the 
world model it is possible to update the model with 
new information contained in the sensor data. In the ex¬ 
ample, the orientation and position of the robot relative 
to the world model can be updated (Fig. 5.4) from the 
matched line segments. 

Finally, it may be possible to develop a dynamical 
system model of the underlying state being estimated. 
Using such a system model, it is possible to predict 
how the world changes over time until new sensory 
data is acquired. This can be used within a feed-forward 
prediction process, which in turn simplifies data associ¬ 
ation for new sensory readings, as shown in Fig. 5.1. 

With this as a prologue, we now turn to discuss each 
step of the perception process in greater detail. 
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5.3 Sensors 


There are a variety of ways to classify sensors depend¬ 
ing on what they measure, and how they measure it. 
As noted previously, proprioceptive sensors are used 
to measure the internal state of a robot, which might 
include position of different degrees of freedom, tem¬ 
perature, voltage on key components, motor current, 
force applied to an effector, and so forth. Exterocep¬ 
tive sensors, on the other hand, generate information 
about the external environment in terms of distance 
to an object, interaction forces, tissue density, and so 
forth. 

Sensors may also be differentiated based on whether 
they are passive or active. In general, an active sen¬ 
sor is one that emits energy into the environment, and 
measures properties of the environment based on the re¬ 
sponse. A passive sensor is one that is not active. Active 
sensors are generally more robust than passive sensors 
since they exert some control over the measured sig¬ 
nal. For example, a passive stereo camera system must 
rely on the appearance of viewed surfaces when per¬ 
forming feature matching for triangulation (Chap. 31), 
whereas structured light systems project a pattern onto 
the scene and are thus less sensitive to scene character¬ 
istics. Even so, absorbtion, scattering or interference of 
the emitted signal can affect the performance of active 
sensors. 

Proprioceptive sensors are typically passive and 
usually measure physical properties of the robot such 
as joint position, velocity, or acceleration, motor torque, 
and so forth. Exteroceptive sensors, on the other hand, 
can be further divided into contact and noncontact 
sensing. The contact sensors are typically the same 
modalities as used for proprioception, while noncontact 
sensor sensors involve most of the modalities that can 
be used for estimation of physical properties at a dis¬ 
tance including intensity, range, direction, size, and so 
forth. 

A classification of typical sensors according to 
method and typical application is shown in Table 5.1. 
More detail on methods of sensing, characterization of 
sensors, and general applications can for example be 

State Ch A Ch B 

51 High Low 

5 2 High High 

53 Low High 

54 Low Low 

Fig. 5.5 Sketch of the quadrature encoder disc, and output from 
photodetectors placed over each of the two pattern. The correspond¬ 
ing state changes are shown on the right 


found in the Handbook of Modern Sensors [5.3] and in 
Part C of this handbook. 

Estimation of rotational motion is fundamental to 
control of robot manipulators and also for estimation 
of ego-motion for mobile systems. The most common 
sensor for measurement of rotation is the quadrature 
encoder. It is composed of a transparent disc, with 
two periodic patterns that are out of phase, as shown 
in Fig. 5.5. Through the use of counters it is possible 


Table 5.1 Classification of sensors frequently used in 
robotics according to sensing objective (proprioception 
(PC)/exteroception (EC)) and method (active/passive) 


Classification 

Sensor type 

Sens 

A/P 

Tactile sensors 

S witches/bumpers 

EC 

P 


Optical barriers 

EC 

A 


Proximity 

EC 

P/A 

Haptic sensors 

Contact arrays 

EC 

P 


Force/torque 

PC/EC 

P 


Resistive 

EC 

P 

Motor/axis sensors 

Brush encoders 

PC 

P 


Potentiometers 

PC 

P 


Resolvers 

PC 

A 


Optical encoders 

PC 

A 


Magnetic encoders 

PC 

A 


Inductive encoders 

PC 

A 


Capacity encoders 

EC 

A 

Heading sensors 

Compass 

EC 

P 


Gyroscopes 

PC 

P 


Inclinometers 

EC 

A/P 

Beacon based 

GPS 

EC 

A 

(postion wrt 

Active optical 

EC 

A 

an inertial 

Radio frequency 
(RF) beacons 

EC 

A 

frame) 

Ultrasound beacon 

EC 

A 


Reflective beacons 

EC 

A 

Ranging 

Capacitive sensor 

EC 

P 


Magnetic sensors 

EC 

P/A 


Camera 

EC 

P/A 


Sonar 

EC 

A 


Laser range 

EC 

A 


Structured light 

EC 

A 

Speed/motion 

Doppler radar 

EC 

A 


Doppler sound 

EC 

A 


Camera 

EC 

P 


Accelerometer 

EC 

P 

Identification 

Camera 

EC 

P 


Radio frequency 
identification RFID 

EC 

A 


Laser ranging 

EC 

A 


Radar 

EC 

A 


Ultrasound 

EC 

A 


Sound 

EC 

P 
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to directly compute the motion and its direction (the 
phasing between sensors A and B in Fig. 5.5). In addi¬ 
tion the disc is frequently fitted with a single dot on the 
outer rim for indexing (specification of a zero index). 
The density of the pattern determines the resolution of 
the measurements. When fitting the sensor to a motor 
before a reduction gear it is easy to achieve accuracies 
beyond 1/1000°. 

For the estimation of force and torque at an end- 
effector it is possible to use piezoelectric elements. 
These elements generate a voltage that is proportional 
to the introduced deformation. Through careful place¬ 
ment it is possible to measure both force and torque. 
The sensors are used in robotic manipulation as part 
of assembly systems, deburring, etc. and also in medi¬ 
cal applications for the estimation of stress and contact. 
Force/torque sensors are widely available in a range of 
sizes and dynamic ranges, including new flexible ar¬ 
rays that can be mounted on a variety of end-effectors 
(Fig. 5.6). These arrays are more commonly referred 
to as tactile sensors as they begin to simulate the hu¬ 
man tactile sense. See Chap. 19 and [5.4,5] for recent 
reviews of the state of the art in tactile sensing. Poten¬ 
tial problems with force sensors are a dead band on 
initial contact, and noisy data from the basic sensing 
elements, which calls for signal processing to clean up 
the data. 

Ego-motion estimation is an important part of al¬ 
most all robotic systems. To this end it is possible to use 
inertial measurement units (IMU). An IMU typically in¬ 
cludes both accelerometers and gyros. Accelerometers 
are sensitive to all types of acceleration, which implies 
that both translation motion and rotation (centripetal 
forces) are measured in combination. Joint IMU units 
allow the estimation of rotation and translation, and al¬ 
low for double integration to estimation the velocity, 
orientation, and position of a system, as for example re¬ 
ported in [5.6]. One of the problems associated with the 
use of an IMU is the need for double integration. Small 
biases and noise can result in significant divergence in 
the final estimate, which calls for use of detailed mod¬ 
els and careful calibration and identification of sensor 
characteristics. An example of data from a cross-bow 
DMU-6x unit for a car driving on an unpaved road is 
shown in Fig. 5.7. 

Much early work on mobile robotics, underwater 
robots, and some medical robotics relies on ultrasonic 
ranging. The general class of sensors are often termed 
sound navigation and ranging (sonars). The general 
principle is that the system emits a sound pulse and 
awaits the return of echoes that have bounced off ob¬ 
jects in the environment. Knowing the transmission 
speed in the medium and the time of flight it is possible 
to compute the distance. The method was widely used 



Fig. 5.6 (a) TactArray, a flexible capacitive array tactile 
sensor from Pressure Profile Systems, Inc., is appropri¬ 
ate for sensing contact locations and areas under sliding 
conditions, (b) Conformable TactArray sensors can fit on 
a human or robotic hand (courtesy Pressure Profile Sys¬ 
tems, Inc.) 


Acceleration 



Fig. 5.7 Example data from an IMU unit for driving on an 
unpaved road 


in early robotics due to the availability of low-cost sen¬ 
sors with adequate performance. In underwater robotics 
this is still a primary sensor. Sonar is discussed in detail 
in Chap. 30. 
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Fig. 5.8 Example laser ranging sensor (SICK LMS291) 
that is widely used in mobile robotics 


Recent progress on environmental modeling and 
navigation has, in many respects, been due to the emer¬ 
gence of low-cost high-fidelity laser scanning systems. 
The SICK series of laser scanners are time-of-flight 
scanners. The scanner sends out a pulse of light and 
measures the time to return. The standard scanner en¬ 
ables estimation of distances up to 80 m at centimeter or 
millimeter accuracy. The scanner measures distances in 
a plane with an angular resolution of 0.5—1°. The field 
of view is 180° resulting in 181—361 range measure¬ 
ments. The sensor data are contaminated by uniformly 
distributed noise, which must be considered in the de¬ 
tection of features or integration of data into a raw 
sensor map (Fig. 5.8). 

Imaging sensors are a rich source of information 
for sensing and estimation. Imaging sensors come in 
a wide variety of configurations, varying according to 
imaging geometry, image resolution, sensor technology 
and the range of sensed spectral bands. Most readers 
are no doubt familiar with the traditional three-CCD, 
perspective color camera. In this case, there are three 
charge-coupled detector (CCD) arrays, each receiving 
a portion of the visible spectrum corresponding roughly 
to the human perception of red, green, and blue colors. 
A common and less expensive alternative is a so-called 
single-chip CCD camera. In this case, a special spatial 
array of color filters, usually referred to as a Bayer filter 
after its inventor Bryce Bayer, is employed. The result¬ 
ing spatial array is subsequently processed (a process 
referred to as demosaicing) to provide color informa¬ 
tion for each pixel. 

In the United States, image sensors traditionally 
contained 480 rows of 640 pixels according to the 
National Television System Committee (NTSC) stan¬ 
dard created for analog transmission of television 
signals. The corresponding European standard, PAL, 
has 576 lines of 768 pixels. More recently, the ad¬ 
vent of digital interfaces such as IEEE 1394 and 




Fig. 5.10 Catadioptric image and the same image mapped 
to a cylindrical surface 


USB 2.0 have allowed camera systems to be de¬ 
veloped with significantly improved resolution rang¬ 
ing into the millions of pixels. At the same time, 
cost-effective infrared (IR) and ultraviolet (UV) cam¬ 
eras have become available, allowing the develop¬ 
ment of advanced multispectral image interpretation 
systems. 

A traditional imaging sensor contains an optical 
system that focuses light on a planar imaging array. In 
most cases, this system can be modeled using the clas¬ 
sical pinhole camera model (shown in Fig. 5.9). Given 
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Fig. 5.11 Picture of proxim and kinect RGB-D sensors 


a point (x, y, z) T in Euclidean space, the corresponding 
camera pixel coordinates ( u , v) T are given by 

, \ f x 

(U-Uc) = -. 

s x z 

(v — v c )=—-, (5.1) 

Sy Z 

where/ is the focal length of the lens system, u c and v c 
are the pixel coordinates of the center of projection, 
and s x and s y are the size of a single pixel on the imag¬ 
ing array. In practice, these models are also augmented 
with low-order models of image distortion. The val¬ 
ues of these parameters for a given camera system can 
be determined experimentally using a variety of meth¬ 
ods [5.7], 

By combining a traditional perspective camera with 
a mirror, creating a so-called catadioptric system, it is 
possible to create imaging geometries that map fields of 
view as large as a hemisphere into a single image. Such 
systems are useful, for example, for surveillance, and 
their geometric properties provide for stable position 
referencing for mobile navigation [5.8]. An example 
image is shown in Fig. 5.10 together with the corre¬ 
sponding image when mapped to a cylindrical surface. 

Active ranging cameras, which combine images 
from an optical camera with a dense range map, are now 
also widely available and quite cost effective. These 
systems perform triangulation between a light projec¬ 
tor that throws a pattern with known structure into the 
scene, and a camera that views the scene and detects 
the pattern. By matching or correlating the pattern ele¬ 
ments between the projector and the camera, and using 
the known relationship between them, it is possible 
to recover depth using standard methods. The cam¬ 
era/projector pair for depth recovery typically operate 
in the infrared to avoid the pattern being visible to 
the human eye. Thus, these systems typically include 
a third visible light camera to acquire a corresponding 
color image. Hence the name RGB-D (red-green-blue- 
depth) cameras. Fig. 5.11 shows two recent low-cost 
RGB-D products. In Fig. 5.12 is show the fused output 
for a tabletop scene. 

The discussion above has touched on the most 
commonly employed robotic sensing devices. Many 
special-purpose sensors are employed for specific ap¬ 
plications. In medicine (Chap. 63), ultrasound. X-ray, 



Fig. 5.12 Example of fused range images from RGB-D 
camera and the same scene as an intensity textured mesh 


computed tomography, and magnetic resonance imag¬ 
ing are commonly employed. Underground mapping 
makes use of ground-penetrating radar [5.9]. Underwa¬ 
ter robotics makes use of many variations on acoustical 
sensors. Further discussions of these more task-specific 
sensing modalities can be found in the application chap¬ 
ters in Part C of this handbook. 
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5.4 Estimation Processes 

As discussed in the introduction, there are many differ¬ 
ent techniques for combining information from sensors. 
The appropriate set of techniques depends, to a great de¬ 
gree, on what is known a priori about the environment, 
what information is necessary for the task at hand, 
and what models for the sensing system are appro¬ 
priate. Common methodologies include simple voting- 
based methods, parametric and nonparametric statisti¬ 
cal estimation techniques, fuzzy logic-based systems, 
and Dempster-Shafer theory. 

To illustrate this point, consider the robot localiza¬ 
tion problem introduced in Sect. 5.2. At the outset, if 
nothing is known about the environment, the robot may 
acquire a laser scan and try to produce an initial model 
of the environment using line segments. Since nothing 
is known a priori, the system must estimate: 

1. The number of line segments 

2. The data association between line segments and ob¬ 
served data values 

3. The parameters of the line segments themselves. 

This is a challenging problem that can be at¬ 
tacked by simple voting techniques such as the 
Hough transform [5.10] or random sample consensus 
(RANSAC) [5.11] or more sophisticated unsupervised 
clustering methods such as £:-means [5.12], expecta¬ 
tion maximization (EM) [5. 1 3] , or generalized principal 
component analysis (GPCA) [5.14]. In many cases, this 
is a computationally intensive, iterative process. 

Conversely, if a prior CAD model for the environ¬ 
ment is known, then the problem is to produce a small 
set of parameters (translation and rotation) of the 
model to match the data. This problem can be solved, 
using feature matching by aligning observed points 
to the model with iterative closest-point algorithms 
(ICP) [5.15] or other efficient combinatorial matching 
algorithms such as Monte Carlo methods [5.16]. The 
best method to apply again depends to a great degree 
on the structure of the environment and what is known 
a priori. 

Once an initial registration is known, new data can 
take advantage of the fact that strong prior knowl¬ 
edge is available. In particular, as the robot moves, 
the sensor data should change in a predictable fash¬ 
ion. Thus is it possible to make use of predictor- 
corrector methods such as the Kalman filter [5.17, 
18] or sequential importance sampling [5.19], provided 
appropriate statistical characterizations of the sensing 
system are available. The data association problem, if 
present, can be addressed using a variety of general 
techniques such as EM [5.12] or more specialized mod¬ 


ifications to the previously cited predictor-corrector 
methods [5.20]. 

It is often the case that sensor data is corrupted by 
occasional nonsensical values. For example, the laser 
range finder in our example may occasionally return 
a spurious range value due to a reflection. Many com¬ 
monly used estimation techniques are not robust to such 
so-called data outliers. Techniques from robust statis¬ 
tics [5.21] can be used to improve the performance of 
sensing and estimation systems in such cases. 

Finally, we may want to consider what information 
is actually important for the task at hand. Most of the 
techniques above presume that the goal is to produce 
an accurate estimate of a set of continuous parame¬ 
ters closely related to the underlying data. However, in 
some tasks, the parameter values themselves may not 
be what is of interest. For example, suppose that the 
goal of our robot is to drive through a doorway. Al¬ 
though this clearly depends on an ability to estimate the 
width of the door (a continuous parameter), the decision 
is ultimately binary. This problem can be codified as 
a decision problem. Decision problems can be modeled 
using concepts from decision theory [5.22] including 
zero-one loss functions, likelihood ratios, or probabil¬ 
ity ratios. For example, in the case of fitting through 
a door, for a low-priority task there may be a low 
cost associated with not attempting to move through 
this particular door (necessitating replanning to find 
an alternative route) relative to attempting to navigate 
through an opening that is too small (risking damage to 
the robot or the door, or both). Conversely, if the task is 
urgent, more risky behavior may be warranted. 

For any given task (or decision), the amount of in¬ 
formation necessary to reach the decision may vary, for 
example, if the doorway is quite wide, it may require 
relatively little information to safely navigate through 
it. Conversely, a tight fit may require close inspec¬ 
tion before a decision can be reached. The problem of 
determining the type and/or amount of information nec¬ 
essary to reach a decision is referred to variously as the 
sequential sampling problem [5.22], the sensor control 
problem, or the sensor planning problem [5.23-25]. 

5.4.1 Point Estimation 

In our robot localization example we saw several cases 
where the key problem was to estimate an unknown 
quantity that can be represented as a point in a vector 
space. Examples include the location of a two-dimen¬ 
sional (2-D) or three-dimensional (3-D) point or the 
location of a robot. We also saw examples where the 
problem was to locate the pose (position and orien- 
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tation) of the robot, or parameters of a line segment. 
The latter differ in that the underlying parameter space 
is not a vector space. This introduces some additional 
unique problems. We refer the reader to [5.26,27] for 
further discussion, and in the remainder of this chapter 
restrict our attention to point estimation problems on 
vector spaces. In our discussion, we assume the reader 
is familiar with multivariate Gaussian distributions as 
described in [5.28] and basic linear algebra [5.29]. 

In the remainder of this section, we consider the fol¬ 
lowing general problem. 

Given: an observation model 

y =f(x, tj). (5.2) 

Estimate : xeRe(n) from observations v e Re(m) 
where r] is an unknown disturbance taking values 
in Re(k) and / is a known mapping from R e(k + n) 
to Re(m). 

We divide our discussion into two topical areas: 

• Methods for performing estimation on batch and se¬ 
quential data when/ is linear, 

• Methods for performing estimation on sequential 

data when/ is nonlinear. 

Estimation Techniques for Batch 
and Sequential Data with Linear Models 
In this section, we discuss linear and linearized es¬ 
timation techniques for sequential data, including the 
Kalman filter and extensions thereof. Our goal is to pro¬ 
vide an overview of techniques available. The reader 
may also wish to consult more in-depth references such 
as [5.18, 30, 31] and (Chap. 35) for additional informa¬ 
tion. 

We first consider the case when/ in (5.2) is linear 
in its arguments. In this case, we can write 

y=Fx + Bq, (5.3) 

where F e Re(m x n) defines the (linear) relationship 
between the unknown x and the observation y and B e 
Re(m x m). For the moment, we will drop B and assume 
that 4 represents the complete disturbance model of the 
system. 

The least-squares method of estimating x from y 
proceeds by solving the optimization problem 

min ||Fjc — y|| 2 . (5.4) 

X 

This optimization has a unique solution x if and only 
if the matrix F has full column rank. In this case, the so¬ 
lution can be computed by solving the following linear 


system 

F t FT = F t v . (5.5) 

In some cases, there may be reason to believe that some 
observed elements are more reliable than others, and 
hence should contribute more to the final estimate. This 
information can be incorporated by modifying (5.4) to 
include a diagonal positive-definite weighting matrix W 
as 

min (Fx: — y) T W(Fx — y) . (5.6) 

X 

The solution is then given by solving 

(F t WF)T = F T Wy . (5.7) 

Although (5.3) included a disturbance component 
(in the form of q), the parameter estimates computed 
in (5.5) or (5.7) made no explicit use of this quantity. 
However, we can often model the noise characteristics 
of the underlying sensor using a statistical model and 
recast our original estimation problem to incorporate 
this information. One common method is to com¬ 
pute the maximum-likelihood estimate (MLE), which is 
a value x such that 

p(y\x) = maxp(y\x) . (5.8) 

X 

For the linear additive model of (5.3), the likelihood 
function can be expressed in a particularly simple form. 
Suppose that q is described by a fixed, known probabil¬ 
ity density function D. The likelihood function is then 
given by 

p(y\x) = D(y-Fx) . (5.9) 

The MLE can be related to the previous least- 
squares method as follows. Suppose that q ~ N( 0, A), 
where N denotes a multivariate Gaussian density func¬ 
tion with (mean) 0 and covariance A . Upon observing 
that the maximizing the value of the likelihood function 
is equivalent to minimizing the negative log of the like¬ 
lihood function, a short series of calculations shows that 
the optimal maximum-likelihood estimate is computed 
by weighted least squares with W = A -1 . 

Finally, there is often a reason to include the idea 
that some parameters are more likely a priori to occur 
as others. For example, when observing a car driving 
on an expressway, a velocity of 60 mph is much more 
likely than either 20 or 300 mph. This information can 
be captured in prior statistics on the unknown value x. 
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Given a prior probably density on x, p(x ), Bayes the¬ 
orem states that 


Taking derivatives with respect to K and setting 
them equal to zero yields the solution 


= p(y\x)p(x) = p(y\x)p(x) 

P(y ) J P(y\x)p(x)dx ' 


(5.10) 


The maximum a posteriori probability (MAP) estimate 
is the value x such that 


p(x\y) = maxp(x\y) . (5.11) 

X 

In general, the solution to this optimization problem can 
be quite complex. Rather than pursue this course fur¬ 
ther, we consider another alternative. Namely, provided 
the second moments of p(x\y) exist, it is possible to pro¬ 
duce a least-squares estimate, in a statistical sense, by 
solving the following optimization problem over an un¬ 
known function S 

mini: 11<5(y) — *|| 2 . (5.12) 

s 

That is, the best function 8 is one that produces an es¬ 
timate of x from y with minimum mean-square error 
(MMSE). Thus, the estimator 8 is often referred to as 
an MMSE estimator. 

It can be shown that, in the general case, the optimal 
decision rule 5* is the conditional mean [5.22] 

8*(y) =E[x\y\ . (5.13) 

Unfortunately, this expression, as with the MAP esti¬ 
mate defined above, can be extremely difficult to com¬ 
pute for the general case. Later we consider methods 
for computing approximations to (5.13). For now, we 
again consider our previous linear observation model 
(5.3) (without B). Additionally, we suppose that x and q 
are independent random variables with finite second 
moments, and both are zero-mean random variables. 
Note that the latter is not really a restriction since it 
can be accomplished by simply defining a new variable 
x' = x—E[x], Finally, we will consider only linear func¬ 
tions 8, that is, we can write x, = <5(y) = Ky. 

With this, (5.12) can be expanded as 


E ||<5(v) — x || 2 = E ||K>’ — jc|| 2 

= E ||K(Fx+ r\) —x\\ 2 
= E\\(KF-I)x\\ 2 +E\\Kq\\ 2 
= tr[(KF-/)A(KF-/) T + K51K T ]. 

(5.14) 

Here, the independence of x and q and the fact that they 
are both zero mean has eliminated several terms. The 
final step makes use of the fact that ||jc || 2 = tr(x* T ). 


K= AF T (FAF T +S) _1 . (5.15) 

Thus, in this case the optimal estimate is given by a lin¬ 
ear function of the observation, where the linear term 
depends only on the variance of the underlying random 
variables and the linear term defining the observation 
system. 

If x is not zero-mean, but has mean pt, it is not hard 
to show that the optimal estimate is 

x= Kv+ (I —KF)M , (5-16) 

and that the variance of the estimate A+ is 

A+=(I —KF)A. (5.17) 

The interested reader may wish to work this out for 
a few simple cases, for example, if A = T, and F = I, 
K = 1 / 21 and thus x = y + pt - a simple average - with 
variance A + = 1 /2 A. 

When both the observation noise and prior statistics 
are Gaussian distributions, then it can be shown that the 
solution we have derived is also the MAP estimate for 
the unknown* [5.22]. 

The Kalman Filter 

With this as background, we are now in a position 
to define the discrete-time Kalman-Bucy filter [5.32] 
for linear systems. Consider the following time-series 
model 


x ,+1 = Gx t + w ,, (5.18) 

y t = F.r, + q ,, (5.19) 

where G is an n x n matrix describing the system time 
evolution and xq is distributed according to a Gaussian 
distribution with mean *o and variance Ao- In addi¬ 
tion w, and q t are zero-mean Gaussian independent 
random variables for all t, w, is independent of uy 
for all t t ', and likewise q, is independent of q,f for 
all t t'. Finally, q, has variance T , and w t has vari¬ 
ance 

Given an observation y\ it is possible, using the 
derivation of the previous section, to compute an up¬ 
dated estimate x\ with variance Ai. Note, that the 
solution is a linear combination of two Gaussian ran¬ 
dom variables: the observation value y l and the prior 
estimate xq. As any linear combination of Gaussian ran¬ 
dom variables is also a Gaussian random variable, it 
follows that the updated estimate is also Gaussian. 
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Fig. 5.13 A summary of the Kalman filter 


Now, we add one additional step: projection through 
the dynamics model. To describe this, superscripts mi¬ 
nus and plus will denote before and after the estimation 
step, respectively. Thus, given an estimate xf" with vari¬ 
ance A, , the projection ahead one time step produces 

x~ +l =Gxf , (5.20) 

\~ +1 = GA + G T + J2,. (5.21) 

At this point, a new observation y , 4.1 is acquired and 
the cycle repeats. The summarization of the complete 
Kalman filtering algorithm for linear systems is shown 
in Fig. 5.13. 

It is possible to show that the Kalman filter is the op¬ 
timal filter, under the stated assumptions, in the mean- 
square sense. It is also the optimal linear filter when 
either or both Gaussian assumptions do not hold. 


the extended Kalman filter (EKF) by making use of the 
Taylor-series expansion of the nonlinear elements about 
the current estimates. Let 3/ (resp. J g ) denote the Jaco¬ 
bian matrix of the function / (resp. g). Supposing that 
an estimate at time step t— 1 exists, the first-order ex¬ 
pansion of (5.22) about this point yields 

x,+i = g/Cri-i) + 3 g {x,-i)(x,-x t+] _) + w, , (5.23) 
y t =ft(x-i) + Jf(x t -i)(x,-x,-i)+ rj,. (5.24) 

Rearranging yields a linear form appropriate for the 
previously defined Kalman filter 

Xt +1 = Jt,+i - g t (xt- 1 ) + 3 g ,x t -i = 3 gl x, + w t , 

(5.25) 

yt = yt -ft(x t - 1 ) + 3f,x t -\ = J f ,xt + th ■ 

(5.26) 

In this form, x and y are new synthetic state and observa¬ 
tion variables, 3 g (x t —\ ) plays the role of G, and 3f{x t —\) 
plays the role of F. 

It is worth noting that the EKF iterations are es¬ 
sentially a form of weighted Newton iterations (i. e., an 
iterative nonlinear estimation method). As a result, it is 
often useful to iterate more than once on the same ob¬ 
servation while holding the variance terms fixed. This 
allows the estimator to converge to a solution in the 
presence of large disturbances or significant nonlinear¬ 
ities. Only after convergence are the variance terms 
updated. This version of the Kalman filter is referred 
to as the iterated extended Kalman filter (IEKF). 

5.4.2 Other Approaches to Estimation 


Nonlinear Estimation Techniques 
for Sequential Data 

The results of the previous subsection presume a linear 
form for the relationship between the observation and 
system state, additive noise, and a linear relationship 
describing the state evolution. Furthermore, the stated 
results are globally optimal for systems with Gaussian 
observation and driving noise, but are only the best lin¬ 
ear estimator if the noise sources are non-Gaussian. 

As noted at the outset, the more general nonlinear 
(discrete-time) system description is 


x t +l = gt(x t ) + w ,, 

y, =f t {x,) + t], , (5.22) 

where, for the moment, the noise model continues to be 
additive. 

Although this model contains nonlinear elements, it 
is still possible to apply a variant of the Kalman filter. 


In the previous section, we reviewed a common and 
widely used estimation method. However, there are sev¬ 
eral alternative methodologies for solving parameter 
estimation problems. Here we briefly introduce two: se¬ 
quential importance sampling and graphical models. 

Sequential Importance Sampling 
Much of the discussion heretofore has centered around 
the notion of approximating everything known about 
the system state using an estimated mean and covari¬ 
ance. An alternative presents itself by simply going 
back to Bayes theorem which states, in general, that 


p(x n \yi,y 2 ...y n ) 


piyuyi • • -ynMplXn) 
piyi > yi ■ ■ •>«) 


(5.27) 


Assuming that y n is independent of all prior observa¬ 
tions and states given x n , and that x n is independent 
of x n —k for k > I given x n —\, this expression simplifies 
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to 


p(xn\x n — i, y w ) 


P(y,,\Xn)p(Xn | *„-l) 

p(y„ | X„-l ) 


(5.28) 


Recall that the optimal mean-square estimate is 
given by the conditional mean which, in this case, is 


8*(y n ) = E[x n | v„] . (5.29) 

In fact, we essentially showed that the Kalman filter is 
a special case of this result for linear systems corrupted 
by Gaussian noise. 

The difficulty in implementing this procedure in the 
general case ultimately comes down to the problem 
of representing and computing with the distributions 
that arise in the nonlinear, non-Gaussian case. However, 
suppose that the heretofore continuous variable x n only 
took on a discrete set of values. In this case, computing 
Bayes theorem and other associated statistical quanti¬ 
ties reduces to a straightforward set of computations on 
this discrete set of variables. This can be simply done 
for any distribution and any set of transformations. 

Sequential important sampling (also known as parti¬ 
cle filtering, condensation, and a variety of other names) 
is a way of applying this approach to continuous vari¬ 
ables in a statistically sound manner. In order to perform 
sequential importance sampling, it is assumed that: 


1. It is possible to sample from the likelihood function 
P(y n I x n ), and 

2. It is possible to sample from the dynamical model 
P{x n j x n —}). 


Note the emphasis on sampling - there is no need to 
explicitly exhibit an analytical form of the likelihood 
function or of the dynamical model. 

Given this, sequential important sampling, in its 
simplest form, can be written as follows: 

1. Let jr n —i = {(x^ l _ l ,wj c l _ l ),k = 1,2,.. .N} repre¬ 
sent a set of sample points x k n _ l together with a set 
of weights w k _ l with Y w k _ l = 1. 

2. Compute a new set of N samples n = 
{{^,\/N),k= 1,2,.. .IV} as: 

a) Choose a sample point JfZ.\ with probability 
proportional to its weight w k ~ 1 ; 

b) Sample from P(x n |*J;_ 1 ) given jcJ with 
weight 1 /N; 

3. Compute n n = {(ji *,P(y„ | x? n )),k = 1,2,.. .N}. 

It is easy to see that this set of steps is now in the form of 
a recursive filter. Furthermore, at any time any statistic 
of the associated distribution can be approximated from 
the set of samples and associated weights. 


Sampling-based filters of this form have found wide 
applicability in a variety of challenging areas where 
linear estimation techniques do not suffice. These tech¬ 
niques have been particularly successful, for problems 
with low state dimension (typically n < 3) and well- 
constrained dynamics. For higher-dimensional prob¬ 
lems or systems exhibiting high dynamic variability, 
the number of particles necessary to obtain good ap¬ 
proximations can become prohibitively large. How¬ 
ever, even in these cases, sampling-based systems can 
sometimes be engineered to produce acceptably good 
results. 

Graphical Models 

Graphical models are a class of models that repre¬ 
sent dependence and independence relationships among 
a set of variables. Common examples of graphical mod¬ 
els include Bayes nets, influence diagrams, and neural 
nets. Graphical models are quite general - indeed much 
of this chapter could have been written by first defin¬ 
ing graphical models, and exploring specializations that 
lead to the Kalman Filter, for example. Here, for reasons 
of space, we focus on Bayes nets as a specific example 
of graphical models. 

A Bayesian network is a directed acyclic graph con¬ 
sisting of nodes representing random variables, and 
directed arcs representing probabilistic relationships 
between pairs of random variables. Let parents(X) de¬ 
note the set of nodes which have arcs terminating at X, 
and let X \, X 2 ,..., X N be the N random variables in the 
graph. Then we can write 


N 

P(X U X 2 . X N ) = n / J ( X, I parents®) . 

i= 1 

(5.30) 

For example, a Bayesian network representing a mo¬ 
bile robot performing localization is shown in Fig. 5.14. 
This graphical model encodes the sequential form of the 
problem and is thus an example of a so-called recurrent 



Fig. 5.14 An example of robot localization expressed as 
a graphical model 
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network. More discussion of such models can be found 
in [5.33]. 

The structure of a Bayesian network encodes var¬ 
ious independence relationships among variables. By 
exploiting these independence relationships, it is pos¬ 
sible to design efficient inference algorithms. In partic¬ 
ular, graphs which are acyclic even in their undirected 
form (referred to as polytrees) admit linear time infer¬ 
ence algorithms. More general graphs can be solved 
using various types of iterative methods. In particular, 
if the distributions in the network are of a continu¬ 
ous type, variations on sequential importance sampling 
can be used to solve problems in an approximate 
sense [5.34]. 

Conditional Random Fields 
In many cases of interest, including many of the ex¬ 
amples in this chapter, the end goal is to infer or 
predict a value or label from observed data. We might 
then frame the problem by exploring the joint distri¬ 
bution P(X, Y) where X represents some data that is 
observed and Y is what we would like to infer. Recall 
that 


P(X, Y) = P(Y\X)P(X) . 

If we take X = x for some observed values x, 
then we see that P(X) becomes constant, and infer¬ 
ring a value for Y depends only on P(Y\X). If we 
were to apply a Bayes Net to this problem, the model 
would represent the complete joint probability distribu¬ 
tion on X and 7, what is referred to as a generative 
model. But, if we know X is always observed, then 
much of this structure is irrelevant to our problem - we 
don’t care about the probabilistic structure of X. This 
observation has given rise to a specialization of graph¬ 
ical models referred to as Conditional Random Fields, 
or CRFs for short. 

The immediate value of CRFs is their economy and 
expressivity compared to graphical models. This has 
immediate positive implications for the complexity of 
both learning and inference. Traditionally CRF models 
are learned using maximum likelihood-based methods 
using gradient descent or other unconstrained optimiza¬ 
tion techniques. However, recent methods like Cutting 
Planes [5.35] and Block Coordinate Frank Wolfe [5.36] 
pose it as a constrained optimization problem in the 
form of a Structural Support Vector Machine. These 
techniques tend to be more computationally efficient 
and are often more accurate. 

CRFs have proven to be very general, and are 
now extremely widely used for image processing, nat¬ 
ural language processing, video processing - nearly 
any problem where there is a series of data elements 


a) 



b) Micro accuracy (%) 



Skip-length (frames) 


Fig. 5.15 (a) A skip chain CRF for inferring symbolic labels (Y) 
from robot kinematic and video data (X) acquired while a user 
performed a surgical training task, (b) The change in classification 
accuracy as a function of the skip length (images courtesy of Colin 
Lea) 


from which some prediction is to be performed. For 
example. Fig. 5.15 shows the graphical structure of 
a skip-chain CRF designed to compute gesture labels 
from kinematic and video data acquired from a surgi¬ 
cal robot [5.37]. This can be viewed as a discriminative 
generalization of a Hidden Markov Model (a genera¬ 
tive model) that is designed to capture dependencies 
over a specified period of time (the skip). The right side 
shows the change in labeling performance as a function 
of the skip length which is now a tunable parameter of 
the model. 

An in-depth discussion of CRFs goes well beyond 
this chapter. The interested reader is referred to [5.38, 
39] to learn more about the underlying theory and appli¬ 
cation of CRFs. Because of their high interest, there are 
a number of open-source packages for developing and 
applying CRFs including PyStruct [5.40] for Python 
and CRF++ for C++. 

5.4.3 Robust Estimation Methods 

In our previous discussions, we generally assumed that 
all of the data was good, meaning that it was perhaps 
corrupted by noise but ultimately carried information 
about the problem at hand. However, in many cases, 
the data may contain so-called outliers - data points 
that are either much more highly corrupted than typical 
data, or which are completely spurious. For example, 
in our mapping application we might occasionally ob¬ 
tain range data through multiple reflections. Thus, while 
scanning a straight wall, most of the points would lie on 
a straight line, but occasionally we would have a data 
point that has a completely inconsistent range value. 

Many common estimation methods are quite sen¬ 
sitive to data outliers. Consider a very simple case: 
estimating a single scalar value x by averaging a series 
of observations X\,X^,.. .X N . Then we can write our 
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estimate x as 

N 

x=J2 X i/N- (5.31) 

;= 1 

Now, without loss of generality, suppose that X N is an 
outlier. We can rewrite the above as 

N— 1 

x=y ' Xj/n + Xtf/n . (5.32) 

It is now easy to see that we can produce any value of x 
by manipulating X n . In short, a single outlier can create 
an arbitrarily poor estimate. More generally, the solu¬ 
tion to any least-squares problem, e.g., estimating a line 
from laser range data, takes the general form x = My. 
By the same argument as above, it is easy to show 
that any least-squares solution is likewise susceptible 
to outliers. 

The field of robust statistics studies the problem of 
estimation or decision making when the underlying data 
are contaminated by outliers. In robust statistics, there 
are two important concepts: the breakdown point and 
influence function. The breakdown point is the propor¬ 
tion of outliers (i. e., data with arbitrarily large errors) 
that an estimator can tolerate before producing arbitrar¬ 
ily large errors in an estimate. We argued above that 
least-squares methods have a breakdown point of 0% 
since the estimate can be perturbed arbitrarily far by 
a single observation. By comparison, we might com¬ 
pute an estimate by taking the median of the data, which 
has a breakdown point of 50% - up to half of the data 
can be outliers and meaningful results may still be pro¬ 
duced. 

Whereas the breakdown point quantifies how many 
outliers can be tolerated, the influence function quan¬ 
tifies how much an outlier affects an estimate. In the 
case of least squares, the influence function is linear. 
One way of creating new estimators with better robust¬ 
ness is the method of M-estimators [5.21]. To produce 
an M-estimate, we consider the following minimization 
problem 

N 

min V p(x,y,) . (5.33) 

* ;=l 

Note that defining p(a, b) = (a — b) 2 leads to a least- 
squares solution. However, we can now choose other 
functions with better resistance to outliers. Fig. 5.16 
shows three common examples. 

Note that, in general, the optimization of (5.33) 
is nonlinear and the result will often not exist in 
closed form. Interestingly, it is often possible to 


a) 




Fig. 5.16 (a) Three common robust M-estimation func¬ 
tions: the square function, the absolute value, and the 
Tukey biweight function, (b) The corresponding influence 
functions 

solve this problem using the method of iteratively 
reweighted least squares (IRLS) [5.30,41]. The idea 
behind IRLS is quite simple. Recall that in (5.7) we in¬ 
troduced a weighting matrix W. Suppose that, through 
some means, we knew which data points were out¬ 
liers. In this case, we could simply set the weights 
for those points to zero, and the result would be 
the least-squares estimate on the remaining (good) 
data. 

In IRLS, we alternate between hypothesizing out¬ 
liers (through reweighting) and solving to produce a so¬ 
lution (through least squares). Typically, the weight for 
a point depends on the residual error of the estimate. 
That is, suppose we compute 

r = y — ¥x. (5.34) 

Let fl(y) = dp/dx |~; then we can set W M - = \fr(y)/rj. 
It can be shown that in many cases this form of 
weighting will lead to convergence. An example of 
using IRLS techniques for video tracking is shown 
in Fig. 5.17. 

Voting-Based Methods 

Another common method for dealing with outliers is to 
choose a set of data and let it vote for a result. We dis- 
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Fig.5.17a-c An example of using an M-estimate imple¬ 
mented via IRLS for visual tracking (after [5.42]). (a) Re¬ 
sults of a face tracker in a single frame of video. The black 
frame corresponds to a tracking algorithm without outlier 
rejection and the white frame corresponds to the algorithm 
with outlier rejection, (b) Magnified view of the region in 
the white frame; (c) the corresponding weighting matrix in 
which darker areas mark outliers 

cuss two common methods: RANSAC [5.11] and least 
median of squares (LMedS) [5.43]. 

In both cases, we start with the idea that, amidst 
all of the data (including outliers), there is an estimate 
that is consistent with the good data. The problem is to 
choose that estimate. Consider, however, our problem 
of estimating a line from laser data, and suppose we 
have 100 laser points. All we really need is to choose 
two points correctly, fit a line, and then count how many 
other points are consistent with this line. If we (conser¬ 
vatively) estimate that 3/4 of the data is good, then the 
odds of choosing two good points is 9/16, or equiv¬ 
alently, the odds of one or both points being outliers 
is 7/16. If we now repeat this process a few (e.g., ten) 
times, then the odds that all of our choices are bad 
is (7/16) 10 = 0.025%. To put it in other terms, there 
is a 99.975% chance we have chosen a good pair of 
points. 

How do we decide to accept a sample? In RANSAC, 
we vote by counting the number of samples that are 
consistent with an estimate to within a given distance 
threshold. For example, we would choose points that 
are within a fixed distance to the line we estimated. We 
choose the candidate estimate with the largest number 
of votes. In LMedS, we instead compute the median dis¬ 
tance of all of the samples to the line. We then choose 
the estimate with the least median value. 

It is not hard to see that LMedS has a breakdown 
point of 50% of the data. RANSAC, on the other hand, 
can have a breakdown point that is potentially larger, 
but it requires the choice of a threshold. RANSAC also 
has the advantage that, once the inliers are identified, 
it is possible to compute a least-squares estimate from 
them, thus reducing the noise in the estimate. 

Both RANSAC and LMedS can also provide good 
starting solutions for a robust iterative method such as 
IRLS. 


5.4.4 Data Association Techniques 

The previous section considered the case where there is 
a known relationship between observations and a quan¬ 
tity to be estimated. However, as was illustrated in our 
initial mobile robot mapping problem, it may be the 
case that we also have to compute this correspondence 
in conjunction with estimation. In this case, an essen¬ 
tial step in estimation is the data association problem: 
producing a correspondence between the observed data 
and quantities to be estimated. 

The literature on this problem is enormous; here we 
will focus on a few specific methods that have found 
wide use. We will also separate our discussion into 
causal (or sequential) association methods commonly 
used when filtering time-series data and noncausal (or 
batch) methods that can be used when the complete data 
set is available for processing. The latter is typically 
treated with methods for data clustering. 

In both cases, we can extend our previous models 
and notation to include uncertainty as to the underlying 
source of the data. To this end, we will use a superscript 
on quantities to denote the observation model. Thus, our 
observation model becomes 

4+1 = /( 4 ) + w t > < 5 - 35 ) 

4 =/?( 4 )+ 4 . ( 5 - 36 ) 

where k = 1 ... M. 

Clustering on Batch Data 

Following the same course as our previous discussion 
on point estimation, let us first consider the case where 
we do not make any statistical assumptions about the 
data, and we have no system dynamics. Thus, we are 
simply given the observations y\,yi,... ,yM- We have 
unknown underlying parameters x 1 ,x 2 ,... ^ (for the 
moment, we take N as known). Our goal it to compute 
an association mapping n such that n(j) = k if and only 
if yj arose from the model parameters x k . 

/(-Means Clustering 

The k-means algorithm for clustering and data asso¬ 
ciation is simple, well established, and forms a good 
starting point for our discussion. Here, we assume that 
f(x) = x - that is, we are provided with noisy obser¬ 
vations of the underlying state vectors. The k-means 
algorithm then proceeds as follows: 

1 . Pick N cluster centers {x 1 }. 

2. For each observation }j, associate it with the closest 

cluster center, that is, set n(j) = i, where 

d(x',yj ) = min dffi.yj) 
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for some distance function d (typically the Eu¬ 
clidean distance). 

3. Estimate the mean of the observation associated 
with each cluster center as 

*'= y (5 - 38) 

j, 7 V(j)=i 

4. Repeat steps 2 and 3. 

In many cases and with good initialization, £-means 
works quite well. However, it can also fail to produce 
good clusters, and there is no guarantee that it will even 
converge to a solution. It is common to repeat the algo¬ 
rithm several times from different initial conditions and 
take the result that has the best outcome. Note also that 
the extension to linear observation models is straight¬ 
forward by including F in (5.3) by defining 

d&,yj) = ||F^—)j|| (5.39) 

and replacing (5.38) with the corresponding least- 
squares estimator. Going a step further, if we have a sta¬ 
tistical model for observed data, then we could make 
use of the likelihood function introduced earlier and 
define d(x',yj ) = p(yj\x') and make use of the MLE in 
(5.38). 

One disadvantage of the Gmeans algorithm is that, 
even when we have known statistical models, it is not 
guaranteed to converge. However, a variation, known as 
expectation maximization, can be shown to converge. 

Expectation Maximization for Data Association 
and Modeling 

The expectation-maximization (EM) algorithm [5.44] is 
a general statistical technique for dealing with missing 
data. In previous discussion, we made use of maxi- 
mum-likelihood estimation to maximize the conditional 
probability of observed data given a set of unknown 
parameters. However, our use of MLE presumed that 
we had complete knowledge of the data. In particular, 
we knew the association between the data elements and 
models. 

Let use now assume that some of our data is miss¬ 
ing. To this end, define l/o and Vu as the observed and 
unobserved data, respectively. We then note that we can 
write 

p(Vo, VuM =p(Vu\Vo,x)p{y 0 \x). (5.40) 

Suppose now that we make a guess for x, and we 
have a distribution over the unknown data f/u (where 
this comes from we will discuss in a minute). It follows 
that we could compute the expected value of the log- 


likelihood function (recall that maximizing the log like¬ 
lihood is equivalent to maximizing the likelihood) as 

Q(x,x) = E Vl] pogpG/o, Vv\x)\Vo,x] . (5.41) 

Note that we differentiate between the fixed value x that 
is usually needed to define the distribution over the un¬ 
known data and the unknown x of the log-likelihood 
function. 

Ideally, we would then like to choose values for x 
that make Q large. Thus, we can choose a new value 
according to the iterative rule 

Xj = argmax£2(jc,.x;_i) . (5.42) 

X 

What can be shown is that this iteration will converge 
to some local maximum of the objective function Q. It 
is important to note that there is no guarantee that this 
is, however, the global maximum. 

How do we connect this with clustering? We con¬ 
sider the observed data to be just that, the data we 
have observed. Let the unobserved data be the associa¬ 
tion values n(j), j = I, 2,... M that determine which 
model the observed data items originate from. Note 
that this is a discrete random variable. Let us fur¬ 
ther assume that N underlying clusters are distributed 
according to a Gaussian distribution with mean x,- 
and covariance A,. Let the unconditional probabil¬ 
ity that a particular data item v 7 - comes from clus¬ 
ter i be a,. The unknown parameters are then 0 = 

{xi, x 2 ,..., x N , A u A 2 , - A n , «!, a 2 , ..., 0 ;^}. We 

now use ~ and + to denote prior and updated parameter 
estimates, respectively. For conciseness, we also define 
Wij = p(nj = i\yj, 9) and we use a superscript to de¬ 
note updated parameter estimates. Then, after a series 
of calculations [5.44], the EM algorithm for data clus¬ 
tering becomes 


E-Step : 


Wij = 

E,pCv/kO') = u 0 )oii ' 

(5.43) 

M-Step : 



r“l” — 

Ejyjwj 

(5.44) 

x i 

’ 

A + = 

E/.h(.v/)'«V.i 

E/ 

(5.45) 

«+ = 

E/ ^j 

(5.46) 

Ei E; Wij ' 


From this, we can see that EM produces a type of 
soft clustering, as opposed to A-means which produces 
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Fig.5.18a-d An example of clustering using expectation maximization. The figures are the results at iterations (a) 1, 
(b) 2, (c) 5, and (d) 10 


specific decisions (in terms of uijj) as to which cluster 
an observation belongs. In fact, the result of estimation 
is the maximum-likelihood estimate of a Gaussian mix¬ 
ture model which has the form 

p(y\0) = ^ otjN(y\xj, Aj) , ( 5 . 47 ) 

J 

where N(-) denotes a Gaussian density function. 
Fig. 5.18 shows the results of executing the EM algo¬ 
rithm on data sampled from a Gaussian mixture model. 

Recursive Filtering 

In the batch methods described above, we do not have 
a priori information on state parameters. In the case 
of recursive filtering, we have the advantage that prior 
state estimates, $P t and A k t , are available for process¬ 
ing at time t+ 1. As before, for data y\,i= 1... N, 
the problem is to determine a mapping n : {1 ... N} —>■ 
{1... M} which associates data element i to model 
k = 7t{i). In some cases, it is also useful to include 
an outlier process to handle data that comes from no 
known model. For this purpose, we can include 0 in the 
range of the function, and use the mapping to zero as an 
outlier. 


Nearest-Neighbor Association 
Analogous to k-mean clustering, a simple way of pro¬ 
ducing a data association is to compute the data associ¬ 
ation value as 

jr(i) = argmin(i(F'.F,yO ■ (5.48) 

j 

However, nearest-neighbor methods do not take into 
account what we know about either the sensor data or 
the estimate. That is, we may have a very very good 
estimate of some model i and a very very bad esti¬ 
mate for some other model j. If a sensor observation 
is equidistance between them, does it make sense to 
flip a coin? Odds are that it is more likely to come 
from j (with a larger variance) than i (with a smaller 
variance). 

A commonly used measure that can take this into 
account is the Mahalanobis distance [5.45]. The idea is 
to weight each value by its variance as 

m(yi,y 2 ) = (3'i-3 , 2)(Ai+A 2 ) _1 (yi-y2) T • (5.49) 

Thus, distances are scaled inversely with uncertainty. In 
the case above, the observation with a higher variance 
would produce the smaller distance, as desired. 
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Even using this as a weighting method, it is still 
possible that we will make an error in data asso¬ 
ciation. From an estimation point of view, this will 
introduce an outlier in the estimation process with, as 
discussed above, potentially disastrous results. Another 
approach, analogous to IRLS, is instead to weight the 
data based on the distance to a model. This leads natu¬ 
rally to the notion of a data association filter. We refer 
the reader to [5.20] for extensive discussions of these 
techniques. 

5.4.5 Modeling Sensors 

To this point, we have introduced several sensing 
modalities, and we have discussed several methods 
for estimation. However, the latter often rely on hav¬ 
ing statistical models for the former. Thus, no chapter 
on sensing and estimation would be complete without 
a short discussion of modeling sensors. 

Developing a sensor model potentially involves four 
major elements: 

1. Creating a physical model 

2. Determining a sensor calibration 

3. Determining an error model 

4. Identifying failure conditions. 

The physical model is the relationship / between 
the underlying quantities of interest ( x ) and the avail¬ 
able data (y). In many cases, this relationship is obvious, 
e.g., the distance from a laser sensor to a surface in the 
world. In others, it may be less so, e.g., what is the right 
model relating intensities in multiple camera images to 
the distance to an observed point? In some cases, it 
may be necessary to include computational processes, 
e.g., feature detection and correspondence, in the sen¬ 
sor model. 

Once a physical model is determined, there is often 
a process of sensor calibration. Such procedures are 
typically specific to the sensor in question, for ex¬ 
ample, the imaging geometry of a perspective camera 
system requires identification of two scale parame¬ 
ters (governing image scale) and the location of the 
optical center (two additional parameters). There are 
also often lens distortion parameters. These parameters 
can only be determined by a careful calibration proce¬ 
dure [5.7], 

Once a calibrated physical sensor model is avail¬ 
able, determining an error model typically involves 
performing an identification of the statistical parame¬ 
ters. Ideally, the first step is to determine an empirical 
distribution on errors. However, this can often be diffi¬ 
cult, as it requires knowing accurate ground truth for the 
underlying unknown parameters. This often requires 


the development of a laboratory setup that can simulate 
the expected sensing situation. 

Given such an empirical distribution, there are sev¬ 
eral important questions, including: 

1. Are observations statistically independent? 

2. Is the error distribution unimodal? 

3. Can the essential aspects of the empirical error be 
captured using common statistical quantities such 
as the data variance? 

We refer the reader to books on statistics and data 
modeling [5.46] for further information on this topic. 

Finally, it is important to understand when sen¬ 
sors can and cannot provide reliable data, for example, 
a laser sensor may be less accurate on dark surfaces than 
on light ones, cameras do not produce meaningful data 
if the lighting is too bright or too dark, and so forth. 
In some cases, there are simple clues to these condi¬ 
tions (e.g., simply looking at the intensity histogram of 
a camera image can quickly determine if conditions are 
suitable for processing). In some cases it is only pos¬ 
sible to detect conditions in context (e.g., two range 
sensors disagree on the distance to a surface). In some 
cases failure is only detectable in retrospect, e.g., after 
a 3-D surface model is built it is apparent that a hy¬ 
pothesized surface would be occluded by another and 
must therefore be a multiple reflection. In a truly robust 
sensing system, all available possibilities for verifying 
sensor operation should be exploited. 

5.4.6 Other Uncertainty Management 
Methods 

Due to the limitations of space, we have necessarily 
limited our discussion to cover the most commonly 
used sensing and estimation methods. It is important 
to note that many other alternative uncertainty manage¬ 
ment methods have been proposed and employed with 
success. 

For example, if it is known that sensing error is 
bounded, constraint-based methods can be quite ef¬ 
fective at performing point estimation [5.47,48]. Al¬ 
ternatively, if only partial probability models can be 
identified, Dempster-Shafer methods can be employed 
to make judgments [5.49]. 

Fuzzy logic allows graded membership of a set. 
With fuzzy set theory it is possible to have partial mem¬ 
bership. As an example in classification of data it might 
be difficult to select between two categories such as av¬ 
erage and tall and gradual shifts may make sense. Such 
methods have for example been used for situation as¬ 
sessment and navigation as reported by [5.50] for the 
DAMN architecture. 



Sensing and Estimation I 5.5 Representations 109 


5.5 Representations 

Sensor data can be used directly for control but it is 
also used for estimation of the state of the robot and/or 
the world. The definition of state and the appropriate 
methods for estimation are closely related to the repre¬ 
sentation adopted for the application. 

There are a rich variety of possible world represen¬ 
tations including most typical geometric elements such 
as points, curves, surfaces, and volumes. A fundamental 
aspect in robotics is the concept of rigid-body pose. The 
pose of a robot or an entity in the world is characterized 
by position and orientation with respect to a reference 
frame. 

In general, pose is represented by the pair (R. H). 
Here R is the orientation of the object represented by 
a rotation matrix with respect to a reference frame. Sim¬ 
ilarly, H represents the translation of the object with 
respect the reference frame. There is a rich set of po¬ 
tential representations for the transformation between 
reference frames as detailed in the chapter on Kinemat¬ 
ics (Chap. 2) and in [5.51]. 

Sensory data is acquired in a local sensor reference 
frame, for example, a sonar transducer, a laser scan¬ 
ner, and a stereo imaging system would all measure 
distances to surfaces in the world relative to their own 
frame. However, if the goal is to combine this infor¬ 
mation into a common world model, the data must be 
transformed into a robot-centered reference frame, or 
possibly into a fixed world (inertial) reference frame. 
In particular, the world-centered reference frame enable 
simple transfer across robot motions and communica¬ 
tion to other robots and/or users. 

For the purposes of discussion, most representations 
for the integration of sensor data can be categorized into 
four general classes of models: 

• Raw sensor data models 

• Grid-based models 

• Feature-based models 

• Symbolic or graphical models. 

Naturally, it is also possible to combine elements of 
these four categories to achieve hybrid models of the 
environment. 

5.5.1 Raw Sensor Representations 

For simple feedback control [5.52] it is common to inte¬ 
grate raw sensory data directly into the control system, 
as in many cases it is unnecessary to have a world model 
for the control. For example, proprioceptive sensing 
is often used in this manner: basic trajectory control 
makes direct use of encoder information from joints. 


and force control operates directly from force or torque 
information from force sensors. 

Raw sensor models are less common with extero¬ 
ceptive sensing, but there are cases where it can be use¬ 
ful. One example is mobile robot mapping from dense 
point data. This approach has in particular been made 
popular for laser range sensors, where scan alignment 
is used for the generation of point-based world models. 
The work by [5.53,54] demonstrates how a number of 
laser range scans can be combined into a joint model of 
the environment. More formally a scan of the environ¬ 
ment at time t is represented as a point set 

T t = {p i = (p i ,9 i )\iel...N}. (5.50) 

Two different scans T, and TV+i are then aligned 
through a standard rigid body transformation. The es¬ 
timation of the transformation is typically achieved 
through use of the ICP algorithm [5.15]: assume 
that H m is an initial estimate of the transformation be¬ 
tween the two point sets and that \\p,— p t +\ || is the 
Euclidean distance between a point from I’, and a point 
from T t . (- 1 . If furthermore CP is a function to locate the 
closest point from one set in the other set, then let C be 
the set of point correspondences between the two sets. 
Through iterations of the following algorithm, 

1. Compute C k = CP[H lk ~ 1] (p h T,+i)]}, 

2. Estimate the that minimizes the LSQ error be¬ 
tween the points in Ck until the error has converged 

an estimate of the scan alignment can be found and 
a joint model of the environment can be constructed. 

The model is simple to construct and well suited 
for integration of sensor data from a single modal¬ 
ity. Typically the model does not include information 
about uncertainty and, as the model grows the complex¬ 
ity, 0(£, |T’,|) becomes an issue. 

5.5.2 Grid-Based Representations 

In a grid-based representation the world is tessellated 
into a number cells. The cells can contain informa¬ 
tion about environmental features such as temperature, 
obstacles, force distribution, etc. The dimensionality 
of the grid is typically two or three, depending on 
the application. The tessellation can either be uni¬ 
form or tree based using quad-tree or oct-trees [5.55]. 
The tree-based methods are in particular well suited 
for handling of inhomogeneous and large-scale data 
sets. In a grid model each cell contains a probability 
over the parameter set. As an example, when using 
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the grid model for representation of a physical envi¬ 
ronment, the cell specifies occupied (O) or free (F) 
and the cell encodes the probability P(occupied). Ini¬ 
tially where there is no information the grid is initial¬ 
ized to P( O) = 0.5 to indicate unknown. It is further 
assumed that sensor models are available that spec¬ 
ify P(R\Sjj), i. e., the probability of detection objects 
for a given sensor and location. Using Bayes theo¬ 
rem (5.10) it is now possible to update the grid model 
according to 

Pij(f+ 1) = 

_ P(R\Sij = Q) P ij(t) _ 

P(R\Sij = O)pij(t) + P(R\Sij = F)(l- Pij (t)) ’ 

where is computed across the grid model whenever 
new data are acquired. 

The grid-based model has been widely used in mo¬ 
bile robotics [5.56,57] and in medical imaging where 
image volumes are quite common [5.58]. Volume mod¬ 
els can be relative large. As an example a millimeter- 
resolution grid model of the human head requires 4 GB 
of storage, and thus demands significant computational 
resources for maintenance. 

5.5.3 Feature Representations 

Both the raw sensor representation and the grid-based 
models contain a minimum of abstraction for the sen¬ 
sory data. In many cases there is an interest in extracting 
features from the sensor data to reduce the storage 
requirement and only preserve data that are invariant 
across motion of the platform or external objects. Fea¬ 
tures span most standard geometric entities such as 
points (p ), lines (/), planes (N,p), curves (p(s )), and 
more general surfaces. For estimation of properties of 
the external world there is a need for a hybrid model in 
which collections of features are integrated into a uni¬ 
fied model of state. 

In general a point is represented in 31(3). Sensors 
have associated noise and, consequently, in most cases 
points have an associated uncertainty, typically mod¬ 
eled as Gaussian with mean p and standard deviation a. 
The estimation of the statistics is achieved using first- 
and second-order moments. 

Line features are more difficult to represent. The 
mathematical line can be represented by the vector pair 
(p,t), i. e., a point on the line and the tangent vector. 
In many practical applications the line has a finite ex¬ 
tent, and there is a need to encode the length of the line, 
which can be achieved using end points, start point, 
tangent, and length. In some cases it is advantageous 



Fig. 5.19 A topological map of a spatial environment 


to have a redundant representation of the line model 
to simplify updating and matching. The relation be¬ 
tween end-point uncertainties and other line parameters 
can be derived analytically, as described in [5.59]. The 
estimation of line parameters is often based on the pre¬ 
viously describe RANSAC method through the use of 
the Plough transform [5.10], which is another voting- 
based method. 

For more complex feature models such as curves 
or surfaces there is a corresponding need to utilize de¬ 
tection methods that facilitate robust segmentation of 
features, and estimation of the associated uncertainty. 
A comprehensive description of such methods is avail¬ 
able from [5.44]. 

5.5.4 Symbolic/Graph-Based Models 

All of the representations presented in Sects. 5.5.1— 
5.5.3 are parametric in nature with limited associated 
semantics. Methods for the recognition of structures, 
spaces, locations, and objects have seen major recent 
progress in particular due to advances in statistical 
learning theory [5.12,60]. Consequently, today there 
exist a variety of methods for the recognition of com¬ 
plex structures in sensor data, such as landmarks, road 
surfaces, body structures, etc. Given the availability 
of recognized structures it is possible to represent the 
environment using the previously discussed graphical 
models. In general a graph is composed of a set of 
nodes N and a set of edges E that connect nodes. 
Both nodes and edges can have attributes associated 
such as labels and distances. One example of a graph 
structure is a topological map of the environment as 
shown in Fig. 5.19. The graph representation could also 
be a semantic model of the environment (objects and 
places) or a representation of the composition of an ob¬ 
ject to assembled. 

In terms of model updating semantic/graph-based 
representations can take advantage of recent advances 
in Bayesian reasoning as presented by Pearl [5.61], and 
exemplified in [5.62]. 
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5.6 Conclusions and Further Readings 


Sensing and estimation continues to be a challenging 
and very active area of robotics research. Several areas 
of sensing such as computer vision and medical imag¬ 
ing are themselves large and diverse research areas. At 
the same time, new fundamental and applied techniques 
in estimation continue to be developed. Indeed, it is fair 
to say that perception continues to be one of the most 
challenging areas of robotics research. 

Given this wealth of activity, no single chapter can 
hope to cover all of the material that can be useful 
in the development of sensor-based robotics. However, 
the methods that have been presented here are rep¬ 
resentative of the most commonly used techniques in 
robotics. In particular, linear techniques such as the 
Kalman filter continue to form the backbone of percep¬ 
tive robotics. Part C of the handbook provides more in- 
depth coverage of several of the key topics in sensing 
and estimation. 


For the reader wishing to learn more, general dis¬ 
cussion on the design, physics, and use of a rich 
variety of sensors can be found in the Handbook of 
Modem Sensors [5.3]. A discussion of sensors for mo¬ 
bile robots can be found in [5.63], though significant 
advances have been achieved since the book was pub¬ 
lished more than a decade ago. Sensing and estimation 
using computer vision is described in detail in [5.64] 
and [5.65]. 

The basic estimation theory is covered in a num¬ 
ber of excellent text books. Much of the detec¬ 
tion and linear estimation theory is covered in depth 
in [5.20] and [5.66]. General statistical estimation is 
covered in [5.12] and [5.13] and the more recently 
updated version [5.44]. Robust methods are described 
in detail in [5.21,43]. In-depth coverage of estima¬ 
tion methods for mobile systems is also covered 
in [5.33]. 
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6. Model Identification 


John Hollerbach, Wisama Khalil, Maxime Gautier 


This chapter discusses how to determine the kine¬ 
matic parameters and the inertial parameters 
of robot manipulators. Both instances of model 
identification are cast into a common framework 
of least-squares parameter estimation, and are 
shown to have common numerical issues relating 
to the identifiability of parameters, adequacy of 
the measurement sets, and numerical robustness. 
These discussions are generic to any parameter 
estimation problem, and can be applied in other 
contexts. 

For kinematic calibration, the main aim is to 
identify the geometric Denavit-Flartenberg (DH) 
parameters, although joint-based parameters 
relating to the sensing and transmission ele¬ 
ments can also be identified. Endpoint sensing 
or endpoint constraints can provide equivalent 
calibration equations. By casting all calibration 
methods as closed-loop calibration, the calibra¬ 
tion index categorizes methods in terms of how 
many equations per pose are generated. 

Inertial parameters may be estimated through 
the execution of a trajectory while sensing one 
or more components of force/torque at a joint. 
Load estimation of a handheld object is simplest 
because of full mobility and full wrist force-torque 
sensing. For link inertial parameter estimation, 
restricted mobility of links nearer the base as well 
as sensing only the joint torque means that not 
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all inertial parameters can be identified. Those 
that can be identified are those that affect joint 
torque, although they may appear in complicated 
linear combinations. 


6.1 Overview 

There are many different kinds of models in robotics, 
whose accurate identification is required for precise 
control. Examples from the previous chapters include 
sensor models, actuator models, kinematic models, dy¬ 
namic models, and flexibility models. System identifi¬ 


cation is the general field concerned with the process 
of identifying models from measurements. Generally 
speaking, there are two types of models: parametric 
and nonparametric models. Parametric models are de¬ 
scribed by a few parameters, which are adequate to 
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characterize the accuracy of a model throughout the 
working range. Examples include sensor gain and off¬ 
set, the DH parameters for links, and rigid-body inertial 
parameters. Parametric models are particularly appro¬ 
priate for robotics, whose components are manmade 
and whose properties are controlled and understood. 

Nonparametric models include the impulse re¬ 
sponse and the Bode plot for linear systems, and Wiener 
and Volterra kernels for nonlinear systems [6. 1]. A non¬ 
parametric model can be used as a stepping stone 
towards identifying a parametric model; for example, 
a Bode plot (graph of amplitude and phase versus input 
frequency) is often used to decide on model order, such 
as whether an actuator should be modeled as a second- 
or third-order system. Otherwise, nonparametric mod¬ 
els are required when the system’s properties are so 
complicated that a few lumped parameters do not suf¬ 
fice. This is particularly true for biological systems. 

This chapter will describe the parametric calibration 
of the following kinds of models: 

1. Kinematic parameters. Kinematic calibration is the 
process of locating coordinate systems of objects 
relative to each other. These objects may be isolated 
from each other, or they may be linked by joints. 
Examples include: 

• Locating a robot relative to a global reference 
frame. 

• Locating a stereo vision system relative to 
a robot. 

• Locating a grasped object relative to the manip¬ 
ulator’s gripper frame. 

• Locating neighboring coordinate systems of 
links of a robot manipulator. 

2. Rigid-body inertial parameters. These parameters 
are required to predict the driving forces and torques 
to move objects and manipulators. 

Suppose there are /V par parameters combined into 
an A par x 1 parameter vector 0 = {<p\,, cpN pal }- The 
parameters may appear linearly or nonlinearly in the 
model. 

Linear model: y l = A l <p, (6.1) 

Nonlinear model: y 1 = fix 1 . <p) , (6.2) 

where y 1 = {y[,. ■ ■ ,y' M } is the Mxl vector of output 
variables and x 1 = {x\.... ,x l n ) is the vector of input 
variables. For the linear model. A 1 is an M x A par matrix 
whose entries Ay are functions of the input variables x 1 . 
Any entry Ay may be a complicated nonlinear function 
of x 1 , but it evaluates to just a number. For the non¬ 
linear model, an explicit equation is shown in which 


the input variables appear in a nonlinear function / = 
{f\, ... ,fu}. Implicit nonlinear models fiy'.x 1 , <p) = 0 
may also appear in calibration [6.2]; they are han¬ 
dled similarly to explicit nonlinear models (Sect. 6.2.2). 
There may be P different measurements; a particu¬ 
lar measurement is indicated by the superscript / = 
1,...,P. 

For linear models, information from different mea¬ 
surements is combined by stacking the P (6.1) 


y = A<p, (6.3) 

where y = {y 1 ,... .y 1 } is an MP x 1 vector of all output 
measurements and A = { A 1 ,..., A ; } is MP x A par . The 
parameters are estimated by ordinary least squares 


<p = (A T A) _1 A T y . 


(6.4) 


In statistics, the matrix A is called the regressor matrix 
and the least-squares solution is called regression [6.3]. 
An example of a linear model is the rigid-body model 
for inertial parameters. 

The Gauss-Newton method [6.3] is typically em¬ 
ployed to estimate the nonlinear model (6.2). First, 
absorb the input variables x 1 (which can be considered 
as a number of constants) into the nonlinear func¬ 
tion f l , now given a superscript. The model is linearized 
through a Taylor series expansion at a current esti¬ 
mate <p k of the parameters at iteration k 


y[ =f'(<P k + A<p) 


=f(r) + 


df'«t>) 


90 0=<p( 

+ higher order terms 
./(0*) + A'A0, 


A 0 


(6.5) 


where y' c are the computed values of the output variables 
and A 1 = df 1 /dip is a lacobian matrix evaluated at <p k . 
Higher-order terms in the Taylor series are ignored, 
yielding the linearized form (6.5). The bold assumption 
is now made that a correction A0 to the parameter esti¬ 
mate <p k causes the computed output variables to equal 
the measurements: y' c = y 1 . Defining Ay 1 = y 1 —f 1 (<p k ) 
as the error between the output measurement and the 
predicted output with the current model <p k , the lin¬ 
earized (6.5) becomes 

Ay' = A ; A0. (6.6) 


The linearized equation is then stacked for P measure¬ 
ments for the estimation form 


Ay = AA0 . 


(6.7) 
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A correction A <j) to the parameter estimates is now 
found by ordinary least squares 

A0 = (A T A) _1 A T Ay . (6.8) 

This process is iterated with a new estimate 
0*+' = + A<j> until the error A <j> becomes 

sufficiently small. The Gauss-Newton method has 
quadratic convergence, which is very fast, provided 
that there is a good initial estimate <j>° of the parameters 
and the nonlinearity is not too severe. An example of 
a nonlinear model is the kinematic model containing 
the DH parameters. The nonlinearity is mild because 
it is due to sines and cosines, so the Gauss-Newton 
method usually has good convergence properties. 

Both for linear and nonlinear estimation, however, 
rank deficiencies and numerical ill-conditioning may 
cause problems in inverting A T A. Rank deficiencies 
may result from two problems: 

1. Inadequate data. The quality of the data for es¬ 
timating the parameters can be quantified by an 
observability index, such as the condition num¬ 
ber [6.4] of the regressor matrix A. A different 
choice of data to maximize the observability index 
might result in more robust estimates. Examples are 
choosing different poses for kinematic calibration, 
or different trajectories for inertial parameter esti¬ 
mation. 

2. Unidentifiable parameters. Perhaps no set of data 
from an experiment can identify some of the param¬ 
eters. A procedure has to be found to eliminate or 
circumvent the unidentifiable parameters. Parame¬ 
ter elimination is usually done by using the singular 
value decomposition of A, while circumvention can 
be achieved by using a priori values and ridge re¬ 


6.2 Kinematic Calibration 

In general, the relative location between coordinate sys¬ 
tems requires six geometric parameters to be calibrated 
(position plus orientation). If there are mechanical con¬ 
straints between the relative movement of the coor¬ 
dinate systems, such as connection by a joint, fewer 
parameters are required. For a rotary joint connecting 
two links, whose axis is a line-bound vector, four ge¬ 
ometric parameters are required. For a prismatic joint, 
whose axis is a free vector, only two geometric param¬ 
eters describing orientation are required. 

In addition, nongeometric parameters are required 
to model sensors and mechanical deflection: 


gression. This does not mean that the parameters are 
intrinsically identifiable, only that the experimen¬ 
tal setup precludes their determination; for example, 
only one of the 10 inertial parameters of the first link 
of a robot manipulator can be identified if the base 
of the manipulator is stationary. A different experi¬ 
mental setup involving accelerating the base of the 
manipulator and measuring the reaction forces and 
torques at the base would allow other parameters to 
be identified [6.5]. 

Ill-conditioning may result from poor scaling of 
measurements or parameters: 

1. The least-squares estimation minimizes the error 
between the predicted and measured output vector. 
Components yj of the output vector y 1 may have 
different units and magnitudes, such as radians ver¬ 
sus meters, in pose measurements for kinematic 
calibration. Moreover, not all components may be 
measured to the same level of accuracy. Normaliz¬ 
ing the output vectors with an appropriately chosen 
weighting matrix may result in a better conditioned 
estimate. 

2. The parameters may also have different units and 
different magnitudes. This can cause problems both 
in terms of convergence criteria and in terms of 
deciding which parameters to eliminate. Again, 
a weighting matrix for the parameters may be in¬ 
troduced to improve the conditioning. 

These numerical issues are generic to any parame¬ 
ter estimation problem, and are discussed at the end of 
this chapter. The next sections discuss individual robot 
models and issues in putting them into parameter esti¬ 
mation form. 


• Joint angle sensors may require a gain and offset 
determination. 

• Camera calibration using an undistorted pinhole 
camera model may require the determination of the 
focal length and an image sensor offset. 

• Joint flexibility from gears leads to angle change 
due to gravity loading and the manipulator’s own 
weight. 

• Base flexibility results from nonrigid attachment of 
the robot to the environment. Depending on how the 
manipulator is outstretched, there will be a varying 
effect on endpoint location. 
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• Thermal effects and vibration may need to be mod¬ 
eled for very fine positioning control. 

This section focuses on determining the geometric 
parameters and the sensor-based nongeometric param¬ 
eters. 

6.2.1 Serial-Link Robot Manipulators 

The modified Denavit-Hartenberg (DH) parameters 
(Fig. 6.1) serve as the main geometric parameters 
(Sect. 2.4); the link transformation matrix is 
i-i T . 

= Rot(x, cn,)Trans(x, a,)Trans(z, <f,)Rat(z, (9,-) . 

(6.9) 

• For an n-joint manipulator with rotary joint i = 
1whose axis z, is a line in space, all four 
parameters a„ dj, a,-, and 0; have to be calibrated. 

• For a prismatic joint i, whose axis z, is a free vector, 
only two parameters describing its orientation (a,- 
and 0,) are required. In principle, the axis z, can be 
positioned anywhere in space. This means two DH 
parameters are arbitrary. One possibility is to inter¬ 
sect Zi with Oj-j-j [6.6,7], which sets dj .pi = 0 and 
a,-)-i = 0. While kinematically correct, this place¬ 
ment is nonintuitive in that it doesn’t correspond to 
the physical location of the prismatic mechanism. It 
is possible to set |_i with a t or 0; to values that 
position Zi in the middle of the prismatic joint’s me¬ 
chanical structure. 

In the case of nearly parallel neighboring axes, the 
common normal is poorly defined and the calibration 
is ill-conditioned. For this case, Hayati and Mirmi- 
rani [6.7] introduced an extra rotational parameter /J, 
about they,_i axis (Fig. 6.2). 

Let lie along a line from O, to axis z,_i, such 
that x'_ l is perpendicular to z,-; the intersection point 



Fig. 6.1 The modified DH parameters 


defines the origin 0,_ 1 . Two rotations are required to re¬ 
late Zj_ 1 to Zi = z-_| : a rotation about x\_\ maps z"_i 

to Zi, and a rotation /f, about y,_i =y'!_ x maps Zi— 1 
to z"_j. The angle 0, is now from x ' i _ 1 to jc, about z,. 
The link transformation is 

i_I T ; 

= Rot(y, /l,')Rot(jc, a,)Trans(x, a,)Rot(z, 9 ',) . 

( 6 . 10 ) 

• For a rotary joint, the parameter /),- is calibrated in¬ 
stead of di . 

• For a prismatic joint, the joint variable dj has to be 
retained. As before, position z,- at some convenient 
location on link i, by specifying two parameters rel¬ 
ative to the coordinate system i + 1. Then proceed 
with the construction of the Hayati parameters for 
di = 0. As dj changes, axis Xj is displaced along z, 
relative to (not shown). The link transforma¬ 
tion is 

'T; = Rot(y, J S,)Rot(x, a,)Trans(x, a,-) 

Trans(z, J,)Rot(z, 0,) . (6.11) 

Although there are five parameters in this transfor¬ 
mation, the process of setting the Hayati parameters 
for dj = 0 is tantamount to setting the value of di— 1 
to locate Oi—i, so there is no net increase in the 
number of parameters. 

The procedures above set the coordinate systems in 
the intermediate links of a serial manipulator. Coordi¬ 
nate systems also have to be set in the base and end 
links, but the procedure for doing so depends on the 
external metrology system and on any physical con¬ 
straints on the end link pose. The last frame may be n 
or ;z + 1, while the first frame may be 0 or —1 (to keep 
consecutive numbers); examples appear below. Collect 
the unknown kinematic parameters into the vectors a, 
d, a, 6 , and fi. and thence into the parameter vec¬ 
tor 0 = {a,d, a, 0,13}. The parameters <j> predict the 



Fig. 6.2 The additional parameter ($j about _y,_i is em¬ 
ployed for nearly parallel axes 
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position and orientation of the last coordinate system 
relative to the first, such as 

Not all six components of pose are necessarily 
used for calibration; the number can vary from one 
to six. Calibration proceeds by observing the error in 
the prediction of a certain number of pose components, 
then employing the nonlinear calibration method (6.7). 
There are two general methods for observing er¬ 
ror: 

• Open-loop calibration utilizes an external metrol¬ 
ogy system to measure the pose components. Be¬ 
cause the manipulator is not in contact with the 
environment during this process, the method is 
termed open loop. 

• Closed-loop calibration utilizes physical con¬ 
straints on the end link pose to substitute for mea¬ 
surements. Deviations from the physical constraint 
represent the prediction error. Because of the con¬ 
tact with the physical constraint, the manipulator 
forms a closed loop with the ground. 

Open-Loop Kinematic Calibration 
The calibration literature contains a variety of metrol¬ 
ogy systems, which can be categorized based on the 
number of pose components measured [6.8]: 

• 1 component. The distance to a single point on the 
end link can be measured in a variety of ways, such 
as an instrumented ball bar [6.9], wire potentiome¬ 
ter [6.10], or laser displacement meter [6.11]. 

• 2 components: A single theodolite has been 
employed to provide two orientation measure¬ 
ments [6.12], A reference length had to be sighted 
for scaling. 

• 3 components: Laser tracking systems provide ac¬ 
curate three-dimensional (3-D) measurements by 
reflecting a beam off a retroreflector mounted on the 
end effector. The beam provides length information, 
while the gimbal drive for laser steering provides 
two orientation measurements [6.13]. Since the 
least precise part of this setup is the angle sensing, 
another approach is to employ three laser tracking 
systems and use only the length information. Com¬ 
mercial 3-D stereo-camera motion tracking systems 
also provide high-accuracy measurements of posi¬ 
tion. 

• 5 components: Lau et al. [6.14] presented a steer¬ 
able laser interferometer with steerable reflector. 
With pitch and yaw measurements, the steerable 
interferometer yields all three components of posi¬ 
tion, while the steerable reflector yields two compo¬ 
nents of orientation. 


• 6 components: Full pose can be inferred from the 
3-D position of multiple points on the last link mea¬ 
sured with a stereo-camera system. A coordinate 
system fit to the cloud of points yields position plus 
orientation [6.15]. 

Vincze et al. [6.16] measured full pose with a single¬ 
beam laser tracking system, by fitting the robot with 
a retroreflector mounted on a universal joint. Posi¬ 
tion is measured using interferometry, as usual. The 
novel aspect is orientation measurement, by imag¬ 
ing of the diffraction pattern of the edges of the 
retroreflector. 

Examples are given for calibrating with three pose 
components and with all six pose components mea¬ 
sured. 

Point Measurement. The 3-D position of a particu¬ 
lar point on the end link can be conveniently located by 
some form of stereo camera system. The camera sys¬ 
tem defines a global coordinate system relative to which 
frame 1 of the robot is located. To provide enough pa¬ 
rameters, an intermediate coordinate system has to be 
introduced; this intermediate coordinate system has in¬ 
dex 0, while the camera system is given index —1 to 
keep consecutive numbers. Two of the eight param¬ 
eters are arbitrary. Figure 6.3 shows one possibility: 
Zo is made parallel (ot 0 = 0) and coincident (ao = 0) 
with z— 1 . The calibrated parameters are do, 0 q, a\, d\, 
ai, and G\. In the case that z— 1 is nearly parallel to zo, 
the measurement coordinate system can be simply re¬ 
defined to avoid using Hayati parameters; for example, 
y— 1 can be redefined as z—\. 

In the end link, the origin O n and axis x„ are un¬ 
specified, as are the associated parameters d„ and 9 n . 
Locating the measured point requires only three pa¬ 
rameters, so the addition of a single coordinate system 
n + 1 is required to provide an additional parameter. 




Fig. 6.3 A camera system (index—1) is located relative to 
the manipulator coordinate system 1 via an intermediate 
coordinate system 0 
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The measured point is defined as the origin 0„_|_i of 
the new coordinate system, and the normal from z„ that 
intersects 0„+i defines the x n axis (Fig. 6.4). Three pa¬ 
rameters are arbitrary; a simple choice is to make z„+i 
parallel to z„ (a,,- pi = 0) and jc„+i collinear with x n 
(0,,-pi = 0 and r/,,+ 1 = 0). The calibrated parameters 
are a n +i, d n , and 8 n . 

From the transformation 

— 1 rpi — 1 rpi n . rp 

— Ao • • • A n _|_i , 

the position ~ x p n - |-i of the measured point relative to 
the camera frame is extracted. Collect the unknown 
kinematic parameters into vectors a.d.a, 6 , and /L and 
thence into the parameter vector <]> = {a,d, a, 0, /?}. 
The nonlinear kinematic model analogous to (6.2) is 

-Vn+l =/(«'. 0), l=h...,P. (6.12) 

where q 1 is the vector of joint variables for pose l. To 
linearize this equation into estimation form (6.6), the 
associated Jacobians for each parameter type are calcu- 



Fig. 6.4 A measured point 0„+i is located on the end link 
by adding a coordinate system n + 1 



Fig. 6.5 Full pose measurement of coordinate system n + 
1 in the end link 
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where a typical column i for each individual parameter 
Jacobian is derived from the screw parameters as if each 
parameter represented an active joint (Sect. 2.8.1) 

. Mxf , DH 


t = , t 
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(6.17) 

t/ -1 __/ . . -1 J/ 

j 0i — yi— 1 X “i-l,n+l » 

(6.18) 

where l d it „J r \= *R,-refers the interorigin vec¬ 
tor to coordinate system —1. Equation (6.13) is stacked 


for all the poses for the final estimation form 


A-'pn+i = JA0 , (6.19) 

which corresponds to (6.7) and which is solved for A</> 
by least squares and for </> by iteration. 


Full Pose Measurement. Suppose that coordinate 
system n + 1 in link n is measured (Fig. 6.5). Coordi¬ 
nate system n is completed in the usual way, and the six 
calibrated parameters to locate coordinate system n + 1 
are d,„ a n , 9 n , a,,+ i, <7„+ 1 , and 6 n +i. If z n +\ is nearly 
parallel to z,„ this axis may simply be permuted with 
other axes, such as y„_|_ i, to avoid using the Hayati pa¬ 
rameters. 

In addition to the position (6.12), orientation equa¬ 
tions of coordinate system n + 1 

-1 R,' + 1 =F(< ? ',0), 1=1 . P (6.20) 

are extracted from — 1 1 , where F is a matrix func¬ 

tion. Linearization of this equation yields 
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where A - l p l n + l are differential orthogonal rotations 
which are the finite-difference counterpart to the an¬ 
gular velocity vector. Continuing with this analogy, 
a Jacobian J similar to that for spatial velocities 
(Sect. 2.8.1) is employed to represent the combined ef¬ 
fect of parameter variations A <j) on changes in position 
A - l p l II _\_ 1 and orientation A - 


(: 


) 


= J'A 0 . 


( 6 . 22 ) 


Compared to (6.13), the Jacobian J f now has six rows, 
as do the individual parameter Jacobians 
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by varying the joint angles, as long as the point of con¬ 
tact is not changed. Before, the measurement system 
for point measurement defined the reference coordinate 
system —1 (Fig. 6.3), and there was an end effector co¬ 
ordinate system n + 1 whose origin 0 „+1 was measured 
(Fig. 6.4). Now the reference coordinate origin Oo has 
index 0 and is collocated with O n +1 (Fig. 6.6). An extra 
coordinate system —1 is not required because a point is 
located relative to coordinate system 1 with only three 
parameters: a\, d\, and Q\. The arbitrary choice oq = 0 
is made, i. e., zo is made parallel to z\■ 

Different poses while maintaining the point contact 
can be generated manually, or they can be generated 
automatically if there is a force control capability. In 
comparison to (6.13), the measured position °p I n ^. 1 = 0 
by definition and the linearized calibration equation is 
simply 

A~y,+ 1 = —~y,+ i.c = J'A0 . (6.28) 
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(6.27) 


As before, A (j> is found by least squares and (J) by itera¬ 
tion. 


Closed-Loop Calibration 

Physical constraints on end-effector position or orien¬ 
tation can substitute for measurements. The location of 
a physical constraint defines the reference coordinate 
system; the measurement of endpoint position or orien¬ 
tation is therefore defined as zero. The deviation from 
the physical constraint due to an incorrect kinematic 
model is cast as a displacement from the reference co¬ 
ordinates. Analogous to point measurement and full 
pose measurement, there are closed-loop methods us¬ 
ing point constraints and full pose constraints. 

Point Constraint. Suppose that the end-effector holds 
a stylus that makes contact with a fixed point in the en¬ 
vironment. The orientation of the stylus can be changed 


The set of poses that may be generated is limited rela¬ 
tive to open-loop calibration, which may affect identifi- 
ability. 

Full Pose Constraint. Analogous to full pose mea¬ 
surement (Fig. 6.5), the end link may be fully con¬ 
strained by rigidly gripping the environment. If the 
manipulator is redundant (Chap. 10), then poses can be 
generated through self motion. Generating such poses 
would require endpoint force/torque sensing or joint 
torque sensing to be accomplished. 

The end link can be considered part of the ground 
due to rigid attachment, and therefore one fewer coor¬ 
dinate system is required than for the fixed point case. 
Fig. 6.7 shows one way of setting up coordinate systems 
0 and n for calibration. Axis zo is set equal and coin¬ 
cident with axis z n . The common normal between zo 
and zi sets the origin Oo and the axis xq. Coordinate sys¬ 
tem n is completed by defining O n = Oo and x n = Xq. 
Six parameters result for calibration, as they must to re- 



Fig. 6.6 Fixed point contact, setting Oo = O n -|_i 


119 


Part A | 6.2 







Part A | 6.2 


120 Part A 


Robotics Foundations 


late coordinate system n to coordinate system 0: 9 n , d n , 
a i, a\, d i, and 6\. 

Similar to (6.28) after index adjustment, the mea¬ 
sured position °pl = 0 by definition and the linearized 
position calibration equation is 



With regard to the error in orientation (6.21), the mea¬ 
sured orientation °R l ll = I. the identity matrix, after 
index adjustment. 

s ( a y,) = (i R ;j (X, C ) T = (XJ T -1 ■ 

(6.30) 

The error (6.22) is then applied as for full pose mea¬ 
surement, after index adjustment. The set of poses that 
may be generated is limited relative to open-loop cali¬ 
bration, which may affect identifiability. 

6.2.2 Parallel Manipulator Calibration 

Parallel manipulators are comprised of multiple closed 
loops. The methods of the previous section could 
readily be extended to calibrate parallel manipula¬ 
tors. Rather than treat parallel and serial manipulators 
differently, and continue to elaborate the calibration 
equations on a case-by-case basis that includes differ¬ 
ent loop arrangements and different sensing arrange¬ 
ments, Hollerbach and Wampler [6.8] presented a uni¬ 
fying methodology termed the calibration index , which 
views all calibration problems as closed-loop calibra¬ 
tion problems. 

Open-loop calibration is converted to closed-loop 
calibration by considering the end effector measure¬ 
ment to form a joint. All measurements, whether from 
joints or metrology systems, are put on an equal foot¬ 
ing, as are all unsensed joints, whether from unsensed 



Fig. 6.7 Fully constrained end link, setting O o = O n 


components of pose, passive environmental constraints, 
or joints in the chain without sensors. In the case of par¬ 
allel linkages, sufficient numbers of loop-closure equa¬ 
tions are formulated to characterize the kinematics and 
are combined at each pose. Because the loop-closure 
equations are implicit functions of all measurements, 
Wampler et al. [6.2] called this calibration method the 
implicit loop method. 

Fig. 6.8 illustrates this idea for a calibrated stereo 
camera metrology system that measures the 3-D coor¬ 
dinates of a distinguished point on the end-effector of 
an uncalibrated robot. On the right this camera system 
has been replaced by a prismatic leg, which stylistically 
represents a six-degree-of-freedom (6-DOF) joint that 
provides equivalent 3-D coordinate measurements. The 
result is a closed-loop mechanism. 

The kinematic loop closure equations / for the i-th 
pose (i = 1. P) arc formed as 


/'(0) =/(*', 0) = 0, (6.31) 

where 0 is a vector of robot parameters to be calibrated, 
x l is a vector of joint sensor readings and possibly exter¬ 
nal sensor readings, and/' absorbs the sensor readings 
x' and so is given an index. Combining (6.31) for the P 
poses into a single matrix equation 

/W = (/ lT ••• r T ) T = 0. (6.32) 

Linearize (6.32) around the nominal values of the pa¬ 
rameters 

A/ = A0 = AA0 . (6.33) 

dtp 

where A f is the deviation of the computed loop closure 
equations from zero, J is the identification Jacobian, 
and A 0 is the correction to be applied to the current 
parameter estimate. The calibration problem is then 
solved by minimizing A f via iterative least squares. 




Fig. 6.8 External metrology system modeled as a 6-DOF 
joint 
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The Calibration Index 

The basis for kinematic calibration is to compute the 
error in positioning a manipulator using the current 
kinematic model. The error may be relative to mea¬ 
surements by an external metrology system (open-loop 
methods), or relative to a physical constraint such as 
contact with a plane (closed-loop methods). An exter¬ 
nal metrology system may measure all six components 
of pose or fewer components such as a point on the 
end link (three components). Likewise, a physical con¬ 
straint can limit between one and six components of 
pose. Constraints and measurements may be mixed. 
The number of measurements and constraints determine 
how many scalar equations per pose are available for 
calibration. 

The calibration index C quantifies the number 
of equations per pose. While this analysis is rather 
straightforward for serial linkages, for parallel linkages 
it can be difficult to infer the number of independent 
equations per pose 

C = S — M , (6.34) 

where A is the sensor index and M is the mobility index. 
The mobility index [6.17] characterizes the degrees of 
freedom in the calibration setup. 

Nj 

M = 6n — nj , (6.35) 

;= 1 

where n is the number of links, Nj is the number of 
joints, and nj is the number of constraints at joint i. n in¬ 
cludes any extra links attached to the robot to constrain 
or measure its motion. Nj includes joints of any addi¬ 
tional linkages added for calibration. For a rotational or 
prismatic joint nj = 5, while for a ball or spherical joint 
nj = 3. For an external measurement system for a freely 
moving end-effector n c Nj = 0, while for rigid attachment 
of the endpoint n c Nj = 6. While generally correct, there 
are exceptions to (6.35) due to special or degenerate 
mechanisms that have to be analyzed on a case-by-case 
basis. 

The sensor index A is the total number of sensors in 
the joints 

Nj 

A=J>, (6.36) 

;= 1 

where A, is the number of sensed degrees in joint i. Usu¬ 
ally Si = 1 for the lower-order pairs typical of actuated 
joints, while S Nj = 6 for full pose measurement of the 
end-effector joint Nj. For an unsensed joint, such as in 
passive environment kinematics, Si = 0. 


If P is the number of poses, then CP is the total num¬ 
ber of equations for the calibration procedure. Clearly 
a larger C means that fewer poses are required, other 
things being equal. For the single-loop case consisting 
of a series of sensed lower-order robot joints (A; = 1, 
nj = 5, i = 1, ..., Nj — 1) with a final joint (A^,, n c Nj ) 
connecting the end-effector to ground, one has from 
(6.34-6.35) 

C = S Nj +ii c Nj . (6.37) 

According to the calibration index, using full end¬ 
point constraints is an equivalent kinematic calibration 
method to using full pose measurements. There is po¬ 
tentially a problem with the smaller range of poses 
available with the endpoint constrained, but otherwise 
the mathematics are the same. 

Categorization of Serial Link Calibration 
Methods 

A great variety of calibration methods have been pro¬ 
posed, varying in the manner of pose measurement or 
endpoint constraints. These are categorized below ac¬ 
cording to the calibration index C and the values of tf Nj 
and S N] : 

• C = 6: n c Nj = 0 and A^y, = 6 corresponds to full 
pose measurement. n c Nj = 6 and Ajy, = 0 corre¬ 
sponds to rigid endpoint attachment. 

• C = 5: n c N = 0 and Ay, = 5 corresponds to 5-DOF 
pose measurement [6.14]. n c Nj = 5 and S Nj — 0 
corresponds to 5-DOF endpoint constraints, such 
as manipulation of an unsensed passive hinge 
joint [6.18]. 

• C = 4: No published method exists for C = 4. 

• C = 3: n c Nj = 0 and Ay, = 3 corresponds to 3-DOF 
pose measurement. n c N = 3 and Ajy, = 0 corre¬ 
sponds to 3-DOF endpoint constraints. 

• C = 2: n c Nj = 0 and A,y, = 2 corresponds to 2-DOF 
pose measurements, such as is provided by a single 
theodolite [6.12]. n c Nj = 2 and Snj - 0 corresponds 
to 2-DOF endpoint constraints. Motion along a line 
provides two constraints [6.19]. 

• C = 1: n c Nj = 0 and Ajy, = 1 corresponds to mea¬ 
surement of just 1-DOF pose, provided by a lin¬ 
ear transducer such as an linear variable differ¬ 
ential transformer (LVDT) [6.9] or wire poten¬ 
tiometer [6.10]. n c Nj = 1 and S Nj - 0 corresponds to 
a plane constraint [6.20]. 

Categorization of Parallel-Link Calibration 
Methods 

To calibrate parallel robots, a loop-closure equation is 
written for each loop j 

0 =/;(*). (6.38) 
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The equations for all the loops are combined. One prob¬ 
lem is to eliminate unsensed degrees of freedom. The 
calibration index is now applied based on the number 
of loops: 

• 2 loops'. A two-loop mechanism has three arms or 
legs, attached to a common platform. An example 
is the RSI Research Ltd. Hand Controller [6.21], 
which employs three 6-DOF arms with three sensed 
joints each. The mobility of this mechanism is M = 
6. As S = 9, therefore C = 3 and closed-loop cali¬ 
bration is possible. 

• 4 loops: Nairn et al. [6.22] calibrated a spherical 
shoulder joint, driven redundantly by four prismatic 
legs (Fig. 6.9). In addition, the platform is con¬ 
strained to rotate about a spherical joint; with the 
four legs, four kinematic loops are formed. For this 
system, M = 3 and 5 = 4, so that C = 1. Hence 
self-calibration is possible. Without the extra leg 
and the sensing it provides, one would have C = 0 
and calibration would not be possible. 

• 5 loops: Wampler et al. [6.2] calibrated the six¬ 
legged Stewart platform (M = 6) via a closed-loop 


6.3 Inertial Parameter Estimation 

A rigid body i has 10 inertial parameters: mass m,-, 
center of mass ro, relative to an origin Oq , and sym¬ 
metric inertia matrix I, referred here to the origin O , 
(Fig. 6.10). The rigid body may be a load held by the 
end effector, or it can be one of the manipulator’s own 
links. By generating a trajectory and measuring forces 
or torques in combination with velocity and acceler¬ 
ation, some or all of the inertial parameters can be 
estimated. 

6.3.1 Link Inertial Parameter Estimation 

The procedure involves: 

1. Formulating the Newton-Euler equations of the 
load dynamics to reveal a linear dependence on the 
inertial parameters. 

2. Estimating the parameters using ordinary least 
squares. 

Begin with the Newton-Euler equations of the last 
link, and assume that suitable filters have been designed 
to estimate the velocity 9, and acceleration 0, at each 
joint i. 

The center of mass of link n is defined as C„, lo¬ 
cated relative to the base origin Oq by r„ = C n — Oq and 



Fig. 6.9 Redundant parallel drive shoulder joint 

procedure. In addition to leg length measurements, 
all angles on one of the legs were measured (5 = 
11). The reason for the extra sensors is to yield 
a unique forward kinematics solution, but a side 
benefit is that, with C = 5, closed-loop calibra¬ 
tion is possible. For a regular Stewart platform 
without the instrumented leg, 5 = 6 and hence 
C = 0. External pose measurement is required; 
for example, with full pose measurement 5=12 
and C = 6; full pose measurement was utilized 
in [6.23]. 


relative to link n ’s origin O n by c„ = C n — O n . All vec¬ 
tors and matrices are expressed in coordinate system n, 
in which the center of mass location c„ and the inertia 
matrix I„ about origin ()„ are constant. 

From (3.35) in Chap. 3, the Newton-Euler equations 
referred to 0„ are 

f n — f/itL + Ai x I /7 v n . (6.39) 

Substituting for the spatial inertia I„, the spatial accel¬ 
eration a„, and the spatial velocity v n , the first term on 



Fig. 6.10 Location of the center of gravity for an interme¬ 
diate link and the constraint forces and torques 
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the right evaluates to 
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where v n = do,, — to,, x v n has been substituted. The 
second term on the right evaluates to 
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Combining and simplifying, 
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Using these expressions, (6.40) can be written as 

fn 

_ ( 0 -S (do„) L(( b n ) + S(rn„)L((« n A 

Vd 0n S(w„) + S(»„)S(w„) 0 / 

( m„ \ 

m„c„ 

l (In)) 

or more compactly, 

fn = A„0„ , (6.41) 

where A„ is a 6 x 10 matrix, and <j)„ is the vector of the 
10 unknown inertial parameters which appear linearly. 

The previous formulation can be extended to all 
the links of a manipulator with n joints (Fig. 6.10). 
(Only manipulators with revolute joints will be con¬ 
sidered, since handling prismatic joints requires only 
trivial modifications to the algorithm.) Define/^ as the 
spatial force at joint i due to movement of link j alone. 
Then/,, is the spatial force at joint i due to movement of 
its own link, and is the same as (6.41) with i substituted 
for n in the A„ matrix 


where the mass moment m„c„ appears as a quantity 
to be estimated in combination. However, since the 
mass m„ is separately estimated from the m„do„ term, 
the center of mass c„ can be extracted. To account 
explicitly for gravity g, we substitute do„ —g for do,, sub¬ 
sequently. 

To formulate an estimation algorithm, the force and 
torque measured by the wrist sensor must be expressed 
in terms of the product of known geometric parameters 
and the unknown inertial parameters. Elements of the 
inertia matrix are vectorized as l(I„) according to the 
following notation 


% = %&, (6.42) 

where 0, is the vector of unknown link i inertial pa¬ 
rameters. Superscript i has been added to indicate that 
vectors are expressed in terms of link i coordinates, so 
that the center of mass 'c,- and the inertia matrix 'I,- are 
constant. 

The total spatial force at joint i is the sum of the 
spatial forces f -,j for all links j distal to joint i 

n 

fi= (6 - 43) 
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where L(«„) is a 3 x 6 matrix of angular velocity ele¬ 
ments and 


/hi hi hi\ 
In = I hi hi hi I 
V 13 hi hi / 


Each spatial force at joint i is determined by trans¬ 
mitting the distal spatial force ifjj across intermediate 
joints. Using the spatial force transform matrix 'X ; F , 


fu+i = ‘X‘+i i+1 fi+u+i = 'x; +1 '+ 1 A i+1 0 i+1 


+ 1 1 


(6.44) 


For convenience, we note that 'Xf = 16x6- To ob¬ 
tain the forces and torques at the f-th joint due to the 
movements of the y-th link, these matrices can be cas¬ 
caded 


'/, 7 = 'Xf +1 '•+ 1 xf +2 ■■■ ^ % 

= 'xf A rfj . (6.45) 
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An upper-diagonal matrix expression for a serial 
kinematic chain can be derived from (6.43) and (6.45) 


f l fi\ 

2 fi 


Vfn) 



! x^a 2 • 

■ A,A 



0 

2 x| 2 a 2 ■ 

■ 2 XP"A„ 


02 

V 0 

0 

■ "x^aJ 


V0»> 


(6.46) 


This equation is linear in the unknown parameters, but 
the left-hand side is composed of a full force-torque 
vector at each joint. Since only the torque r, about the 
joint rotation axis z, can usually be measured, each spa¬ 
tial force must be projected onto the joint rotation 
axis, reducing (6.46) to 

T = K0 , (6.47) 



and K,y = Oixio if i > j- For an n-link manipulator, r is 
an n x 1 vector. 0 is a lOn x 1 vector, and K is an n x 10/z 
matrix. 

For geared electric drives, joint torque may either 
be measured by joint torque sensors (Chap. 28) or esti¬ 
mated from the motor current by using an electric motor 
model (Chap. 8). Most robots do not have joint torque 
sensors, in which case joint friction needs to be taken 
into account. Joint friction typically consumes a large 
fraction of the torque that the motor produces. Coulomb 
and viscous friction are the most important compo¬ 
nents of a friction model, although Stribeck friction 
may need to be modeled for low joint velocities [6.24, 
25]. A friction model is estimated through a process of 
moving one joint at a time, and relating motor torque to 
velocity. Ripple torque, due either to uncompensated in¬ 
homogeneities of the magnetic field in the motor [6.26, 
27], or to positional dependencies of gear tooth interac¬ 
tion [6.24], may need to be modeled. 

Equation 6.47 represents the dynamics of the ma¬ 
nipulator for one sample point. It can be rewritten 


using P data points as with geometric calibration 


II 
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where r is now an nP x 1 vector and K is now 
nP x 10«. 

Unfortunately, one cannot apply simple least- 
squares estimation because K T K is not invertible due 
to the loss of rank from restricted degrees of freedom at 
the proximal links and the lack of full force-torque sens¬ 
ing. Some inertial parameters are completely unidenti¬ 
fiable, while some others can only be identified in linear 
combinations. The general topic of identifiability of pa¬ 
rameters, and how to handle unidentifiable parameters, 
is discussed next. 

An issue with geared electric drives is the rotor in¬ 
ertia. If not known, the rotor inertia can be added to 
the list of 10 inertial parameters to be identified for 
a link [6.28]. For large gear ratios, the rotor inertia can 
dominate the other components of link inertia. 

6.3.2 Load Inertial Parameter Estimation 

The load is considered as a rigid body held by the end 
effector. It has 10 inertial parameters: mass wl, first 
moments /wlCl relative to O n and symmetric inertia 
matrix I L referred here to origin O n (Fig. 6.11). The 
estimation of these parameters can be used to tune the 
control law parameters in order to improve the dynamic 
accuracy of the robot, or to compensate for its dynam¬ 
ics by the control. They can also be exploited to verify 
the load transported by the robot. Two procedures are 
presented to estimate the load inertia parameters: the 
first one assumes that there is a wrist-mounted 6-axis 
force/torque sensor, whereas the second one makes use 
of the joint torques. 


Link i 



Fig. 6.11 Dynamics of the last link 
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Using 6-Axis Forces/Torque Sensor 
Kinematically, the force-torque sensor is part of the last 
link n, mounted near the joint axis z.„ and origin O n . 
The sensor provides readings relative to its own coordi¬ 
nate system, called n + 1 (Fig. 6.11). The remainder of 
the last link is attached to the shaft of the force-torque 
sensor, and so the force-torque sensor is measuring the 
loads on the last link excluding itself. The acceleration 
of the force-torque sensor reference frame would have 
to be calculated based on the amount of offset from O n , 
but we will ignore that complication here and assume 
that the sensor origin O n +i is coincident with O n . 

The estimation of load inertial parameters can be 
carried out using (6.41). 

The quantities inside the A„ matrix are computed 
by direct kinematics computation from the measured 
joint angles, and from the estimated joint velocities 
and accelerations. The estimation is typically done by 
bandpass filtering of the joint angle data [6.29]. The el¬ 
ements of the /„ vector are measured directly by the 
wrist force sensor. For robust estimates in the presence 
of noise, a larger number of P data points are obtained 
by moving the manipulator through a suitable trajec¬ 
tory. Augment/,, and A„ as 



(K\ 


(A\ 

A = 


. /= 



kKj 


<2 


where P is the number of data points. The least-squares 
estimate for <j> L is given by 

0 L = (A t A)-'A t / . (6.50) 

For object recognition, it may be desired to derive 
the inertia about the center of mass, which can be done 
with the parallel axis theorem. An eigenvalue analysis 
can diagonalize the inertia matrix to reveal the principal 
axes and inertia. 

Using Joint Torques 

When the robot is holding a payload, the dynamic iden¬ 
tification (6.48) can be rewritten as follows 

r = K<j> + K l 0 l , (6.51) 

where t contains the measured joint torques when the 
robot is holding a load for P points. Using (6.51) in the 
identification will result in grouping 0 L with </),, and 
with other link parameters. The load parameters can be 
deduced by examining how the load parameters have 
modified the values of the grouped inertial parameters 
using the expressions of grouping relations [6.30]. In 
the following we present three methods to identify <j)\ 
parameters without using the grouping expressions. 


Using a Priori Link Parameters Estimates. In this 
case we suppose that the robot parameters without 
load have been estimated as given in Sec. 6.3.1. By 
substituting the robot parameters into (6.51) the 10 
inertial parameters of the load are identifiable and 
can be estimated using the least squares solution 
of [6.31] 

(r — K<j>) = K l 0 l . (6.52) 

In this method we suppose that the friction parame¬ 
ters are invariant with respect to the payload. If this 
hypothesis is not true the friction parameters must be 
re-estimated at the same step with (j) L . 

Using Joint Torques on the Same Trajectory Twice 
with and Without the Load. Since the vector Ktj) 
of (6.52) is equal to the joint torques without load, 
consequently, the payload inertial parameters could be 
identified by (6.52) after replacing K<j) by To repre¬ 
senting the joint torques on the same trajectory without 
load 

(t-t o ) = K l 0 l . (6.53) 

This method assumes that the difference between 
the joint variables with and without payload is negligi¬ 
ble and that the friction parameters are the same with 
and without load. If the friction parameters vary with 
respect to the load we can estimate this variation at the 
same time as the load inertial parameter. 

Estimating the Robot Parameters and the Load 
Parameters in One Step. In this method the identi¬ 
fication model is constructed by grouping two sets of 
equations: the first using a trajectory without load, and 
the second using a trajectory when the robot is holding 
the load. In this case the two sets of trajectories could 
be different. The link parameters and the payload in¬ 
ertial parameters are estimated using the least squares 
solution of 

(::)-(£ l)(tr 

where the indices a and b indicate the trajectories with¬ 
out and with load, respectively. 

This method has the advantage of using a global 
identification procedure that can avoid the accumula¬ 
tion of errors of previous methods. 

These three methods have been validated experi¬ 
mentally on the 6-DOF robot RX-90 of Staubli. They 
give similar results [6.30]. 
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6.3.3 Identification 

of Total Joint Drive Gains 

For gear electric drive of joint j, the joint torque Xj is 
estimated as tj = v r jgj, where v r j is the control signal 
computed by the numerical controller of the robot (the 
input references of the motor current loop) and gj is the 
total joint j drive gain, given by the gear ratio, the cur¬ 
rent amplifier gain, and the motor torque constant. 

Because of large values of the gear ratio for indus¬ 
trial robots, (> 50), joint drive gain is very sensitive to 
errors and must be accurately measured from special, 
time consuming, heavy tests, on the drive chain [6.32, 
33], 

The methods used to identify the load inertial pa¬ 
rameters using joint torques can be inversed to identify 
the drive gains. In this case some load inertial param¬ 
eters 0 *-l must be known while the other unknown 
parameters 0 „l are identified with the gain g. 

The vector r of sampled joint torques in (6.52) or 
(6.54) is rewritten as 

T=\rg, (6.55) 


of the system 



(6.60) 

But owing to the fact that K* L and K„ L are corre¬ 
lated by the same noisy data (q,q,q), the total least 
squares (TLS) solution is used to avoid biais errors of 
the ordinary LS solution [6.34]. Rewritting (6.60) as 

K tot 0 to t = 0 , (6.61) 

where 

— K„ 0 0 \ 

tot [v rb -K* — K„ l ( ' ' 

0tot = ( g T 0 r 0 Il if • (6.63) 

The singular value decomposition ( SVD ) of K tot is 
given by 


where: g is the (n x 1) vector of the joint drive gains, V, 
regroups the samples of v r 

Using the Same Set of Trajectory Twice, 
with and Without the Load 
Introducing (6.55) and the known load inertial parame¬ 
ters in (6.53) with the expression 

K l 0l = K,l0, l + K „ L 0„ L (6.56) 

gives 

(Vr — V,.„l)^ = K kL<t>kh + K„ l 0„l . (6.57) 

This equation can be used to identify 0„[ and g as the 
LS solution of the system 


[K, L 0 A - L ] = ((V, —V„ iL ) -K„ L )(g r 0„ 7 L f. 

(6.58) 


Using Two Sets of Trajectories 
(Same or Different), the First Set Without Load 
and the Second Set with the Load 
Introducing (6.55) and the known load inertial parame¬ 
ters (6.56) in (6.54) gives 



(6.59) 


Ktot = USV r , 


(6.64) 


where U and V are orthonormal matrices, and S = 
diag(s,) is a diagonal matrix with singular values s ; 
of K tot sorted in decreasing order. There are infin¬ 
ity of vectors 0 tot = X\ c which are the TLS solutions 
of (6.61), depending on a scale factor A and on the 
normalized TLS solution given by the last column V c 
of V, 11 V f || = 1, corresponding to the smallest singular 

A. * 

value s c [6.34]. The unique solution 0 tot assumes that its 

A * 

last coefficient (/> iot (end) must be equal to 1 according 
to (6.63) such as 


0tot — 


Vc 

V c (end) 


(6.65) 


The methods has been successfully experimen¬ 
tally validated on the 6-DOF Staubli TX-40 industrial 
robot [6.35]. 

This approach is very simple to perform and the ex¬ 
perimental results have shown its effectiveness: for the 
global identification of the total joint drive gains and the 
dynamic parameters, it is only necessary to accurately 
weigh a payload mass and to carry standard trajectories 
of industrial robot. 


6.3.4 Link Parameter Estimation 
for More Complex Structures 


Equation (6.59) can be used to identify all the pa¬ 
rameters, g , 0 and 0„ l, as the single global LS solution 


In this section we present the dynamic identification 
model for kinematic tree (spanning tree) robots and 
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kinematic closed-loop robots including parallel robots. 
These models are linear in the inertial parameters and 
can be represented by a model similar to (6.47), thus 
these parameters can be identified using the same tech¬ 
niques. 

Tree-Structured Robots 

For an n-link manipulator with a kinematic tree struc¬ 
ture, the dynamic identification model (6.46) must be 
modified to take into account that the inertial parame¬ 
ters of link j have no effect on the dynamics of links 
which do not belong to the chain from the base to that 
link. For such structures, the link on which link j is ar¬ 
ticulated is denoted by p(j) (parent of j - see Sect. 3.4). 
It could be a link with any number i, such that i < j. 
Thus the nonzero elements of the column of the matrix 
representing the coefficients of the inertial parameters 
of link j in (6.46) would be 

'X;'A/.*X;'.'X; : 'A/.... ,*XpA j , (6.66) 

where b is the first link of the chain connecting link 0 to 
link j, thus p(b) = 0. 

Consequently, in (6.47) of a tree robot we obtain: 
Ky = Oixio if i > j or if link i does not belong to the 
branch connecting link 0 to link j. This means that some 
elements of the upper-right submatrix of K will be zero. 

We note that a serial structure is a particular case 
of a tree structure where p(j) =j — 1. Thus any link m 
such that m < j will belong to the chain from the base 
to link j. 

Closed-Loop Robots 

The dynamic model of a closed-loop structure can be 
obtained by considering first an equivalent spanning 
tree by opening one joint at each closed loop, and then 
using the principle of virtual work such that 

T=G T K,r0. G =(^)’ (6-67) 

with < 7 a an IV x 1 vector of N active joint angles (N dif¬ 
fers from n, the total number of joints), and q a an n x 1 
vector of the joints of the spanning tree structure. 


Parallel Robots 

A parallel robot is a particular case of closed-loop 
robots (Chap. 18). It is composed of a moving platform, 
representing the terminal link, that is connected to the 
base by m parallel legs. The dynamic model of parallel 
robots can be given by [6.36] 



( 6 . 68 ) 


where represents the dynamic model of leg i 

with K, a function of (q,. q r q,), q, is the vector of joint 
angles of leg i, A p 0 p is the Newton-Euler spatial force 
of the platform calculated in terms of the Cartesian vari¬ 
ables of the mobile platform using (6.41), and J is the 
Jacobian matrix of the parallel manipulator. We can ob¬ 
tain (6.68) from (6.67) by supposing that the spanning 
tree structure is obtained by separating the mobile plat¬ 
form from the legs. 

Equation 6.68 can be rewritten as 


T = [j T A/-> (dqi/dq^Kx ■■■ (dqjdqj K„,] 

0par 

T — Kpar (' C' ; ' kfy' 7 ‘ 7 7 ) 0par ■ 


(6.69) 

where 0 par is the vector of the inertial parameters of the 
robot (the legs and the platform). 


/ 

0i 


\0m/ 


In the common case where the legs are identical and 
their inertial parameters are denoted by 0 i eg , the identi¬ 
fication model can be rewritten by the following equa¬ 
tion, which considerably reduces the number of inertial 
parameters to be identified 

T = [J T Ap Er=t(3? i /3?a) T K i ]^J. (6.70) 

The identification of the orthoglide parallel robot is 
given in [6.37]. 


6.4 Identifiability and Numerical Conditioning 


That some inertial parameters in (6.47) are unidentifi¬ 
able does not mean that they are intrinsically identifi¬ 
able, only that the experimental setup does not allow 
them to be identified. Limited motion near the base 
could be fixed by placing the whole manipulator on 


a six-axis moving platform such as a Stewart-Gough 
platform. In fact, for mobile manipulators mounted 
on high-mobility vehicles such as satellites, it may 
be necessary to know the full inertial model of the 
manipulator. Additional sensors could be added, for ex- 
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ample a six-axis force-torque sensor to the base of the 
robot [6.5], to identify some additional (but not all) in¬ 
ertial parameters. 

A similar situation may arise in kinematic cal¬ 
ibration, where for example joint models might be 
augmented to include gear eccentricities, transmission 
and coupling coefficients of gears, joint elasticity, link 
elasticity, and base deflection [6.38]. By instrumenting 
the robot with additional sensors, for example, by plac¬ 
ing joint angle sensors before and after the gears to 
measure joint deflection, it may be possible to identify 
additional parameters of interest. Additional sensing 
with a theodolite was done to measure base deflection 
in [6.12], 

Given that the experimental setup is what it is, there 
may be no recourse but to deal with unidentifiable pa¬ 
rameters, treated in the next section. Another problem 
is that parameters might in principle be identifiable, but 
numerical conditioning prevents their accurate determi¬ 
nation. Issues such as adequacy of the collected data, 
and the effect of different units and magnitudes of pa¬ 
rameters, are treated subsequently. 

6.4.1 Identifiability 

There are two main approaches towards handling 
unidentifiable parameters, depending on whether the 
goal is a structural model or a prediction model. For 
a structural model, the goal is to find the minimum 
parameter set that provides a meaningful physical de¬ 
scription of the system by eliminating parameters until 
all are identifiable. This is done by careful evaluation of 
the effect of each parameter of the original model. 

For a prediction model, the goal is to match the out¬ 
puts to the inputs as closely as possible, and is more of 
a curve-fitting approach. The resulting parameter values 
are not necessarily physically meaningful or accurate. 

Structural Model 

It may be possible in the initial model formation to 
avoid including redundant or unidentifiable parameters. 
At other times, it is not possible to determine a priori 
what the minimal parameter set is, because of system 
complexity or because of numerical difficulties such as 
measurement error or limitations in the collected data. 

A Priori Parameter Elimination. In the initial model 
formation, the choice of representation can immediately 
dictate whether the model is redundant or minimal. 
For kinematic calibration, the minimal parameter set 
includes four parameters for a rotary joint and two pa¬ 
rameters for a prismatic joint. The detailed presentation 
in Sect. 6.2.1 of how to set up the DH/Hayati param¬ 
eters for different kinematic and sensing arrangements 


has the advantage that the parameter set is minimal (ex¬ 
cepting joint models). Once done, the application to any 
manipulator is somewhat formulaic. 

Five- and six-parameter joint models have also been 
proposed, either for the convenience of locating link 
coordinate systems or for the ease of model forma¬ 
tion [6.13, 28, 39]. With such redundant parameter sets, 
extra steps must be taken to handle the numerical diffi¬ 
culties caused by the redundancy, such as by reducing 
the number of parameters in a postprocessing step. 
When there are complicated joint models that include 
gearing effects as mentioned above, the resultant large 
numbers of parameters make the parameter elimination 
problem more difficult. Determining the minimal set of 
inertial parameters offers a different sort of complexity, 
because large numbers of parameters are not identi¬ 
fiable or are identifiable only in linear combinations. 
Again, model reduction is one approach to handle this 
problem. 

Two approaches towards determining a minimal or 
base parameter set are: 

1. Numerical identification of the unidentifiable pa¬ 
rameters or linear combinations of parameters. 

2. Symbolic determination. 

Numerical identification involves rank reduction 
through matrix manipulation of the regressor, either 
through QR decomposition or singular value decom¬ 
position [6.40]. If the kinematic or dynamic model is 
known exactly, then a simulation using exactly gen¬ 
erated data will yield a noise-free regressor matrix A 
in (6.3). For QR decomposition, the regressor matrix is 
factored as 

A = Q R ), (6.71) 

V U M/>-W par ,W par / 

where Q is an M x MP orthogonal matrix, R is an 
Apar xfVpar upper-triangular matrix, and <W_iv parii v pal is 
the zero matrix with dimensions MP — (V par x7V par . The¬ 
oretically, the unidentifiable parameters <p, are those 
whose corresponding diagonal elements /(,, of matrix R 
are zero. In practice, |R„ | is considered to be zero if it is 
less than the numerical zero f [6.41] 

£ = MPe max \Ra\ , (6.72) 

i 

where e is the computer precision [6.41]. Other param¬ 
eters may appear in linear combinations depending on 
how many nonzero elements there are in row j of R. 
Resolving these linear combinations is arbitrary. One 
approach is to zero all elements in the linear combina¬ 
tion save one. The result is a prediction model rather 
than a structural model. 
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Symbolic determination of the base inertial param¬ 
eters has been proposed in [6.40,42]. Using an energy 
formulation, the total energy of link j is expressed 
as hjfpj, where hj is a row vector termed the total energy 
function whose elements are kinematic expressions of 
the angular and linear velocity of link j plus gravity. 
A recursive relation between neighboring links is de¬ 
veloped as 

hj = + qjtjj , (6.73) 

where the elements of the 10 x 10 matrix 7 —1 Xj are func¬ 
tions of the DH parameters defining frame j, and the 
elements of the 1 x 10 row vector t]j depend on the 
linear and angular velocities. The details of these ex¬ 
pressions can be found in [6.40,42]. Grouping rules are 
then developed to find the precise linear combinations 
of parameters. 

The minimum inertial parameters for a tree struc¬ 
ture can be obtained using a closed-form solution simi¬ 
lar to the serial structure case [6.42]. 

The term K tr <J> in (6.67) shows that the minimum 
parameters of the spanning tree structure can be used 
to calculate the dynamics of the closed structure, giving 
by this a first reduction and grouping of the inertial pa¬ 
rameters. However extra parameters can be eliminated 
or grouped when considering the matrix G. The case of 
parallelogram loops can be treated symbolically using 
closed-form relations [6.42]. For general closed loops, 
the minimum parameters must be determined using nu¬ 
merical methods such as QR decomposition [6.40]. 

In the case of parallel robots, we deduce from ( 6 . 68 ) 
that the minimum parameters of the legs can be used 
to calculate K, 0 ,, giving a first reduction of the iner¬ 
tial parameters. However, some other parameters can be 
grouped with the parameters of the platform. The mini¬ 
mum parameters of the Gough-Stewart robot are given 
in [6.43]. 

Data-Driven Parameter Elimination. The singular 
value decomposition of the regressor matrix can show 
which parameters are unidentifiable, weakly identifi¬ 
able, or identifiable only in linear combination. For A par 
parameters, P data points, and M-dimensional output 
measurements at each data point, the regressor ma¬ 
trix A (6.3) or (6.7) can be decomposed as 

A = UZY t , (6.74) 

where U is an MP x MP orthogonal matrix, Y is an 
Ypar x A par orthogonal matrix, and T, is the MP x 7V par 
matrix of singular values 

£ = ( S ), (6.75) 

\™MP iVpar.Vpar / 


where S = diag(yti,.... p r , 0,_0) is the A par x A par 

matrix of ordered singular values, with /.i 1 the largest 
and fi r the smallest nonzero singular value. There may 
be A par —;'zero singular values/r,. 4-1 = ■ = 41 Wpar = 0 . 

Especially when complex joint models are assumed 
that include flexibility, backlash, and gear eccentric¬ 
ity, it is not clear that all parameters can be identified. 
Retaining poorly identifiable parameters will degrade 
the robustness of the calibration; such parameters are 
indicated by zero or very small singular values. The ex¬ 
pansion of (6.7) in terms of (6.74) is 

r 

A y = Pj(vJ A(j))Uj , (6.76) 

1 

where Uj and Vj are the j -th columns of U and V. For 
zero or small singular values fjj, the projection vj A <j> 
of the parameters onto the column vector Vj in general 
represents a linear combination of parameters. It is also 
possible that only a single parameter results from the 
projection. 

To proceed, it is first necessary to scale the pa¬ 
rameters and the output measurements so that the sin¬ 
gular values are comparable. Scaling is discussed in 
Sect. 6.4.3. Small singular values signal that there are 
parameters that are poorly identifiable and that should 
be eliminated. By small singular values, Schroer [6.38] 
suggests the heuristic that the condition number of 
a well-conditioned regressor matrix should be less 
than 100 

k( A) = — < 100 . (6.77) 

Pr 

This heuristic derives from experience of their statisti¬ 
cal community. If the condition number is above 100, 
the singular values are examined, beginning with the 
smallest one, which may be a zero singular value. 

If the condition number is above 100, the lin¬ 
ear sums (6.76) corresponding to the smallest singular 
value ji r are examined. The elements of column v r are 
in one-to-one correspondence with the elements of A <j>. 
If there is an element j of v r that is much larger than 
the others, then the parameter corresponding to that 
column element is a candidate for elimination. This 
procedure will tend to pinpoint parameters that are to¬ 
tally unidentifiable. Isolating the largest element of v r 
is once again only meaningful if the parameters have 
been previously scaled. 

Once the parameter is eliminated, the condition 
number of the reduced regressor is computed again. The 
procedure iterates until the condition number of the re¬ 
gressor is below 100 . 

The previous procedure can be carried out using QR 
decomposition as in (6.71), by replacing e representing 
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the computer precision in (6.72) by a greater value func¬ 
tion of the noise level. 

Prediction Model 

If there are several largest elements of that are 
roughly comparable in magnitude, then these param¬ 
eters may only be identifiable in linear combination. 
There will be the same number of singular values which 
are too small. By examining several columns Vj cor¬ 
responding to the small singular values, these linear 
combinations can become apparent. The linear combi¬ 
nations can be resolved arbitrarily, i. e., all parameter 
elements in the linear combination save one can be set 
to zero. The result of zeroing some parameters is that 
the model is no longer a structural model, but rather 
a prediction model. 

One may also proceed without first removing pa¬ 
rameters. It can be shown by substituting the singular 
value decomposition that 


tional rows to (6.3) that reflect this preference 

(?)♦-(;)■ 

To proceed with the solution, we can treat 0o as a con¬ 
stant. Redefine the parameter vector as 0 = 0 — 0 0 . 
which we expect to be close to zero. Then 

where y = y — A0o. We may not know 0 0 perfectly 
well, so we add a weighting parameter A to express the 
confidence in this value 



The larger is A, the more confidence we have in our 
a priori estimate. The least-squares solution is therefore 


(A t A) -1 A t = V (S- 1 0 Wpar , M/ >_ w J U T . (6.78) 

Consequently the solution to ( 6 . 8 ) can be expressed 
as [6.44] 




ujAy 

— - '1 

fij 


(6.79) 


One can see that poorly identifiable parameters corre¬ 
sponding to small singular values /z ; - greatly perturb the 
estimates, because the weighting is 1 / pj. The strategy is 
to remove their influence. If pj is zero or very small rel¬ 
ative to the largest singular value p\, then set 1 / pj = 0 . 

Parameters that cannot be identified well are simply 
ignored in this procedure, which automatically con¬ 
verges to an identifiable parameter set. The resulting 
parameters can then be used for the model. A disadvan¬ 
tage is that the resulting parameters may not correspond 
to real model parameters. A procedure to project these 
parameters on the real ones is given in [6.40]. 


Incorporating A Priori Parameter Estimates 
Least squares treats parameter values as completely un¬ 
known, i. e.. they could be anywhere in the range from 
—00 to + 00 . Yet often a fairly good initial estimate of 
the parameters is available, for example, from a manu¬ 
facturer’s specifications or in the case of a recalibration. 
It makes sense to incorporate such an a priori preference 
into the least-squares optimization [6.45]. 

Suppose there is a preference for a solution 
near 0 = 0o. Express the preference as 10 = 0o, 
where I is the identity matrix, and append it as addi¬ 


M (at ai) G a i)) I(at ai)T © 

= (A T A + A 2 I) _1 A T y . (6.83) 

This solution is called damped least squares, where A 
is the damping factor. Expanding this solution in terms 
of the singular value decomposition, 

= Y J ( u Jy)-rh 2 v J ■ < 6 - 84 ) 

j— 1 n ~ I_A 

Hence a very small pj is counteracted by the larger A 
value: then the a priori information about parameter val¬ 
ues dominates the information from the data, i. e., the 
data are ignored. Thus for damped least squares, no ex¬ 
plicit action on the singular values is required, because 
the damping factor modifies the singular values. The so¬ 
lution will be perturbed over the normal least-squares 
solution by the magnitude of the A choice. 

Confidence Measure of Parameter Estimates 
After calibration, an estimate M of the covariance of the 
parameter estimates can be derived from the data [6.3]. 
Assume that the task variables Ay have previously been 
scaled for equal uncertainty, that there is no bias, and 
that the errors are uncorrelated. Then 

M = ct 2 (A t A) -1 . (6.85) 

An estimate of the standard deviation a after running 
the calibration procedure is obtained from the y 2 statis¬ 
tic [6.44,46] 

y 2 = (Ay — AA0) T (Ay — AA0) . 


( 6 . 86 ) 
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An unbiased estimator for a 2 is <f J = y 2 / v, where v = 
MP — iVpar is called the statistical degrees of freedom', v 
subtracts from the total number of measurements MP 
the number (V par of parameters estimated, since some of 
the measurements went into determining <j>. 

The estimate M can be used as a basis for elimi¬ 
nating parameters, by choosing those with the largest 
variances. 

6.4.2 Observability 


This is similar to D-optimality. The rationale is that 0\ 
represents the volume of a hyperellipsoid in Ay, de¬ 
fined by (6.7) when A <j> defines a hypersphere, and the 
singular values represent the lengths of axes. Therefore 
maximizing 0\ gives the largest hyperellipsoid volume, 
and hence a good aggregate increase of the singular 
values. O ne can als o derive 0\ from the well-known 
relation y/ det( A T A) = >Lti • • • p r - 

Minimizing the condition number of A as a measure 
of observability has been proposed in [6.40, 52, 53] 


The measurements that are taken will influence the 
accuracy of parameter estimation. In kinematic cali¬ 
bration, the quality of the pose set has been measured 
by an observability index. In inertial parameter esti¬ 
mation, the quality of the identification trajectory has 
been termed persistent excitation [6.47], Regardless 
of whether data is collected statically as in kinematic 
calibration or dynamically as in inertial parameter es¬ 
timation, the result is just a bunch of numbers that go 
into the regressor matrix, and so common terminology 
is appropriate. 

In statistics, optimal experimental design theory has 
given rise to several data measures termed alphabet op¬ 
timalities [6.48]. Some of the most prominent are the 
following: 

• A-optimality minimizes the trace of (A T A )~ 1 to ob¬ 
tain regression designs. 

• D-optimality maximizes the determinant of 
(A t A) -1 . 

• E-optimality maximizes the minimum singular 
value of (A t A) _1 . 

• G-optimality minimizes the maximum prediction 
variance, and does not have a simple expression in 
terms of singular values. 


Although ties to the experimental design liter¬ 
ature have not typically been made [6.49], sev¬ 
eral of the proposed observability indexes for robot 
calibration have an alphabet-optimality counterpart. 
A-optimality does not have a counterpart in the 
robot calibration literature, and conversely a few pro¬ 
posed observability indexes do not have an alphabet- 
optimality counterpart. In [6.49], it is proved that 
E-optimality and G-optimality are equivalent for exact 
design. 

Bonn et al. [6.50,51] proposed an observability 
index (here termed O \ and numbered consecutively be¬ 
low) that maximizes the product of all of the singular 
values 


_ j/pi ■■■ p, r 

s/P 


(6.87) 


0 2 = — . ( 6 . 88 ) 

Pr 

O 2 measures the eccentricity of the hyperellipsoid 
rather than its size. The intermediate singular values are 
not considered to be that pertinent, because minimizing 
the condition number automatically makes all singular 
values become more similar in magnitude and makes 
the hyperellipsoid closer to a hypersphere. 

Nahvi et al. [6.22] argue for maximizing the mini¬ 
mum singular value pt r as an observability measure 

0 3 = p r . (6.89) 


This is similar to E-optimality. The rationale is to make 
the shortest axis as long as possible, regardless of the 
other axes, that is to say, to improve the worst case. 
Consider the following standard result [6.4] 


Pr < 


II Ay II 

l|A0|| 


< Pi 


or more particularly 


(6.90) 


Pr IIA01| < || Ay || . 


(6.91) 


Then maximizing p, r ensures that a given size of pa¬ 
rameter errors || A(j> || has the largest possible influence 
on the pose errors || Ay ||. 

Nahvi and Hollerbach [6.54] proposed the noise 
amplification index O 4 , which can be viewed as com¬ 
bining the condition number O 2 with the minimum 
singular value O 3 

<T r 

0 4 = — . (6.92) 

04 

The rationale is to measure both the eccentricity of the 
ellipse through Oi as well as the size of the ellipse 
through O 3 . The noise amplification index was argued 
to be the most sensitive to measurement error and mod¬ 
eling error. 

Hollerbach and Lokhorst [6.21] found experimen¬ 
tally that the condition number and the minimum sin¬ 
gular value gave about the same good result: their 
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relative magnitudes were almost proportional to the 
root-mean-square (RMS) errors of the final parameter 
errors. The observability index 0\ was not as sensitive 
and not directly related to parameter errors. Sun and 
Hollerbach [6.49] derived general relations among the 
observability indexes and alphabet optimalities 

Oi >A -optimality > O 3 . (6.93) 

They further showed that if /i 1 > 1, then O 3 > O 2 , and 
if fi r < 1, then Oi > O 4 . They also argued that O 3 
(D-optimality) is in general the best index, because it 
minimizes the variance of the parameters and also min¬ 
imizes the uncertainty of the end-effector pose. 

Optimal Experimental Design 
An observability index is typically used to decide how 
many data points to collect. As one begins to add 
data points, the observability increases, then plateaus. 
Adding additional data does not then improve the qual¬ 
ity of the estimates. For kinematic calibration, data may 
be added through random selection, or optimal design 
methods can be employed that can drastically reduce 
the amount of data required [6.55, 56]. Optimal experi¬ 
mental designs work by measuring the effect of adding 
or exchanging data points [6.57]. 

Exciting Trajectories 

For inertial parameter estimation, data points are not in¬ 
dependent because they result from a trajectory rather 
than isolated poses. Therefore the issue is what type 
of trajectory to generate. Industrial robots usually 
have joint position point-to-point trajectory generators. 
A continuous and smooth trajectory is calculated by in¬ 
terpolating between these points, assuming zero initial 
and final velocity and acceleration at each point, and us¬ 
ing polynomial interpolators. Excitation trajectories are 
obtained by minimizing an observability index, using 
nonlinear optimization techniques to calculate the poly¬ 
nomial coefficients, under the constraints of the joint 
positions, velocities, and accelerations limits [6.52], 

It is possible to facilitate the optimization by pro¬ 
ceeding to a sequential excitation procedure. Specific 
trajectories, which structurally excite a small number of 
parameters, are easier to optimize. For example, mov¬ 
ing one joint at a time with constant velocities excites 
friction and gravity parameters. In this approach, se¬ 
quential identification is avoided. However, it is better 
to collect all the data and to proceed to a globally 
weighted least-squares estimation [6.29]. This proce¬ 
dure avoids the accumulation of estimation errors and 
allows the calculation of the confidence intervals (6.85). 

Specific trajectories have been proposed like si¬ 
nusoidal interpolators [6.58], or periodic trajecto¬ 


ries obtained from a spectral analysis of the con¬ 
tribution function of the parameters [6.59]. This is 
a general planning trajectory strategy, which is very 
important to get accurate experimental identifica¬ 
tion [6.60]. 

6.4.3 Scaling 

The numerical conditioning of parameter estimation 
can be improved by scaling both the output measure¬ 
ments (task variable scaling) and the parameters. 

Task Variable Scaling 

When performing a least-squares analysis on the end¬ 
point pose error, position errors and orientation errors 
have to be combined (6.22) 

MAyii 2 HiA-y, +1 ii 2 +iiA-y, +1 ii 2 . (6.94) 

However, position error and orientation error have dif¬ 
ferent units, and so are not comparable. Furthermore, 
not all position or orientation components may be mea¬ 
sured with equal accuracy. 

Ordinary least squares (6.8) weighs all task vari¬ 
ables equally. To weigh these variables differently, the 
general solution is left multiplication of (6.7) by a scal¬ 
ing matrix G [6.45], 

GAy = GAA0 , 

Ay = AA<j> , (6.95) 

where Ay = GAy is the scaled output vector and A = 
GA is the scaled regressor matrix. The weighted least- 
squares solution is 

A<p = (A T A)~ 1 A T Ay = (A T WA) _1 A T WAy , 

(6.96) 

where W = G T G. Often W is a diagonal matrix. 

One approach to scaling position error versus ori¬ 
entation error is to equalize the effect of a parameter 
error A (pi on either position error or orientation er¬ 
ror. Curiously, for human-sized robot arms the effect 
is equal without scaling due to an accident of metric 
units. If 6 is the joint angle resolution, then s = r6 is 
the resulting endpoint position resolution. For human¬ 
sized arms, r = 1 m and hence s = 0. Thus meters and 
radians are directly comparable after all. Thus using 
no scaling factors for the pose parameters makes some 
sense, and may explain why the general disregard for 
scaling in the robotics community has not had more 
consequences. If linkages are much smaller (like fin¬ 
gers) or much larger (like excavators) then the situation 
is different. 
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A more general way of choosing the weighting ma¬ 
trix W is to use a priori information about acceptable 
relative errors. Such information might arise from spec¬ 
ifications of the measurement apparatus. Suppose that 
the output variables are subject to independent Gaus¬ 
sian noise, such that aj is the standard deviation in task 
variable measurement component A yj, for j = 1 ,,m 
pose components. Then the individual diagonal weights 
are Wy = 1 /aj, and define 

R ! = diag [K ) 2 - • • • - «) 2 ] > 

R = diag(R'.RO , 


where the weighting matrix W = R — ', and R is called 
the covariance matrix. 

The weighted least-squares estimate is 

A0 = (A T R _1 A) _1 A T R _1 Ay . (6.97) 

The resulting scaled output variable Ayj = AyJ/aj is 
dimensionless. The larger the uncertainty aj. the less 
this variable influences the least-squares solution rel¬ 
ative to other variables. The standard deviations aj 
are not necessarily the same as endpoint measurement 
accuracy, because model errors and input noise also 
contribute to output errors. 

The weighted least-squares solution using standard 
deviations is variously called the Gauss-Markov esti¬ 
mate, the generalized least-squares estimate, or the best 
linear unbiased estimator (BLUE) [6.3]. It is the mini¬ 
mum covariance estimate (on the parameter error) of all 
unbiased estimators. A significant point is that the stan¬ 
dard deviations of the scaled components of Ay are all 
about the same size, or the covariance R = cov( Ay) = 
I. the identity matrix. Hence the Euclidean norm of the 
error vector Ay is a reasonable measure of its size. 

Often we do not know the covariance matrix R that 
well. An estimate of the standard deviations after run¬ 
ning the calibration procedure is obtained from the / 2 
statistic [6.44,46] 

X 2 = (Ay-AA0) T R -1 (Ay-AA0) . (6.98) 

This equation is the same as the residual error (6.95), 
with substitution for W = R —1 . y : is nothing more than 
the weighted residuals after calibration. The expected 
value of x 2 is 

E(x 2 ) = v = PK-R , (6.99) 

where E is the expectation operator. That is to say, 
the unweighted residuals (Ay —AA0) 2 should, with 
enough measurements, approximate the true covari¬ 
ance. We may now uniformly scale an initial estimate 


of R after a preliminary calibration, based on the value 

of/ 2 . 


R = —R . ( 6 . 100 ) 

v 


where R is the revised estimate of the covariance ma¬ 
trix. 


Parameter Scaling 

Scaling of parameters is important for proper con¬ 
vergence in nonlinear optimization and for singular 
value decomposition. If parameters have vastly differ¬ 
ent magnitudes, then the singular values are not directly 
comparable. Also, parameter scaling can improve the 
conditioning of the regressor A and help to avoid in- 
vertibility problems. 

Whereas left multiplication of A in (6.95) results 
in task variable scaling, right multiplication of A by 
a weighting matrix H results in parameter scaling [6.45] 

Ay = (AH)(H _ 1 A0) = AA0, ( 6 . 101 ) 


where the scaled Jacobian and parameters are A = AH 
and A0 = H -1 A0. The least-squares solution is not 
changed by parameter scaling, whereas it is by task 
variable scaling. 

The most common approach towards parameter 
weighting is column scaling, which does not require 
a priori statistical information. Define a diagonal ma¬ 
trix H = diag(/;i,..., h N ) with elements 


jlM 1 if Iky II 7 ^ 0 
1 )l if|M=o 


( 6 . 102 ) 


where a, is the /-til column of A. Then (6.101) becomes 


Wpar 

Ay = E (6.103) 

j_ 1 Ill'll 


Suppose that Ay has been previously normalized; then 
its length is meaningful. Each a ; /||fly|| is a unit vector, 
and hence each scaled parameter A0 ; j|a / j| is about the 
same size and has the same effect on Ay. 

Schroer [6.38] identified a problem with column 
scaling, namely that poor identifiability of parameters 
can result in very small Euclidean norms, which re¬ 
sult in very large scaling factors. This results in strong 
amplification of uncertainties of A. Instead, Schroer 
proposed scaling based on the effect on the anticipated 
error of the robot (as previously discussed under task 
variable scaling). 
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In an ideal case, one would have a priori knowl¬ 
edge of the expected value 00 of the parameter vector 
and the standard deviation a? of each of the parameter 
vector components. More generally, the parameter dis¬ 
tribution would be described by a covariance matrix M, 
but usually this information is not available. Instead, the 
estimate M of the covariance from (6.85) can be used. 

If the output measurement covariance R 1 and pa¬ 
rameter error covariance M are both available, one may 
define a new least-squares optimization criterion that 
combines output errors with parameter errors to yield 
a new x~ statistic 

X 2 = (Ay-AA0) T R - 1 (Ay-AA0) 

+ A0 T M _1 A0. (6.104) 

Its solution is the minimum-variance estimate, which 
unlike (6.97) is biased 

A0 = (A T R _1 A + M _I ) _I A T R _I Ay . (6.105) 

The Kalman filter recursively solves the same prob¬ 
lem [6.61,62]; when the state is nonvarying, there is 
a constant process, and there is no process noise [6.13, 
63]. The Gauss-Markov estimate is the limiting case 
when M ~ 1 is zero, i.e., there is no a priori informa¬ 
tion about the parameters. Again, there is an issue of 
determining the covariances. As for the Gauss-Markov 
estimate, the expected value of / 2 can be used to uni¬ 
formly scale R and M post facto [6.64]. 

6.4.4 Recursive Least Squares 
and the Kalman Filter 

In a control setting, it may be necessary to update the 
estimate 0 at time step k based upon all the readings up 
to time step A: — 1. It is possible to recast least squares 
recursively to perform this updating [6.3], which is 
strongly analogous to the Kalman Filter. At time step k, 
there is a new reading 

y k = A k 0 + e k , (6.106) 

where y k is of dimension Mx 1, A k is of dimension M x 
N, and 0 is of dimension N x 1. Let R' be the covariance 
of e k . This equation is stacked for all readings up to k, 
and the Gauss-Markov estimate is derived 

yk = A A .0 + e k 

0 , = (A^-'A.r'AjR,- 1 ^ , (6.107) 

where y k is of dimension MKx 1, and A k is of di¬ 
mension MK x N. Rj is the covariance of e k , and is 
a block-diagonal matrix with elements R k . 


To make an analogy to the Kalman filter, define the 
covariance of 0 as P k = (AjR^'A ^.) -1 of dimension 
N x N. Inverting this relation eventually yields a recur¬ 
sive reltionship in terms of the covariance up to the 
previous time, step, plus a new contribution 

p- 1 = P-_1, + (A k ) T (R k r l A k . (6.108) 

Similarly, 

AlRf'y, = A[_jR a -Wi + (A*) r (R*)-y . 

(6.109) 

Substituting the definition of P*, and (6. 109) into (6. 107) 
eventually yields 

k = k-i +p *(A*) r (R*r v - a*' 0,_i). 

( 6 . 110 ) 

In the covariance updating matrix (6.108), the in¬ 
verse P[7_| has to be taken. The updating relationship 
can be reworked into a computationally friendlier form 
using the binomial inverse theorem (see Wikipedia en¬ 
try) 

(A + UBV )- 1 = 

A -1 — A - 1 U(B -1 + VA - 1 U) - 1 VA -1 ( 6 . 111 ) 

Making the identification A = P^ 1; U = (A k ) T , B = 
(R A ) — *, and V = A k , after inversion (6.108) becomes 

Pifc = 

P^ 1 -P k - 1 (A k ) T (R k + A k P k - l (A k ) T r l A k P k - l 

( 6 . 112 ) 

The resulting matrix in parentheses is of size M x M, 
and is typically of much lower dimension than the 
N xN dimension of P^. That is to say, there are usually 
many more parameters than measurement components. 
Hence the inversion is easier. 

Equations ( 6 . 110) and (6.112) comprise a large part 
of the Kalman filter. Thus the Kalman filter is funda¬ 
mentally recursive least squares [6.3, 61, 62]. There are 
many variants to the Kalman filter, including one based 
on the mmse estimate. 

Gautier and Poignet [6.65] applied an extended 
Kalman filter to estimate a robot’s inertial parame¬ 
ters, in which the filter was reworked to include ve¬ 
locity and acceleration estimation. They found that 
there was no advantage to using a Kalman filter over 
least squares, and in fact there was a disadvantage in 
terms of sensitivity to initial conditions and slower 
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convergence. One issue in on-line versus off-line es¬ 
timation is the accuracy of velocity and acceleration 
estimates. In off-line estimation, two sided (or acausal) 


filters can be used, and they typically estimate time 
derivatives more accurately than one-side (or causal) 
filters. 


6.5 Conclusions and Further Reading 

This chapter has presented methods for calibrating 
the kinematic parameters and the inertial parameters 
of robot manipulators. Both methods are instances of 
parameter estimation using least squares. Inertial pa¬ 
rameters appear linearly in the equations of motion, and 
ordinary least squares can be used. Kinematic parame¬ 
ters appear nonlinearly because of sines and cosines, 
and so nonlinear estimation via the Gauss-Newton 
method can be applied. 

There are domain-specific issues for setting up the 
calibration equations. For kinematic calibration, Hayati 
parameters have to be mixed with Denavit-Hartenberg 
parameters to handle the case of nearly parallel joint 
axes. The calibration equations have to take into ac¬ 
count how the endpoint is measured or constrained. 
By examining in a detailed fashion the possible joint 
sequences, including parallel or prismatic joints, a min¬ 
imal parameterization is achieved that obviates prob¬ 
lems of identifiability. 

The calibration index has been presented as cate¬ 
gorizing all kinematic calibration methods. This index 
counts the number of equations generated per pose, by 
calculating the excess of sensing over mobility. A key 
is that all calibration methods can be viewed as closed- 
loop methods, where any endpoint sensing system is 
considered as a joint. Parallel robots are handled by in¬ 
corporating multiple closed loops. 

For inertial parameter estimation.the recursive 
Newton-Euler equations lead to a formulation of the 
regressor matrix as an upper-diagonal matrix. A min¬ 
imal parameterization is straightforward for serial and 
spanning robots. The load inertial parameters estima¬ 
tion has been treated by using either the joint torques 
or the terminal link force-torque sensor measurements. 
The proposed models are exploited to identify the joint 
torques gains supposing known load parameters. 

Numerical methods were presented to handle 
unidentifiable parameters if a minimal parameterization 
has not been achieved. These methods hinge on the 
singular value or QR decomposition of the regressor 
matrix. The singular values can be examined to de¬ 
cide which parameters are not identifiable and should 
be eliminated. Alternatively, a small singular value can 
simply be zeroed to eliminate the effects of poorly iden¬ 
tifiable parameters without explicitly eliminating them. 
The former yields a structural model, while the latter 


yields a prediction model. A priori estimates of param¬ 
eters can be taken into account by using damped least 
squares. 

The adequacy of the measurement set for parame¬ 
ter estimation is captured by different proposals for an 
observability index. This index can be related to the lit¬ 
erature on alphabet optimalities in experimental design. 

Finally, the scaling of measurements or of param¬ 
eters is important for a well-conditioned numerical 
estimation, and is essential in order to compare singu¬ 
lar values. When uncertainties in measurements and in 
parameters are included as weights, the optimal mini¬ 
mum variance estimate can be found, which is related 
to the Kalman filter. If not known beforehand, these un¬ 
certainties can be estimated from the data. 

6.5.1 Relation to Other Chapters 

Estimation involving least squares and Kalman filter¬ 
ing is discussed in Chap. 5 in a similar context. The 
problem of estimating properties of the world through 
sensors is very similar to model identification. Recur¬ 
sive estimation techniques are particularly appropriate 
when a robot needs to update its world model incremen¬ 
tally. For model identification, employing a recursive 
formulation is not particularly helpful, as the machin¬ 
ery of recursive updating can obscure numerical issues 
with the total data. 

The singular value decomposition appears in other 
contexts. Chapter 10 and 16 analyze equal motion ca¬ 
pability in different directions by measures similar to 
the observability indexes: 0 \ has a counterpart in the 
manipulability measure, and O 2 the condition number 
and O 3 the minimum singular value appear again. By 
contrast, the concern in calibration is good data in all 
directions as captured by the singular values. Chap. 10 
employs the singular value decomposition to analyze 
redundant mechanisms. Whereas parameter estimation 
generally is an overconstrained least-squares problem 
(many more measurements than parameters), redundant 
mechanisms are underconstrained (more joint angles 
than task variables). Instead of signalling identifiability 
problems, zero singular values indicate the null space of 
the Jacobian. Damped least squares is used in Chap. 10 
to avoid singularities. Just as the true parameters are 
perturbed by damped least squares in calibration, the 
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trajectory is perturbed in order to get around a numeri¬ 
cal conditioning issue. 

Sensors involved in positioning a robot can have as¬ 
pects of their sensor models calibrated as well, such 
as the gain of a potentiometer. Camera calibration is 
discussed in Chap. 32. Camera models can be deter¬ 
mined at the same time as the kinematic models of robot 
manipulators [6.39, 66], including intrinsic camera pa¬ 
rameters such as the pinhole model discussed in Chap. 5 
as well as the extrinsic parameters of where the camera 
is located. 

6.5.2 Further Reading 

Screw Axis Measurement 

An alternative to nonlinear least squares to estimate 
kinematic parameters is a class of methods which mea¬ 
sure the joint axes as lines in space, termed screw axis 
measurement in [6.8]. One approach is circle point 
analysis, which involves moving one joint at a time to 
generate a circle at a distal measurement point [6.13]. 
Other approaches measure the Jacobian matrix, which 
contains as columns the joint screws [6.67]. With 
knowledge of the joint axes, the kinematic parame¬ 
ters are straightforwardly extracted without nonlinear 
search. The accuracy of this class of methods may not 
be as good as the nonlinear least-squares methods. 

Total Least Squares 

Ordinary least squares assumes that there is only noise 
in the output measurements, but often there is noise 
in the inputs as well. Input noise is known to lead 
to bias errors [6.3]. A framework for handling both 


input and output noise is total least squares [6.34], 
also known as orthogonal distance regression [6.68] 
or errors-in-variables regression [6.69]. Nonlinear to¬ 
tal least squares has been applied to robot calibration 
in [6.2, 70, 71]. In the implicit loop method of [6.2], by 
treating endpoint measurements equally with joint mea¬ 
surements, no distinction is made between input and 
output noise. 

Methods with Direct 
and Inverse Dynamic Models 
The identification of dynamic parameters presented in 
this chapter is based on the Inverse Dynamic Identi¬ 
fication Model and linear LS techniques. A common 
procedure to validate the identification result is to sim¬ 
ulate the direct dynamic model with the identified 
parameters and compare the output with the real sys¬ 
tem. 

Recently two methods which use both the direct and 
the inverse dynamic models have been developed to im¬ 
prove the noise immunity of estimates with respect to 
low rate or corrupted data in the observation matrix 
resulting from noisy measurements and/or bad tuning 
of joint positions band-pass filtering. The first method 
needs only the joint torques, while the velocity and ac¬ 
celeration are obtained from the direct dynamic model. 
It carries out in a combined procedure the identifica¬ 
tion and the validation. Current work aims to use this 
technique to identify robots with flexible joints where 
the flexible position variables are not measurable. The 
second method uses an instrumental variable (IV) ap¬ 
proach. Both methods have been validated on a 6-DOF 
industrial robot [6.72,73]. 


Video-References 
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Calibration of ABB's IRB 120 industrial robot 

available from http://handbookofrobotics.org/view-chapter/06/videodetails/422 
Robot calibration using a touch probe 

available from http://handbookofrobotics.org/view-chapter/06/videodetails/425 

Calibration and accuracy validation of a FANUC LR Mate 200iC industrial robot 

available from http://handbookofrobotics.org/view-chapter/06/videodetails/430 

Dynamic identification of Staubli TX40: Trajectory without load 

available from http://handbookofrobotics.org/view-chapter/06/videodetails/480 

Dynamic identification of Staubli TX40: Trajectory with load 

available from http://handbookofrobotics.org/view-chapter/06/videodetails/481 

Dynamic identification of Kuka LWR: Trajectory without load 

available from http://handbookofrobotics.org/view-chapter/06/videodetails/482 

Dynamic identification of Kuka LWR: Trajectory with load 

available from http://handbookofrobotics.org/view-chapter/06/videodetails/483 

Dynamic identification of a parallel robot: Trajectory with load 

available from http://handbookofrobotics.org/view-chapter/06/videodetails/485 

Dynamic identification of Kuka KR270: Trajectory without load 

available from http://handbookofrobotics.org/view-chapter/06/videodetails/486 
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Dynamic identification of Kuka KR270: trajectory with load 
available from http://handbookofrobotics.org/view-chapter/06/videodetails/487 
I^SHKiliiasESa Dynamic identification of a parallel robot: Trajectory without load 

available from http://handbookofrobotics.org/view-chapter/06/videodetails/488 
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7. Motion Planning 


Lydia E. Kavraki, Steven M. LaValle 


This chapter first provides a formulation of the 
geometric path planning problem in Sect. 7.2 
and then introduces sampling-based planning 
in Sect. 7.3. Sampling-based planners are general 
techniques applicable to a wide set of problems 
and have been successful in dealing with hard 
planning instances. For specific, often simpler, 
planning instances, alternative approaches exist 
and are presented in Sect. 7.4. These approaches 
provide theoretical guarantees and for simple 
planning instances they outperform sampling- 
based planners. Section 7.5 considers problems 
that involve differential constraints, while Sect. 7.6 
overviews several other extensions of the basic 
problem formulation and proposed solutions. Fi¬ 
nally, Sect. 7.8 addresses some important and more 
advanced topics related to motion planning. 


7.1 Robotics Motion Planning . 139 

7.2 Motion Planning Concepts . 140 

7.2.1 Configuration Space . 140 

7.2.2 Geometric Path Planning Problem . 141 

7.2.3 Complexity of Motion Planning . 141 

7.3 Sampling-Based Planning . 141 

7.3.1 Multi-Query Planners: 

Mapping the Connectivity of Cf re e--. 142 


7.3.2 Single-Query Planners: 

Incremental Search . 143 


7.4 Alternative Approaches . 144 

7.4.1 Combinatorial Roadmaps . 145 

7.4.2 Roadmaps in Higher Dimensions... 146 

7.4.3 Potential Fields . 146 

7.5 Differential Constraints . 148 

7.5.1 Concepts and Terminology . 148 

7.5.2 Discretization of Constraints . 149 

7.5.3 Decoupled Approach . 149 

7.5.4 Kinodynamic Planning . 150 


7.6 Extensions and Variations . 151 

7.6.1 Closed Kinematic Chains . 151 

7.6.2 Manipulation Planning . 151 

7.6.3 Time-Varying Problems . 151 

7.6.4 Multiple Robots . 152 

7.6.5 Uncertainty in Predictability . 153 

7.6.6 Sensing Uncertainty . 153 

7.6.7 Optimal Planning . 154 


7.7 Advanced Issues . 

7.7.1 Topology of Configuration Spaces .. 

7.7.2 Sampling Theory . 

7.7.3 Computational Algebraic 

Geometry Techniques . 

7.8 Conclusions and Further Reading . 


154 

154 

155 

156 

157 



Video-References . 158 

References . 158 


7.1 Robotics Motion Planning 

A fundamental robotics task is to plan collision-free 
motions for complex bodies from a start to a goal 
position among a collection of static obstacles. Al¬ 
though relatively simple, this geometric path planning 
problem is computationally hard [7.1]. Extensions of 
this formulation take into account additional problems 


that are inherited from mechanical and sensor limi¬ 
tations of real robots such as uncertainties, feedback, 
and differential constraints, which further complicate 
the development of automated planners. Modern al¬ 
gorithms have been fairly successful in addressing 
hard instances of the basic geometric problem and 
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a lot of effort has been devoted to extend their ca¬ 
pabilities to more challenging instances. These algo¬ 
rithms have had widespread success in applications 
beyond robotics, such as computer animation, vir¬ 


tual prototyping, and computational biology. There 
are many available surveys [7.2—4] and books [7.5— 
7] that cover modern motion planning techniques and 
applications. 


7.2 Motion Planning Concepts 

This section provides a description of the fundamental 
motion planning problem or the geometric path plan¬ 
ning problem. Extensions of this basic formulation to 
more complicated instances will be discussed later in 
the chapter and will be revisited throughout this book. 

7.2.1 Configuration Space 

In path planning, a complete description of the geom¬ 
etry of a robot JA and of a workspace TV is provided. 
The workspace TV = R w , in which N = 2 or N = 3, is 
a static environment populated with obstacles. The goal 
is to find a collision-free path for _7L to move from an 
initial position and orientation to a goal position and 
orientation. 

To achieve this, a complete specification of the 
location of every point on the robot geometry, or 
a configuration q, must be provided. The configura¬ 
tion space, or C-space ( q e C), is the space of all 
possible configurations. The C-space represents the set 
of all transformations that can be applied to a robot 
given its kinematics as described in Chap. 2 (Kine¬ 
matics). It was recognized early on in motion planning 
research [7.8,9] that the C-space is a useful way to 
abstract planning problems in a unified way. The ad¬ 
vantage of this abstraction is that a robot with a com¬ 
plex geometric shape is mapped to a single point in 
the C-space. The number of degrees of freedom of 
a robot system is the dimension of the C-space, or 
the minimum number of parameters needed to specify 
a configuration. 

Let the closed set O C TV represent the (workspace) 
obstacle region, which is usually expressed as a collec¬ 
tion of polyhedra, three-dimensional (3-D) triangles, or 
piecewise-algebraic surfaces. Let the closed set S\(q) C 
TV denote the set of points occupied by the robot when 
at configuration q e C; this set is usually modeled using 
the same primitives as used for O. The C-space obstacle 
region, C 0 bs> is defined as 

C obs = {qeC | 7l( 9 )n(9^0}. (7.1) 

Since O and JA(q) are closed sets in TV, the obstacle 
region is a closed set in C. The set of configurations 
that avoid collision is Cf ree = C \ C 0 b s , and is called the 
free space. 


Simple Examples of C-spaces 
Translating Planar Rigid Bodies. The robot’s con¬ 
figuration can be specified by a reference point ( x, y) on 
the planar rigid body relative to some fixed coordinate 
frame. Therefore the C-space is equivalent to R 2 . Lig- 
ure 7.1 gives an example of a C-space for a triangular 
robot and a single polygonal obstacle. The obstacle re¬ 
gion in the C-space can be traced by sliding the robot 
around the workspace obstacle to find the constraints on 
all q e C. Motion planning for the robot is now equiva¬ 
lent to motion planning for a point in the C-space. 

Planar Arms. Ligure 7.2 gives an example of a two- 
joint planar arm. The bases of both links are pinned, so 
that they can only rotate around the joints, and there are 
no joint limits. Lor this arm, specifying the rotational 
parameters 9\ and 62 provides the configuration. Each 
joint angle 0 ,- corresponds to a point on the unit circle 
S 1 and the C-space is § 'x S 1 = T 2 , the two-dimensional 
(2-D) torus shown in Lig. 7.2. Lor a higher number of 



Configuration space obstacle 


Fig.7.1a,b A robot translating in the plane: (a) a triangu¬ 
lar robot moves in a workspace with a single rectangular 
obstacle. (b) The C-space obstacle 



Fig. 7.2 (a) A two-joint planar arm in which the links are 
pinned and there are no joint limits, (b) The C-space 
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links without joint limits, the C-space can be similarly 
defined as 

C = S 1 xS 1 x---xS 1 . (7.2) 

If a joint has limits, then each corresponding S 1 is often 
replaced with R, even though it is a finite interval. If 
the base of the planar arm is mobile and not pinned, 
then the additional translation parameters must also be 
considered in the arm’s configuration 

C = R 2 xS 1 xS 1 x---xS 1 . (7.3) 

Additional examples of C-spaces are provided in 
Sect. 7.7.1, where topological properties of configura¬ 
tion spaces are discussed. 

7.2.2 Geometric Path Planning Problem 

The basic motion planning problem, also known as the 
piano mover’s problem [7.1], is defined as follows. 
Given: 

1. A workspace TV, where either TV = R 2 or 

tv = r 3 . 

2. An obstacle region O C TV. 

3. A robot defined in TV. Either a rigid body or 
a collection of m links: JTi, JA 2 ,..., JT,„. 

4. The configuration space C (C 0 b s and Cfr ee are then 
defined). 

5. An initial configuration q\ e Cf ree . 

6 . A goal configuration qa e Cf ree - The initial and goal 
configuration are often called a query {qi,qo)- 

Compute a (continuous) path , r : [0, 1] —> Cf ree , 
such that r(0) = qi and r(l) = qo- 

7.2.3 Complexity of Motion Planning 

The main complications in motion planning are that it 
is not easy to directly compute C 0 b s and Cf ree , and the 
dimensionality of the C-space is often quite high. In 
terms of computational complexity, the piano mover’s 
problem was studied early on and it was shown to be 
PSPACE-hard by Reif [7.1]. A series of polynomial- 

7.3 Sampling-Based Planning 

Sampling-based planners are described first because 
they are the method of choice for a very general class 
of problems. The following section will describe other 
planners, some of which were developed before the 
sampling-based framework. The key idea in sampling- 
based planning is to exploit advances in collision 
detection algorithms that compute whether a single con- 


time algorithms for problems with fixed dimension 
suggested an exponential dependence on the problem 
dimensionality [7.10, 11]. A single exponential-time al¬ 
gorithm in the C-space dimensionality was proposed by 
Canny and showed that the problem is PSPACE-com- 
plete [7.12]. Although impractical, the algorithm serves 
as an upper bound on the general version of the basic 
motion planning problem. It applies computational al¬ 
gebraic geometry techniques for modeling the C-space 
in order to construct a roadmap , a one-dimensional 
(1-D) subspace that captures the connectivity of Cf ree ■ 
Additional details about such techniques can be found 
in Sect. 7.7.3. 

The complexity of the problem motivated work in 
path planning research. One direction was to study sub¬ 
classes of the general problem for which polynomial 
time algorithms exist [7.13]. Even some simpler, spe¬ 
cial cases of motion planning, however, are at least 
as challenging, for example, the case of a finite num¬ 
ber of translating, axis-aligned rectangles in R 2 is 
PSPACE-hard as well [7.14]. Some extensions of mo¬ 
tion planning are even harder. For example, a certain 
form of planning under uncertainty in 3-D polyhedral 
environment is NEXPTIME-hard [7.15]. The hardest 
problems in NEXPTIME are believed to require dou¬ 
bly exponential time to solve. 

A different direction was the development of al¬ 
ternative planning paradigms that were practical under 
realistic assumptions. Many combinatorial approaches 
can efficiently construct 1 -D roadmaps for specific 2-D 
or 3-D problems. Potential field-based approaches de¬ 
fine vector fields which can be followed by a robot 
towards the goal. Both approaches, however, do not 
scale well in the general case. They will be described 
in Sect. 7.4. An alternative paradigm, sampling-based 
planning, is a general approach that has been shown to 
be successful in practice for many challenging prob¬ 
lems. It avoids the exact geometric modeling of the C- 
space but it cannot provide the guarantees of a complete 
algorithm. Complete and exact algorithms are able to 
detect that no path can be found. Instead sampling-based 
planning offers a lower level of completeness guarantee. 
This paradigm is described in the following section. 


figuration is collision free. Given this simple primitive, 
a planner samples different configurations to construct 
a data structure that stores 1-D C-space curves, which 
represent collision-free paths. In this way, sampling- 
based planners do not access the C-space obstacles 
directly but only through the collision detector and the 
constructed data structure. Using this level of abstrac- 


Part A | 7.3 



Part A | 7.3 


142 Part A I Robotics Foundations 


tion, the planners are applicable to a wide range of 
problems by tailoring the collision detector to specific 
robots and applications. 

A standard for sampling-based planners is to pro¬ 
vide a weaker, but still interesting, form of com¬ 
pleteness: if a solution path exists, the planner will 
eventually find it. Giving up on the stronger form of 
completeness, which also requires failure to be reported 
in finite time, these techniques are able to solve in prac¬ 
tice problems with more than three degrees of freedom 
that complete approaches cannot address. More details 
on this weaker form of completeness are provided in 
Sect. 7.7.2. 

Different planners follow different approaches on 
how to sample configurations and what kind of 
data structures they construct. Section 7.7.2 provides 
a deeper insight on sampling issues. A typical classi¬ 
fication of sampling-based planners is between multi¬ 
query and single-query approaches: 

• In the first category, the planners construct a road¬ 
map, an undirected graph G that is precomputed 
once so as to map the connectivity properties 
of Cf ree . After this step, multiple queries in the same 
environment can be answered using only the con¬ 
structed roadmap. Such planners are described in 
Sect. 7.3.1. 

• Planners in the second category build tree data 
structures on the fly given a planning query. They at¬ 
tempt to focus on exploring the part of the C-space 
that will lead to solving a specific query as fast as 
possible. They are described in Sect. 7.3.2. 

Both approaches, however, make similar use 
of a collision checking primitive. The objective of 
a collision detector is to report all geometric contacts 
between objects given their geometries and transforma¬ 
tions [7.16-18]. The availability of packages that were 
able to answer collision queries in a fraction of a sec¬ 
ond was critical to the development of sampling-based 
planners. Modem planners use collision detectors as 
a black box. Initially the planner provides the geome¬ 
tries of all the involved objects and specifies which 
of them are mobile. Then, in order to validate a robot 
configuration, a planner provides the corresponding 
robot transformation and a collision detector responds 
on whether the objects collide or not. Many packages 
represent the geometric models hierarchically, avoid 
computing all-pairwise interactions, and conduct 
a binary search to evaluate collisions. Except from 
configurations, a planner must also validate entire 
paths. Some collision detectors return distance-from- 
collision information, which can be used to infer that 
entire neighborhoods in C are valid. It is often more 
expensive, however, to extract this information; instead 


paths are usually validated point by point using a small 
stepping size either incrementally or by employing 
binary search. Some collision detectors are incremental 
by design, which means that they can be faster by 
reusing information from a previous query [7.16]. 
Examples of problems solved by sampling-based 
planners are shown in l<gf 11'»'!£■ and 

7.3.1 Multi-Query Planners: 

Mapping the Connectivity of Cf ree 

Planners that aim to answer multiple queries for a cer¬ 
tain static environment use a preprocessing phase dur¬ 
ing which they attempt to map the connectivity proper¬ 
ties of Cf r ee onto a roadmap. This roadmap has the form 
of a graph G, with vertices as configurations and edges 
as paths. A union of 1-D curves is a roadmap G if it 
satisfies the following properties: 

1. Accessibility: From any q e Cff ee , it is simple and 

efficient to compute a path r : [0, 1] —> Cf ree such 
that r(0) = q and r(1) = s, in which s may be any 
point in S(G). S(G) is the swath of G, the union of 
all configurations reached by all edges and vertices. 
This means that it is always possible to connect 
a planning query pair q\ and q<, to some and s g, 

respectively, in S(G). 

2. Connectivity preserving: The second condition re¬ 
quires that, if there exists a path r : [0,1] — >■ Cf re e 
such that r(0) = q\ and r(l) = qc, then there also 
exists a path z' : [0, 1] —> S(G), such that r^O) = S] 
and r / ( 1) = Sc- Thus, solutions are not missed be¬ 
cause G fails to capture the connectivity of Cf re e- 

The probabilistic roadmap method (PRM) ap¬ 
proach [7.19] attempts to approximate such a road¬ 
map G in a computationally efficient way. The pre¬ 
processing phase of PRM, which can be extended 
to sampling-based roadmaps in general, follows these 
steps: 

1. Initialization: Let G(V,E) represent an undirected 
graph, which is initially empty. Vertices of G 
will correspond to collision-free configurations, and 
edges to collision-free paths that connect vertices. 

2. Configuration sampling: A configuration a(i ) is 
sampled from Cf ree and added to the vertex set V. 
a(-) is an infinite, dense sample sequence and a(i) 
is the r-th point in that sequence. 

3. Neighborhood computation: Usually, a metric is 
defined in the C-space, p:CxC->-R. Vertices q 
already in V are then selected as part of a(i)’s neigh¬ 
borhood if they have small distance according to p. 

4. Edge consideration: For those vertices q that do not 
belong in the same connected component of G with 
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u(i ) the algorithm attempts to connect them with an 
edge. 

5. Local planning method. Given u{i) and q £ Cf ree 
a module is used that attempts to construct 
a path x s : [0, 1] —> Cfr ee suc h that t( 0) = a(i ) and 
r (1 ) = q. Using collision detection, r v must be 
checked to ensure that it does not cause a collision. 

6. Edge insertion: Insert x s into E, as an edge from a (i) 
to q. 

7. Termination: The algorithm is typically terminated 
when a predefined number of collision-free vertices 
N has been added in the roadmap. 

The algorithm is incremental in nature. Computa¬ 
tion can be repeated by starting from an already existing 
graph. A general sampling-based roadmap is summa¬ 
rized in Algorithm 7.1. 


Algorithm 7.1 Sampling-Based Roadmap 
N: number of nodes to include in the roadmap 
1: G.init(); i <— 0; 

2: while i <N do 

3: if a (0 G Cfree then 

4: G.add_ vertex(a(i)); i <— i+ 1; 

5: for q € NEIGHBORHOOD^!'), G) do 

6: if CONNECT (a(i).q) then 

7: G.add_ edge (a(i), q)\ 

8 : end if 

9: end for 

10: end if 

11: end while 


An illustration of the algorithm’s behavior is de¬ 
picted in Fig. 7.3. To solve a query, qi and qc, 
are connected to the roadmap, and graph search is 
performed. 

For the original PRM [7.19], the configuration a(i ) 
was produced using random sampling. For the connec- 



Fig. 7.3 The sampling-based roadmap is constructed in¬ 
crementally by attempting to connect each new sam¬ 
ple, a(i), to nearby vertices in the roadmap 


tion step between q and a(i), the algorithm used straight 
line paths in the C-space. In some cases a connection 
was attempted if q and u( i) were in the same connected 
component in order to improve path quality. There have 
been many subsequent works that try to improve the 
roadmap quality while using fewer samples. Methods 
for concentrating samples at or near the boundary of 
Cffee are presented in [7.20,21]. Methods that move 
samples as far from the boundary as possible appear 
in [7.22,23]. Deterministic sampling techniques, in¬ 
cluding grids, appear in [7.24]. A method of pruning 
vertices based on mutual visibility that leads to a dra¬ 
matic reduction in the number of roadmap vertices 
appears in [7.25]. Theoretical analysis of sampling- 
based roadmaps appears in [7.24, 26, 27] and is briefly 
discussed in Sect. 7.7.2. An experimental comparison 
of sampling-based roadmap variants appears in [7.28]. 
One difficulty in these roadmap approaches is identify¬ 
ing narrow passages. One proposal is to use the bridge 
test for identifying these [7.29]. For other PRM-based 
works, see [7.30-34]. Extended discussion of the topic 
can be found in [7.5,7]. 

7.3.2 Single-Query Planners: 

Incremental Search 

Single-query planning methods focus on a single 
initial-goal configuration pair. They probe and search 
the continuous C-space by extending tree data struc¬ 
tures initialized at these known configurations and 
eventually connecting them. Most single-query meth¬ 
ods conform to the following template: 

1. Initialization: Let G{V,E) represent an undirected 
search graph, for which the vertex set V contains 
a vertex for one (usually q\) or more configura¬ 
tions in Cfree, and the edge set E is empty. Vertices 
of G are collision-free configurations, and edges are 
collision-free paths that connect vertices. 

2. Vertex selection method: Choose a vertex ^ cur £ V 
for expansion. 

3. Local planning method: For some </ new £ Cf ree , 
which may correspond to an existing vertex in V 
but on a different tree or a sampled configuration, 
attempt to construct a path x s : [0,1] —> Cf ree such 
that r(0)=<jr cur and r(l) = <jr new . Using collision 
detection, x s must be checked to ensure that it does 
not cause a collision. If this step fails to produce 
a collision-free path segment, then go to Step 2. 

4. Insert an edge in the graph: Insert x s into E, as an 
edge from q CUT to q new . If q new is not already in V, 
then it is inserted. 

5. Check for a solution: Determine whether G encodes 
a solution path. 
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6. Return to Step 2: Iterate unless a solution has been 
found or some termination condition is satisfied, in 
which case the algorithm reports failure. 

During execution, G may be organized into one or 
more trees. This leads to: 

1. Unidirectional methods, which involve a single tree, 
usually rooted at q\ [7.35], 

2. Bidirectional methods, which involve two trees, 
typically rooted at qi and qo [7.35], and 

3. Multidirectional methods, which may have more 
than two trees [7.36,37]. 

The motivation for using more than one tree is that 
a single tree may become trapped trying to find an exit 
through a narrow opening. Traveling in the opposite 
direction, however, may be easier. As more trees are 
considered it becomes more complicated to determine 
which connections should be made between trees. 

Rapidly Exploring Dense Trees 
The important idea with this family of techniques is that 
the algorithm must incrementally explore the properties 
of the C-space. An algorithm that achieves this objec¬ 
tive is the rapidly exploring random tree (RRT) [7.35], 
which can be generalized to the rapidly exploring dense 
tree (RDT) for any dense sampling, deterministic or 
random [7.7]. The basic idea is to induce a Voronoi bias 
in the exploration process by selecting for expansion the 
point in the tree that is closest to a(i) in each iteration. 
Using random samples, the probability that a vertex is 
chosen is proportional to the volume of its Voronoi re¬ 
gion. The tree construction is outlined as: 


Algorithm 7.2 Rapidly Exploring Dense Trees 
k : the exploration steps of the algorithm 
1: G.init(^i); 

2: for i = 1 to A: do 
3: G.add_ vertex (a (;)); 

4: q n <- NEAREST(5(G),a(i)); 

5: G.add_ edge(</ n ,«(/)); 

6 : end for 


The tree starts at q\, and in each iteration, an edge and 
vertex are added (Fig. 7.4). 


7.4 Alternative Approaches 

Alternative approaches to the sampling-based paradigm 
include potential-field-based techniques and combina¬ 
torial methods that also produce roadmaps, such as cell 



Fig. 7.4 If there is an obstacle, the edge travels up to the 
obstacle boundary, as far as allowed by the collision detec¬ 
tion algorithm 

So far, the problem of reaching qo has not been 
explained. There are several ways to use RDTs in 
a planning algorithm. One approach is to bias a(i) so 
that </g is frequently chosen (perhaps once every 50 
iterations). A more efficient approach is to develop 
a bidirectional search by growing two trees, one from 
each of <34 and qc,. Roughly half of the time is spent 
expanding each tree in the usual way, while the other 
half is spend attempting to connect the trees. The sim¬ 
plest way to connect trees is to let the newest vertex of 
one tree be a substitute for a(i) in extending the other. 
This tricks one RDT into attempting to connect to the 
other while using the basic expansion algorithm [7.38]. 
Several works have extended, adapted, or applied RDTs 
in various applications [7.37,39-42]. Detailed descrip¬ 
tions can be found in [7.5,7], 

Other Tree Algorithms 

Planners based on the idea of expansive spaces are pre¬ 
sented in [7.43-45]. In this case, the algorithm forces 
exploration by choosing vertices for expansion that 
have fewer points in a neighborhood around them. 
In [7.46], additional performance is obtained by self¬ 
tuning random walks, which focus virtually all of their 
effort on exploration. Other successful tree-based al¬ 
gorithms include the path-directed subdivision tree al¬ 
gorithm [7.47] and some of its variants [7.48]. In the 
literature, it is sometimes hard to locate tree-based plan¬ 
ners for ordinary path planning problems as many of 
them (including RRT) were designed and/or applied 
to more complex problems (Sect. 7.5.4). Their perfor¬ 
mance is nevertheless excellent for a variety of path 
planing problems. 


decompositions. These algorithms are able to elegantly 
and efficiently solve a narrow class of problems, and 
are much preferred over the algorithms of Sect. 7.3 in 
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these cases. Most of the combinatorial algorithms are 
of theoretical interest, whereas sampling-based algo¬ 
rithms are motivated primarily by performance issues 
in challenging applications. Nevertheless, given some 
abstractions, the combinatorial algorithms can be used 
to solve practical problems such as autonomous naviga¬ 
tion of mobile planar robots. 

7.4.1 Combinatorial Roadmaps 

Several algorithms exist for the case in which C = R 2 
and Cobs is polygonal. Most of these cannot be di¬ 
rectly extended to higher dimensions; however, some 
of the general principles remain the same. The maxi¬ 
mum clearance roadmap (or retraction method [7.49]) 
constructs a roadmap that keeps paths as far from 
the obstacles as possible. Paths are contributed to the 
roadmap from the three cases shown in Fig. 7.5, which 
correspond to all ways to pair together polygon fea¬ 
tures. The roadmap can be made naively in time 0(/? 4 ) 
by generating all curves shown in Fig. 7.5 for all pos¬ 
sible pairs, computing their intersections, and tracing 
out the roadmap. Several algorithms exist that provide 
better asymptotic running time [7.50], but they are con¬ 
siderably more difficult to implement. The best-known 
algorithm runs in 0 (nlgn) time in which n is the num¬ 
ber of roadmap curves [7.51]. 

An alternative is to compute a shortest-path 
roadmap [7.52], as shown in Fig. 7.6. This is differ¬ 
ent than the roadmap presented in the previous section 
because paths may actually touch the obstacles, which 
must be allowed for paths to be optimal. The roadmap 
vertices are the reflex vertices of C 0 b s , which are ver¬ 
tices for which the interior angle is greater than n. An 
edge exists in the roadmap if and only if a pair of ver¬ 
tices is mutually visible and the line through them pokes 
into Cfree when extended outward from each vertex 
(such lines are called bitangents). An 0(n 2 lgn)-time 
construction algorithm can be formed by using a radial 
sweep algorithm from each reflex vertex. It can theoret¬ 
ically be computed in time 0 (n 2 + m ), in which m is the 
total number of edges in the roadmap [7.53]. 



Edge-edge Vertex-vertex Vertex-edge 


Fig. 7.5 Voronoi roadmap pieces are generated in one of 
three possible cases. The third case leads to a quadratic 
curve 


Figure 7.7 illustrates the vertical cell decomposition 
approach. The idea is to decompose Cf rcc into cells that 
are trapezoids or triangles. Planning in each cell is triv¬ 
ial because it is convex. A roadmap is made by placing 
a point in the center of each cell and each boundary be¬ 
tween cells. Any graph search algorithm can be used 
to find a collision-free path quickly. The cell decom¬ 
position can be constructed in 0 (n lg n) time using the 
plane-sweep principle [7.54, 55]. Imagine that a vertical 
line sweeps from x = —00 to x = 00 , stopping at places 
where a polygon vertex is encountered. In these cases, 
a cell boundary may be necessary above and/or below 
the vertex. The order in which segments stab the ver¬ 
tical line is maintained in a balanced search tree. This 
enables the determination of the vertical cell boundary 
limits in time 0(lg n). The whole algorithm runs in time 
0(n lgn) because there are O(n) vertices at which the 
sweep line can stop (also, the vertices need to be sorted 
at the outset, which requires time 0 (n lg n)). 



Fig. 7.6 The shortest-path roadmap includes edges be¬ 
tween consecutive reflex vertices on Cobs and also bitan¬ 
gent edges 



Fig. 7.7 The roadmap derived from the vertical cell de¬ 
composition 
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7.4.2 Roadmaps in Higher Dimensions 

It would be convenient if the methods of Sect. 7.4.1 
directly extend into higher dimensions. Although this 
unfortunately does not occur, some of the general ideas 
extend. To consider a cell decomposition in higher di¬ 
mensions, there are two main requirements: (1) each 
cell should be simple enough that motion planning 
within a cell is trivial; (2) the cells should fit together 
nicely. A sufficient condition for the first requirement 
is that cells are convex; more general shapes may be 
allowed; however, the cells should not contain holes 
under any circumstances. For the second requirement, 
a sufficient condition is that the cells can be organized 
into a singular complex. This means that for any two 
^-dimensional cells for d < n, if the boundaries of the 
cells intersect, then the common boundary must itself 
be a complete cell (of lower dimension). 

In two-dimensional polygonal C-spaces, triangula¬ 
tion methods define nice cell decompositions that are 
appropriate for motion planning. Finding good trian¬ 
gulations, which for example means trying to avoid 
thin triangles, is given considerable attention in com¬ 
putational geometry [7.55]. Determining the decom¬ 
position of a polygonal obstacle region with holes 
that uses the smallest number of convex cells is NP- 
hard [7.56]. Therefore, we are willing to tolerate nonop- 
timal decompositions. 

In three-dimensional C-spaces, if C 0 b s is polyhedral, 
then the vertical decomposition method directly extends 
by applying the plane sweep recursively, for example, 
the critical events may occur at each z coordinate, at 
which point changes a 2-D vertical decomposition over 
the x and y coordinates are maintained. The polyhe¬ 
dral case is obtained for a translating polyhedral robot 
among polyhedral obstacles in R 3 ; however, for most 
interesting problems, C 0 b s becomes nonlinear. Suppose 
C = R 2 xS 1 , which corresponds to a robot that can 
translate and rotate in the plane. Suppose the robot and 
obstacles are polygonal. For the case of a line-segment 
robot, an 0(n 5 ) algorithm that is not too difficult to im¬ 
plement is given in [7.57]. The approaches for more 
general models and C-spaces are extremely difficult to 
use in practice; they are mainly of theoretical interest 
and are summarized in Sect. 7.7.3. 

7.4.3 Potential Fields 

A different approach for motion planning is inspired 
from obstacle avoidance techniques [7.58]. It does 
not explicitly construct a roadmap, but instead con¬ 
structs a differentiable real-valued function U : R'" —> 
R, called a potential function, that guides the mo¬ 
tion of the moving object. The potential is typically 


constructed so that it consists of an attractive compo¬ 
nent U & (q), which pulls the robot towards the goal, 
and a repulsive component U t (q), which pushes the 
robot away from the obstacles, as shown in Fig. 7.8. 
The gradient of the potential function is the vec¬ 
tor VU(q) = DU(q) J = [§[(«/),..., §^(q)]\ which 

a) 



Fig.7.8a-c An attractive and a repulsive component define 
a potential function, (a) An attractive potential, (b) a re¬ 
pulsive potential, (c) an attractive and repulsive component 
define a potential function 
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points in the direction that locally maximally in¬ 
creases U. After the definition of U, a path can be 
computed by starting from qi and applying gradient de¬ 
scent: 

1 . q(0) = q\\ i = 0; 

2. while VU{q{i)) 0 do 

3. q(i+ 1) = q(i) + V (/(</(/)) 

4. i = i+] 

However, this gradient-descent approach does not 
guarantee a solution to the problem. Gradient de¬ 
scent can only reach a local minimum of U(q), which 
may not correspond to the goal state </g, as shown 
in Fig. 7.9. 

A planner that makes uses of potential functions 
and attempts to avoid the issue of local minima is 
the randomized potential planner [7.59]. The idea is 
to combine potential functions with random walks by 
employing multiple planning modes. In one mode, 
gradient descent is applied until a local minimum is 
reached. Another mode uses random walks to try to es¬ 
cape local minima. A third mode performs backtracking 
whenever several attempts to escape a local minimum 
have failed. In many ways, this approach can be consid¬ 
ered as a sampling-based planner. It also provides the 
weaker completeness guarantee but it requires param¬ 
eter tuning. Recent sampling-based methods achieve 
better performance by spending more time exploring 
the space, rather than focusing heavily on a potential 
function. 

The gradient of the potential function can be also 
used to define a vector field, which assigns a motion 



Fig.7.9a,b Two examples of the local minimum problem 
with potential functions 


for the robot at any arbitrary configuration q e C. This 
is an important advantage of the approach, beyond its 
computational efficiency, since it does not only com¬ 
pute a single path, but also a feedback control strategy. 
This makes the approach more robust against control 
and sensing errors. Most of the techniques in feedback 
motion planning are based on the idea of navigation 
functions [7.60], which are potential functions properly 
constructed so as to have a single minimum. A function 
4 > ■ Cfree —^ [ 0 , 1 ] is called a navigation function if it: 

• Is smooth (or at least C k for k> 2 ), 

• Has a unique minimum at qo in the connected com¬ 
ponent of the free space that contains qo, 

• Is uniformly maximal on the free-space boundaries, 

• and is Morse, which means that all its critical 
points, such as saddle points, are isolated and can 
be avoided with small random perturbations. 

Navigation functions can be constructed for sphere 
boundary spaces centered at q\ that contain only spher¬ 
ical obstacles, as illustrated in Fig. 7.10. Then they 
can be extended to a large family of C-spaces that are 
diffeomorphic to sphere spaces, such as star-shaped 
spaces, as shown in Fig. 7.10. A more elaborate descrip¬ 
tion of strategies for feedback motion planning will be 
presented in Chap. 47. 
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Putting the issue of local minima aside, another 
major challenge for such potential function based ap¬ 
proaches is constructing and representing the C-space 


7.5 Differential Constraints 

Robot motions must usually conform to both global 
and local constraints. Global constraints on C have 
been considered in the form of obstacles and possi¬ 
bly joint limits. Local constraints are modeled with 
differential equations, and are therefore called differen¬ 
tial constraints. These limit the velocities, and possibly 
accelerations, at every point due to kinematic consid¬ 
erations, such as wheels in contact, and dynamical 
considerations, such as the conservation of angular 
momentum. 

7.5.1 Concepts and Terminology 

Let q denote a velocity vector. Differential constraints 
on C can be expressed either implicitly in the form 
gi(q. q) = 0 or parametrically in the form x =f(q , u). 
The implicit form is more general but often more dif¬ 
ficult to understand and utilize. In the parametric form, 
a vector-valued equation indicates the velocity that is 
obtained for a given q and u, in which u is an input, 
chosen from some input space, U. Let T denote an in¬ 
terval of time, starting at t = 0. 

To model dynamics, the concepts are extended into 
a phase space X of the C-space. Usually each point 
x e X represents both a configuration and velocity, 
x = (q,q). Both implicit and parametric representa¬ 
tions are possible, yielding gi(x,x) = Oandi =f(x,u), 
respectively. The latter is a common control system 
definition. Note that i: = (q, q), which implies that ac¬ 
celeration constraints and full system dynamics can be 
expressed. 

Planning in the state space X could lead to 
a straightforward definition of X D b s by declaring x e 
X obs if and only if q e C 0 bs for x = (q. q). However, an¬ 
other interesting possibility exists which provides some 
intuition about the difficulty of planning with dynam¬ 
ics. This possibility is based on the notion of a region of 
inevitable collision, which is defined as 

X r i c = {x(0) e X | for any It e Tf 00 , 3f > 0 

such that jc(r)GX 0 bs} , (7.4) 

in which x(t) is the state at time t obtained by integrat¬ 
ing the control function It: T —> U from jt(0). The set 
Tioo is a predefined set of all possible control func¬ 
tions. X ric denotes the set of states in which the robot 


in the first place. This issue makes the applications of 
these techniques too complicated for high-dimensional 
problems. 


is either in collision or, because of momentum, it can¬ 
not do anything to avoid collision. It can be considered 
as an invisible obstacle region that grows with speed 
(Fig. 7.11). 

Under the general heading of planning under differ¬ 
ential constraints , there are many important categories 
of problems that have received considerable attention 
in research literature. The term nonholonomic plan¬ 
ning was introduced for wheeled mobile robots [7.61]. 
A simple example is that a car cannot move sideways, 
thereby making parallel parking more difficult. In gen¬ 
eral, a nonholonomic constraint is a differential equality 
constraint that cannot be integrated into a constraint 
that involves no derivatives. Typically, nonholonomic 
constraints that appear in robotics are kinematic, and 
arise from wheels in contact [7.62]. Nonholonomic con¬ 
straints may also arise from dynamics. 

If a planning problem involves constraints on at 
least velocity and acceleration, the problem is often re¬ 
ferred to as kinodynamic planning [7.63]. Usually, the 
model expresses a. fully actuated system, which means 
that it can be expressed as q = h(q. q, u), in which U 
contains an open set that includes the origin of R" 
(here, n is the dimension of both U and C). It is pos¬ 
sible for a problem to be nonholonomic, kinodynamic, 
both, or neither; however, in recent times, the terms are 
not used with much precision. 



Fig. 7.11 The region of inevitable collision grows quadrat- 
ically with the speed 
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a) 




Two stages 

Fig. 7.12 (a) A reachability tree for the Dubins car with three ac¬ 
tions. (b) A 2-stage tree is shown. The k-\h stage produces 3 k new 
vertices 


Trajectory planning is another important term, 
which has referred mainly to the problem of deter¬ 
mining both a path and velocity function for a robot 
arm (e.g., PUMA 560). In the treatment below, all of 
these will be referred to as planning under differential 
constraints. 

7.5.2 Discretization of Constraints 

The only known methods for complete and optimal 
planning under differential constraints in the presence 
of obstacles are for the double integrator system with 
X = R [7.64] and X = R 2 [7.65]. To develop planning 
algorithms in this context, several discretizations are 
often needed. For ordinary motion planning, only C 
needed to be discretized; with differential constraints, 
T and possibly also U require discretization, in addition 
toC(orX). 

Discretization of the differential constraints is one 
of the most important issues. To solve challenging plan¬ 
ning problems efficiently, it is often necessary to de¬ 
fine motion primitives for the particular dynamical sys¬ 
tem [7.40, 66, 67], One of the simplest ways to discretize 
the differential constraints is to construct a discrete-time 
model , which is characterized by three aspects: 

1. The time interval T is partitioned into intervals of 
length At. This enables stages to be assigned, in 
which stage k indicates that (k— l)Af time has 
elapsed. 

2. A finite subset I/d of the action space U is cho¬ 
sen. If U is already finite, then this selection may 
be U d = U. 

3. The action u (t ) must remain constant over each time 
interval. 

From an initial state, x, a reachability tree can be 
formed by applying all sequences of discretized actions. 
Figure 7.12 shows the path of this tree for the Dubins 
car, which is a kinematic model of a car that drives in 
the plane at unit speed and cannot move in reverse. The 
edges of the tree are circular arcs and line segments. 
For general systems, each trajectory segment in the tree 
is determined by numerical integration of x =f(x,u ) 
for a given u. In general, this can be viewed as an in¬ 
cremental simulator that takes an input and produces 
a trajectoiy segment according to x =j(x. u). 

7.5.3 Decoupled Approach 

A popular paradigm for trajectory planning and other 
problems that involve dynamics is to decouple the 
problem into first planning a path and then computing 
a timing function along the path by performing a search 
in the space spanned by (s, i), in which s is the path 



s 


Fig. 7.13 An illustration of the bang-bang approach to 
computing a time-optimal trajectory. The solution trajec¬ 
tory is obtained by connecting the dots 

parameter and s is its first derivative. This leads to a di¬ 
agram such as the one shown in Fig. 7.13, in which 
the upper region ,S' 0 h, must be avoided because the cor¬ 
responding motion of the mechanical system violates 
the differential constraints. Most methods are based on 
early work in [7.68,69], and determine a bang-bang 
-control, which means that they switch between ac¬ 
celerating and decelerating at full speed. This applies 
to determining time-optimal trajectories (optimal once 
constrained to the path). Dynamic programming can be 
used for more general problems [7.70]. 

For some problems and nonholonomic systems, 
steering methods have been developed to solve the 
two-point boundary value problem efficiently [7.62, 
71]. This means that, for any pair of states, a trajectory 
that ignores obstacles but satisfies the differential con¬ 
straints can be obtained. Moreover, for some systems. 
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Fig. 7.14 Reachability graph from the origin, shown after three 
stages (the true edges are actually parabolic arcs when accelera¬ 
tion or deceleration occurs). Note that a lattice is obtained, but the 
distance traveled in one stage increases as |^| increases 

the complete set of optimal trajectories has been charac¬ 
terized [7.72,73]. These control-based approaches en¬ 
able straightforward adaptation of the sampling-based 
roadmap approach [7.74,75]. One decoupled approach 
is to first plan a path that ignores differential constraints, 
and then incrementally transform it into one that obeys 
the constraints [7.62, 76]. 

7.5.4 Kinodynamic Planning 

Due to the great difficulty of planning under differential 
constraints, many successful planning algorithms that 
address kinodynamic problems directly in the phase 
space X are sampling based. 

Sampling-based planning algorithms proceed by 
exploring one or more reachability trees. Many paral¬ 
lels can be drawn with searching on a grid; however, 
reachability trees are more complicated because they do 
not necessarily involve a regular lattice structure. The 
vertex set of reachability trees is dense in most cases. 


It is therefore not clear how to search a bounded re¬ 
gion exhaustively at a fixed resolution. It is also difficult 
to design approaches that behave like a multiresolution 
grid, in which refinements can be made arbitrarily to 
ensure resolution completeness. 

Many algorithms attempt to convert the reachabil¬ 
ity tree into a lattice. This is the basis of the origi¬ 
nal kinodynamic planning work [7.63], in which the 
discrete-time approximation to the double integrator, 
q = u, is forced onto a lattice as shown in Fig. 7.14. 
This enables an approximation algorithm to be devel¬ 
oped that solves the kinodynamic planning problem in 
time polynomial in the approximation quality 1/e and 
the number of primitives that define the obstacles. Gen¬ 
eralizations of the methods to fully actuated systems are 
described in [7.7], Surprisingly, it is even possible to 
obtain a lattice for some underactuated, nonholonomic 
systems [7.77], 

If the reachability tree does not form a lattice, then 
one approach is to force it to behave as a lattice by 
imposing a regular cell decomposition over X (or C), 
and allowing no more than one vertex per cell to be ex¬ 
panded in the reachability graph (Fig. 7.15). This idea 
was introduced in [7.78]. In their version of this ap¬ 
proach, the reachability graph is expanded by dynamic 
programming. Each cell is initially marked as being in 
collision or being collision free, but not yet visited. As 
cells are visited during the search, they become marked 
as such. If a potential new vertex lands in a visited cell, 
it is not saved. This has the effect of pruning the reach¬ 
ability tree. 

Other related approaches do not try to force the 
reachability tree onto a lattice. RRTs were designed 
to expand the tree in a way that is biased toward 
covering as much new territory as possible in each it¬ 
eration [7.79]. Planners that are based on the concept 
of expansive trees attempt to control the density of ver- 




Fig. 7.15 (a) The first four stages of 
a dense reachability graph for the 
Dubins car. (b) One possible search 
graph, obtained by allowing at most 
one vertex per cell. Many branches 
are pruned away. In this simple exam¬ 
ple there are no cell divisions along 
the 0-axis 
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tices in the tree by analyzing neighborhoods [7.44]. The 
path-directed subdivision tree planner expands a tree, 
while building an adaptive subdivision of the state 
space, so as to avoid resampling the same regions of the 
space [7.47, 80]. Such approaches can be biased to ac¬ 


7.6 Extensions and Variations 

A brief overview of other important extensions to the 
basic motion planning problem are presented in this 
section. 

7.6.1 Closed Kinematic Chains 

In many cases, the robot may be consist of links that 
form closed loops. This arises in many important appli¬ 
cations, for example, if two arms grasp an object then 
a loop is formed and a humanoid robot forms a loop if 
both legs touch the ground. For parallel robots, loops 
are intentionally designed into the robot [7.81]; a clas¬ 
sic example is the Stewart-Gough platform. To model 
closed-chain problems, the loops are broken so that 
a kinematic tree of links is obtained. The main compli¬ 
cation is that constraints on C of the form h(q) = 0 are 
introduced, which require that the loops are maintained. 
This causes great trouble for most planning algorithms 
because without loops a parameterization of C was 
available. The closure constraints restrict the planning 
to a lower-dimensional subset of C for which no param¬ 
eterization is given. Computing a parameterization is 
generally difficult or impossible [7.82], although there 
has been recent progress for some special cases [7.83]. 

Sampling-based approaches can generally be adap¬ 
ted to handle closed chains. The main difficulty is that 
the samples u(i) over C are unlikely to be configu¬ 
rations that satisfy closure. In [7.84], both RRTs and 
PRMs were adapted to closed chains. RRTs performed 
much better because a costly optimization was required 
in the PRM to move samples onto the closure sub¬ 
space; RRTs on the other hand do not require samples 
to lie in this subspace. By decomposing chains into ac¬ 
tive and passive links, followed by inverse kinematics 
computations, performance was dramatically improved 
for PRMs in [7.85]. This idea was further improved by 
the introduction of the random loop generator (RLG). 
Based on this, some of the most challenging closed- 
chain planning problems ever solved appear in [7.86]. 

7.6.2 Manipulation Planning 

In most forms of motion planning, the robot is not 
allowed to touch obstacles. Suppose instead that it is ex¬ 


celerate the expansion of the tree towards a goal, while 
still providing the weaker probabilistic completeness 
guarantee [7.48]. l<s>M3EI$EB provides an example of 
the use of a tree-based planner together with a physics- 
engine which accounts for the constraints. 


pected to interact with its environment by manipulating 
objects. The goal may be to bring an object from one 
place to another, or to rearrange a collection of objects. 
This leads to a kind of hybrid motion planning prob¬ 
lem, which mixes discrete and continuous spaces. There 
are discrete modes that correspond to whether the robot 
is carrying a part [7.87]. In the transit mode, the robot 
moves toward a part. In the transfer mode, it carries the 
part. Transitions between modes require meeting spe¬ 
cific grasping and stability requirement. One important 
variant of manipulation planning is assembly planning, 
in which the goal is to fit a collection of pieces together 
to make an assembled product [7.88]. Most motion 
planning work makes limiting assumptions on the kinds 
of interaction that can occur between the robot and the 
objects. For richer models of manipulation, see [7.89]. 

7.6.3 Time-Varying Problems 

Suppose that the workspace contains moving obsta¬ 
cles whose trajectories are specified as a function of 
time. Let Tel denote the time interval, which may 
be bounded or unbounded. A state X is defined as 
X = C x T, in which C is the usual C-space of the robot. 
The obstacle region in X is characterized as 

*obs = {( 9 , t)ex | Mq) n O(t) ± 0} , (7.5) 

in which (9(f) is a time-varying obstacle. Many plan¬ 
ning algorithms can be adapted to X, which has only 
one more dimension than C. The main complication is 
that time must always increase along a path through X. 

For the easiest version of the problem, there is no 
bound on the robot speed. In this case, virtually any 
sampling-based algorithm can be adapted. Incremental 
searching and sampling methods apply with little mod¬ 
ification, except that paths are directed so that forward 
time progress is made. Using bidirectional approaches 
is more difficult for time-varying problems because the 
goal is usually not a single point due to the time de¬ 
pendency. Sampling-based roadmaps can be adapted; 
however, a directed roadmap is needed, in which every 
edge must be directed to yield a time-monotonic path. 


Part A | 7.6 





Part A | 7.6 


152 


Part A 


Robotics Foundations 


If the motion model is algebraic (i. e., expressed 
with polynomials) then A obs is semi-algebraic. This en¬ 
ables cylindrical algebraic decomposition to apply. If 
X obs is polyhedral, as depicted in Fig. 7.16, then verti¬ 
cal decomposition can be used. It is best to first sweep 
the plane along the T -axis, stopping at the critical times 
when the linear motion changes. 

There has been no consideration so far of the speed 
at which the robot must move to avoid obstacles. It is 
obviously impractical in many applications if the solu¬ 
tion requires the robot to move arbitrarily fast. One step 
towards making a realistic model is to enforce a bound 
on the speed of the robot. Unfortunately, the problem 
is considerably more difficult. Even for piecewise-lin- 
ear motions of obstacles in the plane, the problem has 
been established to be PSPACE-hard [7.90]. A com¬ 
plete algorithm based on the shortest-path roadmap is 
presented in [7.91]. 

An alternative to defining the problem in C x T is 
to decouple it into a path planning part and a motion 
timing part. A collision-free path in the absence of ob¬ 
stacles is first computed. A search in a 2-D space is then 
performed to determine the toning function (or time 
scaling) for the path. 


Substantial attention has been devoted to the prob¬ 
lem of planning for multiple robots (|43>M1M5EB and 
losBEEHiEB). Suppose there are m robots. A state 
space is defined that considers the configurations of all 
robots simultaneously, 

X= C 1 xC 2 x---xC m . (7.6) 

A state x e X specifies all robot configurations, and may 
be expressed as x = (q l ,q 2 ,... ,q'"). The dimension 
of X is N, which is N = dim(C'). 

There are two sources of obstacle regions in 
the state space: (1) robot-obstacle collisions, and 
(2) robot-robot collisions. For each i such that 1 < i < 
m, the subset of X that corresponds to robot TV in col¬ 
lision with the obstacle region (9 is 

A' bs = {* e X | JA\q‘) nO/0). (7.7) 

This models the robot-obstacle collisions. 

For each pair, TV and TV, of robots, the subset of X 
that corresponds to TV in collision with TV is 

= {x ex\ JA'(q l ) n JV(q') / 0}. (7.8) 


7.6.4 Multiple Robots 

A simple extension to the basic motion planning prob¬ 
lem can be made to handle multibody robots by includ¬ 
ing robot self-intersections; however, it is important to 
specify the pairs of bodies for which collision is unac¬ 
ceptable, for example, consecutive links in a robot arm 
are allowed to touch. 



Fig. 7.16 A time-varying example with linear obstacle motion 


Both (7.7) and (7.8) will be combined in (7.9) to 
yield A obs . The obstacle region in X is 

Xobs=(Q X obs) U f U 4b. )• < 7 - 9 ’ 

Once these definitions have been made, any general- 
purpose planning algorithm can be applied because X 
and X obs appear no different from C and C 0 bs> except 
that the dimension N may be very high. Approaches 
that plan directly in X are called centralized. The high 
dimensionality of X motivates the development of de¬ 
coupled approaches that handle some aspects of the 
planning independently for each robot. Decoupled ap¬ 
proaches are usually more efficient, but this usually 
comes at the expense of sacrificing completeness. An 
early decoupled approach is prioritized planning [7.92, 
93], in which a path and timing function is computed 
for the r-th robot while treating the first i — 1 robots as 
moving obstacles as they follow their paths. Another 
decoupled approach is fixed-path coordination [7.94], 
in which the paths are planned independently for each 
robot, and then their timing functions are determined 
by computing a collision-free path through an m- 
dimensional coordination space. Each axis in this space 
corresponds to the domain of the path of one robot. 
Fig. 7.17 shows an example. The idea has been gen¬ 
eralized to coordination on roadmaps [7.95,96]. 
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7.6.5 Uncertainty in Predictability 

If the execution of the plan is not predictable, then feed¬ 
back is needed. The uncertainty may be modeled either 
implicitly, which means that the plan is able to respond 
to unexpected future configurations, or explicitly, which 
means that the uncertainty is precisely characterized 
and analyzed in the development of a plan. Potential- 
function-based approaches are one way of achieving 
feedback motion planning. 

A plan can be represented as a vector field 
over Cfree, in which each vector indicates the required 
velocity. The integral curves of the field should flow 
into the goal without leaving C 0 bs- If dynamics are 
a concern, then the vector field can be tracked by an ac¬ 
celeration-based control model 

u = K(f(q) — q) + V(< 7 ) . (7.10) 



Fig. 7.17 The obstacles that arise from coordinating m robots are 
always cylindrical. The set of all \m(m— 1) axis-aligned 2-D pro¬ 
jections completely characterizes X 0 b s 


in which K is a scalar gain constant. Alternatively, 
a vector field may be designed directly on the phase 
space, X\ however, there are not methods to compute 
such fields efficiently under general conditions. This 
can also be considered as a feedback control problem 
with implicit, nonlinear constraints on X. 

If the uncertainty is modeled explicitly, then a game 
against nature is obtained, in which the uncertainty 
is caused by a special decision maker called nature. 
The decisions of nature can either be modeled non- 
deterministically, which means that a set of possible 
actions is specified, or probabilistically, which means 
that a probability distribution or density is specified 
over the nature actions. Under nondeterministic un¬ 
certainty, worst-case analysis is usually performed to 
select a plan; under probabilistic uncertainty, expected- 
case analysis is usually performed. Numerous ap¬ 
proaches exist for such problems, including value itera¬ 
tion, Dijkstra-like algorithms, and reinforcement learn¬ 
ing algorithms [7.7]. 

7.6.6 Sensing Uncertainty 

Consider solving tasks such as localization, map build¬ 
ing, manipulation, target tracking, and pursuit-evasion 
(hide-and-seek) with limited sensing. If the current con¬ 
figuration or state is not known during execution, then 
the problem may appear quite different. Information is 
obtained from sensors, and the planning problem nat¬ 
urally lives in an information space or I -space [7.7, 
Chap. 11]. The state may include the configuration, 
velocities, or even the map of the environment (e.g., 
obstacles). The most basic I-space is the set of all 
histories that can be obtained during execution, based 
on all sensing observations, actions previously applied, 


and the initial conditions. The goal in developing ef¬ 
ficient algorithms in this context is to determine in¬ 
formation mappings that reduce the I-space size or 
complexity so that plans that can be computed that 
use information feedback. The traditional way to use 
the information state is for estimating the state. This 
is sufficient for solving many tasks, but it is often not 
necessary. It may be possible to design and execute suc¬ 
cessful plans without ever knowing the current state. 
This can lead to more robust robot systems which may 
also be cheaper to manufacture due to weaker sensing 
requirements. 

Two important families of I-spaces are nondeter¬ 
ministic and probabilistic. A nondeterministic informa¬ 
tion state (I-state) is a set of states that are possible 
given the available history of sensor observations and 
actions applied during execution. The nondeterministic 
I-space is the set of all possibilities. Similarly, a proba¬ 
bilistic I-state is a probability density function over the 
state space, conditioned on the available history. The 
probabilistic I-space is often called the belief space, 
which represents the set of all probability density func¬ 
tions. Both filtering and planning over these spaces 
remains a topic of active research. One of most use¬ 
ful and classical results is the Kalman filter [7.97], 
for which the belief space reduces to Gaussians, al¬ 
lowing it to be completely parametrized by mean and 
covariance of state. Many approaches to reasoning 
in these I-spaces attempt to reduce its complexity, 
through combinatorial reasoning in the case of nonde¬ 
terministic I-spaces [7.98] and through approximations. 
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sampling, and dimensionality reduction techniques in 
belief spaces [7.99-103], For example, a sampling- 
based roadmap can be constructed directly in the belief 
space [7.104], 

7.6.7 Optimal Planning 

In most formulations of planning, computing an opti¬ 
mal solution is given little or no importance. Several 
factors have contributed to this trend. One of the most 
fundamental is that a natural criterion of optimality 
often does not exist. Unlike control theory, where op¬ 
timality is a central goal, the main task in planning is 
to avoid obstacles. Imagine walking quickly through 
a furniture store. Moving along the shortest possible 
path would cause you to touch the corners of obsta¬ 
cles, which might not be desirable. You could try to 
maximize clearance, but this causes longer paths, which 
again might not be desirable. Another factor is that so¬ 
lutions produced by path planning algorithms tend to 
not be excessively long, especially after some quick 
post-processing in the case of sampling-based planners. 
Finally, the computational complexity of the optimal 
planning problem is typically worse than its feasible 
(not necessarily optimal) counterpart. One of the most 
notable exceptions is planning in field robotics (outdoor 
vehicles in unstructured terrain), for which cost func- 

7.7 Advanced Issues 

We cover here a series of more advanced issues, such 
as topics from topology and sampling theory, and how 
they influence the performance of motion planners. The 
last section is devoted to computational algebraic geom¬ 
etry techniques that achieve completeness in the general 
case. Rather than being a practical alternative, these 
techniques serve as an upper bound on the best asymp¬ 
totic running time that could be obtained. 

7.7.1 Topology of Configuration Spaces 

Manifolds 

One reason that the topology of a C-space is important 
is because it affects its representation. Another reason is 
that, if a path-planning algorithm can solve problems in 
a topological space, then that algorithm may carry over 
to topologically equivalent spaces. 

The following definitions are important in order to 
describe the topology of C-space. A map 0 : S —> T is 
called a homeomorphism if 0 is a bijection and both 0 
and 0 1 are continuous. When such a map exists, S 
and T are said to be homeomorphic. A set S is an 
77 -dimensional manifold if it is locally homeomorphic 


tion determines navigability at each point, leading to the 
well-known family of D* algorithms [7.105,106]. 

In spite of these issues, several useful approaches 
and interesting ideas have emerged. As mentioned in 
Sect. 7.4.1, the shortest-path roadmap is an effective 
multiple-query approach in the case of a 2-D, polyg¬ 
onal C-space. Alternatively, the continuous Dijkstra 
method provides an effective single-query approach 
by propagating wavefronts that correspond to level 
sets of the optimal cost from the initial configuration 
[7.107,108]. The wavefronts are propagated combina- 
torially, stopping only at critical events, leading to an 
exact, optimal solution. In the case of 3-D polyhedral 
C-spaces, the shortest path problem already becomes 
PSPACE-hard [7.15]. However, algorithms that produce 
approximately optimal solutions exist [7.109-11 1], and 
are useful in C-spaces of several dimensions. Dijkstra- 
like approaches can be adapted to include various forms 
of uncertainty and differential constraints, and are all 
derived in some way from value iteration methods 
introduced by Bellman in the 1950s. See Chaps. 7, 
10, and 14 of [7.7] for more discussion. Pushing into 
even higher dimensions, recent sampling-based plan¬ 
ning methods have produced asymptotically optimal 
versions RRTs [7.112] and PRMs [7.112,113], which 
have been shown to produce paths that improve in qual¬ 
ity as time progresses. 


to R' ! , meaning that each point in S possesses a neigh¬ 
borhood that is homeomorphic to R". For more details, 
see [7.114,115]. 

In the vast majority of motion planning problems, 
the configuration space is a manifold. An example 
of a C-space that is not a manifold is the closed 
unit square: [0, 1] x [0, 1] C R 2 , which is a manifold 
with boundary obtained by pasting the one-dimensional 
boundary on the two-dimensional open set (0,1) x 
(0. 1). When a C-space is a manifold, then we can 
represent it with just n parameters, in which n is the 
dimension of the configuration space. Although an 77- 
dimensional manifold can be represented using as few 
as 77 parameters, due to constraints it might be eas¬ 
ier to use a representation that has higher number of 
parameters, e.g., the unit circle S 1 can be represented 
as S 1 = {(x,y)\}r + y 1 = 1} by embedding S 1 in R 2 . 
Similarly, the torus T 2 can be embedded in R 3 . 

Representation 

Embeddings into higher-dimensional spaces can facili¬ 
tate many C-space operations. For example, the orien¬ 
tation of a rigid body in space can be represented by 
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a n x n matrix of real numbers. The n 2 matrix entries 
must satisfy a number of smooth equality constraints, 

making the manifold of such matrices a submanifold 

2 

of R m . One advantage is that these matrices can be 
multiplied to get another matrix in the manifold. For ex¬ 
ample, the orientation of a rigid-body in /7-dimensional 
space (7? = 2 or 3) is described by the set SO(n ), the set 
of all 77 x // rotation matrices. The position and orienta¬ 
tion of a rigid body is represented by the set SE(i 1 ), the 
set of all 77 x 77 homogeneous transformation matrices. 
These matrix groups can be used to (1) represent rigid- 
body configurations, (2) change the reference frame for 
the representation of a configuration, and (3) displace 
a configuration. 

There are numerous parameterizations of .S’0(3) 
[7.116] but unit quaternions correctly preserve the C- 
space topology as S 1 represents 2-D rotations. Quater¬ 
nions were introduced in Chap. 2. There is, however, 
a two-to-one correspondence between unit quaternions 
and 3-D rotation matrices. This causes a topological is¬ 
sue that is similar to the equivalence of 0 and lie for 
2-D rotations. One way to account for this is to declare 
antipodal (opposite) points on S 3 to be equivalent. In 
planning, only the upper hemisphere of S 3 is needed, 
and paths that cross the equator instantly reappear on 
the opposite side of S 3 , heading back into the northern 
hemisphere. In topology, this is called a real projective 
space: RP 3 . Hence, the C-space of a 3-D body capable 
only of rotation is RP 3 . If both translation and rotation 
are allowed, then SE( 3), the set of all 4 x 4 homogeneous 
transformation matrices, yields 

C = R 3 x RP 3 , (7.11) 

which is six dimensional. A configuration q e C can 
be expressed using quaternions with seven coordinates, 
(x , y, z, a, b, c, d), in which a 2 + b 2 + c 2 + d 2 = 1. More 
examples can be found in Table 7.1. 

7.7.2 Sampling Theory 

Since the most successful paradigm for motion plan¬ 
ning today is the sampling-based framework, presented 


Table 7.1 Some common robots and their C-spaces 


Type of robot 

C-space 

representation 

Mobile robot translating in the plane 

R 2 

Mobile robot translating and rotating 
in the plane 

SE( 2) or R 2 x 5 1 

Rigid body translating in the three-space 

R 3 

A spacecraft 

SE( 3) or R 3 x 50(3) 

An 72 -joint revolute arm 

jn 

A planar mobile robot with an attached 
72 -joint arm 

SE(2) x T" 


in Sect. 7.3, sampling theory becomes relevant to the 
motion planning problem. 

Metrics in Configuration/State Spaces 
Virtually all sampling-based methods require some no¬ 
tion of distance on C. For example, the sampling-based 
roadmap method selects candidate vertices to connect 
a new configuration given a distance-defined neigh¬ 
borhood. Similarly, the rapidly exploring dense trees 
expands the tree from the nearest node of the tree 
to a newly sampled configuration. Usually, a metric, 
p : C x C —R, is defined, which satisfies the standard 
axioms: nonnegativity, reflexivity, symmetry, and the 
triangle inequality. 

Two difficult issues that arise in constructing a met¬ 
ric are: (1) the topology of C must be respected, and 
(2) several different quantities, such as linear and angu¬ 
lar displacements, must be compared in some way. To 
illustrate the second issue, consider defining a metric p z 
for a space constructed as Z = X x Y as 

p z (z,z) = p z (x,y,x',y') 

= C\p x {x,x’) + c 2 Py(y, /). (7.12) 

Above, Ci and C2 are arbitrary positive constants that 
indicate the relative weights of the two components. 
For a 2-D rotation, expressed as «,• = cos 0, and 
b, = sin 9j, a useful metric is 

p(ai,b u a 2 ,b 2 ) = cos~ 1 {a l a 2 + bib 2 ) . (7.13) 

The 3-D equivalent is obtained by defining 

p 0 (hi,h 2 ) = cos -1 (aia 2 + bib 2 + CiC 2 + did 2 ) , 

(7.14) 

in which each h[ = ( at,bi,Ci,di ) is a unit quaternion. 
The metric is defined as p(h\,h 2 ) = mm(po(h\,li 2 ), 
Po(h 1, — h 2 )), by respecting the required identification 
of antipodal points. This computes the shortest distance 
in R 4 , for a path constrained to the unit sphere. 

In some algorithms, defining volume on C may 
also be important. In general, this leads to a measure 
space, for which the volume function (called the mea¬ 
sure) must satisfy axioms that resemble the probability 
axioms, but without normalization. For transformation 
groups, one must be careful to define volumes in a way 
that is invariant with respect to transformations. Such 
volumes are called Haar measures. Defining volumes 
via balls using the metric definitions (7.13) and (7.14) 
actually satisfy this concern. 

Probabilistic Versus Deterministic Sampling 
The C-space may be sampled probabilistically or de¬ 
terministically. Either way, the requirement is usu¬ 
ally that a dense sequence a of samples is obtained. 
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This means that, in the limit as the number of sam¬ 
ples tends to infinity, the samples become arbitrarily 
close to every point in C. For probabilistic sampling, 
this denseness (with probability one) ensures prob¬ 
abilistic completeness of a planning algorithm. For 
deterministic sampling, it ensures resolution complete¬ 
ness, which means that, if a solution exists, the algo¬ 
rithm is guaranteed to find it; otherwise, it may run 
forever. 

For probabilistic sampling, samples are selected 
randomly over C, using a uniform probability den¬ 
sity function. To obtain uniformity in a meaning¬ 
ful way, the Haar measure should be used. This 
is straightforward in many cases; 50(3) however is 
tricky. A uniform (with respect to Flaar measure) 
random quaternion is selected as follows. Choose 
three points u \, 112 , M 3 e [0,1] uniformly at random, and 
let [7.117] 

h = ( y/1 — iii sin27tU2, ^/l — «i cos 2 ttu 2 , 

^/M7sin2jTM3, ^/lll COS lltU^) . 

(7.15) 

Even though random samples are uniform in some 
sense, they are also required to have some irregularity 
to satisfy statistical tests. This has motivated the de¬ 
velopment of deterministic sampling schemes that offer 
better performance [7.118]. Instead of being concerned 
with randomness, deterministic sampling techniques 
are designed to optimize criteria, such as discrepancy 
and dispersion. Discrepancy penalizes regularity in the 
sample, which frequently causes trouble in numerical 
integration. Dispersion gives the radius of the largest 
empty (not containing samples) ball. Thus, driving dis¬ 
persion down quickly means that the whole space is 
explored quickly. Deterministic samples may be ir¬ 
regular neighborhood structure (appearing much like 
random samples), or regular neighborhood structure, 
which means that points are arranged along a grid or 
lattice. For more details in the context of motion plan¬ 
ning, see [7.7], 

7.7.3 Computational Algebraic 
Geometry Techniques 

Sampling-based algorithms provide good practical per¬ 
formance at the expense of achieving only a weaker 
form of completeness. On the other hand, complete al¬ 
gorithms, which are the focus of this section, are able to 
deduce that there is no solution to a planning problem. 

Complete algorithms are able to solve virtually any 
motion planning problem as long as C 0 b s is represented 
by patches of algebraic surfaces. Formally, the model 
must be semi-algebraic, which means that it is formed 


from unions and intersections of roots of multivariate 
polynomials in q, and for computability, the polyno¬ 
mials must have rational coefficients (otherwise roots 
may not have finite representations). The set of all roots 
to polynomials with rational coefficients is called real 
algebraic numbers and has many nice computational 
properties. See [7.12,119-121] for more information 
on the exact representation and calculation with real al¬ 
gebraic numbers. For a gentle introduction to algebraic 
geometry, see [7.82], 

To use techniques based on algebraic geometry, the 
first step is to convert the models into the required 
polynomials. Suppose that the models, the robot, TA, 
and the obstacles O are semi-algebraic (this includes 
polyhedral models). For any number of attached 2-D 
or 3-D bodies, the kinematic transformations can be 
expressed using polynomials. Since polynomial trans¬ 
formations of polynomials yield polynomials, the trans¬ 
formed robot model is polynomial. The algebraic sur¬ 
faces that comprise C 0 bs are computed by carefully 
considering all contact types, which characterize all 
ways to pair a robot feature (faces, edges, vertices) 
with an obstacle feature [7.6,7,9,122]. This step al¬ 
ready produces too many model primitives to be useful 
in most applications. 

Once the semi-algebraic representation has been ob¬ 
tained, powerful techniques from algebraic geometry 
can be exploited. One of the most widely known al¬ 
gorithms, cylindrical algebraic decomposition [7.119, 
123,124], provides the information needed to solve the 
motion planning problem. It was originally designed 
to determine whether Tarski sentences, which involve 
quantifiers and polynomials, are satisfiable, and to find 
an equivalent expression that does not involve quanti¬ 
fiers. The decomposition produces a finite set of cells 
over which the signs of the polynomials remain fixed. 
This enables a systematic approach to satisfiability and 
quantifier elimination. It was recognized by Schwartz 
and Sharir [7.121] that it also solves motion planning. 

The method is conceptually simple, but there are 
many difficult technical details. The decomposition is 
called cylindrical because the cells are organized into 
vertical columns of cells, see Fig. 7.18 for a 2-D ex¬ 
ample. There are two kinds of critical events, shown 
in Fig. 7.19. At critical points, rays are extended indef¬ 
initely in both vertical directions. The decomposition 
differs from the vertical decomposition in Fig. 7.7 
because there the rays were only extended until the 
next obstacle was hit. Here, columns of cells are 
obtained. 

In n dimensions, each column represents a chain of 
cells. The first and last cells are /7-dimensional and un¬ 
bounded. The remaining cells are bounded and alternate 
between being (n — 1)-dimensional and /7-dimensional. 
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The bounded n-dimensional cells are bounded above 
and below by the roots of single multivariate polynomi¬ 
als. This makes it simple to describe the cells and their 
connectivity. To compute this cell decomposition, the 
algorithm constructs a cascading chain of projections. 
In the first step, C 0 bs is projected from R" to R"~'. 
This is followed by a projection into R" —2 . This re¬ 
peats until M is obtained with a univariate polynomial 
that encodes the places at which all critical boundaries 
need to be placed. In a second phase of the algorithm, 
a series of liftings is performed. Each lifting takes the 
polynomials and cell decomposition over M' and lifts 
them via columns of cells to R' - * -1 . A single lifting is 
illustrated in Fig. 7.18b. The running time of the full 
algorithm depends on the particular methods used to 
perform the algebraic computations. The total running 
time required to use cylindrical algebraic decomposi¬ 
tion for motion planning is bounded by (m<i) 0<l , in 
which m is the number of polynomials to describe C 0 bs 
(a huge number), and d is the maximum algebraic de¬ 
gree. (It may seem odd for O(-) to appear in the middle 
of an expression. In this context, it means that there 
exists some c e [0, 00 ) such that the running time is 
bounded by (md) c . Note that another O is not neces¬ 
sary in the front of the whole formula.) The main point 
to remember is that the algorithm is doubly exponen¬ 
tial in the dimension of C (even the number of cells is 
doubly exponential). 

Although performing the cylindrical decomposition 
is sufficient for solving motion planning, it computes 
more information than is necessary. This motivates 
Canny ’s roadmap algorithm [7.12], which produces 
a roadmap directly from the semi-algebraic set, rather 
than constructing a cell decomposition along the way. 
Since there are doubly exponentially many cells in the 
cylindrical algebraic decomposition, avoiding this con¬ 
struction pays off. The resulting roadmap method of 
Canny solves the motion planning problem in time that 
is again polynomial in the number of polynomials and 
polynomial in the algebraic degree, but is only singly 
exponential in dimension [7.12]. 

The basic idea is to find silhouette curves in R 2 
of Cobs in R" • The method finds zero-dimensional crit¬ 
ical points and one-dimensional critical curves. The 
critical curves become roadmap edges, and the criti- 



Fig. 7.18 (a) A face modeled with four algebraic primitives, and 
(b) a cylindrical algebraic decomposition of the face 



Fig. 7.19 Critical points occur either when the surface 
folds over in the vertical direction or when surfaces 
intersect 


cal points are places at which the algorithm recursively 
finds silhouettes of (/? — 1)-dimensional slices of C 0 b s - 
These contribute more critical points and curves. The 
curves are added to the roadmap, and the algorithm 
recurses again on the critical points. The recursive it¬ 
erations terminate at n = 2. Canny showed that the 
resulting union of critical curves preserves the connec¬ 
tivity of C 0 bs (and hence, Cf ree ). Some of the technical 
issues are: the algorithm works with a stratification 
of C 0 bs into manifolds; there are strong general position 
assumptions that are hard to meet; paths are actually 
considered along the boundary of Cf ree ; and the method 
does not produce a parameterized solution path. For 
improvements to Canny’s algorithm and many other im¬ 
portant details, see [7.119]. 


7.8 Conclusions and Further Reading 

The brief survey given here hardly does justice to mo¬ 
tion planning, which is a rich and active research field. 
For more details, we recommend consulting two recent 
textbooks [7.5,7]. In addition, see the classic textbook 


of Latombe [7.6], the classic papers in [7.4], and the re¬ 
cent surveys in [7.2, 3]. Furthermore, consult the related 
handbook chapters that were indicated throughout this 
chapter. 
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Video-References 



Powder transfer task using demonstration-guided motion planning 
available from http://handbookofrobotics.org/view-chapter/07/videodetails/17 
Simulation of a large crowd 

available from http://handbookofrobotics.org/view-chapter/07/videodetails/21 
Motion planning in multi-robot scenario 

available from http://handbookofrobotics.org/view-chapter/07/videodetails/22 
Alpha puzzle 

available from http://handbookofrobotics.org/view-chapter/07/videodetails/23 
Kinodynamic motion planning for a car-like robot 

available from http://handbookofrobotics.org/view-chapter/07/videodetails/24 
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8. Motion Control 


Wan Kyun Chung, Li-Chen Fu, Torsten Kroger 


This chapter will focus on the motion control of 
robotic rigid manipulators. In other words, this 
chapter does not treat the motion control of mobile 
robots, flexible manipulators, and manipulators 
with elastic joints. The main challenge in the mo¬ 
tion control problem of rigid manipulators is the 
complexity of their dynamics and uncertainties. 
The former results from nonlinearity and coupling 
in the robot manipulators. The latter is twofold: 
structured and unstructured. Structured uncer¬ 
tainty means imprecise knowledge of the dynamic 
parameters and will be touched upon in this chap¬ 
ter, whereas unstructured uncertainty results from 
joint and link flexibility, actuator dynamics, fric¬ 
tion, sensor noise, and unknown environment 
dynamics, and will be treated in other chapters. 

In this chapter, we begin with an introduc¬ 
tion to motion control of robot manipulators from 
a fundamental viewpoint, followed by a survey 
and brief review of the relevant advanced mate¬ 
rials. Specifically, the dynamic model and useful 
properties of robot manipulators are recalled in 
Sect. 8.1. The joint and operational space control 
approaches, two different viewpoints on control of 
robot manipulators, are compared in Sect. 8.2. 
Independent joint control and proportional- 
integral-derivative (PID) control, widely adopted 
in the field of industrial robots, are presented 
in Sects. 8.3 and 8.4, respectively. Tracking con¬ 
trol, based on feedback linearization, is introduced 
in Sect. 8.5. The computed-torque control and its 
variants are described in Sect. 8.6. Adaptive con¬ 
trol is introduced in Sect. 8.7 to solve the problem 
of structural uncertainty, whereas the optimality 
and robustness issues are covered in Sect. 8.8. To 
compute suitable set point signals as input values 
for these motion controllers, Sect. 8.9 introduces 
reference trajectory planning concepts. Since most 
controllers of robot manipulators are implemented 


by using microprocessors, the issues of digital im¬ 
plementation are discussed in Sect. 8.10. Finally, 
learning control, one popular approach to intelli¬ 
gent control, is illustrated in Sect. 8.11. 
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8.7.4 Adaptive Control 

with Desired Compensation . 180 

8.7.5 Summary . 180 

8.8 Optimal and Robust Control . 181 
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8.1 Introduction to Motion Control 

The dynamical model of robot manipulators will be re¬ 
called in this section. Furthermore, important properties 
of this dynamical model, which is useful in controller 
design, will then be addressed. Finally, different control 
tasks of the robot manipulators will be defined. 

8.1.1 Dynamical Model 

For motion control, the dynamical model of rigid robot 
manipulators is conveniently described by Lagrange 
dynamics. Let the robot manipulator have n links 
and let the (n x l)-vector q of joint variables he q = 
[q\,. .. , <g , „] T . The dynamic model of the robot manipu¬ 
lator is then described by Lagrange’s equation [8.1-6] 

H(q)q + C(q,q)q + T g (q) = r , (8.1) 

where Yt(q) is the (n x n) inertia matrix, C(q, q)q is the 
(n x l)-vector of Coriolis and centrifugal forces, T g (q) 
is the ( n x l)-vectorof gravity force, and r is the (n x 1)- 
vector of joint control inputs to be designed. Friction 
and disturbance input have been neglected here. 


Remark 8.1 

Other contributions to the dynamic description of the 
robot manipulators may include the dynamics of the 
actuators, joint and link flexibility, friction, noise, and 
disturbances. Without loss of generality, the case of the 
rigid robot manipulators is stressed here. 


The control schemes that we will introduce in this 
chapter are based on some important properties of the 
dynamical model of robot manipulators. Before giving 
a detailed introduction to these different schemes, let us 
first give a list of those properties. 
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Property 8.1 

The inertia matrix is a symmetric positive-definite ma¬ 
trix, which can be expressed 

A h I n < H( 9 ) < A H I„ , (8.2) 

where Ah and Ah denote positive constants. 

Property 8.2 

The matrix N(#, q) = H(<jr) — 2C(q, q) is skew- 
symmetric for a particular choice of C (q, q) (which is 
always possible), i. e., 

Z T N(q,q)z = 0 (8.3) 

for any (n x l)-vectorz. 

Property 8.3 

The (n x nj-matrix C(q, q) satisfies 

II C(q.q) || < c 0 || q || (8.4) 

for some bounded constant c 0 . 


Property 8.4 

The gravity force/torque vector satisfies 

II Tg(?) ||<g 0 (8.5) 

for some bounded constant g a . 


Property 8.5 

The equation of motion is linear in the inertia pa¬ 
rameters. In other words, there is a (rxl) constant 

































Motion Control I 8.1 Introduction to Motion Control 


vector a and an (n x r) regressor matrix Y (q.q.q) such 
that 

H(q)q + C(q,q)q+T g (q) = Y (q.q.q)a. (8.6) 

The vector a is comprised of link masses, moments of 
inertia, and the link, in various combinations. 

Property 8.6 

The mapping t —>■ q is passive; i. e., there exists a > 0 
such that 

t 

J q T (/3)T(l3)d/3 > — a , Vr< oc . (8.7) 

0 


Remarks 8.1 

• Properties 8.3 and 8.4 are very useful since they al¬ 
low us to establish upper bounds on the nonlinear 
terms in the dynamical model. As we will see fur¬ 
ther, several control schemes require knowledge of 
such upper bounds. 

• In Property 8.5, the parameter vector a is comprised 
of several variables in various combinations. The di¬ 
mensionality of the parameter space is not unique, 
and the search over the parameter space is an im¬ 
portant problem. 

• In this section, we assume that the robot manipu¬ 
lator is fully actuated and this indicates that there 
is an independent control input for each degree of 
freedom (DOF). In contrast, the robot manipulators 
with joint or link flexibility are no longer fully actu¬ 
ated and the control problems are more difficult in 
general. 


8.1.2 Control Tasks 

It is instructive for comparative purposes to classify 
control objectives into the following two classes: 

• Trajectory tracking is aimed at following a time- 
varying joint reference trajectory specified within 


the manipulator workspace. In general, this desired 
trajectory is assumed to comply with the actua¬ 
tors’ capacity. In other words, the joint velocity and 
acceleration associated with the desired trajectory 
should not violate, respectively, the velocity and ac¬ 
celeration limit of the manipulator. In practice, the 
capacity of actuators is set by torque limits, which 
result in bounds on the acceleration that are com¬ 
plex and state dependent. 

• Regulation sometimes is also called point-to-point 
control. A fixed configuration in the joint space 
is specified; the objective is to bring to and keep 
the joint variable at the desired position in spite of 
torque disturbances and independently of the initial 
conditions. The behavior of transients and over¬ 
shooting, are in general, not guaranteed. 

The selection of the controller may depend on the 
type of task to be performed. For example, tasks only 
requiring the manipulator to move from one position to 
another without requiring significant precision during 
the motion between these two points can be solved by 
regulators, whereas such as welding, painting, and so 
on, require tracking controllers. 


Remarks 8.2 

• The regulation problem may be seen as a special 
case of the tracking problem (for which the desired 
joint velocity and acceleration are zero). 

• The task specification above is given in the joint 
space and results in joint space control, which is 
the main content of this chapter. Sometimes, the 
task specification of the robot manipulators in terms 
of the desired trajectory of the end-effector (e.g., 
control with eye-in-hand) is carried out in the task 
space and gives rise to the operational space control, 
which will be introduced in Sect. 8.2. 


8.1.3 Summary 

In this section, we introduced the dynamical model of 
the robot manipulators and important properties of this 
dynamical model. Finally, we defined different control 
tasks of the robot manipulators. 
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8.2 Joint Space Versus Operational Space Control 


In a motion control problem, the manipulator moves to 
a position to pick up an object, transports that object 
to another location, and deposits it. Such a task is an 
integral part of any higher-level manipulation tasks such 
as painting or spot-welding. 

Tasks are usually specified in the task space in terms 
of a desired trajectory of the end-effector, while con¬ 
trol actions are performed in the joint space to achieve 
the desired goals. This fact naturally leads to two kinds 
of general control methods, namely joint space con¬ 
trol and operational space control (task space control) 
schemes. 

8.2.1 Joint Space Control 

The main goal of the joint space control is to design 
a feedback controller such that the joint coordinates 
q(t) G R" track the desired motion q,i(t) as closely as 
possible. To this end, consider the equations of mo¬ 
tion (8.1) of an w-DOF manipulator expressed in the 
joint space [8.2,4]. In this case, the control of robot 
manipulators is naturally achieved in the joint space, 
since the control inputs are the joint torques. Neverthe¬ 
less, the user specifies a motion in terms of end-effector 
coordinates, and thus it is necessary to understand the 
following strategy. 

Figure 8.1 shows the basic outline of the joint space 
control methods. Firstly, the desired motion, which is 
described in terms of end-effector coordinates, is con¬ 
verted to a corresponding joint trajectory using the 
inverse kinematics of the manipulator. Then the feed¬ 
back controller determines the joint torque necessary to 
move the manipulator along the desired trajectory spec¬ 
ified in joint coordinates starting from measurements of 
the current joint states [8. 1,4, 7, 8]. 

Since it is always assumed that the desired task is 
given in terms of the time sequence of the joint mo¬ 
tion, joint space control schemes are quite adequate in 
situations where manipulator tasks can be accurately 
preplanned and little or no online trajectory adjustments 
are necessary [8.1,4, 7, 9]. Typically, inverse kinemat¬ 
ics is performed for some intermediate task points, and 
the joint trajectory is interpolated using the intermediate 
joint solutions. Although the command trajectory con¬ 
sists of straight-line motions in end-effector coordinates 



Fig. 8.1 Generic concept of joint space control 


between interpolation points, the resulting joint motion 
consists of curvilinear segments that match the desired 
end-effector trajectory at the interpolation points. 

In fact, the joint space control includes simple 
proportional-derivative (PD) control, PID control, in¬ 
verse dynamic control, Lyapunov-based control, and 
passivity-based control, as explained in the following 
sections. 

8.2.2 Operational Space Control 

In more complicated and less certain environments, 
end-effector motion may be subject to online modifi¬ 
cations in order to accommodate unexpected events or 
to respond to sensor inputs. There are a variety of tasks 
in manufacturing where these type of control problem 
arise. In particular, it is essential when controlling the 
interaction between the manipulator and environment 
is of concern. 

Since the desired task is often specified in the 
operational space and requires precise control of the 
end-effector motion, joint space control schemes are 
not suitable in these situations. This motivated a dif¬ 
ferent approach, which can develop control schemes 
directly based on the dynamics expressed in the oper¬ 
ational space [8.10,11]. 

Let us suppose that the Jacobian matrix, denoted 
by J(^) G R" Xn , transforms the joint velocity (q g R ") 
to the task velocity (x G R") according to 

x = J(q)q . (8.8) 

Furthermore, assume that it is invertible. Then, the op¬ 
erational space dynamics is expressed as follows 

f c = A(q)x+T(q,q)x + i)(q), (8.9) 

where f c G R" denotes the command forces in the op¬ 
erational space; the pseudo-inertia matrix is defined by 

A( 9 )=J- T ( 9 )H( ? )J- 1 ( 9 ), (8.10) 

and T (q. q) and q(q) are given by 

T (q . q) = J~ T (q)C(q, q)3~\q) 

- A(< 7 )J(< 7 )J _I (< 7 ) , 

q(q) = J~ T (q)r g (q). 

The task space variables are usually reconstructed 
from the joint space variables, via the kinematic map¬ 
pings. In fact, it is quite rare to have sensors to directly 
measure end-effector positions and velocities. Also, it 
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is worth remarking that an analytical Jacobian is uti¬ 
lized since the control schemes operate directly on task 
space quantities, i. e., the end-effector position and ori¬ 
entation. 

The main goal of the operational space control is 
to design a feedback controller that allows execution 
of an end-effector motion x(t) e R" that tracks the de¬ 
sired end-effector motion x ( |(/j as closely as possible. 
To this end, consider the equations of motion (8.9) of 
the manipulator expressed in the operational space. For 
this case, Fig. 8.2 shows a schematic diagram of the 
operational space control methods. There are several 
advantages to such an approach because operational 



Fig. 8.2 Basic concept of operational space control 


space controllers employ a feedback loop that directly 
minimizes task errors. Inverse kinematics need not be 
calculated explicitly, since the control algorithm em¬ 
beds the velocity-level forward kinematics (8.8), as 
shown in the figure. Now, motion between points can 
be a straight-line segment in the task space. 


8.3 Independent-Joint Control 

By independent-joint control (i. e., decentralized con¬ 
trol), we mean that the control inputs of each joint only 
depends on the measurement of the corresponding joint 
displacement and velocity. Due to its simple structure, 
this kind of control schemes offers many advantages. 
For example, by using independent-joint control, com¬ 
munication among different joints is saved. Moreover, 
since the computational load of controllers may be re¬ 
duced, only low-cost hardware is required in actual 
implementations. Finally, independent-joint control has 
the feature of scalability, since the controllers on all 
the joints have the same formulation. In this section, 
two kinds of design of independent-joint control will 
be introduced: one focused on the dynamical model 
of each joint (i. e., based on the single-joint model) 
and the other based on the analysis of the overall dy¬ 
namical model (i. e., the multijoint model) of robot 
manipulators. 

8.3.1 Controller Design Based 
on the Single-Joint Model 

The simplest independent-joint control strategy is to 
control each joint axis as a single-input single-output 
(SISO) system. Coupling effects among joints due to 
varying configuration during motion are treated as dis¬ 
turbance inputs. Without loss of generality, the actuator 
is taken as a rotary electric direct-current (DC) mo¬ 
tor. Hence, the block diagram of the control scheme of 
joint i can be represented in the domain of the complex 
variables as shown in Fig. 8.3. In this scheme, 6 is the 
angular variable of the motor, J is the effective inertia 
viewed from the motor side, J? a is the armature resis¬ 
tance (auto-inductance being neglected), and k t and k v 
are, respectively, the torque and motor constants. Fur¬ 
thermore, G v denotes the voltage gain of the power 


amplifier so that the reference input is the input volt¬ 
age V c of the amplifier instead of the armature voltage 
V a . It has also been assumed that F m k v k t /J? a , i. e., 
the mechanical (viscous) friction coefficient has been 
neglected with respect to the electrical coefficient. Now 
the input-output transfer function of the motor can be 
written as 


M(s) 


■s(l + -S'7m) ’ 


( 8 . 11 ) 


where k m = G v /k v and T m = R a J/k v k t are, respectively, 
the voltage-to-velocity gain and the time constant of the 
motor. 

To guide selection of the control structure, start by 
noticing that an effective rejection of the disturbance d 
on the output 6 is ensured by 

1. A large value of the amplifier before the point of 
intervention of the disturbance 

2. The presence of an integral action in the con¬ 
troller so as to cancel the effect of the gravitational 
component on the output at the steady state (i. e., 
constant 6). 

In this case, as shown in Fig. 8.4, the types of 
control action with position and velocity feedback are 



Fig. 8.3 Block scheme of a joint-driven system (after [8.4]) 
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Fig. 8.4 Block scheme of position and velocity feedback (after [8.4]) 


characterized by [8.4] 


GJs) = K P , G v (s) = Ky 1 + V7v , (8.12) 

s 

where G p (s) and G v (s) correspond to the position and 
velocity control actions, respectively. It is worth not¬ 
ing that the inner control action G v (s) is in a form of 
propositional-integral (PI) control to yield zero error in 
the steady state under a constant disturbance d. Further¬ 
more, and kjy are both transducer constants, and 
the amplifier gain Ky has been embedded in the gain of 
the inner controller. From the scheme of Fig. 8.4, the 
transfer function of the forward path is 


P(s) = 


k m K P K w (l + sT w ) 


s(l+sT m ) 
while that of the return path is 


H(s) = k TP 


V ^fp^TP / 


(8.13) 


(8.14) 


The zero of the controller at s = —1/Ty can be chosen 
so as to cancel the effects of the real pole of the motor at 
s = — 1 / T m . Then, by setting Ty = T m , the poles of the 
closed-loop system move on the root locus as a function 
of the loop gain, k m Kykjy. By increasing the position 
feedback gain K P , it is possible to confine the closed- 
loop poles to a region of the complex plane with large 
absolute real part. Then, the actual location can be es¬ 
tablished by a suitable choice of Ky. 

The closed-loop input-output transfer function is 


1 

® ( 4 ) _ kpp 

©rO) ] sk TP s 2 

Kpkjp k m KpkjpKy 


(8.15) 


which can be compared with the typical transfer func¬ 
tion of a second-order system 


W (s) = 


1 

^TP 


2 s 2 

1 -I-1- j 

a>n «- 


(8.16) 


It can be recognized that, with a suitable choice of the 
gains, it is possible to obtain any value of natural fre¬ 
quency co n and damping ratio f. Hence, if co n and £ are 
given as design specifications, the following relations 
can be found 


Kykjy — — and KpkypKy — —^ . (8.17) 

k m k m 

For given transducer constants £tp and k-py, Ky and 
Kp will be chosen to satisfy the two equations above, 
respectively. On the other hand, the closed-loop distur¬ 
bance/output function is 


sR d 

©(A) = hKpK T pKy(l+sT m ) 
D{s) skpy s 2 

Kpkpp k m K P kppKy 


(8.18) 


which shows that the disturbance rejection factor is 
2 lr(s) = KpkppKy and is fixed, provided that Ky and K P 
have been chosen via the approach above. Concerning 
the disturbance dynamics, the zero at the origin intro¬ 
duced by the PI, a real pole at 5 = — 1/T m , and the pair 
of complex poles with real part — £<y n should be kept in 
mind. In this case, an estimate T R of the output recov¬ 
ery time needed by the control system to recover from 
the effect of a disturbance on the joint position can be 
evaluated by analyzing models of the transfer function 
above. Such an estimate can reasonably be expressed as 
T r = max{7j n , 1 /£«}. 
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8.3.2 Controller Design Based 
on the Multijoint Model 

In recent years, independent-joint control schemes 
based on the complete dynamic model of the robot 
manipulators (i. e., a multijoint model) have been pro¬ 
posed. For example, following the approach of com- 
puted-torque-like control, [8.12] dealt with the regula¬ 
tion task for horizontal motions, and [8.13] and [8.14] 
handled the tracking task for arbitrary smooth trajec¬ 
tories. Since the overall dynamic model is considered, 
the effects of coupling among joints are handled. These 
schemes will be introduced in detail in Sect. 8.6. 

8.3.3 Summary 

In this section, we have presented two independent- 
joint control schemes: one is based on the single-joint 
model and the other is based on the multijoint model. 


8.4 PID Control 

Traditionally, control design in robot manipulators can 
be understood as the simple fact of tuning of a PD or 
PID compensator at the level of each motor driving 
the manipulator joints [8.1]. Fundamentally, a PD con¬ 
troller is a position and velocity feedback that has good 
closed-loop properties when applied to a double inte¬ 
grator system. 

The PID control has a long history since Ziegler 
and Nichols’ PID tuning rules were published in 
1942 [8.15], Actually, the strong point of PID control 
lies in its simplicity and clear physical meaning. Sim¬ 
ple control is preferable to complex control, at least in 
industry, if the performance enhancement obtained by 
using complex control is not significant enough. The 
physical meanings of PID control [8.16] are as follows: 

• P-control means the present effort making a present 
state into desired state 

• I-control means the accumulated effort using the ex¬ 
perience information of previous states 

• D-control means the predictive effort reflecting the 
information about trends in future states. 

8.4.1 PD Control for Regulation 

A simple design method for manipulator control is to 
utilize a linear control scheme based on the lineariza¬ 
tion of the system about an operating point. An example 
of this method is a PD control with a gravity compen¬ 
sation scheme [8.17, 18]. Gravity compensation acts as 


The former focuses on the dynamics of a single joint 
and regards the interaction among joints as a distur¬ 
bance. This control scheme is simple but may not be 
suitable for high-speed tracking. Hence, we introduce 
the latter, which considers the overall dynamical model 
of robot manipulators such that the interaction among 
joints can be handled. 

Further Reading 

There are different types of feedback applied in the 
independent-joint control based on the single-joint 
model (such as pure position feedback or position, 
velocity, and acceleration feedback). A complete dis¬ 
cussion is given in [8.4]. When the joint control servos 
are required to track reference trajectories with high 
speeds and accelerations, the tracking capabilities of the 
above schemes are inevitably degraded. A possible rem¬ 
edy is to adopt decentralized feedforward compensation 
to reduce tracking errors [8.4, 5]. 


a bias correction, compensating only for the amount of 
forces that create overshooting and an asymmetric tran¬ 
sient behavior. Formally, it has the following form 

t = Kpf^d — q) — Ky<jr + t g (q) , (8.19) 

where K P and Ky € R nXn are positive-definite gain ma¬ 
trices. This controller is very useful for set-point regula¬ 
tion, i. e., c/,i = constant [8.7, 18]. When this controller 
is applied to (8.1), the closed-loop equation becomes 

H(#)</ + C(q.q)q + K v q- K P e q = 0 , (8.20) 

where e q = — q. and the equilibrium point is y = 

[e q , <jr T ] J = 0. Now, the stability achieved by PD control 
with gravity compensation can be analyzed accord¬ 
ing to the closed-loop dynamic (8.20). Consider the 
positive-definite function 

V = ^q T H(q)q + ie q K v e q . 

Then, the derivative of function becomes negative 
semidefinite for any value of q by using Property 8.2 
in Sect. 8.1, i. e., 

V = Hr T Ky?<—A min (Ky)||«jr|| 2 , (8.21) 

where A^lKy) means the smallest eigenvalue of 
K v . By invoking the Lyapunov stability theory and 
LaSalle’s theorem [8.1], it can be shown that the regu¬ 
lation error will converge asymptotically to zero, while 
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their high-order derivatives remain bounded. This con¬ 
troller requires knowledge of the gravity components 
(structure and parameters), though it is simple. 

Now, consider simple PD control without gravity 
compensation 

r = Kp(g d -qO -K w q , (8.22) 

then the closed-loop dynamic equation becomes 

H (q)q + C(q, q)q + T g (q) + K v ? - K P e q = 0 . 

(8.23) 

Consider the positive definite function 

V= i q T H(q)q+^e q K v e q +U(q) + U 0 , 

where U(q) denotes the potential energy with the rela¬ 
tion of dU(q)/dq = r g (q) and Uq is a suitable constant. 
Taking time derivative of V along the closed-loop dy¬ 
namics (8.23) gives the same result (8.21) with previous 
one using gravity compensation. In this case, the control 
system must be stable in the sense of Lyapunov, but it 
can not conclude that the regulation error will converge 
to zero by LaSalle’s theorem [8.1]. Actually, the system 
precision (the size of the regulation error vector) will 
depend on the size of the gain matrix K P in the follow¬ 
ing form 

IM < l|K P :l ||go , (8.24) 

where go is that in Property 8.4 in Sect. 8.1. Hence, the 
regulation error can be arbitrarily reduced by increasing 
K P ; nevertheless, measurement noise and other unmod¬ 
eled dynamics, such as actuator friction, will limit the 
use of high gains in practice. 

8.4.2 PID Control for Regulation 

An integral action may be added to the previous PD 
control in order to deal with gravity forces, which to 
some extent can be considered as a constant disturbance 
(from the local point of view). The PID regulation con¬ 
troller can be written in the following general form 

r = K P (q d -q) + Ki J f(q d -q)dt-K v q , 

where K| e A'" x " is a positive-definite gain matrix, and: 

• lff(q d — q) = < 7 d — q, we have PID control. 

• If Ki f {—q)dt is added, we have PI 2 D control. 

• If/(-) = tanh(-), we have PD + nonlinear integral 
control. 

Global asymptotic stability (GAS) by PID control 
was proved in [8.12] for robotic motion control sys¬ 
tem including external disturbances, such as Coulomb 


friction. Also, Tomei proved the GAS of PD control 
in [8.19] by using an adaptation for gravity term. On 
the other hand, Ortega et al. showed in [8.20] that 
PI 2 D control could yield semiglobal asymptotic sta¬ 
bility (SGAS) in the presence of gravity and bounded 
external disturbances. Also, Angeli proved in [8.21] that 
PD control could achieve the input-output-to-state sta¬ 
bility (IOSS) for robotic systems. Also, Ramirez et al. 
proved the SGAS (with some conditions) for PID gains 
in [8.22], Also, Kelly proved in [8.23] that PD plus 
nonlinear integral control could achieve GAS under 
gravity. 

Actually, a large integral action in PID control can 
cause instability of the motion control system. In order 
to avoid this, the integral gain should be bounded by an 
upper limit of the following form [8.1] 

kpky 


where Ah is that in Property 8.1 in Sect. 8.1, K P = fc P I, 
Kj = A t I, and K v = Ayl. This relation gives the guide¬ 
lines for gain selection implicitly. Also, PID control has 
generated a great variety of PID control plus something, 
e.g., PID plus friction compensator, PID plus gravity 
compensator, PID plus disturbance observer. 

8.4.3 PID Gain Tuning 

The PID control can be utilized for trajectory tracking 
as well as set-point regulation. True tracking control 
will be treated after Sect. 8.5. In this section, the simple 
but useful PID gain tuning method will be introduced 
for practical use. The general PID controller can be 
written in the following general form 

r = K v e q + K P e q + Ki J e q dt , 

or, in another form, 

r = + 4^-1 j ^e q + K P e q + Ki J e q dt j . 

(8.25) 

In a fundamental stability analysis of tracking con¬ 
trol systems, Qu and Dorsey proved in [8.24] that 
PD control could satisfy uniform ultimate boundedness 
(UUB). Also, Berghuis and Nijmeijer proposed output 
feedback PD control, which satisfies semiglobal uni¬ 
form ultimate boundedness (SGUUB) in [8.25] under 
gravity and a bounded disturbance. Recently, Choi et al. 
suggested inverse optimal PID control [8.26], assuring 
extended disturbance input-to-state stability (ISS). 
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Actually, if a PID controller (8.25) is repeatedly ap¬ 
plied to the same set point or desired trajectory, then the 
maximum error will be proportional to the gains in the 
following form 


and its automatic tuning rule as follow 

d K, ,, , 

- = T i s~ i (t), for i = 1, ■ • • , n , (8.27) 


max ||e 0 (t)|| « , = • (8.26) 

y 2ky 2 + 1 


where tf denotes the final execution time of a given 
task and K = k\. This relation can be utilized to tune 
the gain of a PID controller and is referred to as the 
compound tuning rule [8.16]. The compound tuning 
rule implicitly includes simple tuning rules as follows 
(y tuning method): 

• Square tuning: max ||e q || oc y 2 , for a small k 

• Linear tuning: max ||e q || oc y, for a large k. 

For example, suppose we select positive constant 
diagonal matrices K P = A P I, Ki = Aril, while satisfying 
kp > 2k\. For small k , the maximum error will be re¬ 
duced by i according to the square tuning rule, if we 
reduce the value y by For large k, the maximum 
error will be proportionally reduced as y according to 
the linear tuning rule. This means that we can tune the 
PID controller using only one parameter y when the 
other gain parameters are fixed [8.16] 

Although these rules are very useful in tuning the con¬ 
trol performance, they can be utilized only for repetitive 
experiments for the same set point or desired trajec¬ 
tory because the tuning rules consist of proportional 
relations. 

8.4.4 Automatic Tuning 

For simplicity, define the composite error to be 


where T, implies an update gain parameter for i-th con¬ 
trol joint. 

For practical use of the PID control, a target perfor¬ 
mance denoted by 12 is specified in advance 

sup |s(t)| = 12 , 

0<f<? 


in order to maintain the composite error within the re¬ 
gion Q. Moreover, since the auto-tuning rule has the 
decentralized type (8.27), we suggest the decentralized 
criterion for auto-tuning as follows 



(8.28) 


where n is the number of the joint coordinates. As 
soon as the composite error arrives at the tuning region 
(8.28), the auto-tuning rule is implemented to assist the 
achievement of target performance. On the contrary, if 
the composite error stays in non-tuning region, namely, 
|.y,| < Q / V2n, then the auto-tuning process stops. For 
this case, we expect that the gain K updated by an auto¬ 
tuning rule (8.27) would be larger than the matrix 
able to achieve the target performance 12. As a matter 
of fact, the auto-tuning rule plays a nonlinear damping 
role in the auto-tuning region. 

Matlab Example (Multimedia) 

Simple automatic tuning example for one-link manipu¬ 
lator control system is shown in the multimedia source 
to help readers’ understanding. 


s(t) =e q + K P e q + Ki J e q d t. 

Now simple auto-tuning PID control is suggested by 
choosing one tuning parameter K as shown in the fol¬ 
lowing control form 


Further Reading 

The PID-type controllers were designed to solve the 
regulation control problem. They have the advantage 
of requiring knowledge of neither the model structure 
nor the model parameters. Also, the stability achieved 
by PID-type controllers was presented in this section. 
A range of books and papers [8.1, 15,16,22,27, 28] are 
available to the robotics audience, detailing the indi¬ 
vidual tuning methods used in PID control and their 
concrete proofs. 
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8.5 Tracking Control 

While independent PID controls are adequate in most 
set-point regulation problems, there are many tasks that 
require effective trajectory tracking capabilities such as 
plasma-welding, laser-cutting or high-speed operations 
in the presence of obstacles. In this case, employing lo¬ 
cal schemes requires moving slowly through a number 
of intermediate set points, thus considerably delay¬ 
ing the completion of the task. Therefore, to improve 
the trajectory tracking performance, controllers should 
take account of the manipulator dynamic model via 
a computed-torque-like technique. 

The tracking control problem in the joint or task 
space consists of following a given time-varying trajec¬ 
tory q d (t ) or jc c i(/) and its successive derivatives <jr d (f) or 
x,i(l) and q d (t) or x d (t), which describe the desired ve¬ 
locity and acceleration, respectively. To obtain success¬ 
ful performance, significant effort has been devoted to 
the development of model-based control strategies [8.1, 
2, 7]. Among the control approaches reported in the lit¬ 
erature, typical methods include the inverse dynamics 
control, the feedback linearization technique, and the 
passivity-based control method. 

8.5.1 Inverse Dynamics Control 

Though the inverse dynamics control has a theoretical 
background, such as the theory of feedback lineariza¬ 
tion discussed later, its starting point is mechanical en¬ 
gineering intuition based on cancelling nonlinear terms 
and decoupling the dynamics of each link. Inverse dy¬ 
namics control in joint space has the form 

t =H(q)v+ C(q,q)q + T g (q) , (8.29) 

which, applied to (8.1), yields a set of n decoupled lin¬ 
ear systems, e.g., q = v, where v is an auxiliary control 
input to be designed. Typical choices for v are 

v = q d + K v (q d -q)+ K P (q d -q) , (8.30) 

or with an integral component 

v = q d + K v (q d -q) + Kp(q d -q) 

+ K 1 J(q i -q)dt , (8.31) 

leading to the error dynamics equation 
Cq + Ky£q + KpCq = 0 
for an auxiliary control input (8.30), and 
e^ 3) + K v e q + K P e q + Kje q = 0 , 


if an auxiliary control input (8.31) is used. Both error 
dynamics are exponentially stable by a suitable choice 
of the gain matrices Ky and K P (and Kq). 

Alternatively, inverse dynamics control can be de¬ 
scribed in the operational space. Consider the opera¬ 
tional space dynamics (8.9). If the following inverse 
dynamics control is used in the operational space, 

/ c = A (q) (x d + K v e x + K P t\ ) + T (q,q)x + q(q), 
where e x = x d —x, the resulting error dynamics is 

e x + Kye, + K P e x = 0 , (8.32) 

and it is also exponentially stable. One apparent advan¬ 
tage of using this controller is that K P and K v can be 
selected with a clear physical meaning in operational 
space. However, as can be seen in (8.10), A (q) becomes 
very large when the robot approaches singular config¬ 
urations [8.8]. This means that large forces in some 
direction are needed to move the arm. 

8.5.2 Feedback Linearization 

This approach generalizes the concept of inverse dy¬ 
namics of rigid manipulators. The basic idea of feed¬ 
back linearization is to construct a transformation as 
a so-called inner-loop control , which exactly linearizes 
the nonlinear system after a suitable state space change 
of coordinates. One can then design a second stage or 
outer-loop control in the new coordinates to satisfy the 
traditional control design specifications such as track¬ 
ing, disturbance rejection, etc. [8.5,29]. The full power 
of the feedback linearization scheme for manipulator 
control becomes apparent if one includes in the dy¬ 
namic description of the manipulator the transmission 
dynamics, such as the elasticity resulting from shaft 
windup, gear elasticity, etc. [8.5]. 

In recent years, an impressive volume of litera¬ 
ture has emerged in the area of differential-geometric 
methods for nonlinear systems. Most of the results 
in this area are intended to give abstract coordinate- 
free descriptions of various geometric properties of 
nonlinear systems and as such are difficult for the non¬ 
mathematician to follow. It is our intention here to give 
only the basic idea of the feedback linearization scheme 
and to introduce a simple version of this technique 
that finds an immediate application to the manipula¬ 
tor control problem. The reader is referred to [8.30] for 
a comprehensive treatment of the feedback linearization 
technique using differential-geometric methods. 

Let us now develop a simple approach to the de¬ 
termination of linear state-space representations of the 
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manipulator dynamics (8.1) by considering a general 
sort of output £ G R p 

£ = h(q) +r(t) , (8.33) 

where h(q) is a general predetermined function of the 
joint coordinate q e R" and r(t) is a general predeter¬ 
mined time function. The control objective will be to 
select the joint torque inputs t in order to make the out¬ 
put |(t) go to zero. 

The choice of h(q) and r(t) is based on the control 
purpose. For example, if h(q) = —q and r(t) = q,i(t)-, 
the desired joint space trajectory we would like the 
manipulator to follow, then £ (7) = q^(t ) — q(t) = e q (7) 
is the joint space tracking error. Forcing § (7) to zero 
in this case would cause the joint variables q(t) to 
track their desired values qd(t), resulting in a manipula¬ 
tor trajectory-following problem. As another example, 
| (7) could represent the operational space tracking er¬ 
ror §(t) =x&(t)— x{t) = e x (t). Then, controlling |(7) 
to zero would result in trajectory following directly in 
operational space where the desired motion is usually 
specified. 

To determine a linear state-variable model for ma¬ 
nipulator controller design, let us simply differentiate 
the output | (7) twice to obtain 

dh 

k = ~q + r=Tq + r, (8.34) 

3 q 

1 = Tq + Tq + r , (8.35) 


Define the control input function 

u = r + Tq + T:(q)W~ x (q)[T - n(q,q)] . (8.40) 

Now we may define a state y (7) G R 2p byy = (|£) and 
write the manipulator dynamics as 



This is a linear state-space system of the form 

y = Aji + Bh , (8.42) 

driven by the control input u. Due to the special form of 
A and B, this system is called the Brunovsky canonical 
form and it is always controllable from u(t). 

Since (8.40) is said to be a linearizing transforma¬ 
tion for the manipulator dynamic equation, one may 
invert this transformation to obtain the joint torque 

r = H(q)T + (q)(u-r-jq) + n(q,q) , (8.43) 

where T+ denotes the Moore-Penrose inverse of the 
transformation matrix T(q). 

In the special case | = e q (r), and if we select u(t ) so 
that (8.41) is stable by the PD feedback u = — K P £ — 
Ky|, then T = —1„ and the control input torque r(7) 
defined by (8.43) makes the manipulator move in such 
a way thaty(7) goes to zero. In this case, the feedback 
linearizing control and the inverse dynamics control be¬ 
come the same. 

8.5.3 Passivity-Based Control 


where we defined a (p x n) transformation matrix of the 
form 

dh(ri = (dh_ dh_ 3/t\ 

3 q \3<71 3<?2 dq„) 

(8.36) 

Given the output h(q), it is straightforward to com¬ 
pute the transformation T(</) associated with h(q). In 
the special case where £ represents the operational 
space velocity error, then T(<?) denotes the Jacobian ma¬ 
trix J(g). 

According to (8.1), 

q = W\q)[T-n{q,q)\ , (8.37) 

with the nonlinear terms represented by 

n{q,q) = C(q, q)q + r g (q) . (8.38) 

Then (8.35) yields 

| = r + Tq + r T(q)13~ l {q)[x-n{q,q)\ . 


This method explicitly uses the passivity properties 
of the Lagrangian system [8.31,32], In comparison 
with the inverse dynamics method, passivity-based con¬ 
trollers are expected to have better robust properties 
because they do not rely on the exact cancellation of the 
manipulator nonlinearities. The passivity-based control 
input is given by 

q T = q d + ae q , a > 0 , 

r = H (q)q r + C (q, q)q, + r g (q) + K v e q + K P e q . 

(8.44) 

With (8.44), we obtain the following closed-loop sys¬ 
tem 

H(#)s q + C(< 7 , q)s q + K v e q + K P e q = 0 , (8.45) 

where s q = e q + ae q . Let us choose a Lyapunov func¬ 
tion V(y,t) as follows 

T7 1 T /aKv + K P + crH aH\ 1 T _ 
V= 2 y { aH H) y= 2 yFy 

(8.46) 


(8.39) 
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Since the above equation is positive definite, it has 
a unique equilibrium at the origin, i.e., y = (e q ,e q ) T 
= 0. Moreover, V can be bounded by 

CTmlbll 2 <;y T Py < ^mIItII 2 , o- M > a m > o. (8.47) 
The time derivative of V gives 

V = —e q Kye q — ae q K P e q = —y T Qy < 0 , (8.48) 

where Q = diag[aKp. K v ]. Since Q is positive definite 
and quadratic in y, it can be also bounded by 

Kmlbll 2 <T T Qy < ^m||j || 2 , *m > > o. (8.49) 

Then, from the bound of the Lyapunov function V, we 
get 

V < —Kmllyll 2 = -2??V , 17 = — , (8.50) 

O'M 

which finally yields 

7(f) < V(0)e~ 2m . (8.51) 

8.6 Computed-Torque Control 

Through the years many kinds of robot control schemes 
have been proposed. Most of these can be considered 
as special cases of the class of computed-torque control 
(Fig. 8.5) which is the technique of applying feedback 
linearization to nonlinear systems in general [8.37, 38]. 
In the section, computed-torque control will be first 
introduced, and its variant, so-called computed-torque- 
like control, will be introduced later. 

8.6.1 Computed-Torque Control 

Consider the control input (8.29) 

r = H(q)v + C(q,q)q + T g (q) , 

which is also known as computed-torque control; it con¬ 
sists of an inner nonlinear compensation loop and an 



Fig. 8.5 Computed-torque control 


It has been shown that the value of a affects the track¬ 
ing result dramatically [8.33]. The manipulator tends to 
vibrate for small values of a. Larger values of a allow 
better tracking performance and protect s q from being 
spoiled by the velocity measurement noise when the po¬ 
sition error is small. In [8.34], it was suggested that 

K P = crKy (8.52) 

be used for quadratic optimization. 

8.5.4 Summary 

In this section, we have reviewed some of the model- 
based motion control methods proposed to date. Under 
some control approaches, the closed-loop system has 
either asymptotic stability or globally exponential sta¬ 
bility. However, such ideal performance cannot be ob¬ 
tained in practical implementation because factors such 
as sampling rate, measurement noise, disturbances, and 
unmodeled dynamics will limit the achievable gain and 
the performance of the control algorithms [8.33, 35, 
36]. 


outer loop with an exogenous control signal v. Substi¬ 
tuting this control law into the dynamical model of the 
robot manipulator, it follows that 

q = V . (8.53) 

It is important to note that this control input converts 
a complicated nonlinear controller design problem into 
a simple design problem for a linear system consist¬ 
ing of n decoupled subsystems. One approach to the 
outer-loop control v is propositional-derivative (PD) 
feedback, as in (8.30) 

v = q d + K v £ q -I- K P <? q , 

in which case the overall control input becomes 

r = H(< 7 )(ij d + K v e q + K P <? q ) + C(q,q)q + t g (q) , 

and the resulting linear error dynamics are 

e q + K v e q + K P e q = 0 . (8.54) 

According to linear system theory, convergence of the 
tracking error to zero is guaranteed [8.29,39]. 


Remark 8.2 

One usually lets Ky and K P be ( 11 x 11 ) diagonal 
positive-definite gain matrices (i.e., Ky = diag(Lv, 1 , 
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■ ■ ■ , k\ n ) > 0, K P = diag(fcp i, ■ ■ • , kp n ) > 0) to guar¬ 
antee the stability of the error system. However, the for¬ 
mat of the foregoing control never leads to independent 
joint control because the outer-loop multiplier H(</) and 
the full nonlinear compensation term C(q,q)q + r g (q) 
in the inner loop scramble all joint signals among dif¬ 
ferent control channels. 


8.6.2 Computed-Torque-Like Control 

It is worth noting that the implementation of computed- 
torque control requires that parameters of the dynamical 
model are accurately known and the control input is 
computed in real time. In order to avoid those problems, 
several variations of this control scheme have been pro¬ 
posed, for example, computed-torque-like control. An 
entire class of computed-torque-like controllers can be 
obtained by modifying the computed-torque control as 

t = U(q)v+ C(q,q)q+i(q) , (8.55) 

where " represents the computed or nominal value 
and indicates that the theoretically exact feedback lin¬ 
earization cannot be achieved in practice due to the 
uncertainty in the systems. The overall control scheme 
is depicted in Fig. 8.6. 

Computed-Torque-Like Control 
with Variable-Structure Compensation 
Since there is parametric uncertainty, compensation is 
required in the outer-loop design to achieve trajectory 
tracking. The following shows a computed-torque-like 
control with variable-structure compensation 

w = <j d + K v e q + K P e q + At; , (8.56) 

where the variable-structure compensation Aw is de¬ 
vised as 


Aw = 


| ~p{x, t) 

1 ° 


B T Px 

B T Pr 


if 

if 


B t Pjt ||jt0, 

B T Pr ||=0, 

(8.57) 



Fig. 8.6 Computed-torque-like control 


where x = (e^, B = (0, I„) T , P is a (2 n x 2 n) sym¬ 

metric positive-definite matrix satisfying 

PA + A t P=-Q, (8.58) 


with A being defined as 




(8.59) 


Q being any appropriate (2 n x In) symmetric positive- 
definite matrix. 


p(x, t) = -- [aP+ || K UK x || +H0(x, 0], 

1 —a 

(8.60) 

where a and ft are positive constants such that 
||H _1 (< 7 )H(< 7 ) —1„|| < a < 1 for all q e R" and 
su Pte[o 00 ) II 9d(0 II < P, respectively, K is the (n x 2 n) 
matrix defined as K = [KpKy], Ay being a positive 
constant such that || H —1 ((jr) ||< Ay for all q e R", and 
the function tp being defined as 

|| [C(q. q) - C (q, q)\q + [r g (?) - r g (g)] || 

<<j>{xj). (8.61) 

Convergence of the tracking error to zero can be shown 
using the Lyapunov function 

V = jc t Pjc , (8.62) 

following the stability analysis in [8.5,40]. 

Remarks 8.3 

• By Property 8.1 in Sect. 8.1, there exist positive 
constants Ay and Ah such that Ah <|| H 1 (qr) ||< Ay 
for all q e R". If we choose 

- 1 Ay TAh 

H = -I„ , where c=—-- , (8.63) 

c 2 

it can be shown that 

~ Ay — Ah 

|| H- 1 ( 9 )H( 9 ) -1„ ||< s a < 1 , 

Ay + Ah 

(8.64) 

which indicates that there is always at least one 
choice of H for some a < 1. 

• Due to the discontinuity in Aw, chattering phe¬ 
nomenon may occur when the control scheme is 
applied. It is worth noting that chattering is often 
undesirable since the high-frequency component in 
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the control can excite unmodeled dynamic effect 
(such as joint flexibility) [8.6,29,38]. In order to 
avoid chattering, the variable-structure compensa¬ 
tion can be modified to become smooth, i. e., 


Av = 


—p(x, t) 
-p(x, t) 


B t Pjc 

|| B' Pjc 
B 1 P.v 


£ 


if || B t Pjc || > £ , 

if || B T Pr || < £ , 

(8.65) 


where £ is a positive constant and is used as the 
boundary layer. Following this modification, con¬ 
vergence of tracking errors to a residual set can be 
ensured, and the size of this residual set can be made 
smaller by use of a smaller value of s. 


Computed-Torque-Like Control 
with Independent-Joint Compensation 
Obviously, the previous compensation scheme is cen¬ 
tralized, which implies that the online computation load 
is heavy and high-cost hardware is required for prac¬ 
tical implementation. In order to solve this problem, 
a scheme with independent-joint compensation is in¬ 
troduced below. In this scheme, a computed-torque-like 
control is designed with estimates as 

H(<jf) = I, C(q,q) = 0, r(q)=0. (8.66) 


for some suitable positive constants fa, fa, and fa, 
where /? = (fa, fa, fa) 7 and 

w(0,«) = [l,||?||,||?||] T . (8.69) 

Finally, £,-, i e {1,..., n }, is the variable length of the 
boundary layer, satisfying 

£; = ~g, Si, £(0) > 0 , gi> 0. (8.70) 

It is worth pointing out that the term w in this control 
scheme is devised as the desired compensation rather 
than feedback. Furthermore, this scheme is in a form of 
independent-joint control and hence has the advantages 
introduced before. The convergence of the tracking er¬ 
ror to zero can be shown using the Lyapunov function 



(8.71) 

whose time derivative along the trajectory of the closed- 
loop systems follows 



with a being some positive constant, if sufficiently large 
K P and y are used. The detailed analysis of stability, 
which requires Properties 8.3 and 8.4, can be found 
in [8.13]. 


Then, we use the outer-loop v as 


V = KyCq + Kp£q + A V , 


(8.67) 


where the positive constants K P and Ky are selected 
to be sufficiently large, and the i-th component A», of 
Av = (vi, ...,v n ) T is 

~[p T w(q d ,q d )Y 

if I ^ |< 


A Vt = < 


P 7 w(q d ,q d ) 


-P J w(q d ,q d )^j 


if | Si \> 


P 7 w(q d ,q d ) 


( 8 . 68 ) 


In this compensation, s ; = e q i + A,e q>i , i e {1,..., n}, 
and A;,i e {1. n} are positive constants. Further¬ 

more, following the properties of robot manipulators, 
we have 


Remark 8.3 

Similar to computed-torque-like control with variable- 
structure compensation, we can consider the nonzero 
boundary layer as 

£, = ~g,E, , £(0) > 0 , gi, Ofj > 0 . (8.73) 

Following this modification, tracking errors converge to 
a residual set. The size of this residual set can be made 
smaller by the use of a smaller value of s (i. e., smaller 

a;)- 


For the task of point-to-point control, one PD con¬ 
troller with gravity compensation is designed with the 
estimates 

H(g) = I, C(q,q) = 0 . i g (q) = t g (q) , (8.74) 

with r g (q) being the gravity term of the dynamical 
model of the robot manipulators. Then, we use the 
outer-loop v as 

V = KyCq + Kpgq , (8.75) 


H (q)q d + C (q. q)q d + t g (q) || such that the control input becomes 

< fa + fa || q || + || q ||= P 7 w(q,q) r = K v e q + K P e q + t g (q) . 


(8.76) 
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This scheme is much simpler to implement than the 
exact computed-torque controller. The convergence of 
the tracking error to zero can be shown using the Lya¬ 
punov function 

V = H (q)e q + ^q K P g q - (8-77) 

whose time derivative along the solution trajectories of 
the closed-loop system is 

V = -e^Kv^q . (8.78) 

The detailed analysis of stability is given in [8.12], It 
is necessary to note that this result is for the case of 
regulation, rather than for the case of tracking, since 
the former theoretical base, which relies on LaSalle’s 
lemma, requires the system be autonomous (time invari¬ 
ant) [8.38,41,42], 


Remark 8.4 

If we neglect gravity in the dynamical model of the 
robot manipulators, then the gravity estimation can be 
omitted here, i. e., i „(q) = 0, so that the control law be¬ 
comes 

r = v = Kye q + K P e q , (8.79) 


8.7 Adaptive Control 

An adaptive controller differs from an ordinary con¬ 
troller in that the controller parameters are time varying, 
and there is a mechanism for adjusting these param¬ 
eters online based on some signals in the closed-loop 
systems. By the use of such control scheme, the con¬ 
trol goal can be achieved even if there is parametric 
uncertainty in the plant. In this section, we will in¬ 
troduce several adaptive control schemes to deal with 
the case of imperfect knowledge of dynamical parame¬ 
ters of the robot manipulators. The control performance 
of those adaptive control schemes, including adap¬ 
tive computed-torque control, adaptive inertia-related 
control, adaptive control based on passivity, and adap¬ 
tive control with desired compensation, are basically 
derived from Property 8.5. Finally, the condition of 
persistent excitation, which is important in parameter 
convergence, will be addressed. 

8.7.1 Adaptive Computed-Torque Control 

The computed-torque control scheme is appealing, 
since it allows the designer to transform a MIMO highly 
coupled nonlinear system into a very simple decou¬ 


which leads to pure PD control. The gain matrices, Kp 
and K v , can be selected to be diagonal so that this PD 
control is in a form of independent-joint control devel¬ 
oped based on the multijoint dynamic model. 


8.6.3 Summary 

In this section, we have presented two control schemes: 
the computed-torque and computed-torque-like con¬ 
trol. The former transforms a multi-input multi-output 
(MIMO) nonlinear robotic system into a very simple 
decoupled linear closed-loop system whose control de¬ 
sign is a well-established problem. Since the practical 
implementation of the former control requires pre¬ 
knowledge of all the manipulator parameters and its 
payload, which may not be realistic, the latter was in¬ 
troduced to relax the foregoing requirement and still 
to achieve the objective of tracking subject to system’s 
uncertainty. 

Further Reading 

The PD control with different feedforward compensa¬ 
tion for tracking control is investigated in [8.43]. An 
adaptive control scheme based on PD control is pre¬ 
sented in [8.19]. 


pled linear system, whose control design is a well- 
established problem. However, this method of feedback 
linearization relies on perfect knowledge of the system 
parameters, and failure to have this will cause erroneous 
parametric estimates, resulting in a mismatch term in 
the closed-loop model of the error system. That term 
can be interpreted as a nonlinear perturbation acting at 
the input of the closed-loop system. In order to solve 
the problem due to parametric uncertainty, we instead 
consider the inverse dynamics method with parameter 
estimates of 

r = H(tf)(<? d + K v e q + K P e q ) + C(q,q)q + r g (q) , 

(8.80) 

where H, C, r g have the same functional form as H, C, 
T g . From Property 8.5 of the dynamics model, we have 

H(q)q + C(q.q)q+i g (q) = Y (q,q,q)a, (8.81) 

where Y (q,q,q), called the regressor, is a known (n x r) 
function matrix and a is the (rxl) vector that sum¬ 
marizes all the estimated parameters. Substituting this 
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control input r into the manipulator dynamics gives the 
following closed-loop error model 

H(tf)(<? q + K v £ q + K P e q ) = Y (q.q.'q)a , (8.82) 

where a = a — a. In order to acquire an appropriate 
adaptive law, we first assume that the acceleration term 
is measurable, and that the estimated inertia matrix 
H(</) is never singular. Now, for convenience, the error 
equation is rewritten as 

x = Ax + BH~ 1 (< 7 )Y(rjr,<jr.?)fl , (8.83) 

with* = (e^) T , 



The adaptive law is considered as 

a = — T -1 Y t (<jf, q, ((/)B t Pjc , (8.85) 

where T is an (r x r) positive-definite constant matrix, 
and P is a ( 2n x 2 n) symmetric positive-definite constant 
matrix satisfying 

PA + A t P = -Q, (8.86) 

with Q being a symmetric positive-definite constant 
matrix with coherent dimension. In this adaptive law, 
we made two assumptions: 

• The joint acceleration q is measurable, and 

• The bounded range of the unknown parameter is 
available. 

The first assumption is to ensure that the regres¬ 
sor Y (q.q.'q) is known a priori, whereas the second 
assumption is to allow one to keep the estimate H(</) 
nonsingular by restricting the estimated parameter a to 
lie within a range about the true parameter value. 

Convergence of the tracking error and maintaining 
boundedness of all internal signals can actually be guar¬ 
anteed by Lyapunov stability theory with the Lyapunov 
function 

V = —x t Qx . (8.87) 

Detailed stability analysis is given in [8.2]. 


Remark 8.5 

For practical and theoretical reasons, the first assump¬ 
tion above is hardly acceptable. In most cases, it is not 
easy to obtain an accurate measure of acceleration; the 
robustness of the above adaptive control scheme with 


respect to such a disturbance has to be established. 
Moreover, from a pure theoretical viewpoint, measur¬ 
ing q. q. 'q means that not only do we need the whole 
system state vector, but we also need its derivative. 


8.7.2 Adaptive Inertia-Related Control 

Another adaptive control scheme is now introduced. 
This proposed scheme does not require the measure¬ 
ment of the manipulator’s acceleration nor does it 
require inversion of the estimated inertia matrix. Hence, 
the drawbacks of the adaptive computed-torque control 
scheme are avoided. Let us consider the control input 

r = H(^r)u + C(q,q)v + i g (q) + K D s , (8.88) 

where the auxiliary signals v and s are defined as v = 
q d + Ae q and s = v — q = e q + Ae q , with A being an 
(n x n) positive-definite matrix. Following Property 8.5 
of the dynamic model, we have 

H(tf)i> + C (q,q)v + r g (q) = Y (q.q. v. i>)a , 

(8.89) 

where Y(•,•,•,•) is an (nxr) matrix of known time 
functions. The formulation above is the same type of 
the parameter separation that was used in the formula¬ 
tion of the adaptive computed-torque control. Note that 
Y (q.q. v. v) is independent of the joint acceleration. 
Similar to the formulation above, we also have 

H(q)i> + C(q,q)v + i g (q) = Y (q.q, v, v)a . 

(8.90) 

Substituting the control input into the equation of mo¬ 
tion, it follows that 

Yl(q)'q + C(q,q)q + T g (p) 

= H(tf)F + C (q,q)v + i g (q) + K D s . 

Since q = i) — s.q = v —s, the previous result can be 
rewritten as 

H(</)i + C(q.q)s + Kd* = Y (q.q, v, i))a , (8.91) 

where a = a — a. The adaptive law is considered as 

a = rY T {q,q,v,v)s . (8.92) 

The convergence of the tracking error to zero with 
boundedness on all internal signals can be shown 
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through Lyapunov stability theory using the following unknown constant parameters. We use the expression 
Lyapunov-like function given above to define 


V = ^s T H(</)s + l a , (8.9B) 

whose time derivative along the trajectories of the 
closed-loop systems can be found to be 

V = —s t KdS . (8.94) 

The detailed stability analysis is given in [8.32]. 


Remark 8.6 

• The restrictions for adaptive computed-torque con¬ 
trol formerly seen have been removed here. 

• The term K D s introduces a PD-type linear stabiliz¬ 
ing control action to the error system model. 

• The estimated parameters converge to the true pa¬ 
rameters provided the reference trajectory satisfies 
the condition of persistency of excitation, 

to+t 

ail, < J Y T (qd-q d ’ W v)Y(q d ,q d , v, v)dt 

to 

< a 2 I r , 

for all to, where cti, a 2 > and t are all positive con¬ 
stants. 


8.7.3 Adaptive Control Based on Passivity 

Taking a physics viewpoint of control, we see that the 
concept of passivity has become popular for the devel¬ 
opment of adaptive control schemes. Here, we illustrate 
how the concept of passivity can be used to design 
a class of adaptive control laws for robot manipulators. 
First, we define an auxiliary filtered tracking error sig¬ 
nal r as 

/• = F _1 (i)e q , (8.95) 

where 


Z<p = K(q)\q d + K(j)c q 


+ V(q.q) 


q„ + -KCs)e q 

S 


+ T g (q) . 


where Z is a known (n x r ) regression matrix and <p is 
a vector of unknown system parameters in the adap¬ 
tive context. It is important to note that the above can 
be arranged such that Z and r do not depend on the 
measurement of the joint acceleration q. The adaptive 
control scheme given here is called the passivity ap¬ 
proach because the mapping of — r —> Zip is constructed 
to be a passive mapping. That is, we develop an adap¬ 
tive law such that 


J — r T (a)Z(a)q>(a)da > -fi (8.97) 

0 


is satisfied for all time and for some positive scalar con¬ 
stant /3. For this class of adaptive controllers, the control 
input is given by 


r = Zip + K D r , 


(8.98) 


Detailed analysis of stability is given in [8.44]. 


Remarks 8.4 

• If K(.v) is selected such that H(.s) has a relative de¬ 
gree of one, Z and r will not depend on q. 

• Many types of control schemes can be generated 
from the adaptive passive control approach by se¬ 
lected different transfer function matrices K(j) in 
the definition of r. 

• Note that, by defining K(.sj = .vA such that F(.v) = 
(sl„ + A) -1 , we have the control input 

r = Zip — K D r , 
with 

Zip = H(< 7 )(<j d + Ae q ) + C (q, q)(q d + Ae q ) 

+ i g (q) . 


F ~\s) 


sl„ + -K(.v) 
s 


(8.96) 


and s is the Laplace transform variable. The (n x n) ma¬ 
trix K(s) is chosen such that F(s) is a strictly proper, 
stable transfer function matrix. As in the preceding 
schemes, the adaptive control strategies has close ties 
to the ability to separate the known functions from the 


The adaptive law may be chosen as 
ip = rZ T (e q + Ae q ) 

to satisfy the condition of passive mapping. This in¬ 
dicates that adaptive inertia-related control can be 
viewed as a special case of adaptive passive control. 
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8.7.4 Adaptive Control 

with Desired Compensation 

In order to implement the adaptive control scheme, 
one needs to calculate the elements of Y (q.q.'q) in 
real time. However, this procedure may be excessively 
time consuming since it involves computations with 
highly nonlinear functions of joint position and veloc¬ 
ity. Consequently, the real-time implementation of such 
a scheme is rather difficult. To overcome this diffi¬ 
culty, the adaptive control with desired compensation 
was proposed and is discussed here. In other words, 
the variables q , q, and q are replaced with the desired 
ones, namely, q d . q ( \, and q d . Since the desired quan¬ 
tities are known in advance, all their corresponding 
calculations can be performed offline, which renders 
real-time implementation more plausible. Let us con¬ 
sider the control input 

r = Y(q d ,q d ,q d )a + k^s + k p e q + k n \\e q \\ 2 s , (8.99) 

where the positive constants fc a , k v , and k„ are suffi¬ 
ciently large, and the auxiliary signal s is defined as 
s = e q + e q . The adaptive law is considered as 

a = -TY J (q d ,q d ,q i )s . (8.100) 

It is worth noting that the desired compensation is 
adopted in both the control and adaptive laws such that 
the computational load can be drastically reduced. For 
the sake of this analysis, we note that 

\\Y(q,q,q)a- Y(q d ,q d ,q d )a\\ 

— £l|| e qll + ?2|kq||" + &IMI + ? 4 ||*|| ll e qll > 

where £i, £2, £3. and £4 are positive constants. In order 
to achieve trajectory tracking, it is required that 

> £2 + £4 , 


, £l , j. , £2 

ky > ~ + £3 + — , 

(i. e., the gains k d , k p , and k v must be sufficiently 
large). The convergence of the tracking error to zero 
with boundedness on all internal signals can be proved 
through application of Lyapunov stability theory with 
the following Lyapunov-like function 

V = -s t H(#)s + ifcpCqCq + ^a T r -1 fl , (8.101) 

whose time derivative along the trajectories of the 
closed-loop system can be derived as 

V < —x t Qx , (8.102) 


where 



/ £2 -£1 \ 

Q = 4 2 

^ -£l ^-£3 -£4 

' 2 4 ' 

Detailed stability analysis can be found in [8.45]. 

8.7.5 Summary 

Since the computed-torque control suffers from para¬ 
metric uncertainty, a variety of adaptive control 
schemes have been proposed. Firstly, we have pre¬ 
sented an adaptive control scheme based on computed- 
torque control. Then, in order to overcome the men¬ 
tioned drawbacks such as the measurability of the joint 
acceleration and the invertibility of the estimated iner¬ 
tia matrix, we presented an alternative adaptive control 
scheme that is free of these drawbacks. Recently, to 
incorporate a physics viewpoint into control, adaptive 
passivity-based control has become popular, and hence 
is introduced and discussed here. Finally, to reduce the 
computational load of the adaptive schemes, we pre¬ 
sented an adaptive control with desired compensation. 

Further Reading 

A computationally very fast scheme dealing with 
adaptive control of rigid manipulators was presented 
in [8.46]. The stability analysis was completed by as¬ 
suming that the joint dynamics are decoupled, i. e., 
that each joint is considered as an independent second- 
order linear system. A decentralized high-order adap¬ 
tive variable-structure control is discussed in [8.47], 
the proposed scheme makes both position and velocity 
tracking errors of robot manipulators globally converge 
to zero asymptotically while allowing all signals in 
closed-loop systems to be bounded without the infor¬ 
mation of manipulator parameters. Other pioneering 
works in the field can be found, for example, in [8.48, 
49] ; although none of the fundamental dynamic model 
properties are used, the complete dynamics are taken 
into account, but the control input is discontinuous and 
may lead to chattering. Positive definiteness of the in¬ 
ertia matrix is explicitly used in [8.50], although it was 
assumed that some time-varying quantities remain con¬ 
stant during the adaptation. It is interesting to note that 
all of these schemes were based on the concept of 
model reference adaptive control (MRAC) developed 
in [8.51] for linear systems. Therefore, they are concep¬ 
tually very different from the truly nonlinear schemes 
presented in this section. 
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A passive-based modified version of the least- 
squares estimation scheme has been proposed in [8.52] 
and [8.53], which guaranteed closed-loop stability of 
the scheme. Other schemes can be found in [8.54], 
where no use is made of the skew-symmetry property, 
and in [8.55], where the recursive Newton-Euler for¬ 
mulations is used instead of the Lagrange formulation 
to derive the manipulator dynamics, and thus computa¬ 
tion is simplified to facilitate practical implementation. 

Even though adaptive control provides a solution to 
the problem of parametric uncertainty, the robustness 

8.8 Optimal and Robust Control 

Given a nonlinear system, such as robotic manipula¬ 
tors, one can develop many stabilizing controls [8.29, 
41]. In other words, the stability of the control system 
cannot determine a unique controller. It is natural that 
one seeks an optimal controller among the many sta¬ 
ble ones. However, the design of an optimal controller 
is possible provided that a rather exact information on 
the target system is available, such as an exact sys¬ 
tem model [8.34,59]. In the presence of discrepancy 
between the real system and its mathematical model, 
a designed optimal controller is no longer optimal, 
and may even end up being instable in the actual sys¬ 
tem. Generally speaking, the optimal control design 
framework is not the best one to deal with system un¬ 
certainty. To handle system uncertainty from the control 
design stage, a robust control design framework is nec¬ 
essary [8.60]. One of main objectives of robust control 
is to keep the controlled system stable even in presence 
of uncertainties in the mathematical model, unmodeled 
dynamics, and the like. 

Let us consider an affine nonlinear system described 
by nonlinear time-varying differential equation in the 
state x = (x\ , X2, ■ ■ ■ , x n ) T e R" 

x(t) = /( x, t) + G(x , t)u + P(jt, t)w , (8.103) 

where u e R"' is the control input, and w e R w is the 
disturbance. Without disturbances or unmodeled dy¬ 
namics, the system simplifies to 

x(t) =f(x, t) + G(x, t)u . (8.104) 

Actually, there are many kinds of methods describing 
the nonlinear system according to the objective of con¬ 
trol [8.1, 16,21,23,34, 54], 

8.8.1 Quadratic Optimal Control 

Every optimal controller is based on its own cost func¬ 
tion [8.61,62]. One can define a cost function as [8.63, 


of adaptive controllers remains a topic of great interest 
in the field. Indeed, measurement noise or unmodeled 
dynamics (e.g., flexibility) may result in unbounded 
closed-loop signals. In particular, the estimated param¬ 
eters may diverge; this is a well-known phenomenon 
in adaptive control and is called parameter drift. So¬ 
lutions inspired from the adaptive control of linear 
systems have been studied [8.56,57], where a modi¬ 
fied estimation ensures boundedness of the estimates. 
In [8.58], the controller in [8.32] is modified to enhance 
robustness. 


64] 

z = H(x, t)x + K(x, t)u , 

such that H T (x, f)K(x, t ) = 0, K T (x, r)K(x, t) = R(x, t) 
> 0, and H t (jc, r)H(x, t) = Q(x, t) > 0. Then, we have 

- z T z = ^x T Q(x, t)x + ^t/ T R(x, t)u . 

The quadratic optimal control for the system (8. 104) 
is found by solving, for a first order differentiable 
positive-definite function V{x, r), the Hamilton-.!acobi- 
Bellman (HJB) equation [8.34,59] 

0 = HJB (x, t; V ) = V,(x. t) + V x (x, t)f(x, t ) 

- l -V x (x. t)G(x, t)R~\x, t) G t (jc, t)V T x {x, t) 

where V t = ^ and V x = . Then the quadratic opti¬ 

mal control is defined by 

u = — R — *(jc, r)G T (jc, t)V^(x, t) . (8.105) 

Note that the HJB equation is a nonlinear second-order 
partial differential equation in V(x,t). 

Unlike the aforementioned optimal control prob¬ 
lem, the so-called inverse quadratic optimal control 
problem is to find a set of Q(x, t) and R(x, 1) for which 
the HJB equation has a solution V(x,t). Then the in¬ 
verse quadratic optimal control is defined by (8.105). 

8.8.2 Nonlinear 3-Coo Control 

When disturbances are not negligible, one can deal with 
their effect such that 

t t 

J z T (x, r)z(x, r)dr — Y 2 J w 1 wdx , (8.106) 

o o 
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where y > 0 specifies the Z, 2 -gain of the closed-loop 
system from the disturbance input w to the cost vari¬ 
able z. This is called the Z- 2 -gain attenuation require¬ 
ment [8.63-65]. One systematic way to design an opti¬ 
mal and robust control is given by the nonlinear J-foo 
optimal control. Let y > 0 be given, by solving the fol¬ 
lowing equation 

HJIy (x, t; V) 

= V,(x, 1) + V x (x, t)f(x, t ) 

- l -V x (x, f){G(x, t)R- l (x, t)G T (x. t) 
-y- 2 P(x,f)P T (x,f)}Vj(x,f) 

+ rQfc 0 < 0 , (8.107) 

then the control is defined by 

u = —R -1 (jc, f)G T (x, t)V J(x, t) . (8.108) 

The partial differential inequality (8.107) is called 
the Hamilton-Jacobi-Isaac (HJI) inequality. Then one 
can define the inverse nonlinear 3f 00 optimal control 
problem that finds a set of Q(x, t) and R(x, t) such that 
the L 2 -gain requirement is achieved for a prescribed L 2 - 
gain y [8.66]. 

Two things deserve further comment. The first is 
that the Lq-gain requirement is only valid for distur¬ 
bance signals w whose Z, 2 -norm is bounded. The sec¬ 
ond is that '} l„o optimal control is not uniquely defined. 
Hence, one can choose a quadratic optimal among 
many 3f 00 optimal controllers. Precisely speaking, the 
control (8.108) should be called 3i oo suboptimal con¬ 
trol, since the desired Z, 2 -gain is prescribed a priori. 
A true 3^oo optimal control is to find a minimal value 
of y, such that the Z, 2 -gain requirement is achieved. 

8.8.3 Passivity-Based Design 
of Nonlinear 3-Coo Control 

There are many methodologies for the design of optimal 
and/or robust controls. Among these, passivity-based 
controls can take full advantage of the properties de¬ 
scribed above [8.31]. They consist of two parts: one 
coming from the reference motion compensation while 
preserving the passivity of the system, and the other to 
achieve stability, robustness, and/or optimality [8.66, 
67], 

Let us suppose that the dynamic parameters are 
identified as H(^), C(q,q), and r„(q), whose counter¬ 
parts are H(</), C(q, q). and t g (q), respectively. Then, 
passivity-based control generates the following tracking 
control laws 

t = H(q)q K{ + C(q,q)q K{ +T g (q)-u , (8.109) 



Fig. 8.7 The closed-loop system according to (8. Ill) 


where q Tef is the reference acceleration defined by 

9ref = 9d + Kv^q + Kptfq , (8.110) 

where Ky = diag{Ly > 0 and K P = diag{£p ,/} > o. 
Two parameters are involved in generating the refer¬ 
ence acceleration. Sometimes the following alternative 
method can be adopted 

i/ref = 'Ll + KvCq . 

This reduces the order of the closed-loop system be¬ 
cause the state x = (e 7 , e 7 ) 7 is sufficient for the system 
description, while the definition of (8.110) requires the 
state x = (/e 7 ,e 7 ,e 7 ) T . 

In Fig. 8.7, the closed-loop dynamics under the con¬ 
trol is given by 

H(#)e re f + C(q, q)e m { = u + w, (8.111) 

where 

Href — Cq T H V Cq T RpC q , 

(?ref — ^q T Kv^q T Kp ^ ^q ■ 

If d(t) = 0 and H = H, C = C, r" g = r g , then w = 0. 
Otherwise, the disturbance is defined as 

w = H(q)q Kf + C(q,q)q K f+'T g (q)+d(t) , (8.112) 

where H = H—H, C = C—C, andT g = 'r g — T g . It is of 
particular interest that the system (8.111) defines a pas¬ 
sive mapping between u + w and e m f. 

According to the manner in which the auxiliary 
control input u is specified, passivity-based control 
can achieve stability, robustness, and/or optimality 
(Fig. 8.7). 
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8.8.4 A Solution to Inverse 
Nonlinear Control 

Let us define the auxiliary control input by the 
reference-error feedback 

n = -ffR’ l (x,t)c K f, (8.113) 


Given y, one can set K = ff^jl for a > 1. This 
yields K y = (a - 1)^1. 

When the inertia matrix is identified as a diagonal 
constant matrix such as H = diagj/n,}, one should set 
C = 0. In addition, one can set r g = 0. Then this results 
in a decoupled PID control of the form 


where a > 1 is arbitrary. Then, the control provides the 
inverse nonlinear 3f 00 optimality. 


Theorem 8.1 Inverse Nonlinear Jfoo Optimal¬ 
ity [8.66] 

Let the reference acceleration generation gain matrices 
K v and K P satisfy 

Ky > 2K P . (8.114) 

Then for a given y > 0, the reference error feedback 


ti = rhi (i/d,/ + £v,;<? q ,/ + k P ,,e q ,i) 

for a > 1, which can be rewritten as 


T; = mq^i + ( fhikV'i + a yS 
ky„ 


/2 )^' 


Y 


kp,, 


+ ( rhtkpj + a-f - J e qJ + a—fj e q j . 


/■ 


(8.119) 


u = — Ke re f = —K 




(8.115) 


This leads to a PID control with the desired acceleration 
feedforward [8.68] given by 


satisfies the Lq-gain attenuation requirement for 


/KjjKy 

0 

0 \ 


Q = 0 

(Ky 2Kp)Ky 

0 

, (8.116) 

V 0 

0 

K yj 


R = K _1 , 



(8.117) 

provided that 





K y = K—^1 > 0 . (8.118) 

r 



f e q .i, 


(8.120a) 

where 


k* . = mfyj + a— , 
y- 

(8.120b) 

k Pi = niikpj+a— , 

(8.120c) 

, * kp :i 

v 

(8.120d) 


8.9 Trajectory Generation and Planning 


This section deals with the problem of reference tra¬ 
jectory generation, that is, the computation of desired 
position, velocity, acceleration and/or force/torque sig¬ 
nals that are used as input values for the robot motion 
controllers introduced in Sects 8.3-8.8. 

8.9.1 Geometric Paths and Trajectories 

Path Planning 

A path is a geometric representation of a plan to move 
from a start to a target pose. The task of planning 
is to find a collision-free path among a collection of 


static and dynamic obstacles. Path planning can also 
include the consideration of dynamic constraints such 
as workspace boundaries, maximum velocities, maxi¬ 
mum accelerations, and maximum jerks. We distinguish 
between online and offline path planning algorithms. 
Offline planned paths are static and calculated prior 
to execution. Online methods require algorithms that 
meet real-time constraints (i. e., algorithms that do not 
exceed a determinable worst-case computation time) 
to enable path (re-)calculations and/or adaptations dur¬ 
ing the robot motions in order to react to and interact 
with dynamic environments. This means that a robot 
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moves along a path that has not necessarily been com¬ 
puted completely, and which may change during the 
movement. Details about path planning concepts are de¬ 
scribed in Chap. 7, in Parts D and E, and specifically in 
Chap. 47. 

Trajectory Planning 

A trajectory is more than a path: It also includes 
velocities, accelerations, and/or jerks along a path 
A common method is computing tra¬ 
jectories for a priori specified paths, which fulfill 
a certain criterion (e.g., minimum execution time). We 
distinguish between online and offline trajectory plan¬ 
ning methods. An offline calculated trajectory cannot 
be influenced during its execution, while online trajec¬ 
tory planning methods can (re-)calculate and/or adapt 
robot motions behavior during the movement. The rea¬ 
sons for this (re-)calculation and/or adaptation can 
vary: improvement of accuracy, better utilization of cur¬ 
rently available dynamics, reaction to and interaction 
with a dynamic environment, or reaction to (sensor) 
events. Besides the distinction between online and of¬ 
fline methods, we can further distinguish between (1) 
one-dimensional (1-D) and multi-dimensional trajecto¬ 
ries and (2) single-point and multi-point trajectories. 
Multi-point trajectories typically relate to a path. 

8.9.2 Joint Space and Operational 
Space Trajectories 

Depending on the control state space, trajectory gener¬ 
ators provide set points for tracking controllers in joint 
space or in operational space. In either space, a trajec¬ 
tory can be represented in several different ways: cubic 
splines, quintic splines, higher-order splines, harmonics 
(sinusoidal functions), exponential functions, Fourier 
series, and more. 

Joint Space Trajectories 

Consider the torque control input (8.29) 

r = H (q)v + C (q, q)q + r g (q) , 
and the PD controller (8.30) 

v =q d + K v (q d -q) + K P (q d -q) , (8.121) 

or PID controller (8.31), respectively, the task of a tra¬ 
jectory generator in joint space coordinates is com¬ 
puting the signals q d (t), q d {t), and These three 

signals contain the reference trajectory and are used as 
input values for the tracking controller. 

During nominal operation the joint torques re¬ 
quired to execute a trajectory should not exceed joint 


force/torque limits r^At) and r max (f), 

TminO) < r(t) < T max (0 V t G E. (8.122) 

Operational Space Trajectories 
Similarly to (8.29), we can also consider trajectories 
represented by jfd, id, and id for an operational space 
controller (8.9) 

/c = A (q)fi + T (q, q)x + q(q) , 
with a PD control law of 

li =i d + K v (id-i) + K P (jc d -x) . (8.123) 

With the transformation into joint space using the in¬ 
verse of (8.8), the operational space trajectory generator 
must assure that the limits given in (8.122) are not 
violated. It is the responsibility of the path planner 
(Chap. 7) that all points along the trajectory are in the 
robot workspace and that start and goal poses can be 
reached in the same joint configuration. It is the respon¬ 
sibility of the trajectory planner that joint torque and 
velocity constraints are not violated even in the pres¬ 
ence of kinematic singularities. 

8.9.3 Trajectory Representations 

Mathematical Representations 
Functions for q d (t) (8.121) andx d (8.123) can be repre¬ 
sented in several ways that are described here. 

Polynomial Trajectories. One of the simplest ways to 
represent a robot trajectory is a polynomial function of 
degree m for each joint i e {1, ..., n} 

£/i(0 = <Z;,0 + fli.lt + Cti.2^ 4" ••• + , (8.124) 

so that q(t) can be composed (or x(t) in operational 
space, respectively). In the simplest case, cubic poly¬ 
nomials are used, which, however, leads to non-steady 
acceleration signals with infinite jerks. Quintic and 
higher-order polynomials allow for steady acceleration 
signals as well as arbitrary position, velocity, and accel¬ 
eration vectors and the beginning and at the end of the 
trajectory. To determine the coefficients a,j V (i, j ) e 
{1, ..., n} x {0, ..., m} of (8.124), the execution time 
ftrgt, at which the target state will be reached needs to 
be known. The left part of Fig. 8.8a shows a quintic 
trajectory for three DOFs starting at to = 0s with an 
execution time of ttr gt = 2.5 s. To connect a trajectory 
segment to preceding and succeeding segments, the fol- 
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a) q{t) (mm) 



q(t) (mm/s) 



b) q(t) (mm) 



q(t) (mm/s) 




Fig.8.8a,b Two sample trajectories for a three-DOF robot (n = 3). The trajectory in (a) is represented by quintic splines, 
and the trajectory in (b) by piecewise polynomials 


lowing six constraints need to be satisfied for all n joints 

r/i (/(}) — ///.start — tti, 0 
///(/trgt) — Qi ,trgt — ®i ,0 T Cli ] /trgt "F a, 32 /ti-gt 
+ 

< 7 ;(/o) = < 7 ;,o = fl/,1 

(/trgt) — <7/,trgt — a,-. ] T 2«g 2 /trgt T 3a,g 3 /jrgt 
+ 4a,, 4 ^ + Sajgst^ 

§,(/o) = §;,o = 20,^2 

r/,)/trgt) — ///.trgt — 2a,- 2 + 6a,g3/trgt T 12a,-. 4 /~ gt 
+ 20a,-, 5 ^, • 

A unique closed-form solution can be computed, 
so that all polynomial coefficients a,- ; - V (i, /) e 


{1.n} x {0, ..., 5} can be determined for one tra¬ 

jectory segment. 

Piecewise Polynomials. Polynomials of different de¬ 
grees can be concatenated to represent a trajectory 
between an initial state ( qo , q 0 . q 0 ) and a target state 
(< 7 trgt , q agl , q trgt ). For instance, the classical double- 
S velocity profile [8.69] with trapezoidal acceleration 
and deceleration profiles and a cruise-velocity segment 
in the middle consists of seven polynomials of de¬ 
grees 3-2-3-1-3-2-3 (m in (8.124)). The right part of 
Fig. 8.8 shows a trajectory represented by piecewise 
polynomials. To compute time-optimal trajectories un¬ 
der purely kinematic constraints (e.g., q max , <7 max , ij' max , 
etc.), piecewise polynomials are used, because they 
allow always using one signal at its kinematic limit 
(Fig. 8.8b and [8.70]). 
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Trigonometric Trajectories. Similarly to (8.124), 
trigonometric functions can be used to represent har¬ 
monic, cycloidal, and elliptic trajectories [8.71,72], 
A simple example for a harmonic trajectory for one 
joint i is 


q,(t) 


Qi, trgt qi, 0 
2 


^1 — cos 


rr (t-tp) 

hi'gt to 


+ Qi ,0 • 


(8.125) 


While any order of derivatives of trigonometric func¬ 
tions is continuous, they might be discontinuous at to 
and ftr gt . 


Other Representations. Exponential Trajectories and 
Fourier Series Expansions [8.72] are particularly suited 
to minimize natural vibrations on robot mechanisms in¬ 
troduced by reference trajectories. 


Trajectories and Paths 

To draw the connection to Chap. 7 and Parts D and E, 
trajectories and paths are often tightly coupled. 


Trajectories Along Predefined Paths. A path in joint 
space can be described by a function q(s(t)) with s e 
[fo, ftrgt], where the start configuration of the path is q (fo) 
and the target configuration is q(t lT gt ). To move a robot 
along the path, an appropriate function s(t) needs to be 
computed that does not violate any of the kinematic 
and dynamic constraints [8.73-75]. If a path is given 
in operational space, Jc(s(r)) can be mapped to q(t) 
(Sect. 8.2). 


Multi-Dimensional Trajectories. Instead of using 
a one-dimensional function s(t) to parameterize a path 
segment, trajectories can also be described by individ¬ 
ual functions for each DOF i to represent q(s(t)) or 
x(s(t)), respectively. To connect two arbitrary states, the 
signals for each individual degree of freedom need to be 
time-synchronized [8.70], so that all DOFs reach their 
target state of motion at the very same instant. Those 
trajectories may also be phase-synchronized [8.76], so 
that the trajectories of all DOFs are derived from a one 
master DOF and only scaled by a factor to achieve 
homothety [8.77]. The two trajectories in Fig. 8.8 are 
time-synchronized but not phase-synchronized. 

Multi-Point Trajectories. If instead of an entirely de¬ 
fined geometric path or a motion to a single way point, 
an entire series of geometric way points is given, the 
trajectory that connects all way points in a given state 
space needs to be computed. Trajectory segments be¬ 
tween two way points can be represented with any of 


the above mentioned representations as long as the po¬ 
sition signal and its derivatives up an appropriate order 
are continuous (at least C 1 continuous). Splines, B- 
splines, or Bezier splines are used to generate either 
a reference trajectory or a geometric path, which then 
can be parameterized with a function s(t). 

8.9.4 Trajectory Planning Algorithms 

The following part provides an overview of online and 
offline trajectory planning concepts. 

Constraints 

Constraints for trajectory planners can manifold: 

• Kinematic : maximum velocities, accelerations, 
jerks, etc. and workspace space limits 

• Dynamic: maximum joint or actuator forces and/or 
torques 

• Geometric: no collisions with static and dynamic 
objects in the workspace 

• Temporal, reaching a state within a given time in¬ 
terval or at a given time. 


These and other constraints may have to be taken 
into account at the same time. Depending on the robot 
and the task, additional optimization criteria may be 
considered (e.g., time-optimality, minimum-jerk, max¬ 
imum distance to workspace boundaries, minimum 
energy). 

Offline Trajectory Planning 
Kahn and Roth [8.78] showed results using optimal, 
linear control theory to achieve a near-time-optimal 
solution for linearized manipulators. The resulting tra¬ 
jectories are jerk-limited and lead to smaller trajectory¬ 
following errors and to less excitation of structural 
natural frequencies in the system. 

The work of Brady [8.79] introduced several 
techniques of trajectory planning in joint space and 
Paul [8.80] and Taylor [8.81] published works about the 
planning of trajectories in Cartesian space in parallel to 
Brady. Lin et al. [8.82] published another purely kine¬ 
matic approach as did Castain and Paul [8.69]. 

Hollerbach [8.83] first introduced the consideration 
of the nonlinear inverse robot dynamics for the genera¬ 
tion of manipulator trajectories. 

During the middle of the 1980s, three groups de¬ 
veloped techniques for time-optimal trajectory planning 
for arbitrarily specified paths: Bobrow [8.73], Shin and 
McKay [8.74], and Pfeiffer and Johanni [8.75]. Tra¬ 
jectories are composed of three curves: the maximum 
acceleration curve, the maximum velocity curve, and 
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the maximum deceleration curve. The proposed algo¬ 
rithms find the intersection points of these three curves. 

These algorithms have become the fundament for 
many follow-up works: Kyriakopoulos and Sridis [8.84] 
added minimum-jerk criteria; Slotine and Yang aban¬ 
doned the computationally expensive calculation of 
the maximum velocity curve [8.85]; Shiller and 
Lu added methods for handling dynamic singulari¬ 
ties [8.86]; Fiorini and Shiller extended the algorithm 
for known dynamic environments with moving obsta¬ 
cles [8.87], 

Online Trajectory Planning 
An online modification of a planned trajectory may 
have several reasons: (i) The trajectory becomes 
adapted in order to improve the accuracy with a path 
specified beforehand; (ii) The robotic system reacts on 
sensor signals and/or events that cannot be predicted be¬ 
forehand, because the robot acts in a (partly) unknown 
and dynamic environment. 

Improving Path Accuracy. All previously described 
off-line trajectory planning methods assume a dynamic 
model that describes the behavior of the real robot ex¬ 
actly. In practice, this is often not the case, and some 
robot parameters are only estimated, some dynamic ef¬ 
fects remain unmodeled, and system parameters may 
change during operation. If this is the case, the result¬ 
ing robot motion is not time-optimal anymore and/or 
the maximum actuator forces and/or torques are ex¬ 
ceeded, which leads to an undesired difference between 
the specified and the executed path. 

Dahl and Nielsen [8.88] extended [8.73-75] by 
adapting the acceleration along the path, so that 
the underlying trajectory-following controller becomes 
adapted depending on the current state of motion. The 


8.10 Digital Implementation 

Most controllers introduced in the previous sections are 
digitally implemented on microprocessors. In this sec¬ 
tion basic but essential practical issues related to their 
computer implementation are discussed. When the con¬ 
troller is implemented on a computer control system, 
the analog inputs are read and the outputs are set with 
a certain sampling period. This is a drawback compared 
to analog implementations, since sampling introduces 
time delays into the control loop. Figure 8.9 shows the 
overall block diagram of control system with a boxed 
digital implementation part. When a digital computer is 
used to implement a control law, it is convenient to di¬ 


approaches of Cao et al. [8.89,90] use cubic splines 
to generate smooth paths in joint space with time- 
optimal trajectories. Constantinescu and Croft [8.91] 
suggest a further improvement to the approach of [8.86] 
with objective to limit the derivative of actuator 
forces/torques. Macfarlane and Croft extended this ap¬ 
proach further by adding jerk limitations to quintic 
splines (Fig. 8.8) [8.92]. 

Sensor-Based Trajectory Adaptation 
The last paragraph presented an overview of online 
trajectory generation methods for improving the path 
accuracy, while this one focuses on the online consid¬ 
eration of sensor signals, for instance, for the purpose 
of collision avoidance 

or switching between controllers or control gains 
(Ids &m'll.H.mi |43>MZEH*E31). 

In 1988, Andersson presented an online trajectory 
planning for a Ping-Pong-playing PUMA 260 manip¬ 
ulator that computes parameterized quintic polynomi¬ 
als [8.93, 94]. Based on [8.73-75], Lloyd and Hayward 
proposed a technique to transition between two differ¬ 
ent trajectory segments [8.95] using a transition win¬ 
dow [8.81]. Aim et al. introduced a method to connect 
two arbitrary motion states online, which does not take 
into account kinematic or dynamic constraints [8.96]. 
Broquere et al., Haschke et al., and Kroger extended 
this approach for multi-dimensional trajectories, so that 
kinematic constraints are taken into account [8.70,97, 
98], 

Further Reading 

Overviews of the domain of robot reference trajectory 
generation can found in the textbooks of Biagiotti and 
Melchiorri [8.72], Craig [8.99], Fu et al. [8.100], and 
Spong et al. [8.101]. 


vide coding sequence in the interrupt routine into four 
process routines, as shown in Fig. 8.10. Reading the 
input signal from sensors and writing the control sig¬ 
nal to digital-to-analog (D/A) converters synchronized 
at the correct frequency is very important. Therefore, 
these processes are located in the first routine. After 
saving counter values and extracting D/A values, which 
are already calculated one step before, the next routine 
produces reference values. Control routines with filters 
follow and produce scalar or vector control outputs. Fi¬ 
nally, the user interface for checking parameter values 
is made and will be used for tuning and debugging. 
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Fig. 8.9 Digital implementation of system control 
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ControMnterrupt () 

{ 

Read_Write(); 
y Reference_Generate(); 
Output_process(); 
Output_Debug(); 

} 




Fig. 8.10 The sequence in the interrupt routine for digital 
control 


using tools in the .y-domain. In order to realize those re¬ 
sults in program code, understanding the Z-transform 
is essential. All controllers and filters designed in the 
.y-domain can be easily translated to a program code 
through a Z-transform because it has the form of dig¬ 
itized difference sequences. 

A PID controller is used as an example. In transfer 
function form, this controller has the basic structure 

Y(s) Ki 

— y — = K P -\ - \-sKy. (8.126) 

E(s ) s 

There are several methods for transformation from the 
frequency domain to the discrete domain. For stability 
conservation, backward Euler and Tustin algorithms are 
often used. Though the Tustin algorithm is known as 
the more exact one, the backward Euler algorithm is 
utilized in the following procedure. 

After substituting the backward Euler equation 
into (8.126), 

l-z _1 

s = -, 

T 

the following discrete form is produced 


8.10.1 Z-Transform for Motion Control 
Implementation 


T(z) _ a + fiz 1 + yz 2 
E(z)~ m-z- 1 ) 


(8.127) 


Continuous-time systems are transformed into discrete¬ 
time systems by using the Z-transform. A discrete¬ 
time system is used to obtain a mathematical model 
that gives the behavior of a physical process at the 
sampling points, although the physical process is still 
a continuous-time system. A Laplace transform is used 
for the analysis of control system in the .y-domain. In 
most cases, the design of controllers and filters are done 


where 

a = K{T 2 + KpT + K v , 
ft = -KpT-2Ky , 

Y =Ky. 

Sometimes a differentiator .y in the PID controller 
makes the implementation infeasible when the mea- 
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surement noise is severe. One can remedy the con¬ 
troller (8.126) by attaching a lowpass filter with filter 
time constant a 


—^ = .Kp + — -1-—— 

E(s) s as + 1 


Ky . 


(8.128) 


Again, substituting the backward Euler equation 
into (8.128) produces 


7(z) _ a + jiz 1 + yz 2 
E(z) — 


(8.129) 


where 


or — K P + K\ T + 


Ky 

a + T' 


P 


(2a + T)Kp + aTKi + 2 K v 
a + T 


Y = 


a Kp + Ky 
a + T 


8 = 
l/r = 


2o- + r 

a + T ’ 
a 

~ a + T ’ 


in which the filter time constant a is determined from 
a cutoff frequency f c [Hz] for removing noises such as 


8.10.2 Digital Control for Coding 


Inverse Z-transform produces a difference equation for 
digital control. Furthermore the difference equation can 
be directly converted into the control program code. 
Since the inverse Z-transform of Y(z) is y k and z —1 

implies the previous sample time, z —1 7(z) = _v*_ 1 and 

z~ 2 Y(z) =y k - 2 - 

Now, the PID controller expressed by (8.127) is re¬ 
arranged using the difference equation 

T(yk~yk-i) = ote k + Pe k -i + ye k -2 , 

Yk ~}’k -1 = 7 p(uek + fie k -\ + yek-2) ■ 

(8.130) 

For pratical use, the PID controller can be directly 
coded in the program as follows 

Yk = K P _ c e k + Ky, c e k + Ki :C e[ , 

where 


— Pk, desired Pk •> 

= Vk, desired Vk 

= {p k, desired Pk —1,desired) (Pk Pk —l) 


— e k ~ e k— 1 , 

k 

4 = 4-1 +e k = J2 e k’ 

i= 0 

in which p k is the present position, v k is the present ve¬ 
locity, desired means reference to be followed, and c 
means coded form for digitial control. Now let us ob¬ 
tain the difference between the present control output 
and the previous one 


Yk-Yk -1 

= [Kp, c ek~ K PjC ek—i] 

+ [Ky , c (ek — ek-\) — Ky c (ek— \ —e k - 2 )] 

+ [K hc e[-K Uc e i k _ l ] 

= (Kp, c + Ky c + K lc )e k 

- (Kp, c + 2Ky c )ek—\ + Ky tC ek-2 ■ 

(8.131) 

Comparing the parameters in (8.130) and (8.131), 
one obtains 

j. = Kp + y + K i T = ( k p.c + Ky, c + Kp c ) , 

B 2 Ky „ 

j = ~Kp - — = —(Kp, c + 2A'v.c) . 



which shows that there is a relation between the de¬ 
signed and coded forms of the gains 


K P:C = K P , 



K hc = K\T . (8.132) 

As the sampling frequency is increased in the same sys¬ 
tem, the coded Ky gain should be increased and the 
coded K\ gain should be decreased. Using this method, 
the designed controller can be coded in a digital signal 
processor (DSP) or microprocessor. However, sufficient 
analysis and simulation for control algorithms should 
be performed beforehand to obtain successful control 
system performance. 

In addition, the PID controller with lowpass fil¬ 
ter (8.129) can be implemented as 

Yk ~ 8y k -i - fyk -2 = ae k + Pe k -i + ye k - 2 , 

Yk = 8y k -i + fy k -i + ae k + Pe k -i + ye k -2 . 

(8.133) 

Using the same procedures, one can arrive at the similar 
control program code for digital control. 
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PID Control Experiment (Multimedia) control system are shown in the multimedia source to 

According as the gains change, the performance vari- help readers’ understanding l<s>M3E[E^®l. 
ations of PID controller implemented in the digital 


8.11 Learning Control 

Since many robotic applications, such as pick-and- 
place operations, paintings, and circuit-board assembly, 
involve repetitive motions, it is natural to consider 
the use of data gathered in previous cycles to try to 
improve the performance of the manipulator in subse¬ 
quent cycles. This is the basic idea of repetitive control 
or learning control. Consider the robot model given 
in Sect. 8.1 and suppose that one is given a desired 
joint trajectory q d (t ) on a finite time interval 0 < t < T. 
The reference trajectory q d is used in repeated trails of 
the manipulator, assuming either that the trajectory is 
periodic, q d (T) = </<i(0), (repetitive control) or that the 
robot is reinitialized to lie on the desired trajectory at 
the beginning of each trail (learning control). Hereafter, 
we use the term learning control to mean either repeti¬ 
tive or learning control. 

8.11.1 Pure P-Type Learning Control 

Let T/c be the input torque during the A-th cycle, which 
produces an output q k (t), 0 < t < 7b n d. Now, let us con¬ 
sider the following set of assumptions: 

• Assumption 1: Every trial ends at a fixed time of 
duration 7b n d > 0. 

• Assumption 2: Repetition of the initial setting is sat¬ 
isfied. 

• Assumption 3: Invariance of the system dynamics is 
ensured throughout these repeated trails. 

• Assumption 4: Every output q k can be measured and 
thus the error signal A q k = q k — q,\ can be utilized 
in the construction of the next input t*+ i- 

• Assumption 5: The dynamics of the robot manipu¬ 
lators is invertible. 

The learning control problem is to determine a re¬ 
cursive learning law L 

T k+ i=L[r k (t),Aq k (t)], 0 < t < T bnd , (8.134) 

where A q k (t) = q k (t) — q d {t), such that || A^.|| —> 0 as 
k —> 00 in some suitably defined function norm, || ■ ||. 
The initial control input can be any control input that 
produces a stable output, such as PD control. Such 
learning control schemes are attractive because accurate 
models of the dynamics need not be known a priori. 

Several approaches have been used to generate 
a suitable learning law L and to prove convergence of 


the output error. A pure P-type learning law is one of 
the form 

n+lW = r k (t)~ QAq k (t) , (8.135) 

and is given this name because the correction term for 
the input torque at each iteration is proportional to the 
error A q k . Now let tj be defined by the computed- 
torque control, i. e., 


r d (t) 


H[?d(0]<?dM + C[q d (t),q d (t)]q d (t) 

+ Tg[«d(0] ■ 

(8.136) 


One should recall that the function T k actually does not 
need to be computed; it is sufficient to know that it ex¬ 
ists. Considering the P-type learning control law, we 
have 


AT/t +1 (f) = At k (t) - $A q k (t) , (8.137) 

where At k (t) = t k (t) — t j(t), so that 

II At*. + 1 (0|| 2 < ||AT t (0|| 2 -/8||$A 9i (f)|| 2 (8.138) 
provided there exist positive constant A and /l such that 


Tbnd 

J e~ Xt Aq k T AT k {t)dt> A^.(t)|| 2 

0 

(8.139) 

for all k. It then follows from the inequality above that 
A q k —y 0 in the norm sense as k —> 00 . Detailed stabil¬ 
ity analysis of this control scheme is given in [8.102, 
103], 

8.11.2 P-Type Learning Control 
with a Forgetting Factor 

Although pure P-type learning control achieves the de¬ 
sired goal, several strict assumptions may be not valid 
in actual implementations, for example, there may be an 
initial setting error. Furthermore, there may be small but 
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nonrepeatable fluctuations of dynamics. Finally, there 
may exit a (bounded) measurement noise ^ such that 

A q k (t) + $*(f) = [q k (t) + $*(*)] - q d (t ) . (8.140) 

Thus the learning control scheme may fail. In order to 
enhance the robustness of P-type learning control, a for¬ 
getting factor is introduced in the form 

rjt+i (0 = (1 — a)T k (t) + oiTo(t) 

-$[A + $*(*)]• (8.141) 

The original idea of using a forgetting factor in learning 
control originated with [8.104]. 

It has been rigorously proven that P-type learning 
control with a forgetting factor guarantees convergence 
to a neighborhood of the desired one of size 0(a). 
Moreover, if the content of a long-term memory is re¬ 
freshed after every k trials, where k is of 0(1 /a), then 
the trajectories converge to an e-neighborhood of the 


desired control goal. The size of e is dependent on the 
magnitude of the initial setting error, the nonrepeat¬ 
able fluctuations of the dynamics, and the measurement 
noise. For a detailed stability investigation, please refer 
to [8.105,106]. 

8.11.3 Summary 

By applying learning control, the performance of repeti¬ 
tive tasks (such as painting or pick-and-place operation) 
is improved by utilizing data gathered in the previous 
cycles. In this section, two learning control schemes 
were introduced. First, pure P-type learning control and 
its robustness problem were described. Then P-type 
learning control with a forgetting factor was presented, 
enhancing the robustness of learning control. 

Further Reading 

Rigorous and refined exploration of learning control is 
first discussed independently in [8.2, 12]. 
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Gain change of the PID controller 

available from http://handbookofrobotics.org/view-chapter/08/videodetails/25 
Safe human-robot cooperation 

available from http://handbookofrobotics.org/view-chapter/08/videodetails/757 

Virtual whiskers - Highly responsive robot collision avoidance 

available from http://handbookofrobotics.org/view-chapter/08/videodetails/758 

JediBot - Experiments in human-robot sword-fighting 

available from http://handbookofrobotics.org/view-chapter/08/videodetails/759 

Different jerk limits of robot arm trajectories 

available from http://handbookofrobotics.org/view-chapter/08/videodetails/760 
Sensor-based online trajectory generation 

available from http://handbookofrobotics.org/view-chapter/08/videodetails/761 
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9. Force Control 


Luigi Villani, Joris De Schutter 


A fundamental requirement for the success of 
a manipulation task is the capability to handle 
the physical contact between a robot and the en¬ 
vironment. Pure motion control turns out to be 
inadequate because the unavoidable modeling 
errors and uncertainties may cause a rise of the 
contact force, ultimately leading to an unstable 
behavior during the interaction, especially in the 
presence of rigid environments. Force feedback 
and force control becomes mandatory to achieve 
a robust and versatile behavior of a robotic sys¬ 
tem in poorly structured environments as well as 
safe and dependable operation in the presence of 
humans. This chapter starts from the analysis of 
indirect force control strategies, conceived to keep 
the contact forces limited by ensuring a suitable 
compliant behavior to the end effector, without 
requiring an accurate model of the environment. 
Then the problem of interaction tasks modeling 
is analyzed, considering both the case of a rigid 
environment and the case of a compliant environ¬ 
ment. For the specification of an interaction task, 
natural constraints set by the task geometry and 
artificial constraints set by the control strategy are 
established, with respect to suitable task frames. 
This formulation is the essential premise to the 
synthesis of hybrid force/motion control schemes. 
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9.1 Background 


Research on robot force control has flourished in the 
past three decades. Such a wide interest is motivated 
by the general desire of providing robotic systems 
with enhanced sensory capabilities. Robots using force, 
touch, distance, and visual feedback are expected to au¬ 
tonomously operate in unstructured environments other 
than the typical industrial shop floor. 


Since the early work on telemanipulation, the use 
of force feedback was conceived to assist the human 
operator in the remote handling of objects with a slave 
manipulator. More recently, cooperative robot systems 
have been developed where two or more manipula¬ 
tors (viz. the fingers of a dexterous robot hand) are 
to be controlled so as to limit the exchanged forces 
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and avoid squeezing of a commonly held object. Force 
control plays a fundamental role also in the achieve¬ 
ment of robust and versatile behavior of robotic systems 
in open-ended environments, providing intelligent re¬ 
sponse in unforeseen situations and enhancing human- 
robot interaction. 

9.1.1 From Motion Control 
to Interaction Control 

Control of the physical interaction between a robot 
manipulator and the environment is crucial for the suc¬ 
cessful execution of a number of practical tasks where 
the robot end-effector has to manipulate an object or 
perform some operation on a surface. Typical exam¬ 
ples in industrial settings include polishing, deburring, 
machining or assembly. A complete classification of 
possible robot tasks, considering also nonindustrial ap¬ 
plications, is practically infeasible in view of the large 
variety of cases that may occur, nor would such a clas¬ 
sification be really useful to find a general strategy to 
control the interaction with the environment. 

During contact, the environment may set constraints 
on the geometric paths that can be followed by the end- 
effector, denoted as kinematic constraints. This situa¬ 
tion, corresponding to the contact with a stiff surface, 
is generally referred to as constrained motion. In other 
cases, the contact task is characterized by a dynamic 
interaction between the robot and the environment that 
can be inertial (as in pushing a block), dissipative (as 
in sliding on a surface with friction) or elastic (as in 
pushing against an elastically compliant wall). In all 
these cases, the use of a pure motion control strategy for 
controlling interaction is prone to failure, as explained 
below. 

Successful execution of an interaction task with the 
environment by using motion control could be obtained 
only if the task were accurately planned. This would in 
turn require an accurate model of both the robot manip¬ 
ulator (kinematics and dynamics) and the environment 
(geometry and mechanical features). A manipulator 
model may be known with sufficient precision, but 
a detailed description of the environment is difficult to 
obtain. 

To understand the importance of task planning accu¬ 
racy, it is sufficient to observe that in order to perform 
a mechanical part mating with a positional approach the 
relative positioning of the parts should be guaranteed 
with an accuracy of an order of magnitude greater than 
part mechanical tolerance. Once the absolute position 
of one part is exactly known, the manipulator should 
guide the motion of the other with the same accuracy. 

In practice, the planning errors may give rise to 
a contact force and moment, causing a deviation of the 


end-effector from the desired trajectory. On the other 
hand, the control system reacts to reduce such devia¬ 
tions. This ultimately leads to a build-up of the contact 
force until saturation of the joint actuators is reached or 
breakage of the parts in contact occurs. 

The higher the environment stiffness and position 
control accuracy are, the more easily a situation like 
the one just described can occur. This drawback can be 
overcome if a compliant behavior is ensured during the 
interaction. This compliant behavior can be achieved ei¬ 
ther in a passive or in an active fashion. 

Passive Interaction Control 
In passive interaction control the trajectory of the robot 
end-effector is modified by the interaction forces due to 
the inherent compliance of the robot. The compliance 
may be due to the structural compliance of the links, 
joints, and end-effector, or to the compliance of the po¬ 
sition servo. Soft robot arms with elastic joints or links 
are purposely designed for intrinsically safe interaction 
with humans. In industrial applications, a mechanical 
device with passive compliance, known as the remote 
center of compliance (RCC) device [9.1], is widely 
adopted. An RCC is a compliant end-effector mounted 
on a rigid robot, designed and optimized for peg-into- 
hole assembly operations. 

The passive approach to interaction control is 
very simple and cheap, because it does not require 
force/torque sensors; also, the preprogrammed trajec¬ 
tory of the end-effector must not be changed at ex¬ 
ecution time; moreover, the response of a passive 
compliance mechanism is much faster than active repo¬ 
sitioning by a computer control algorithm. However, 
the use of passive compliance in industrial applications 
lacks flexibility, since for every robotic task a special- 
purpose compliant end-effector has to be designed and 
mounted. Also, it can only deal with small position and 
orientation deviations of the programmed trajectory. Fi¬ 
nally, since no forces are measured, it can not guarantee 
that high contact forces will never occur. 

Active Interaction Control 

In active interaction control, the compliance of the 
robotic system is mainly ensured by a purposely de¬ 
signed control system. This approach usually requires 
the measurement of the contact force and moment, 
which are fed back to the controller and used to mod¬ 
ify or even generate online the desired trajectory of the 
robot end-effector. 

Active interaction control may overcome the afore¬ 
mentioned disadvantages of passive interaction control, 
but it is usually slower, more expensive, and more so¬ 
phisticated. To obtain a reasonable task execution speed 
and disturbance rejection capability, active interaction 
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control has to be used in combination with some degree 
of passive compliance [9.2]: feedback, by definition, 
always comes after a motion and force error has oc¬ 
curred, hence some passive compliance is needed in 
order to keep the reaction forces below an acceptable 
threshold. 

Force Measurements 

For a general force-controlled task, six force compo¬ 
nents are required to provide complete contact force 
information: three translational force components and 
three torques. Often, a force/torque sensor is mounted 
at the robot wrist [9.3], but other possibilities exist, for 
example, force sensors can be placed on the fingertips 
of robotic hands [9.4]; also, external forces and mo¬ 
ments can be estimated via shaft torque measurements 
of joint torque sensors [9.5,6]. However, the majority 
of the applications of force control (including indus¬ 
trial applications) is concerned with wrist force/torque 
sensors. In this case, the weight and inertia of the tool 
mounted between the sensor and the environment (i. e., 
the robot end-effector) is assumed to be negligible or 
suitably compensated from the force/torque measure¬ 
ments. The force signals may be obtained using strain 
measurements, which results in a stiff sensor, or de¬ 
formation measurements (e.g., optically), resulting in 
a compliant sensor. The latter approach has an advan¬ 
tage if additional passive compliance is desired. 

9.1.2 From Indirect Force Control 
to Hybrid Force/Motion Control 

Active interaction control strategies can be grouped into 
two categories: those performing indirect force control 
and those performing direct force control. The main dif¬ 
ference between the two categories is that the former 
achieve force control via motion control, without ex¬ 
plicit closure of a force feedback loop; the latter instead 
offer the possibility of controlling the contact force and 
moment to a desired value, thanks to the closure of 
a force feedback loop. 

To the first category belongs impedance control (or 
admittance control) [9.7, 8], where the deviation of the 
end-effector motion from the desired motion due to the 
interaction with the environment is related to the con¬ 
tact force through a mechanical impedance/admittance 
with adjustable parameters. A robot manipulator under 
impedance (or admittance) control is described by an 
equivalent mass-spring-damper system with adjustable 
parameters. This relationship is an impedance if the 
robot control reacts to the motion deviation by gener¬ 
ating forces, while it corresponds to an admittance if 
the robot control reacts to interaction forces by impos¬ 
ing a deviation from the desired motion. Special cases 


of impedance and admittance control are stiffness con¬ 
trol and compliance control [9.9], respectively, where 
only the static relationship between the end-effector po¬ 
sition and orientation deviation from the desired motion 
and the contact force and moment is considered. Notice 
that, in the robot control literature, the terms impedance 
control and admittance control are often used to refer to 
the same control scheme; the same happens for stiffness 
and compliance control. Moreover, if only the relation¬ 
ship between the contact force and moment and the end- 
effector linear and angular velocity is of interest, the 
corresponding control scheme is referred to as damping 
control [9.10]. 

Indirect force control schemes do not require, in 
principle, measurements of contact forces and mo¬ 
ments; the resulting impedance or admittance is typi¬ 
cally nonlinear and coupled. However, if a force/torque 
sensor is available, then force measurements can be 
used in the control scheme to achieve a linear and de¬ 
coupled behavior. 

Differently from indirect force control, direct force 
control requires an explicit model of the interaction 
task. In fact, the user has to specify the desired motion 
and the desired contact force and moment in a con¬ 
sistent way with respect to the constraints imposed 
by the environment. A widely adopted strategy be¬ 
longing to this category is hybrid force/motion control, 
which aims at controlling the motion along the uncon¬ 
strained task directions and force (and moment) along 
the constrained task directions. The starting point is the 
observation that, for many robotic tasks, it is possible to 
introduce an orthogonal reference frame, known as the 
compliance frame [9.11] (or task frame [9.12]) which 
allows one to specify the task in terms of natural and 
artificial constrains acting along and about the three 
orthogonal axes of this frame. Based on this decompo¬ 
sition, hybrid force/motion control allows simultaneous 
control of both the contact force and the end-effector 
motion in two mutually independent subspaces. Simple 
selection matrices acting on both the desired and feed¬ 
back quantities serve this purpose for planar contact 
surfaces [9.13], whereas suitable projection matrices 
must be used for general contact tasks, which can also 
be derived from the explicit constraint equations [9. 14- 
lb]. Several implementation of hybrid motion control 
schemes are available, e.g., based on inverse dynam¬ 
ics control in the operational space [9.17], passivity- 
based control [9.18], or outer force control loops closed 
around inner motion loops, typically available in indus¬ 
trial robots [9.2], 

If an accurate model of the environment is not 
available, the force control action and the motion con¬ 
trol action can be superimposed, resulting in a parallel 
force/position control scheme. In this approach, the 


Part A | 9.1 



Part A | 9.2 


198 Part A I Robotics Foundations 


force controller is designed so as to dominate the 
motion controller; hence, a position error would be tol- 

9.2 Indirect Force Control 

To gain insight into the problems arising at the inter¬ 
action between the end-effector of a robot manipulator 
and the environment, it is worth analyzing the effects of 
a motion control strategy in the presence of a contact 
force and moment. To this aim, assume that a ref¬ 
erence frame S e is attached to the end-effector, and 
let p e denote the position vector of the origin and R e 
the rotation matrix with respect to a fixed base frame. 
The end-effector velocity is denoted by the 6x1 twist 
vector v e = (pj wJ) T where p c is the translational ve¬ 
locity and a> e the angular velocity, and can be computed 
from the n x 1 joint velocity vector q using the linear 
mapping 

v e = i(q)q. (9.1) 

The matrix J is the 6xn end-effector geometric Ja¬ 
cobian. For simplicity, the case of nonredundant non¬ 
singular manipulators is considered; therefore, n = 6 
and the Jacobian is a square nonsingular matrix. The 
force f e and moment m e applied by the end-effector 
to the environment are the components of the wrench 
h t = (f e ml)\ 

It is useful to consider the operational space formu¬ 
lation of the dynamic model of a rigid robot manipula¬ 
tor in contact with the environment 

A(q)v s . + T(q,q)v e + q(q) = h c -h e , (9.2) 

where 

A( 9 ) = (JH( 9 )-‘J t )- 1 

is the 6x6 operational space inertia matrix, 

T(q.q) = J" T C( q , q)i~' - A ( q) jj" 1 

is the wrench including centrifugal and Coriolis effects, 
and )/( q) = J~ r g(q) is the wrench of the gravitational 
effects; H( q), C( q, q) and g( q) are the corresponding 
quantities defined in the joint space. The vector h c = 
J^ t t is the equivalent end-effector wrench correspond¬ 
ing to the input joint torques r. 

9.2.1 Stiffness Control 

In the classical operational space formulation, the end- 
effector position and orientation is described by a 


erated along the constrained task directions in order to 
ensure force regulation [9.19]. 


6 x 1 vector jc e = (pj <pJ) T , where <p e is a set of Eu¬ 
ler angles extracted from R . Hence, a desired end- 
effector position and orientation can be assigned in 
terms of a vector x d , corresponding to the position of 
the origin p,\ and the rotation matrix Rj of a desired 
frame The end-effector error can be denoted as 
Axde = x d —x e , and the corresponding velocity error, 
assuming a constant x d , can be expressed as Aide = 
-X e = —A _1 ( <p e )V e , With 

A( <? e > = ( 0 T(? e )) ’ 

where I is the 3x3 identity matrix, 0 is a 3 x 3 null 
matrix, and T is the 3x3 matrix of the mapping 
00 e = T( <p e )<p e , depending on the particular choice of 
the Euler angles. 

Consider the motion control law 

he = A^ T ( <Pe)Kp Ax de - K D t> e + 1 i(q) , (9.3) 

corresponding to a simple proportional-derivative 
(PD) + gravity compensation control in the operational 
space, where K P and K D are symmetric and positive- 
definite 6x6 matrices. 

In the absence of interaction with the environment 
(i. e., when h c = 0), the equilibrium v e = 0, Ax de = 0 
for the closed-loop system, corresponding to the de¬ 
sired position and orientation for the end-effector, is 
asymptotically stable. The stability proof is based on 
the positive-definite Lyapunov function 

1 T 1 

V= -v e A(q)v e + -Ax de KpAx de , 

whose time derivative along the trajectories of the 
closed-loop system is the negative semidehnite function 

V = -vjK D v e . (9.4) 

In the presence of a constant wrench h e , using a similar 
Lyapunov argument, a different asymptotically stable 
equilibrium can be found, with a nonnull Ax de - The new 
equilibrium is the solution of the equation 

A~ T ( (p e )KpAXde - he = 0 , 
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which can be written in the form 

A* de = Kp 1 A t ( <p e )h e , (9.5) 

or, equivalently, as 

h e = A^ t ( <p e )KpAx de . (9.6) 

Equation (9.6) shows that in the steady state the end- 
effector, under a proportional control action on the 
position and orientation error, behaves as a six-degree- 
of-freedom (DOF) spring in respect of the external 
force and moment h e . Thus, the matrix Kp plays the 
role of an active stiffness , meaning that it is possible 
to act on the elements of Kp so as to ensure a suit¬ 
able elastic behavior of the end-effector during the 
interaction. Analogously, (9.5) represents a compliance 
relationship, where the matrix KJT 1 plays the role of 
an active compliance. This approach, consisting of as¬ 
signing a desired position and orientation and a suitable 
static relationship between the deviation of the end-ef¬ 
fector position and orientation from the desired motion 
and the force exerted on the environment, is known as 
stiffness control. 

The selection of the stiffness/compliance parame¬ 
ters is not easy, and strongly depends on the task to be 
executed. A higher value of the active stiffness means 
a higher accuracy of the position control at the expense 
of higher interaction forces. Hence, if it is expected to 
meet some physical constraint in a particular direction, 
the end-effector stiffness in that direction should be 
made low to ensure low interaction forces. Conversely, 
along the directions where physical constraints are not 
expected, the end-effector stiffness should be made high 
so as to follow closely the desired position. This allows 
discrepancies between the desired and achievable posi¬ 
tions due to the constraints imposed by the environment 
to be resolved without excessive contact forces and mo¬ 
ments. 

It must be pointed out, however, that a selective 
stiffness behavior along different directions cannot be 
effectively assigned in practice on the basis of (9.6). 
This can easily be understood by using the classi¬ 
cal definition of a mechanical stiffness for two bodies 
connected by a 6-DOF spring, in terms of the linear 
mapping between the infinitesimal twist displacement 
of the two bodies at an unloaded equilibrium and the 
elastic wrench. 

In the case of the active stiffness, the two bod¬ 
ies are, respectively, the end-effector, with the attached 
frame E e , and a virtual body, attached to the desired 
frame E d . Hence, from (9.6), the following mapping 
can be derived 

h e = A~ T ( <p e )KpA -1 ( <p e )8.*: de , (9.7) 


in the case of an infinitesimal twist displacement &c de 
defined as 



where A p dt = p d —p e is the time derivative of the posi¬ 
tion error A p dt = p d —p e and A« de = <w d — w e is the 
angular velocity error. Equation (9.7) shows that the 
actual stiffness matrix is A -T (^> e )KpA — 1 (^> e ), which 
depends on the end-effector orientation through the vec¬ 
tor (p e , so that, in practice, the selection of the stiffness 
parameters is quite difficult. 

This problem can be overcome by defining a ge¬ 
ometrically consistent active stiffness, with the same 
structure and properties as ideal mechanical springs. 

Mechanical Springs 

Consider two elastically coupled rigid bodies A and B 
and two reference frames E. d and X'b, attached to A 
and B , respectively. Assuming that at equilibrium 
frames E^ and E d coincide, the compliant behavior 
near the equilibrium can be described by the linear map¬ 
ping 

(9-8) 

where /t b is the elastic wrench applied to body B, ex¬ 
pressed in frame B , in the presence of an infinitesimal 
twist displacement 8jc ab of frame I7 a with respect to 
frame E^, expressed in frame B. The elastic wrench 
and the infinitesimal twist displacement in (9.8) can 
also be expressed equivalently in frame E^, since A a 
and Et, coincide at equilibrium. Therefore, /i b = /t b 
and 8x ab = 8x ab ; moreover, for the elastic wrench ap¬ 
plied to body A, hi = K t 8x ba = —/i b being 8x ba = 
—8jc ab . This property of the mapping (9.8) is known as 
port symmetry. 

In (9.8), K is the 6x6 symmetric positive-semidef- 
inite stiffness matrix. The 3x3 matrices K t and K 0 , 
called respectively the translational stiffness and ro¬ 
tational stiffness, are also symmetric. It can be shown 
that, if the 3x3 matrix K c , called the coupling stiffness 
is symmetric, there is maximum decoupling between 
rotation and translation. In this case, the point corre¬ 
sponding to the coinciding origins of the frames E a 
and E\, is called the center of stiffness. Similar defini¬ 
tions and results can be formulated for the case of the 
compliance matrix C = K —1 . In particular, it is possi¬ 
ble to define a center of compliance in the case that the 
off-diagonal blocks of the compliance matrix are sym¬ 
metric. The center of stiffness and compliance do not 
necessarily coincide. 
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There are special cases in which no coupling ex¬ 
ists between translation and rotation, i. e., a relative 
translation of the bodies results in a wrench correspond¬ 
ing to a pure force along an axis through the center 
of stiffness; also, a relative rotation of the bodies re¬ 
sults in a wrench that is equivalent to a pure torque 
about an axis through the centers of stiffness. In these 
cases, the center of stiffness and compliance coincide. 
Mechanical systems with completely decoupled behav¬ 
ior are, e.g., the remote center of compliance (RCC) 
devices. 

Since K t is symmetric, there exists a rotation ma¬ 
trix R ( with respect to the frame If, = X b at equilib¬ 
rium, such that K t = R t T, R, 1 , and T t is a diagonal 
matrix whose diagonal elements are the principal trans¬ 
lational stiffnessess in the directions corresponding to 
the columns of the rotation matrix R t , known as the 
principal axes of translational stiffness. Analogously, 
K 0 can be expressed as K„ = R r o R' . where the 
diagonal elements of r o are the principal rotational 
stiffnesses about the axes corresponding to the columns 
of rotation matrix R 0 , known as the principal axes of 
rotational stiffness. Moreover, assuming that the ori¬ 
gins of X a and X b at equilibrium coincide with the 
center of stiffness, the expression K c = R c IT R r can 
be found, where the diagonal elements of T c are the 
principal coupling stiffnesses along the directions cor¬ 
responding to the columns of the rotation matrix R c , 
known as the principal axes of coupling stiffness. In 
sum, a 6 x 6 stiffness matrix can be specified, with re¬ 
spect to a frame with origin in the center of stiffness, in 
terms of the principal stiffness parameters and principal 
axes. 

Notice that the mechanical stiffness defined by (9.8) 
describes the behavior of an ideal 6-DOF spring which 
stores potential energy. The potential energy function 
of an ideal stiffness depends only on the relative posi¬ 
tion and orientation of the two attached bodies and is 
port symmetric. A physical 6-DOF spring has a pre¬ 
dominant behavior similar to the ideal one, but never¬ 
theless it always has parasitic effects causing energy 
dissipation. 

Geometrically Consistent Active Stiffness 
To achieve a geometrically consistent 6-DOF active 
stiffness, a suitable definition of the proportional con¬ 
trol action in control law (9.3) is required. This control 
action can be interpreted as the elastic wrench applied 
to the end-effector, in the presence of a finite displace¬ 
ment of the desired frame X d with respect to the end- 
effector frame X e . Hence, the properties of the ideal 
mechanical stiffness for small displacements should be 
extended to the case of finite displacements. Moreover, 
to guarantee asymptotic stability in the sense of Lya¬ 


punov, a suitable potential elastic energy function must 
be defined. 

For simplicity, it is assumed that the coupling stiff¬ 
ness matrix is zero. Hence, the potential elastic energy 
can be computed as the sum of a translational potential 
energy and a rotational potential energy. 

The translational potential energy can be defined as 

V) = ^Apj e Kp t Ap de , (9.9) 

with 

Kp ( = -RdK Pt Rj + -R e K Pt Rj . 

where K Pt is a 3 x 3 symmetric positive-definite matrix. 
The use of Kp t in lieu of K Pt in (9.9) guarantees that 
the potential energy is port symmetric also in the case 
of finite displacements. Matrices Kp t and K Pt coincide 
at equilibrium (i. e., when R d = R e ) and in the case of 
isotropic translational stiffness (i. e., when K Pt = Ap t I). 

The computation of the power V t yields 

Vt = A p e Zf' At + A«"i« e At , 

where A p e de is the time derivative of the posi¬ 
tion displacement A p e de = R 1 ( p,\ —p e ), while A« Ae = 
Rj(a> d — The vectors f At and p, c Al are, respec¬ 
tively, the elastic force and moment applied to the end- 
effector in the presence of the finite position displace¬ 
ment A Pfe. These vectors have the following expres¬ 
sions when computed in the base frame 

/A( = Kp,Ap de /n At = Kp t A/7 de , (9.10) 

with 

K" = -S(A/» de )R d K Pt RT , 

where S(-) is the skew-symmetric operator performing 
the vector product. The vector A At = (f \, /n A t ) T * s 
the elastic wrench applied to the end-effector in the 
presence of a finite position displacement Ap de and 
a null orientation displacement. The moment in is 
null in the case of isotropic translational stiffness. 

To define the rotational potential energy, a suitable 
definition of the orientation displacement between the 
frames X d and X e has to be adopted. A possible choice 
is the vector part of the unit quaternion {)? de ,f de } that 
can be extracted from matrix R|j = RjR d . Hence, the 
orientation potential energy has the form 

V 0 = 2e d jK Po e de 


(9.11) 
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where Kp 0 is a 3 x 3 symmetric positive-definite matrix. 
The function V 0 is port symmetric because e = — e^ d . 

The computation of the power V a yields 

Vo = &(of e m e Ao , 
where 

»!Ao = Kp 0 e de , (9.12) 

with 

Kp 0 = 2E T (j/de, fde)ReKp 0 Rj , 

and E(jjde.Cde) = )?del — S( fde)- The above equations 
show that a finite orientation displacement e de = Rj f de 
produces an elastic wrench h Ao = (0 T w^ o ) T which is 
equivalent to a pure moment. 

Hence, the total elastic wrench in the presence 
of a finite position and orientation displacement of 
the desired frame with respect to the end-effector 
frame E e can be defined in the base frame as 

h a = /*At T h Ao . (9.13) 

where h A{ and h a 0 are computed according to (9.10) 
and (9.12), respectively. 

Using (9.13) for the computation of the elastic 
wrench in the case of an infinitesimal twist displace¬ 
ment 8;cjj e near the equilibrium, and discarding the high- 
order infinitesimal terms, yields the linear mapping 

^ = K P 8^ e =( K 0 Pt k °J&4. (9.14) 

Therefore, K P represents the stiffness matrix of an ideal 
spring with respect to a frame E e (coinciding with Xji 
at equilibrium) with the origin at the center of stiff¬ 
ness. Moreover, it can be shown, using definition (9.13), 
that the physical/geometrical meaning of the principal 
stiffnesses and of the principal axes for the matri¬ 
ces K Pt and K Po are preserved also in the case of large 
displacements. 

The above results imply that the active stiffness ma¬ 
trix K P can be set in a geometrically consistent way 
with respect to the task at hand. 

Notice that geometrical consistency can also be en¬ 
sured with different definitions of the orientation error 
in the potential orientation energy (9.11), for example, 
any error based on the angle/axis representation of FT 1 
can be adopted (the unit quaternion belongs to this cat¬ 
egory), or, more generally, homogeneous matrices or 
exponential coordinates (for the case of both position 
and orientation errors). Also, the XYZ Euler angles ex¬ 
tracted from the matrix FT 1 could be used; however, in 


this case, it can be shown that the principal axes of 
rotational stiffness cannot be set arbitrarily but must co¬ 
incide with those of the end-effector frame. 

Stiffness control with a geometrically consistent ac¬ 
tive stiffness can be defined using the control law 

h c = h A ~K D v e +rj(q) , (9.15) 

with h a in (9.13). The asymptotic stability about the 
equilibrium in the case h e = 0 can be proven using the 
Lyapunov function 

V = -vJ\(q)v e + V t + V 0 , 

with Vt and V D given in (9.9) and (9.11), respectively, 
whose time derivative along the trajectories of the 
closed-loop system, in case the frame E& is motion¬ 
less, has the same expression as in (9.4). When h e ^ 
0, a different asymptotically stable equilibrium can 
be found, corresponding to a nonnull displacement of 
the desired frame E ( \ with respect to the end-effector 
frame E e . The new equilibrium is the solution of the 
equation h a = h e . 

Stiffness control allows to keep the interaction force 
and moment limited at the expense of the end-effector 
position and orientation error, with a proper choice of 
the stiffness matrix, without the need of a force/torque 
sensor. However, in the presence of disturbances (e.g., 
joint friction) which can be modeled as an equiva¬ 
lent end-effector wrench, the adoption of low values 
for the active stiffness may produce large deviations 
with respect to the desired end-effector position and 
orientation, also in the absence of interaction with the 
environment. 

9.2.2 Impedance Control 

Stiffness control is designed to achieve a desired static 
behavior of the interaction. In fact, the dynamics of 
the controlled system depends on that of the robot 
manipulator, which is nonlinear and coupled. A more 
demanding objective may be that of achieving a de¬ 
sired dynamic behavior for the end-effector, e.g., that of 
a second-order mechanical system with six degrees of 
freedom, characterized by a given mass, damping, and 
stiffness, known as mechanical impedance. 

The starting point to pursue this goal may be the ac¬ 
celeration-resolved approach used for motion control, 
which is aimed at decoupling and linearizing the non¬ 
linear robot dynamics at the acceleration level via an 
inverse dynamics control law. In the presence of inter¬ 
action with the environment, the control law 

h c = A(q)a + T(q.q)q+ri(q) + ho 


(9.16) 
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cast into the dynamic model (9.2) results in 

i) e = a , (9.17) 

where a is a properly designed control input with the 
meaning of an acceleration referred to the base frame. 
Considering the identity v e = Rji>° + Rju°, with 



the choice 

a = Rja e + Rj^e (9.18) 

gives 

i>l = cc e , (9.19) 

where the control input a e has the meaning of an ac¬ 
celeration referred to the end-effector frame E c . Hence, 
setting 

« e = ^ + KM 1 (K D A4+/i e A -liJ), (9.20) 

the following expression can be found for the closed- 
loop system 

K M A^ e + K D A^ e + A e A =^, (9.21) 

where Km and Kq are 6 x 6 symmetric and positive- 
definite matrices, A v e Ae = v A — v e e , Ai> de = ujj — v\, 
and v e A are, respectively, the acceleration and the ve¬ 
locity of a desired frame E& and h c A is the elastic 
wrench (9.13); all the quantities are referred to the end- 
effector frame E e . 

The above equation describing the dynamic behav¬ 
ior of the controlled end-effector can be interpreted as 
a generalized mechanical impedance. The asymptotic 



Fig. 9.1 Impedance control 


stability of the equilibrium in the case h e = 0 can be 
proven by considering the Lyapunov function 

V = i Av£K m Ai4 + V t + Vo , (9.22) 

where V t and V„ are defined in (9.9) and (9.11), respec¬ 
tively, and whose time derivative along the trajectories 
of system (9.21) is the negative semidehnite function 

V=-A<K d Ai4. 

When h t / 0, a different asymptotically stable equilib¬ 
rium can be found, corresponding to a nonnull displace¬ 
ment of the desired frame E^ with respect to the end- 
effector frame E e . The new equilibrium is the solution 
of the equation A a = h e . 

In case E& is constant, (9.21) has the meaning of 
a true 6-DOF mechanical impedance if Km is chosen as 



where m is a mass and M is a 3 x 3 inertia tensor, and K D 
is chosen as a block-diagonal matrix with 3x3 blocks. 
The physically equivalent system is a body of mass m 
with inertia tensor M with respect to a frame E e at¬ 
tached to the body, subject to an external wrench h L c . 
This body is connected to a virtual body attached to 
frame X d through a 6-DOF ideal spring with stiffness 
matrix K P and is subject to viscous forces and moments 
with damping K D . The function V in (9.22) represents 
the total energy of the body: the sum of the kinetic and 
potential elastic energy. 

A block diagram of the resulting impedance control 
is sketched in Fig. 9.1. The impedance control com¬ 
putes the acceleration input as in (9.18) and (9.20) on 
the basis of the position and orientation feedback as 
well as the force and moment measurements. Then, the 
inverse dynamics control law computes the torques for 
the joint actuators r = J 1 h c with h c in (9.16). This con¬ 
trol scheme, in the absence of interaction, guarantees 
that the end-effector frame E e asymptotically follows 
the desired frame A d . In the presence of contact with the 
environment, a compliant dynamic behavior is imposed 
on the end-effector, according to the impedance (9.21), 
and the contact wrench is bounded at the expense 
of a finite position and orientation displacement be¬ 
tween Ad and E e . Differently from stiffness control, 
a force/torque sensor is required for the measurement 
of the contact force and moment. 

In the case that the force/torque sensor is not avail¬ 
able, the measure of the external wrench h e cannot 
be used in the controller and thus the configuration- 
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independent impedance behaviour (9.21) cannot be 
obtained anymore. However, a desired impedance be¬ 
haviour can be still achieved with the control law 

h c = A(q)v d + T(q,q)v d + K D Av de +h A + q(q) 

(9.23) 

in place of (9.16), where K D is a 6 x 6 positive-definite 
matrix and h A is the elastic wrench (9.13). The result¬ 
ing closed-loop equation is 

A(q)Av dt + (T (q,q) + K D )Ai> de + h A =h e , 

representing an impedance behaviour which preserves 
the actual operational space inertia matrix A (q) of the 
robot. In the above equation, the centrifugal and Cori¬ 
olis wrench T(<jr, q) At; de is required to preserve the 
mechanical properties of a configuration-dependent in¬ 
ertia and to prove the stability similarly to (9.21). In 
case F d is constant, control law (9.23) reduces to the 
stiffness control law (9.15). 

Pioneering experiments on stiffness control and 
impedance control and without without force sensing 
are presented in reports 

experiments on impedance control based on a geomet¬ 
rically consistent active stiffness. 

Implementation Issues 

The selection of good impedance parameters ensuring 
a satisfactory behavior is not an easy task. In fact, the 
dynamics of the closed-loop system is different in free 
space and during interaction. The control objectives are 
different as well, since motion tracking and disturbance 
rejection must be ensured in free space, while, during 
the interaction, the main goal is achieving a suitable 
compliant dynamic behavior for the end-effector. No¬ 
tice also that the dynamics of the controlled system 
during the interaction depends on the dynamics of the 
environment. 

To gain insight into these problems, assume that 
the interaction of the end-effector with the environment 
can be approximated by that derived from an ideal 6- 
DOF spring connecting end-effector frame S e to the 
environment frame Therefore, according to (9.8), 
the elastic wrench exerted by the end-effector on the 
environment, in the presence of an infinitesimal twist 
displacement of with respect to E 0 , can be com¬ 
puted as 


presence of interaction, while the contact wrench is null 
when the end-effector moves in free space. 

The disturbances acting on the robot manipulator 
and the unmodeled dynamics (joint friction, modeling 
errors, etc.) may be taken into account by introducing 
an additive term on the right-hand side of the dynamic 
model of the robot manipulator (9.2), corresponding to 
an equivalent disturbance wrench acting on the end- 
effector. This term produces an additive acceleration 
disturbance y e on the right-hand side of (9.19). There¬ 
fore, using the control law (9.20), the following closed- 
loop impedance equation can be found 

KjyiAi> de + KdA u de + /i^ = h e e + K M y c . (9.25) 

The timing procedure for the impedance parameters 
can be set up starting from the linearized model that 
can be computed from (9.25) in the case of infinitesimal 
displacements, i. e., 

Kjyi8x de + K D 8x e de + (K P + K)&t4 
= K8^ C + K M y e , (9.26) 

where (9.24) and the equality 8xj: 0 = —8jc^ e + 8x do have 
been used. The above equation is valid both for con¬ 
strained (K / 0) and for free motion (K = 0). 

It is evident that suitable dynamics of the position 
and orientation errors can be set by suitably choosing 
the matrix gains Km, K d , and Kp. This task is easier 
under the hypothesis that all the matrices are diagonal, 
resulting in a decoupled behavior for the six compo¬ 
nents of the infinitesimal twist displacement. In this 
case, the transient behavior of each component can be 
set, e.g., by assigning the natural frequency and damp¬ 
ing ratio with the relations 


Wn 


< kp + k 


K 


1 k D 

2 y/k M (k P + k) 


Hence, if the gains are chosen so that a given natural 
frequency and damping ratio are ensured during the in¬ 
teraction (i. e., for k ^ 0), a smaller natural frequency 
with a higher damping ratio will be obtained when the 
end-effector moves in free space (i. e., for k = 0). As for 
the steady-state performance, the end-effector error for 
the generic component is 


8x de 


k 

(kp + k) 


8x do T 


kp + k V 


hi = K8x(. 


(9.24) and the corresponding interaction force is 


where S e and U 0 coincide at equilibrium and K is 
a stiffness matrix. The above model holds only in the 


h 


■ 

kp k 


8xdo 


ky[k 

kp + k y ' 
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Fig. 9.2 Impedance control 
with inner motion control 
loop (admittance control) 


The above relations show that, during interaction, the 
contact force can be made small at the expense of 
a large position error in steady state, as long as the ac¬ 
tive stiffness kp is set low with respect to the stiffness 
of the environment k, and vice versa. However, both the 
contact force and the position error also depend on the 
external disturbance y, in particular, the lower k P , the 
higher the influence of y on both 8jCde and h. Moreover, 
a low active stiffness kp may result in a large position er¬ 
ror also in the absence of interaction (i. e., when k = 0). 

Admittance Control 

A solution to this drawback can be devised by separat¬ 
ing motion control from impedance control as follows. 
The motion control action is purposefully made stiff so 
as to enhance disturbance rejection but, rather than en¬ 
suring tracking of the desired end-effector position and 
orientation, it ensures tracking of a reference position 
and orientation resulting from the impedance control 
action. In other words, the desired position and orien¬ 
tation, together with the measured contact wrench, are 
input to the impedance equation which, via a suitable 
integration, generates the position and orientation to be 
used as a reference for the motion control. 

To implement this solution, it is worth introducing 
a reference frame other than the desired frame X',j. This 
frame is referred to as the compliant frame E c , and is 
specified by the quantities p c , R c , v c , and v c that are 
computed from p^, Rd, v^, and ltd and the measured 
wrench h c , by integrating the equation 

K M Av c dc + K D Av c dc + h c A =h c , (9.27) 

where h c A is the elastic wrench in the presence of a fi¬ 
nite displacement between the desired frame and the 
compliant frame E c . Then, a motion control strategy, 
based on inverse dynamics, is designed so that the end- 
effector frame E t is taken to coincide with the compli¬ 
ant frame E c . To guarantee the stability of the overall 
system, the bandwidth of the motion controller should 
be higher than the bandwidth of the impedance con¬ 
troller. 


A block diagram of the resulting scheme is sketched 
in Fig. 9.2. It is evident that, in the absence of in¬ 
teraction, the compliant frame E c coincides with the 
desired frame Dj and the dynamics of the position and 
orientation error, as well as the disturbance rejection ca¬ 
pabilities, depend only on the gains of the inner motion 
control loop. On the other hand, the dynamic behav¬ 
ior in the presence of interaction is imposed by the 
impedance gains (9.27). 

The control scheme of Fig. 9.2 is also known as ad¬ 
mittance control because, in (9.27), the measured force 
(the input) is used to compute the motion of the compli¬ 
ant frame (the output), given the motion of the desired 
frame; a mapping with a force as input and a position 
or velocity as output corresponds to a mechanical ad¬ 
mittance. Vice versa, (9.21), mapping the end-effector 
displacement (the input) from the desired motion tra¬ 
jectory into the contact wrench (the output), has the 
meaning of a mechanical impedance. Experiments on 
admittance control are reported in l-gAa'II'Jfm. 

Simplified Schemes 

The inverse dynamics control is model based and 
requires modification of current industrial robot con¬ 
trollers, which are usually equipped with indepen¬ 
dent proportional-integral (PI) joint velocity controllers 
with very high bandwidth. These controllers are able 
to decouple the robot dynamics to a large extent, espe¬ 
cially in the case of slow motion, and to mitigate the 
effects of external forces on the manipulator motion if 
the environment is sufficiently compliant. Hence, the 
closed-loop dynamics of the controlled robot can be ap¬ 
proximated by 

in joint space, or equivalently 

i) e =V r (9.28) 

in the operational space, where q r and v t are the control 
signals for the inner velocity motion loop generated by 
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a suitably designed outer control loop. These control 
signals are related by 

q, = r\q)v t . 

The velocity v t , corresponding to a velocity-resolved 
control, can be computed as 

v° = vl + K» l (h° A -hi) , 

where the control input has been referred to the end- 
effector frame, K D is a 6 x 6 positive-definite matrix 
and /ja is the elastic wrench (9.13) with stiffness ma¬ 
trix K P . The resulting closed-loop equation is 

K d A v e Ae + h e A =hl, 


9.3 Interaction Tasks 

Indirect force control does not require explicit knowl¬ 
edge of the environment, although to achieve a satisfac¬ 
tory dynamic behavior the control parameters have to be 
tuned for a particular task. On the other hand, a model 
of the interaction task is required for the synthesis of 
direct force control algorithms. 

An interaction task is characterized by complex 
contact situations between the manipulator and the en¬ 
vironment. To guarantee proper task execution, it is 
necessary to have an analytical description of the in¬ 
teraction force and moment, which is very demanding 
from a modeling viewpoint. 

A real contact situation is a naturally distributed 
phenomenon in which the local characteristics of the 
contact surfaces as well as the global dynamics of 
the manipulator and environment are involved. In 
detail: 

• The environment imposes kinematic constraints on 
the end-effector motion, due to one or more con¬ 
tacts of different type, and a reaction wrench arises 
when the end-effector tends to violate the con¬ 
straints (e.g., the case of a robot sliding a rigid tool 
on a frictionless rigid surface). 

• The end-effector, while being subject to kinematic 
constraints, may also exert a dynamic wrench on 
the environment, in the presence of environment dy¬ 
namics (e.g., the case of a robot turning a crank, 
when the crank dynamics is relevant, or a robot 
pushing against a compliant surface). 

• The contact wrench may depend on the structural 
compliance of the robot, due to the finite stiffness 
of the joints and links of the manipulator, as well as 


corresponding to a compliant behavior of the end-effec¬ 
tor characterized by a damping K D and a stiffness K P . 
In the case K P = 0, the resulting scheme is known as 
damping control. 

Alternatively, an admittance-type control scheme 
can be adopted, where the motion of a compliant 
frame X c can be computed as the solution of the dif¬ 
ferential equation 

Kd Attj c + h c A = h% 

in terms of the position p c , orientation R c , and velocity 
twist v c , where the inputs are the motion variables of 
the desired frame and the contact wrench h c t . The 
motion variables of X c are then input to an inner po¬ 
sition and velocity controller. In the case K D = 0, the 
resulting scheme is known as compliance control. 


of the wrist force/torque sensor or of the tool (e.g., 
an end-effector mounted on an RCC device). 

• Local deformation of the contact surfaces may oc¬ 
cur during the interaction, producing distributed 
contact areas (e.g., the case of a soft contact surface 
of the tool or of the environment). 

• Static and dynamic friction may occur in the case of 
non ideally smooth contact surfaces. 

The design of the interaction control and the perfor¬ 
mance analysis are usually carried out under simplify¬ 
ing assumptions. The following two cases are consid¬ 
ered: 

1. The robot and the environment are perfectly rigid 
and purely kinematics constraints are imposed by 
the environment, 

2. The robot is perfectly rigid, all the compliance in 
the system is localized in the environment, and the 
contact wrench is approximated by a linear elastic 
model. 

In both cases, frictionless contact is assumed. It is ob¬ 
vious that these situations are only ideal. However, the 
robustness of the control should be able to cope with 
situations where some of the ieal assumptions are re¬ 
laxed. In that case the control laws may be adapted to 
deal with nonideal characteristics. 

9.3.1 Rigid Environment 

The kinematic constraints imposed by the environ¬ 
ment can be represented by a set of equations that the 
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variables describing the end-effector position and ori¬ 
entation must satisfy; since these variables depend on 
the joint variables through the direct kinematic equa¬ 
tions, the constraint equations can also be expressed in 
the joint space as 

</>(q) = 0. (9.29) 

The vector <J> is an m x 1 function, with men, where n is 
the number of joints of the manipulator, assumed to be 
nonredundant; without loss of generality, the case n = 6 
is considered. Constraints of the form (9.29), involv¬ 
ing only the generalized coordinates of the system, are 
known as holonomic constraints. The case of time-vary¬ 
ing constraints of the form <j>( q,t) = 0 is not considered 
here but can be analyzed in a similar way. Moreover, 
only bilateral constraints expressed by equalities of the 
form (9.29) are of concern; this means that the end- 
effector always keeps contact with the environment. 
The analysis presented here is known as kinetostatic 
analysis. 

It is assumed that the vector (9.29) is twice differ¬ 
entiable and that its m components are linearly indepen¬ 
dent at least locally in a neighborhood of the operating 
point. Hence, differentiation of (9.29) yields 

U(q)q = 0. (9-30) 

where J<^( q) = d(/)/dq is the m x 6 Jacobian of (/>( q), 
known as the constraint Jacobian. By virtue of the 
above assumption, J</>(< 7 ) is of rank m at least locally 
in a neighborhood of the operating point. 

In the absence of friction, the generalized inter¬ 
action forces are represented by a reaction wrench 
that tends to violate the constraints. This end-effector 
wrench produces reaction torques at the joints that can 
be computed using the principle of virtual work as 

T e = JJ(?)A . 

where A is an m x 1 vector of Lagrange multipliers. The 
end-effector wrench corresponding to r e can be com¬ 
puted as 

h e = J~ T ( q)r e = S f (<jr)A , (9.31) 

where 

Sf = J -T («)jJ(«). (9.32) 

From (9.31) it follows that h e belongs to the m- 
dimensional vector space spanned by the columns of 
the 6 x m matrix Sf. The inverse of the linear transfor¬ 
mation (9.31) is computed as 

A = S \(q)h t , 


where sj denotes a weighted pseudoinverse of the ma¬ 
trix Sf, i. e., 

Sj = (SjFWSf ) -1 Sj W , ( 9 . 34 ) 

where W is a suitable weighting matrix. 

Notice that, while the range space of the matrix Sf 
in (9.32) is uniquely defined by the geometry of the 
contact, the matrix Sf itself is not unique; also, the con¬ 
straint equations (9.29), the corresponding Jacobian 
as well as the pseudoinverse Sf and the vector A are not 
uniquely defined. 

In general, the physical units of measure of the ele¬ 
ments of A are not homogeneous and the columns of the 
matrix Sf , as well as of the matrix SJ , do not necessar¬ 
ily represent homogeneous entities. This may produce 
invariance problems in the transformation (9.33) if h e 
represents a measured wrench that is subject to distur¬ 
bances and, as a result, may have components outside 
the range space of Sf. If a physical unit or a reference 
frame is changed, the matrix Sf undergoes a transforma¬ 
tion; however, the result of (9.33) with the transformed 
pseudoinverse in general depends on the adopted phys¬ 
ical units or on the reference frame. The reason is that 
the pseudoinverse is the weighted least-squares solution 
of a minimization problem based on the norm of the 
vector h e — Sf( q) A, and the invariance can be guaran¬ 
teed only if a physically consistent norm of this vector 
is used. In the ideal case that h t is in the range space 
of Sf, there is a unique solution for A in (9.33), regard¬ 
less of the weighting matrix, and hence the invariance 
problem does not appear. 

A possible solution consists of choosing Sf so that 
its columns represent linearly independent wrenches. 
This implies that (9.31) gives h e as a linear combi¬ 
nation of wrenches and A is a dimensionless vector. 
A physically consistent norm on the wrench space can 
be defined based on the quadratic form hjK~ 1 h e , which 
has the meaning of an elastic energy if K is a positive- 
definite matrix corresponding to a stiffness. Hence, the 
choice W = K —1 can be made for the weighting matrix 
of the pseudoinverse. 

Notice that, for a given Sf, the constraint Jaco¬ 
bian can be computed from (9.32) as J^( q) = SjFj( q)\ 
moreover, the constraint equations can be derived by in¬ 
tegrating (9.30). 

Using (9.1) and (9.32), the equality (9.30) can be 
rewritten in the form 

j0(?)J“ 1 (?)J(?)? = S>e=O, (9.35) 

which, by virtue of (9.31), is equivalent to 

hjv e = 0. 


( 9 . 33 ) 


( 9 . 36 ) 



Force Control I 9.3 Interaction Tasks 


Equation (9.36) represents the kinetostatic relation¬ 
ship, known as reciprocity, between the ideal reaction 
wrench h t (belonging to the so-called force-controlled 
subspace) and the end-effector twist that obeys the con¬ 
straints (belonging to the so-called velocity-controlled 
subspace). The concept of reciprocity, expressing the 
physical fact that, in the hypothesis of rigid and fric¬ 
tionless contact, the wrench does not cause any work 
against the twist, is often confused with the concept of 
orthogonality, which makes no sense in this case be¬ 
cause twists and wrenches belong to different spaces. 

Equations ((9.35)) and (9.36) imply that the veloc¬ 
ity-controlled subspace is the reciprocal complement of 
the m-dimensional force-controlled subspace, identified 
by the range of matrix Sf. Hence, the dimension of the 
velocity-controlled subspace is 6 — m and a6x(6-m) 
matrix S v can be defined, whose columns span the ve¬ 
locity-controlled subspace, i. e., 

v e = S v (q)v , (9.37) 

where v is a suitable (6 — m) x 1 vector. From (9.35) 
and (9.37) the following equality holds 

Sf(9)Sv( 9 ) = 0; (9.38) 

moreover, the inverse of the linear transformation (9.37) 
can be computed as 

v = $l(q)v t , (9.39) 

where S] denotes a suitable weighted pseudoinverse of 
the matrix S v , computed as in (9.34). 

Notice that, as for the case of Sf, although the range 
space of the matrix S v is uniquely defined, the choice 
of the matrix S v itself is not unique. Moreover, the 
columns of S v are not necessarily twists and the scalar v 
may have nonhomogeneous physical dimensions. How¬ 
ever, in order to avoid invariance problems analogous 
to that considered for the case of Sf, it is convenient 
to select the columns of S v as twists so that the vec¬ 
tor v is dimensionless; moreover, the weighting matrix 
used to compute the pseudoinverse in (9.39) can be set 
as W = M, being M a 6 x 6 inertia matrix; this corre¬ 
sponds to defining a norm in the space of twists based 
on the kinetic energy. It is worth observing that the 
transformation matrices of twists and wrenches, corre¬ 
sponding to a change of reference frame, are different; 
however, if twists are defined with angular velocity 
on top and translational velocity on bottom, then their 
transformation matrix is the same as for wrenches. 

The matrix S v may also have an interpretation in 
terms of Jacobians, as for Sf in (9.32). Due to the pres¬ 
ence of m independent holonomic constraints (9.29), 


the configuration of the robot in contact with the en¬ 
vironment can be described in terms of a (6 — m) x 1 
vector r of independent variables. From the implicit 
function theorem, this vector can be defined as 

r = f(q) , (9.40) 

where x/r( q) is any (6 — m) x 1 twice-differentiable vec¬ 
tor function such that the m components of <j)( q) and 
the n — m components of \j/ ( q) are linearly independent 
at least locally in a neighborhood of the operating point. 
This means that the mapping (9.40), together with the 
constraint (9.29), is locally invertible, with inverse de¬ 
fined as 

q = p(r), (9.41) 

where p(r) is a 6 x 1 twice-differentiable vector func¬ 
tion. Equation (9.41) explicitly provides all the joint 
vectors which satisfy the constraint (9.29). Moreover, 
the joint velocity vectors that satisfy (9.30) can be com¬ 
puted as 

q = 3p(r)r, 

where J p (r) = dp/dr is a 6 x (6 — m) full-rank Jacobian 
matrix. Therefore, the following equality holds 

J <t>(q)J P (r) = 0 , 

which can be interpreted as a reciprocity condition be¬ 
tween the subspace of the reaction torques spanned by 
the columns of the matrix Jj and the subspace of the 
constrained joint velocities spanned by the columns of 
the matrix J p . 

Rewriting the above equation as 

J <i> (q)J(qr 1 J(q)J P (r) = 0, 

and taking into account (9.32) and (9.38), the matrix S v 
can be expressed as 

S v = J(q)Jp(r) , (9.42) 

which, by virtue of (9.40) and (9.41), it can be equiva¬ 
lently expressed as a function of either q or r 

The matrices Sf and S v , and their pseudoinverse sj 
and Sj are known as selection matrices. They play 
a fundamental role for the task specification, i. e., the 
specification of the desired end-effector motion and in¬ 
teraction forces and moments, as well as for the control 
synthesis. 
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9.3.2 Compliant Environment 

In many applications, the interaction wrench between 
the end-effector and a compliant environment can 
be approximated by an ideal elastic model of the 
form (9.24). However, since the stiffness matrix K 
is positive definite, this model describes a fully con¬ 
strained case, when the environment deformation co¬ 
incides with the infinitesimal twist displacement of the 
end-effector. In general, however, the end-effector mo¬ 
tion is only partially constrained by the environment 
and this situation can be modeled by introducing a suit¬ 
able positive-semidefinite stiffness matrix. 

The stiffness matrix describing the partially con¬ 
strained interaction between the end-effector and the 
environment can be computed by modeling the en¬ 
vironment as a couple of rigid bodies, S and O, 
connected through an ideal 6-DOF spring of compli¬ 
ance C = K 1 . Body S is attached to a frame E s and 
is in contact with the end-effector; body O is attached 
to a frame XJ 0 which, at equilibrium, coincides with 
frame S s . The environment deformation about the equi¬ 
librium, in the presence of a wrench h s , is represented 
by the infinitesimal twist displacement 8x so between 
frames E s and which can be computed as 

8x so = C h s . (9.43) 

All the quantities hereafter are referred to frame E s but 
the superscript s is omitted for brevity. 

For the considered contact situation, the end- 
effector twist does not completely belong to the ideal 
velocity subspace, corresponding to a rigid environ¬ 
ment, because the environment can deform. Therefore, 
the infinitesimal twist displacement of the end-effector 
frame E e with respect to A7 0 can be decomposed as 

8jc e0 = 8x v + SjCf , (9.44) 

where 8x v is the end-effector infinitesimal twist dis¬ 
placement in the velocity-controlled subspace, defined 
as the 6 — m reciprocal complement of the force-con- 
trolled subspace, while 8xf is the end-effector infinites¬ 
imal twist displacement corresponding to the environ¬ 
ment deformation. Hence, 

8x v = Py&Ceo . (9.45) 

8x f = (I —P v )8x eo = (I —P v )8* so , (9.46) 

where P v = S v Sj and S v and Sj are defined as in the 
rigid environment case. Being P V P V = Pv. the matrix P v 
is a projection matrix that filters out all the end-effector 
twists (and infinitesimal twist displacements) that are 
not in the range space of S v . Moreover, I — P v is a pro¬ 
jection matrix that filters out all the end-effector twists 


(and infinitesimal twist displacements) that are in the 
range space of S v . The twists P v i> are denoted as twists 
of freedom while the twists (I — P v )u are denoted as 
twists of constraint. 

In the hypothesis of frictionless contact, the interac¬ 
tion wrench between the end-effector and the environ¬ 
ment is restricted to a force-controlled subspace defined 
by the m-dimensional range space of a matrix Sf, as for 
the rigid environment case, i. e., 

ft e = S f A = h s , (9.47) 

where A is an m x 1 dimensionless vector. Premulti¬ 
plying both sides of (9.44) by S^ and using (9.43), 
(9.45), (9.46), and (9.47) yields 

8x eo = SjF C SfA , 

where the identity S^Py = 0 has been exploited. There¬ 
fore, the following elastic model can be found 

h t = SfA = K'Sxeo , (9.48) 

where K' = Sf(S^CSf)~ 1 S^F is the positive-semidefi¬ 
nite stiffness matrix corresponding to the partially con¬ 
strained interaction. 

If the compliance matrix C is adopted as a weight¬ 
ing matrix for the computation of Sf, then K' can be 
expressed as 

K' = PfK , (9.49) 

where Pf = SfSjL Being PfPf = Pf, matrix Pf is a pro¬ 
jection matrix that filters out all the end-effector 
wrenches that are not in the range space of Sf. 

The compliance matrix for the partially constrained 
interaction cannot be computed as the inverse of K\ 
since this matrix is of rank m < 6. However, us¬ 
ing (9.46), (9.43), and (9.47), the following equality can 
be found 

8x f = C 'h e , 

where the matrix 

C' = (I-P V )C, (9.50) 

of rank 6 — m, is positive semidefinite. If the stiffness 
matrix K is adopted as a weighting matrix for the com¬ 
putation of S( , then the matrix C' has the noticeable 
expression C' = C — S v (S', r K S v ) -1 Sy, showing that C' 
is symmetric. 



Force Control I 9.3 Interaction Tasks 


9.3.3 Task Specification 

An interaction task can be assigned in terms of a desired 
end-effector wrench h c | and twist In order to be con¬ 

sistent with the constraints, these vectors must lie in the 
force- and velocity-controlled subspaces, respectively. 
This can be guaranteed by specifying vectors X^ and 
and computing h c \ and Ud as 


and the desired velocities: 

• A nonzero linear velocity along the z t -axis 

• An arbitrary angular velocity about the z t -axis. 

The task continues until a large reaction force in the 
Zt direction is measured, indicating that the peg has hit 
the bottom of the hole, not represented in the figure. 
Hence, the matrices Sf and S v can be chosen as 


h d — SfAd, rtd — S v t>d * 

where Sf and S v have to be suitably defined on the basis 
of the geometry of the task, and so that invariance with 
respect to the choice of the reference frame and change 
of physical units is guaranteed. 

Many robotic tasks have a set of orthogonal ref¬ 
erence frames in which the task specification is very 
easy and intuitive. Such frames are called task frames 
or compliance frames. An interaction task can be spec¬ 
ified by assigning a desired force/torque or a desired 
linear/angular velocity along/about each of the frame 
axes. The desired quantities are known as artificial con¬ 
straints because they are imposed by the controller; 
these constraints, in the case of rigid contact, are 
complementary to those imposed by the environment, 
known as natural constraints. 

Some examples of task frame definition and task 
specification are given below. 

Peg-in-Hole 

The goal of this task is to push the peg into the hole 
while avoiding wedging and jamming. The peg has two 
degrees of motion freedom, hence the dimension of the 
velocity-controlled subspace is 6 — m = 2, while the di¬ 
mension of the force-controlled subspace is m = 4. The 
task frame can be chosen as shown in Fig. 9.3 and the 
task can be achieved by assigning the following desired 
forces and torques: 


/I 0 0 0\ 
0 10 0 
0 0 0 0 
0 0 10 
0 0 0 1 
\0 0 0 0 / 


(0 0 \ 
0 0 


0 0 
\0 1 / 


where the columns of Sf have the dimensions of 
wrenches and those of S v have the dimensions of twists, 
defined in the task frame, and they transform accord¬ 
ingly when changing the reference frame. The task 
frame can be chosen either attached to the end-effector 
or to the environment. 

Turning a Crank 

The goal of this task is turning a crank with an idle han¬ 
dle. The handle has two degrees of motion freedom, 
corresponding to the rotation about the z t -axis and to 
the rotation about the rotation axis of the crank. Hence 
the dimension of the velocity-controlled subspace is 
6 — m = 2, while the dimension of the force-controlled 
subspace is m = 4. The task frame can be assumed as 
in the Fig. 9.4, attached to the crank. The task can be 
achieved by assigning the following desired forces and 
torques: 

• Zero forces along the x t and z t axes 

• Zero torques about the x t and y t axes. 


• Zero forces along the x t and y t axes 

• Zero torques about the x t and y t axes. 



Fig. 9.3 Insertion of a cylindrical peg into a hole 


and the following desired velocities: 
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• A nonzero linear velocity along the y t -axis 

• An arbitrary angular velocity about the z t -axis. 

Hence, the matrices Sf and S v can be chosen as 


/I 0 0 0\ 
0 0 0 0 
0 10 0 
0 0 10 
0 0 0 1 
\0 0 0 0 / 


(0 0 \ 
1 0 


0 0 

\0 V 


referred to the task frame. In this case, the task 
frame is fixed with respect to the crank, but in mo¬ 
tion with respect both the end-effector frame (fixed 
to the handle) and to the base frame of the robot. 
Hence, the matrices Sf and S v are time variant when 
referred either to the end-effector frame or to the base 
frame. 


Hence, the matrices Sf and S v can be chosen as 


0 0\ 

1 0 
0 0 
0 0 ' 

0 0 
0 \) 

The elements of the 6x6 stiffness matrix K 7 , corre¬ 
sponding to the partially constrained interaction of the 
end-effector with the environment, are all zero except 
those of the 3x3 principal minor formed by rows 
3,4,5 and columns 3,4,5 of K', which can be com¬ 
puted as 


S f = 


/0 0 0 \ 
0 0 0 
1 0 0 
0 1 0 
0 0 1 
\0 0 0 / 


s v = 


/I 

0 

0 

0 

0 

Vo 



c 3,3 

C3,4 

^3,5 

C4,3 

C4,4 

£4,5 

, c 5,3 

C5,4 

^5,5 


Sliding a Block on a Planar Elastic Surface 
The goal of this task is to slide a prismatic block over 
a planar surface along the jc t -axis, while pushing with 
a given force against the elastic planar surface. The ob¬ 
ject has three degrees of motion freedom, hence the 
dimension of the velocity-controlled subspace is 6 — 
m = 3 while the dimension of the force-controlled 
subspace is m = 3. The task frame can be chosen at¬ 
tached to the environment, as shown in Fig. 9.5, and 
the task can be achieved by assigning the desired 
velocities: 

• A nonzero velocity along the jc t -axis 

• A zero velocity along the y t -axis 

• A zero angular velocity about the z t -axis, 

and the desired forces and torques: 

• A nonzero force along the z t -axis 

• Zero torques about the jr t and y t axes. 



Fig. 9.5 Sliding of a prismatic object on a planar elastic 
surface 


where cq = Cjj are the elements of the compliance ma¬ 
trix C. 

General Contact Model 

The task frame concept has proven to be very useful 
for the specification of a variety of practical robotic 
tasks. However, it only applies to task geometries with 
limited complexity, for which separate control modes 
can be assigned independently to three pure transla¬ 
tional and three pure rotational directions along the axes 
of a single frame. For more complex situations, as in 
the case of multiple-point contact, a task frame may 
not exist and more complex contact models have to be 
adopted. A possible solution is represented by the vir¬ 
tual contact manipulator model, where each individual 
contact is modeled by a virtual kinematic chain between 
the manipulated object and the environment, giving the 
manipulated object (instantaneously) the same motion 
freedom as the contact. The velocities and force kine¬ 
matics of the parallel manipulator, formed by the virtual 
manipulators of all individual contacts, can be derived 
using the standard kinematics procedures of real manip¬ 
ulators and allow the construction of bases for the twist 
and wrench spaces of the total motion constraint. 

A more general approach, known as constraint- 
based task specification opens up new applications in¬ 
volving complex geometries and/or the use of multiple 
sensors (force/torque, distance, visual sensors) for con¬ 
trolling different directions in space simultaneously. 
The concept of task frame is extended to that of mul¬ 
tiple feature frames. Each of the feature frames makes 
it possible to model part of the task geometry using 
translational and rotational directions along the axes of 
a frame; also, part of the constraints is specified in each 
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a) 


b) 



Fig.9.6a,b Estimation of orientation error: (a) velocity- 
based approach, (b) force-based approach 

of the feature frames. The total model and the total set 
of constraints are achieved by collecting the partial task 
and constraints descriptions, each expressed in the indi¬ 
vidual feature frames. 

9.3.4 Sensor-Based 

Contact Model Estimation 

The task specification relies on the definition of the ve¬ 
locity-controlled subspace and of the force-controlled 
subspace assuming that an accurate model of the con¬ 
tact is available all the time. On the other hand, in 
most practical implementations, the selection matri¬ 
ces Sf and S v are not exactly known, however many 
interaction control strategies turn out to be rather robust 
against modeling errors. In fact, to cope reliably with 

9.4 Hybrid Force/Motion Control 

The aim of hybrid force/motion control is to split up 
simultaneous control of both end-effector motion and 
contact forces into two separate decoupled subprob¬ 
lems. In the following, the main control approaches in 
the hybrid framework are presented for the cases of both 
rigid and compliant environments. 

9.4.1 Acceleration-Resolved Approach 

As for the case of motion control, the acceleration- 
resolved approach is aimed at decoupling and lineariz- 


these situations is exactly why force control is used. The 
robustness of the force controller increases if the matri¬ 
ces Sf and S v can be continuously updated, using mo¬ 
tion and/or force measurements, during task execution. 

In detail, a nominal model is assumed to be avail¬ 
able; when the contact situation evolves differently 
from what the model predicts, the measured and pre¬ 
dicted motion and force will begin to deviate. These 
small incompatibilities can be measured and can then 
be used to adapt the model online, using algorithms de¬ 
rived from classical state-space prediction-correction 
estimators such as the Kalman filter. 

Figure 9.6 reports an example of error between 
nominal and measured motion and force variables, typ¬ 
ical of a two-dimensional contour-following task. The 
orientation of the contact normal changes if the envi¬ 
ronment is notplanar. Hence an angular error 6 appears 
between the nominal contact normal, aligned to the 34 - 
axis of the task frame (the frame with axes x t and y t ), 
and the real contact normal, aligned to the y r -axis of the 
real task frame (the frame with axes x r and y r ). This 
angle can be estimated with either velocity or force 
measurements only: 

• Velocity-based approach : the executed linear veloc¬ 
ity v, which is tangent to the real contour (aligned 
to the v,-axis), does not completely lie along the 
Xt-axis, but has a small component Vy t along the 
y t -axis. The orientation error 0 can then be approx¬ 
imated by 6 = tan ~ 1 {v yi /v Xt ). 

• Force-based approach: the measured (ideal) contact 
force / does not completely lie along the nomi¬ 
nal normal direction, aligned to the y r axis, but has 
a small component f Xl along the ;c t -axis. The ori¬ 
entation error 6 can then be approximated by 0 = 
tan -1 (/*//*). 

The velocity-based approach is disturbed by me¬ 
chanical compliance in the system; the force-based 
approach is disturbed by contact friction. 


ing the nonlinear robot dynamics at the acceleration 
level, via an inverse dynamics control law. In the pres¬ 
ence of interaction with the environment, a complete 
decoupling between the force- and velocity-controlled 
subspaces is sought. The basic idea is that of designing 
a model-based inner control loop to compensate for the 
nonlinear dynamics of the robot manipulator and de¬ 
couple the force and velocity subspaces; hence an outer 
control loop is designed to ensure disturbance rejec¬ 
tion and tracking of the desired end-effector force and 
motion. 
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Rigid Environment 

In the case of a rigid environment, the external wrench 
can be written in the form h e = SfA. The force multi¬ 
plier A can be computed from (9.2) by multiplying both 
sides of (9.2) by the weighted pseudo-inverse sj with 
weight A 1 ( q) and using the time derivative of the last 
equality (9.35). This yields 

X = s](q)[h c -p(q,q)] + A f (q)S]v t , (9.51) 

where Af(^) = (Sf A _ 1 Sf ) _1 and fi(q,q) = Tq + q. 
Replacing again (9.51) into (9.2) gives 

A( 0 )t; e + SfA f ( 0 )Sj' v e =Nf(q)[h c -/i(q,q)] , 

(9.52) 

with Nf = I — SfSf. Notice that NfNf = Nf and NfSf = 
0, hence the 6 x 6 matrix Nf is a projection matrix that 
filters out all the end-effector wrenches lying in the 
range of Sf. These correspond to wrenches that tend 
to violate the constraints. Equation (9.52) represents 
a set of six second-order differential equations whose 
solution, if initialized on the constraints, automatically 
satisfy (9.29) at all times. 

The reduced-order dynamics of the constrained sys¬ 
tem is described by 6 — m second-order equations that 
are obtained by premultiplying both sides of (9.52) by 
the matrix Sj and substituting the acceleration with 

v e = S v v + S v v . 

The resulting equations are 

A v (<jr)v = |^/t c - fi{q,q) - A(#)S v vJ , (9.53) 

where 

A v = S^ASv 

and the identities (9.38) and 
SjN f = 

have been used. Moreover, expression (9.51) can be 
rewritten as 

X =S](q) |* c -/t(g,40-A(g)SvvJ , (9.54) 

where the identity SjFS v = —Sf S v has been exploited. 

Equation (9.54) reveals that the force multiplier vec¬ 
tor X instantaneously depends also on the applied input 
wrench h c . Hence, by suitably choosing h c , it is possi¬ 
ble to directly control the m independent components 
of the end-effector wrench that tend to violate the con¬ 
straints; these components can be computed from the m 


force multipliers through (9.31). On the other hand, the 
6 — m independent components of the end-effector ve¬ 
locity twist can be controlled through h c via (9.53). 

An inverse-dynamics inner control loop can be de¬ 
signed by choosing the control wrench h c as 

h c = A(q)S w a v + S { fx + fi(q,q) + A(q)S v v , 

(9.55) 

where a v and f x are properly designed control inputs. 

Substituting (9.55) into (9.53) and (9.51) yields 

v = a v , 

X=fx, 

showing that control law (9.55) allows complete de¬ 
coupling between the force- and velocity-controlled 
subspaces. 

It is worth noticing that, for the implementation 
of the control law (9.55), the constraint (9.29) as well 
as (9.40) defining the vector of the configuration vari¬ 
ables for the constrained system are not required, pro¬ 
vided that the matrices Sf and S v are known or estimated 
online. In these cases, the task can easily be assigned by 
specifying a desired force, in terms of the vector Ad(f), 
and a desired velocity, in terms of the vector Vd(t); 
moreover, a force/velocity control is implemented. 

The desired force Ad (t) can be achieved by setting 

fx=X d (t), (9.56) 

but this choice is very sensitive to disturbance forces, 
since it contains no force feedback. Alternative choices 
are 

fx = A d (t) + K PX [A d (0 — A(r)] . (9.57) 

or 

t 

/l=Ad(f)+Ku J [A d ( t) — A( t)] dr , (9.58) 

o 

where K P ^ and K|;. are suitable positive-definite matrix 
gains. The proportional feedback is able to reduce the 
force error due to disturbance forces, while the integral 
action is able to compensate for constant bias distur¬ 
bances. 

The implementation of force feedback requires the 
computation of the force multiplier A from the mea¬ 
surement of the end-effector wrench h e , which can be 
achieved by using (9.33). 

Velocity control is achieved by setting 

ay = VdU) + K Pv [v d (f) - v(0] 

t 

+ Ki y J [v d (r) — v(t)] dr , (9.59) 

o 
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where K Pv and K| U are suitable matrix gains. It 
is straightforward to show that asymptotic tracking 
of v d (f) and Vd(f) is ensured with exponential conver¬ 
gence for any choice of positive-definite matrices K Pv 
and Kiv. 

The computation of the vector v from the available 
measurements can be achieved using (9.39), where the 
end-effector twist is computed from joint position and 
velocity measurements through (9.1). 

Equations (9.57) or (9.58) and (9.59) represent the 
outer control loop ensuring force/velocity control and 
disturbance rejection. 

When (9.29) and (9.40) are known, the matrices Sf 
and S v can be computed according to (9.32) and (9.42) 
and a force/position control can be designed by speci¬ 
fying a desired force A d (f) and a desired position r d (f). 

Force control can be designed as above, while posi¬ 
tion control can be achieved by setting 

«v = hU) + K Dr [r d (f) - v(t)] + K Pr [r d (t) - r(f)] . 

Asymptotic tracking of /•<[(/), r A (t) and r d (t) is en¬ 
sured with exponential convergence for any choice of 
positive-definite matrices Ko r and Kp r . The vector r 
required for position feedback can be computed from 
joint position measurements via (9.40). 

Compliant Environment 

In the case of a compliant environment, according to the 
decomposition (9.44) of the end-effector displacement, 
the end-effector twist can be decomposed as 

v e = S v v + C'SfA , (9.60) 

where the first term is a twist of freedom, the second 
term is a twist of constraint, the vector v is defined as 
in (9.42), and C' is defined in (9.50). Assuming a con¬ 
stant contact geometry and compliance, i. e., S v = 0, 
C' = 0, and Sf = 0, a similar decomposition holds in 
terms of acceleration 

v e = S v v + C'Sfii . (9.61) 

An inverse-dynamics control law (9.16) can be 
adopted, resulting in the closed loop (9.17), where a 
is a properly designed control input. 

In view of the acceleration decomposition (9.61), 
the choice 

a = S v a y + C'S f fx (9.62) 

allows decoupling of the force control from veloc¬ 
ity control. In fact, substituting (9.61) and (9.62) 


into (9.17) and premultiplying both sides of the result¬ 
ing equation once by and once by Sf, the following 
decoupled equations can be derived 

v=«v, (9.63) 

A =f x ■ (9.64) 

Hence, by choosing a v according to (9.59) as for the 
rigid environment case, asymptotic tracking of a desired 
velocity v d (t) and acceleration v d (f) is ensured, with 
exponential convergence. The control input f \ can be 
chosen as 

fx =A d (t) + K DA [A d (f)-A(f)] 

+ K p a [A d (f) — X(t)] , (9.65) 

ensuring asymptotic tracking of a desired force tra¬ 

jectory (A d (f), A d (r), A d (t)) with exponential conver¬ 
gence for any choice of positive-definite matrices K D/ ^ 
and K P ^. 

Differently from the rigid environment case, feed¬ 
back of A is required for the implementation of the 
force control law (9.65). This quantity could be com¬ 
puted from end-effector wrench measurements h e as 

A = SjAe. 

However, since the wrench measurement signal is often 
noisy, the feedback of A is often replaced by 

A = S f t K / J( ? )?, (9.66) 

where joint velocities are measured using tachometers 
or computed from joint positions via numerical differ¬ 
entiation and K' is the positive-semidefinite stiffness 
matrix (9.49) describing the partially constrained inter¬ 
action. For the computation of (9.66), only knowledge 
(or an estimate) of K' is required, and not that of the 
full stiffness matrix K. Also, the implementation of the 
control law (9.62) requires knowledge (or an estimate) 
of the compliance matrix C ' of the partially constrained 
interaction and not that of the full compliance matrix C. 

If the contact geometry is known, but only an es¬ 
timate of the stiffness/compliance of the environment 
is available, the control law (9.62), with (9.65), may 
still guarantee the convergence of the force error, if 
a constant desired force A d is assigned. In this case, the 
control law (9.62) has the form 

a. = S v a v + C'Sffx > 

where C = (I — P V )C and C is an estimate of the com¬ 
pliance matrix. Hence, (9.63) still holds, while, in lieu 
of (9.64), the following equality can be found 

A = Wx , 


Part A | 9.4 



Part A | 9.4 


214 Part A I Robotics Foundations 


where Lf = (S/C Sf) — 1 S|FC Sf is a nonsingular matrix. 
Thus, the force- and velocity-controlled subspaces re¬ 
main decoupled and the velocity control law (9.59) 
does not need to be modified. On the other hand, if 
the feedback of the time derivative of A is computed 

using (9.66), only an estimate X can be obtained. Us¬ 
ing (9.66), (9.60) and (9.48), the following equality can 
be found 

X = hf'X . 

Therefore, computing the force control law fx as 

in (9.65) with a constant A d , X in lieu of X and Kda = 
K d aI, the dynamics of the closed-loop system is 


X + Kda^- + LfKpA-A — LfKpAA C | 


showing that exponential asymptotic stability of the 
equilibrium X = Ad can be ensured, also in the pres¬ 
ence of the uncertain matrix Lf, with a suitable choice 
of the gains Kda and Kpa . 

9.4.2 Passivity-Based Approach 

The passivity-based approach exploits the passivity 
properties of the dynamic model of the manipula¬ 
tor, which hold also for the constrained dynamic 
model (9.2). It can be easily shown that the choice of 
the matrix C(q,q) that guarantees the skew symme¬ 
try of the matrix H( q) — 2C( q, q) in the joint space, 
also makes the matrix A (< 7 ) — 2T ( q. q) skew symmet¬ 
ric. This fundamental property of Lagrangian systems 
is at the base of passivity-based control algorithms. 

Rigid Environment 

The control wrench h c can be chosen as 


Ac 


A(?)S v v r + T'(q,q)v r + (Sj) T K v (v r - v) 

+ q(q) + S f /A , 

(9.67) 


where T'(q,q) = T S v + AS V , K„ is a suitable sym¬ 
metric and positive-definite matrix, and v r and fx are 
properly designed control inputs. 

Substituting (9.67) into (9.2) yields 

A( 9 )SvS„ +T'(q,q)s v + (Sy) T K v s v 
+ S f (/ A — A) = 0 , (9.68) 


with s v = v r — v and s v = v r — v, showing that the 
closed-loop system remains nonlinear and coupled. 


Premultiplying both sides of (9.68) by the ma¬ 
trix S v , the following expression for the reduced-order 
dynamics is achieved 

A v (q)s v + T v (q,q)s v +K v s v = 0 , (9.69) 

with r v = SyT(^, q)S v + Sy A(<jr)S v ; it can easily be 
shown that the skew symmetry of the matrix A ( q) — 
2T ( q. if) implies that the matrix A v ( q) — 2T v (<jr, q) is 
skew symmetric as well. 

On the other hand, premultiplying both sides 
of (9.68) by the matrix Sj (q), the following ex¬ 
pression for the force dynamics can be found 

fx - A = —Sf ( q) [r'( q. q) - (S+) T K V ] , 

(9.70) 

being sj the weighted pseudo-inverse of Sf with 
weight A -1 (q). The above equation shows that the 
force multiplier A instantaneously depends on the con¬ 
trol input fx but also on the error s v in the velocity- 
controlled subspace. 

The asymptotic stability of the reduced-order sys¬ 
tem (9.69) can be ensured with the choices 

v r = v d + aAv , (9.71) 

v r = Vd + aAjt w , (9.72) 

where a is a positive gain, v d and v d are the desired ac¬ 
celeration and velocity, respectively, Av = v d — v, and 

t 

AjCy = j Av( r)dr . 

0 

The stability proof is based on the positive-definite Lya¬ 
punov function 

V = |syA v (^)s y + aAx^K,,Axy , 

whose time derivative along the trajectories of (9.69), 

V = — Av t K v A v — orAjc^KyA jc v , 

is a definite semi-negative function. Hence, Av = 0, 
Axy = 0, and s v = 0 asymptotically. Therefore, track¬ 
ing of the desired velocity v d (f) is ensured. Moreover, 
the right-hand side of (9.70) remains bounded and 
vanishes asymptotically. Hence, tracking of the de¬ 
sired force A d (f) can be ensured by setting fx as for 
the acceleration-resolved approach, according to the 
choices (9.56)-(9.58). 

Notice that position control can be achieved if a de¬ 
sired positionr d (0 is assigned for the vectorr in (9.40), 
provided that the matrices Sf and S v are computed ac¬ 
cording to (9.32) and (9.42), and the vectors v d = r d , 
v c i = r*d, and Ajc v = r d —r are used in (9.71) and (9.72). 
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Compliant Environment 

The control wrench h c can be chosen as 

h c = A(q)v r + T(q.q)v t 

+ K s (v r -v e )+h e + q(q) , (9.7B) 

where K s is a suitable symmetric positive-definite ma¬ 
trix while v, and its time derivative i) r are chosen as 

v t = 

v t = v& + a Av , 

where a is the positive gain, v& and its time deriva¬ 
tive hj are properly designed control inputs, Av = 
Vd — v e , and 

t 

Ax = j Avdr . 

0 

Substituting (9.73) into (9.2) yields 

A(q)s+ T(^r)s + K s s = 0 , (9.74) 

with s = i) T — i) e and s = v T — v t . 

The asymptotic stability of system (9.74) can be en¬ 
sured by setting 

Vd — T C SfAd , 

where Vd(f) is a desired velocity trajectory and A c i(0 is 
the desired force trajectory. The stability proof is based 
on the positive-definite Lyapunov function 

V= -s J A(q)s + a Ax T K s Ax . 

whose time derivative along the trajectories of (9.74), 

V = —Av J K s Av — a 2 Ax T K s Ax , 

is a negative-definite function. Hence, Av = 0 
and Ax = 0, asymptotically. In the case of constant 
contact geometry and stiffness, the following equalities 
hold 

Av = S v (v d — v) + C'S f (A d — A) , 

t 

Ax = S v J (v d - v)dr + C'SfUd - A) , 

0 

showing that both the velocity and force tracking errors, 
belonging to reciprocal subspaces, converge asymptoti¬ 
cally to zero. 


9.4.3 Velocity-Resolved Approach 

The acceleration-resolved approach, as well as the 
passivity-based approach, require modification of the 
current industrial robot controllers. As for the case of 
impedance control, if the contact is sufficiently com¬ 
pliant, the closed-loop dynamics of a motion-controlled 
robot can be approximated by (9.28), corresponding to 
a velocity-resolved control. 

To achieve force and velocity control, according to 
the end-effector twist decomposition (9.60), the control 
input v r can be chosen as 

v r = S v v v + C'Sf/A , (9.75) 

with 

t 

Vv = v d (t) + Ki y J [v d (r)-v(r)]dr , (9.76) 

o 

and 

/A=A d « + Kp A [AdW-A(r)], (9.77) 

where Ki v and K PA are suitable symmetric and 
positive-definite matrix gains. Decoupling between 
velocity- and force-controlled subspaces and exponen¬ 
tial asymptotic stability of the closed-loop system can 
be proven as for the acceleration-resolved approach. 
Also, since the force error has second-order dynamics, 
an integral action can be added to (9.77) to improve the 
disturbance rejection capabilities, i. e., 

/a = A d (t) + K pA [A d (f)-A(r)] 

t 

+Ku J [A d (» — A(r)] dr , (9.78) 

o 

and the exponential asymptotic stability is guaranteed 
if the matrices K pA and Ku are symmetric and positive 
definite. 

If an estimate C of the stiffness matrix of the en¬ 
vironment is used in (9.75), as for the acceleration- 
resolved approach, the exponential convergence of A 
to a constant Ad can still be ensured for both (9.77) 
and (9.78). 

In some applications, besides the stiffness ma¬ 
trix, also the geometry of the environment is uncer¬ 
tain. In these cases, a force/motion control law sim¬ 
ilar to (9.75) can be implemented, without using the 
selection matrices S v and Sf to separate the force- 
controlled subspace from the velocity-controlled sub¬ 
space. The motion control law can be set as in (9.76), 
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but using full velocity feedback. Also, the force con¬ 
trol law can be set as in (9.78), but using full force 
and moment feedback. That is, both motion control 
and force control are applied in all directions of the 
six-dimensional (6-D) space. The resulting control, 
known as force control with feedforward motion or 
parallel force/position control guarantees force reg¬ 
ulation at the expense of position errors along the 
constrained task directions, thanks to the dominance 


of the force controller over the position controller en¬ 
sured by the presence of the integral action on the 
force error. This approach is tested in the experiments 
reported in and where the 

concept of task frame is also exploited. A framework 
for robotic assembly, where a standard position-based 
robot controller is integrated with an external con¬ 
troller performing force-controlled skills is presented 
in 


9.5 Conclusions and Further Reading 

This chapter has summarized the main approaches to 
force control in a unifying perspective. However, there 
are many aspects that have not been considered and 
that must be carefully taken into account when dealing 
with interaction robotic tasks. The two major paradigms 
of force control (impedance and hybrid force/motion 
control) are based on several simplifying assumptions 
that are only partially satisfied in practical implemen¬ 
tations. In fact, the performance of a force-controlled 
robotic system depends on the interaction with a chang¬ 
ing environment which is very difficult to model and 
identify correctly. A general contact situation is far 
from being completely predictable, not only quantita¬ 
tively, but also qualitatively: the contact configuration 
can change abruptly, or be of a different type than ex¬ 
pected. Hence, the standard performance indices used 
to evaluate a control system, i. e., stability, bandwidth, 
accuracy, and robustness, cannot be defined by consid¬ 
ering the robotic system alone, as for the case of robot 
motion control, but must be always referred to the par¬ 
ticular contact situation at hand. Also, a classification 
of all these different situations is not easy, especially in 
the case of dynamics environments and when the task 
involves multiple contacts acting in parallel. 

Due to the inherent complexity of the force control 
problem, a large number of research papers on this topic 
have been published in the past three decades. A de¬ 
scription of the state of the art of the first decade is 
provided in [9.20], whereas the progress of the second 
decade is surveyed in [9.21] and [9.22], More recently, 
two monographs on force control [9.23,24] have ap¬ 
peared. In the following, a list of references is provided, 
where more details on the arguments presented in the 
chapter, as well as topics not covered here, can be 
found. 

9.5.1 Indirect Force Control 

The concept of generalized spring and damping for 
force control in joint coordinates was originally pro¬ 


posed in [9.3] and the implementation discussed 
in [9.10]. Stiffness control in Cartesian coordinates was 
proposed in [9.9]. Devices based on the remote center 
of compliance were discussed in [9.25] for successful 
mating of rigid parts. The original idea of a mechanical 
impedance model used for controlling the interaction 
between the manipulator and the environment is pre¬ 
sented in [9.7], and a similar formulation is given 
in [9.8]. Stability of impedance control was analyzed 
in [9.26] and the problems of interaction with a stiff en¬ 
vironment were considered in [9.27], 

Adaptive impedance control algorithms [9.28,29] 
have been proposed to overcome uncertainties in the 
dynamic parameters of the robot manipulator, while ro¬ 
bust control schemes can be found in [9.30]. Impedance 
control has also been used in the hybrid force/motion 
control framework [9.31]. 

A reference work on modeling 6-DOF (spatial) 
stiffness is [9.32], while the properties of spatial 
compliance have been analyzed in detail in [9.33— 
35] ; a 6-DOF variable compliant wrist was proposed 
in [9.36], while several studies concerning the con¬ 
struction of programmed compliances, optimized for 
specific tasks, have been proposed [9.37,38]. The en¬ 
ergy-based approach to derive a spatial impedance was 
introduced in [9.39], using rotation matrices; various 
6 -DOF impedance control schemes based on different 
representations of end-effector orientation, including 
the unit quaternion, can be found in [9.40]. The quater¬ 
nion-based formulation is extended to the case of non- 
block-diagonal stiffness matrix in [9.41]. A rigorous 
treatment of spatial impedance control in a passivity 
framework can be found in [9.42], 

More recently, impedance control was proposed as 
an effective approach to enhance safety in applications 
where humans and robots share the same workspace 
and may have physical interaction. To this end, the 
passive compliance of lightweight robots is combined 
with the active compliance ensured by impedance con¬ 
trol [9.43,44], 
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9.5.2 Task Specification 

The concepts of natural and artificial constraints and 
of compliance frame were introduced in [9.11]. These 
ideas have been systematically developed in [9.12,45] 
within the task frame formalism. Theoretical issues on 
the reciprocity of generalized force and velocity direc¬ 
tions are discussed in [9.46,47], while invariance in 
computing the generalized inverse in robotics is ad¬ 
dressed in [9.48]. The issue of partially constrained 
tasks is considered in [9.49], where the models of 
positive-semidefinite stiffness and compliance matrices 
are developed. The problem of estimating geomet¬ 
ric uncertainties is considered in [9.50,51], as well 
as the issue of linking constraint-based task specifi¬ 
cation with real-time task execution control. This ap¬ 
proach is generalized in [9.52], where a systematic 
constraint-based methodology to specify complex tasks 
is presented. 

9.5.3 Hybrid Force/Motion Control 

Early works on force control can be found in [9.10]. 
The original hybrid force/positon control concept was 
introduced in [9.13], based on the natural and arti¬ 
ficial constraint task formulation [9.11]. The explicit 
inclusion of the manipulator dynamic model was pre¬ 
sented in [9. 17], and a systematic approach to modeling 
the interaction with a dynamic environment was de¬ 
veloped in [9.53]. The constrained formulation with 
inverse dynamic controllers is treated in [9.14,54] in 
the Cartesian space as well as in [9.15] joint space. 
The constrained approach was also used in [9.16] with 
a controller based on linearized equations. The invari¬ 


ance problems pointed out in [9.47] were correctly 
addressed, among other papers, in [9.46,55]. Trans¬ 
position of model-based schemes from unconstrained 
motion control to constrained cases was accomplished 
for adaptive control in [9.18, 56, 57] and for robust con¬ 
trol in [9.58]. 

Approaches designed to cope with uncertainties in 
the environment geometry are the force control with 
feedforward motion scheme proposed in [9.2] and the 
parallel force/position control [9.19], based on the con¬ 
cept of dominance of force control on motion control, 
thanks to the use of an integral action on the force er¬ 
ror. A parallel force/position regulator was developed 
in [9.59]. The integral action for removing steady-state 
force errors has traditionally been used; its stability was 
proven in [9.60], while robustness with respect to force 
measurement delays was investigated in [9.61,62]. 

In the absence of force/torque sensors, suitable al¬ 
gorithms can be adopted to estimate the contact force, 
based, e.g., on disturbance observers [9.63], or on the 
error of a position control [9.64]. 

It has generally been recognized that force con¬ 
trol may cause unstable behavior during contact with 
environment. Dynamic models for explaining this phe¬ 
nomenon were introduced in [9.65] and experimen¬ 
tal investigations can be found in [9.66] and [9.67]. 
Moreover, control schemes are usually derived on the 
assumption that the manipulator end-effector is in con¬ 
tact with the environment and that this contact is not 
lost. Impact phenomena may occur and deserve careful 
consideration, and there is a need for global analy¬ 
sis of control schemes including the transition from 
noncontact to contact situations and vice versa, see 
e.g., [9.68-70]. 


Video-References 
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Recent research in impedance control 

available from http://handbookofrobotics.org/view-chapter/09/videodetails/684 
Integration of force strategies and natural admittance control 
available from http://handbookofrobotics.org/view-chapter/09/videodetails/685 
Experiments of spatial impedance control 

available from http://handbookofrobotics.org/view-chapter/09/videodetails/686 
Compliant robot motion; Control and task specification 

available from http://handbookofrobotics.org/view-chapter/09/videodetails/687 
COMRADE: Compliant motion research and development environment 
available from http://handbookofrobotics.org/view-chapter/09/videodetails/691 
Robotic assembly of emergency stop buttons 

available from http://handbookofrobotics.org/view-chapter/09/videodetails/692 
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10. Redundant Robots 



Stefano Chiaverini, Giuseppe Oriolo, Anthony A. Maciejewski 


This chapter focuses on redundancy resolution 
schemes, i.e., the techniques for exploiting the 
redundant degrees of freedom in the solution of 
the inverse kinematics problem. This is obviously 
an issue of major relevance for motion planning 
and control purposes. 

In particular, task-oriented kinematics and 
the basic methods for its inversion at the veloc¬ 
ity (first-order differential) level are first recalled, 
with a discussion of the main techniques for han¬ 
dling kinematic singularities. Next, different first- 
order methods to solve kinematic redundancy are 
arranged in two main categories, namely those 
based on the optimization of suitable performance 
criteria and those relying on the augmentation of 
the task space. Redundancy resolution methods at 
the acceleration (second-order differential) level 
are then considered in order to take into account 
dynamics issues, e.g., torque minimization. Con¬ 
ditions under which a cyclic task motion results 
in a cyclic joint motion are also discussed; this is 
a major issue when a redundant manipulator is 
used to execute a repetitive task, e.g., in industrial 
applications. The use of kinematic redundancy for 
fault tolerance is analyzed in detail. Suggestions 
for further reading are given in a final section. 


10.1 Overview 

Kinematic redundancy occurs when a robotic manipu¬ 
lator has more degrees of freedom than those strictly 
required to execute a given task. This means that, 
in principle, no manipulator is inherently redundant; 
rather, there are certain tasks with respect to which it 
may become redundant. Because it is widely recognized 
that a fully general spatial task consists of following an 
end-effector motion trajectory requiring six degrees of 
freedom, a robot arm with seven or more joints is fre¬ 
quently considered as a typical example of an inherently 
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redundant manipulator. However, even robot arms with 
fewer degrees of freedom, like conventional six-joint 
industrial manipulators, may become kinematically re¬ 
dundant for specific tasks, such as simple end-effector 
positioning without constraints on the orientation. 

The motivation for introducing kinematic redun¬ 
dancy in the mechanical structure of a manipulator goes 
beyond that for using redundancy in traditional engi¬ 
neering design, namely, increasing robustness to faults 
so as to improve reliability (e.g., redundant processors 
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or sensors). In fact, endowing robotic manipulators with 
kinematic redundancy is mainly aimed at increasing 
dexterity, as is the case with a human arm. 

The minimal-complexity approach that character¬ 
ized early manipulator designs had the objective of 
minimizing cost and maintenance. For example, this led 
to the development of selective compliance assembly 
robot arm (SCARA) robots for pick-and-place opera¬ 
tions where products had been designed for assembly, 
i. e., using a single axis of insertion. However, giving 
a manipulator the minimum number of joints required 
to execute its task results in a serious limitation in real- 
world applications where, in addition to the singular¬ 
ity problem, joint limits or workspace obstacles are 
present. All of these give rise to forbidden regions in the 
joint space that must be avoided during operation, thus 
requiring a carefully structured (and static) workspace 
where the motion of the manipulator can be planned in 
advance; this is the typical situation for workcells in tra¬ 
ditional industrial applications. 

On the other hand, the presence of additional de¬ 
grees of freedom besides those strictly required to 
execute the task allows motions of the manipulator that 




Fig. 10.2 The 7-DOF Mitsubishi PA-10 manipulator 


do not displace the end effector (the so-called self-mo¬ 
tions or internal motions); this implies that the same 
task at the end-effector level can be executed in sev¬ 
eral ways at the joint level, giving the possibility of 
avoiding the forbidden regions and ultimately resulting 
in a more versatile mechanism. Such a feature is key to 
allowing operation in unstructured or dynamically vary- 



Fig. 10.3 The 8-DOF DEXTER by Scienzia Machinale 



Fig. 10.4 A 5-finger hand by Schunk 
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ing environments that characterize advanced industrial 
applications and service robotics scenarios. 

In practice, if properly managed, the increased 
dexterity characterizing kinematically redundant robots 
may allow them to not only avoid singularities, joint 
limits, and workspace obstacles, but also to minimize 
torque/energy over a given task, ultimately meaning 
that the robotic manipulator can achieve a higher de¬ 
gree of autonomy. 

The biological archetype of a kinematically re¬ 
dundant manipulator is the human arm, which, not 
surprisingly, also inspires the terminology used to char¬ 
acterize the structure of serial-chain manipulators. In 
fact, the human arm has three degrees of freedom at 
the shoulder, one degree of freedom at the elbow and 
three degrees of freedom at the wrist. The available re¬ 
dundancy can be easily verified by locking one’s wrist, 
e.g., on a table and moving the elbow while keeping 
the shoulder still. The kinematic arrangement of the hu¬ 
man arm has been replicated in a number of robots often 
termed as human-arm-like manipulators. This family 



Fig. 10.5 The Baxter working robot by Rethink Robotics 



Fig. 10.6 The NASA (National Aeronautics and Space 
Administration) Robonaut 


of 7-DOF (degree of freedom) manipulators includes 
the DLR (Deutsches Zentrum filr Luft- und Raum- 
fahrt) Lightweight Robot (Fig. 10.1) and the Mitsubishi 
PA-10 robot (Fig. 10.2). An example of an 8-DOF robot 
is the DEXTER by Scienzia Machinale (Fig. 10.3). Ma¬ 
nipulators with a larger number of joints are often called 
hyperredundant robots, and include many snake-like 
robots described in the literature. 

The use of two or more robotic structures to execute 
a task (as in the case of multifingered hands, humanoid 
mockups or cooperating manipulators, Figs. 10.4-10.7, 
respectively) also gives rise to kinematic redundancy. 
Redundant mechanisms also include vehicle-manipu¬ 
lator systems (Fig. 10.8); in this case, however, the 
possible presence of nonholonomic constraints on the 



Fig. 10.7 A cooperating dual-arm concept robot by Asea 
Brown Boveri (ABB) 



Fig. 10.8 A huge vehicle-manipulator system by KUKA 
Aktiengesellschaft 
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motion of the base must be properly taken into account 
in order to determine the actual degree of redundancy. 

Although the realization of a kinematically redun¬ 
dant structure raises a number of issues from the point 
of view of mechanical design, this chapter focuses 


10.2 Task-Oriented Kinematics 

The relationship between the variables representing the 
configuration of an articulated manipulator and those 
describing an assigned task in the appropriate space can 
be established at the position, velocity or acceleration 
level. In particular, consideration of the first-order task 
kinematics inevitably results in discussion of the task 
Jacobian matrix, which is central to many redundancy 
resolution techniques. 

10.2.1 Task-Space Formulation 

A manipulator consists of a chain of rigid bodies articu¬ 
lated by joints. If q, denotes the variable characterizing 
the relative displacement of body i with respect to 
body i— 1, the vector q = (qi...q N ) 7 uniquely de¬ 
scribes the configuration of an (V-joint serial-chain 
manipulator. Joint i may be either prismatic or revolute, 
in which case q, measures the relative translation or ro¬ 
tation of the attached links, respectively. 

While the manipulator is naturally described and ac¬ 
tuated in the joint space, its operation is conveniently 
specified in terms of the vector t = (fi... t M ) 7 , which 
typically defines the location of the manipulator end- 
effector in a suitably defined task space. In the general 
case, it is of dimension M = 6 and t is chosen so that 
its first three components represent the position of the 
end effector, while its last three components represent 
a minimal description of the end-effector orientation 
(such as Euler angles or the roll-pitch-yaw represen¬ 
tation), i. e., 

t = [Px Py Pz Ot P y] T • 

Typically, one has A > M, so that the joints can provide 
at least the number of degrees of freedom required for 
the end-effector task. If A > M strictly, the manipulator 
is kinematically redundant. 

The relationship between the joint-space coordinate 
vector q and the task-space coordinate vector t is ex¬ 
pressed by the direct kinematics equation 

t = k t (q), (10.1) 

where k t is a nonlinear vector function. 


on redundancy resolution schemes, i. e., techniques for 
exploiting the redundant degrees of freedom in the so¬ 
lution of the inverse kinematics problem. This is an 
issue of major relevance for motion planning and con¬ 
trol purposes. 


Task Jacobian and Geometric Jacobian 

It is useful to consider the first-order differential kine¬ 
matics [10.1] 

t = Mq)q, ( 10 . 2 ) 

that can be obtained by differentiating (10.1) with re¬ 
spect to time. In (10.2), t is the task-space velocity 
vector, q is the joint-space velocity vector, and ijq) = 
dk t /dq is the Mx A task Jacobian matrix (also called 
the analytic Jacobian). 

Remarkably, the components of i relative to the end- 
effector orientation express the rate of change of the 
parameters characterizing the adopted minimal repre¬ 
sentation; they are not the components of the angular 
velocity vector of the end-effector. Indeed, denoting 
by \ N the 3x1 translational velocity vector and by « v 
the 3 x 1 angular velocity vector of the end-effector, and 
defining the end-effector spatial velocity ?;, v as 



the following relationship holds 

i=T(t)v N , (10.4) 

where T is an Mx 6 transformation matrix that is a func¬ 
tion of t only. In the case of M = 6, the transformation 
matrix T takes the form 

T =(! I). 110 9 

where I and 0 are, respectively, the identity and null 
matrix of proper dimensions, and R is a 3 x 3 matrix 
that specifically depends on the minimal representation 
used to describe the end-effector orientation. 

For a given manipulator, the mapping 

v N = J(q)q (10.6) 

relates a joint-space velocity to the corresponding end- 
effector velocity through the 6 x A geometric Jacobian 
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matrix J. The geometric Jacobian matrix is of major 
concern in the kinematic analysis of a manipulator be¬ 
cause it allows description of the motion capabilities of 
the end-effector (in terms of its free rigid-body spatial 
velocity) as a result of the velocity commands at the 
joints in the current configuration. As a matter of fact, 
if J Pi qi denotes the contribution of q , to \ N , Zi-\ is the 
unit vector of the axis of joint i and /y-i, e is the vector 
expressing the position of the end effector frame with 
respect to the joint i frame, it is 

Zj —1 if joint i is prismatic 

Zi- 1 x ri-i, e if joint i is revolute , 

whereas if J oi qj denotes the contribution of q t to 00 N , it 
is 

0 if joint i is prismatic 
Zi- 1 if joint; is revolute . 

By comparing (10.2), (10.4), and (10.6), the relation 
between the geometric Jacobian and the task Jacobian 
can be found as 

J t (?) = T(0J(?). (10.7) 

Second-Order Differential Kinematics 
While the first-order differential kinematics (10.2) re¬ 
lates task-space to joint-space velocities, further differ¬ 
entiation with respect to time provides an analogous 
relationship between accelerations 

t = ii(q)q + iM-q)q • ( 10 . 8 ) 

This equation is also known as the second-order differ¬ 
ential kinematics. 

10.2.2 Singularities 

In this section the occurrence of singular configurations 
is considered to properly analyze their effect on inverse 
kinematics solutions. 

Representation and Kinematic Singularities 
A robot configuration q is singular if the task Jacobian 
matrix J t is rank-deficient there. Considering the role 
of J t in (10.2) and (10.8), it is easy to realize that at 
a singular configuration it is impossible to generate end- 
effector task velocities or accelerations in certain direc¬ 
tions. Further insight can be gained by looking at (10.7), 
which indicates that a singularity may be due to a loss 
of rank of the transformation matrix T and/or of the ge¬ 
ometric Jacobian matrix J. 


Rank deficiencies of T are only related to the math¬ 
ematical relationship established by R between the 
angular velocity vector of the end-effector and the com¬ 
ponents of t relative to the end-effector orientation; 
because the expression of R depends on the adopted 
minimal representation of the orientation, a configu¬ 
ration at which T is singular is then referred to as 
a representation singularity. Remarkably, any minimal 
description of the end-effector orientation experiences 
the occurrence of representation singularities. Further¬ 
more, a given configuration may or may not yield 
a representation singularity depending on which de¬ 
scription of orientation is used. 

A representation singularity is not directly related 
to the true motion capabilities of the manipulator struc¬ 
ture, which can instead be inferred by the analysis of 
the geometric Jacobian matrix. Rank deficiencies of J 
are in fact related to loss of mobility of the manipulator 
end-effector; indeed, end-effector velocities exist in this 
case that are unfeasible for any velocity commanded at 
the joints. A configuration at which J is singular is re¬ 
ferred to as a kinematic singularity. 

Because this chapter focuses on the inversion of 
the task differential kinematics (10.2) and (10.8), in 
the following the task Jacobian matrix and its singu¬ 
larities (which include representation and kinematic 
singularities) are studied in detail. The case N > M is 
considered, which encompasses conventional as well as 
redundant manipulators. 

Singular Value Decomposition of the Jacobian 
To analyze the linear mapping (10.2), the singular 
value decomposition (SVD) of the Jacobian matrix is 
adopted; remarkably, this powerful numerical tool is the 
sole reliable method to compute the rank of a matrix 
and to study near-singular linear mappings. The classic 
Golub-Reinsch algorithm [10.2], which is the most ef¬ 
ficient and numerically stable algorithm to compute the 
SVD of an arbitrary matrix, may however be compu¬ 
tationally demanding in view of real-time applications. 
A faster algorithm that takes advantage of the nature of 
robotic matrix calculations has been proposed [10.3]; 
this makes it possible to improve real-time kinematic 
control techniques. 

The SVD of the task Jacobian matrix can be written 
in the form 

M 

J t = U27V T = J2 a ‘ u ‘ v ^ (10 ' 9) 

;=i 

where U is the MxM orthonormal matrix of the output 
singular vectors it,-, V is the N x N orthonormal ma¬ 
trix of the input singular vectors Vj, and 27 = (S 0) is 
the MxN matrix whose MxM diagonal submatrix S 
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contains the singular values er,- of the matrix J t . It is 
worth noting that the SVD is a well-behaved function 
of its matrix argument; therefore, the singular values 
and associated subspaces do not change radically in 
the neighborhood of the current configuration. Let¬ 
ting rank (J t ) = R, the following hold 

0\ > <72 > • • • > OR > CTfi+l = • • • = 0 , 

2£(J t ) = span{Mi- ,u R } , 

Af(J t ) = sptm{v R+u . ,.,v N }. 

If the task Jacobian is full-rank ( R = M), all the sin¬ 
gular values are nonzero, the range space of J t is the 
entire K M , and the null space of J t has dimension N—M. 
In a singular configuration, R < M\ thus, the last M — R 
singular values are zero, the range space of J, is an R- 
dimensional subspace of R M , and the dimension of the 
null space of J, increases to N — R. An interpretation 
of this from a kinematic viewpoint is presented in the 
following. 

Feasible Velocities. At each configuration, the range 
space of J, is the set of task-space velocities that can 
be obtained as a result of all possible joint-space veloc¬ 
ities q: therefore, it constitutes the so-called subspace 
of feasible velocities for the end-effector task. A basis 
for J t ) is given by the first R output singular vectors; 
accordingly, the effect of a singularity is to decrease the 
dimension of the range space of J, that corresponds to 
a reduction of the space of feasible velocities. 

Null-Space Velocities. At each configuration, the null 
space of J t is the set of joint-space velocities that yield 
zero task velocity; these are thus shortly called null- 
space velocities. A basis for AT(J t ) is given by the N—R 
last input singular vectors, which represent independent 
linear combinations of the velocities at each joint. From 
this viewpoint, the effect of a singularity is to increase 
the dimension of the null space of J, by introducing 
a further independent linear combination of joint veloc¬ 
ities that produces a zero task velocity. 

According to (10.2) and (10.9), a joint velocity 
along the i-th input singular vector results in a task ve¬ 
locity that lies along the i-th output singular vector 

VpeR q = p Vi =>■ t = Ofp Uj . 

Thus, the i-th singular value of J t can be viewed as 
a gain factor relating motion along the v, direction of 
the joint velocity space to the resulting motion along 
the Uj direction of the task velocity space. When a sin¬ 
gularity is approached, the R-th singular value a R tends 
to zero and the task velocity produced by a fixed joint 


velocity along v R is decreased proportionally. At a sin¬ 
gular configuration, the joint-space velocity along v R 
belongs to the null-space velocities and task velocities 
along u R become unfeasible for the manipulator. 

In the generic case, the joint-space velocity q is an 
arbitrary linear combination of individual joint veloci¬ 
ties with nonzero components along all the v t . Its effect 
can be analyzed by combining the single effects of the 
above components. Remarkably, the components of q in 
the null space of J t produce a change in the configura¬ 
tion of the manipulator without affecting its task veloc¬ 
ity. This can be exploited to achieve additional goals - 
such as obstacle or singularity avoidance - in addition 
to the realization of a desired task motion, and consti¬ 
tutes the core of redundancy resolution approaches. 

Distance from Singularities 
Clearly, the effect of a singularity is experienced not 
only at the singular configuration itself but also in its 
neighborhood (Sect. 10.3.2). For this reason, it is im¬ 
portant to be able to characterize the distance from 
singularities through suitable measures; these can then 
be exploited to counteract undesirable effects. 

Because each singularity is associated with a re¬ 
duction in the rank of J t , one conceptually simple 
possibility in the case of a square Jacobian matrix (M = 
N) is to compute its determinant. A generalization of 
this idea that works also for nonsquare Jacobians is the 
manipulability measure [10.4], defined as 

M = VlJ, J?|- 

It can be recognized that the manipulability measure is 
equal to the product of the singular values of J t , i. e., 

M 

m= n ’ 

i= 1 

and thus its zeros coincide with the singularities. 

Another possible measure of distance from a sin¬ 
gular configuration is the condition number of the 
Jacobian matrix [10.5], defined as 

K = - . 

Om 

The condition number has values ranging from 1, at 
configurations where all the singular values are equal, 
to 00 , at singular configurations. Note that when k = 1 
the end effector has the same motion capability in all 
task space directions because all the singular values are 
equal - i. e., the arm is at an isotropic configuration - 
whereas at a singularity it loses mobility in some task- 
space direction. 
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An even more direct measure of the distance from 
singular configurations is the smallest singular value of 
the Jacobian matrix [10.5], i. e., 

Omiti = OjM • 

A computationally light estimate of the smallest sin¬ 
gular value can be obtained either via numerical meth¬ 
ods [10.3, 6, 7] or based on a kinematic analysis of the 
robot structure [10.8]. 


It must be noted that the manipulability measure 
may remain constant even in the presence of variations 
of either the condition number or the smallest singu¬ 
lar value of J t . On the other hand, because the smallest 
singular value changes more radically near singularities 
than the other singular values, it dominates the behav¬ 
ior of the determinant and the condition number of the 
Jacobian matrix; therefore, the most effective measure 
of distance from singular configurations is the smallest 
singular value of J t [10.5]. 


10.3 Inverse Differential Kinematics 

In order to accomplish a task, a proper joint motion 
must be commanded to the manipulator; therefore, it 
is necessary to derive mathematical relations that allow 
one to compute joint-space variables corresponding to 
the assigned task-space variables. This is the objective 
of the inverse kinematics problem. 

The inverse kinematics problem can be solved by 
inverting the direct kinematics equation (10.1), the first- 
order differential kinematics (10.2) or the second-order 
differential kinematics (10.8). If the task is time-vary¬ 
ing (i. e., if a desired trajectory t(t) is assigned), it is 
convenient to solve the differential kinematic relation¬ 
ships because these represent linear equations using the 
task Jacobian matrix [10.9]. 

10.3.1 The General Solution 


Under the assumption that the manipulator is kinemat¬ 
ically redundant (i. e., M < N), one solution of (10.2) 
or (10.8) can be expressed by resorting to the pseudoin¬ 
verse J t ' of the task Jacobian matrix [10.1, 10]; this is 
the unique matrix satisfying the Moore-Penrose condi¬ 
tions [10.11-13] 


Jt J J, 

J t .Jt J, f 

(J, J, f ) T 
(J, + Jt) T 


= J, , 

= J, + , 

= Jt Jt f . 
= Jt + Jt- 


( 10 . 10 ) 


If J t is low-rectangular and full-rank, its pseudoinverse 
can be expressed as 

j t f = j^(j t jfr 1 ■ (10.11) 

If J t is square, expression (10.11) reduces to the stan¬ 
dard inverse matrix. 

The general solution of (10.2) can be written as 

j = jfr+(I-jJjt)«o. (10.12) 


Here, I — jJ J t represents the orthogonal projection ma¬ 
trix into the null space of J t , and qo is an arbitrary joint- 
space velocity; the second part of the solution is there¬ 
fore a null-space velocity. Equation (10.12) provides 
all least-squares solution to the end-effector task con¬ 
straint (10.2), i.e., it minimizes ||f — J^||. In particular, 
if J t is low-rectangular and full-rank, all joint veloci¬ 
ties in the form (10.12) exactly realize the assigned task 
velocity. By acting on qo, one can still obtain differ¬ 
ent joint velocities that give the same end-effector task 
velocity; therefore, as will be discussed later in detail, 
solutions in the form of (10.12) are typically used to 
represent redundancy resolution techniques. 

The particular solution obtained by setting qo = 0 
in (10.12), 

q = i}'t, (10.13) 

provides the least-squares solution of (10.2) with mini¬ 
mum norm, and is known as the pseudoinverse solution. 
In terms of the inverse differential kinematics problem, 
the least-squares property quantifies the accuracy of the 
end-effector task realization, while the minimum norm 
property may be relevant for the feasibility of the joint- 
space velocities. 

As for the second-order kinematics (10.8), its least- 
squares solutions can be expressed in the general form 

q = J?(i- jt?) + (I-Jt + Jt )?0 , (10.14) 

where qo is an arbitrary joint-space acceleration. As 
above, choosing qo = 0 in (10.14) gives the minimum- 
norm acceleration solution 

q = J?(i-j,q). (10.15) 

10.3.2 Singularity Robustness 

We now investigate the kinematics aspects involved in 
the first-order inverse mappings (10.12) and (10.13) 
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with respect to the handling of singularities. With ref¬ 
erence to the SVD of J t in (10.9), let us consider the 
following decomposition of the matrix J,^ 

J t + = YZ f \J T = — VjiiJ , (10.16) 

where, as above, R denotes the rank of the task Jacobian 
matrix. Analogously to (10.9), the following hold 

CTi > <72 > . . . > Or > (Tr+ 1 = . = 0 , 

n(J ] t ) = JV- L (Jt) = span{t)i,..., v K } , 

■2V~(Jt) = 21^ (J t ) = span{« R+1 ,...,%}. 

Notice that, if the Jacobian matrix is full-rank, 
the range space of J, 1 is an M-dimensional subspace 
of R w and the null space of J,^ is trivial. In a singular 
configuration (R < M ), the range space of J,' is an R- 
dimensional subspace of R w , and an M— i?-dimensional 
null space of J,' exists. 

The range space of is the set of joint-space ve¬ 
locities q that can be obtained via the inverse kinematic 
mapping (10.13) as a result of all possible task ve¬ 
locities i. Because these q belong to the orthogonal 
complement of the null space of J t , the pseudoinverse 
solution (10.13) satisfies the least-squares condition, as 
expected. 

The null space of jJ is the set of the task veloci¬ 
ties t that yield null joint-space velocity in the current 
configuration; these t belong to the orthogonal comple¬ 
ment of the space of feasible task velocities. Therefore, 
one effect of the pseudoinverse solution (10.13) is to 
filter out the unfeasible components of the commanded 
task velocity while allowing exact tracking of the fea¬ 
sible components; this is related to the minimum-norm 
property. 

If the assigned task velocity is aligned with Ui, 
the corresponding joint-space velocity - computed 
via (10.13) - is obtained along v ,■ modulated by the 
factor 1/a,. When a singularity is approached, the J?-th 
singular value tends to zero and a fixed task veloc¬ 
ity command along u R requires joint-space velocities 
along v R that grow unboundedly in proportion to the 
factor 1 /or. At the singular configuration, the u R direc¬ 
tion becomes unfeasible for the task variables and v R 
adds to the null-space velocities of the arm. 

According to the above, two main problems are 
inherently related to the basic inverse differential kine¬ 
matics solution (10.13), namely: 

• At near-singular configurations, excessive joint- 
space velocities may result, due to the component 
of t along the direction which becomes unfeasible 
at the singularity. 


• At the singular configuration, discontinuity of the 

joint-space solution is experienced if t has a non¬ 
null unfeasible component. 

The same is obviously true for the complete inverse 
solution (10.12). 

Both the above problems are of major concern 
for kinematic control of manipulators, where the com¬ 
puted joint-space velocities must actually be executed 
by the robot arm. This has motivated the develop¬ 
ment of modified inverse differential mappings, so as 
to ensure proper behavior of the manipulator through¬ 
out its workspace independent of the configuration of 
the robot. A reasonable approach is to preserve the 
mapping (10.13) far from singularities and to set up lo¬ 
cal modifications of it only inside a region enclosing 
the singular configuration; the definition of the region 
depends on a suitable measure of distance from the 
singularity, while the modified mapping must ensure 
feasible and continuous joint velocities. 

Modification of the Planned Trajectory 
One way to tackle the problem of singularities is to 
act at the planning level by either designing trajectories 
that do not incur in unavoidable singularities or assign¬ 
ing task-space motion commands that are feasible for 
the robot arm. However, these solutions rely on perfect 
task planning and cannot be used in real-time sensory 
control applications, in which motion commands are 
generated online. 

Because singularities are a function of configu¬ 
ration and not workspace location, in general, it is 
not possible to avoid singularities by restricting the 
workspace trajectory of the manipulator. There do exist 
cases where avoidance of singular configurations at the 
motion planning level is relatively simple, e.g., when 
dealing with those associated with workspace bound¬ 
ary such as the elbow singularity of an anthropomorphic 
arm. However, this approach may be difficult for those 
singularities, like the wrist one, that may occur every¬ 
where in the workspace. 

Another possibility is to perform a joint-space inter¬ 
polation when the planned trajectory is close to a singu¬ 
larity [10.14]; however, this may cause large errors in 
tracking the originally assigned task-space motion. 

Acting in the task space, a method based on time- 
scale transformation is presented in [10.15], which 
slows down the manipulator’s motion when a singular¬ 
ity is approached; this technique, however, fails at the 
singularity. 

Because a task-space robot control system must 
be able to guide the motion of the manipulator safely 
through singularities, considerable research effort has 
been devoted to the derivation of well-defined and 
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continuous inverse kinematic mappings instead of the 
above ad hoc approaches. 

Removal of Dependent Rows or Columns 
of the Jacobian Matrix 

The first requirement of solution (10.1 3) is the availabil¬ 
ity of a general algorithm to compute the pseudoinverse 
of J t also when the Jacobian is singular. Several tech¬ 
niques presented in the literature can be arranged in this 
framework; they consist either in removing the unfeasi¬ 
ble end-effector reference components [10.16] or in us¬ 
ing nonsingular blocks of the Jacobian matrix [10.10]. 
The main problem with this type of solution is the spec¬ 
ification of the directions of unfeasible velocities in 
a systematic way and the need to have smooth transi¬ 
tions between the usual inverse kinematic algorithm and 
the algorithms used close to the singularities. 

A systematic procedure to compute the pseudoin¬ 
verse of the Jacobian matrix can be devised by taking 
advantage of a kinematic analysis of the manipulator 
structure because for many manipulators it is possible to 
identify and describe classes of singular configurations 
in suitably defined link-fixed frames. This approach has 
been demonstrated for a six-degree-of-freedom elbow 
geometry in [10.17, 18], 

As for the continuity of the solution across a sin¬ 
gularity, it must be observed that, while the singular 
vectors change very little in the neighborhood of the 
singularity, the term 

1 T . 

— v M u M t 

Gm 

suddenly disappears from (10.16) when R becomes less 
than M at the singular configuration. 

One possibility to avoid this problem is to make 

0 

in the neighborhood of the singularity, which means 
avoiding commanding task velocities along the direc¬ 
tion that becomes unfeasible at the singular configu¬ 
ration. This would require some method for efficiently 
computing the singular subspace associated with near¬ 
zero singular value(s) as described in the upcoming sec¬ 
tion on numerical filtering. 

Independently from the assigned t, continuity of 
the pseudoinverse solution can be ensured by treat¬ 
ing the manipulator as singular in a suitably defined 
region around each singularity through a modified Ja¬ 
cobian J t , so that M — R extra degrees of freedom are 
available [10.17-19]; these can be used without signif¬ 
icantly affecting the end-effector velocity, because the 
modified Jacobian J t approximates J t inside the region. 


The described approach is difficult to generalize for 
multiple singularities. For the typical anthropomorphic 
geometry of industrial manipulators, however, only the 
wrist singularity is of primary concern because it may 
occur everywhere in the workspace, while the elbow 
and shoulder singularities are naturally characterized in 
the task space and thus can be avoided during planning. 

Regularization/Damped Least-Squares 
Technique 

The use of the damped least-squares technique in the in¬ 
verse differential kinematics problem has been indepen¬ 
dently proposed in [10.9, 20]. The method corresponds 
to solving the optimization problem 

min(||f — Jt ?|| 2 + A 2 1| 9 1| 2 ) , (10.17) 

4 

which results in a solution q that minimizes the end- 
effector tracking error from the set of all joint veloci¬ 
ties that do not exceed ||g||. Here, A e R is the damping 
factor that represents the trade-off between the least- 
squares and the mimimum-norm properties. When A 
is zero, the solution to (10.17) and (10.13) become 
identical. 

The solution to (10.17) can be written in two equiv¬ 
alent forms 

q = J^(JtJ^ + A 2 I) -1 f , ( 10 . 18 ) 

q = (Jt'Jt + A 2 I ) _1 J^f . (10.19) 

The computational load of (10.18) is lower than that 
of (10.19), because usually N>M. In the remainder, 
we will refer to the damped least-squares solution as 

q = J*(q)i, (10.20) 

whenever explicit specification of the computation used 
is not essential. 

Condition (10.17) implies consideration of accu¬ 
racy and feasibility at the same time when choosing 
the joint-space velocity required to match the given 
t. In this regard it is essential to suitably select the 
value to be assigned to the damping factor: small val¬ 
ues of A give accurate solutions but low robustness to 
the occurrence of singular and near-singular configura¬ 
tions; high values of A result in low tracking accuracy 
even when a feasible and accurate solution would be 
possible. 

In the framework of singular value decomposition, 
the solution ( 10 . 20 ) can be written as 

i= 1 1 
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Remarkably, we have 

3l(J?) = 2l(J t + ) = ^(J.) 

= span{v 1 ,...,t; fi } , 

JV (J*) = FK (jJ) = R 1 - (J t ) 

= span{H fi+1 . u M } • 


As with Jf", if the Jacobian matrix is full-rank, the 
range space of J* is an M-dimensional subspace of 
and the null space of J* is trivial; in a singular con¬ 
figuration, the range space of J* is an A’-dimcnsional 
subspace of R w , and an M — R-dimensional null space 
of J* exists. 

It is clear that, with respect to the pure least- 
squares solution (10.13), in (10.21) the components for 
which <7; A are little influenced by the damping fac¬ 
tor, because in this case 

07 ^ J_ 

of + A 2 ay ' 

On the other hand, when a singularity is approached, 
the smallest singular value tends to zero while the as¬ 
sociated component of the solution is driven to zero 
by the factor a,-/A 2 ; this progressively reduces the 
joint velocity required to achieve near-degenerate com¬ 
ponents of the commanded t. At the singularity, the 
solutions (10.20) and (10.13) behave identically as 
long as the remaining singular values are significantly 
larger than the damping factor. One can show that the 
maximum gain factor of 1/(2A) relating the task veloc¬ 
ity component along m, to the resulting joint velocity 
along Vj occurs when a,- = A. 

Selection of the Damping Factor. According to 
above, A determines the degree of approximation in¬ 
troduced with respect to the pure least-squares solution 
given by the pseudoinverse. An optimal choice for A 
requires consideration of the smallest non-null singu¬ 
lar value experienced along the given trajectory and of 
the minimum damping needed to ensure feasible joint 
velocities. 

To achieve good performance in the entire manipu¬ 
lator’s workspace the use of a configuration-varying 
damping factor has been proposed [10.9]. The natural 
choice is to adjust A as a function of some measure of 
distance from the singularity at the current configura¬ 
tion of the robot arm. Far from singular configurations 
feasible joint velocities are obtained; thus the accuracy 
requirement prevails and low damping must be used. 
Close to a singularity, task velocity commands along the 
unfeasible directions give large joint velocities and the 
accuracy requirement must be relaxed; in these cases, 
high damping is needed. 


A first proposal was to adjust the damping fac¬ 
tor as a function of the manipulability measure [10.9], 
Because a more effective measure of distance from sin¬ 
gular configurations is the smallest singular value of the 
Jacobian matrix [10.5], its use has been considered later 
for building a variable damping factor. 

If an estimate a M of the smallest singular value is 
available, the following choice for the damping factor 
can be adopted [10.21] 


A 2 



when o M > e 
otherwise, 


( 10 . 22 ) 


which ensures continuity and good shaping of the solu¬ 
tion. In (10.22), £ defines the size of the singular region, 
in which damping is used, and A max sets the maximum 
value of the damping factor, which occurs just at the 
singularity. 


Numerical Filtering. As can be recognized 
from (10.21), the damping factor affects the accu¬ 
racy of the solution along each end-effector velocity 
component; however, the sole unfeasible direction is 
responsible for the (possible) loss of tracking abil¬ 
ity. To overcome this problem, selective filtering of 
the end-effector velocity components was proposed 
in [10.6]. The method can be generalized as follows; if 
estimates m, of the output singular vectors associated 
with the M — K smallest singular values - which give 
the components to become unfeasible - are available, 
the solution can be written in the form 


? = J^(j t J^ + A 2 I+/l 2 £ u t u) 

i=K +1 


i, (10.23) 


where ft provides the largest contribution to damping 
only along the unfeasible components. This solution is 
an approximation to applying a greater selective filter¬ 
ing to the near singular components, i. e., 


E 


u i t ; 

^r- v ' Ult 


+ 


E 

i=K+l 


or + A 2 + 1 


:V,uji. 


(10.24) 


where the quality of the approximation is based on 
the accuracy of the estimates of u in (10.23). No¬ 
tice that K < R; nevertheless, a nonzero value of A is 
kept to guarantee satisfactory conditioning of the map¬ 
ping (10.23) even for incorrect estimates of the output 
singular vectors. 

Similarly, additional damping can be provided to 
the joint-space velocity components given by the in- 
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put singular vectors associated to the smallest singular 
values, since they are close to becoming the null-space 
velocities [10.22]. The solution can be generalized in 
the form 

q= (jJj t + X 2 I+p 2 J2 Wj) (10.25) 

V i=K +1 / 


where f) provides the largest contribution to damp¬ 
ing only along the N — K estimated null-space velocity 
components v t . Once again, this solution is an approx¬ 
imation to applying a greater selective filtering to the 
near singular components, i. e.. 




»7 

of + A 2 


T 

VjUj 


t 


,=r +1 -. 2 + A2 +^ 


2 Vjll] t , 


(10.26) 


where the quality of the approximation is based on the 
accuracy of the estimates v, in (10.25). Also in this case 
a nonzero value of A is kept to guarantee satisfactory 
conditioning of the mapping (10.25) even for incorrect 
estimates of the input singular vectors. 

Comparison of (10.26) with (10.24) shows that, in 
the case of accurate estimate of the singular vectors, the 
solutions (10.23) and (10.25) behave identically. 


10.3.3 Joint Trajectory Reconstruction 


When solving the inverse kinematics problem at the 
first-order differential level, one obtains the joint ve¬ 
locity profile q(t) that corresponds to an assigned end- 
effector task velocity profile t(t)\ however, the robot 
motion controller needs reference position trajectories 
in addition to the velocity references for the joints. 
The reconstruction of the joint-position profile from the 
joint-velocity profile obtained from an end-effector task 
velocity profile t(t) realizes a kinematic inversion that 
can be seen as an inverse kinematics algorithm. 

If the joint-velocity profile were completely speci¬ 
fied (e.g., by its analytical expression) the correspond¬ 
ing joint-position profile could be obtained from time 
integration 

t 

q(t) = q(t 0 ) + J q(r) dr . (10.27) 

fo 

However, digital implementation of the robot control 
system makes it more likely that a discrete-time se¬ 
quence of samples qt of the computed joint velocities 
at the time instants 4 will be available, i. e., 

qk = q{tk) ■ 


For this reason, a discrete-time numerical approxima¬ 
tion of the continuous-time integral (10.27) must be 
suitably devised. 

Accurate discrete-time approximations of a conti¬ 
nuous-time integral usually require a trade-off between 
the complexity of the interpolating algorithm and the 
length of the time step. In real-time applications, such 
as robot motion control, high-order interpolation results 
in large finite-time delays that degrade the dynamic per¬ 
formance of the control loop. This time delay can be 
reduced by suitably shortening the time step; in fact, if 
the time step is sufficiently short even a low-order inter¬ 
polation may give acceptable accuracy of the numerical 
integration. It is typical to use first-order interpolation, 
i. e., an Euler forward rectangular rule that transforms 
the integral (10.27) into 

k— 1 

qk = qo + ^qi, At, (10.28) 

h =0 

where At is the time step. Equation (10.28) is usually 
written in the more effective recursive form 


qk = qk-\ + qk-i At. 


Closed-Loop Inverse Kinematics (CLIK) 

No matter what kind of interpolation is used, the 
small though unavoidable error suffered at each nu¬ 
merical integration step accumulates, leading to long¬ 
term drifting of the reconstructed profile from the 
exact joint-position profile. Another source of error 
affecting any integral reconstruction method is the 
possible uncertainty in the initial value of the joint 
position. 

Algorithmic solutions that overcome these prob¬ 
lems are based on the use of a feedback correction term; 
these are termed closed-loop inverse kinematics (CLIK) 
algorithms [10.23]. Considering, e.g., the case of first- 
order kinematics, the joint velocity at the fc-th time in¬ 
stant is computed as (compare with ( 10 . 12 )) 

qk = J,' (qk) {ik + K [f k - (?*)]} 

+ [l-J t t («*)Jt(«*)] q ok , (10.29) 

where K is a constant positive-definite gain matrix. 

Second-order CLIK algorithms are also available 
that solve for joint positions, velocities, and acceler¬ 
ations [10.24,25]. The CLIK approach was originally 
proposed in [10.26] and [10.27] based on the Jacobian 
transpose in lieu of the pseudoinverse, which provides 
remarkable computational savings and provides inher¬ 
ent singularity handling capabilities [10.28]. 
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10.4 Redundancy Resolution via Optimization 


For a kinematically redundant manipulator, the inverse 
kinematics problem admits an infinite number of so¬ 
lutions, so that a criterion to select one of them is 
needed. In this section we consider the problem of re¬ 
dundancy resolution via optimization at the first-order 
differential kinematics level. Before discussing algo¬ 
rithmic schemes for computing joint velocities, we 
provide a short review of possible performance criteria. 

10.4.1 Performance Criteria 

The availability of degrees of freedom in excess of those 
strictly needed to execute a given task can be used to 
improve the value of performance criteria during the 
motion. These criteria may depend on the robot joint 
configuration only, or also involve velocities and/or ac¬ 
celerations. 

Among the additional objectives that can be pursued 
by defining a suitable criterion, arguably the most pop¬ 
ular is singularity avoidance. In fact, one of the original 
motivations for introducing kinematic redundancy was 
to increase the size of the workspace the manipulator 
could reach without being in a singular configuration 
{unavoidable singularities) for achieving general mo¬ 
tion trajectory tracking tasks. A discussion of avoidable 
and unavoidable singularities in redundant manipula¬ 
tors can be found in [10.29]. If the assigned end-effector 
task does not pass through unavoidable singularities, it 
is in principle always possible to compute a joint trajec¬ 
tory along which the task Jacobian J, is continuously 
full-rank. To this end, possible performance criteria 
are the configuration-dependent functions introduced 
in Sect. 10.2.2 that characterize the distance from singu¬ 
larities, i. e., the manipulability measure, the condition 
number, and the smallest singular value of J t . Maxi¬ 
mizing (or keeping as large as possible) these functions 
during the motion is a reasonable plan in order to avoid 
singular configurations during the motion. 

Because kinematic inversion can produce arbitrarily 
large joint velocities in the vicinity of singular con¬ 
figurations, a conceptually different possibility is to 
minimize the norm of the joint velocity generated by the 
redundancy resolution scheme. However, this approach 
guarantees singularity avoidance only if such a norm is 
minimized in an integral sense along the motion of the 
manipulator. Local minimization of the norm [10.10] 
does not lead to singularity avoidance in any practical 
sense [10.29]. 

Redundancy can be also used to keep the manipu¬ 
lator linkage away from undesired regions of the joint 
space, for example, mechanical joint limits that are typ¬ 
ically present in robot manipulators may be avoided by 


minimizing the cost function [10.30] 

•_ j Vvi.max * 11 , min/ 


where [<^ m in , qt, max] is the available range for joint i and 
< 7 ^mid is its midpoint. Another interesting application is 
obstacle avoidance, which can be enforced by minimiz¬ 
ing suitable artificial potential functions defined on the 
basis of the image of the obstacle region in the configu¬ 
ration space [10.31,32]. 

Many other performance criteria have been pro¬ 
posed in the literature; some of them are mentioned in 
Sects. 10.5 and 10.8. 


10.4.2 Local Optimization 


The simplest form of local optimization is represented 
by the pseudoinverse solution (10.13), which provides 
the joint velocity with the minimum norm among those 
that realize the task constraint. Clearly, the joint move¬ 
ment generated by this locally optimal solution does not 
provide global velocity minimization along the entire 
manipulator motion. This means that, despite the local 
minimization of joint velocities, singularity avoidance 
is not guaranteed [10.29]. 

Another possibility is to use the general solu¬ 
tion (10.12), choosing the arbitrary joint velocity qo 
in the direction of the antigradient of a scalar config¬ 
uration-dependent performance criteria H(<?) that one 
would like to minimize 

?o = -1 h VH( ? ), (10.30) 

where ku is a scalar step size and VH(^) denotes 
the gradient of H at the current joint configuration. 
This leads to the following redundancy resolution 
scheme [10.30] 

q = jJi-kad-ijDVRlq) . (10.31) 

Because its second term is the projection of the anti¬ 
gradient of H in the null space of the task Jacobian, 
the above expression is reminiscent of the projected 
gradient method for constrained minimization [10.33]. 
In particular, it can be shown [10.34] that the inverse 
kinematic solution (10.31) minimizes the complete 
quadratic function 

Uq. q) = l ^q T q + k u q T VH(^) 

at the current configuration q. Thus, (10.31) represents 
a natural trade-off between the unconstrained local 




Redundant Robots I 10.5 Redundancy Resolution via Task Augmentation 233 


minimization of the performance criteria H (which 
would lead to choose q = — f'uVH(r/)) and the satis¬ 
faction of constraint (10.2) by a minimum-norm joint 
velocity. 

The choice of the step size % is critical for the per¬ 
formance of the redundancy resolution scheme (10.31). 
In particular, a small value of the step size may slow 
down the minimization of the performance criteria, but 
on the other hand a large value may even lead to an 
increase of H (recall that the antigradient is the local 
direction of maximum decrease). In practice, one may 
use a simplified line search technique such as Armijo’s 
rule [10.33] to identify at each configuration an appro¬ 
priate value of kft in a reasonable time. 

10.4.3 Global Optimization 

The main advantage of the redundancy resolution 
scheme (10.31) is its simplicity: if the computation 
of VH(z/) and k\\ is efficient, such a scheme is a realistic 
option for real-time kinematic inversion. Its disadvan¬ 
tage is to be found in the local nature of the optimization 


process, which can lead to unsatisfactory performance 
over long tasks, for example, use of (10.31) with H = 
—fi (the manipulability measure) will perform better 
than the simple pseudoinverse solution, but still cannot 
guarantee singularity avoidance. 

It is therefore natural to consider the possibility of 
selecting qo in (10.12) so as to minimize integral criteria 
of the form 

ft 

f H(</) d t. 

ti 

defined over the duration [t;, Zf] of the entire task (e.g., 
the integral of manipulability along the motion). Un¬ 
fortunately, the solution of this problem (naturally for¬ 
mulated within the framework of calculus of variations) 
may not exist, and, in general, admits no closed form. 
One way to make the problem solvable is to include 
a quadratic form in the joint velocities or accelerations 
inside the integral. However, this is more easily done at 
the second-order kinematic level (Sect. 10.5). 


10.5 Redundancy Resolution via Task Augmentation 


Another approach to redundancy resolution consists of 
augmenting the task vector so as to tackle additional 
objectives expressed as constraints. In this section, we 
review the basic task augmentation techniques for solv¬ 
ing the first-order differential kinematics (10.2). 

10.5.1 Extended Jacobian 


The extended Jacobian technique, proposed by Bail- 
lieul [10.35] and later revisited by Chang [10.36], en¬ 
forces an appropriate number of functional constraints 
to be fulfilled along with the original end-effector task 
so as to identify a single solution among the infinite 
ones that are compatible with the end-effector task. 

Let us consider an objective function g(q) to be op¬ 
timized and let also Nj t (q) be a matrix spanning the null 
space of J t at a nonsingular configuration q. e.g., 

N Jt = I — Jt"Jt • 


It can be recognized that, for a given to, if qo is a con¬ 
figuration at which the function g(q) is at an extreme 
under the constraint to = k { (qo), one has 


dg(g) 

^ q=qo 


N Jt (?o) = 0 T 


(10.32) 


If the Jacobian J t has full rank M, then Nj t has rank 
N—M\ therefore, (10.32) yields a set of N — M inde¬ 


pendent constraints that can be written in vector form 
as 


h{q) = 0; 

for example, these can be obtained by taking the scalar 
product of the gradient dg(q)/dq by each of the N — 
M vectors constituting a base of the null space of J t , 
i. e., 


Hq) = ( v M+i(q)---V N (q))^ . 

At this point, the condition (10.32) implies that the 
equation 


(k t (q 0 )\ = ( 

\h(qo)) 

is satisfied. 

For motion starting from to with qo that tracks a tra¬ 
jectory t(t) by keeping g(q) extremized at each time one 
has 


k t (9(0)\ = (t(t)\ 

h(q(t))) 
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By differentiating both sides with respect to time 
one obtains 

dh(q) j q = ^ , (10.33) 

where the matrix premultiplying the vector q is square 
and is called the extended Jacobian J ext . 

Therefore, if the initial configuration qo extrem- 
izes g(q ) and provided that J ex i does not become sin¬ 
gular, the time integral of the inverse mapping 

« = (10-34) 

tracks the assigned end-effector trajectory t(t) propagat¬ 
ing joint configurations that extremize g(q). 

The extended Jacobian method has a major advan¬ 
tage over pseudoinverse techniques of the form (10.13) 
in that it is cyclic (Sect. 10.6). Moreover, solu¬ 
tion (10.34) can be made equivalent to (10.12) via 
suitable choice of the vector qo [10.35, 37]. One relative 
disadvantage is its performance near algorithmic singu¬ 
larities (Sect. 10.5.3). 

10.5.2 Augmented Jacobian 

Another approach, the so-called task-space augmenta¬ 
tion, introduces a constraint task to be fulfilled along 
with the end-effector task; then, an augmented Jacobian 
matrix is set-up whose inverse gives the sought joint 
velocity solution. The concept of task-space augmen¬ 
tation has been independently introduced by Sciavicco 
and Siciliano [10.28,38,39] and Egeland [10.40] and 
later revisited by Seraji in the framework of the config¬ 
uration control method [10.41], 

In detail, let us consider the vector t c = 
(tc,i ■ ■-t c ,p) T that describes the tasks to be ful¬ 
filled in addition to the M-dimensional end-effector 
task t. In the general case P < N — M, although full 
redundancy exploitation suggests the consideration 
of exactly as many additional tasks as the number of 
redundant degrees of freedom, i. e., P = N — M. 

The relation between the joint-space coordinate 
vector q and the constraint-task vector t c can be con¬ 
sidered as a direct kinematics equation 

tc=k c (q ), (10.35) 

where k c is a continuous nonlinear vector function. Ac¬ 
cordingly, it is useful to consider the mapping 

(10.36) 


which can be obtained by differentiating Eq. (10.35). 
In (10.36), t c is the constraint-task velocity vector, 
and J c (<7) = dk c /dq is the Px N constraint-task Jaco¬ 
bian matrix. 

At this point, an augmented-task vector can be de¬ 
fined by stacking the end-effector task vector with the 
constraint-task vector as 



According to this definition, finding a joint configura¬ 
tion q that results in some desired value for t. cl means 
satisfying both the end-effector and the constraint task 
at the same time. 

A solution to this problem can be found at the dif¬ 
ferential level by inverting the mapping 

4=Ja (q)q, (10.37) 

where the matrix 



is referred to as the augmented Jacobian. 

A particular choice for the constraint-task vector 
is t c =h(q), with h defined as explained in 10.4.1, 
which allows the augmented Jacobian method to embed 
the extended Jacobian one. 

10.5.3 Algorithmic Singularities 

The specification of additional goals besides tracking 
the end-effector velocity raises the possibility that con¬ 
figurations exist at which the augmented kinematics 
problem is singular while the sole end-effector task 
kinematics is not; these configurations are referred to 
as algorithmic singularities [10.35]. In terms of the ve¬ 
locity mappings (10.33) and ( 10.37), an algorithmically 
singular configuration is one at which the extended 
and the augmented Jacobians, respectively, are singular 
while J t is full-rank. 

When he introduced task-augmentation redundancy 
resolution methods, Baillieul pointed out that algo¬ 
rithmic singularities are not a specific problem of the 
extended Jacobian technique but that they arise from the 
way in which the constraint task conflicts with the end- 
effector task [10.35,37]. This is easily understandable 
in the simple problem of trajectory tracking with obsta¬ 
cle avoidance: if the assigned trajectory passes through 
an obstacle either the trajectory is tracked or the ob¬ 
stacle is avoided so that both tasks cannot be achieved. 
If the origin of the conflict between the two tasks has 


k = J c (?)9 • 
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a clear meaning the algorithmic singularity may then be 
avoided by keenly specifying the constraint task case by 
case [10.23]. In more general situations analytical tools 
may be useful in finding algorithmic singularities and 
guiding the choice of the constraint function [10.42] or 
in finding configurations that better harmonize the two 
tasks [10.43]. 

In terms of the tasks definitions given by (10.2) 
and (10.36), one can see that the two tasks are in conflict 
when 

R (Jj) n R (jJ) / {0} , (10.38) 

which is the condition for the occurrence of an algorith¬ 
mic singularity. On the other hand, when 

K(jJ)n2i(jT) = {0}, 

the two tasks are compatible becuse the two inverse 
mappings are linearly independent; a special case of 
task compatibility is when 

R (jJ) = R ± (Jj) , (10.39) 

and the two mappings are orthogonal. 

At the algorithmic singularities the augmented Ja¬ 
cobian cannot be inverted but singularity-robust tech¬ 
niques can be adopted. Because the exact solution does 
not exist there are reconstruction errors and these affect 
both the task vectors. To counteract this problem the 
use of weighted damped least-squares has been consid¬ 
ered to invert the augmented Jacobian matrix [10.44, 
45]. A different approach is the so-called task priority 
inverse kinematics. 

10.5.4 Task Priority 

Conflicts between the end-effector task and the con¬ 
straint task are handled in the framework of the task- 
priority strategy by suitably assigning an order of pri¬ 
ority to the given tasks and then satisfying the lower- 
priority task only in the null space of the higher-priority 
task [10.46,47]. In the typical case, an end-effector task 
is considered as the primary task, although examples 
can occur when it becomes the secondary task [10.23]. 
The idea is that, when an exact solution does not exist, 
the reconstruction error should only affect the lower- 
priority task. 

With reference to solution (10.12), the task-prior¬ 
ity method consists of computing qo to suitably achieve 
the P-dimensional constraint-task velocity t c . The pro¬ 
jection of qo onto the null space of J t ensures that the 


lower priority of the constraint task will not affect the 
higher priority end-effector task [10.48]. 

When the secondary task t c is orthogonal to the pri¬ 
mary task i (in the sense of (10.39)) the joint velocity 

q 0 = il(q)i c (10.40) 

would easily solve the problem, because it would al¬ 
ready be a null-space velocity for the primary-task 
velocity mapping (10.2), i.e., projection through Nj t 
would not be needed. However, in the general case the 
two tasks may be compatible but not orthogonal or may 
conflict and there may not exist a joint velocity solu¬ 
tion that ensures the achievement of both t and i c . To be 
consistent with the defined order of priority between the 
two tasks, a reasonable choice is then to guarantee exact 
tracking of the primary-task velocity while minimizing 
the constraint-task velocity reconstruction error t c — J c q ; 
this gives [10.49] 

<7o= [jc(I-j;'Jt)] + (ic-J c J t + i). (10.41) 

Finally, by observing that the null-space projection 
operator is both hermitian and idempotent, the solu¬ 
tion given by (10.12) and (10.41) can be simplified 
to [10.46] 

q = J, + H [jc(I-tfjof (ic- J c Jt + i) • (10.42) 

It can be recognized that the problem of algorith¬ 
mic singularities still remains; in fact, when condi¬ 
tion (10.38) is satisfied the matrix J c (I — J,^J t ) loses 
rank with full-rank J, and J c . However, in contrast to 
the task-space augmentation approach, correct primary- 
task solutions are expected as long as the sole primary- 
task Jacobian matrix is full-rank. Away from the algo¬ 
rithmic singularities the task-priority strategy gives the 
same solution as the task-space augmentation approach; 
this implies that close to an algorithmic singularity the 
solution becomes ill-conditioned and large joint veloc¬ 
ities may result. This problem can be solved by using 
a continuous version of the truncated SVD as described 
in [10.3]. 

Another approach is to relax minimization of the 
secondary-task velocity reconstruction constraint and 
simply pursue tracking of the components of (10.40) 
that do not conflict with the primary task [10.50,51], 
namely 

q = jjt +0-Jt t Jt)J&. 


(10.43) 
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An intuitive justification of this solution can be given 
as follows: The pseudoinverses J t ' and are used 
to solve separately for the joint velocities associated 
with the respective task velocities; the joint veloc¬ 
ity associated with the (secondary) constraint task is 
then projected onto the null space of J t to remove 
the components that would interfere with the (primary) 
end-effector task and finally added to the joint veloc¬ 
ity associated with the end-effector task. As a result, 
a nice property of solution (10.43) is that algorith¬ 
mic singularities are decoupled from the singularities 
Of J c . 

By construction, the solution (10.43) leads to 
larger constraint-task reconstruction errors than solu¬ 


tion (10.42); this is the price to be paid for smoother 
trajectories for the joint velocity when tracking con¬ 
flicting tasks. In any case, a CLIK implementation will 
allow one to recover from the secondary-task tracking 
error due to the algorithmic singularities. In this case, 
the CLIK formulation becomes 

q = + (I - J t + J t )jJ w c , 

with 

w t = it+K t (t-k t (q)) , 
w c = f c + K C ( t c -k c (q )) . 


10.6 Second-Order Redundancy Resolution 


Redundancy resolution at the acceleration level allows 
the consideration of dynamic performance along the 
manipulator motion. Moreover, the obtained accelera¬ 
tion profiles can be directly used as reference signals 
(together with the corresponding positions and veloci¬ 
ties) of a task-space dynamic controller. On the other 
hand, a second-order redundancy resolution scheme is 
invariably more demanding in terms of computational 
load. 

The simplest scheme operating at the acceleration 
level is represented by (10.15), i.e.. the solution with 
the minimum norm among those that realize the task 
constraint (10.8). Similar to the velocity-level pseu¬ 
doinverse solution, the joint movement generated by 
this locally optimal solution does not result in global 
acceleration minimization along the entire manipulator 
motion. Remarkably, however, the use of (10.15) does 
lead to the minimization of the integral index 

if 

J q T qdt, 

ti 

over [ti, ff] provided that the appropriate boundary con¬ 
ditions are satisfied [10.52]. For example, in the case 
of free endpoints (neither joint positions nor velocities 
specified at t\ and tf) the boundary conditions to be sat¬ 
isfied are split and expressed as 

q(t) = i}Kt), t = ti,tf. 

Thus, in spite of the apparent simplicity and elegance of 
the solution (10.15), actual minimization of the above 
integral requires the solution of a two-point boundary 
value problem (TPBVP), which is a computationally 
intensive numerical procedure impractical for real-time 


kinematic control. However, this may be perfectly ac¬ 
ceptable for offline redundancy resolution in an indus¬ 
trial setting. 

More flexibility in the choice of (both local and 
global) performance criteria is obviously obtained by 
considering the full second-order solution (10.14). Let 
the manipulator dynamic model be expressed as 

r = H(q)q+ c(q,q) + T g (q) , (10.44) 

where r is the vector of actuator torques, H is the 
manipulator inertia matrix, c is the vector of centrifu¬ 
gal/Coriolis terms, and r g is the gravitational torque 
vector. Choosing the null-space acceleration in (10.14) 
as 

?o = -[H(I-J t t J t )] t r , (10.45) 

with 

r = H J t ^ (if — j,(jr) -l-c + T g 

leads to the local minimization of the actuator torque 
norm r T r [10.53]. This particular redundancy resolu¬ 
tion scheme performs reasonably over short tasks, but 
may lead to instability (more precisely, very high joint 
torques) in the long run, essentially due to the build-up 
of null-space joint velocities. Note also that the matrix 
product H(I—J t ^J t ) in (10.45) is not full-rank; however, 
its pseudoinverse can be efficiently calculated using the 
procedure given in [10.54]. Minimization of the integral 
joint torque is also possible [10.55]; this solution obvi¬ 
ously avoids the instability problem, but once again the 
solution of a TPBVP is required. 
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Another interesting inverse solution is the following 

q = q) + (I— J t ^ H Ji) H — 'c , (10.46) 

which is slightly modified with respect to the general 
formula (10.14) in view of the use of a weighted pseu¬ 
doinverse. In particular, H is the inertia-weighted task 
Jacobian pseudoinverse, given in the full-rank case by 
the following expression 

10.7 Cyclicity 

A common drawback of redundancy resolution 
schemes based on differential kinematics is the lack of 
cyclicity (also called repeatability)', in general, the joint- 
space trajectory corresponding to a cyclic task space 
trajectory is not cyclic itself (i. e., the final position of 
the joints does not coincide with the initial position). 
This phenomenon is clearly undesirable, because it ba¬ 
sically means that the behavior of the manipulator along 
periodic tasks to be repeated over and over is unpre¬ 
dictable. 

A mathematical condition for cyclicity exists for 
a particular class of redundancy resolution meth¬ 
ods [10.56]. In particular, consider any scheme of the 
form 

q = G t (q)i. (10.47) 

where G t is any generalized inverse of the task Jaco¬ 
bian J t , i. e., an N x M matrix such that J t G t J t = J t 
(the pseudoinverse matrix in (10.13) is a particular 
generalized inverse). Assume that the assigned taskf(t) 
describes a cyclic trajectory in a simply connected re¬ 
gion of the task space, and denote by gii(q) the z'-th 
column of G t . A necessary and sufficient condition 
for (10.47) to generate cyclic joint trajectories is that 

10.8 Fault Tolerance 

One natural use for kinematic redundancy is that of fault 
tolerance, i. e., if there are more degrees of freedom 
than the minimum required to complete a task then one 
should, theoretically, be able to tolerate a joint failure 
and still complete the desired task. While component 
failures for robots employed in structured and benign 
environments where regular maintenance can be per¬ 
formed are relatively rare, there are many important 


The solution (10.46) minimizes 

ff 

I l -q J H(q)qdt, 

h 

i. e., the integral over [ 4 , If] of the manipulator kinetic 
energy [10.52]. Once again, the correct boundary con¬ 
ditions must be used; for example, the case of free 
endpoints leads to 

9(f)=Jt + H^)’ ~t = ti,tf. 


the distribution 

A G M) = span{g t i(?). gm(q)} 

is involutive (i. e., it is closed under the Lie bracket op¬ 
eration). 

It should be emphasized that the involutivity of Ag, 
is a strong condition, because it must be satisfied at 
any configuration. This suggests that most generalized 
inverses are not cyclic. Note also that the above con¬ 
dition for cyclicity depends on the chosen generalized 
inverse (i. e., pseudoinverse, weighted pseudoinverse, 
and so on) as well as on the form of J t , which in turn 
is related to the mechanical structure of the manip¬ 
ulator. This means that cyclicity must be established 
on a case-by-case basis. However, it is possible to 
design a repeatable inverse that can approximate any 
desired generalized inverse over a specified subset of 
the workspace [10.57]. 

As for redundancy resolution schemes which do not 
fall in the class (10.47) - namely, those entailed by the 
general inverse solution ( 10 . 12 ) - they are in general 
not cyclic. In particular, this is true for when local op¬ 
timization is used to solve redundancy, as in (10.31). 
A notable exception is the extended Jacobian method, 
which is always cyclic. 


applications, although less common, where this is not 
true, e.g., in space or underwater exploration and in nu¬ 
clear environments. The failure rates for components 
in such harsh environments are relatively high [10.58] 
and maintenance is not possible. Many of these com¬ 
ponent failures will result in a robot’s joint becoming 
immobilized, i. e., a locked joint failure mode [10.59]. 
Component failures that result in other common fail- 
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ure modes, e.g., free-swinging joint failures [10.60], are 
also frequently transformed into the locked joint failure 
mode by failure recovery mechanisms that employ fail 
safe brakes [10.61]. 

A large body of work on fault-tolerant manipulators 
has focused on the kinematic properties of redundant 
robots, both of serial and parallel structure [10.62], 
These analyses have been performed both on the lo¬ 
cal properties associated with the manipulator Jaco¬ 
bian [10.63,64] as well as the global characteristics 
such as the resulting workspace following a particu¬ 
lar failure [10.65-67], Clearly both local and global 
kinematic properties are related, e.g., if a robot is at 
a workspace boundary then the Jacobian is singular. 

The effect of locked joint failures on the motion 
capabilities of a robotic manipulator are most easily 
characterized when the manipulator Jacobian is repre¬ 
sented as a collection of columns 

Jmxn = ( 3 i 3 2 3 n ) , 

where J, represents the end-effector velocity due to the 
velocity of joint i. For an arbitrary single joint failure at 
joint/, assuming that the failed joint can be locked, the 
resulting (Mx(iV-l)) Jacobian will be missing the/-th 
column, where / can range from 1 to N. This Jacobian 
will be denoted by a preceding superscript so that in 
general 

= ( 31 3 i ■ ■ ■ 3 f —1 3 f +1 • • ■ 3 n)- 

As noted above, the local velocity properties of 
a manipulator Jacobian are frequently quantified in 
terms of the singular values. Most local dexterity mea¬ 
sures can be defined in terms of simple combinations 
of these singular values such as their product (deter¬ 
minant), sum (trace), or ratio (condition number). The 
most significant of the singular values is a M , the mini¬ 
mum singular value, because it is by definition the mea¬ 
sure of proximity to a singularity and tends to dominate 
the behavior of both the manipulability (determinant) 
and the condition number. The minimum singular value 
is also a measure of the worst-case dexterity over all 
possible end-effector motions. One definition of failure 
tolerance is based on the worst-case dexterity following 
an arbitrary locked joint failure. Because 1 o M denotes 
the minimum singular value of f J, it is a measure of 
the worst-case dexterity if joint/ fails. If all joints are 
equally likely to fail, then a measure of the worst-case 
failure tolerance is given by 


Physically, this amounts to minimizing the worst-case 
increase in joint velocity when a joint is locked and 


the others must accelerate to maintain the desired end 
effector trajectory. In addition, maximizing K is equiv¬ 
alent to locally maximizing the distance to a post-failure 
workspace boundary [10.68], To insure that manipula¬ 
tor performance is optimal prior to a failure, one may 
want to further define an optimally failure tolerant Ja¬ 
cobian as having all equal singular values due to the 
desirable properties of isotropic manipulator configu¬ 
rations. Under these conditions, to guarantee that the 
minimum 1 cr M is as large as possible they should all be 
equal. It is easy to show that the worst-case dexterity of 
an isotropic manipulator that experiences a single joint 
failure is governed by the inequality 


if f /( N-M) 

K = mm to M 5 o^j — , 

where a denotes the norm of the original Jacobian. The 
best case of equality occurs if the manipulator is in 
an optimally failure tolerant configuration. The above 
inequality makes sense from a physical point of view 
because it represents the ratio of the degree of redun¬ 
dancy to the original number of degrees of freedom. 
Using the above definition of an optimally failure tol¬ 
erant configuration one can identify the structure of the 
Jacobian required to obtain this property [10.63]. In par¬ 
ticular, one can show that the optimally failure tolerant 
criteria requires that each joint contributes equally to 
the null space of the Jacobian transformation. Physi¬ 
cally, this means that the redundancy of the robot is 
uniformly distributed among all the joints so that a fail- 



Fig. 10.9 A four degree-of-freedom spatial positioning 
manipulator in a configuration that is optimally fault 
tolerant 
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ure at any one joint can be compensated for by the 
remaining joints. 

A simple example of an optimally failure tolerant 
configuration that is easy to visualize is that of a spatial 
positioning manipulator with four joints whose Jaco¬ 
bian is given by 


Jv = 


/— v/3/4 
0 

V 0 


71712 

-7273 

0 


71712 7T7i2\ 
776 7176 
-7172 7172/ 


where J v represents the linear velocity portion of a ma¬ 
nipulator Jacobian. The null space at this configura¬ 
tion is given by (1/2)[1 1 1 1] T , which illustrates that 


each joint contributes equally to the null space mo¬ 
tion, thus distributing the redundancy proportionally 
to all degrees of freedom. If the four possible sin¬ 
gle locked joint failures are considered, one can show 
that 

f 1 

— ^max — ^ ‘ 

/ = 1, .... 4 , 

which satisfies the optimally failure tolerant criterion. 
It is then possible to design physical robot manipula¬ 
tors that possess this Jacobian along with its associated 
optimal kinematic properties. An example of one such 
robot is shown in Fig. 10.9 [10.69]. 


10.9 Conclusion and Further Reading 

Research on kinematically redundant robots has been 
flourishing for over two decades now, and is still very 
active. The number of papers dealing with this subject 
is therefore enormous. The few works cited below are 
simply a small addition to the fundamental contribu¬ 
tions already referenced so far, and constitute by no 
means an exhaustive list. 

The mechanical design of kinematically redundant 
manipulators has been studied in many papers [10.70- 
74]. In particular, the superiority of human-arm-like 
manipulators over conventional 6-DOF robots was first 
advocated in [10.75]. Reconfiguration of this arm so as 
to ensure full mobility in the whole workspace is stud¬ 
ied in [10.76,77], 

A general analysis of the inverse kinematic prob¬ 
lem for redundant manipulators is presented in [10.78]. 

In particular, the geometric structure of self-motions is 
analyzed in [10.79]. 

Weighting the damped least-squares solution for 
guaranteed singularity avoidance in anisotropic end-ef¬ 
fector tasks was proposed in [10.9, 21, 80]. 

In addition to singularity avoidance, redundancy 
has also been exploited to achieve obstacle avoid¬ 


ance [10.37,46,81,82], minimization of the effects of 
joint elasticity [10.83], fault tolerance [10.60], reduc¬ 
tion of impact force [10.84,85], and maximization of 
various dexterity measures [10.4,5,43,86]. A com¬ 
pletely different approach for obstacle avoidance with 
redundant robots is presented in [10.87]. 

A review of redundancy resolution via local opti¬ 
mization is given in [10.88]. A numerically efficient 
alternative for redundancy resolution via local opti¬ 
mization is proposed in [10.89]. A redundancy resolu¬ 
tion approach with somehow intermediate characteris¬ 
tics between local and global optimization is presented 
in [10.90]. Other methods for second-order redundancy 
resolution are discussed, e.g., in [10.91]. Also worth 
citing is the dynamically consistent generalized inverse 
of [10.92], 

The cyclicity issue was first pointed out in [10.93], 
and further investigated, e.g., in [10.94,95]. General 
formalisms for redundancy resolution of vehicle-ma¬ 
nipulator systems subject to nonholonomic constraints 
are described in [10.96, 97]. 


Video-References 
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KUKA LBR iiwa - Kinematic redundancy 

available from http://handbookofrobotics.org/view-chapter/10/videodetails/813 

Free floating autonomous valve turning (task priority redundancy control + task concurrence) 

available from http://handbookofrobotics.org/view-chapter/10/videodetails/814 

Human inspired tele-impedance and minimum effort controllerfor improved manipulation performance 
available from http://handbookofrobotics.org/view-chapter/10/videodetails/815 
Human motion mapping to a robot arm with redundancy resolution 
available from http://handbookofrobotics.org/view-chapter/10/videodetails/816 
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Configuration space control of KUKA lightweight robot LWR with EXARM exoskeieton 

available from http://handbookofrobotics.org/view-chapter/10/videodetails/817 

FlexIRob - Teaching nullspace constraints in physical human-robot interaction 

available from http://handbookofrobotics.org/view-chapter/10/videodetails/818 

Visual servoing control of baxter robot arms with obstacle avoidance using kinematic redundancy 

available from http://handbookofrobotics.org/view-chapter/10/videodetails/819 
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11. Robots with Flexible Elements 


Alessandro De Luca, Wayne J. Book 


Design issues, dynamic modeling, trajectory plan¬ 
ning, and feedback control problems are presented 
for robot manipulators having components with 
mechanical flexibility, either concentrated at the 
joints or distributed along the links. The chapter 
is divided accordingly into two main parts. Sim¬ 
ilarities or differences between the two types of 
flexibility are pointed out wherever appropriate. 

For robots with flexible joints, the dy¬ 
namic model is derived in detail by following 
a Lagrangian approach and possible simplified 
versions are discussed. The problem of computing 
the nominal torques that produce a desired robot 
motion is then solved. Regulation and trajectory 
tracking tasks are addressed by means of linear 
and nonlinear feedback control designs. 

For robots with flexible links, relevant factors 
that lead to the consideration of distributed flexi¬ 
bility are analyzed. Dynamic models are presented, 
based on the treatment of flexibility through 
lumped elements, transfer matrices, or assumed 
modes. Several specific issues are then highlighted, 
including the selection of sensors, the model order 
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used for control design, and the generation of 
effective commands that reduce or eliminate 
residual vibrations in rest-to-rest maneuvers. 
Feedback control alternatives are finally discussed. 

In each of the two parts of this chapter, a sec¬ 
tion is devoted to the illustration of the original 
references and to further readings on the subject. 


The standard assumption underlying robot kinematics, 
dynamics, and control design is that manipulators con¬ 
sist only of rigid bodies (links and motion transmission 
components). This is, however, an ideal situation that 
may be considered valid only for slow motion and small 
interacting forces. In practice, mechanical flexibility in 
robot manipulators is present for two main reasons: the 
use of compliant transmission elements and the reduc¬ 
tion of the mass of the moving links through the use 
of lightweight material and slender design. These two 
kinds of flexibility introduce static and dynamic deflec¬ 
tions between the position of the driving actuators and 
the position of the manipulator end-effector. If flexibil¬ 


ity is not taken into account when considering robot 
design and control, a degradation of the overall ex¬ 
pected performance of the robot typically occurs. 

From a modeling point of view, flexibility can be as¬ 
sumed as concentrated at the robot joints or distributed 
(in different ways) along the robot links. The dynamic 
modeling steps are similar, with the need to introduce 
additional generalized coordinates besides those used 
to describe the rigid motion of the robot arm. How¬ 
ever, the properties of the resulting models are quite 
different from a control point of view. Therefore, the 
case of robots with flexible joints and of robots with 
flexible links are presented in this chapter mostly as 
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separate items, pointing out similarities or structural 
differences wherever appropriate. For both classes of 
flexible robots, the relevant design issues, dynamic 
modeling, inverse dynamics algorithms, and control 
laws for set-point regulation and trajectory tracking will 
be discussed. Indeed, joint and link flexibilities may 


11.1 Robots with Flexible Joints 

The presence of joint flexibility is common in current 
industrial robots, when motion transmission/reduction 
elements such as belts - (as in the selective compliant 
assembly robot arm (SCARA) family), long shafts (as 
in the forearm of the Unimation Puma arm), cables, har¬ 
monic drives, or cycloidal gears are used. The purpose 
of these components is to allow relocation of the actu¬ 
ators next to the robot base, thus improving dynamic 
efficiency, or to guarantee high reduction ratios with 
power-efficient compact inline devices. 

However, when subject to the forces/torques arising 
in normal robot operation, these components are intrin¬ 
sically flexible (e.g., the flexspline in harmonic drives; 
Fig. 11.1) and introduce a time-varying displacement 
between the position of the actuators and that of the 
driven links. Without a specific control action, an os¬ 
cillatory behavior, typically of small magnitude but at 
a relatively high frequency, is observed at the robot end- 
effector level during free motion. Moreover, some form 
of instability (e.g., chattering) may occur in tasks in¬ 
volving contact with the environment. 

Recently, flexible actuation/transmission elements 
have been deliberately selected in soft robots (such 
as those driven by serial elastic actuation (SEA) or 
variable stiffness actuation (VS A), Chap. 21) intended 
for physical human-robot interaction and for achiev¬ 
ing more natural motion characteristics. In fact, this 
form of mechanical compliance guarantees an iner- 



Wave generator Flexspline Circular spline 


Fig. 11.1 The components of a harmonic drive 


be present (and dynamically interact) at the same time. 
Many of the presented results can be extended to this 
case as well. 

In the following, it is assumed that the reader is 
already familiar with basic issues on kinematics, dy¬ 
namics, and control of rigid robots (Chaps. 2, 3, 8). 


tial decoupling between the actuator and the (possi¬ 
bly, lightweight) link, thus reducing the kinetic energy 
involved in undesired collisions with humans. Such 
a safety-oriented mechanical design should be traded- 
off with a more complex control design aimed at 
preserving the same performance of rigid robots in 
terms of speed and accuracy of the end-effector motion 
(Chap. 69). 

Figures 11.2 and 11.3 show two examples of robot 
manipulators with flexible joints, the 8 R robot Dex- 



Fig. 11.2 The cable-driven robot Dexter by Scienzia 
Machinate 
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ter and the 1R lightweight manipulator DLR LWR-III. 
The first robot has motors 3—8 located inside the sec¬ 
ond link, with motion transmitted to the distal links 
through steel cables and pulleys; the second robot has 
a modular structure, with each motor integrated in the 
associated joint and using an harmonic drive as reduc¬ 
tion element. The joint stiffnesses of the Dexter robot 
vary between 120 and 6300Nm/rad depending on the 
joint, while for the DLR LWR-III manipulator, the range 
is 6000—15 000 N m/rad. These numerical values refer 
to deflections and associated torques evaluated after the 
reduction gears. 

Deflection of the flexible transmission elements can 
be modeled as being concentrated at the joints of the 
robot, thus limiting the complexity of the associated 
equations of motion. When compared to the rigid case, 
the dynamic model of robots with flexible joints (and 
rigid links) requires twice the number of generalized co¬ 
ordinates to completely characterize the configuration 
of all rigid bodies (motors and links) constituting the 
arm. 

As a result, the case of flexible joint robots is one ex¬ 
ample in which the number of control inputs is strictly 
less than the number of mechanical degrees of freedom. 
This suggests that the design of control schemes re- 



Fig. 11.3 The lightweight manipulator DLR LWR-III by 
the German Aerospace Center 


alizing standard motion tasks is expected to be more 
difficult than for rigid robots. Moreover, the implemen¬ 
tation of a full state feedback law will require twice 
the number of sensors, measuring quantities that are 
before and after (or across) the joint deformation. On 
the other hand, the motor torques used to command 
the robot as well as the disturbance torques due to 
joint flexibility perform work on the same joint axes 
(they are physically collocated). It should be noted 
that this is one relevant difference with respect to the 
case of flexibility distributed along the links. This char¬ 
acteristic is very helpful for rejecting vibrations and 
controlling the overall robot motion. Roughly speak¬ 
ing, we will be able to use input commands acting 
before the source of flexibility to ensure that output 
variables defined beyond the flexibility behave in a de¬ 
sired way. 

11.1.1 Dynamic Modeling 

Dynamic models that explicitly include joint flexibil¬ 
ity are used to evaluate quantitatively vibratory ef¬ 
fects superposed on the rigid motion, verify whether 
the control laws obtained under the assumption of 
joint rigidity can still work in practice (or to what 
extent they should be modified), and enable the de¬ 
sign of new model-based feedforward and feedback 
controllers. 

We consider a robot with flexible joints as an open 
kinematic chain having N + 1 rigid bodies, the base and 
the N links, interconnected by N (rotary or prismatic) 
joints undergoing deflection, and actuated by N elec¬ 
trical drives. From a mechanical point of view, each 
motor (with its stator and rotor) is an additional rigid 
body with its inertial properties. All joints are consid¬ 
ered to be flexible, though mixed situations may be 
encountered due to the use of different transmission de¬ 
vices. When reduction gearings are present, they are 
modeled as being placed before the joint deflection 
occurs. 

The following standard assumptions are made: 

A1 Joint deflections are small, so that flexibility effects 
are limited to the domain of linear elasticity. 

A2 The actuators’ rotors are modeled as uniform bodies 
having their center of mass on the rotation axis. 

A3 Each motor is located on the robot arm in a position 
preceding the driven link. (This can be generalized 
to the case of multiple motors simultaneously driv¬ 
ing multiple distal links.) 

The first assumption supports the terminology of 
robots with elastic joints often used in the literature. 
The elasticity at joint i is modeled by a spring of stiff- 
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ness Kj > 0, which is torsional for rotational joints and 
linear for translational ones. Figure 11.4 shows a sin¬ 
gle link driven by a motor through a rotational elastic 
joint. Nonlinear stiffness characteristics for the flexible 
joint can also be considered, provided that the map from 
deflection to force is smooth and invertible. Assump¬ 
tion A2 is a basic requirement for the long life of an 
electrical drive, and thus very reasonable. As we will 
see, it implies that the inertia matrix and the gravity 
term in the robot dynamic model will be independent of 
the angular position of the motors. A typical arrange¬ 
ment of motors satisfying assumption A3 is shown in 
Fig. 11.5. The simplest situation is when the i-th motor 
moves link i and is mounted on link i — 1 with its rota¬ 
tion axis aligned with the i-th joint. This is, for instance, 
the case for the LWR-III manipulator. The dislocation 
of the actuating motors along the structure has a great 
influence on the structure of the dynamic equations of 
motion. 

For kinematic and dynamic analysis, 2A frames are 
attached to the 2A moving rigid bodies (links and mo¬ 
tors) in the robot chain: the link frames L, and the 

motor frames R t , for i = 1. N (Fig. 11.5). For the 

definition of the link frames the standard Denavit- 
Hartenberg convention can be used. The frames R, are 
attached to the motor stators, aligned with the axes of 



Fig. 11.4 Schematic representation of an elastic joint 



Fig. 11.5 Arrangement of motors and links in an open kinematic 
chain 


symmetry of the motors and with the z-axis along the 
spinning direction of the rotor. 

Accordingly, 2 N generalized coordinates will be 
needed. A possible set of coordinates is given by 



where q is the A'-vector of link positions and 9 is the 
A-vector of motor (i. e., rotor) positions, as reflected 
through the transmission/reduction gears. This choice 
of variables is quite convenient because: 

1. The model will be formally independent of the re¬ 
duction ratios. 

2. These position variables will have a similar dy¬ 
namic range. 

3. The kinematics of the robot will be a function of 
the link variables q only (these variables are already 
beyond the joint flexibility) so that all issues related 
to direct/inverse kinematics will be identical to the 
case of fully rigid robots. 

For some considerations, it is also useful to define 
the variable 6 m , namely the A-vector of motor positions 
before reduction, which are the quantities directly mea¬ 
sured by encoders mounted on the motors. In the case 
of a motor directly placed on the f-th joint axis, one has 
6 m j = nidi, where n,- > 1 is the reduction ratio at the i-th 
joint. In addition, for i = 1,..., A, the difference 

Si = qi - 6j 

is the deflection at the i-th joint, while 
tj.i = Ki(6i - q t ) 

is the torque transmitted to the i-th link through the 
associated elastic spring (Fig. 1 1.4) - the quantity mea¬ 
sured by a joint torque sensor, whenever present. Note 
that for robots with flexible links, the set (0,8) is typ¬ 
ically used in the dynamic modeling, where 8 is the 
vector of link deflection coordinates. 

Following a Lagrangian approach, the single energy 
contributions to the Lagrangian 

£ = T'(0,0)-TI(0) 

will be derived next. 

The potential energy of the robot is due to gravity 
and joint elasticity. The gravity part is related to the po¬ 
sition of the barycenter of the links (each of mass mi) 
and of the motors (of mass m rj ). Because of assump¬ 
tion A2, the latter will be independent of 0. Thus, 

TIgrav — Lgrav.lmk (</) t Tfg mv ,motor(*Z) • 
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For the elastic part of the potential, because of assump¬ 
tion Al, we have 

'Uelas = i:; (q ~ 0 ) T K(<? - 9 ) , 

K = diag (Ki,...,K n ) . 

As a result, 

U(e) = U gmv (q) + n,Uq-0). 

The kinetic energy of the robot is the sum of the 
link and rotor contributions. For the links, there is no 
difference with respect to the standard rigid robot case 
and it will be sufficient to write in general 

1 T 

r iiak =-q l M h (q)q, ( 11 . 1 ) 

with the positive-definite symmetric link inertia ma¬ 
trix Ml(#). For the rotors, some more details are needed 

N 

^"rotor = ^ ' ^"rotor; 

= J2 ( \ m « v l v n + \ R ‘uV% K ‘Vr^ , 

( 11 . 2 ) 

where v n is the linear velocity of the center of mass of 
the t-th rotor and co n is the angular velocity of the i-th 
rotor body. All quantities in the angular contributions 
in (11.2) are conveniently expressed in the local frame 
Rj. According to assumption A2, the rotor inertia matrix 
is then diagonal 

"Ir, = diag(7 r ixx < Ir iyy , Ir izz ) , 

with I rjja = I riyy , while v n can be expressed as a func¬ 
tion of q and q only. Moreover, due to assumption A3, 
the angular velocity of the i-th rotor has the general ex¬ 
pression 

Rl °>T l = J25nj(9)ii+\ 0 V (11-3) 

i =1 \6m,i / 


where B is the constant diagonal inertia matrix col¬ 
lecting the rotors inertial components I T . around their 
spinning axes, as reflected through the reduction ratios, 
M R (q) contains the rotor masses (and, possibly, the ro¬ 
tor inertial components along the other principal axes), 
and the square matrix S(<?) expresses the inertial cou¬ 
plings between the rotors and the previous links in the 
robot chain. 

A simple example illustrates the derivation and the 
actual expressions of the terms in (11.4). We also in¬ 
clude the reduction elements to show how the rotor 
inertial component around the spinning axis will appear 
in the kinetic energy, weighted by different powers of 
the reduction ratios. Consider a planar robot with two 
rotational elastic joints, a first link of length i\, and 
motors mounted directly on the joint axes. The kinetic 
energies of the two rotors are 


rr _ j n2 _ j M 2/i2 

1 rotori — 2 l nzz U m,\ ~ ^ 1 U l 

^"rotor2 = 2 “I - ^ ^ r 2zS& 1 "F 

= - m n i\q\ + - I r2zz (q 2 i + 2n 2 qi9 2 + n\0\) , 


leading to 


B = 


M r = 


( m n l\ 

V 0 


0 n) 

trizA) 



^■2 zz n2 \ 

0 )' 



SB-'S 1 



In this specific case, the matrix S (as well as Mr) is 
constant. Note that for large reduction ratios the 
dominant inertial effect due to the rotors is given by 
the matrix B. Also, if the second motor was mounted 
remotely on the first joint (as it is often the case in the 
class of SCARA arms), or still close to the second joint 
but with the spinning axis orthogonal to the axis of this 
joint, then the matrix S would be zero. 

In general, as a consequence of assumption A3, 
the matrix S(q) has always a strictly upper triangular 
structure with a cascaded dependence of its nonzero 
elements 


where J n j(q) is the /-th column of the Jacobian relat¬ 
ing the link velocities q to the angular velocity of the 
r-th rotor in the robot chain. By substituting (1 1 .3) into 
(1 1 .2) and expressing 6 in terms of 0 , it can be shown 
that 

Tjotor = \ q 7 [Mr(?) + S( ? )B- 1 S T ( 9 )] q 
+ q J S(q)6 + ~6 T B0 , 


S (<?) = 


/o S12 Si 3 (<? 2 ) Su{qi-q?>) 



Slivte, ■■■ ■ 4 W-l)'' 

o o s 23 

S 24 (<? 3 ) 



SiNiqi . qN— 1) 

0 0 0 

S34 



Swiqi, ■■■ ■ qN— 1) 

0 0 0 


0 

Sn— 2 ,N— 

Sn— 2 ,N(qN-1 ) 

0 0 0 


0 

0 

Sn— l,N 

^0 0 0 


0 

0 

0 

(11.5 
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Summing up, the total kinetic energy of the robot is 


T =- © T Wf(0)0 

n(, 


M (q) 
S T (q) 


with 




M( 9 ) = M l ( 9 ) + M r (q) + S(q)B- l S T (q) . ( 11 . 6 ) 


As anticipated, the total inertia matrix !M of the robot 
depends only on q. 

Using the Lagrange equations, finally yields the 
complete dynamic model 


/M(?) S (q)\ (q\ (ciq.^ + c^q.qj)^ 
\S T (q) B ) \0) V c 2 (q.q) ) 

(g(q) + K(q-0)\ / 0 \ 

V K (0-q) ) \r)’ 

(11.7) 


where the inertial terms (related to the total inertia ma¬ 
trix !M(q)), the Coriolis and centrifugal terms (collec¬ 
tively denoted by c to t( 0 . 0 )), and the potential terms 
(3'U(0)/30) T have been written separately. In par¬ 
ticular. g(q) = (dll gtllw (q)/dq) T while Tj = K(0 — q) is 
the elastic torque transmitted through the joints. 

The first N and the last N equations of the dynamic 
model (11.7) are referred to as the link and the motor 
equations, respectively. 

On the right-hand side of (11.7), all nonconserva¬ 
tive generalized forces should appear. When dissipative 
effects are not considered, only the motor torque r 
producing work on the 0 variable is present (i. e., the 
torques on the motor output shafts, amplified by the 
reduction ratios) in the motor equations. If the robot 
end-effector is in contact with the environment, the 
zero on the right-hand side of the link equations should 
be replaced by Tex, = J T (<7)F, where J(</) is the robot 
Jacobian and F the forces/torques acting from the envi¬ 
ronment on the robot. 

In the presence of energy-dissipating effects, addi¬ 
tional terms appear on the right-hand side of (11.7). For 
instance, viscous friction at both sides of the transmis¬ 
sions and spring damping of the (visco)elastic joints 
give rise to the vector term 


\-F S 0-V(0-q)J 

where the diagonal, positive-definite matrices F ? , F#, 
and D contain, respectively, the viscous coefficients on 


the link side and on the motor side, and the damping 
of the elastic springs at the joints. More general forms 
of nonlinear friction tf can be considered. Note that, 
in principle, friction acting on the motor side can be 
fully compensated by a suitable choice of the control 
torque r, while this is not true for friction acting on the 
link side, due to the noncollocation. 


Model Properties 

All elements in the 2/V-vector c to t(0. 0) of velocity- 
dependent terms in (1 1 .7) are independent of the motor 
position 0. The specific dependence of the A-vectors c, 
Ci, and C 2 follows from the general expression of the 
components of c tot based on Christoffel symbols 


Ctot,,(0. ©) — ^ ® 



dSM 

3©; 


© . 


for i = 1 ,..., 2 N, where 2 M, is the i-th column of the 
total inertia matrix .7H (0). In particular, the veloc¬ 
ity vectors c, and c 2 arise only in the presence of 
a configuration-dependent S(q) matrix. Performing cal¬ 
culations, it can be shown that: 


1. The vector ci does not contain quadratic velocity 
terms in q or 0 , but only mixed quadratic terms #,<j. 

2. When the matrix S is constant (in particular, zero), 
both Ci andc 2 vanish. 

The dynamic model (11.7) also shares some prop¬ 
erties of the rigid case, such as: 

• The model equations admit a linear parametrization 
in terms of a suitable set of dynamic coefficients, in¬ 
cluding joint stiffnesses and motor inertias, which is 
useful for model identification and adaptive control. 

• The Coriolis and centrifugal terms can always 
be factorized as c to t(0. 0) = C(0,0)0, in such 
a way that matrix TAJ — 2C is skew-symmetric - 
a property used in control analysis. 

• For robots having only rotational joints, the gradi¬ 
ent of the gravity vector g(q) is globally bounded in 
norm by a constant. 

Finally, when the joint stiffness is extremely large 
(K — > 00 ), then 0 —> q while tj —s- r. It is easy to check 
that the dynamic model (1 1 .7) collapses in the limit into 
the standard model of fully rigid robots (including links 
and motors). 


Reduced Model 

In general, the link and motor equations in (11.7) are 
dynamically coupled through the elastic torque tj at 
the joints, but also (at the acceleration level) via the in¬ 
ertial components of matrix S (q) - usually a path of 
low energy transfer. The presence and actual relevance 
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of these inertial couplings depend on the kinematic ar¬ 
rangement of the manipulator and, in particular, on the 
specific placement of the motors and transmission de¬ 
vices. There are cases in which the matrix S is constant 
(e.g., the planar case, as in the previous example, with 
any number of links) or zero (e.g., for a single link with 
elastic joint or for a robot with N = 2 links having the 
two joint axes orthogonal and the motors mounted at the 
joints). As a result, the dynamic equations will simplify 
considerably. 

For a generic robot with elastic joints, one can take 
advantage of the presence of large reduction ratios (with 
the nfs of the order of 100—150) and simply neglect en¬ 
ergy contributions due to the inertial couplings between 
the motors and the links (see again the 2 R planar ex¬ 
ample). This is equivalent to considering the following 
simplifying assumption: 

A4 The angular velocity of the rotors is due only to 

their own spinning, i. e., 

Ri (o n = (0 0 0„,,,) T , i=l,...,N, 

instead of the complete expression (11.3). 

As a result, the total angular kinetic energy of the 
rotors is just ^ 6 B H (or S = 0), and the dynamic 
model (11.7) reduces to 

M (q)'q + c(q, q) + g(q) + K(tf - 6 ) = 0 

B'6+K(6 -q) = t , ( 11 . 9 ) 

with M(<jr) = M| (</) + Mr(^). The main feature of this 
model is that the link and motor equations are dynam¬ 
ically coupled only through the elastic torque tj. In 
addition, the motor equations are now fully linear. 

We note that the complete model (11.7) and the 
reduced model (11.9) display different characteristics 
with respect to certain control problems. In fact, the re¬ 
duced model is always feedback linearizable by static 
state feedback, whereas this is never the case for the 
complete model as soon as a coupling S 0 is present. 

Singular Perturbation Model 
It is interesting to show the two-time-scale dynamic be¬ 
havior that it is induced in robots with elastic joints 
when the joint stiffness K is relatively large, but still 
finite. This behavior can be made explicit by a simple 
linear change of coordinates, namely replacing 6 with 
the joint torque tj. For the sake of simplicity, this is 
illustrated on the reduced model only (without dissipa¬ 
tion). 

Since the diagonal joint stiffness matrix is assumed 
to have large but similar elements, a common scalar fac¬ 


tor 1 /e 2 1 can be extracted as 

K = K = diag i y k\ . K N ^j . 

The slow subsystem is then given by the link equations, 
once they are rewritten as 

M(?)jj+ £•(?, 9 )+#(?) = Tj . (11.10) 

To obtain the dynamics of the fast subsystem, the joint 
torque is differentiated twice, the motor and link ac¬ 
celerations are replaced from (11.9), and the above 
definition of K is used. This leads to 

6 2 Tj =K {B _1 t - [B _1 +M -1 (<7)] Tj 

+M _1 (?)[c(qr,40 +g(?)]} • (11.11) 

For small 6, (1 1.10) and (11.11) represent a singularly 
perturbed system. The two separate time scales govern¬ 
ing the slow and fast dynamics are t and a = t/e, since 

2 .. , d 2 rj d 2 tj 

6 TJ ~ ' 

This model serves as a basis for composite control 
schemes, where the control torque has the general form 

r = x s (q,q,t) + €Tf(q,q, Tj,Tj) . (11.12) 

This includes a slow action r s , designed when ne¬ 
glecting joint elasticity, and an additional action Tf for 
locally stabilizing the fast flexible dynamics around 
a suitable manifold in the state space. It can be verified 
that, when setting e = 0 in ( 11.1 0)-( 11.12), the equiva¬ 
lent rigid robot model is recovered as 

[Mfa) + B] q + c(q, q) + g(q) = r s . 

A similar singular perturbation model (and control de¬ 
sign) can also be derived for robot manipulators with 
flexible links. 

11.1.2 Inverse Dynamics 

Given a desired motion of a robot, we wish to compute 
the nominal torque needed to reproduce this motion in 
ideal conditions (the inverse dynamics problem). Such 
nominal torque can be used as the feedforward term in 
a trajectory-tracking control law. 

For rigid robots, the inverse dynamics is a straight¬ 
forward algebraic computation obtained by replacing 
the desired motion of the generalized coordinates in the 
dynamic model. The minimum requirement for exact 
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reproduction is that the planned motion has a contin¬ 
uously differentiable desired velocity. For robots with 
elastic joints, a motion task can be conveniently ex¬ 
pressed in terms of a desired link trajectory q = q d (t) 
(possibly obtained from the kinematic inversion of 
a desired motion in Cartesian space). The additional 
complexity lies in the fact that not all robot coordi¬ 
nates are directly assigned in this way, so that additional 
derivations should be performed. This will require some 
higher degree of smoothness of the desired trajectory 
<jr d (f) e [0, 7], where the final time T may or may not be 
finite. 

Reduced Model 

Consider first the case of the reduced model (11.7) and 
set n(q,q) = c(q,q) +g(q) for compactness. By evalu¬ 
ating the link equations on the desired link motion 


where e, is the i-th unit vector and M,- is the i-th col¬ 
umn of matrix IYT(</). This and other similar expressions 
can be obtained by symbolic manipulation software. 
From (11.16), it follows that the minimum require¬ 
ment for the exact reproducibility of the desired motion 
is that q d (t) admits a continuously differentiable jerk 
(i. e., that (f) exists in the time interval [0, 7]). Such 
a smoother requirement should not come unexpected, 
in view of the flexible nature of the system. 

Similarly to the rigid case, a recursive numerical 
Newton-Euler algorithm can be defined, which com¬ 
putes efficiently the inverse dynamics torque (11.16) in 
C)(N) asymptotic complexity. For the reduced model of 
robots with elastic joints, the algorithm involves for¬ 
ward recursion of motion variables up to the fourth 
differential order and backward recursion of second 
time derivatives of forces and torques. 


M( 9 d )? d + n (q d , q d ) + Kq d = K0 d , (11.13) 

the nominal position 0 d of the motors associated with 
the desired link motion is readily obtained. The nom¬ 
inal elastic torque at the joints is Tj id = K(0 d — q d )\ 
note that, from (11.13), this quantity can be expressed 
as a function of q d , q d , and q d , which is independent 
of K. Differentiating (11.13) leads to the expression for 
the nominal motor velocity 6 d . 


M(g d )tf d 3] + M(q d )q d + n (q d .q d ) + Kq d = K0 d , 

(11.14) 

where the notation y hi = d'y/dt‘ is used. Differentiating 
once more, we obtain 

M(? d )? d 4] + 2 M( 9 d )?J, ,] + h (q d , q d ) 

+ [M( 9d ) + K] q d = K0 d . (11.15) 

to be used in the motor equations evaluated along 
the desired motion. After simplifications, the nominal 
torque is obtained as 

r d = [M(<jr d ) + B] q d + n (q d , q d ) 

+ BK- I [M( 9d )^ 4] +2M( 9d )^ 3] 

+M(<jr d )ij d + ii (<7 d .? d )J , (11.16) 

where the extra contributions to the nominal torque of 
the rigid case due to joint elasticity can be clearly rec¬ 
ognized. The evaluation of r d involves the computation 
of first- and second-order partial derivatives of the dy¬ 
namic model terms. For instance, one needs to compute 


MMO] = 


E 


3M,-(<7) 


3 q 


9=9dM 




Complete Model 

Some more analysis is needed for the model (11.9). For 
ease of exposition, and without loss of generality, the 
case of a constant matrix S is considered. When evalu¬ 
ating the link equations on the desired link motion 

M (?d)9d + S ^d + H (9d-9d) + K ?d = K °d • (11-17) 

the additional presence of the motor acceleration on 
the left-hand side does not allow solving directly for 
the motor position 6 d as a function of (q d , <jr d , q d ) only. 
However, the strictly upper triangular structure (11.5) 
of S allows one to define 0 d componentwise, using the 
scalar equations in (11.17) recursively. The (V-th equa¬ 
tion is in fact independent of 0 d , 

Mjj(q d )q d +O T 0d+n N (q d ,q d )+K N q dtN = K N 6 djN , 

so that this equation can be used to define 


#dJV —/at (?d-9d'?d) 

and, after double differentiation, its second time deriva¬ 
tive 

dd,N =In (?d.9d’---.?d 41 ) • 

In the (N — l)-th equation, 

Mj_j(<7 d )i7 d + S N — l,N@d,N 
+ n N — 1 {q d ,q d )+ K N —\q dtN —\ = K N — iO d ,N— 1 , 

the acceleration 0 d ,N has already been determined in the 
previous step. Therefore, this equation can be used sim¬ 
ilarly to define 

6 a,n-i =/v- 1 (#d' 9 d>-#d 4] ) 
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and, after two time differentiations, also 

0&.N-1 =fw -1 (? d - ?d-■••- 9d 61 ) • 

Note that whenever Sn—i,n = 0, there will be no in¬ 
crease in the degree of the highest order derivatives 
of q d within the functional dependence of 9 d , n— i • This 
argument also applies recursively to the following steps. 
Proceeding backward through the link equations, the 
scalar computations end up with the definition of 

OdA =/t {q d ,q d . 

and 

a -f"( n „12(JV+1)]\ 

“d.l — Ji < 7 d->< 7 d J > 

where the dependence on the highest possible differ¬ 
ential degree of q d is shown. With 0 d =f"( ■) made 
available in this way, the nominal torque is finally 
computed from the motor equations, again evaluated 
along the desired motion. Using (11.17) to substitute 
for K(0 d — q A ) yields 

r d = [M(<jr d ) + S T ] q d + n (q d ,q d ) 

+ (B + S) 0 d ( 9d .^ d ,..., 9 t 2(A,+ 1>] ). 

( 11 . 18 ) 

As a result, the presence of inertial motor-link cou¬ 
plings considerably increases the complexity of the 
solution to the inverse dynamics problem. The exact re¬ 
production of a desired link trajectory by the nominal 
torque in (11.18) requires that q d (t) has a continu¬ 
ously differentiable (2N + l)-th time derivative (i. e., 
that 9 d 2<W+1> ‘'(f) exists in the time interval [0. T]). For 
rest-to-rest motions, this implies a trajectory plan that 
imposes a very slow start and ending phases. 

We finally remark that the possibility of express¬ 
ing the evolution of the state and input of a system 
algebraically in terms of the evolution of an output 
variable (in our case q) and a finite number of its deriva¬ 
tives is sometimes referred to as the flatness property. 
The above inverse dynamics analysis shows that q is 
a flat output for robots with elastic joints modeled ei¬ 
ther by (11.7) or (11.9). 

Note also that, when a constant q d is considered, 
these computations all provide the same condition for 
the associated motor position 

0 d = q d + K-'g(q d ) (11.19) 

and nominal static torque 

Td=g(? d )- (11.20) 


Presence of Dissipative Terms 
Inclusion of dissipative terms in the inverse dynamics 
deserves some additional comments. Any model of fric¬ 
tional effects acting on the motor side of the transmis¬ 
sions can be included in the computation without extra 
requirements. Friction at the link side should instead be 
described by a smooth model, because of the need to 
differentiate the link equations. Thus, some functional 
approximations may be needed (e.g., replacing discon¬ 
tinuous sign functions with hyperbolic tangents) before 
including this term in (11.13) or (11.17). 

On the other hand, the presence of a nonnegligi- 
ble spring damping D in (11.8) changes the structure 
of the computations. While this reduces the order of 
the derivatives of q d involved, the problem itself will 
become nonalgebraic; in fact, the inverse system will 
require the use of the solution of a dynamical system, 
albeit a simple one. 

Consider for instance the model (11.9), including 
now all the dissipative terms given in (11.8). When eval¬ 
uating the link equations, 

M(? d )ij d + n (q d , q d ) + (D + F (/ ) q d + K? d 

= D<i d + K0 d , 

( 11 . 21 ) 

the motor velocity 6 d additionally appears on the right- 
hand side. Differentiating (11.21) leads to 

D6 d + K6 d = w d , (11.22) 

with 

w d = M( 9 d )^, 3] + [M(? d ) + D + F q j q d 
+ h(q d ,q d ) + Kq d . 

Equation (11.22) represents a first-order linear asymp¬ 
totically stable dynamical system (internal dynam¬ 
ics), with state 0 d and forcing signal w d (t). For 
a given 0 d (O), its solution 0 d (t) and the associated solu¬ 
tion derivative 0 d (t) are needed to evaluate the nominal 
torque in the motor equations. This yields 

T d = M(q d )q d + n (q d ,q d ) + F q tf d + B0 d + F o 0 d , 

where (11.21) has been used to replace the term D((? d — 
q d ) + K(0 d — q A ). In this case, the desired link trajec¬ 
tory q d (t) should have a continuously differentiable 
acceleration (# d should exist, since it is used in the def¬ 
inition of w d ). Note that any initialization 0 d (O) of the 
internal dynamics (11.22) is feasible (i.e., it produces 
a specific torque profile T d (t) yielding the same link 
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motion q d (t)), with the associated motor position # d (0) 
initialized from (11.21). Indeed, we should match the 
actual initial state of the robot. For instance, starting 
from an equilibrium state implies the unique choice 
0 d (O) = O. 

A similar procedure can also be applied to the com¬ 
plete model (11.7) in the presence of spring damping. 
Again a dynamical inverse system will be needed, while 
the smoothness requirement on q d (t) would be even 
more dramatically reduced in that case. 

We finally remark that, for robots with flexible 
links, the inverse dynamics problem gives also rise to 
an internal dynamics, independently of the presence of 
modal damping. Moreover, when specifying a desired 
motion for the tip of the flexible arm, the associated in¬ 
ternal dynamics will be unstable and this critical issue 
has to be tackled to determine a feasible solution. 

11.1.3 Regulation Control 

We now consider the problem of controlling the motion 
of a robot with joint elasticity to a constant configura¬ 
tion. In this problem, no trajectory planning is involved 
and a feedback law should be found that achieves 
asymptotic stabilization of a desired closed-loop equi¬ 
librium. Global solutions (i. e., those valid when start¬ 
ing from any initial state) are indeed preferred. 

From the analysis in the previous section, it should 
be clear that one needs to define only a constant ref¬ 
erence q d (with q d (t) = 0 ) for the link coordinates. 




Fig. 11.6 Bode diagrams of the torque to motor velocity transfer 
function 


From (11.19), a unique reference 0 d for the motor vari¬ 
ables is in fact associated with the desired q d (which, 
in turn, may result from a desired pose of the robot 
end-effector). Furthermore, (11.20) provides the needed 
static torque to be applied at steady state by any feasible 
controller. 

A major aspect of the presence of joint elasticity 
is that the feedback part of the control law can de¬ 
pend in general on four variables for each joint: the 
motor and link position, and the motor and link veloc¬ 
ity. However, in most robots where joint elasticity is 
not explicitly considered in the system design, at most 
two sensors are available for measurement at the joints: 
a position sensor (e.g., an encoder) and, in some cases, 
a tachometer as a velocity sensor. When no velocity 
sensors are present, velocity is typically reconstructed 
by suitable numerical differentiation of high-resolution 
position measurements. Due to the presence of joint 
elasticity, the position/velocity quantities that are ac¬ 
tually measured depend on where these sensors are 
mounted in the motor/transmission assembly. 

A single link driven through an elastic joint is intro¬ 
duced as a paradigmatic case study to show what can 
be achieved using different sets of partial state mea¬ 
surements. This simple situation provides indications 
on how to handle the general multilink case. 

In the absence of gravity, it will be shown that 
a proportional-derivative (PD) controller based only 
on motor measurements is sufficient to achieve the de¬ 
sired regulation task. In the presence of gravity, various 
gravity compensation schemes can be added to the PD 
feedback controller. These control results mimic the sit¬ 
uation of robots with rigid joints, once the convenient 
quantities for use in the feedback have been identified. 

Single Elastic Joint Example 
Consider a single link rotating on a horizontal plane 
(thus, without gravity) and actuated with a motor 
through an elastic joint coupling (Fig. 1 1.4). When vis¬ 
cous friction on motor and link side as well as damping 
across the spring are included, the dynamic model is 

Mq + D{q - 6) + K(q - 8) + F q q = 0 , 

B6 + D{6 - q) + K(6 - q) + F e 0 = r , 

where the same, now scalar, notation of Sect. 11.1.1 
has been used. Since this system is described by lin¬ 
ear equations, Laplace transforms are used to derive the 
transfer functions of interest, namely from input torque 
to motor position 

6is) _ Ms 2 + (D + F q )s + K 

r(s) den(s) 
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and from input torque to link position 

q(s ) Ds + K 
r(s) den(s) 

with the common denominator den(s) given by 

den(.s) = | MBs 3 + [M(D + F g ) + B(D + F q )] s 2 
+ [(M + B)K + (F q + F e )D + F q F e ] s 
+ {F q + Fg)K^s . 

In the case of link position output, the transfer function 
has a larger relative degree (or pole-zero excess). Fig¬ 
ures 11.6 and 11.7 show typical frequency responses in 
the two cases. For clarity, the outputs have been cho¬ 
sen at the velocity level. In the Bode diagram of the 
magnitude for the motor velocity output, note the pres¬ 
ence of an antiresonance/resonance behavior. Similarly, 
there is a pure resonance (more pronounced for weak 
or zero spring damping D) for the link velocity output. 
The phase profile in Fig. 11 .7, with a high-frequency lag 
of 270°, indicates the greater control difficulty encoun¬ 
tered when closing the feedback loop on link quantities. 
In experimental tests on a robot joint, such plots are 
quite characteristic of the presence of concentrated joint 
elasticity and can be used to assess the relevance of this 
phenomenon and for identifying model parameters. 

For the analysis of the stabilizing properties of dif¬ 
ferent feedback arrangements, all dissipative effects 
will be neglected in the following (D = F q = Fq = 0 
is anyway the worst case). Consider first the transfer 
function relative to the motor position output 

6> (.s) I Ms 2 +K 

——\ = - 11 23 

r ( s ) Inodiss [MBS 2 + (M + B)K\S 2 

Apart from a double pole at the origin, this transfer 
function has a pair of imaginary zeros and poles, with 
the zero pair occurring at the so-called locked frequency 



which characterizes the oscillations occurring when the 
motor is locked (8 = 0), e.g., by a high-gain positional 
feedback. This frequency is used to assess the per¬ 
formance limit of a simple PD control for the motor 
variables. In order to achieve enough damping in the 
closed-loop system, the bandwidth should be limited, 
as a rule of thumb, to one-third of a>i. Faster transients 
can be achieved only by taking into account the fourth- 
order dynamics of the elastic joint assembly. Note also 


Link velocity output 

Magnitude (dB) 




Fig. 11.7 Bode diagrams of the torque to link velocity transfer func¬ 
tion 


that the frequency of the zero pair is always lower than 
that of the pole pair in (11.23). This is related to the 
passivity of the mapping from torque to motor veloc¬ 
ity, a property useful for stability and robust or adaptive 
control design. 

Since the control objective is to regulate the link po¬ 
sition output, we are also interested in the open-loop 
transfer function 


g(s) _ _ K _ 

„odiss“ [MB s 2 + (M + E)K] s 2 • 


(11.24) 


This transfer function has no zeros (in fact, for this to 
happen it is sufficient that D = 0) so that the relative 
degree is now four, the maximum possible. We shall see 
that the nonlinear counterpart of the absence of zeros 
in (11.24) will play a relevant role also for trajectory 
control in the multilink case of robots with elastic joints. 

It is worth mentioning that this situation is com¬ 
pletely different from the case in which the elastic 
spring would be moved out of the joint and placed any¬ 
where along the link - a simple one-mode approximate 
model of link flexibility. In that case, the analogous 
transfer function will possess a negative and a positive 
real zero, symmetrically placed w.r.t. the origin (a non¬ 
minimum-phase system). This indicates the criticality 
of a direct inversion of the system input-output map for 
the execution of a desired link trajectory. 

With the desired position given in terms of the link 
variable q d, the most natural choice for the design of 
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a linear stabilizing feedback using one position and one 
velocity variable, is to close a PD loop from the link 
variables, 

r = u q — (K Pq q + K Dq q) , (11.25) 

with position and velocity gains K Pq and K D q , and with 
u q = K Pq q d being the external input used for defining 
the set point. It is easy to verify that the closed-loop 
poles will be unstable no matter how the gains are cho¬ 
sen, so that error feedback from link variables alone 
should be avoided. In a similar way, also the combi¬ 
nation of motor position and link velocity feedback is 
always unstable. 

Another mixed feedback strategy is to use link po¬ 
sition and motor velocity 

T = u q~ {Kp, q q + Ko m 6^ . (11.26) 

This combination corresponds, e.g., to the case of 
a tachometer integrated in a direct-current (DC) motor 
and of an optical encoder placed on the load shaft for 
sensing its position (without any knowledge about the 
relevance of joint elasticity). Use of (11.26) leads to the 
closed-loop characteristic equation 

BMs 4 + MK D , m s 3 + (B + M)Ks 2 
T KK D m s T KK P ^ q = 0 . 

Using Routh’s criterion, asymptotic stability occurs 
if and only if the motor velocity gain Ko, m > 0 and 
the link position gain satisfies 0 < K P q < K, i. e., the 
proportional feedback should not override the spring 
stiffness. The existence of such an upper bound limits 
the usefulness of this scheme. 

Finally, consider performing feedback from the mo¬ 
tor variables 

v = ug — ^Kp rin 9 + Kp> m d^j , (11.27) 

with uq = K P m 6d = K P m qd (due to the absence of 
gravity). The closed-loop system will be asymptotically 
stable provided that both K P _ m and K D m are strictly pos¬ 
itive (and otherwise arbitrarily large). This favorable 
situation lends itself to a convenient generalization in 
the multilink case. 

Note that other partial state feedback combinations 
would be possible, depending on the available sensing 
devices. For instance, mounting a strain gauge on the 
transmission shaft provides a direct measure of the elas¬ 
tic torque rj = K(0 — q) for control use. Strain gauges 
are also useful sensors for flexible links. Indeed, full- 
state feedback may be designed so as to guarantee 


asymptotic stability and improve the transient behav¬ 
ior considerably; however, this would be obtained at the 
cost of additional sensors and requires proper tuning of 
the four gains. 

PD Control Using only Motor Variables 
For the general multilink case in the absence of gravity, 
consider the PD control law based on motor position 
and velocity feedback 

t = K P (0 d -0)-K D 0 , (11.28) 

with symmetric (typically, diagonal) positive-definite 
gain matrices K/. and K 0 . Since g(q) = 0, it follows 
from (11.19) that the reference value for the motor posi¬ 
tion is 0 d = q d (neither a joint deflection is present nor 
an input torque is needed at steady state). 

The control law (11.28) globally asymptotically sta¬ 
bilizes the desired equilibrium state q = 0 = q d , q = 
0=0. This can be shown through a Lyapunov ar¬ 
gument, completed by the application of La Salle’s 
theorem. In fact, a candidate Lyapunov function is given 
by the sum of the total energy of the system (kinetic 
plus elastic potential) and of the control energy due to 
the proportional term (a virtual elastic potential energy) 

V = l -0 J M(Q)Q + l -(q- 0) T K(q - 0) 

+ Uo d -0) T K P (0 d -0)>O. (11.29) 

Computing the time derivative of V along the tra¬ 
jectories of the closed-loop system given by (11.7) 
(or (11.9)) and (11.28), and taking into account the 
skew-symmetry of 2M — 2C, leads to 

V = -0 T K d 0 <0. 

The inclusion of dissipative terms (viscous friction and 
spring damping) would render V even more negative- 
semidefinite. The analysis is completed by verifying 
that the maximum invariant set contained in the set of 
states such that V = 0 (i. e., those with 0 = 0) collapses 
into the desired unique equilibrium state. 

We point out that an identical control law is able 
to globally regulate, in the absence of gravity, robots 
with flexible links to a desired joint configuration. In 
that case, 0 in (11.28) would be the rigid coordinates at 
the base of the flexible links of the robot. 

PD with Constant Gravity Compensation 
The presence of gravity requires the addition of some 
form of gravity compensation to the PD action (11.28). 
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Moreover, this will typically need an additional struc¬ 
tural assumption and may demand some caution in the 
selection of the control gains. 

Before proceeding, we recall a basic property of the 
gravity vector g(q) (under assumption A2, the gravity 
vector in (11.7) is the same appearing in the dynamics 
of the equivalent rigid robot). For robots with rotational 
joints, elastic or not, a positive constant a exists such 
that 


< a , Wqe R N . (11.30) 

The norm of a matrix A (q) is that induced by the Eu¬ 
clidean norm for vectors, i. e.. 


dg(g) 

dq 


II A|| = v/A max (ATA) . 


Inequality (11.30) implies that 


llg(9i)-g(9 2 )ll < a \\q\-qiW > 

(11.31) 


In common practice, robot joints are never unre¬ 
alistically soft. More precisely, they are stiff enough 
to have, under the load of the robot’s own weight, 
a unique equilibrium link position q e associated with 
any assigned motor position 0 e - the reverse of the re¬ 
lationship expressed by (11.19). This situation is by no 
means restrictive, but will be stated for clarity as a fur¬ 
ther modeling assumption: 

A5 The lowest joint stiffness is larger than the upper 
bound on the gradient of gravity forces acting on 
the robot, or 


> a. . 

The simplest modification for handling the presence 
of gravity is to consider the addition of a constant term 
that compensates exactly the gravity load at the desired 
steady state. According to (11.19) and (11.20), the con¬ 
trol law (11.28) is then modified to 

r =K P (0 i -6)-K D 9+g(q i ), (11.32) 


with symmetric, typically diagonal. K/> > 0 (as a mini¬ 
mum) and > 0 , and with the motor reference given 
by 0 d =q d + K~ l g(q d ). 

A sufficient condition guaranteeing that q = q d , 
0 = 0 d , q = 0 = 0 will be the unique globally asymp¬ 
totically stable equilibrium for the system (11.7) under 
the control law (11.32) is that 



K ) 
K + K p) 


> a , 


(11.33) 


with a defined in (11.30). Taking into account the 
diagonal structure of K and K/>, and thanks to assump¬ 
tion A5, it is always possible to fulfill this condition 
by increasing the smallest proportional gain in the con¬ 
troller (or, the smallest eigenvalue of K/> if this matrix 
has not been chosen diagonal). 

In the following, we sketch the motivation for this 
condition and the associated proof of asymptotic stabil¬ 
ity. The equilibrium configurations of the closed-loop 
system are the solutions of 

K(q-0)+g(q) = O, 
K(0-q)-K P (0 d -0)-g(q d ) = O. 


Indeed, the pair ( q d , 0 d ) satisfies these equations. How¬ 
ever, in order to obtain a global result one should 
guarantee that this pair is the unique solution. There¬ 
fore, recalling (11.19). the null term K (0 d —q d ) —g(q d ) 
can be added/subtracted to both equations to obtain 

K(?-g d )-K(0-0 d ) =g(q i )-g(q) 

-K (q-q d ) + (K + K P )(0-0 d ) = 0, 

where the matrix in condition (11.33) can be clearly rec¬ 
ognized. Taking the norms of terms on both sides and 
bounding gravity terms using (11.31), the introduced 
condition (11.33) implies that the pair (q A , 0 d ) is in fact 
the unique equilibrium. To show asymptotic stability, 
a candidate Lyapunov function is built based on the one 
introduced in (11.29) in the absence of gravity 

l gI = T + Tig rav {q) — ilgrav((7d) — (S ~ 9d) f>(*?d) 

- ^(tfcOKT'g^d) >0. 

(11.34) 

The positive-definiteness of V g i and the fact that its 
unique minimum is at the desired state are again im¬ 
plied by condition (11.33). (The last constant term 
in (11.34) is used to set to zero the minimum value of 
V g i at the equilibrium.). By the usual computations, it 

•T 

follows that Vgi = —0 K d 0 < 0 and the result is ob¬ 
tained applying La Salle’s theorem. 

The control law (11.32) is based only on the know¬ 
ledge of the gravity term g(q d ) and of the joint stiff¬ 
ness K. The latter appears in fact in the definition of the 
motor position reference 0 d . Uncertainties in the grav¬ 
itational term g(q d ) and in the joint stiffness K affect 
the performance of the controller. Still, the existence of 
a unique closed-loop equilibrium and its asymptotic sta¬ 
bility are preserved when the gravity bound by a is still 
correct and condition (11.33) holds for the true stiffness 
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values. Indeed, the robot will converge to an equilib¬ 
rium that is different from the desired one - the better 
the estimates K and g(q d ) used, the closer the actual 
equilibrium will be to the desired one. 

PD with Online Gravity Compensation 
Similarly to the rigid robot case, a better transient be¬ 
havior is expected if gravity compensation (or, more 
precisely, its perfect cancellation) is performed at all 
configurations during motion. However, the gravity 
vector in (11.7) depends on the link variables q, which 
are assumed not to be measurable at this stage. It is easy 
to see that using g(9), with the measured motor posi¬ 
tions in place of the link positions, leads in general to 
an incorrect closed-loop equilibrium. Moreover, even 
if q were available, adding g(q) to a motor PD error 
feedback has no guarantee of success because this com¬ 
pensation, which appears in the motor equation, does 
not instantaneously cancel the gravity load acting on the 
links. 

With this in mind, a PD control with online gravity 
compensation can be introduced as follows. Define the 
variable 

0 = 6-K- l g{q d ) , (11.35) 

which is a gravity-biased modification of the measured 
motor position 0 , and let 

T = K P (O d -0)-K D 6+g(O), (11.36) 

where K/> > 0 and K/, > 0 are both symmetric (and typ¬ 
ically diagonal) matrices. The control law (11.36) can 
still be implemented using only motor variables. The 
term g(0) provides only an approximate cancellation 
(though, of a large part) of gravity during motion, but 
leads to the correct gravity compensation at steady state. 
In fact, by using (11.19) and (11.35), it follows that 

9 d := 0 d -K~ l g(q d ) = q d , 
so that g(0 d )=g(q d ). 

Global asymptotic stability of the desired equi¬ 
librium can be guaranteed under the same condi¬ 
tion (11.33) used for the constant gravity compensation 
case. A slightly different Lyapunov candidate is de¬ 
fined, starting again from the one in (11.29), as 

V g2 = V+U g m (q)-'U pm tf) 
- l -g r (q d )K- l g(q d )>0, 

to be compared with (11.34). 


The use of an online gravity compensation scheme 
typically provides a smoother time course and a notice¬ 
able reduction of the positional transient errors, with 
no additional control effort in terms of peak and av¬ 
erage torques. We note that choices of low position 
gains, even when largely violating the sufficient con¬ 
dition (11.33) for stability, may still work, contrary to 
the case of constant gravity compensation. However 
even for increasing values of joint stiffness, in the limit 
for K —> 00 , the range of feasible values of that 
guarantee exact regulation does not extend down to 
zero. 

A possibility to refine the previous online gravity 
compensation scheme, again based only on motor posi¬ 
tion measurement, is offered by the use of a fast iterative 
algorithm that elaborates the measure 6 in order to 
generate a quasistatic estimate q(0) of the current (un¬ 
measured) q. In fact, in any steady-state configuration 
(q s , 0 S ), a direct mapping is defined from q s to 0 S 

0s = h g (q s ) := q s + K -1 g(g s ) . 

Assumption A5 is sufficient to guarantee the exis¬ 
tence and uniqueness also of the inverse mapping q s = 
h^ l (9 s ). For a measured 0 , the function 

q = T(q):=0-K- l g(q ) 

is then a contraction mapping and the iteration 

q i+] = T(q t ) , i = 0,1,2,... 

will converge to the unique fixed point of this mapping, 
which is exactly the value q(9) = h~ l (9). A suitable 
initialization for q 0 is either the measured 0 or the 
value q computed at the previous sampling instant. In 
this way, just two or three iterations are needed to ob¬ 
tain sufficient accuracy and this is fast enough to be 
implemented within a sensing/control sampling interval 
of a digital robot controller. With this iterative scheme 
running in the background, the regulation control law 
becomes 

t =K P (0 d -9)-K D 0+g(q(0)), (11.37) 

with symmetric (diagonal) K/> > 0 and K/j > 0. A proof 
of the global asymptotic stability of this control scheme 
can be given by further modifying the previous Lya¬ 
punov candidates. The advantage of (11.37) is that any 
positive value of the feedback gain K/> is allowed, thus 
recovering in full the operative working conditions of 
the rigid case with exact gravity cancellation. 

With reference to the reduced model (11.9), a fur¬ 
ther improvement can be obtained via a nonlinear PD- 
like controller that removes completely the gravity from 
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the actual motion of the robot links. For this, consider where u is an auxiliary input to be designed, transforms 
the control law the motor equations into 


r = K P [ qi/ - 0 + K-'*( 9 )] - K d ( 9- Kr'giq)) 

+ g(q) + BK~ 1 g(q) 

( 11 . 38 ) 

with symmetric (diagonal) Kp > 0 and > 0. The 
last two terms are those needed to cancel both static 
and dynamic effects due to gravity. Even when as¬ 
sumption A5 does not hold, the control law (11.38) 
guarantees global asymptotic stability of the desired 
equilibrium state. This result is a byproduct of feedback 
linearization design (Sect. 11.1 .4), though much simpler 
than the complete one, and can be proven using again 
a Lyapunov analysis. Since a lower bound on the joint 
stiffness is not required (the joints may be arbitrarily 
soft), the same approach can be used also for robots us¬ 
ing variable stiffness actuation (VSA). The price to pay 
is that an estimate of the link acceleration q is needed 
when computing the term g in (11.38). 

The two online gravity compensation schemes 
(11.36) and (11 .37) realize a compliance control in the 
joint space with only motor measurements. The same 
idea can also be extended to the case of Cartesian 
compliance control by evaluating the direct kinematics 
and the Jacobian (transpose) of the robot arm with 6 
or, respectively, q(0) in place of q. The role of a 
damping action within a compliance/impedance con¬ 
trol scheme can be clearly understood when looking at 
laq iM'JMfrlta and IdaMaEItllEI. 


Full-State Feedback 

When the feedback law is based on a full set of meas¬ 
urements of the robot state, the transient performance 
of regulation control laws can be improved. Taking 
advantage of the availability of a joint torque sen¬ 
sor, we present a convenient design for the reduced 
model (11.9) of robots with elastic joints, including 
spring damping as a dissipation effect. Full-state feed¬ 
back can be obtained in two phases, by combining 
a preliminary torque feedback with the motor feedback 
law in (11.28). 

Using Tj = K(0 — q), the motor equation can be 
rewritten as 

B0 + Tj + DK~*f j = r . 

The joint torque feedback 

r = BB^w + (I - BBg') (rj + DK^'rj) , 

( 11 . 39 ) 


BqO + Tj + DK 1 T J = M . 

In this way, the apparent motor inertia can be reduced to 
a desired, arbitrary small value Bg, with clear benefits 
in terms of vibration damping. For instance in the linear 
and scalar case, a very small B shifts the pair of complex 
poles in (11.23) to a very high frequency, with the joint 
behaving almost as a rigid one. 

Setting now 

u = K P , e (0 d -0)-K Did 0 +g(q d ) 
in (11.39) leads to the state feedback controller 

r = K/>(0d — 0) — K d 0 + K r [g( 9 d ) — Tj] 

-K S Tj+g( 9d ), (11.40) 


with gains 


Ky, = BBg‘Kp.0 , 

K d = BBg 'Kfl g , 

K r = BBg 1 — I 

K s = (BB-'-I) DKT 1 . 

Indeed, the law (11.40) can also be rewritten in terms of 
(0 , q, 0, q), if the torque sensor is not available. How¬ 
ever, keeping this structure of the gains preserves the 
interesting physical interpretation of what the full-state 
feedback controller is able to achieve. 

11.1.4 Trajectory Tracking 

As for rigid robots, the problem of tracking desired 
time-varying trajectories for robots with elastic joints 
is harder than that of achieving constant regulation. In 
general, solving this problem requires the use of full- 
state feedback and knowledge of all the terms in the 
dynamic model. 

Under these conditions, we shall focus on the feed¬ 
back linearization approach, namely a nonlinear state 
feedback law that leads to a closed-loop system with de¬ 
coupled and exactly linear behavior for all the N joints 
of the robot (in fact, the link variables q). The track¬ 
ing errors along the reference trajectory are forced to 
be globally exponentially stable, with a decaying rate 
that can be directly specified through the choice of 
the scalar feedback gains in the controller. This funda¬ 
mental result is the direct extension of the well-known 
computed torque method for rigid robots. Because of its 
relevant properties, the feedback linearization approach 
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can be used as a reference to assess the performance 
of any other trajectory tracking control law, which may 
possibly be designed using less/approximate model in¬ 
formation and/or only partial-state feedback. 

However, in the presence of joint elasticity, the 
design of a feedback linearization law is not straight¬ 
forward. Furthermore, as soon as S 7 ^ 0, the dynamic 
model (11.7) will not satisfy the necessary conditions 
for exact linearization (nor those for input-output de¬ 
coupling) when only a static (or, instantaneous) feed¬ 
back law from the full state is allowed. Therefore, we 
will restrict our attention to the more tractable case 
of the reduced dynamic model (11.9) and sketch only 
briefly the more general picture. 

As a second much simpler approach to trajectory 
tracking problems, we also present a linear control 
design that makes use of a model-based feedforward 
command and a precomputed state reference trajectory, 
as obtained from the inverse dynamics in Sect. 11.1.2, 
with the addition of a linear feedback from the full 
state. In this case, convergence to the desired trajec¬ 
tory is only locally guaranteed, i. e., the tracking errors 
should be small enough, but the control implementation 
is straightforward and the real-time computational bur¬ 
den is considerably reduced. 

Feedback Linearization 

Consider the reduced model (11.9) and let the de¬ 
sired motion be specified by a smooth reference tra¬ 
jectory q A {t) for the robot links. The control design 
will proceed by system inversion, in a similar way 
to the inverse dynamics computations of Sect. 11.1.2 
but using now the current measures of the state vari¬ 
ables (q, 6 ,q, 6) instead of the reference state evolution 
(q d , 0 A ,q d , 0 d ). Notably, there is no need to transform 
the robot equations into their state-space description, 
which is the standard form used in control design for 
general nonlinear systems; we will use the robot model 
directly in its second-order differential form (typical of 
mechanical systems). 

The outcome of the inversion procedure will be the 
definition of a torque r, in the form of a static state 
feedback control law, which cancels the original robot 
dynamics and replaces it with a desired linear and de¬ 
coupled one of suitable differential order. In this sense, 
the control law stiffens the dynamics of the robot with 
elastic joints, irrespective of the fact that K is already 
relatively large (stiff joint) or small (soft joint). The 
feasibility of inverting the system from the chosen out¬ 
put q without causing instability problems (related to 
the presence of unobservable dynamics in the closed- 
loop system, after cancellation) is a relevant property 
of robots with elastic joints. In fact, this is the direct 
generalization to the nonlinear multiple-input multiple- 


output (MIMO) case of the possibility of inverting 
a scalar transfer function in the absence of zeros (see 
also the considerations made on (11.24)). 

Rewrite the link equation in the compact form 

M(q)'q + n(q, q) + K(q-0) = 0 . (11.41) 

None of the above quantities depends instantaneously 
on the input torque r. Therefore, we can differentiate 
once w.r.t. time and obtain 

M(< 7 )? [3] + M (q)q + h(q,q) + K(q-0) = O. 

(11.42) 

Proceeding one step further leads to 

M( 9 ) 9 [4] + 2M( 9 )? [3] + M( 9 )jj 

+ii(q,q) + K(ij -0) = 0 . (11.43) 

where 0 now appears. The motor acceleration is at the 
same differential level of r in the motor equation 

B6 + K(0-q) = T, (11.44) 

and thus, replacing 0 from (11.44), we get 

M(q )# [4] + 2M(#)# [j] + M(q)q + ii(q, q) + Kij 
= KB"” 1 [r-K (6-q)] . 

(11.45) 

We note that, using (11.41), the last term K(0 —q) 
in (11.45) may also be replaced by M(q)q + n(q , q). 

Since the matrix A(<jr) = 1 is always 

nonsingular, an arbitrary value v can be assigned to 
the fourth derivative of q by a suitable choice of the 
input torque r. The matrix A(<jr) is the so-called de¬ 
coupling matrix of the system and its nonsingularity is 
a necessary and sufficient condition for imposing a de¬ 
coupled input-output behavior by nonlinear static state 
feedback. Moreover, (11.45) indicates that each com¬ 
ponent qj of q needs to be differentiated r, = 4 times in 
order to be algebraically related to the input torque r 
(r, is the relative degree of q t , when this is chosen as 
a system output). Since there are N link variables, the 
total sum of the relative degrees is AN, equal to the di¬ 
mension of the state of a robot with elastic joints. All 
these facts taken together lead to the conclusion that, 
when inverting ( 11 .45) to determine the input r that im¬ 
poses q ^ = v, there will be no dynamics left other than 
the one appearing in the closed-loop input-output map. 

Therefore, choose 

t = BK~' |^M(^r) v + a (q, q, q, <jr [3] ) j 
+ [M(q) + B]q + n(q,q) , 


(11.46) 
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Fig. 11.8 Exploded view of a joint of the DLR LWR-III lightweight 
manipulator and its sensor suite 


with 

a(q,q,'q, # [3] ) = + 2M(#)# [3] +ii(q, q) , 

where the terms in a have been ordered according to 
the dependence on increasing orders of derivatives of q. 
It is easy to verify that the control law (11.46) leads to 
a closed-loop system fully described by 

q [4] = v , (11.47) 

i. e., chains of four input-output integrators from each 
auxiliary input Vi to each link position output q t , for 
i = 1,.... N. Thus, the robot system has been exactly 
linearized and decoupled by the nonlinear feedback 
law (11.46). The improvement in performance can 
be appreciated in a comparative way by looking at 
Id aMI.H.im and \<sa mmiMbM 

The complete control law (11.46) is expressed 
as a function of the so-called linearizing coordinates 
(q, q, q, q ^) only. This has led to some misunderstand¬ 
ings in the past, as it seemed that the feedback lineariza¬ 
tion approach for robots with elastic joints would need 
direct measures of the link acceleration q and jerk 
which are impossible to obtain with currently available 
sensors (or would require multiple numerical differen¬ 
tiations of position measures in real time, with critical 
noise problems). 

When considering the latest technology in the field, 
it is now feasible to have a set of sensors for elas¬ 
tic joints measuring in a reliable and accurate way the 
motor position 6 (and, possibly, also its velocity 6 ), 
the joint torque tj =K (6 —q), as well as the link 
position q. For instance, this is the arrangement of sen¬ 
sors available at each joint of the LWR-III lightweight 
manipulator, where expressly designed high-resolution 
incremental encoders of the magneto-resistive type are 
used for the motor position, joint torque sensing is 
based on a full bridge of strain gauges, and a high- 
end capacitive potentiometer is used for the absolute 
link position (Fig. 11.8). Therefore, only one numeri¬ 
cal differentiation is needed in order to obtain a good 
estimate of q and/or tj as well. Note that, depending 
on the specific sensor resolution, it may also be con¬ 
venient to evaluate q using the measures of 6 and r i 
as 6 — K -1 tj. 

With this in mind, it is easy to see that the following 
three sets of AN variables 

(q,q/q,q l3] ) , (q,6,qj), (q,T S ,q,ii) 

are all equivalent state variables for a robot with elas¬ 
tic joints, related by globally invertible transforma¬ 
tions. Therefore, under the assumption that the dynamic 


model is available, we can completely rewrite the feed¬ 
back linearizing control law (11.46) in terms of the 
more conventional state (q,0,q,0) or, taking advan¬ 
tage of a joint torque sensor, in terms of (q , Tj,<jr, Tj). 
In particular, as a byproduct of (11.41) and (11.42), we 
have 

q = M~* (q) [K(0 - q) - n(q , q)\ 

= M -1 ^) [Tj -n(q,q)] , (11.48) 


<7 [31 = M 1 (q) [K(0 -q)-M(q)q-h{q,q)^ 

= M -1 (tf) [rj-M(9)£-«(qr,£)j , 

(11.49) 

where the acceleration q appearing in (11.49) has al¬ 
ready been computed through (11.48). Therefore, the 
exact linearizing and decoupling control law can be 
rewritten in terms of a static state feedback law of the 
form r = r(q, 0 ,q, 6, v) or r = r(q, tj ,q, i j, v). In¬ 
deed, the evaluation of the various derivatives of the 
dynamic model terms that are present in these expres¬ 
sions should be properly organized or customized in 
order to optimize computations. 

Based on the resulting (11.47), the trajectory¬ 
tracking problem is solved by setting 

v = + K 3 -q l3] ) + K 2 (q d - q) 

+ K l (q d -q) + K 0 (q d -q) , (11.50) 


and 
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where it is assumed that the reference trajectory q d (t) is 
(at least) three times continuously differentiable (i. e., 
the fourth derivative gj, 4 -' exists), and the diagonal ma¬ 
trices Ko,..., K 3 have scalar elements K. , such that 

s 4 + K^jS^ + KtjS^ + K\ jS + Kqj , i = 1,..., N , 

are Hurwitz polynomials. In view of the obtained de¬ 
coupling, the trajectory error 

c,(0 = ~ <k(t) 

for the r'-th link satisfies then 

+ K2j'ii + Ki je, + K d je, = 0 ;, 

yielding e, (f) —s- 0 in a global exponential way, for any 
initial state. A series of remarks are in order: 

• When the initial state (< 7 ( 0 ), 0(O),<jr(O), 0(0)) is 
matched with the reference trajectory and its first 
three derivatives at time t = 0 (for this check, 
(11.48) and (11.49) are to be used), exact repro¬ 
duction of the reference trajectory is achieved at all 
times. 

• In the case of a discontinuity of the reference posi¬ 
tion trajectory, or of any of its first three derivatives, 
at a time t* e [0, T], a trajectory error occurs at 
t = t* that will again decay to zero, independently 
for each link and with the prescribed exponential 
rate. 

• The choice of the gains ..., K d j can be made 
by a pole placement technique (equivalent in this 
case to an eigenvalue assignment). Let Ai,..., A 4 
be four poles with negative real parts, possibly 
given in complex pairs and/or coincident, speci¬ 
fying the desired transient of the trajectory error. 
These closed-loop poles will be assigned by the 
unique choice of real and positive gains 

Kij = —(A 1 + A 2 + A 3 + A 4 ) , 

Ki,i = Ai(A 2 + A 3 + A 4 ) + A 2 (A 3 + A 4 ) + A 3 A 4 , 
K\,i = — [-A. 1 A 2 (A 3 + A 4 ) + A 3 A 4 (Ai + A 2 )] , 

Ko,i = A 1 A 2 A 3 A 4 . 

When the values of link and motor inertias are very 
different from each other, or when the joint stiffness 
is very large, the above fixed choice of gains has the 
drawback of generating too large control efforts. In 
those cases, a more tailored set of eigenvalues can 
be assigned scheduling their placement as a func¬ 
tion of the physical data of robot inertias and joint 
stiffnesses. 

• When compared to the computed torque method 
for rigid robots, the feedback linearization control 


for trajectory tracking given by (11.46)—(11.50) re¬ 
quires the inversion of the inertia matrix M(<jr) and 
the additional evaluation of derivatives of the inertia 
matrix and of other terms in the dynamic model. 

• The feedback linearization approach can also be 
applied without any changes in the presence of 
viscous (or otherwise smooth) friction at the mo¬ 
tor and link side. The inclusion of spring damping 
leads instead to a third-order decoupled differen¬ 
tial relation between the auxiliary input v and q. 
leaving thus a /V-dimensional unobservable dynam¬ 
ics in the closed-loop system, which is, however, 
still asymptotically stable. In this case, only input- 
output (and not full-state) linearization and decou¬ 
pling is achieved. 

We conclude with some considerations on exact 
linearization/decoupling by feedback for the general 
model (11.7) of robots with elastic joints. Unfortu¬ 
nately, due to the presence of the inertial coupling 
matrix S(q) between links and rotors, the above con¬ 
trol design can no longer be applied. Consider as an 
illustration the case of a constant S in (11.7), with the 
associated model simplifications. Solving for 0 from 
the motor equations, 6 = B~ 1 (r — tj — S T ij), and sub¬ 
stituting into the link equations leads to 

[M(</) - SB-'S 1 ]? + n(q, q) - (I + SB -1 ) r j 

= —SB - ’r . 

In this case, the input torque r appears already in the 
expression for the link acceleration q. Using (11.6), the 
expression of the decoupling matrix is then 

A( 9 ) = - [M L (q) + Mr(?)] -1 SB” 1 , 

which is never full rank in view of the structure (11.5) 
of S. As a consequence, the necessary condition for 
obtaining (at least) input-output linearization and de¬ 
coupling by static state feedback fails to hold. 

However, by resorting to the use of a larger class 
of control laws, it is still possible to obtain an exact 
linearization and decoupling result. For this purpose, 
consider a dynamic state feedback controller of the form 

r = a{q,q, 0,0,%) + fi(q, q, 0, 0, %)v , 
k = y(q,q,0,6,$) + 8(q,q,9j,$)v , (11.51) 

where £ el 1 ' is the state of the dynamic compensator, 
of, /L y. and 8 are suitable nonlinear vector functions, 
and v e is (as before) an external input used for 
trajectory-tracking purposes. It is possible to show in 
general that a dynamic compensator (11.51) of order 
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at most v = 2N(N — 1) can be designed so as to yield 
a closed-loop system that is globally described by 

? [2(JV+D] = v j (11.52) 

in place of (11.47). Note the coincidence of this differ¬ 
ential order with the one found for exact reproducibility 
of a desired trajectory q d (t) in Sect. 11.1.2. The track¬ 
ing problem is then solved by working on (11.52), with 
a direct generalization of the linear stabilization design 
in (11.50). 

It is interesting to give a physical interpretation for 
this control result. The structural obstruction to input- 
output decoupling of the model (11.7) is related to the 
fact that the motion of a link attached to an elastic joint 
is affected too soon (at the second differential level) by 
torques originating at other elastic joints. This is due to 
the inertial couplings present between motors and links. 
The addition of dynamics (i. e., of integrators) in the 
controller slows down these low-energy paths and al¬ 
lows for the high-energy effects of the elastic torque 
at the local joint (a fourth-order differential path) to 
come into play. This dynamic balancing allows both 
input-output decoupling and exact linearization of the 
extended system (having the robot and the controller 
states). 

Linear Control Design 

The feedback linearization approach to trajectory track¬ 
ing gives rise to a rather complex nonlinear control 
law. Its main advantage of globally enforcing a linear 
and decoupled behavior to the trajectory error dynamics 
should be traded-off with a control design that achieves 
only local stability around the reference trajectory, but 
is much simpler to implement (and may run at higher 
sampling rates). 

For this, the inverse dynamics results of Sect. 11.1.2 
can be used, assuming either the reduced or the com¬ 
plete robot model, with and without dissipative terms. 
Given a sufficiently smooth desired link trajectory q d (t), 
we have seen that it is always possible to associate: 

1. The nominal torque r d (t) needed for its exact repro¬ 
duction. 

2. The reference evolution of all other state variables 

(e.g., 0 d (t), as given by (11.13), or r w (f)). 

These signals define a sort of steady-state (though, 
time-varying) operation for the system. 

A simpler tracking controller combines a model- 
based feedforward term with a linear feedback term 
using the trajectory error. The linear feedback locally 
stabilizes the system around the reference state trajec¬ 
tory, whereas the feedforward torque is responsible for 
maintaining the robot along the desired motion as soon 


as the error has vanished (see some examples on the 
tracking of square paths in l<gf’il'H'im). 

Using full-state feedback, two possible controllers 
of this kind are 

t = r d + K P g(0 A — 0) + K 0 ,o(0d — 0) 

+ K/> i9 (< 7 d — q) +K/) j9 (q A — q) (11.53) 

and 

T = T d + Kp g(0 d — 0) + K 0 ,q{0 d — 0) 

+ — Tj) + Kfl J (TJ4 — Tj) . 

(11.54) 

These trajectory tracking schemes are the most com¬ 
mon in the control practice for robots with elastic joints. 
In the absence of full-state measurements, they can be 
combined with an observer of the unmeasurable quan¬ 
tities. An even simpler realization is 

r = T d + K P (0 d -0) + K D (0 d -0), (11.55) 

which uses only motor measurements and relies on the 
results obtained for the regulation case. 

The different gain matrices used in (11.53)—(11.55) 
have to be tuned using a linear approximation of the 
robot system. This approximation may be obtained at 
a fixed equilibrium point or around the actual reference 
trajectory, leading respectively to a linear time-invariant 
or to a linear time-varying system. While the existence 
of (possibly time-varying) stabilizing feedback matri¬ 
ces is guaranteed by the controllability of these linear 
approximations, the validity of the approach is only lo¬ 
cal in nature and the region of convergence will depend 
both on the given trajectory and on the robustness of the 
designed linear feedback. 

It should be mentioned that such a control approach 
to trajectory tracking problems can be used also for 
robots with flexible links. Once the inverse dynamics 
problem has been solved (for the case of link flexi¬ 
bility, this typically results in a noncausal solution), 
a controller of the form (11.55) (or (11.53), with link 
deflection 5 and deflection rate 8 replacing, respec¬ 
tively, q and <jr), can be directly applied. 

11.1.5 Further Reading 

This section contains the main references for the part of 
this chapter on robots with joint flexibility. In addition, 
we point out to a larger bibliography for topics that have 
not been included here. 

Early interest in problems arising from the pres¬ 
ence of flexible transmissions in industrial robots dates 
back to [11.1,2], with first experimental findings on the 
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GE P-50 arm. Relevant mechanical considerations in¬ 
volved in the design of robot arms and in the evaluation 
of their compliant elements can be found in [1 1.3]. 

One of the first studies on the inclusion of joint elas¬ 
ticity in the dynamic modeling of robot arms is due 
to [11.4]. The detailed analysis of the model structure 
presented in Sect. 11.1.1 comes from [11 .5], with some 
updates from [11.6]. The simplifying assumption lead¬ 
ing to the reduced model was introduced in [11.7]. The 
special case of motors mounted on the driven links is 
considered in [11.8], The observation that joints with 
limited elasticity lead to a singularly perturbed dynamic 
model was first recognized in [11.9]. Symbolic ma¬ 
nipulation programs for the automatic generation of 
dynamic models of robots with elastic joints were also 
developed very early [11.10]. 

The inverse dynamics computations in Sect. 11.1.2 
can be traced back to [11.11, 12]. The use of programs 
such as Modelica for the numerical evaluation of in¬ 
verse dynamics is highlighted in [11.6]. An efficient 
Newton-Euler inverse dynamics algorithm for robots 
with elastic joints has been proposed in [11.12]. 

The first specific control designs were based 
on decentralized linear controllers, see, e.g., [11.13] 
and [11.14], where a fourth-order dynamics local to 
each elastic joint was used. However, schemes with 
proved global convergence characteristics appeared 
only later. 

In Sect. 11.1.3, the PD controller with constant grav¬ 
ity compensation is a contribution of [ 11. 1 5]. This was 
also extended to the case of robots with flexible links 
in [11.16]. The first two versions of regulation control 
with online gravity compensation are due, respectively, 
to [11.17] and [11.18]. Both these control laws were 
extended to Cartesian compliance schemes in [11.19- 
21], The PD-like controller (11.38), with perfect can¬ 
cellation of gravity effects on link motion, is a result 
of [11.22]. A general regulation framework based on 
energy shaping has been proposed in [11.23]. In the 
absence of any information on gravity, a proportional- 
integral-derivative (PID) regulator with semiglobal sta¬ 
bility properties has been presented in [11.24], while 
a global learning scheme was proposed in [1 1.25]. 

The presentation of a full-state feedback design 
for regulation (and tracking) follows the idea used 
in [11.26]. Special interest has been devoted over the 
years to joint torque feedback, from the placement of 
torque sensors on transmission shafts [11.27] or within 
harmonic drives [11.28] to their use for achieving ro¬ 
bust control performance [1 1.29]. 

The fact that the reduced model of robots with elas¬ 
tic joints can always be feedback linearized via static 
state feedback was shown for the first time in [11.7]. 
A previous similar result for a specific robot kinematics 


can be found in [11.30]. The discrete-time implemen¬ 
tation of the feedback linearization approach has been 
studied in [1 1.31], while its robustness properties were 
analyzed in [11.32]. Feedback linearization and input- 
output decoupling were also considered for the case 
of viscoelastic joints in [11.33], and for robots with 
joints of mixed type, some rigid and some elastic, 
in [11.34]. The same idea of inversion control, lead¬ 
ing to input-output decoupling and linearization, has 
also been successfully applied to the tracking prob¬ 
lem of joint-level trajectories in robots with flexible 
links [11.35]. 

For the general model of robots with elastic joints, 
the use of dynamic state feedback was first proposed 
in [11.36]. A comparative study of the errors induced 
by neglecting the motor-link inertia couplings (thus, for 
robots that are not linearizable by static state feedback) 
was carried out in [11.37]. The general algorithm for 
constructing the dynamic linearizing feedback is pre¬ 
sented in [11.38, 39]. 

The benefit of combining a stabilizing feedback 
with a feedforward command in robots with flexi¬ 
ble joints was experimentally shown for the first time 
in [1 1.39] (see also the l<£ 2 >jHti 2 H!Ui!l) and then formal¬ 
ized in [11.11]. 

Other control design methodologies available 
for trajectory tracking, which were not discussed 
in Sect. 11.1.4, are based on singular perturbation, 
backstepping, or passivity. Nonlinear controllers based 
on the two-time scale separation property were pro¬ 
posed by [11.40] and [11.41]. These corrective con¬ 
trollers are an outcome of the singular perturbation 
model form and should be preferred in the case of very 
stiff joints, as they do not generate high-gain laws in 
the limit. Backstepping is based on the idea of con¬ 
sidering the joint elastic torque [11.42] or the motor 
position [1 1.43] as an intermediate fictitious input to be 
used to control the link equations, and then to design 
the actual torque input in the motor equations so as to 
follow the reference behavior for the previous interme¬ 
diate input. The main advantage is that this design may 
be easier to be transformed into an adaptive version. 
Adaptive control results for robots with elastic joints 
include the high-gain (approximate) schemes [11.44] 
and [11.45], as well as the global (but very complex) 
solution obtained in [11.46], both analyzed using only 
the reduced dynamic model. Moreover, robust control 
schemes have been proposed in [11.47], based on slid¬ 
ing mode techniques, and in [1 1.48-52], using iterative 
learning on repetitive tasks. 

As for optimal control results, the intrinsic fourth- 
order dynamics associated to flexible joints has barred 
so far the derivation of analytical or numerically effi¬ 
cient results. Finding the time-optimal velocity profile 
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on a constrained geometric path is a solved problem 
in the rigid case, but only an approximate solution is 
available for robots with elastic joints [11.49]. A use¬ 
ful generalization from the rigid to the elastic case is 
the dynamic trajectory scaling algorithm of [11.50], 
which provides a closed-form expression to the uniform 
time scaling that recovers feasibility of commanded 
torques w.r.t. their bounds. Selected optimal control 
problems for single viscoelastic joints, involving either 
the maximization of stored potential energy or the min¬ 
imization of rest-to-rest motion time, have been tackled 
and solved in [11.51] and [11.52], respectively. 

Different state observers have been presented, start¬ 
ing from an approximate one [11.53] up to the exact 
ones in [11.5,54], in which a tracking controller based 
on the estimated state was also tested. In all cases, link 
position or link position and velocity measurements are 


11.2 Robots with Flexible Links 

Arm flexibility is a dynamic behavior in which kinetic 
energy interacts with elastic potential energy. Kinetic 
energy is stored in moving inertia, and potential energy 
is stored in compliant members. Flexible links provide 
both the inertia and compliance distributed throughout 
the member. Flexibility manifests itself as mechanical 
oscillations and static deflections, greatly complicating 
the motion control of a mechanical arm. If the time to 
settle the oscillations is significant relative to the cycle 
time of the overall task, flexibility will be a major con¬ 
sideration in the arm design. 

11.2.1 Design Issues 

We will deal with these issues in a modular way, sepa¬ 
rating link flexibility and joint flexibility. The first topic 
of link flexibility deals with the inherent spatial dis¬ 
tribution of compliance and inertia, perhaps with both 
effects existing in the same element of material. The 
links of a manipulator arm can usually be stiffened by 
adding structural mass or by improving the material 
properties or distribution over the links. At some points, 
the links may be treated as rigid. The designer must 
realize that an arm with heavy rigid links and flexible 
joints may be more flexible dynamically than a lighter 
arm with more link flexibility and less inertia. The con¬ 
sequences are predicted by the natural frequency of the 
first flexible mode of motion, with the joint actuators 
locked, which can be approximated by 



assumed. On the other hand, the use of motor position 
and link acceleration has been shown to be a feasible 
alternative for robust state reconstruction [1 1.55], 

Finally, the force control problem in constrained 
tasks for robots with elastic joints has been tackled fol¬ 
lowing a singular perturbation technique [1 1.56,57], an 
inverse dynamics approach [11 .58], or an adaptive strat¬ 
egy [11.59]. 

A unified passivity-based approach has been pre¬ 
sented in [11.60], where an inner torque feedback loop 
is incorporated for shaping (viz., reducing) the motor 
inertia. This enables approximate gravity compensa¬ 
tion (as done in Sect. 11.1.3), as well as to assign 
a desired Cartesian stiffness relation for the link po¬ 
sitions, mimicking the impedance control of the rigid 
case for handling interactions and contacts with the 
environment. 


where k e ff is the effective spring constant and 7 e ff is the 
effective inertia. For the elastic joint case, k e ff and 7 e ff are 
directly available from the elastic joint parameters and 
mass properties of the arm. The effective values appli¬ 
cable to a simple beam in bending, for example, would 
result from boundary conditions that are clamped at one 
end and free at the other and would be selected to yield 
a good approximation to the natural frequency. This nat¬ 
ural frequency is readily available f rom han dbooks on 
vibration in this case as co \ = 3.52 y/ El/ml 4 . (The vari¬ 
ables are used in (1 1 .57) and (11.58) and defined below.) 
More complex geometries can be evaluated by various 
means, including the transfer matrix method discussed 
in Sect. 11.2.2 on modeling. The arm should be con¬ 
sidered flexible when this frequency is low enough to 
interfere with the design of controllers based on rigid 
assumptions. The common PD control of a single joint 
on a flexible link can only achieve adequate damping of 
a closed-loop design when the magnitude of the closed 
loop poles is less than about 1/3 of co 1 [11.61]. 

A distillation of the key aspects of the mechanics of 
materials is now in order. In some cases static compli¬ 
ance effects represent a member sufficiently, whereas 
in other cases the true distributed nature of compliance 
and mass and its dynamics must be modeled. A long 
structural member with forces / and moments M per¬ 
pendicular to its long axis is subject to bending. Static 
bending relates the moment at the axial location x to the 
displacement w 

d 2 w 

M(x) = EI(x)— , 


an = 


(11.56) 


(11.57) 
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where E is the elastic modulus of the material, and I(x) 
is the area moment of inertia about the neutral axis of 
the cross section. The deflection at any point is obtained 
by integrating this equation from a reference point to 
a desired point, such as the end of the link. 

This is a description of compliance that can be 
used when mass is isolated from elasticity. If mass 
is distributed throughout the beam with material mass 
density per unit volume p, the time t must be incorpo¬ 
rated so that 


d 2 w(x,t) d 2 ( d 2 w(x,t ) 

m,,) -3^ + SU £7W ^r- 


= 0 . 


(11.58) 


Here m(x) is the mass density per unit length, incor¬ 
porating the material properties and the cross-sectional 
area at x. Assumptions implicit in this equation, which 
is called the Bernoulli-Euler equation, include the min¬ 
imal impact of shear distortion and rotational inertia of 
the cross section of the beam, which is valid for long 
beams. The Timoshenko beam model relaxes these as¬ 
sumptions but is not included here. 

Torsional deflection of an angle 9 results if a twist¬ 
ing moment T about the long beam axis occurs, such 
that, in the static case 


6 = 


T1 

JG 


uerT , 


(11.59) 


where G is the shear modulus of the material, and J is 
the polar area moment of inertia about the neutral axis 
of the beam. Again, the addition of a distributed mass 
and considering the dynamics produces a partial differ¬ 
ential equation with independent variables in space and 
time 


d 2 9 

^W = CJ a? 


(11.60) 


where fj,(x ) is the rotary mass moment of inertia per 
unit length of the shaft. While (11.60) is potentially ap¬ 
propriate for links, the smaller rotational mass moment 
of inertia of a long slender shaft relative to the links it 
drives results in the static compliance effects of torsion 
given by (11.59) being of most importance. 

Tension and compression effects should also be 
acknowledged, although they tend to be the least sig¬ 
nificant. Here deflection S in the v-direction (the long 
axis of the member) results from axial force F a and is 
calculated as 8 = ^ 


. F,L 

0 = —- = UxfFz , 

AE 


(11.61) 


where L is the element length and A is its cross sec¬ 
tional area. In the dynamic case, with mass density p 
and displacement f, the axial motion is described by 




(11.62) 


Note that these effects are special cases of the more 
general elastic behavior of a member subject to accel¬ 
eration and external loading, specialized to the member 
with one long axis. The same general phenomena ap¬ 
ply to joint structures (e.g., bearings and couplings) 
and drive-train components (e.g., gears and cables). The 
functional constraints on the structural design of these 
members may be more severe and limit the designer’s 
ability to make them more rigid by adding more mate¬ 
rial. Under this constraint, the combined compliance of 
the flexible joint design and the rigidized link design re¬ 
sults in inferior performance of the combination. A less 
rigid link with less inertia will raise the lowest natural 
frequency. Thus the design of the link and the joint must 
be coordinated. 

Flexibility is only one of several link design con¬ 
straints. Static compliance is obviously another based 
on the above discussion. Buckling and strength are 
two more considerations that constrain some of the 
same design parameters. While it has been shown that 
flexibility is a dominant constraint for typical arm de¬ 
sign regimes [1 1.62], a simple optimization of bending 
stiffness of a beam with a tubular geometry illustrates 
the fallacy of ignoring buckling. If the radius of the 
tube is varied to maximize bending stiffness with con¬ 
strained total mass, the radius increases without bound, 
producing a thin shell of infinitesimal thickness and in¬ 
finite radius that lacks robustness. The true constraint 
becomes local buckling of the tube under slight pertur¬ 
bations of the applied load. A thin metal can illustrates 
this behavior as it crushes. Strength is another applica¬ 
ble constraint. Various materials differ in their strength, 
typically represented by their elastic limit or endurance 
limit on the stress at a point in the member. A suffi¬ 
ciently stiff member may fail due to stress, particularly 
at points of stress concentration. 

In this presentation of options to overcome flexibil¬ 
ity, and link flexibility in particular, radically different 
operational and design strategies should be recognized 
even though they are not the core of the treatment. The 
elastic modulus relates stress to strain, but physically 
the strain rate is sometimes a relevant term, providing 
structural damping to augment the joint or control- 
based damping. Composite materials have substantially 
more damping inherent in their behavior. Damping 
can be enhanced with passive damping treatments, 
with constrained layer damping having particularly pro¬ 
nounced effects [11.63]. Smart materials can also be 
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configured to enhance damping [1 1.64]. An operational 
strategy known as bracing can be combined with re¬ 
dundant actuation to achieve a large workspace for 
a flexible arm that provides gross motion and a smaller 
precision workspace after docking the large arm with 
a stationary structure, as described in [11.65]. 

The implication above is that flexibility was always 
due to the moving parts of the arm. However, the arm 
base itself may be a source of significant flexibility. 
If the base is stationary, the analysis may be almost 
identical to that of the flexible links of an arm. How¬ 
ever, if the arm is mounted on a vehicle, tires or tracks 
may be the source of flexibility and empirical evalua¬ 
tion of these new elements might be the best solution. 
The vehicle could be a boat, aircraft, or space vehicle. 
Here there is no elastic connection to an inertial refer¬ 
ence frame and some fundamental changes in approach 
will be needed. These changes are not unexplored and 
in some cases are simple extensions of the following 
approaches. 

11.2.2 Modeling of Flexible Link Arms 

Mathematical models of link flexibility trade accuracy 
for ease of use. It is therefore key to determine the 
effects that are important to represent accurately. The 
techniques presented here will assume linear elastic¬ 
ity with light damping. Rotational motions must be of 
modest angular rate so that centrifugal stiffening can be 
ignored. Small deflections will generally be assumed. 
These are reasonable assumptions for most robotic de¬ 
vices, but can be violated in more exotic applications, 
requiring reevaluation of the assumptions and more tor¬ 
tuous modeling approaches. 

Four model types will be examined: 

1. Lumped-element models having compliance or in¬ 
ertia but not both in a given member. 

2. Finite-element models, which will be only briefly 
addressed. 

3. Assumed mode models which readily include non¬ 
linear dynamic behavior. 

4. Transfer matrix models that incorporate the true 
distributed intermingling of compliance and inertia 
and consequently the infinite-dimensional nature of 
flexible link arms. 

With the limited space available here, the discussion 
will be preliminary but should enable the reader to pur¬ 
sue one or more alternatives knowledgably. 

For lumped elements, one may build on the rigid 
kinematics presented in the section on fundamentals of 
kinematics (Chap. 2) that establishes rigid transforma¬ 
tion matrices A,-. The 4x4 homogenous transformations 
that describe position can be used to describe deflection 


as well. Assuming small motions and static behavior (or 
negligible member mass), the transformation produced 
by elastic bending, torsion, and compression is 


( 1 

a eFiF'r, + Ol0MiM'zi 

a 0FiF' zi — Ol0MiM‘ Yt 

V 0 


ueFiF'y,-cieMiM' Zi 

1 

a Tl M' xi 

0 


-a eF iF‘ zi + asMiM'y, 
—u Ti Mxi 

1 

0 


uaF'x, \ 

<XXF,F‘ Yl + dxMiM'zi 
UXFiF' zi + CtxMiM'yj 

1 


(11.63) 


where 

da = coefficient in compression, 
displacement/force , 

an = coefficient in torsion, angle/moment, 

dQFi = coefficient in bending, angle/force , 
ag Mi = coefficient in bending, angle/moment, 

d-XFi = coefficient in bending, displacement/force , 

dxMi = coefficient in bending, 
displacement/moment, 

F J xj = the force at the end of link i in the 
X direction of coordinate frame j , 

F J Yj = the force at the end of link i in the 

Y direction of coordinate frame j , 

F J zi = the force at the end of link i in the 
Z direction of coordinate frame j , 

M J xj = the moment at the end of link i in the 
X direction of coordinate frame j , 

M' Yj = the moment at the end of link i in the 

Y direction of coordinate frame j , 

M J zi = the moment at the end of link i in the 
Z direction of coordinate frame j . 

The listed coefficients depend on the construction of the 
element and are readily found for a slender beam from 
simple strength of materials. In other cases, a finite- 
element model or empirical measurements may be more 
readily used. 

Alternating transformations for the undeformed and 
deformed aspects of N members (links or joints) pro¬ 
duces the position vector of the end of arm (EOA) 


P — (A1E1A2E2... AiEiAf+iEi+i... A n E n ) 


M 
0 
0 

W 

(11.64) 
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If the massless elements connect rigid lumped masses, 
a linear spatial model is readily obtained, as described 
in [11.66], Servo-controlled joints can also be inserted 
into this model. This analysis proceeds by examining 
the deflection of the elastic members produced by joint 
motion and the resulting forces and moments on the 
rigid inertias at the ends of the chain of links. The ref¬ 
erence concentrates on the special case of only two 
inertias at each end of the chain but the technique is 
easily extended to inertias in the middle of the chain as 
well. The results are six linear second-order equations 
for each inertia and one first-order equation for each 
joint. 

If the compliance and inertia properties are both to 
be treated as distributed throughout the same element, 
the partial differential equations (PDEs) introduced in 
the previous section for bending, torsion, and com¬ 
pression must be employed. Still insisting on a linear 
treatment, the general solution of these equations can 
be found in terms of each element with adjoining el¬ 
ements, i. e., the boundary conditions on the PDE. 
The convenient approach of Laplace transformation of 
these linear equations into the frequency domain, fol¬ 
lowed by factoring out the boundary conditions into 
a matrix-vector product, results in a technique known 
as the transfer matrix method (TMM) [11.67]. This 
method has been applied productively to general elas- 
tomechanics problems as well as specifically to flexible 
arms [11.68]. 

The transfer matrix relates variables at two stations 
along the arm with the number of variables depending 
on the model complexity. For the planar bending, de¬ 
flection of the beam is shown in Fig. 11.9 
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at station 1 


zo = Tzi , 

(11.65) 

where T is the appropriate element transfer matrix. 

If the element is a simple rotary spring with con¬ 
stant k and a damper with constant b, then the angle 
of rotation 9 is related to the moment by a differential 



Fig. 11.9 State vector for the bending transfer matrix 


equation with the following Laplace-domain represen¬ 
tation 


M = —L(k6 + bd) => k@(s) + bs&(s) = Mq = M\ , 
Wo = W\ , 

Vo = Ft , 

% = V\-0(s) = <Pi - M i . 

k + bs 

Note that the only variable changed across the element 
is the angle related to the moment. The transfer matrix 
representation conventionally uses zero initial condi¬ 
tions and converts to the Fourier representation with 
s = ico. This leads to the transfer function 


T = C(kw) = 


(\ 0 
0 1 


k-\-b(iCO) 

0 0 1 0 
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\0 0 
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which has the same form of the joint controller as 
shown in (11.69). 

The beam model is much more complex, and the 
Euler-Bernoulli model gives 
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where 


( 11 . 66 ) 


P 4 = co 2 l 4 f e/(EI); a = l 2 /(El) , 

Co = (cosh /) + cos P)/2 , 

Ci = (sinh/3 + sin/J)/(2 P ), 
c 2 = (cosh p — cos p )/(2/S 2 ) , 

C 3 = (sinh/S — sin /S)/(2/S 3 ) , 

fi = density/unit length , 

co = circular frequency of vibration , 

E = elastic modulus , 

I = cross-sectional area moment of inertia . 


The spatial variable has been transformed to the spatial 
Laplace variable but no longer appears explicitly. The 
Laplace time variable remains and is designated by s 
or by the frequency variable co = —is, where i = y/—l. 
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The other transfer matrices needed for a simple planar 
model are as follows. 

For a rotation in the plane of angle (p 


/ 0 0 0 \ 

cos (p 

0 10 0 
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\m s w 2 sin <p tan ip 0 0 cos ip) 


(11.67) 
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(11.71) 


where m s is the sum of all outboard masses from the 
angle to the end of the arm. This is an approximation 
resulting from ignoring compression elasticity. These 
elements appear as an additional mass to the extent they 
are translated with beam compression. 

For a rigid mass, simply applying Newton’s laws 
and collecting term yields 
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0 
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( 11 . 68 ) 
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where m is the mass of the body, is the mass moment 
of inertia about an axis through the center of mass and 
perpendicular to the plane of the arm, l is the length of 
the mass (the distance between the points of attachment 
at stations i and i+ 1), and //2 is the distance to the 
center of mass from station i. 

For a controlled joint with the controller transfer 
function (joint torque/joint angle) = k(ico) 
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(11.69) 


The determinant of the 2x2 matrix must be zero 
when <y is a natural frequency, or a system eigenvalue if 
it is complex. This can be found by a numerical search. 
Evaluating variables (position, angle, shear, and mo¬ 
ment) along the length of the arm with the eigenvalues 
used for the values of co yields the eigenfunctions. If 
boundary conditions are not zero but known for given 
frequencies, the frequency response of the system can 
be obtained. Flence Bode plots are readily obtained 
for boundary condition forcing. Interior forcing can be 
handled using an extended state vector as described 
in [11.67], which was applied to arms in [11.61] and 
most recently updated with modern programming tech¬ 
niques in [1 1.68]. Inverse fast Fourier transform (FFT) 
will produce a time-domain response from the fre¬ 
quency response. 

The creation of a state-space model in the time do¬ 
main is attractive because it enables the use of powerful 
state-space design techniques and is compatible with 
the nonlinear behavior of arms undergoing large mo¬ 
tion and experiencing centrifugal and Coriolis forces. 
The assumed modes method allows such a model to be 
constructed efficiently. The presentation here follows 
the early introduction of a recursive manner to calcu¬ 
late these equations introduced in [11.70]. The flexible 
kinematics of the links must be expressed as a sum of 
basis functions also known as assumed modes (shapes) 
(Pi(x) with time-variable amplitudes 5,- (?) 


When combined to represent the planar arm with two 
joints as shown in Fig. 11.10, this analysis yields a com¬ 
posite transfer matrix that relates the four state variables 
at the two ends of the arm. Further analysis results from 
imposing the known boundary conditions at these ends. 

The transfer matrix representation of an arm pre¬ 
sented in [11.69] and pictured in Fig. ll.lOis shown in 
(11.70). 
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In the example, the left end is pinned and the right end 
is free, yielding 


OO 

w(x, t) = Sj(t)<pi(x). ( 11 . 72 ) 
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The amplitudes and their derivatives become the states 
of the model. Compatible joint angle variables and their 
derivatives are also included as the rigid state variables. 
Joint angles tangent to the ends of the flexible links 
are compatible with mode shapes based on the clamped 
boundary conditions at one end and free boundary con¬ 
ditions at the other. This choice is generally chosen for 
simulation and the rigid coordinates are directly mea¬ 
sured by standard joint angle transducers. If rigid coor¬ 
dinates are angles measured between lines connecting 
successive joint axes, the end of arm is known in terms 
of rigid coordinates alone and pinned-pinned boundary 
conditions are consistent. This produces advantages for 
inverse dynamics calculations as seen shortly. 

The flexible and rigid kinematics combined de¬ 
scribes the position and velocity of every point on the 
arm and can be used to express the kinetic energy ‘t 
and the potential energy V. These expressions are used 
in the conservative form of Lagrange’s equation 

d dr dr dy 

-:- 1 -= Fi , (11.73) 

dt dcji dqi dqt 

where F, is the force that does work as q, is varied. 

The kinematics of the deflection are again described 
by 4 x 4 transformation matrices, but this time the as¬ 
sumed mode shapes j are summed to give the position 
of link i 
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where Xy, yy, and zy are the Xj, y,-, and Zi displacement 
components of mode j of link i’s deflection, respec¬ 
tively, Sy is the time-varying amplitude of mode j of 
link i, and m ; is the number of modes used to describe 
the deflection of link i 
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(11.77) 


This description is used in the formation of the po¬ 
tential and kinetic energy contributions of each element 
of mass. The integral over the link accumulates the to¬ 
tal energy for that link and the sum over all links gives 
the energy of the arm system. Interchanging the order 
of integration and summation allows key invariant par¬ 
ameters to be identified, such as the modal mass that 
multiplies the second derivative of modal amplitude and 
the modal stiffness that multiplies the amplitude itself. 
The intermediate steps are aimed at the evaluation of 
Lagrange’s equations, which separate into derivatives 
related to the rigid variables and the flexible variables. If 
the rigid variables 6 are the joint variables, two groups 
of equations result 

d hr\ dr av e dy g 

dr ^ 9 9j J ddj d6j ddj 

9 d(dr\ dr aVe dy g 

d' \ dSjf) d$jf dSjf dfyf 

where the subscript e indicates elasticity, subscript g in¬ 
dicates gravity, and 8 indicates the amplitude of a mode 
shape. 

The simulation form of the equations is achieved by 
grouping all second derivatives of the rigid and flexible 
coordinates, which yields 

M,.,.(< 7 ) 

M/v(<7 ) 1V% J \8 J 

= -(o ^J+N(q,q) + G(q)+R(q,q,Q), 

(11.80) 

where 6 is a vector of rigid coordinates, usually the 
joint variables, 8 is the vector of flexible coordinates, 
q = [6 T <S T ] T , My is the mass matrix for rigid and flex¬ 
ible coordinates corresponding to the rigid ( i,j = r ) 
or flexible (i,j =/) coordinates and equations, N(q, q) 
contains the nonlinear Coriolis and centrifugal terms, 
G(q) captures gravity effects, Q represents the ex¬ 
ternally applied forces, and R captures the effect of 
external forces and all other nonconservative forces in¬ 
cluding friction. 

On the other hand, the inverse dynamics form of the 
equations is (assuming negligible friction) 


= Fj (11.78) 
= 0, (11.79) 
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In general, Q is dependent on the vector T of motor 
torques or forces at the joints, and the distribution of 
these effects is such that 



where T has the same dimensionality as q r . By rear¬ 
ranging the inverse dynamics equation, we obtain 



(m;;)* + ' v( «> +gw ’ 

(11.8B) 


which is arranged with a vector of unknowns on the 
left-hand side of the equation and the prespecified rigid 
and solvable flexible coordinates on the right. This is at 
least approximately true, with the caveat that the non¬ 
linear and gravity terms have a weak influence on the 
immediate flexible coordinates, which are not simulta¬ 
neously known but which can be estimated for the next 
time step. 

One consequence of the distributed flexibility is 
non-minimum-phase behavior. This can occur for other 
cases where the excitation is separated from the output 
point by mass and flexibility, a condition known as non¬ 
collocation. These systems are characterized by a step 
response in which the initial movement is opposite in 
sign from the final movement. This is simply charac¬ 
terized for linear systems as a transfer function zero in 
the right half-plane. As will be shown in the section on 
Command Generation, it may be possible to overcome 
this problem by redefining the output. 


11.2.3 Control 


Appropriate reaction to flexible dynamics requires ap¬ 
propriate sensors. While a Kalman filter can theoreti¬ 
cally observe flexible link states from joint measure¬ 
ments, it is based on back driving the joint from the 
various modes of flexible dynamics. More reliable indi¬ 
cations of flexibility include strain gauges, accelerome¬ 
ters, and optical sensors. 

Sensors for Flexibility Control 
The strain of a beam element is a direct indication 
of curvature, which is the second spatial derivative 
of beam deflection, as indicated in (11.57). Modern 
semiconductor strain gauges have high gauge factors 
producing a good signal-to-noise ratio. The second spa¬ 
tial derivative relationship means that a slight change 
in location can result in a large change in the reading, 
but once affixed to the arm this is not a problem. More 
of a problem is the relatively short lifetime of a strain 
gauge subject to wear and tear if not carefully protected. 


By suitably placing strain gauges, good measurements 
of multiple assumed modes are possible. 

Optical measurement of deflection can be very ver¬ 
satile but is subject to interference between a target 
measurement point and the sensor, particularly if the 
sensor is stationary. If the sensor is mounted on the link, 
this difficulty is reduced but optical measurements are 
then usually a complex combination of the rotation of 
the optical axis of the sensor and the deflection of the 
target. Machine vision senses a wide field of view but 
typically with less precision. Multiple points can be de¬ 
tected, which is attractive in measuring multiple modes. 
Machine vision requires considerable processing and, 
even for enhanced targets or retroreflective fiducials, 
sample rates can be slow relative to some modes of con¬ 
trol interest. 

Accelerometers are attractive transducers in that 
they do not require explicit grounding to a fixed frame. 
Microelectromechanical systems (MEMSs) accelerom¬ 
eters provide the required low-frequency sensitivity and 
are very inexpensive. However, they are subject to ori¬ 
entation in a gravity field, which must be compensated 
for by knowing the orientation of the sensor. If one is 
interested in position, accelerometers require two inte¬ 
grations and are consequently subject to drift in position 
measurement. Consequently, the combined use of sen¬ 
sors is appealing. For example, vision sensors, which 
are relatively slow but give direct position information 
for the end of a flexible member, can be combined with 
accelerometers, which give information about the same 
point at high sample rates but with noise and drift prob¬ 
lems. The two sensor readings can be combined with 
a Kalman filter or other fusion scheme. 

Issues Related to Model Order 
Realizing that the theoretical order of a flexible arm 
is infinite, one should be prepared to deal with cer¬ 
tain issues related to model order. The first of these is 
possible aliasing problems arising from finite sampling 
rates. Antialiasing filters should be employed, and even 
more effective is the use of analog damping by materi¬ 
als, damping treatments, or the back electromotive force 
(emf) of actuators to provide an inherent damping of the 
system under digital control. A more subtle problem is 
that of measurement and control spill over. 

Control of Special Configurations: 

Macro/Micro Redundancy 
Several special configurations have been proposed to 
enable improved controlled performance of flexible link 
arms. The problem with noncollocation noted above is 
a case in point. In some cases, a large range of motion 
and high accuracy cannot be achieved with a single set 
of actuators. Redundant degrees of freedom can then 
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be employed with the macro degrees of freedom that 
provide large range of motion but with inherent lack 
of precision or accuracy. Vibrations are particularly dif¬ 
ficult to eliminate with a long arm, and construction 
accuracy and even thermal drift can exceed the toler¬ 
ance limits. If the gross motion is completed quickly, 
residual vibrations may remain and be very lightly 
damped. Actuators constructed for gross motions of 
large axes do not have the bandwidth capabilities to 
damp these vibrations actively, but small motion axes 
typically do. This has been illustrated in several in¬ 
stances for long-reach manipulators when exploring 
nuclear waste cleanup solutions [11.71-74]. Using the 
inertial forces generated at the end of the arm also 
has the advantage of creating forces collocated with 
the measurements being used for control, assuming the 
base of the micro arm is at the end of the macro arm 
and the location of sensors. Book and Loper [11.73] 
and George and Book [1 1.75] have shown that this con¬ 
trol is possible based on accelerometers in multiple 
degrees of freedom. It is also possible to excite unde¬ 
sired modes which have not been accounted for. If the 
full six degrees of freedom are used it is possible to 
argue stability based on a simple passive analogy. Gen¬ 
erating prescribed forces at the base of an articulated 
micromanipulator is a challenging task, requiring the 
solution of a new type of inverse dynamics problem, 
which solves for motions or joint torques to produce 
the prescribed base forces and moments. 

Command Generation for Flexible Link Arms 
via Command Shaping 

The trajectory undertaken by a flexible arm can dra¬ 
matically affect the consequences of flexibility. Purely 


Single impulse 


1 

Single impulse 
shaped with 
a three-term 
command shaper 



Fig. 11.11 Impulse response of a shaper and the effect on a vibra¬ 
tory system. Time is normalized by the intersample time T s and T&, 
chosen such that A = Ta/T s is an integer 


open-loop anticipation of the flexible dynamics has 
been used to create motion profiles wherein the motion 
cancels the incipient oscillation that had been created by 
earlier motions. This strategy is referred to as command 
shaping. A more complicated strategy to implement is 
a true inverse-dynamics-based trajectory wherein the 
rigid and flexible states are all orchestrated to arrive 
at the desired point in state space, typically at a static 
equilibrium. The high order of the flexible model is 
further complicated by the non-minimum-phase nature 
of the noncollocated system and noncausal inverse is 
necessary to produce a physically realizable trajectory 

Time delay command shaping is a variation of the 
deadbeat control introduced by [11.76,77], It can be 
couched in terms of a finite impulse response filter with 
a simple impulse response as depicted in Fig. 11.11 
with a single input impulse producing a small number 
(typically two to four) of output impulses with ap¬ 
propriate time spacing and amplitudes. Extensions by 
Singer and Seering [11.78] and Singhose et al. [11.79] 
have produced more robust varieties under the trade¬ 
mark Input Shaping. The robustness is enhanced by 
using more output impulses and appropriate selection 
of the impulse spacing. The optimal arbitrary time- 
delay (OAT) filter [11.80,81] reduces this selection 
to a simple formula for linear vibrations that depends 
on the natural frequency and damping ratio of those 
modes to be eliminated from the response. The simplic¬ 
ity of implementation is illustrated in Fig. 11.12, where 
the relationships between the filter parameters and the 
mode parameters are 


coefficient 1 = — 
M 


coefficient 2 = 


—(2 cos (a>d7d) e 


M 


coefficient 3 = 


-2^CO a T d 


M = 1—2 cos (a>d7d) e ^ a)nTd 


j g 2£ci> n 7d 

®d = w n \/1 — £ 2 = damped natural frequency 

co a = undamped natural frequency 
£ = damping ratio 
T& = time delay selected, an integer 
number of samples . 

(11.84) 


Note that the formulation here is oriented toward 
digital systems with fixed sampling intervals such that 
the time delay of the filter is adjusted to an integer 
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number of sampling intervals. Command shaping has 
been applied to a wide range of flexible systems includ¬ 
ing disk drives, cranes, and large robot arms, such as 
the example by CAMotion (l<£2>Kii!H!UM) as shown in 
Fig. 11.13. Note that this example contains a flexible 
drive belt, effectively a flexible joint, as well as flexi¬ 
ble links that contribute to the compliance of the lowest 
mode that is quenched with an OAT filter. 

Adaptive versions of command shaping have also 
been demonstrated. A version oriented toward repetitive 
motion applications is presented in [11.82]. 

Command Generation for Flexible Link Arms 
via Inverse Dynamics 

In contrast to the scant system knowledge required for 
command shaping, the inverse dynamics of a flexi¬ 
ble arm requires a great deal of knowledge about the 
system. The creation of an appropriate model was dis¬ 
cussed above. An assumed modes model, for example, 
is well adapted to inverse dynamics if the rigid coordi¬ 
nates are not the joint angles, but the angles between 
lines connecting the joint axes (where it is assumed that 
joints are rotational). In this case, the desired end-of- 
arm motion can be expressed solely in terms of rigid 
variables, that is, the same variables q c \ used in the flex¬ 
ible joint case, and the desired flexible variables are 
found from the second-order equations driven by the 
known rigid positions and velocities. The remaining 
difficulty is the unstable zero dynamics of the system. 

For the purposes of introduction, only the linear sit¬ 
uation will be treated here in the fashion of [11.83]. 
First, the rigid coordinates and their derivatives are 
found from the desired end-of-arm motion profile. The 
flexible equations are separated into causal stable zeros 
and anticausal unstable zeros, and the stable inverse is 
solved forward in time. Physically the motion response 
responds after the torque inputs. The computational 
input however is the response and the torque is cal¬ 
culated as an output, so no physical laws are violated 
in solving the anticausal portion beginning at the fi¬ 
nal time when flexible coordinates are specified to be 
zero and proceeding to the initial time and beyond. 
In fact, the exact solution extends from negative infin¬ 
ity (where the anticausal solution decays) to positive 
infinity (where the causal solution decays). For realis¬ 
tic dynamics, the necessary time for the input is only 
slightly more than the motion time for the end of the 
arm because the zeros are often quite far from the ori¬ 
gin in both directions. If the end-of-arm motion begins 
at time zero, shortly before that, the joint torques be¬ 
gin to pre-shape the flexible links without moving the 
end of arm. Similarly, when the end of the arm reaches 
its final position, the joint torques must release the 
arm deflection in a manner that is not seen from the 



Fig. 11.12 Block diagram representation of the command shaping 
algorithm 



Fig. 11.13 A large flexible linking arm with command shaping 


Generated joint trajectory 

Joint angle (rad) Displacement (in) 



Time (s) 

Fig. 11.14 Inverse dynamics driven motion of a single-link flexible 
arm 

end of arm (or any other chosen output location). Fig¬ 
ure 11.14 shows the example from Kwon [11.84] for 
a single-link arm and the |43®ZEEtEEl shows the be¬ 
havior improvement. 

Since inversion of the flexible plant as described 
earlier containing nonminimum phase or unstable ze¬ 
ros can result in an anticausal input torque to achieve 
an arbitrary trajectory, it is appropriate to look for other 
trajectories with special characteristics that avoid these 
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problems. The location of zeros are dependent on where 
on the flexible link the trajectory is specified. De Luca 
et al. studied the one flexible link rotational arm and the 
migration of zeros as the point of interest is shifted from 
tip to joint. They utilize a point where zeros are nonex¬ 
istent as the output point, a point that enables inversion 
to proceed without the causality problem [11.85-87], 
When this point reaches the desired angle with zero 
deflection, the end point will also be at the desired an¬ 
gle. Joint velocity and deflection velocities must also 
be brought to zero resulting in a rest-to-rest motion 
completed in specified time and with no residual os¬ 
cillations. The cited references show the equivalence of 
this development in the time or frequency domain for 
linear systems. For a two link arm (only the forearm is 
flexible), the nonlinear behavior demanded a time do¬ 
main approach [11.86]. 

De Luca and Di Giovanni [11.85] address rest-to- 
rest motion of a single flexible link and carries through 
an example for a single flexible mode. The later ref¬ 
erence [11.87] also provides experimental results for 
which a brief video (l<SAKii!ll!Ukl) is available. The 
strength of this approach is its relative simplicity in 
achieving the desired final position described by a sin¬ 
gle polynomial. The degree of the single polynomial 
required to meet all the boundary conditions is 4n e + 3 
where n e is the number of flexible modes. One diffi¬ 
culty is the inability to efficiently constrain the peak 
torque required. Torque profiles developed in this sim¬ 
plest of strategies have high peaks but for most of the 
trajectory (at start and end) are relatively low. Hence 
the peak torque capabilities of the actuator may not be 
efficiently used. It is not necessary for a single polyno¬ 
mial to describe the entire trajectory, of course, but the 
complexity of developing the response without zeros 
becomes more challenging for piece wise polynomials, 
and arises from meeting all the boundary conditions on 
polynomial segments. The nonlinear case arising from 
a two link arm with the forearm flexible [11.86] pro¬ 
vides further exploration of the inversion approach for 
flexible arms. 

The model employed by De Luca varies slightly 
since deflection is measured from an axis passing 
through the dynamic center of mass of the flexible arm 
as shown in Fig. 11.15, not the tangent to the neutral 
axis of the beam at the rotary actuator. This modifies the 
boundary conditions on the shape equation and hence 
the mode shapes. For an introduction, a single flexible 
link is considered here. Variables are defined as follows: 

• Jo, Jp' rotary inertia of the rotary actuator, tip pay- 

load, respectively 

• m p : mass of the tip payload 


• 9,9 c ,9 t : angles through center of mass, tangent to 
actuated end, and through tip, respectively 

• x : axis through the center of mass (COM) 

• y: deflection perpendicular to x. 

Consider the case with no damping on the modes. 
While modal damping is readily incorporated, its typ¬ 
ically low values justify an abbreviated treatment here 
for the sake of space and simplicity. The output y spec¬ 
ified is the sum of the angle (to the center of mass), the 
angular velocity, deflections and deflection rate, each 
times appropriate gains. These gains are found by spec¬ 
ifying the transfer function from joint torque to y to be 
of maximum relative degree, i. e., it has no zeros. Equiv¬ 
alently, the successive derivatives of y do not explicitly 
involve the torque until the (2 n t + l)-th derivative. Note 
that these characteristics ascribed to differentially flat 
systems may yield a more complete understanding of 
this approach [1 1.88, 89], 

yfo) __ K _ 

r(H * 2 IT=i (■ s2 + «, 2 ) ’ 



When transformed to the time domain, the resulting 
form of the torque based on derivatives of the desired 
output will be 


D(0 = 


ns 


cot 


2(«e + l) 

y d 


(o + J2 a 'Td +2) ( f ) 


The a,- values are readily found from the convolution 
of the polynomial or expansion of the denominator of. 
y(s) /t(s). But what must be used for yd? With the given 
definition of y, the desired value of yj for the rest-to-rest 
trajectory will be achieved by creating an interpolating 
trajectory with the desired initial and final values of Vd 



Fig. 11.15 A definition of variables for the causal inversion 
of De Luca 
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and derivatives of Vd at the initial and final times that are 
zero. The number of derivatives to set equal to zero are 
at least 2n e + 1, but it may be preferred to set additional 
derivatives equal to zero to obtain a smoother torque at 
the ends of the trajectory. As explained in [1 1.85] this is 
a transformation of states that results also in the desired 
rest-to-rest behavior of the joint angle, the deflection 
and their derivatives. For more robustness, the feedfor¬ 
ward command should be supplemented with feedback 
control, for example a simple PD or PID control of the 
joint angle to follow y&. 

Shown in Fig. 11.16 are the torque and resulting 
bending deflection for a 2.2 s move using this combi¬ 
nation on a single link flexible arm [1 1.87], 

To best comprehend these results, some details of 
the flexible arm in the experiment are needed. The 
desired trajectory was defined by a polynomial of de¬ 
gree 19. The arm length was 0.655 m, of sheet steel 
2 mm thick and 51 mm wide, directly driven by a DC 
motor with inertia of 1.888 xl0“ 3 kgm 2 and no tip 
mass. The resulting natural frequencies were 14.4, 13.2 
and 69.3 Hz with damping ratios of 0.0001, 0.001, and 
0.008, respectively. Frequency errors of the model were 
less than 1%. Friction compensation was applied based 
on static and viscous models of the friction under con¬ 
tinuous rotation. 

Feedback Control of Flexible Links 
Flexibility in the links is in some cases controlled in the 
same way as concentrated flexibility in joints. Herein 
the focus is on approaches that apply specifically to 
link flexibility; consequently the use of Kalman filtering 
to estimate tip position and other broadly used tech¬ 
niques are not reexamined. A long history of research 


on this topic exists. Cannon and Schmitz [11 .90], Truck- 
enbrot [1 1.91], and others showed that these techniques 
work for flexible link arms and that the noncollocated 
nature of the system aggravates the sensitivity to mod¬ 
eling errors. 

Strain measurements provide a valuable indication 
of the state of a flexible link, and strain rate, ob¬ 
tained from filtered differentiation of the typically noisy 
strain signal, was shown by Hastings and Book [1 1.92] 
to be an appropriate way of overcoming the limita¬ 
tions imposed by purely joint feedback control. Yuan 
et al. [11 .93] produced an adaptive version of strain-rate 
feedback. 

Strain measurements readily lend themselves to cal¬ 
culation of the modal amplitudes, and the number of 
appropriately placed strain gage measurements N can 
be equal to the number of mode amplitudes to be evalu¬ 
ated. In this case, no estimation algorithm is necessary, 
although filtering is usually needed. If the mode shape 
(Pi of a beam is known, strain is proportional to the sec¬ 
ond derivative of <pi 


By inverting the coefficient matrix thus formed, it is 
simple to obtain the amplitude of N modes as 
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Fig. 11.16 (a) Applied torque for rest to rest motion with PD control, (b) Bending deformation at the tip 
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Additional attention is necessary if beam torsion is 
a significant part of the mode deflection, but the prin¬ 
ciple still applies. Active control of high-frequency 
modes may not be warranted, and feedback of the one or 
two lowest-frequency modes can be managed by clas¬ 
sical or state-space design techniques to increase the 
damping of that mode. 

The use of the measurement of the end-of-arm po¬ 
sition has perpetual appeal and is very effective for 
rigid arms for the elimination of variability due to arm 
distortion and the change over time of arm calibra¬ 
tion. Machine vision can eliminate uncertainty in the 
target position as well as the arm position. These ap¬ 
proaches are plagued by the noncollocated and hence 
non-minimum-phase dynamics of link flexibility, how¬ 
ever, and may require particular caution. Wang and 
Vidyasagar [11.94] proposed an alternative formula¬ 
tion of EOA measurements in the control formulation 
that he called the reflected tip position. For a rotating 
link of angle 9 and length /, the deflected position is 
16 + S while the reflected tip position 19 — S is minimum 
phase. This combination of joint and tip measure - 



Fig. 11.17 Robotic arm large and flexible (RALF) used in 
EOA feedback experiments 


ments produced the desired undeflected position but 
would not serve for EOA tracking control. Obergfell 
and Book [1 1.95] synthesized an output based on strain 
feedback combined with EOA position that allowed 
tracking control as well. They first produced a passive 
system with deflection feedback obtained from optical 
measurements of the link deflection, and then devised 
a traditional feedback control of the tip position mea¬ 
sured by machine vision of the large two link arm 
shown in Fig. 11.17 for the experimental robot RALF 
(robot arm long and flexible). Figures 11.18 and 11.19 
compare the path for a large square traced in a plane 
with only the deflection feedback and with both deflec¬ 
tion and EOA position feedback. 

Robust Estimation 
for Flexible Link Systems 

The state feedback algorithms readily designed for lin¬ 
ear flexible link systems are dependent on information 
about states that cannot be directly measured and are 
theoretically infinite in number. Truncation of the states 
to represent the most dominant low frequency modes 
is a ready solution to the second problem and state 
estimators or observers present a viable approach to 
the first problem. (Estimator and observer are com¬ 
monly used interchangeably, but the term observer is 
more common will be used in this section.) For lin¬ 
ear systems a Luenberger observer or the stochastically 
based Kalman filter are commonly used. Unfortunately, 
observers require an accurate system model or the esti¬ 
mates may be poor and the observer, a dynamic system 


Tip ^-position (m) 



Fig. 11.18 Path followed without EOA position feedback. 
(Commanded trajectory is a square partially shown in light 
brown. Actual trajectory shown in dark brown) 
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of order as high as the order of the system itself, may 
even be unstable. Hence the robustness of the state ob¬ 
server is a critical subject for the control of flexible 
link systems. The quality of the control is ultimately 
the best measure of the quality of the estimator. Robust 
observer design has been the subject of considerable re¬ 
search and, in large part, mirrors the development of 
robust controllers. This is natural since an observer is 
a dynamic system which, like the plant itself, can be 
controlled to achieve a desired outcome. Control ef¬ 
fort of the observer is utilized to drive convergence of 
the estimated outputs of the observer to the measured 
outputs of the real system. Hence research on estima¬ 
tors based on H-infinity, fuzzy logic, neural network, 
sliding mode, adaptive, and other design techniques 
abound. In this chapter, two types of observers will be 
discussed: sliding mode observers and multiple model 
observers. 

For all observers the placement of sensors is cru¬ 
cial and for estimation of multiple modes it is desirable, 
although not theoretically essential, to use multiple sen¬ 
sors. Joint sensors (position and velocity) are typically 
available for general pose (joint) control, but for de¬ 
flection sensing strain gages and accelerometers are 
currently most appropriate. The MEMS accelerome¬ 
ters now readily available and inexpensive will be the 
principle choice here. Strain gages are more fragile 
and generally provide more complex readings of the 
mode behavior. Accelerometers are complicated by the 
fact that acceleration is not dependent on just states z 
which include the mode amplitudes and their deriva¬ 
tives, but also on the input u creating the link motion. 
Acceleration measurement of a point p on the link 
is affected by both the states z and the input u as 
follows, 

x v = mmc)\z 

+ {P( P mm 1 Q}u, 

where x p is the acceleration in the direction perpendic¬ 
ular to the beam axis, 0 is the matrix of eigenvectors, z 
is the state vector, composed of joint motion as well as 
beam deflections, p is the location of the sensor along 
the beam axis, a> 2 is the diagonal matrix of natural 
frequencies squared, C is the modal damping matrix, 
f(p) expresses the sensor motion in terms of the modal 
states variables, and Q relates the input to the system 
states. 

As Post [1 1.96] has explained, the ability of a given 
sensor to detect a mode depends on the amplitude of the 
mode at the sensor location. Overall the singular val¬ 
ues of the observability grammian, also used for testing 
the system observability, is a reasonable but not perfect 


Tip _v-position (m) 
5.24 

5.22 



Tip .v-position (m) 

Fig. 11.19 Path followed with EOA feedback from camera. 
(Commanded trajectory is a square partially shown in light 
brown. Actual trajectory shown in dark brown ) 

metric. A robust measurement requires the placement 
of sensors so as to avoid nodes (zero crossings) of the 
modes to be sensed even when parameters of the sys¬ 
tem change or are inaccurate. A single sensor (strain 
gage or accelerometer) is much more likely to have this 
problem than multiple sensors. The next sections de¬ 
pend on suitably placed sensors. For those modes not 
being controlled, one should minimize their influence 
by placing sensors near the node of those modes. This 
helps to avoid spillover of these modes that interferes 
with the system as approximated. 

Parametric Errors in System Model 
Parametric errors are inaccuracies in the parameters of 
a model with an accurate representation of the sys¬ 
tem structure. For flexible link systems this might arise 
from inaccurate payload mass, stiffness values, stiff¬ 
ness distribution, boundary conditions, etc., all of which 
lead to inaccurate mode shapes, natural frequencies 
and input coefficients. Errors are inevitable to some 
degree. As a consequence of parametric errors, one 
of the most powerful results of linear feedback con¬ 
trol of estimated states, the separation principle which 
says the design of the observer does not change the 
eigenvalues of the controlled system and vice versa, is 
compromised. The rapid convergence of observer error 
to zero will in general not occur with parametric er¬ 
ror. The net effect is a major blow to the effectiveness 
of the classical Luenberger observer. The Kalman fil¬ 
ter, which optimizes observer gains under conditions of 
Gaussian process and measurement noise, is likewise 


275 


Part A 111.2 
























Part A 111.2 


276 Part A I Robotics Foundations 


compromised. Without good state estimates the effec¬ 
tiveness of the powerful state feedback approaches are 
in jeopardy. 

One currently popular tack to improve the robust¬ 
ness of observers is the sliding mode observer. There 
are several variations of this approach, and the tech¬ 
nique by Wallcot and Zak [11.97,98] was followed by 
Post [1 1.96] in exploring the consequences of paramet¬ 
ric error. As typical, a sliding surface is defined which is 
intended to guide the estimate error to zero. The reach¬ 
ing phase moves from the initial error state to the sliding 
surface and when that surface is crossed the sign on the 
control changes. This rapid switching may be modified 
by establishing a boundary layer near the switching sur¬ 
face that substitutes a linear behavior. 

In this section, we will use z as the state vector, z 
as its estimate, L as the feedback of the observer, y as 
the measurement and K$ as the sliding mode gain. The 
hat on a variable will generally denote its estimate and 
an over dot its time derivative. The symbol A proceed¬ 
ing a symbol indicates the variation of the true system 
parameter from its model. The linear plant model is 
thus 

z = Az + Bu ; y = Cz + Du . 

The equation approaching the switching surface is 

z = Az + Bu + L(y — y) + Aj s sgn(>' - y) . 

When the boundary layer is included, the control 
is changed in a region near the switching sur¬ 
face where the design is essentially a conventional 
linear observer. For present purposes we will not 
employ a boundary layer but examine the pure 
sliding mode case, for which the error equation 
is 

e = Aoe — K r sgn(Ce) + AAz + A Bu , 

which clearly has a steady state error solution that is 
not zero except in the case that AAz = — A Bu which 
tends to occur when the overall system comes to rest. 
Thus the convergence of the sliding mode observer has 
similar issues found with the conventional linear ob¬ 
servers. They can be stable but can converge to the 
wrong steady state value leading to error, particularly 
after the observer’s state estimates should have con¬ 
verged to the slower changing system state. However, 
when the overall plant reaches steady state and z = 0, 
u = 0, the observer error will also converge to zero. In 
contrast, the usual intent is to design the observer to 
quickly converge to the true state estimate in order to 
appropriately control the plant. 


Successful Estimation 
with Model Parameter Uncertainty 
Models are seldom, if ever, exact in their representation 
of a physical system. In addition to error in parame¬ 
ter identification, parameters can change with time and 
operating conditions. For example, robots acquire and 
release payloads and move to different poses which 
may change parameters. As reported before, conver¬ 
gence of state observers is adversely affected by these 
changes which can lead to poor performance and even 
instability. In a recent work, Post [11.96] described 
a multiple model estimation scheme that successfully 
overcomes these limitations for a flexible link robot in 
both simulation and experiment. Other authors, e.g., 
Chalhoub and Kfoury [11.99] have used fuzzy logic 
techniques to address the needs for improving sliding 
mode observers. 

Since the true state of the system is not known, the 
steady state estimation error cannot be calculated, but 
if it is known to exist, Post has shown it to be propor¬ 
tional to parameter error. Hence multiple models based 
on a different set of sensors can be used to establish 
which is most accurate. The difference in estimated 
states based on two sensor selections S z (t) = z S| — z. S2 
is then a metric for choosing between alternative ob¬ 
servers in an array of observers with different assumed 
parameters. 

The observers themselves (conventional Luen- 
berger observers or Kalman filters) have the form 

z.v, 0) = Az si (?) + Bu(t) + L S1 [y V] (t) - y.v, (?)] . 

where z si is the estimate of the state vector z using the 
sensor array 1, A is the plant matrix, B is the input ma¬ 
trix, L S1 is the gain matrix to be designed, y si is the 
sensor measurement of sensor array 1 , and y si estimate 
of sensor measurement of sensor array 1 , and similarly 
for sensor S 2 . The difference between states and their 
estimates converges rapidly (relative to the states them¬ 
selves) to steady state and hence — z S2 converges to 
zero and hence 

8 z (t) = yj [AAz.it) + A Bu] 

+ * 2 [A C si zit) + A D n u(t)] 

+ *3 [A C S2 zit) + A D si u(t)) 

is found to be the steady state slowly changing (rel¬ 
ative to the observer dynamics) difference in the two 
observers based on sensors ,V| and s 2 . Here, the /,- are 
constant matrices based on the model parameters and 
not the true values of the system. A indicates the de¬ 
viation of the true matrices from the model’s matrices 
following that symbol. Note that subscripts si and s 2 
indicate different sensors involved in the measurement 
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and feed through matrices C and D, respectively. Also 
note that this equation is somewhat complicated by the 
feedthrough matrices occurring due to the use of ac¬ 
celerometers to sense the motion. 

Incorporating the specific representation of a linear 
flexible manipulator, the equations are 

_ / O(nXn) I(nXn) \ __ ( 0(„xm) 

V 1 W (nXn) — Q nX„)J \Q(nXn,) 

where C denotes modal damping and Q denotes the 
modal forcing coefficient. The sensor’s output is 

ysi = { - [1, Vr (Psi), ■■■• tniPsi)] ^( n x n )W(„ x „) 

- [ 1 , flip Si), ■■■, fniPsi )] ®(„x„)C(„x n ) | Z 
+ [ 1 , fl(Psi), • • • - tn(Psi)} 4>(nXn)Q(nXn)U ■ 



Let Psi = [1. if lip si) _, if nip si)] and one can attribute 

to the sub-matrices the parametric change matrices 

iAAzit ) + A Bu) = 

(aT)) [( -Aw2 “ A ^) + ■ 

(ACsiZ.it) + A D si u) = 

Psi& [(-A« 2 - Ac) z(t) + AQu(t)] , 

Sz(t) = [zi + XlPs^ + X3 

x [(-Aa> 2 - Ac);(f) + Agn] , 

which will be coalesced to give 

&z(0 f^(2 nXn)Y(t) 1 

where it can be seen that A consists of known model 
parameters and y is an unknown metric of parameter 
errors that can’t be solved from this equation. However, 
using the pseudo inverse we can produce a least squares 
metric y m (t) which evaluates the relative accuracy of 
two observers based on different models 

(A T A)~ 1 A T S*(t) = y m (t) . 

The smaller the norm of y m (t), the more accurate the 
observer based on a given model is expected to be. 

While various methods of using y m (t) can be con¬ 
trived, Post [11.96] utilized a switching approach he 
referred to as the multiple model switching adaptive 
estimator (MMSAE), visualized in Fig. 11.20. The 
weighting value Wj is based on y m ;(0 for the observer 



Fig. 11.20 Flow chart for the multiple model switching adaptive es¬ 
timator (MMSAE) 


based on a given model, then the estimate 2 , from that 
observer was used. Figure 11.21 shows schematically 
the flow inside the estimator blocks E\ through £ 5 . The 
parameter varied in Post’s experiments was arm pay- 
load. Multiple parameters could be varied but would 
lead to a higher dimensional formulation and poten- 



Fig. 11.21 Estimation module with weight scaling 



Fig. 11.22 Gantry robot used in MMSAE experiments (after 
Post [11.96]) 
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tially many estimators numbering N T = N 1 N 2 • • • N p for 
the variation of p parameters, each with N, values rep¬ 
resented. 

Post [11.96] conducted experiments with a linear 
model of the gantry robot as illustrated in Fig. 11.22 
and schematically modeled in Fig. 11.23, with the pa¬ 
rameters given in Table 11.1 and sensor placements at 
0.015, 0.25, and 0.42 m from the actuated end of the 
beam. He compared: 

• Conventional PID (l<s>Kll>H!llilJI). 

• Fixed gain linear-quadratic regulator (LQR) plus 
Luenberger state space designs (l^ssMUESiEIl). 

• MMSAE design observers and LQR controls. Five 
potential payloads were chosen with spacing in even 
increments between 0.2 and 0.3 kg, i. e., an end¬ 
point mass between 0.35 and 0.55. 

The penalty matrices were kept the same for the 
LQR controls but the resulting gains changed based on 
the model employed by the MMSAE selection. While 
various measures of performance could be demon¬ 
strated involving somewhat arbitrary selection of con¬ 



troller types, controller gains and metrics, the following 
characteristics are considered most critical: 

• Stability with various payloads. 

• Cycle completion time, with requirements for set¬ 
tling of vibration amplitude at prescribed stops. 

• Variation of cycle times with changing payloads. 

Since reasonable work cycles often pick up and 
drop off payloads, the test included these operations. 
Cycle times are graphed in Fig. 11.24. The light bars 
show the cycle times for the complete cycle with seg¬ 
ments of the task carrying the payload and others with 
the gripper empty. The dark bars show operation with 
no payload carried and only the gripper mass present for 
the entire cycle. In addition to the pair of bars labeled 
PID and MMSAE (with obvious meanings) are pairs la¬ 
beled SS 0.25 through SS 0.40. These latter labels refer 
to the designed-for tip mass (payload plus gripper). The 
most obvious message from this figure is that the state 
space control is a vast improvement over PID, reducing 
cycle time by about 90%. What is not obvious from this 
figure is that some means of gain change is needed to 
accommodate the payload variations. With a designed- 
for tip mass of 0.45 kg and above, the operation without 
payload was unstable. This is still below the actual tip 
mass of 0.5 kg. Hence MMSAE was essential for a suc¬ 
cessful completion of the task with these design criteria 
(i. e., LQR penalty matrices and resulting gains). The 
exact tip mass does not need to be known, only the 
range of masses. The near identical cycle times with 
and without the additional payload seen on the chart for 
MMSAE is also a practical advantage. 

11.2.4 Further Reading 


Fig. 11.23 Schematic of gantry robot model 
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Fig. 11.24 Cycle times with implemented control methods 
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The topic of flexible link robot control is widely cov¬ 
ered with papers on modeling and control. The reader 
wishing to probe further may want to begin with a paper 
giving some perspective on the nature of the problem. 
This was the goal of Book in [1 1.100]. Progress to date 
in this field from the theoretical perspective is signifi- 


Table 11.1 Parameters of the gantry robot model 


Parameter 

Value 

Unit 

Payload (ra t ) 

0.281 

kg 

Cart mass (ra c ) 

10 

kg 

Arm length (L) 

0.42 

m 

Elastic modulus ( E) 

7 xlO -10 

N/m 

Density (p) 

2700 

kg/m 3 

Area moment (/) 

1.0114 x 10“ 10 

m 4 

Belt stiffness (A"b) 

2.1814x10 s 

N/m 

Cart damper (Cd) 

100 

N s/m 

Structural damping coefficient (y) 

0.0005 

N s/m 
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cant and has been collected by Canudas de Witt et al. 
in [11.101]; this reference includes a wide range of 
research approaches, but the examples of the practi¬ 
cal application of this theory are more limited. The 
German Aerospace Center has produced lightweight 
space arms [11.102] and deployed them in space. 
The startup CAMotion, Inc. [11.103] has produced 
large lightweight industrial robots using the command¬ 
shaping approach to improve cycle time. High-speed 
arms such as the FlexPicker [11.104] achieve a light 
weight and avoid flexibility because of their parallel 
structure, but suffer the usual disadvantages of parallel 
actuation: restricted workspace considering the space 
occupied by the robot structure. 

Trautt and Bayo’s early exploration of non- 
minimum-phase inverses is described in [11.105]. This 
is a particularly interesting variation of the work by 


Kwon [11.83,84] in that it uses the frequency domain 
and deals with nonzero initial conditions. The con¬ 
cept of using two time scales, a fast time scale for the 
flexible dynamics and a slow time scale for the rigid 
dynamics, can be found in Siciliano and Book [1 1.106]. 
The singular perturbation approach was used earlier by 
Spong for elastic joint arms. Ghorbel and Spong also 
consider flexible links and contact with a rigid environ¬ 
ment in [11.107]. Achieving robustness is a problem, 
particularly with end-point sensors. Sliding mode con¬ 
trol [11.108] and passivity-based control [11.109] ad¬ 
dress this problem in effective but widely varying ways. 
While these seem to be the more promising methods 
of control, one will find many more approaches to this 
challenging control problem. 7/oo, adaptive, fuzzy, and 
other controllers have all been applied with varying de¬ 
grees of success. 
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Cartesian impedance control with damping off 

available from http://handbookofrobotics.org/view-chapter/11/videodetails/133 
Cartesian impedance control with damping on 

available from http://handbookofrobotics.org/view-chapter/11/videodetails/134 

Control laws for a single-link arm with an elastic joint 

available from http://handbookofrobotics.org/view-chapter/11/videodetails/135 

Feedforward/feedback law for path tracking with a KUKA KR15/2 robot 

available from http://handbookofrobotics.org/view-chapter/11/videodetails/136 

Trajectory generation and control for a KUKA IR 161/60 robot 

available from http://handbookofrobotics.org/view-chapter/11/videodetaiis/770 

Input shaping on a lightweight gantry robot 

available from http://handbookofrobotics.org/view-chapter/11/videodetails/777 
Inverse dynamics control for a flexible link 

available from http://handbookofrobotics.org/view-chapter/11/videodetails/778 
Rest-to-rest motion for a flexible link 

available from http://handbookofrobotics.org/view-chapter/11/videodetails/779 
PID response to impulse in presence of link flexibility 

available from http://handbookofrobotics.org/view-chapter/11/videodetaiis/780 

State feedback response to impulse in presence of link flexibility 

available from http://handbookofrobotics.org/view-chapter/11/videodetails/781 
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12. Robotic Systems Architectures 

and Programming 


David Kortenkamp, Reid Simmons, Davide Brugali 


Robot software systems tend to be complex. This 
complexity is due, in large part, to the need to con¬ 
trol diverse sensors and actuators in real time, in 
the face of significant uncertainty and noise. Robot 
systems must work to achieve tasks while moni¬ 
toring for, and reacting to, unexpected situations. 
Doing all this concurrently and asynchronously 
adds immensely to system complexity. 

The use of a well-conceived architecture, 
together with programming tools that support 
the architecture, can often help to manage that 
complexity. Currently, there is no single architec¬ 
ture that is best for all applications - different 
architectures have different advantages and dis¬ 
advantages. It is important to understand those 
strengths and weaknesses when choosing an ar¬ 
chitectural approach for a given application. 

This chapter presents various approaches to 
architecting robotic systems. It starts by defining 
terms and setting the context, including a re¬ 
counting of the historical developments in the 
area of robot architectures. The chapter then 
discusses in more depth the major types of ar¬ 
chitectural components in use today - behavioral 
control (Chap. 13), executives, and task planners 
(Chap. 14) - along with commonly used techniques 
for interconnecting connecting those components. 
Throughout, emphasis will be placed on pro¬ 
gramming tools and environments that support 
these architectures. A case study is then pre¬ 
sented, followed by a brief discussion of further 
reading. 
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12.1 Overview 

The term robot architecture is often used to refer to 
two related, but distinct, concepts. Architectural struc¬ 


ture refers to how a system is divided into subsystems 
and how those subsystems interact. The structure of 


Part A 






























Part A 112.1 


284 Part A I Robotics Foundations 


a robot system is often represented informally using tra¬ 
ditional boxes and arrows diagrams or more formally 
using techniques such as unified modeling language 
(UML) [12.1]. In contrast, architectural style refers 
to the computational concepts that underlie a given 
system. For instance, one robot system might use 
a publish-subscribe message passing style of commu¬ 
nication, while another may use a more synchronous 
client-server approach. 

All robotic systems use some architectural struc¬ 
ture and style. However, in many existing robot 
systems it is difficult to pin down precisely the 
architecture being used. In fact, a single robot 
system will often use several styles together. In 
part, this is because the system implementation 
may not have clean subsystem boundaries, blurring 
the architectural structure. Similarly, the architec¬ 
ture and the domain-specific implementation are of¬ 
ten intimately tied together, blurring the architectural 
style(s). 

This is unfortunate, as a well-conceived, clean 
architecture can have significant advantages in the 
specification, execution, and validation of robot sys¬ 
tems. In general, robot architectures facilitate develop¬ 
ment by providing beneficial constraints on the design 
and implementation of robotic systems, without being 
overly restrictive. For instance, separating behaviors 
into modular units helps to increase understandabil- 
ity and reusability, and can facilitate unit testing and 
validation. 

12.1.1 Special Needs of Robot Architectures 

In some sense, one may consider robot architectures 
as software engineering. However, robot architectures 
are distinguished from other software architectures be¬ 
cause of the special needs of robot systems. The most 
important of these, from the architectural perspective, 
are that robot systems need to interact asynchronously, 
in real time, with an uncertain, often dynamic, en¬ 
vironment. In addition, many robot systems need to 
respond at varying temporal scopes - from millisec¬ 
ond feedback control to minutes, or hours, for complex 
tasks. 

To handle these requirements, many robot architec¬ 
tures include capabilities for acting in real time, con¬ 
trolling actuators and sensors, supporting concurrency, 
detecting and reacting to exceptional situations, dealing 
with uncertainty, and integrating high-level (symbolic) 
planning with low-level (numerical) control. 

While the same capability can often be imple¬ 
mented using different architectural styles, there may 
be advantages of using one particular style over another. 
As an example, consider how a robot system’s style 


of communications can impact on its reliability. Many 
robot systems are designed as asynchronous processes 
that communicate using message passing. One popular 
communication style is client-server, in which a mes¬ 
sage request from the client is paired with a response 
from the server. An alternate communication paradigm 
is publish-subscribe, in which messages are broadcast 
asynchronously and all modules that have previously 
indicated an interest in such messages receive a copy. 
With client-server-style message passing, modules typ¬ 
ically send a request and then block, waiting for the 
response. If the response never comes (e.g., the server 
module crashes) then deadlock can occur. Even if the 
module does not block, the control flow still typically 
expects a response, which may lead to unexpected re¬ 
sults if the response never arrives or if a response to 
some other request happens to arrive first. In contrast, 
systems that use publish-subscribe tend to be more 
reliable: because messages are assumed to arrive asyn¬ 
chronously, the control flow typically does not assume 
any particular order in which messages are processed, 
and so missing or out-of-order messages tend to have 
less impact. 

12.1.2 Modularity and Hierarchy 

One common feature of robot architectures is mod¬ 
ular decomposition of systems into simpler, largely 
independent pieces. As mentioned above, robot sys¬ 
tems are often designed as communicating processes, 
where the communications interface is typically small 
and relatively low bandwidth. This design enables the 
processes/modules to handle interactions with the envi¬ 
ronment asynchronously, while minimizing interactions 
with one another. Typically, this decreases overall sys¬ 
tem complexity and increases overall reliability. 

Often, system decomposition is hierarchical - mod¬ 
ular components are themselves built on top of other 
modular components. Architectures that explicitly sup¬ 
port this type of layered decomposition reduce system 
complexity through abstraction. However, while hierar¬ 
chical decomposition of robotic systems is generally re¬ 
garded as a desirable quality, debate continues over the 
dimensions along which to decompose. Some architec¬ 
tures [12.2] decompose along a temporal dimension - 
each layer in the hierarchy operates at a characteris¬ 
tic frequency an order of magnitude slower than the 
layer below. In other architectures [12.3-6], the hier¬ 
archy is based on task abstraction - tasks at one layer 
are achieved by invoking a set of tasks at lower lev¬ 
els. In some situations, decomposition based on spatial 
abstraction may be more useful, such as when dealing 
with both local and global navigation [12.7]. The main 
point is that different applications need to decompose 
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problems in different ways, and architectural styles can 
often be found to accommodate those different needs. 

12.1.3 Software Development Tools 

While significant benefit accrues from designing sys¬ 
tems using well-defined architectural styles, many ar¬ 
chitectural styles also have associated software tools 
that facilitate adhering to that style during implemen¬ 
tation. These tools can take the form of libraries of 
functions calls, specialized programming languages, or 
graphical editors. The tools make the constraints of the 
architectural style explicit, while hiding the complexity 
of the underlying concepts. 

For instance, inter-process communication libraries, 
such as common object request broker architecture 
(CORBA) [12.8] and inter-process communication 
(IPC) package [12.9], make it easy to implement 
message passing styles, such as client-server and 
publish-subscribe, respectively. Languages, such as 
Subsumption [12.10] and Skills [12.11] facilitate the 


12.2 History 

Robot architectures and programming began in the 
late 1960s with the Shakey robot at Stanford Univer¬ 
sity [12.16] (Fig. 12.1). Shakey had a camera, a range 
finder, and bump sensors, and was connected to DEC 
PDP-10 and PDP-15 computers via radio and video 
links. Shakey’s architecture was decomposed into three 
functional elements: sensing, planning, and execut¬ 
ing [12.17]. The sensing system translated the camera 
image into an internal world model. The planner took 
the internal world model and a goal and generated 
a plan (i. e., a series of actions) that would achieve 
the goal. The executor took the plan and sent the ac¬ 
tions to the robot. This approach has been called the 
sense-plan-act (SPA) paradigm (Fig. 12.2). Its main ar¬ 
chitectural features are that sensing flowed into a world 
model, which was then used by the planner, and that 
plan was executed without directly using the sensors 
that created the model. For many years, robotic control 
architectures and programming focused almost exclu¬ 
sively on the SPA paradigm. 

12.2.1 Subsumption 

In the early 1980s, it became apparent that the SPA 
paradigm had problems. First, planning in any real- 
world domain took a long time, and the robot would 
be blocked, waiting for planning to complete. Second, 
and more importantly, execution of a plan without in¬ 


development of data-driven, real-time behaviors, while 
languages such as the execution support language 
(ESL) [12.12] and the the plan execution interchange 
language (PLEXIL) [12.13] provide support for reli¬ 
ably achieving higher-level tasks. Graphical editors, 
such as found in ControlShell [12.14], Labview [12.15] 
and open robot controller computer aided design (OR- 
CCAD) [12.6], provide constrained ways to assemble 
systems, and then automatically generate code that ad¬ 
heres to the architectural style. 

In each case, the tools facilitate the development 
of software in the given style and, more importantly, 
make it impossible (or, at least, very difficult) to vi¬ 
olate the constraints of that architectural style. The 
result is that systems implemented using such tools are 
typically easier to implement, understand, debug, ver¬ 
ify, and maintain. They also tend to be more reliable, 
since the tools provide well-engineered capabilities for 
commonly needed control constructs, such as message 
passing, interfacing with actuators and sensors, and 
handling concurrent tasks. 


volving sensing was dangerous in a dynamic world. 
Several new robot control architecture paradigms be- 



Fig. 12.1 Shakey (after [12.18]) 
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Fig. 12.2 The sense-plan-act (SPA) paradigm (af¬ 
ter [12.3], with permission) 
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Fig. 12.3 Example of the Subsumption architecture (after [12.3]) 


gan to emerge, including reactive planning, in which 
plans were generated quickly and relied more directly 
on sensed information instead of internal models [12.4, 
19]. The most influential work, however, was the sub¬ 
sumption architecture of Brooks [12.3]. A subsumption 
architecture is built from layers of interacting finite- 
state machines - each connecting sensors to actua¬ 
tors directly (Fig. 12.3). These finite-state machines 
were called behaviors (leading some to call Subsump¬ 
tion behavior-based or behavioral robotics [12.20]; see 
also Chap. 38). Since multiple behaviors could be ac¬ 
tive at any one time, Subsumption had an arbitration 
mechanism that enabled higher-level behaviors to over¬ 
ride signals from lower-level behaviors. For example, 
the robot might have a behavior that simply drives 
the robot in random directions. This behavior is al¬ 
ways active and the robot is always driving somewhere. 
A second, higher-level behavior could take sensor in¬ 
put, detect obstacles, and steer the robot away from 
them. It is also always active. In an environment with 
no obstacles, the higher-level behavior never generates 
a signal. However, if it detects an obstacle it overrides 
the lower-level behavior and steers the robot away. As 
soon as the obstacle is gone (and the higher-level be¬ 
havior stops sending signals), the lower-level behavior 
gets control again. Multiple, interacting layers of behav¬ 
iors could be built to produce more and more complex 
robots. 

Many robots were built using the subsumption ap¬ 
proach - most at MIT [12.21-23]. They were quite 
successful. Whereas SPA robots were slow and pon¬ 
derous, Subsumption robots were fast and reactive. 
A dynamic world did not bother them because they 
constantly sensed the world and reacted to it. These 
robots scampered around like insects or small rodents. 
Several behavioral architectures arose in addition to 


Subsumption, often with different arbitration schemes 
for combining the outputs of behaviors [12.24, 25]. 

A popular example of behavior-based architectures 
is Arkin’s motor-control schemas [12.26]. In this bi¬ 
ologically inspired approach, motor and perceptual 
schemas [12.27] are dynamically connected to one 
another. The motor schemas generate response vec¬ 
tors based on the outputs of the perceptual schemas, 
which are then combined in a manner similar to po¬ 
tential fields [12.28]. To achieve more complex tasks, 
the autonomous robot architecture (AuRA) architec¬ 
ture [12.29,30] added a navigation planner and a plan 
sequencer, based on finite-state acceptors (FSAs), to the 
reactive schemas. 

However, behavior-based robots soon reached lim¬ 
its in their capabilities. It proved very difficult to 
compose behaviors to achieve long-range goals and 
it proved almost impossible to optimize robot behav¬ 
ior. For example, a behavior-based robot that delivered 
mail in an office building could easily be built by sim¬ 
ply wandering around the office building and having 
behaviors looking for rooms and then overriding the 
wandering and entering the office. It was much more 
difficult to use the behavioral style of architecture to 
design a system that reasoned about the day’s mail to 
visit the offices in an optimal order to minimize delivery 
time. In essence, robots needed the planning capabili¬ 
ties of the early architectures wedded to the reactivity 
of the behavior-based architectures. This realization led 
to the development of layered, or tiered, robot control 
architectures. 

12.2.2 Layered Robot Control Architectures 

One of the first steps towards the integration of reac¬ 
tivity and deliberation was the reactive action packages 
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(RAPs) system created by Firby. In his thesis [12.31], 
we see the first outline of an integrated, three-layer ar¬ 
chitecture. The middle layer of that architecture, and 
the subject of the thesis, was the RAPs system. Firby 
also speculated on the form and function of the other 
two tiers, specifically with the idea of integrating classic 
deliberative methods with the ideas of the emerging sit¬ 
uated reasoning community, but those layers were never 
implemented. Later, Firby would integrate RAPs with 
a continuous low-level control layer [12.32]. 

Independently and concurrently, Bonasso at 
MITRE [12.33] devised an architecture that began at 
the bottom layer with robot behaviors programmed 
in the Rex language as synchronous circuits [12.34]. 
These Rex machines guaranteed consistent semantics 
between the agent’s internal states and those of the 
world. The middle layer was a conditional sequencer 
implemented in the goal as parallel programs (GAPPs) 
language [12.35], which would continuously activate 
and deactivate the Rex skills until the robot’s task was 
complete. This sequencer based on GAPPs was ap¬ 
pealing because it could be synthesized through more 
traditional planning techniques [12.36]. This work 
culminated in the three tiers (3T) architecture (named 
after its three tiers of interacting control processes - 
planning, sequencing, and real-time control), which 
has been used on many generations of robots [12.37], 

Architectures similar to 3T (Fig. 12.4) have been 
developed subsequently. One example is a three layer 
architecture for navigating through intricate situations 
(ATLANTIS) [12.38], which leaves much more control 
at the sequencing tier. In ATLANTIS, the delibera¬ 
tive tier must be specifically called by the sequencing 
tier. A third example is Sariclis ’ intelligent control ar¬ 
chitecture [12.39], The architecture begins with the 
servo systems available on a given robot and aug¬ 
ments them to integrate the execution algorithms of the 
next level, using VxWorks and the VME (Versa Mod¬ 
ule Europa) bus. The next level consists of a set of 
coordinating routines for each lower subsystem, e.g., 
vision, arm motion, and navigation. These are im¬ 
plemented in Petri net transducers (PNTs), a type of 
scheduling mechanism, and activated by a dispatcher 
connected to the organizational level. The organiza¬ 
tional level is a planner implemented as a Boltzmann 
neural network. Essentially the neural network finds 
a sequence of actions that will match the required com¬ 
mand received as text input, and then the dispatcher 
executes each of these steps via the network of PNT 
coordinators. 

The LAAS architecture for autonomous systems 
(LAAS) is a three-layered architecture that includes 
software tools to support development/programming at 
each layer [12.40]. The lowest layer (functional) con¬ 
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Fig. 12.4 Prototype three-tiered architecture 


sists of a network of modules, which are dynamically 
parameterized control and perceptual algorithms. Mod¬ 
ules are written in the generator of modules (GenoM) 
language, which produces standardized templates that 
facilitate the integration of modules with one another. 
Unlike most other three-layered architectures, the ex¬ 
ecutive layer is fairly simple - it is purely reactive 
and does no task decomposition. It serves mainly as 
a bridge - receiving task sequences from the highest 
layer and selecting and parameterizing tasks to send 
to the functional layer. The executive is written in the 
Kheops language, which automatically generates deci¬ 
sion networks that can be formally verified. At the top, 
the decision layer consists of a planner, implemented 
using the indexed time table (IxTeT) temporal plan¬ 
ner [12.41,42], and a supervisor, implemented using 
procedural reasoning system (PRS) [12.43, 44]. The su¬ 
pervisor is similar to the executive layer of other three¬ 
layered architectures - it decomposes tasks, chooses 
alternative methods for achieving tasks, and monitors 
execution. By combining the planner and supervisor 
in one layer, LAAS achieves a tighter connection be¬ 
tween the two, enabling more flexibility in when, and 
how, replanning occurs. The LAAS architecture actu¬ 
ally allows for multiple decisional layers at increasingly 
higher levels of abstraction, such as a high-level mission 
layer and a lower-level task layer. 

Remote agent is an architecture for the autonomous 
control of spacecraft [12.45]. It actually consists of 
four layers - a control (behavioral) layer, an execu¬ 
tive, a planner/scheduler, and mode identification and 
recovery (MIR) that combines fault detection and re¬ 
covery. The control layer is the traditional spacecraft 
real-time control system. The executive is the core of 
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the architecture - it decomposes, selects, and moni¬ 
tors task execution, performs fault recovery, and does 
resource management, turning devices on and off at ap¬ 
propriate times to conserve limited spacecraft power. 
The planner/scheduler is a batch process that takes 
goals, an initial (projected) state, and currently sched¬ 
uled activities, and produces plans that include flexible 
ranges on start and end times of tasks. The plan also in¬ 
cludes a task to reinvoke the planner to produce the next 
plan segment. An important part of the remote agent 
is configuration management - configuring hardware 
to support tasks and monitoring that the hardware re¬ 
mains in known, stable states. The role of configuration 
management is split between the executive, which uses 
reactive procedures, and MIR, which uses declarative 
models of the spacecraft and deliberative algorithms to 
determine how to reconfigure the hardware in response 
to detected faults [12.46]. 

The Syndicate architecture [12.47] extends the 3T 
model to multirobot coordination (Chap. 51). In this 
architecture, each layer interfaces not only with the lay¬ 
ers above and below, as usual, but also with the layers 
of the other robots at the same level (Fig. 12.5). In 
this way, distributed control loops can be designed at 
multiple levels of abstraction. The version of Syndicate 
in [12.48] used a distributed market-based approach for 
task allocation at the planning layer. 

Other noteworthy multitiered architectures have ap¬ 
peared in the literature. The National Bureau of Stan¬ 
dards (NBS) developed for the Aeronautics and Space 
Agency (NASA) the NASA/NBS standard reference 
model (NASREM) [12.2,49], later named real-time 
control system (RCS), was an early reference model 
for telerobotic control (Fig. 12.6). It is a many-tiered 
model in which each layer has the same general struc¬ 
ture, but operates at increasingly lower frequency as 
it moves from the servo level to the reasoning lev¬ 
els. With the exception of maintaining a global world 



Fig. 12.5 The Syndicate multirobot architecture 


model, NASREM, in its original inception, provided 
for all the data and control paths that are present in 
architectures such as 3T, but NASREM was a refer¬ 
ence model, not an implementation. The subsequent 
implementations of NASREM followed primarily the 
traditional sense-plan-act approach and were mainly 
applied to telerobotic applications, as opposed to au¬ 
tonomous robots. A notable exception was the early 
work of Blidberg/Chappell [12.50]. 

While three-layered robot architectures are very 
popular, various two-layered architectures have been 
investigated by researchers. The coupled layered archi¬ 
tecture for robot autonomy (CLARAty) was designed 
to provide reusable software for NASA’s space robots, 
especially planetary rovers [12.52,53]. CLARAty con¬ 
sists of a functional and a decision layer. The functional 
layer is a hierarchy of object-oriented algorithms that 
provide more and more abstract interfaces to the robot, 
such as motor control, vehicle control, sensor-based 
navigation, and mobile manipulation. Each object pro¬ 
vides a generic interface that is hardware independent, 
so that the same algorithms can run on different hard¬ 
ware. The decision layer combines planning and ex¬ 
ecutive capabilities. Similar to the LAAS architecture. 
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Fig. 12.6 The real-time control system (RCS) reference ar¬ 
chitecture (after [12.51]) 
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this is done to provide for tighter coordination between 
planning and execution, enabling continual replanning 
in response to dynamic contingencies. 

Closed-loop execution and recovery (CLEaR) 
[12.54] is one instantiation of the CLARAty de¬ 
cision layer. CLEaR combines the continuous ac¬ 
tivity scheduling, planning, execution and replan¬ 
ning (CASPER) repair-based planner [12.55] and 
the task description language (TDL) executive lan¬ 
guage [12.56]. CLEaR provides a tightly coupled ap¬ 
proach to goal- and event-driven behavior. At its heart 
is the capability to do fast, continuous replanning, based 
on frequent state and resource updates from execution 
monitoring. This enables the planner to react to many 
exceptional situations, which can be important in cases 
where there are many tasks, few resources, and sig¬ 
nificant uncertainty. In CLEaR, both the planning and 
executive components are able to handle resource con¬ 
flicts and exceptional situations - heuristics are used to 
decide which component should be involved in a given 
situation. The onboard autonomous science investiga¬ 
tion system (OASIS) system [12.57] extends CLEaR to 
include science data analysis so that the architecture 
can be driven by opportunistic science-related goals 
(such as finding unusual rocks or formations). OASIS 
is planner-centric, releasing tasks to the executive com¬ 
ponent just a few seconds before their scheduled start 
times. 

The cooperative intelligent real-time control archi¬ 
tecture (CIRCA) is a two-layered architecture con¬ 
cerned with guaranteeing reliable behavior [12.58, 59]. 
It embodies the notion of bounded reactivity - an ac¬ 
knowledgement that the resources of the robot are not 


12.3 Architectural Components 

We will take the three-tiered architecture as the pro¬ 
totype for the components discussed in this chapter. 
Figure 12.4 shows a typical three-tiered architecture. 
The lowest tier (or layer) is behavioral control and is 
the layer tied most closely to sensors and actuators. 
The second tier is the executive layer and is respon¬ 
sible for choosing the current behaviors of the robot 
to achieve a task. The highest tier is the task-planning 
layer and it is responsible for achieving long-term goals 
of the robot within resource constraints. Using the ex¬ 
ample of an office delivery robot, the behavioral layer 
is responsible for moving the robot around rooms and 
hallways, for avoiding obstacles, for opening doors, etc. 
The executive layer coordinates the behavioral layer to 
achieve tasks such as leaving a room, going to an office, 
etc. The task-planning layer is responsible for decid¬ 


always sufficient to guarantee that all tasks can be 
achieved. CIRCA consists of a real-time system (RTS) 
and an artificial intelligence (AI) system (AIS) that are 
largely independent. The RTS executes a cyclic sched¬ 
ule of test action pairs (TAPs) that have guaranteed 
worst-case behavior in terms of sensing the environ¬ 
ment and conditionally acting in response. It is the 
responsibility of the AIS to create a schedule that is 
guaranteed to prevent catastrophic failures from occur¬ 
ring, while running in hard real time. The AIS does this 
by planning over a state-transition graph that includes 
transitions for actions, exogenous events, and the pas¬ 
sage of time (e.g., if the robot waits too long, bad things 
can happen). The AIS tests each plan (set of TAPs) to 
see if it can actually be scheduled. If not, it alters the 
planning model, either by eliminating tasks (based on 
goal prioritization) or by changing parameters of be¬ 
haviors (e.g., reducing the robot’s velocity). The AIS 
continues this until it finds a plan that can be success¬ 
fully scheduled, in which case it downloads the new 
plan to the RTS in an atomic operation. 

Like CIRCA, ORCCAD is a two-layered architec¬ 
ture that is concerned with guaranteed reliability [12.6, 
60]. In the case of ORCCAD, this guarantee is achieved 
through formal verification techniques. Robot tasks 
(lower-level behaviors) and robot procedures (higher- 
level actions) are defined in higher-level languages that 
are then translated into the Esterel programming lan¬ 
guage [12.61], for logical verification, or the Timed- 
Argus language [12.62], for temporal verification. The 
verification is geared toward liveness and safety prop¬ 
erties, as well as verifying lack of contention for 
resources. 


ing the order of deliveries to minimize time, taking into 
account delivery priorities, scheduling, recharging, etc. 
The task-planning layer sends tasks (e.g., exit the room, 
go to office 110) to the executive. All these tiers need to 
work together and exchange information. The next sec¬ 
tion deals with the problem of connecting components 
to each other. We then discuss each component of the 
three-tiered prototype architecture in detail. 

12.3.1 Connecting Components 

All of the architecture components that have been dis¬ 
cussed in this chapter need to communicate with each 
other. They need to both exchange data and send com¬ 
mands. The choice of how components communicate 
(often called the middleware) is one of the most im- 


Part A 112.3 



Part A 112.B 


290 Part A I Robotics Foundations 


portant and most constraining of the many decisions 
a robot architecture designer will make. From previous 
experience, a great deal of the problems and a majority 
of the debugging time in developing robot architectures 
have to do with communication between components. 
In addition, once a communication mechanism is cho¬ 
sen it becomes extremely difficult to change, so early 
decisions persist for many years. Many developers roll 
their own communication protocols, usually built on top 
of Unix sockets. While this allows for customization 
of messages, it fails to take advantage of the reliabil¬ 
ity, efficiency, and ease of use that externally available 
communication packages provide. There are two ba¬ 
sic approaches to communication: client-server and 
publish-subscribe. 

Client-Server 

In a client-server (also called a point-to-point) commu¬ 
nication protocol, components talk directly with other 
components. A good example of this is remote pro¬ 
cedure call (RPC) protocols in which one component 
(the client) can call functions and procedures of an¬ 
other component (the server). A modern, and popular, 
variation on this is the common object request broker 
architecture (CORBA). CORBA allows for one com¬ 
ponent to call object methods that are implemented by 
another component. All method calls are defined in an 
interface definition language (IDL) file that is language 
independent. Every component uses the same IDL to 
generate code that compiles with component to handle 
communication. The advantage of this is that, when an 
IDL file is changed, all components that use that IDL 
can be recompiled automatically (by using make or sim¬ 
ilar code configuration tools). CORBA object request 
brokers (ORBs) are available for most major object- 
oriented languages. Although free ORBs are available, 
many commercial ORBs offer additional features and 
support. One disadvantage of CORBA is that it intro¬ 
duces quite a bit of additional code into applications. 
Some competitors have tried to address this issue, such 
as the internet communications engine (ICE), which 
has its own version of an IDL file called the specifica¬ 
tion language for ICE (SLICE). The biggest advantage 
of a client-server protocol is that the interfaces are 
very clearly defined in advance and everyone knows 
when the interface has changed. Another advantage is 
that it allows for a distributed approach to communica¬ 
tion with no central module that must distribute data. 
A disadvantage of client-server protocols is that they 
introduce significant overhead, especially if many com¬ 
ponents need the same information. It should be noted 
that CORBA and ICE also have a broadcast mechanism 
(called an event channel, or the notification service, in 
CORBA). 


Publish-Subscribe 

In a publish-subscribe (also called a broadcast) proto¬ 
col, a component publishes data and any other com¬ 
ponent can subscribe to that data. Typically, a cen¬ 
tralized process routes data between publishers and 
subscribers. In a typical architecture, most components 
both publish information and subscribe to information 
published by other components. There are several exist¬ 
ing publish-subscribe middleware solutions. A popular 
one for robotics is the real-time innovations’ (RTI) 
data distribution service (DDS), formerly the network 
data distribution service (NDDS) [12.63]. Another pop¬ 
ular publish-subscribe paradigm is IPC developed at 
Carnegie Mellon University [12.9]. Many publish-sub¬ 
scribe protocols are converging on using extensible 
markup language (XML) descriptions to define the data 
being published, with the added convenience of trans¬ 
mitting XML over HTTR which allows for significant 
interoperability with Web-based applications. Publish- 
subscribe protocols have a large advantage in being 
simple to use and having low overhead. They are es¬ 
pecially useful when it is unknown how many different 
components might need a piece of data (e.g., multiple 
user interfaces). Also, components do not get bogged 
down with repeated requests for information from many 
different sources. Publish-subscribe protocols are of¬ 
ten more difficult to debug because the syntax of the 
message is often hidden in a simple string type. Thus 
problems are not revealed until runtime when a com¬ 
ponent tries, and fails, to parse an incoming message. 
Publish-subscribe protocols are also not as readable 
when it comes to sending commands from one mod¬ 
ule to another. Instead of calling an explicit method or 
function with parameters, a command is issued by pub¬ 
lishing a message with the command and parameters 
in it and then having that message be parsed by a sub¬ 
scriber. Finally, publish-subscribe protocols often use 
a single central server to dispatch messages to all sub¬ 
scribers, providing a single point of failure and potential 
bottleneck. 

JAUS 

Recently, a standard has emerged in the defense 
robotics community not only for a communication pro¬ 
tocol but also for definitions of messages that are sent 
via that communication protocol. The joint architecture 
for unmanned systems (JAUS) defines a set of reusable 
messages and interfaces that can be used to command 
autonomous systems [12.64-66]. These reusable com¬ 
ponents reduce the cost of integrating new hardware 
components into autonomous systems. Reuse also al¬ 
lows for components developed for one autonomous 
system to be used by another autonomous system. JAUS 
has two components: a domain model and a reference 
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architecture. The domain model is a representation of 
the unmanned systems’ functions and information. It 
contains a description of the system’s functional and in¬ 
formational capabilities. The former includes models of 
the system’s maneuvering, navigational, sensing, pay- 
load, and manipulation capabilities. The latter includes 
models of the system’s internal data, such as maps 
and system status. The reference architecture provides 
a well-defined set of messages. Messages cause actions 
to commence, information to be exchanged, and events 
to occur. Everything that occurs in a JAUS system is 
precipitated by messages. This strategy makes JAUS 
a component-based, message-passing architecture. 

The JAUS reference architecture defines a system 
hierarchy, as shown in Fig. 12.7. The topology defines 
the system as the collection of vehicles, operator control 
units (OCUs), and infrastructure necessary to provide 
the full robotic capability. Subsystems are individual 
units (e.g., vehicles or OCUs) in the system. Nodes 
define a distinct processing capability within the archi¬ 
tecture and route JAUS messages to components. Com¬ 
ponents provide the different execution capabilities and 
respond directly to command messages. Components 
might be sensors (e.g., a SICK laser or a vision sensor), 
actuators (a manipulator or a mobile base) or payloads 
(weapons or task sensors). The topology (the layout of 
particular system, subsystems, nodes, and components) 
is defined by the system implementers based on task 
requirements. 

At the core of JAUS is a set of well-defined mes¬ 
sages. JAUS supports the following message types. 

• Command : Initiate mode changes or actions 

• Query : Used to solicit information from a compo¬ 
nent 

• Inform : Response to a query 

• Event set up: Passes parameters to set up an event 

• Event notification: Sent when the event happens 

JAUS has about 30 predefined messages that can 
be used to control robots. There are messages for con¬ 


trol of a robotic vehicle. For example, the global vector 
driver message performs closed-loop control of the de¬ 
sired global heading, altitude, and speed of a mobile 
vehicle. There are also sensor messages such as global 
pose sensor , which distributes the global position and 
orientation of the vehicle. There are also manipulation 
messages in JAUS. For example, the set joint positions 
message sets the desired joint position values. The set 
tool point message specifies the coordinates of the end- 
effector tool point in terms of the coordinate system at¬ 
tached to the end-effector. 

JAUS also has user-definable messages. Messages 
have headers that follow a specific format and include 
message type, destination address (e.g., system, subsys¬ 
tem, node, and component), priority, etc. While JAUS 
is primarily point to point, JAUS messages can also be 
marked as broadcast and distributed to all components. 
JAUS also defines coordinate systems for navigation 
and manipulation to ensure all components understand 
any coordinates sent to them. 

12.3.2 Behavioral Control 

Behavioral control represents the lowest level of con¬ 
trol in a robot architecture. It directly connects sensors 
and actuators. While these are typically hand-crafted 
functions written in C or C++, there have been spe¬ 
cialized languages developed for behavioral control, 
including ALFA [12.67], Behavioral Language [12.68], 
and Rex [12.69]. It is at this level that traditional con¬ 
trol theory (e.g., proportional-integral-derivative (PID) 
functions, Kalman filters, etc.) resides. In architectures 
such as 3T, the behavioral layer functions as a Brook- 
sian machine - that is, the layer is composed of a small 
number of behaviors (also called skills) that perceive 
the environment and carry out the actions of the robot. 

Example 

Consider an office delivery robot that operates in a typ¬ 
ical office building. The behavioral control layer con- 



Fig. 12.7 The JAUS reference ar¬ 
chitecture topology (after JAUS 
Reference Architecture docu¬ 
ment [12.64]) 
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tains the control functions necessary to move around in 
the building and carry out delivery tasks. Assuming the 
robot has an a priori map of the building, some possible 
behaviors for this robot include: 

1. Move to location while avoiding obstacles 

2. Move down hallway while avoiding obstacles 

3. Find a door 

4. Find a door knob 

5. Grasp a door knob 

6. Turn a door knob 

7. Go through door 

8. Determine location 

9. Find office number 

10. Announce delivery. 

Each of these behaviors ties sensors (vision, range 
sensing, etc.) to actuators (wheel motors, manipulator 
motors, etc.) in a tight loop. In architectures such as 
Subsumption, all behaviors are running concurrently 
with a hierarchical control scheme inhibiting the out¬ 
puts of certain behaviors. In AuRA [12.30], behaviors 
are combined using potential functions. Other architec¬ 
tures [12.25, 69] use explicit arbitration mechanisms to 
choose amongst potentially conflicting behaviors. 

In architectures such as 3T [12.37], not all of the be¬ 
haviors are active at the same time. Typically, only a few 
behaviors that do not conflict would be active at a time 
(e.g., behaviors 2 and 9 in the example above). The ex¬ 
ecutive layer (Sect. 12.3.3) is responsible for activating 
and deactivating behaviors to achieve higher-level tasks 
and to avoid conflicts between two behaviors competing 
for the same resource (e.g., an actuator). 

Situated Behaviors 

An important aspect of these behaviors is that they be 
situated. This means that the behavior works only in 
very specific situations. For example, behavior 2 above 
moves down a hallway, but this is appropriate only 
when the robot is situated in a hallway. Similarly, be¬ 
havior 5, which grasps a door knob, is appropriate only 
when the robot is within grasping distance of a door 
knob. The behavior is not responsible for putting the 
robot in the particular situation. However, it should rec¬ 
ognize that the situation is not appropriate and signal as 
such. 

Cognizant Failure 

A key requirement for behaviors is that they know when 
they are not working. This is called cognizant fail¬ 
ure [12.70]. For example, behavior 5 in our example 
(grasping the door knob) should not continually grasp at 
air if it is failing. More succinctly, the behavior should 
not continue to bang its head against the wall. A com¬ 
mon problem with early Subsumption robots is that the 


behaviors did not know they were failing and contin¬ 
ued to take actions that were not resulting in progress. 
It is not the job of the behavioral control layer to decide 
what to do in a failure situation; it is only necessary to 
announce that the behavior has failed and halt activity. 

Implementation Constraints 
The behavioral control layer is designed to bring the 
speed and reactivity of Subsumption to robot control. 
For this reason, the behaviors in the behavioral control 
layer need to follow the philosophies of Subsumption. 
In particular, the algorithms used for behaviors should 
be constant in state and time complexity. There should 
be little or no search at the behavioral control level, 
and little iteration. Behaviors should simply be trans¬ 
fer functions that take in signals (from sensors or other 
behaviors) and send out signals (to actuators or other 
behaviors), and repeat these several times a second. 
This will allow for reactivity to changing environments. 
More controversial is how much state should be al¬ 
lowed at the behavioral level. Brooks famously said 
several years ago to use the world as its own best 
model [12.68] - that is, instead of maintaining inter¬ 
nal models of the world and querying those models, 
the robot should instead directly sense the world to get 
its data. State such as maps, models, etc. belong at the 
higher levels of the three-tiered prototype architecture, 
not at the behavioral control layer. Certain exceptions, 
such as maintaining state for data filtering calculations, 
could be made on a case-by-case basis. Gat [12.71] ar¬ 
gues that any state kept at the behavioral layer should 
be ephemeral and limited. 

12.3.3 Executive 

The executive layer is the interface between the nu¬ 
merical behavioral control and the symbolic planning 
layers. The executive is responsible for translating high- 
level plans into low-level behaviors, invoking behav¬ 
iors at the appropriate times, monitoring execution, and 
handling exceptions. Some executives also allocate and 
monitor resource usage, although that functionality is 
more commonly performed by the planning layer. 

Example 

Continuing the example of an office delivery robot, the 
main high-level task would be to deliver mail to a given 
office. The executive would decompose this task into 
a set of subtasks. It may use a geometric path planner 
to determine the sequence of corridors to move down 
and intersections at which to turn. If there are door¬ 
ways along the route, a task would be inserted to open 
and pass through the door. At the last corridor, the ex¬ 
ecutive would add a concurrent task that looks for the 
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office number. The final subtasks would be to announce 
that the person has mail and to concurrently monitor 
whether the mail has been picked up. If it is not picked 
up after some period of time, an exception would be 
triggered that invokes some recovery action (perhaps 
announcing again, perhaps checking to make sure the 
robot is at the correct office, perhaps notifying the plan¬ 
ning layer to reschedule the delivery for a later time). 

Capabilities 

The example above illustrates many of the capabilities 
of the executive layer. First, the executive decomposes 
high-level tasks (goals) into low-level tasks (behav¬ 
iors). This is typically done in a procedural fashion: the 
knowledge encoded in the executive describes how to 
achieve tasks, rather than describing what needs to be 
done and having the executive figure out the how by 
itself. Sometimes, though, the executive may also use 
specialized planning techniques, such as the route plan¬ 
ner used in the example above. The decomposition is 
typically a hierarchical task tree (Fig. 12.8), with the 
leaves of the task tree being invocations and parameter- 
izations of behaviors. 

Besides decomposing tasks into subtasks, execu¬ 
tives add and maintain temporal constraints between 
tasks (usually between sibling tasks only, but some ex¬ 
ecutive languages permit temporal constraints between 
any pair of tasks). The most common constraints are se¬ 
rial and concurrent, but most executives support more 
expressive constraint languages, such as having one 
task begin 10 s after another one starts or having one 
task end when another ends. 



Fig. 12.8 Hierarchical task tree for mail-delivery task 
(lozenge nodes are interior; rectangular nodes are leaves; 
hexagonal node is an execution monitor; solid arrows are 
parent-child relationships; dashed arrows are sequential 
constraints) 


The executive is responsible for dispatching tasks 
when their temporal constraints are satisfied. In some 
executives, tasks may also specify resources (e.g., the 
robot’s motors or camera) that must be available before 
the task can be dispatched. As with behaviors, arbitrat¬ 
ing between conflicting tasks can be a problem. In the 
case of executives, however, this arbitration is typically 
either programmed in explicitly (e.g., a rule that says 
what to do in cases where the robot’s attempt to avoid 
obstacles takes it off the preferred route) or handled us¬ 
ing priorities (e.g., recharging is more important than 
mail delivery). 

The final two important executive capabilities are 
execution monitoring and error recovery. One may 
wonder why these capabilities are needed if the un¬ 
derlying behaviors are reliable. There are two reasons. 
First, as described in Sect. 12.3.2, the behaviors are sit¬ 
uated, and the situation may change unexpectedly. For 
instance, a behavior may be implemented assuming that 
a person is available to pick up the mail, but that may 
not always be the case. Second, in trying to achieve 
some goal, the behavior may move the robot into a state 
that is unexpected by the executive. For instance, people 
may take advantage of the robot’s obstacle avoidance 
behavior to herd it into a closet. While the behavior 
layer may, in fact, keep the robot safe in such situations, 
the executive needs to detect the situation in order to get 
the robot back on track. 

Typically, execution monitoring is implemented as 
a concurrent task that either analyzes sensor data di¬ 
rectly or activates a behavior that sends a signal to the 
executive when the monitored situation arises. These 
correspond to polling and interrupt-driven monitors, re¬ 
spectively. 

Executives support various responses to monitors 
being triggered. A monitor may spawn subtasks that 
handle the situation, it may terminate already spawned 
subtasks, it may cause the parent task to fail, or it may 
raise an exception. The latter two responses involve the 
error recovery (also called exception handling) capa¬ 
bility. Many executives have tasks return status values 
(success or failure) and allow parent tasks to execute 
conditionally based on the return values. Other ex¬ 
ecutives use a hierarchical exception mechanism that 
throws named exceptions to ancestor nodes in the task 
tree. The closest task that has registered a handler for 
that exception tries to handle it; if it cannot, it rethrows 
the exception up the tree. This mechanism, which is in¬ 
spired by the exception handling mechanisms of C++, 
Java, and Lisp, is strictly more expressive than the re¬ 
turn-value mechanism, but it is also much more difficult 
to design systems using that approach, due to the non¬ 
local nature of the control flow. 
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Implementation Constraints 
The underlying formalism for most executives is a hi¬ 
erarchical finite-state controller. Petri nets [12.72] are 
a popular choice for representing executive functions. 
In addition, various languages have been developed 
specifically to assist programmers in implementing 
executive-level capabilities. We briefly discuss as¬ 
pects of several of these languages: reactive action 
packages (RAPs) [12.4,31], the procedural reasoning 
system (PRS) [12.43,44], the execution support lan¬ 
guage (ESL) [12.12], the task description language 
(TDL) [12.56], and the plan execution interchange lan¬ 
guage (P I.E XIT.) [12.13], 

These languages all share features and exhibit dif¬ 
ferences. One distinction is whether the language is 
stand-alone (RAPs, PRS, PLEXIL) or an extension of 
an existing language (ESL is an extension of Common 
Lisp; TDL is an extension of C++). Stand-alone lan¬ 
guages are typically easier to analyze and verify, but 
extensions are more flexible, especially with respect to 
integration with legacy software. While stand-alone ex¬ 
ecutive languages all support interfaces to user-defined 
functions. These interfaces are usually limited in ca¬ 
pability (such as what types of data structures can be 
passed around). 

All of these executive languages provide support 
for hierarchical decomposition of tasks into subtasks. 
All except PLEXIL allow for recursive invocation of 
tasks. RAPs, TDL, and PLEXIL have syntax that dis¬ 
tinguishes leaf nodes of the task tree/graph from interior 
nodes. 

All these languages provide capabilities for express¬ 
ing conditionals and iteration, although with RAPs and 
PLEXIL these are not core-language constructs, but 
must be expressed as combinations of other constructs. 
Except for TDL, the languages all provide explicit sup¬ 
port for encoding pre- and post-conditions of the tasks 
and for specifying success criteria. With TDL, these 
concepts must be programmed in, using more primi¬ 
tive constructs. The stand-alone languages all enable 
local variables to be defined within a task description, 
but provide for only limited computation with those 
variables. Obviously, with extension languages the full 
capability of the base language is available for defining 
tasks. 

All the languages support the simple serial (se¬ 
quential) and concurrent (parallel) temporal constraints 
between tasks, as well as timeouts that can be specified 
to trigger after waiting a specified amount of time. In 
addition, TDL directly supports a wide range of tem¬ 
poral constraints - one can specify constraints between 
the start and end times of tasks (e.g., task B starts after 
task A starts or task C ends after task D starts) as well 
as metric constraints (e.g., task B starts 10 seconds after 


task A ends or task C starts at lpm). ESL and PLEXIL 
support the signaling of events (e.g., when tasks tran¬ 
sition to new states) that can be used to implement 
similarly expressive types of constraints. In addition, 
ESL and TDL support task termination based on the 
occurrence of events (e.g., task B terminates when task 
A starts). 

The languages presented differ considerably in how 
they deal with execution monitoring and exception han¬ 
dling. ESL and TDL both provide explicit execution 
monitoring constructs and support exceptions that are 
thrown and then caught by registered handlers in a hi¬ 
erarchical fashion. This type of exception handling is 
similar to that used in C++, Java, and Lisp. ESL and 
TDL also support clean-up procedures that can be in¬ 
voked when tasks are terminated. RAPs and PLEXIL 
use return values to signal failure, and do not have hi¬ 
erarchical exception handling. PLEXIL, though, does 
support clean up procedures that are run when tasks 
fail. PRS has support for execution monitoring, but not 
exception handling. ESL and PRS support the notion 
of resources that can be shared. Both provide support 
for automatically preventing contention amongst tasks 
for the resources. In the other executive languages, this 
must be implemented separately (although there are 
plans to extend PLEXIL in this area). 

Finally, RAPs, PRS and ESL all include a symbolic 
database (world model) that connects either directly 
to sensors or to the behavior layer to maintain syn¬ 
chrony with the real world. Queries to the database are 
used to determine the truth of preconditions, to deter¬ 
mine which methods are applicable, etc. PLEXIL has 
the concept of a lookup that performs a similar func¬ 
tion, although it is transparent to the task how this is 
implemented (e.g., by a database lookup or by invok¬ 
ing a behavior-level function). TDL leaves it up to the 
programmer to specify how the tasks connect to the 
world. 

12.3.4 Planning 

The planning component of our prototype layered ar¬ 
chitecture is responsible for determining the long-range 
activities of the robot based on high-level goals. Where 
the behavioral control component is concerned with the 
here-and-now and the executive is concerned with what 
has just happened and what should happen next, the 
planning component looks towards the future. In our 
running example of an office delivery robot, the plan¬ 
ning component would look at the day’s deliveries, the 
resources of the robot, and a map, and determine the op¬ 
timal delivery routes and schedule, including when the 
robot should recharge. The planning component is also 
responsible for replanning when the situation changes. 
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For example, if an office is locked, the planning compo¬ 
nent would determine a new delivery schedule that puts 
that office’s delivery later in the day. 

Types of Planning 

Chapter 9 describes approaches to robot planning in de¬ 
tail. Here, we summarize issues with respect to different 
types of planners as they relate to layered architectures. 

The two most common approaches used are hierar¬ 
chical task net (HTN) planners and planner/schedulers. 
HTN planners [12.73,74] decompose tasks into sub¬ 
tasks, in a manner similar to what many executives do. 
The main differences are that HTN planners typically 
operate at higher levels of abstraction, take resource uti¬ 
lization into account, and have methods for dealing with 
conflicts between tasks (e.g., tasks needing the same re¬ 
sources, or one task negating a precondition needed by 
another task). The knowledge needed by HTN planners 
is typically fairly easy to specify, since one indicates 
directly how tasks are to be achieved. 

Planner/schedulers [12.75,76] are useful in do¬ 
mains where time and resources are limited. They 
create high-level plans that schedule when tasks should 
occur, but typically leave it to the executive to determine 
exactly how to achieve the tasks. Planner/schedulers 
typically work by laying out tasks on time lines, with 
separate time lines for the various resources that are 
available on the robot (motors, power, communication, 
etc.). The knowledge needed by planner/schedulers 
includes the goals that tasks achieve, the resources 
they need, their duration, and any constraints between 
tasks. 

Many architectures provide for specialized planning 
experts that are capable of solving particular prob¬ 
lems efficiently. In particular, these include motion 
planners, such as path planners and trajectory plan¬ 
ners. Sometimes, the planning layer of the architecture 
invokes these specialized planners directly; in other ar¬ 
chitectural styles, the motion planners are part of the 
lower levels of the architecture (the executive, or even 
the behavioral layer). Where to put these specialized 
planners is often a question of style and performance 
(Sect. 12.5). 

Additionally, some architectures provide for mul¬ 
tiple planning layers [12.40,45,77]. Often, there is 
a mission planning layer at the very top that plans 
at a very abstract level, over relatively long periods 
of time. This layer is responsible mainly for select¬ 
ing which high-level goals are to be achieved over the 
next period of time (and, in some cases, determining 
in which order to achieve them) in order to maximize 
some objective function (e.g., net reward). The lower 
task planning layer is then responsible for determining 
exactly how and when to achieve each goal. This break¬ 


down is usually done for efficiency reasons, since it is 
difficult to plan simultaneously at both a detailed level 
and over a long time horizon. 

Integrating Planning and Execution 
There are two main approaches to the integration of 
the planning and execution components in robotic ar¬ 
chitectures. The first approach is that the planning 
component is invoked as needed by the executive 
and returns a plan. The planning component is then 
dormant until called again. Architectures such as AT¬ 
LANTIS [12.71] and Remote Agent [12.45] use this 
approach, which requires that the executive either leave 
enough time for planning to complete or that it safes 
the system until planning is complete. In the Remote 
Agent, for instance, a special planning task is explicitly 
scheduled. 

The second approach is that the planning compo¬ 
nent sends high-level tasks down to the executive as 
required and monitors the progress of those tasks. If 
tasks fail, replanning is done immediately. In this ap¬ 
proach, the planning component is always running and 
always planning and replanning. Signals must pass in 
real time between the planner and the executive to keep 
them synchronized. Architectures such as 3T [12.37] 
use this second approach. The first approach is useful 
when the system is relatively static, so that planning 
can occur infrequently, at relatively predictable times. 
The second approach is more suited to dynamic envi¬ 
ronments, where replanning is more frequent and less 
predictable. 

Other decisions that need to be made when inte¬ 
grating planning and execution are when to stop task 
decomposition, where to monitor plan execution, and 
how to handle exceptions. By planning all the way 
down to primitive actions/behaviors, the planner has 
a very good notion of what will happen during ex¬ 
ecution, but at a price of much more computation. 
Also, some task decompositions are easier to describe 
procedurally (using an executive language) rather than 
declaratively (using a planning language). Similarly, 
monitoring at the executive level tends to be more ef¬ 
ficient, since the monitoring happens closer to the robot 
sensors, but the planner may be able to use its more 
global knowledge to detect exceptions earlier and/or 
more accurately. With respect to handling exceptions, 
executives can handle many on their own, at the price 
of breaking the expectations used by the planner in 
scheduling tasks. On the other hand, having exceptions 
handled by the planner typically involves replanning, 
which can be computationally expensive. 

For all these integration issues, however, a middle 
ground usually exists. For instance, one can choose to 
decompose some tasks more deeply than others, or han- 
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die certain exceptions in the executive and others in the a compromise and is determined by analyzing the do- 
planner. In general, the right approach usually involves main in detail (Sect. 12.5). 


12.4 Case Study - GRACE 

In this section, we present the architecture of a fairly 
complex autonomous mobile robot. Graduate robot at¬ 
tending conference (GRACE) resulted from the efforts 
of five research institutions (Carnegie Mellon, Naval 
Research Laboratory, Northwestern University, Met- 
rica, and Swarthmore College) to tackle the American 
Association for Artificial Intelligence (AAAI) Robot 
Challenge. The Challenge was for a robot to attend the 
AAAI National Conference on Artificial Intelligence as 
a participant - the robot must find the registration desk 
(without knowing the layout of the convention center 
beforehand), register for the conference, and then, af¬ 
ter being provided with a map, find its way to a given 
location in time to give a technical talk about itself. 

The architectural design of the robot was particu¬ 
larly important given the complexity of the task and the 
need to integrate techniques that had been previously 
developed by the five institutions. These techniques 
included localization in a dynamic environment, safe 
navigation in the presence of moving people, path 
planning, dynamic replanning, visual tracking of peo¬ 
ple, signs and landmarks, gesture and face recognition, 
speech recognition and natural language understanding, 
speech generation, knowledge representation, and so¬ 
cial interaction with people. 

GRACE is built on top of an real world inter¬ 
face (RWI) B21 base and has an expressive computer- 
animated face projected on a flat-panel liquid-crystal 
display (LCD) screen (Fig. 12.9). Sensors that come 
with the B21 include touch, infrared, and sonar sensors. 



Fig. 12.9 The robot GRACE 


Near the base is a SICK scanning laser range finder 
that provides a 180° field of view. In addition, GRACE 
has several cameras, including a stereo camera head on 
a pan-tilt unit (PTU) built by Metrica TRACLabs and 
a single-color camera with pan-tilt-zoom capability, 
built by Canon. GRACE can speak using a high-qual¬ 
ity speech-generation software (Festival), and receive 
speech responses using a wireless microphone head¬ 
set (a Shure TC computer wireless transmitter/receiver 
pair). 

The behavioral layer of the architecture consisted of 
individual processes that controlled particular pieces of 
hardware. These programs provided abstract interfaces 
to either control the hardware or return information 
from sensors. To accommodate the different coding 
styles of the various groups involved, both synchronous, 
blocking and asynchronous, nonblocking calls were 
supported by most of the interfaces (for the nonblock¬ 
ing calls, the interfaces allowed programmers to specify 
a callback function to be invoked when data was re¬ 
turned). Interfaces at the behavioral level included robot 
motion and localization (this interface also provided 
laser information), speech recognition, speech genera¬ 
tion, facial animation, color vision, and stereo vision 
(Fig. 12.10). 

The architecture used individual processes for each 
of the behavioral capabilities, mainly because the un¬ 
derlying code had been developed by different orga¬ 
nizations. While having a large number of processes 
run concurrently is somewhat inefficient, trying to inte¬ 
grate everything into a monolithic process was thought 
to be too difficult. In addition, the use of separate pro¬ 
cesses facilitated development and debugging, since 
one needed to run only those aspects of the system that 
were being tested. 

The executive layer consisted of separate programs 
for achieving each subtask of the challenge - finding the 
registration desk, riding the elevator, standing in line, 
interacting with the person at the desk, navigating to the 
talk, and giving the talk (Fig. 12.10). As is common in 
many implemented robotic systems, the GRACE archi¬ 
tecture did not have a planning layer - since the high- 
level plan was fixed and relatively straightforward, it 
was coded explicitly. Several of the executive-layer pro¬ 
grams were written using TDL (Sect. 12.3.3), which 
facilitated concurrent control and monitoring of the var¬ 
ious tasks. 











Robotic Systems Architectures and Programming I 12.4 Case Study - GRACE 297 



Fig. 12.10 GRACE’S architectural structure 


One particularly involved task was finding the regis¬ 
tration desk (recall that GRACE had no idea where the 
booth was, or even what the convention center looked 
like). TDL was used to create a finite-state machine that 
allowed GRACE to maintain multiple goals, such as us¬ 
ing an elevator to get to a particular floor and following 
directions to find the elevator (Fig. 12.11). The top-level 
goal was to find the registration desk. Intermediate sub¬ 
goals were created as GRACE interacted with people 
to determine the directions to the desk. If there were 
no directions to follow, GRACE performed a random 
walk until a person was detected using its laser scanner. 
GRACE then engaged in conversation with the person 
to obtain directions. GRACE could handle simple com¬ 
mands, such as turn left and go forward five meters, as 
well as higher-level instructions, such as take the ele¬ 
vator and turn left at the next intersection. In addition, 
GRACE could ask questions, such as am I at the regis¬ 
tration desk? and is this the elevator? The TDL-based 
finite-state machine was used to determine which inter¬ 
actions were appropriate at various times and to prevent 
the robot from getting confused. 

Communication between processes used the in¬ 
terprocess communication (IPC) messaging pack¬ 
age [12.9,78], IPC supports both publish-subscribe 
and client-server messaging, and enables complex data 
structures to be passed transparently between processes. 
One side benefit of using IPC to communicate between 
processes was the ability to log all message traffic (both 
message name and data content). This proved invalu¬ 
able, at times, in determining why the system failed 


Destination 



to act as expected - did a process send out a message 
with invalid data? Did it fail to send out a message in 
a timely fashion? Was the receiving process blocked, 
for some reason? Was there a timing issue? While wad¬ 
ing through the message traffic was often tedious, in 


Fig. 12.11 Finite-state machine for GRACE’S task for following di¬ 
rections to the registration booth 
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some cases it was the only way to catch intermittent 
bugs. 

In July 2002, GRACE successfully completed the 
challenge at the Shaw Convention Centre in Edmonton, 
Canada. The processes at the behavioral level gener¬ 
ally worked as anticipated - this was largely attributed 
to the fact that those modules were ported from previ¬ 
ously developed (and hence well-tested) systems. While 
generally functional, the executive-level processes had 


12.5 The Art of Robot Architectures 

Designing a robot architecture is much more of an art 
than a science. The goal of an architecture is to make 
programming a robot easier, safer, and more flexible. 
Thus, the decisions made by a developer of a robot ar¬ 
chitecture are influenced by their own prior experiences 
(e.g., what programming languages they are familiar 
with), their robot and its environment, and the tasks that 
need to be performed. The choice of a robot architecture 
should not be taken lightly, as it is the authors’ experi¬ 
ences that early architectural decisions often persist for 
years. Changing robot architectures is a difficult propo¬ 
sition and can set back progress while a great deal of 
code is reimplemented. 

The art of designing a robotic architecture starts 
with a set of questions that the designer needs to ask. 
These questions include: 

• What are the tasks the robot will be performing? 
Are they long-term tasks? Short-term? User-initi¬ 
ated? Robot-initiated? Are the tasks repetitive or 
different across time? 

• What actions are necessary to perform the tasks? 
How are those actions represented? How are those 
actions coordinated? How fast do actions need to be 
selected/changed? At what speed do each of the ac¬ 
tions need to run in order to keep the robot safe? 

• What data is necessary to do the tasks? How will 
the robot obtain that data from the environment or 
from the users? What sensors will produce the data? 
What representations will be used for the data? 
What processes will abstract the sensory data into 
representations internal to the architecture? How of¬ 
ten does the data need to be updated? How often can 
it be updated? 

• What computational capabilities will the robot 
have? What data will these computational capa¬ 
bilities produce? What data will they consume? 
How will the computational capabilities of a robot 
be divided, structured, and interconnected? What 
is the best decomposition/granularity of computa¬ 


more problems with off-nominal situations. This is 
largely attributed to problems in sensor interpretation, 
as well as mistaken assumptions about what the con¬ 
vention center was going to look like (for instance, 
it turned out that some of the barriers were made of 
glass, which is largely invisible to the laser). Overall, 
however, the architecture itself worked as expected, en¬ 
abling a large body of complex software to be integrated 
rather quickly and operate together effectively. 


tional capabilities? How much does each compu¬ 
tational capability have to know about the other 
capabilities? Are there legacy computational ca¬ 
pabilities (from other robots, other robot projects, 
etc.) that will be used? Where will the different 
computational capabilities reside (e.g., onboard or 
offboard)? 

• Who are the robot’s users? What will they com¬ 
mand the robot to do? What information will they 
want to see from the robot? What understanding do 
they need of the robot’s computational capabilities? 
How will the user know what the robot is doing? Is 
the user interaction peer to peer, supervisory, or as 
a bystander? 

• How will the robot be evaluated? What are the suc¬ 
cess criteria? What are the failure modes? What is 
the mitigation for those failure modes? 

• Will the robot architecture be used for more than 
one set of tasks? For more than one kind of robot? 
By more than one team of developers? 

Once designers have answers to all (or most) of 
these questions, they can then begin building some use 
cases for the types of operations they want the robot 
to perform and how they want users to interact with 
it. These use cases should specify the outward behav¬ 
ior of the robot with respect to its environment and 
its users. From the use cases, an initial partitioning of 
robot functionality can be developed. This partition¬ 
ing should be accompanied by a sequence diagram that 
shows the transfer of information and control over time 
amongst the various components of the robot architec¬ 
ture [12.79]. After this, a more formal specification of 
the interfaces between architectural components can be 
developed. This may be done using a language such as 
the interface definition language (IDL) of CORBA or 
by defining the messages to be distributed in a publish- 
subscribe protocol. This is an important step, as once 
implementation begins it is very costly to change inter¬ 
faces. If an interface does change, all stakeholders need 
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to be notified and need to agree to the change. The most 
common integration problems in robot architectures are 
mismatches between what components expect and what 
they are receiving in the way of data. 

An advantage of tiered architectures with clear in¬ 
terface definitions is that the different layers can be 
developed in parallel. The behavioral control layer can 
be implemented and tested on the robot using a hu¬ 
man as an executive. The executive can be implemented 
and tested using state machine stubs for the expected 
behaviors on the robot. The planning layer can be im¬ 
plemented and tested using stubs for the tasks in the 
executive. The stubs merely acknowledge that they 


were called and report back appropriately. Then, the 
tiers can be integrated to test timing and other runtime 
issues. This parallel approach speeds up the develop¬ 
ment of a robot architecture, but is possible only if the 
roles and interfaces between components are clearly 
defined and respected. There is still considerable real¬ 
time debugging necessary during integration. In our 
experiences, most of the development time in robot ar¬ 
chitectures is still spent on the behavioral control layer - 
that is, sensing and acting are still the hard parts of robot 
control, as compared to execution and planning. Having 
a good, robust behavioral control layer goes a long way 
towards having a competent robot architecture. 


12.6 Implementing Robotic Systems Architectures 


Robot System Architectures offer a coarse-gain view 
of the structure of a robot system, which is modelled 
according to the functional decomposition (or compo¬ 
sition) of parts. Unfortunately, there are concerns that 
cannot be effectively described and modularised using 
the concept of composition of high cohesion functional 
modules. Such concerns relate to the software systems 
as a whole hence crosscutting their modular structure. 
Non-functional requirements such as real-time perfor¬ 
mance, fault tolerance, and safety, are typical examples 
of properties that emerge from multiple parts of a sys¬ 
tem and cannot be confined into individual modules. In 
order to meet these requirements, software development 
efforts are mainly focused on delivering highly efficient 
implementations of control applications that exploit at 
best the capabilities of specific hardware platforms to 
perform a specific set of tasks in specific operational 
environments. 

The race towards performance and the pressure 
to develop proof-of-concept code to test a new the¬ 
ory [12.80] push robotic engineers to neglect other 
quality attributes of a software system, such as main¬ 
tainability, interoperability, and reusability. As a result, 
a huge corpus of software applications, which imple¬ 
ment the entire spectrum of robot functionality, algo¬ 
rithms, and control paradigms, is potentially available 
in robotic research laboratories as open source libraries. 
Unfortunately, they are often not reusable even in 
slightly different application scenarios, because the as¬ 
sumptions and constraints about the computational and 
robotics hardware, and the software and operational en¬ 
vironments are hidden and hard coded in the software 
implementation. 

A large variety of today’s robotic systems is de¬ 
signed according to the relatively small number of 
robot control architecture paradigms discussed in the 


previous sections. Nevertheless, the software that im¬ 
plements their control applications differs significantly 
from system to system. These differences relate, for 
example, to the data structures defined to store rele¬ 
vant information (e.g., the map of the environment, the 
robot kinematic model), the application programming 
interfaces (APIs) to drive sensors and actuators [12.81], 
the information model used to represent key concepts 
(e.g., geometric relations and coordinate representa¬ 
tions). These differences indicate that the most common 
robot functionalities have been re-implemented from 
scratch innumerable times. 

During the last few years, many ideas from software 
engineering (such as component-based development 
and model-driven engineering) have been progressively 
introduced in the construction of robotic software sys¬ 
tems to simplify their development and improve their 
quality (see [12.82] for a survey). 

Modern robot control systems are typically de¬ 
signed as (logically) distributed component-based sys¬ 
tems [12.83,84]. Here, components are units of im¬ 
plementation and represent a code-based way of con¬ 
sidering the system. As an analogy, in the electronics 
domain re-usable off-the-shelf electronic components 
have been available for many years in the form of inte¬ 
grated chips (ICs) which can be bought and deployed in 
other parts of the world. This is possible because each 
IC packages a clear set of functionality and provides 
a well-defined external interface. 

In Robot System Architectures the interactions be¬ 
tween software components are usually more complex 
compared to more traditional business applications. In 
Robotics, the software developer faces the complexity 
of event-based, reactive, and distributed interactions be¬ 
tween sensors and motors and between several process¬ 
ing algorithms. Managing concurrent access to shared 
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resources by multiple (distributed) activities is one of 
the main issues, as thoroughly discussed in [12.85]. For 
this reason, robotic-specific component-based frame¬ 
works and toolkits have been developed, which offer 
mechanisms for real-time execution, synchronous and 
asynchronous communication, data flow and control 
flow management, and system configuration. 

The following section reviews some of the most re¬ 
cent approaches and frameworks that have been specif¬ 
ically defined for robotics. 

12.6.1 Agile Robot Development Network 
(aRDnet) 

aRDnet [12.86] is a software suite developed at the 
German Aerospace Center (DLR) Institute of Robotics 
and Mechatronics that supports the development of dis¬ 
tributed component-based systems for complex mecha- 
tronic systems with hard real-time requirements. The 
aRDnet suite has been used to implement computa¬ 
tionally demanding control loops in the kHz range 
running over all the 41 degrees of freedom (DOF) of 
the upper humanoid body Justin, such as impedance 
control and collision avoidance. The robot control sys¬ 
tem is structured as a network of functional blocks 
and communication links distributed over a network of 
computers connected by a fast digital bus like the Giga- 
bit-Ethernet. 

An aRDnet functional block is a software mod¬ 
ule with multiple input and output ports that hide the 
actual transportation protocol, i. e., user datagram pro¬ 
tocol (UDP) sockets or EtherCAT to exchange data with 
interconnected blocks. Blocks can be executed by indi¬ 
vidual processes or can be grouped in synchronization 
groups (e.g., for blocks interfacing the robot hardware) 
executed iteratively by a single process according to the 
synchronous data flow model of computation. Differ¬ 
ent synchronization groups can interact asynchronously 
through non blocking read and write operations. 

12.6.2 Yet Another Robot Platform (YARP) 

YARP [12.87] is an open-source project developed at 
the LIRA-Lab of the University of Genova in col¬ 
laboration with MIT Computer Science and Artificial 
Intelligence Laboratory (CSAIL). It has been conceived 
with the same objectives as aRDnet, i. e., the devel¬ 
opment of distributed control systems for high DOF 
robots, such as the humanoid bodies. It consists in 
a lightweight software library for concurrent and dis¬ 
tributed programming that has been used and tested on 
several operating systems. 

Similarly to aRDnet, distributed software mod¬ 
ules exchange data through input/output asynchronous 


communication ports. Differently from aRDnet, YARP 
ports can manage multiple connections for a given 
module at different data rates according to different 
protocols (e.g., transfer control protocol (TCP), UDP, 
multicast) allowing for the configuration of the quality 
of service (QOS) of the inter-module communication: 
an output port can send data to many destinations, 
while an input port can receive data from multiple 
connections. 

12.6.3 Open Robot Control Software 
(0R0C0S) 

Orocos is one of the oldest open source framework 
in robotics, under development since 2001, and with 
professional industrial applications and products us¬ 
ing it since about 2005 [12.88], The focus of Orocos 
has always been to provide a hard real-time capa¬ 
ble component framework, the so-called Real-Time 
Toolkit (RTT) implemented in C++ and as indepen¬ 
dent as possible from any communication middleware 
and operating system. Similarly to aRDnet and YARP, 
components interact with each other by exchanging 
data and events asynchronously through lock-free in¬ 
put/output ports according to the Data Flow communi¬ 
cation paradigm. 

The distinguish feature of OROCOS is the defini¬ 
tion of a component model that specifies a standard 
behaviour for concurrent activities. Components with 
real-time, deterministic and cyclic behaviour get fixed 
and cyclic time budgets for computation and within 
a computation cycle they must reach stable interme¬ 
diate states. OROCOS components are implemented 
as extension (inheritance) of the base class TaskCon- 
text, have their own thread of execution, and can be 
deployed as objects that share the same address space 
or as executables that communicate using the CORBA 
middleware. 

12.6.4 Smartsoft 

Smartsoft [12.89] is an open source framework that 
specifically addresses issues related to communica¬ 
tion among software components of a robotic con¬ 
trol system. Similarly to OROCOS, it defines a port- 
based component model, whose distinguished feature 
is a rich set of standard component interfaces (called 
communication patterns) with strictly defined interac¬ 
tion semantics: 

• Send: Defines one-way communication with 
a client/server relationship 

• Query: Two-way request communication with 
a client/server relationship 



Robotic Systems Architectures and Programming I 12.6 Implementing Robotic Systems Architectures 301 


• Push newest/push timed: 1-to-n distribution (broad¬ 
cast) with a publisher/subscriber relationship 

• Event. Asynchronous conditioned notification with 

a client/server relationship 

• Dynamic wiring: Dynamic component wiring with 

a master/slave relationship. 

Smartsoft is one of the first open source frame¬ 
works that has developed an open source toolchain 
for robotic software development based on the Eclipse 
Modeling Project. The toolchain implements a work- 
flow that guides the software developer from high level 
design of the robot system architecture down to the 
components implementation and deployment through 
automatic model-to-model transformations. 

12.6.5 Robot Operating System (ROS) 

ROS [12.90] is a message-based peer-to-peer communi¬ 
cation infrastructure supporting the easy integration of 
independently developed software components, called 
ROS nodes. A ROS system is thus a computation 
graph consisting of a set of nodes communicating with 
one another. Nodes are blocks of functional code and 
are implemented as classes that wrap robotic soft¬ 
ware libraries and provide access to the communication 
mechanisms of the underlying infrastructure (the ROS 
core). Nodes are organized into packages (file sys¬ 
tem folders containing libraries, nodes, and message 
definitions), which are grouped into thematic stacks 
(i. e., the navigation stack). Messages are typed data 
structures that can be nested into compound messages 
and are exchanged between nodes according to the 
publish/subscribe communication paradigm in an asyn¬ 
chronous manner without the need for the interacting 
nodes to know each other and to participate to the in¬ 
teraction at the same time. Messages are organized by 
topics, which correspond to information subjects that 
allow subscribers to recognize the events they are inter¬ 
ested in. When a node receives a message belonging to 
a subscribed topic, a message handler is invoked asyn¬ 
chronously, which performs some computation on the 
message payload data and possibly generates a new 
message of a given topic to publish the computation 
results. 

The distinguish characteristics of ROS is the lack 
of enforced architecture in ROS libraries, i. e., ROS 
nodes are designed as individually reusable compo¬ 
nents. An application is built by configuring individual 
nodes in such a way that they can exchange mes¬ 
sages belonging to common topics. This characteristic 


favours the decentralized development of many small 
packages by experts in the various robotics sub-do¬ 
mains. 

12.6.6 GenoM/BIP 

GenoM [12.91] is a component-oriented software pack¬ 
age developed by the LAAS CNRS robotics group that 
is used for specifying and implementing the functional 
level of robot system architectures. GenoM components 
are collections of control services, which execute a fi¬ 
nite state automaton and are implemented by a set of C 
functions called codels, which get appropriately called 
during specific state transitions (i. e., start, exec, error, 
etc.). GenoM components exchange data and events 
through posters that are regions of shared memory. 

The distinguish feature of the GenoM component 
model is the integration with the Behavior-Interaction- 
Priorities (BIPs) framework. BIP is a software frame¬ 
work and toolset for formally modeling and verifying 
complex, real-time component-based systems to guar¬ 
antee correctness-by-construction of robotic control 
systems. BIP is used to produce a formal interaction 
model, which can be used for system level coordi¬ 
nation to run (using the BIP engine) the functional 
layer composed of all the GenoM modules. BIP al¬ 
lows the hierarchical construction of compound compo¬ 
nents by using connectors, which interconnect the ports 
of GenoM components and models two basic modes 
of communication, namely, caller/provider and broad¬ 
caster/listener. 

12.6.7 Best Practice in Robotics (BRICS) 

Complementary to the projects described above 
BRICS [12.92], a joint research project funded by the 
European Commission, has formalized the robot devel¬ 
opment process according to the principles of Model 
Driven Engineering [12.93] and has provided tools, 
models, and functional libraries, which allow reducing 
the development time of robot software systems. 

In particular, the concepts of Component Frame¬ 
work and Software Product Line have been introduced 
in the robot development process [12.94,95] as illus¬ 
trated in IdajjM'imfriw This approach promotes the 
routine use of existing software or software knowl¬ 
edge to construct new software, so that similarities in 
requirements, architectures and design between appli¬ 
cations can be exploited to achieve substantial ben¬ 
efits in software quality, productivity, and business 
performance. 
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12.7 Conclusions and Further Reading 

Robot architectures are designed to facilitate the con- £ 
current execution of task-achieving behaviors. They en- i 
able systems to control actuators, interpret sensors, plan, t 
monitor execution, and deal with unexpected contin- t 
gencies and opportunities. They provide the conceptual c 
framework within which domain-dependent software 
development can take place, and they often provide pro- e 
gramming tools that facilitate that development. c 

While no single architecture has proven to be best s 
for all applications, researchers have developed a va- 1 
riety of approaches that can be applied in different I 
situations. While there is not yet a specific formula t 
for determining which architecture will be best suited r 
for a given application, this chapter provides some l 


guidelines to help developers in selecting the right 
architecture for the job. That being said, layered archi¬ 
tectures have proven to be increasingly popular, due to 
their flexibility and ability to operate at multiple levels 
of abstraction simultaneously. 

The book Kortenkamp et al. [12.96] provide sev¬ 
eral chapters on architectures that have influenced this 
chapter. Most text books in robotics [12.20, 97,98] have 
sections on robot architectures. For many years in the 
mid 1990s, the AAAI Spring Symposia on Artificial 
Intelligence had sessions devoted to robot architec¬ 
tures, although proceedings from those symposia are 
not widely available. More information on GRACE can 
be found in [12.99-101], 
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available from http://handbookofrobotics.org/view-chapter/12/videodetails/273 
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IB. Behavior-Based Systems 


Francois Michaud, Monica Nicolescu 


Nature is filled with examples of autonomous 
creatures capable of dealing with the diversity, 
unpredictability, and rapidly changing conditions 
of the real world. Such creatures must make de¬ 
cisions and take actions based on incomplete 
perception, time constraints, limited knowledge 
about the world, cognition, reasoning and phys¬ 
ical capabilities, in uncontrolled conditions and 
with very limited cues about the intent of others. 
Consequently, one way of evaluating intelligence 
is based on the creature's ability to make the most 
of what it has available to handle the complexi¬ 
ties of the real world. The main objective of this 
chapter is to explain behavior-based systems and 
their use in autonomous control problems and 
applications. The chapter is organized as follows. 
Section 13.1 overviews robot control, introduc¬ 
ing behavior-based systems in relation to other 
established approaches to robot control. Sec¬ 
tion 13.2 follows by outlining the basic principles of 
behavior-based systems that make them distinct 
from other types of robot control architectures. The 
concept of basis behaviors, the means of modu¬ 
larizing behavior-based systems, is presented in 
Sect. 13.3. Section 13.4 describes how behaviors are 
used as building blocks for creating representations 
for use by behavior-based systems, enabling the 
robot to reason about the world and about itself in 
that world. Section 13.5 presents several different 
classes of learning methods for behavior-based 
systems, validated on single-robot and multi¬ 
robot systems. Section 13.6 provides an overview 
of various robotics problems and application do¬ 
mains that have successfully been addressed or 
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are currently being studied with behavior-based 
control. Finally, Sect. 13.7 concludes the chapter. 
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13.1 Robot Control Approaches 

Situated robotics deals with embodied machines in 
complex, challenging, often dynamically changing en¬ 
vironments. Situatedness thus refers to existing in 
a complex, challenging environment, and having one’s 
behavior strongly affected by it. In contrast, robots that 
exist in static, unchanging environments are usually not 
thought to be situated. These include assembly robots 
operating in complex but highly structured, fixed, and 
strongly predictable environments, specifically engi¬ 
neered and controlled to enable the robot to accomplish 
very specific tasks. The predictability and stability of 
the environment have a direct impact on the complexity 
of the robot that must operate in it; situated robots there¬ 
fore present a significant challenge for the designer. 

Robot control, also referred to as robot decision¬ 
making or robot computational architecture, is the 
process of taking information about the environment 
through the robot’s sensors, processing it as necessary 
in order to make decisions about how to act, and execut¬ 
ing actions in the environment. The complexity of the 
environment, i. e., the level of situatedness, has a direct 
impact on the complexity of control, which is, in turn, 
directly related to the robot’s task. Control architectures 
are covered in Chap. 12 of the Handbook. 

While there are infinitely many possible ways to 
program a robot, there are fundamentally four classes 
of robot control methods, described below. 

13.1.1 Deliberative - Think, Then Act 

In deliberative control, the robot uses all of the avail¬ 
able sensory information, and all of the internally stored 
knowledge, to reason about what actions to take next. 
The control system is usually organized using a func¬ 
tional decomposition of the decision-making processes, 
consisting of a sensory processing module, a modeling 
module, a planning module, a value judgment mod¬ 
ule, and an execution module [13.1]. Such functional 
decomposition allows complex operations to be per¬ 
formed, but implies strong sequential interdependencies 
between the decision-making modules. 

Reasoning in deliberative systems is typically in 
the form of planning, requiring a search of possible 
state-action sequences and their outcomes. Planning, 
a major component of artificial intelligence, is known 
to be a computationally complex process. The process 
requires the robot to perform a sequence of sense-plan- 
act steps (e.g., combine the sensory data into a map 
of the world, then use the planner to find a path in 
the map, then send steps of the plan to the robot’s 
wheels ) [13.2-4]. The robot must construct and then po¬ 
tentially evaluate all possible plans until it finds one that 


enables it to reach its goal, solve the task, or decide on 
a trajectory to execute. Shakey, an early mobile robot 
that used STRIPS, a general planner, is an example of 
such a system applied to the problem of avoiding obsta¬ 
cles and navigating based on vision data [13.5]. 

Planning requires the existence of an internal, sym¬ 
bolic representation of the world, which allows the 
robot to look ahead into the future and predict the out¬ 
comes of possible actions in various states, so as to 
generate plans. The internal model, thus, must be kept 
accurate and up to date. When there is sufficient time 
to generate a plan and the world model is accurate, this 
approach allows the robot to act strategically, selecting 
the best course of action for a given situation. However, 
being situated in a noisy, dynamic world usually makes 
this impossible [13.6,7]. Today, no situated robots are 
purely deliberative. The advent of alternative architec¬ 
tures was driven by the need for faster yet appropriate 
action in response to the demands of complex and dy¬ 
namically changing real-world environments. 

13.1.2 Reactive - Don't Think, (Re)Act 

Reactive control is a technique for tightly coupling sen¬ 
sory inputs and effector outputs, typically involving 
no intervening reasoning [13.8] to allow the robot to 
respond very quickly to changing and unstructured en¬ 
vironments [13.9]. Reactive control is inspired by the 
biological notion of stimulus-response', it does not re¬ 
quire the acquisition or maintenance of world models, 
as it does not rely on the types of complex reasoning 
processes utilized in deliberative control. Rather, rule- 
based methods involving a minimal amount of compu¬ 
tation, and no internal representations or knowledge of 
the world are typically used. Reactive systems achieve 
rapid real-time responses by embedding the robot’s 
controller in a collection of preprogrammed, concur¬ 
rent condition-action rules with minimal internal state 
(e.g., if bumped, stop', if stopped, back up) [13.8, 10]. 
This makes reactive control especially well suited to dy¬ 
namic and unstructured worlds where having access to 
a world model is not a realistic option. Furthermore, the 
minimal amount of computation involved means that 
reactive systems are able to respond in a timely man¬ 
ner to rapidly changing environments. 

Reactive control is a powerful and effective con¬ 
trol method that abounds in nature; insects, which 
vastly outnumber vertebrates, are largely reactive. How¬ 
ever. limitations to pure reactivity include the inability 
to store (much if any) information or have memory 
or internal representations of the world [13.11], and 
therefore the inability to learn and improve over time. 
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Reactive control trades off complexity of reasoning for 
fast reaction time. Formal analysis has shown that, for 
environments and tasks that can be characterized a pri¬ 
ori, reactive controllers can be very powerful, and if 
properly structured, capable of optimal performance in 
particular classes of problems [ 13. 12,1 3]. In other types 
of environments and tasks where internal models, mem¬ 
ory, and learning are required, reactive control is not 
sufficient. 

13.1.3 Hybrid - Think and Act Concurrently 

Hybrid control aims to combine the best aspects of re¬ 
active and deliberative control: the real-time response 
of reactivity and the rationality and optimality of de¬ 
liberation. As a result, hybrid control systems con¬ 
tain two different components, the reactive/concurrent 
condition-action rules and the deliberative ones, which 
must interact in order to produce a coherent output. This 
is challenging because the reactive component deals 
with the robot’s immediate needs, such as moving while 
avoiding obstacles, and thus operates on a very fast 
time scale and uses direct external sensory data and 
signals. In contrast, the deliberative component uses 
highly abstracted, symbolic, internal representations of 
the world, and operates on them on a longer time scale, 
for example to perform global path planning or plan for 
high-level decision-making. As long as the outputs of 
the two components tire not in conflict, the system re¬ 
quires no further coordination. However, the two parts 
of the system must interact if they are to benefit from 
each other. Consequently, the reactive system must 
override the deliberative one if the world presents some 
unexpected and immediate challenge. Analogously, the 
deliberative component must inform the reactive one 
in order to guide the robot toward more efficient and 
optimal trajectories and goals. The interaction of the 
two parts of the system requires an intermediate com¬ 
ponent, which reconciles the different representations 
used by the other two and any conflicts between their 
outputs. The construction of this intermediate compo¬ 
nent is typically the greatest challenge of hybrid system 
design. 

Hybrid systems are referred to as layered, or tiered, 
robot control architecture (Chap. 12). Many are re¬ 
ferred to as three-layer architectures because of their 
structure, which consists of the reactive (execution) 
layer, intermediate (coordination) layer, and delibera¬ 
tive (organization/planning) layer. The layers are orga¬ 
nized according to the principle of increasing precision 
of control in the lower layers with decreasing intel¬ 
ligence [13.14]. A great deal of research has been 
invested into the design these components and their in¬ 
teractions [13.2, 15-21], 


Three-layer architectures aim to harness the best 
of reactive control in the form of dynamic, concurrent 
and time-responsive control, and the best of delibera¬ 
tive control, in the form of globally efficient actions 
over a long time scale. However, there are complex 
issues involved in interfacing these fundamentally dif¬ 
fering components, and the manner in which their 
functionality should be partitioned is not yet well un¬ 
derstood [13.22], 

13.1.4 Behavior-Based Control - 
Think the Way You Act 

Behavior-based control employs a set of distributed, in¬ 
teracting modules, called behaviors, that collectively 
achieve the desired system-level behavior. To an ex¬ 
ternal observer, behaviors are patterns of the robot’s 
activity emerging from interactions between the robot 
and its environment. To a programmer, behaviors are 
control modules that cluster sets of constraints in order 
to achieve and maintain a goal [13.22, 23]. Each behav¬ 
ior receives inputs from sensors and/or other behaviors 
in the system, and provides outputs to the robot’s ac¬ 
tuators or to other behaviors. Thus, a behavior-based 
controller is a structured network of interacting behav¬ 
iors, with no centralized world representation or focus 
of control. Instead, individual behaviors and networks 
of behaviors maintain any state information and mod¬ 
els. 

Well-designed behavior-based systems take advan¬ 
tage of the dynamics of interaction among the behaviors 
themselves, and between the behaviors and the environ¬ 
ment. The functionality of behavior-based systems can 
be said to emerge from those interactions and is thus 
neither a property of the robot or the environment in 
isolation, but rather a result of the interplay between 
them [13.22]. Unlike reactive control, which utilizes 
collections of reactive rules with little if any state and 
no representation, behavior-based control utilizes col¬ 
lections of behaviors, which have no such constraints: 
behaviors do have state and can be used to construct 
representations, thereby enabling reasoning, planning, 
and learning. 

13.1.5 When to Use What 

Characterizing a given robot computational architec¬ 
ture based on these four classes of control is often 
a matter of degree, as architectures attempt to com¬ 
bine the advantages of these paradigms, especially 
the responsiveness, robustness, and flexibility of the 
behavior-based approach with the use of abstract rep¬ 
resentational knowledge for reasoning and planning 
about the world [13.22] or for managing multiple 
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conflicting goals. For example, AuRA (autonomous 
robot architecture) uses a planner to select behav¬ 
iors [13.22] and 3T uses behaviors in the execution 
layer of a three-level hierarchical architecture [13.24]; 
both of these architectures dynamically reconfigure be¬ 
haviors according to reasoning based on available world 
knowledge [13.22], Goller et al. [13.25] use a set of 15 
navigational behaviors with 17 activating behaviors, 
combined with a task planner and a topological navi¬ 
gation module. 

Each of the above approaches to robot control has 
its strengths and weaknesses, and all play important 
and successful roles in certain robot control problems 
and applications. Each offers interesting but different 
insights, and no single approach should be seen as 
ideal or otherwise in the absolute. Rather, the choice 
of robot control methodology should be based on the 
particular task, environment, and robot. Robot control 
presents fundamental tradeoffs having to do with time 
scale of response, system organization, and modular¬ 
ity: thinking allows looking ahead to avoid mistakes, 
but only as long as sufficient, accurate, up-to-date in¬ 
formation is available, otherwise reacting may be the 
best way to handle the world. As a consequence of 
these inherent tradeoffs, it is important to have different 
methodologies at our disposal rather than having to fit 
all controller needs into a single methodology. Select¬ 
ing an appropriate control methodology and designing 
an architecture within it is best determined by the sit¬ 


uatedness properties of the problem, the nature of the 
task, the level of efficiency or optimality needed, and 
the capabilities of the robot, both in terms of hardware, 
world modeling, and computation. 

For example, reactive control is the best choice 
for environments demanding immediate response, but 
such speed of reaction comes at the price of being 
myopic, not looking into the past or the future. Re¬ 
active systems are also a popular choice in highly 
stochastic environments, and environments that can be 
properly characterized so as to be encoded in a reac¬ 
tive input-output mapping. Deliberative systems, on the 
other hand, are the only choice for domains that re¬ 
quire a great deal of strategy and optimization, and 
in turn search and planning. Such domains, how¬ 
ever, are not typical of situated robotics, but more so 
of scheduling, game playing, and system configura¬ 
tion, among others. Hybrid systems are well suited 
for environments and tasks where internal models and 
planning are needed, and the real-time demands are 
few, or sufficiently independent of the higher-level 
reasoning. 

Behavior-based systems, in contrast, are best suited 
for environments with significant dynamic changes, 
where fast response and adaptivity are crucial, but the 
ability to do some looking ahead and avoid past mis¬ 
takes is required. Those capabilities are spread over 
the active behaviors, using representations if neces¬ 
sary [13.23], as discussed in the following. 


13.2 Basic Principles of Behavior-Based Systems 


Behavior-based robotics was developed for situ¬ 
ated robots, allowing them to adapt to the dy¬ 
namics of real-world environments without operat¬ 
ing upon abstract representations of reality [13.11], 
but also giving them more computational capabil¬ 
ity and expressivity than are available to reactive 
robots. Behavior-based systems maintain a tight cou¬ 
pling of sensing and action through behaviors, and 
use the behavior structure for representation and learn¬ 
ing. Therefore, it is uncommon for a behavior to 
perform extensive computation or reasoning relying 
on a traditional world model, unless such computa¬ 
tion can be done in a timely manner in response 
to dynamic and fast-changing environment and task 
demands. 

Figure 13.1 summarizes the general components of 
low-level behavior-based systems. Note that there is 
a distinction between activation conditions, which al¬ 
low the behavior to generate actions, and stimuli, from 
which actions are generated. 


The basic principles of behavior-based control can 

be summarized briefly as follows: 

• Behaviors are implemented as control laws (some¬ 
times similar to those used in control theory), either 
in software or hardware, as a processing element or 
as a procedure. 

• Each behavior can take inputs from the robot’s 
sensors (e.g., proximity sensors, range detectors, 
contact sensors, camera) and/or from other modules 
in the system, and can send outputs to the robot’s ef¬ 
fectors (e.g., wheels, grippers, arm, speech) and/or 
to other modules. 

• Many different behaviors may independently re¬ 
ceive input from the same sensors and output action 
commands to the same actuators. 

• Behaviors are encoded to be relatively simple, and 
are added to the system incrementally. 

• Behaviors (or subsets thereof) are executed con¬ 
currently, not sequentially, in order to exploit par- 



Behavior-Based Systems I 13.2 Basic Principles of Behavior-Based Systems 311 



Fig. 13.1 A general schematic of one type of behavior-based systems 


allelism and speed of computation, as well as the 

interaction dynamics among behaviors and between 

behaviors and the environment. 

Behaviors are designed at a variety of abstraction 
levels, facilitating bottom-up construction of behavior- 
based systems. New behaviors are introduced into the 
system incrementally, from the simple to the more 
complex, until their interaction results in the desired 
overall capabilities of the robot. In general, behaviors 
encode time-extended processes, not atomic actions 
that are typical of feedback control (e.g., go-forward- 
by-a-small-increment or tum-by-a-small-angle). The 
interaction and integration of temporal and spatial ef¬ 
fects are of key importance in behavior-based systems. 
Merely having one process controlling an actuator for 
predetermined intervals of time, or using as many pro¬ 
cesses as there are effectors to control them, does not 
suffice as the basis for behavior-based control. It is 
the combined effect of concurrent processes over time 
and driven by perception and internal states that cre¬ 
ates the relevant behavior-based dynamics in a control 
system. 

As a first step, survival behaviors, such as collision- 
avoidance, are implemented. These behaviors are often 
reactive in nature, since reactive rules can and of¬ 
ten do form components of simple behaviors. Next, 
behaviors are added that provide more complex capa¬ 
bilities, such as wall-following, target-chasing, homing, 
find-object, get-recharged, avoid-the-light, aggregate- 
with-group, pick-up-object, find-landmark. Depend¬ 
ing on the system being designed, behaviors imple¬ 
menting distributed representations may be added, 
as may be behaviors capable of learning about the 
world and/or the robot itself, and operating on those 


representations and learned information. Representa¬ 
tion and learning are addressed in more detail in 
Sect. 13.4. 

Behavior-based systems are typically designed so 
the effects of the behaviors interact largely in the en¬ 
vironment rather than internally through the system, 
taking advantage of the richness of the interaction dy¬ 
namics by exploiting the properties of situatedness. 
These dynamics are sometimes called emergent behav¬ 
iors because they emerge from the interactions and are 
not internally specified by the robot’s program. There¬ 
fore, the internal behavior structure of a behavior-based 
system need not necessarily mirror its externally man¬ 
ifested behavior. For example, a robot that flocks with 
other robots may not have a specific flocking behavior; 
instead, its interaction with the environment and other 
robots may result in flocking, although its only behav¬ 
iors may be avoid-collisions, stay-close-to-the-group, 
and keep-going. 

For such an approach to work, a behavior-based 
system must resolve the issue of choosing a particu¬ 
lar action or behavior from multiple options, a process 
known as action selection [13.26] (or also behavior 
coordination [13.27], behavior selection [13.28], or be¬ 
havior fusion [13.29]) problem. This is one of the 
central design challenges of behavior-based systems. 
One approach to action selection is the use of a pre¬ 
defined behavior hierarchy, in which commands from 
the highest-ranking active behavior are sent to the actu¬ 
ator and all others are ignored. Numerous approaches 
based on other principles as well as ad hoc methods 
for addressing the action selection problem have been 
developed and demonstrated on robotic systems. These 
methods aim to provide increased flexibility but, in 
some cases, may do so at the cost of reducing the ef- 
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ficiency or the analyzability of the resulting control 
systems. Developed methods include varieties of motor 
schemas [13.16], command fusion [13.30], spreading 
of activation through a behavior network [13.31,32], 
behavior auctions [13.33,34], and fuzzy logic [13.35, 
36], among many others. Pirjanian [13.27] presents 
a survey of action selection mechanisms. Some frame¬ 
works also support the use of several action selection 
mechanisms, such as APOC [13.28] (allowing dynamic 
selection and changes) and iB2C [13.29] (along with 
development guidelines, analysis tools and visualiza¬ 
tion techniques). 

13.2.1 Misconceptions 

Because behavior-based systems are not always simple 
to describe or implement, they are also often misun¬ 
derstood. The most common misconception equates 
reactive and behavior-based systems. Historically, the 
advent of behavior-based systems was inspired by re¬ 
active systems; both maintain real-time couplings be¬ 
tween sensing and action [13.18,37], and are struc¬ 
tured and developed bottom-up, consisting of dis¬ 
tributed modules. However, behavior-based systems 
are fundamentally more powerful than reactive sys¬ 
tems, because they can store representations [13.38], 
while reactive systems cannot. Reactive systems are 
limited by their lack of internal state; they are in¬ 
capable of using internal representations and learn¬ 
ing. Behavior-based systems overcome this limitation 
because their underlying unit of representation, the 
behavior, can store state internally, in a distributed 
fashion. 

The means by which state and representation are 
distributed in behavior-based systems is one of the 
sources of the flexibility of the control methodol¬ 
ogy. Representations in behavior-based systems are 
distributed, so as to best match and utilize the un¬ 
derlying behavior structure that causes the robot to 
act. This is how thinking can be organized in much 
the same way as acting. Thus if a robot needs to 
plan ahead, it does so in a network of communicat¬ 
ing behaviors, rather than a single centralized plan¬ 
ner. If a robot needs to store a large map, the map 
might be distributed over multiple behavior modules 
representing its components, such as a network of 
landmarks, as in [13.39], or a network of param¬ 
eterized navigation behaviors, as in [13.40,41], so 
that reasoning about the map/environment/task can 
be done in an active fashion, through using message 
passing within the behavior network. The planning 
and reasoning components of the behavior-based sys¬ 
tem use the same mechanisms as the sensing- and 
action-oriented behaviors, and as a result do not op¬ 


erate on a fundamentally different time scale and 
representation relative to one another. Various forms 
of distributed representations are used, ranging from 
static table structures and networks, to active pro¬ 
cedural processes implemented within the behavior 
networks. 

Another area of common misconception relates to 
the comparison between behavior-based systems and 
hybrid systems. Because the two use such different 
modularization strategies, it is often assumed that one 
approach (usually hybrid) has improved expressive ca¬ 
pabilities. In fact, behavior-based and hybrid systems 
have the same expressive and computational capabili¬ 
ties: both can exploit representations and look ahead, 
but they do so in very different ways. This has re¬ 
sulted in different application domains being best suited 
to behavior-based versus hybrid systems. Specifically, 
hybrid systems dominate the domain of single-robot 
control, unless the task is so time-demanding that a re¬ 
active system must be used. Behavior-based systems 
dominate the domain of multi-robot control because 
the notion of collections of behaviors within the sys¬ 
tem scales well to collections of such robots, resulting 
in robust, adaptive group behavior [13.42,43]. See 
Chap. 53 on multiple mobile robot systems for more 
details. 

Like hybrid systems, behavior-based systems may 
be organized in layers, but unlike hybrid systems, 
the layers do not differ from each other drastically 
in terms of organization, time scale or representation 
used. Behavior-based systems typically do not employ 
the hierarchical/sequential division favored by hybrid 
approaches. Behavior-based systems do provide both 
low-level control and high-level deliberation; the latter 
can be performed by one or more distributed repre¬ 
sentations that compute(s) over the other behaviors or 
modules, often directly utilizing low-level behaviors 
and their outputs. The resulting systems, built from the 
bottom-up, do not divide into differently represented 
and independent components, and consist of elements 
directly tied in some ways to behaviors. The power, el¬ 
egance, and complexity of behavior-based systems all 
stem from the ways in which their constituent behav¬ 
iors are designed, coordinated, and used. For instance, 
monitoring how behaviors for low-level control con¬ 
tribute to the commands issued over time can be used 
to evaluate how appropriate the robot operates in its 
environment based on its set of activated behaviors. Dif¬ 
ferent methods can be used to implement such a mecha¬ 
nism, such as temporal analyses [13.36, 44— 47] , pattern 
recognition techniques [13.48] and graph-based repre¬ 
sentations [13.49-51]. 

Therefore, the principles to keep in mind when ex¬ 
tending behavior-based to higher decisional levels are: 
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1. Use behaviors as the building block of both 
decision-making and action execution processes; 

2. Use distributed parallel evaluation and concurrent 
control over lower-level behaviors, which take real¬ 
time inputs from sensory data and send real-time 
commands to effectors; 


13.3 Basis Behaviors 

The process of designing a set of behaviors for a robot 
is referred to as behavior synthesis, and is typically per¬ 
formed by hand, although some methods for automated 
synthesis behaviors have been developed and success¬ 
fully demonstrated [13.52,53], In all cases, behaviors 
perform a particular activity, attain a goal, or maintain 
some state. The notion of defining an optimal behavior 
set for a given robot or task has been considered, but 
it is generally accepted that such a notion is not realis¬ 
tic as it is dependent on too many specifics of a given 
system and environment that cannot currently be effec¬ 
tively formalized. 

Mataric et al. [13.43,54] describe basis behav¬ 
iors, also referred to as primitive behaviors, as a tool 
for structuring and thus simplifying behavior syn¬ 
thesis. Basis behaviors are a set of behaviors such 
that each is necessary, in the sense that each either 
achieves, or helps to achieve, a relevant goal that 
cannot be achieved without it by other members of 
that set. Furthermore, the basis behavior set is suffi¬ 
cient for achieving the goals mandated for the con¬ 
troller. The term basis was chosen to be indicative 
of the similar notion within linear algebra. The prop¬ 


3. Have no centralized components, each module car¬ 
rying out its own responsibilities. 

The following sections describe and illustrate in 
more detail how behavior-based principles can be used 
to control robots. 


erty of necessity or parsimony is analogous to the 
idea of linear independence; the idea of sufficiency is 
similar to the linear algebraic concept of span. Ba¬ 
sis behaviors should be simple, stable, robust, and 
scalable. 

Another organizational principle of basis behaviors 
is orthogonality. Two behaviors are orthogonal if they 
do not interfere with one another, each inducing no 
side-effects in the other. This is often achieved by hav¬ 
ing behaviors take mutually exclusive sensory inputs. 
Another method is to have different behaviors control 
separate effectors. This form of factorization is only 
feasible when the robot’s dynamics do not inhibit their 
separability. In contrast, autonomous helicopter control 
is an example of a highly coupled system; Saripalli 
et al. [13.55] demonstrated how behavior-based control 
can be effectively applied to robust autonomous heli¬ 
copter flight. 

Basis behavior design principles have been applied 
to single-robot as well as multi-robot behavior-based 
systems in a variety of applications, ranging from nav¬ 
igation to foraging, coordinated group movement, box 
pushing, and others. 


13.4 Representation in Behavior-Based Systems 


Embedding representation into behavior-based systems 
involves the challenge of conserving the basic principles 
of the approach at all levels of system decision-making. 
Combining abstract reasoning processes with behaviors 
must be done in a way that exploits interaction dynam¬ 
ics and desirable emergent system properties. 

Mataric et al. [13.38, 56] describe work with a robot 
named Toto and as shown by l^aJEIiTEEM, which 
introduced the use of distributed representation into 
behavior-based systems. Toto’s capabilities included 
safe navigation, landmark detection, map learning, and 
path planning in the learned map representation, all 
within the behavior-based framework. To exploit the 
principles underlying behavior-based control, Toto’s 
representation was not a centralized map. Instead, any 


newly discovered landmark in the environment was as¬ 
signed to a new map representation behavior, which 
stored the landmark descriptor (type, estimated Carte¬ 
sian location, and compass orientation). Whenever sen¬ 
sory inputs matched the landmark descriptor, the robot 
localized to that particular landmark and the behavior 
became active. The following is pseudo-code for each 
landmark behavior: 


Algorithm 13.1 

my-behavior-type: C 
my-compass-direction: 0 
my-approximate-location: (x,v) 
my-approximate-length: 6.5 
whenever received (input) 
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if input(behavior-type) = my-behavior-type 

and 

input(compass-direction) = 
my-compass-direction then 
active j- true 

end if 


As new landmarks are discovered, they are added 
to the map representation behavior network. In this 
way, the topology of the resulting map network is iso¬ 
morphic to the topology of the network graph in the 
physical environment Toto has explored. The edges 
in the network graph are also communication links 
in the behavior network, allowing landmark behaviors 
to communicate through local message passing. Con¬ 
sequently, a currently active map behavior can send 
a message to its topological neighbor(s), thereby in¬ 
dicating expectation of it being the next recognized 
landmark and facilitating Toto’s localization. Planning 
in the network takes place through the use of the 
same message-passing mechanism. The goal landmark 
(which could be selected by the user as part of the 
task, such as go to this particular corridor or go to 
the nearest north-facing wall), sends messages (i. e., 
spreads activation) from itself to its neighbors, which 
pass it on throughout the network. As messages are 
passed, the length of each landmark in the graph is 
accrued, thereby estimating the length of each path. 
The shortest path arriving at the currently active net¬ 
work behavior indicates the best direction to pursue 
toward the goal. This is equivalent to a distributed Di- 


jkstra search. Importantly, because this search is an 
active ongoing process within a behavior map rather 
than a static process (as it would be in a centralized map 
representation), if the robot is picked up and moved 
to another location, as soon as it localizes, it contin¬ 
ues on the optimal path to the goal: each landmark 
makes a local decision as to where to go next toward 
the goal, and no unique global path is stored in any cen¬ 
tral location/representation. Thus, the path is constantly 
refreshed and updated; if any route is blocked, the link 
in the graph is disconnected and the shortest path is up¬ 
dated dynamically. 

Toto exemplifies how, in a behavior-based system, 
a representation can be stored in a distributed fashion, 
so as to best match the underlying behavior structure 
that produces the robot’s external goal-driven activity. 
If Toto needs to make high-level decisions (such as 
planning ahead to a distant goal), it does so in a net¬ 
work of communicating behaviors, rather than a single 
centralized component. This results in scalable and ef¬ 
ficient computation for the system as a whole, since 
the typically slower decision-making processes such 
as planning are distributed and modularized in a way 
that makes them more consistent with the time scale 
and representation of the rest of the system. Note the 
fundamental difference between this general attempt 
of behavior-based systems to homogenize their rep¬ 
resentation through the use of behaviors as universal 
modules, compared to hybrid systems which rely on 
inherently different representations and time scales at 
different levels of the system. 


13.5 Learning in Behavior-Based Systems 


The ability to improve performance over time and to 
reason about the world, in the context of a chang¬ 
ing and dynamic environment, are important areas of 
research in situated robotics. Unlike in classical ma¬ 
chine learning where the goal is typically to optimize 
performance over a long period of time, in situated 
learning the aim is to adapt relatively quickly toward 
attaining efficiency in the light of uncertainty. Models 
from biology are often considered, given its proper¬ 
ties of learning directly from environmental feedback. 
Variations and adaptations of machine learning, and in 
particular reinforcement learning, have been effectively 
applied to behavior-based robots, which have demon¬ 
strated learning to walk [13.57], communicate [13.58], 
navigate and create topological maps [13.38,59], di¬ 
vide tasks [13.23,60], behave socially [13.61], and 
even identify opponents and score goals in robot soc¬ 
cer [13.62]. Methods from artificial life, evolutionary 


computation/genetic algorithms, fuzzy logic, vision and 
learning, multi-agent systems, and many other research 
areas continue to be actively explored and applied to 
behavior-based robots as their role in animal modeling 
and practical applications continue to develop. 

When operating in unpredictable and partially ob¬ 
servable environments, an autonomous robot must ex¬ 
amine the evolution of its general states, and try to 
capture what emerges from the interaction dynamics 
with its environment. Temporal integration of differ¬ 
ent types of observations is an important aspect of 
that capability [13.63,64]. Work on motivational sys¬ 
tems [13.47,65-68] has shown that a balance between 
planning and reactivity for goal management can be 
achieved using internal variables activated or inhibited 
by different factors [13.42, 46,69, 70]. Motivations can 
be cyclic (e.g., circadian rhythms) or change in vari¬ 
ous temporally dependent ways [13.68]. In general, the 
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notion of motivations is used to efficiently balance the 
need to adapt to the contingencies of the world and to 
accomplish the robot’s goals. 

In the following subsections, we discuss four suc¬ 
cessfully validated classes of learning approaches in 
behavior-based systems. The approaches differ in what 
is learned and where learning algorithms are applied, 
but in all cases behaviors are used as the underlying 
building blocks for the learning process. 

13.5.1 Reinforcement Learning 

in Behavior-Based Systems 

Behaviors are recognized as excellent substrates for 
speeding up reinforcement learning (RL), which is 
known to suffer from the curse of dimensionality. 
The earliest examples of RL in behavior-based sys¬ 
tems demonstrated hexapod walking [13.57] and box¬ 
pushing [13.71]. Both decomposed the control system 
into a small set of behaviors, and used generalized in¬ 
put states, thereby effectively reducing the size of the 
state space. In the box-pushing example, the learning 
problem was broken up into modularized policies for 
learning separate mutually exclusive behaviors: for get¬ 
ting out when stuck, for finding the box when lost and 
not stuck, and for pushing the box when in contact with 
one and not stuck. The modularization into behaviors 
resulted in greatly accelerated as well as more robust 
learning. More recently, Kober et al. [13.72] demon¬ 
strated the use of reinforcement learning to learn a set of 
meta-parameters that enable the adaptation of learned 
motor plans, such as a table tennis pass or a dart throw¬ 
ing task, to new situations. 

Scaling up reinforcement learning to multi-robot 
behavior-based systems has also been explored [13.23, 
73]. In multi-robot systems, the environment presents 
further challenges of nonstationarity and credit assign¬ 
ment, due to the presence of other agents and concurrent 
learners. The problem was studied in the context of 
a foraging task with four robots, each initially equipped 
with a small set of basis behaviors ( searching, homing, 
picking-up, dropping, following, avoiding ) and learn¬ 
ing individual behavior selection policies, i. e., which 
behavior to execute under which conditions. Due to 
interference among concurrent learners, this problem 
could not be solved directly by standard RL. Shaping, 
a concept from psychology [13.74], was introduced; it 
was subsequently adopted in robot RL [13.75]. Shaping 
pushes the reward closer to the subgoals of the behav¬ 
ior, and thus encourages the learner to incrementally 
improve its behaviors by searching the behavior space 
more effectively. Mataric [13.73] introduced shaping 
through progress estimators, measures of progress to¬ 
ward the goal of a given behavior during its execution. 


This form of reward shaping addresses two issues as¬ 
sociated with delayed reward: behavior termination and 
fortuitous reward. Behavior termination is event-driven; 
the duration of any given behavior is determined by 
the interaction dynamics with the environment, and can 
vary greatly. Progress estimators provide a principled 
means for deciding when a behavior may be terminated 
even if its goal is not reached and an externally gen¬ 
erated event has not occurred. Fortuitous reward refers 
to reward ascribed to a particular situation-behavior (or 
state-action) pair which is actually a result of previous 
behaviors/actions. It manifests as follows: previous be¬ 
haviors lead the system near the goal, but some event 
induced a behavior switch, and subsequent achievement 
of the goal is ascribed most strongly to the final be¬ 
havior, rather than the previous ones. Shaped reward 
in the form of progress estimators effectively elimi¬ 
nates this effect. Because it provides feedback during 
behavior execution, it rewards the previous benefi¬ 
cial behaviors more strongly than the final one, thus 
more appropriately assigning credit. This case there¬ 
fore illustrates how reinforcement learning has been 
successfully applied to behavior-based robotics, in par¬ 
ticular at the level of behavior selection. The learning 
process is accelerated by the behavior structure, which 
provides a higher-level representation of actions and 
time-extended dynamics. 

13.5.2 Learning Behavior Networks 

The modularization of behavior-based systems as net¬ 
works of behaviors allows for learning to be ap¬ 
plied at the network level as well. Nicolescu and 
Mataric [13.40,41] developed the notion of abstract 
behaviors, which separate the activation conditions of 
a behavior from its output actions (so-called primitive 
behaviors, which share the same principles as basis be¬ 
haviors described Sect. 13.3); this allows for a more 
general set of activation conditions to be associated 
with the primitive behaviors. While this is not neces¬ 
sary for any single task, it is what provides generality 
to the representation. An abstract behavior is a pairing 
of a given behavior’s activation conditions (i. e., precon¬ 
ditions) and its effects (i. e., postconditions); the result 
is an abstract and general operator much like those used 
in classical deliberative systems (Fig. 13.2). Primitive 
behaviors, which typically consist of a small basis set, 
may involve one or an entire collection of sequential or 
concurrently executing behaviors. 

Networks of such behaviors are then used to spec¬ 
ify strategies or general plans in a way that merges 
the advantages of both abstract representations and 
behavior-based systems. The nodes in the networks are 
abstract behaviors, and the links between them repre- 
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sent precondition and postcondition dependencies. The 
task plan or strategy is represented as a network of such 
behaviors. As in any behavior-based system, when the 
conditions of a behavior are met, the behavior is acti¬ 
vated. Analogously, when the conditions of an abstract 
behavior are met, the behavior activates one or more 
primitive behaviors which achieve the effects speci¬ 
fied in its postconditions. The network topology at the 
abstract behavior level encodes any task-specific behav¬ 
ior sequences, freeing up the primitive behaviors to be 
reused for a variety of tasks. Thus, since abstract behav¬ 
ior networks are computationally lightweight, solutions 
for multiple tasks can be encoded within a single system 
and dynamically switched. 

Nicolescu and Mataric [13.40,41] introduced 
a means for automatically generating such networks of¬ 
fline as well as at runtime. The latter enables a learning 
robot to acquire task descriptions dynamically, while 
observing its environment, which can include other 
robots and/or a teacher. The methodology was validated 
on a mobile robot following a human and acquiring 
a representation of the human-demonstrated task by 
observing the activation of its own abstract behavior 
pre- and postconditions, thus resulting in a new ab¬ 
stract behavior network representing the demonstrated 
task [13.76]. The robot was able to acquire novel be¬ 
havior sequences and combinations (i. e., concurrently 
executing behaviors), resulting in successful learning of 
tasks involving visiting various targets in particular or¬ 
der as shown in and pick¬ 

ing up, transporting, and delivering objects as shown 
in |43>M3EBH3i and l<sf'il'JVtM, dealing with barri¬ 
ers, and maneuvering obstacle courses in specific ways 
as shown in and 

This approach has been extended to learning of 
more general task representations, in which every goal 
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is achieved through a linear superposition (or fusion) 
of robot primitives, and sequencing across goals is 
achieved through arbitration [13.77]. In this work, con¬ 
trollers are built from two components: behavior prim¬ 
itives (BPs) and fusion primitives (FPs), which can be 
sequenced to form behavior networks. The behavior 
primitives perform a set of actions under given environ¬ 
mental conditions and are represented using a schema- 
based approach [13.78]. A fusion primitive encapsulates 
a set of multiple concurrently running primitive be¬ 
haviors through linear combination of the motor com¬ 
mands, expressed as a vector in the robot’s coordinate 
system. The FPs combine these vectors by weighting 
them and fusing them through vector addition. The 
method allows for learning of both the goals involved in 
the task (represented as the sequence of goal-achieving 
fusion primitives) and also of the particular ways in 
which the same goals can be achieved (captured in 
the behavior weights). Using this learning approach, 
a robot is capable of learning multiple navigation styles 
(such as staying on the center/left/right side of a corri¬ 
dor, preferences for turning left or right at T-junctions 
and particular styles of navigating around obstacles). 

13.5.3 Learning from Demonstration 
in Behavior-Based Systems 

Learning by demonstration has also been combined 
with behavior-based systems to produce one-shot learn¬ 
ing and teaching mechanisms for robot control [13.40, 
79]. Furthermore, this type of behavior-based archi¬ 
tecture has been used to learn ship navigation strate¬ 
gies [13.80]; during a learning phase, an instructor 
selects behaviors for a ship to execute in order to reach 
a specific goal, and a subsequent offline stage then gen¬ 
erates dependency links between the behaviors that it 
witnessed during the learning phase. 

Another form of learning by demonstration has used 
probabilistic methods to select and combine multiple 
behaviors [13.81]. There, the problem of learning what 
behaviors to execute during autonomous navigation is 
treated as a state estimation problem. During a learning 
phase, the robot observes commands used by a teacher. 
A particle filter then fuses the control commands that 
were demonstrated by the teacher to estimate behavior 
activation. The method produces a robust controller that 
is well suited for dynamic environments. 

Learning by demonstration has also been used in 
a case-base reasoning framework [13.82], to teach a 4- 
legged Aibo robot different low-level navigation behav¬ 
iors, such as following a ball and reaching a target. The 
learned behaviors encapsulate the kinematics and dy¬ 
namics of the robot’s control through the instruction 
process, which would be more difficult to achieve if de- 
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signed by hand, and can successfully be combined into 
a newer, higher-level behavior, thus enabling hierarchi¬ 
cal representations of robot tasks. 

A method based on decision trees and Support Vec¬ 
tor Machines has been used in [13.83] to find mappings 
between sensory data and patterns produced by behav¬ 
iors in a learned feature space. The mapping allows 
a robot to identify behaviors shown by a human demon¬ 
strator and to further learn a sequential representation of 
the demonstrated task. 

13.5.4 Learning from History 
of Behavior Use 

Most deliberative approaches derive knowledge for rea¬ 
soning about the world from sensor inputs and actions 
taken by the robot. This results in complex state-space 
representations of the world and does not take into con¬ 
sideration the context in which these sensations/actions 
are taken. As already discussed, behaviors, which are 
readily used as low-level control blocks driven by what 
is experienced from the interactions with the environ¬ 
ment, can also serve as an abstract representation to 
model those interactions. One approach is to use history 
information [13.84], i. e., to explicitly take into consid¬ 
eration the time sequence of observations in order to 
make a decision. Applying this idea to behavior-based 
systems, by knowing the purpose of each of the behav¬ 
iors and by observing their history of use, the robot 
can reason and ground its intentions in what it is ex¬ 
periencing in its operating environment. The concept of 
abstract behavior is exploited here as well, to activate 
behaviors and as a representation on which to learn. 

Learning from history has been validated in 
behavior-based systems capable of making the robot 
change its behavior selection strategy for foraging col¬ 
ored objects (blocks) to a homing region in nonsta¬ 
tionary, dynamically changing environments involving 
multiple concurrently learning robots [13.49-51]. In 
that foraging task, the robot is given two subtasks: 
search for a block (searching task), and bring it to the 
home region (homing task). The robot is given behav¬ 
iors to accomplish these tasks: one behavior for the 
searching task, called searching-block, and two for the 
homing task, homing and drop-block. A velocity con¬ 
trol behavior is also used in both of these tasks to make 
the robot move. All these behaviors are referred to as 
task behaviors. Conditions for activating task behaviors 
are preprogrammed based on the presence or absence 
of a block in front of the robot and the proximity of the 
home region. 

The robot also needs to navigate safely in the en¬ 
vironment. In this approach, an avoidance behavior is 
activated unless the robot is near home and carrying 


a block; otherwise it is disabled to allow the robot to 
approach the home region. This type of behavior, used 
for handling harmful situations and interference while 
accomplishing a task, is referred to as a maintenance 
behavior. The designer determines the conditions in 
which maintenance behaviors should be used, but can¬ 
not indicate when they will occur during the task, as 
that is tied to the dynamics of the interaction between 
the robot and its environment. 

The robot learns to use alternative behaviors 
( follow-side , rest, and turn-randomly), which introduce 
variety into its action repertoire by changing the way 
it accomplishes its tasks. In contrast to other types of 
behaviors, these have no a priori activation conditions; 
the objective is to allow the robot to learn when to acti¬ 
vate these behaviors according to past experiences, for 
some preset periods of time, when it is experiencing 
interference in accomplishing its task. Figure 13.3 il¬ 
lustrates how the behaviors are prioritized using a fixed 
suppression mechanism, similar to the subsumption 
architecture [13.9] but with the difference that the ac¬ 
tivated behaviors, i. e., those allowed to issue outputs, 
change dynamically. 

Following behavior-based principles, an activated 
behavior may or may not be used to control the robot, 



Fig. 13.3 Organization of the behavior level and the interaction 
model level. Behavior on colored background represent an example 
of activated behaviors for the searching task, with Turn-randomly as 
a chosen alternative behavior. For clarity, sensory inputs to behav¬ 
iors are not shown 
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depending on the sensory conditions it monitors and 
the arbitration mechanism. An activated behavior is 
used only when it provides commands that actually 
control the robot. Whenever a behavior is used, its cor¬ 
responding symbol is sent to the interaction model, 
generating the sequence of behaviors used over time. 
Separate learning trees are used for each task; determin¬ 
ing which one to use is done based on the activated task 
behavior. 

The algorithm uses a tree structure to store the his¬ 
tory of behavior use. The upper part of Fig. 13.3 shows 
a typical tree with nodes storing the behavior (H for 
use by homing and drop-block) used for controlling the 
robot while accomplishing its task, and n, the number 
of times a transition between the node itself and its suc¬ 
cessor has been made (as observed from behavior use). 
Initially, the tree for a particular task is empty and is 
incrementally constructed as the robot goes about its 
task. Leaf nodes are labeled with E (for end-node) and 
store the total performance of the particular tree path. 
Whenever a path is completely reused, when the same 
sequence of behaviors is used, the E node is updated 
with the average of the stored and current performances 
for recent trials. 

Learning is done through reinforcement. Depending 
on the domains and the tasks, a variety of factors can 
be used to evaluate performance, and different metrics 
can be used with this learning algorithm. To see how 
far the idea of self-evaluation and learning by observ¬ 
ing behavior use can be taken, the evaluation function 
used here is not based on characteristics about the envi¬ 
ronment or the task. Instead, it is based on the amount 
of time behaviors are used. Comparison between the 
time spent using behaviors associated with the tasks 
and the time spent exploiting maintenance behaviors is 
used to derive the evaluation criterion. Consequently, 


behavioral selection strategy is derived from what can 
be learned from the experiences of the robot in its en¬ 
vironment, without having to characterize a priori what 
can be considered to be optimal operating conditions in 
the environment. 

Using the tree and the evaluation function, the algo¬ 
rithm has two options for using a maintenance behavior: 

1. Not to make any changes in its active behavior set 

(the observe option); 

2. To activate an alternative behavior. 

The sequence of nodes in a tree path characterizes the 
interactions experienced by the robot in the world. Dif¬ 
ferent selection criteria can be used by comparing the 
performance at the current position in the tree with 
the expected performance, following an exploration (to 
learn the effects of alternative behaviors) then exploita¬ 
tion (to exploit what was learned in previous trials) 
strategy. The expected performance of a given node 
is the sum of the stored end-node performances in its 
subpaths, multiplied by the frequency of use of the sub¬ 
paths relative to the current position in the tree. Finally, 
since this algorithm is used in noisy and nonstationary 
conditions, deleting paths is necessary to keep the inter¬ 
action model up to date. This is done by keeping only 
a fixed number of the most recent paths in the tree. 

Results obtained with this approach show that the 
robot is able to learn unanticipated strategies (like rest¬ 
ing in front of a static obstacle to increase the turning 
angle when it starts to move again, making it locate 
a target) and original ones (like yielding when close to 
other robots or following walls when the center of the 
pen is crowded). Developing such capabilities is impor¬ 
tant in general, because it makes it possible for robots 
to learn in nonstationary environments, which are com¬ 
mon in natural settings. 


13.6 Applications and Continuing Work 


Behavior-based robots have demonstrated various stan¬ 
dard capabilities, including obstacle avoidance, nav¬ 
igation, terrain mapping, following, chasing/pursuit, 
object manipulation, task division and cooperation, and 
learning maps [13.85], navigation and walking. They 
have also demonstrated some novel applications like 
large-scale group behaviors including flocking, for¬ 
aging, and soccer playing, human-robot interaction, 
and modeling insect [13.86] and even human behav¬ 
ior [13.87-89]. Behavior-based methods span mobile 
robots, shopping carts [13.25], underwater vehicles, 
planetary rovers for space exploration, assistive [13.90], 
interactive and social robots [13.91,92], as well as 


snake-like modular robots [13.93], robots capable of 
grasping, manipulation, walking, running, and many 
others. Consumer market products, such as the iRobot 
Roomba Robotic Floorvac [13.94], also use behavior- 
based control, demonstrating its broad applicability. 

The use of behavior-based architectures for robot 
control has evolved from single-system implementa¬ 
tions to approaches that combine forms of learning, 
state estimation, and distributed computation. Behav¬ 
iors have been combined with fuzzy inference systems 
for indoor navigation using mobile robots [13.35,36, 
95-97], where a command fusion module acts as an 
arbiter that combines multiple fuzzy behavior outputs 
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into a single control signal. This strategy ensures the 
robot is capable of making inferences in the face of 
uncertainty. 

Behavior-based methods have been employed in 
multi-robot systems from the outset [13.43], as il¬ 
lustrated by and are still used [13.98, 

99]. Multi-robot researchers also consider tasks re¬ 
quiring tightly coupled cooperation; [13.100] for an 
overview and discussion. Such tasks typically require 
low-level sensor-sharing and/or for higher-level explicit 
coordination. Behavior-based controllers have been de¬ 
veloped and extended in order to address these chal¬ 
lenges. For example, Parker et al. [13.101] considered 
reusable behavior units that can be automatically redis¬ 
tributed for low-level information processing. Werger 
and Mataric [13.102] described broadcast of local el¬ 
igibility (BLE) to enable higher-level group behav¬ 
iors by allowing communications to influence each 
robot’s local action selection mechanism. Gerkey and 
Mataric [13.103, 104] demonstrated scalable and effi¬ 
cient market-based coordination algorithms for multi¬ 
robot coordination in a variety of tasks, including 
tightly coupled ones (e.g., box-pushing [13.105]). 
A similar strategy was applied to a robot sensor net¬ 
work, which achieves concurrent, sequential and asyn¬ 
chronous multi-robot task allocation [13.106]. The 
market-based auction algorithm developed in this sys¬ 
tem uses a fitness function that enhances fault tolerance 
and the fairness of the task allocation process. Sedat 
and Ay dan [ 13. 1 07] use a fractal conductivity-based ap¬ 
proach to generate movements of a coordinated team of 
mobile sensors. The team explores an environment to 
reach a target that is a source of a potential field chemi¬ 
cal substance contaminated into sea or lake water. 

Several researchers have shown that behavior-based 
controllers allow for sophisticated coordination through 
a utility-centered model of the collective task. Behav¬ 
iors use this representation to produce actions that 
consider each robot’s impact on the performance of the 
group as a whole. Iocchi et al. [13.108] have shown 
this in heterogeneous multi-robot system, while Batalin 
and Sukhatme [13.109] demonstrated that complex, 
interrelated and dynamic tasks can be performed in 
coordinated ways in robots with behavior-based con¬ 
trollers interacting with a sensor network. Stroupe and 
Batch [13.110] considered a mapping task and devel¬ 
oped move value estimation for robot teams (MVERT), 
essentially a behavior-based method for maximizing 
group knowledge. 

Because behavior-based methods lend itself natu¬ 
rally to multi-robot control problems, they have had 
a significant influence on that field of research. Sim¬ 
mons et al. [13.111] described a hybrid architecture 
designed for group-level coordination which employs 


behaviors as a method for organizing low-level safety- 
critical controller code. Behaviors have also been used 
to structure controllers and networked communication 
in minimalist systems [13.112]. Some multi-robot re¬ 
search with a control-theoretic flavor has used separate 
executable processes that can be switched on and off 
dynamically depending on task constraints, in a manner 
very similar to behavior-based control [13.113]. 

Behavior-based controllers have also been built us¬ 
ing genetic algorithms [13.114], in which the controller 
learned through coevolution five behavioral compo¬ 
nents (waypoint prediction, force field trajectory, speed 
regulation, reversing and heading alignment), enabling 
it to outperform all its opponents in a real time car 
racing game held during the 2007 IEEE Congress on 
Evolutionary Computation (CEC). 

Behavior-based architectures have also been used in 
complex vision systems for identification rather than 
control. In those contexts, each behavior represents 
a small unit of visual computation, such as frame differ¬ 
encing, motion detection, and edge detection, resulting 
in biologically inspired vision and attention behav¬ 
ior [13.115, 116]. 

Behavior-based architectures have also been devel¬ 
oped for the control and learning in humanoid robots. 
In the past, the Cog project demonstrated behavior- 
based control for articulated manual and eye-hand co¬ 
ordination [13.117,118]. Edsinger [13.119] developed 
a lightweight behavior architecture for organizing per¬ 
ception and control of Domo. The architecture allows 
for specification of time-contingent behaviors and dis¬ 
tributed computation, resulting in a real-time controller 
that allows Domo to operate in an environment with 
humans. Kismet [13.120] involved several behavior- 
based systems that controlled perception, motivation, 
attention, behavior, and movement in a human-robot 
interaction (HRI) context. Each behavior represented 
Kismet’s individual drives and motivations. Situated 
modules and episode rules were introduced as part of an 
HRI architecture by Ishiguro et al. [13.121] and Kanda 
et al. [13.122]; they employed an ordered set of general- 
use situated modules and behaviors that are contingent 
on a condition, used in sequences defined by a set of 
task-specific episode rules. 

13.6.1 Adaptive Behavior Selection - 
A Case Study 

Designing robots capable of safely and effectively in¬ 
teracting with humans in real-world settings is one of 
the ultimate challenges in robotics. Representation and 
abstract reasoning are necessary to keep up with and 
adapt to the inherently complex properties of such en¬ 
vironments and tasks. 
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Beliefs, emotional signaling 



Fig. 13.4 Hybrid Behavior- 
Based Architecture (HBBA) 


Adaptation in behavior-based systems [13.28] gen¬ 
erally consists of modifying internal parameters of 
the behaviors, or modifying the set of activated be¬ 
haviors. Many possible avenues to implement such 
mechanisms exist. One possibility consists of adding 
to a behavior-based architecture the idea of intention¬ 
ally activating, monitoring, and configuring behaviors. 
An architectural framework based on this idea has 
evolved over the years, having been implemented on di¬ 
verse robotic platforms and tested in various operational 
conditions: 

• EMIB - Emotion, Motivation and Intentional Be¬ 
haviors - has been applied to robots with different 
capabilities, such as a robot that uses activation vari¬ 
ables, topological localization and mapping, and 
fuzzy behaviors to explore and characterize an envi¬ 
ronment [13.45,46], and on an autonomous rolling 
robot that only uses simple sensors and a micro¬ 
controller to generate purposeful movements for 
conducting studies regarding on toddler-robot inter¬ 
action [13.123, 124]. 

• MBA - Motivated Behavioral Architecture [13. 125, 

126]). In the American Association for Artificial 
Intelligence (AAAI) mobile robot competition and 
as shown in MBA was used with 

the Spartacus robot to demonstrate how a behavior- 
based system could integrate planning and sequenc¬ 
ing tasks under temporal constraints, interleaving 
planning and execution [13.127] (as in [13.128] 


and [13.129]), and with spatial localization capa¬ 
bilities using a previously generated metric map. 
The system also used behavioral message read¬ 
ing [13.130], sound processing capabilities with an 
eight-microphone system for source localization, 
tracking, real-time sound separation [13.131, 132], 
and a touch screen interface to allow the robot to 
acquire information about where it is in the world, 
what it should do, and how it should do it [13.125]. 

HBBA - Hybrid Behavior-Based Architec¬ 
ture [13.133] - illustrated by Fig. 13.4, is the current 
instantiation of the architectural framework. The 
Behavioral Layer consists of behaviors that are acti¬ 
vated and configured according to what are referred 
to as the Intentions of the system (derived by the 
Motivation Layer). Intentions are associated with the 
activation and configuration of behaviors and with 
the modulation of perceptual modules. The highest 
layer uses Motivations acting as distributed processes 
(just like Behaviors) and manifesting Desires for the 
satisfaction or inhibition of Intentions. The Intention 
Workspace processes these Desires to issue Intentions. 
The Intention Workspace can also provide Emotional 
Signaling (detecting situations based on models 
of how Behaviors are exploited over time [13.44, 
134-136]), and generates Beliefs (i. e., knowledge 
about the interaction dynamics of the robot and the 
environment) which can be used to identify conflicts 
between Desires and their satisfaction. The objective 
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Fig. 13.5 Photographs of 
the front and back of the 
robot, IRL-1 (30 DOF and 
55 sensors). DEAs stand for 
Differential Elastic Actuators 
(after [13.133]) 


is to have Intentions emerge from the interaction of 
independent motivations. Desires are communicated 
to the Intention Workspace, which plays a role similar 
to a coordination layer in a hybrid architecture, or to 
action selection with motivations acting as behaviors 
and Intentions emerging from the interaction of inde¬ 
pendent motivations. Our design requirements for the 
Intention Workspace module is to implement generic 
mechanisms (i. e., mechanisms that are independent of 
the robot’s physical capabilities and intended usage) to 
generate Beliefs (i. e., data structures that represent de¬ 
sires, knowledge about the interaction dynamics of the 
robot and the environment), and identify conflicts be¬ 
tween Desires and their satisfaction. The Exploitation 
link is used to derive information about the effective 
use of Behaviors as influenced by the events occurring 
in the world and the Action Selection mechanism. 
We believe that observing the exploitation of behav¬ 
iors over time is an important source of information 
about the emerging functionality that comes from the 
Behaviors and the Motivations, because it combines 
both a representation of the environment (from the 
behavioral percepts associated with Behaviors) and of 
the control policy (from the internal decision-making 
processes). 

HBBA is currently being used with IRL-1 [13.137, 
138], an interactive omnidirectional platform shown in 
Fig. 13.5, and ksaMHHJl. 

IRL-1 has behaviors for safe navigation, recharge 
and social interaction (with gestures, voice and vision). 
Action selection is priority-based. Motivations are im¬ 
plemented to provide the intrinsic modes of operation 
of the robot (safe navigation while exploring the world, 
interact with people and ensuring energetic autonomy), 
to localize and plan paths to desired locations, and even¬ 
tually to plan and schedule tasks [13.139]. Just like 


action selection at the behavior-based level, the Inten¬ 
tion Workspace plays a key role. More specifically, in 

HBBA, it involves: 

• A tree representation of desires. Each desire can be 
associated with a particular configuration and acti¬ 
vation of one or more behaviors. The tree represen¬ 
tation of desires provides a general representation 
of Desires’ interdependencies issued by the inde¬ 
pendent motivations, from high-level/abstract de¬ 
sires (e.g., Deliver message ) to primitive/behavior- 
related desires (e.g., Avoid obstacles), allowing for 
behavior configurations to arise in a distributed 
fashion based on the capabilities available to the 
robot. This allows motivations to exchange informa¬ 
tion asynchronously on how to activate, configure, 
and monitor behaviors. Lor instance, one motiva¬ 
tion may monitor the robot’s energy level to issue 
a recharging desire, which would be associated with 
a Recharge behavior that would make the robot 
opportunistically detect and dock to a charging sta¬ 
tion. Meanwhile, if the robot knows where it is and 
can determine a path to a nearby charging station, 
a path planning motivation could add a sub-desire 
of navigating to this position, using a Goto behav¬ 
ior. If a person is detected, a social motivation may 
provide an alternative desire by engaging into a con¬ 
versation to ask for guidance toward the nearest 
charging station. The robot would then have three 
alternative solutions to fulfill its desire to recharge. 

• A representation of the history of behavior exploita¬ 
tions (similar to one described in Sect. 13.5.4), in¬ 
dexed by intentions. Lor instance, for the recharging 
desire example, if being guided by people reveals 
not to be as reliable as using the internal map, a pref¬ 
erence could be given to the second option. 
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• An attention selection mechanism to module com¬ 
putational load, by percepts selection or out¬ 
put complexity reduction. For instance, artificial 
audition [13.131, 132] can be modulated to de¬ 
tect sounds, localize sound sources or separate 
sound sources depending on the presence of peo¬ 
ple surrounding the robot and its intentions to 
interact. A Simultaneous Localization and Map¬ 
ping (SLAM) algorithm can change resolution de¬ 
pending on available processing load of onboard 
computers. 

• An emotion-based mechanism to regulate conflicts 
between desires and intentions. Robot’s adaptabil¬ 
ity in dynamic, continuous and unpredictable en¬ 


vironments ultimately depends on the detection 
of the situations for which their regular decision¬ 
making process is not appropriate. Psychologists 
have identified that one of the functions of hu¬ 
man emotion is to highlight these kinds of situ¬ 
ations, allowing other cognitive processes to ad¬ 
dress them. Therefore, an emotional process that 
detect such situations from the use of temporal 
models of intentions could be beneficial [13.44]. 
This process does not rely on a priori knowl¬ 
edge of specific environmental conditions nor of 
the robot’s mission objectives, but only on tempo¬ 
ral analysis of how behaviors satisfy the robot’s 
intentions. 


13.7 Conclusions and Further Reading 

This chapter has described behavior-based control, f 
a methodology for single- and multi-robot control r 
aimed at situated robots operating in unconstrained, f 
challenging, and dynamic conditions in the real c 
world. While inspired by the philosophy of reac- i 
tive control, behavior-based systems are fundamen- £ 
tally more expressive and powerful, enabling rep¬ 
resentation, planning, and learning capabilities. Dis- i 
tributed behaviors are used as the underlying building I 
blocks for these capabilities, allowing behavior-based i 
systems to take advantage of dynamic interactions 1 
with the environment rather than rely solely on ex- c 


plicit reasoning and planning. As the complexity of 
robots continues to increase, behavior-based princi¬ 
ples and their applications in robot architectures and 
deployed systems will evolve as well, demonstrating 
increasingly higher levels of situated intelligence and 
autonomy. 

Interested readers can find more information regard¬ 
ing behavior-based systems in other chapters of this 
Handbook, as well as in Brooks [13.1 40 1, Arkin [13.22], 
in artificial intelligence and robotics textbooks [13.141, 
142], and in introductory textbooks on mobile robot 
control [13.143-145]. 
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14. Al Reasoning Methods for Robotics 


Michael Beetz, Raja Chatila, Joachim Hertzberg, Federico Pecora 


Artificial intelligence (Al) reasoning technology in¬ 
volving, e.g., inference, planning, and learning, 
has a track record with a healthy number of suc¬ 
cessful applications. So can it be used as a toolbox 
of methods for autonomous mobile robots? Not 
necessarily, as reasoning on a mobile robot about 
its dynamic, partially known environment may 
differ substantially from that in knowledge-based 
pure software systems, where most of the named 
successes have been registered. Moreover, recent 
knowledge about the robot's environment can¬ 
not be given a priori, but needs to be updated 
from sensor data, involving challenging problems 
of symbol grounding and knowledge base change. 

This chapter sketches the main robotics¬ 
relevant topics of symbol-based Al reasoning. Basic 
methods of knowledge representation and infer¬ 
ence are described in general, covering both logic- 
and probability-based approaches. The chap¬ 
ter first gives a motivation by example, to what 
extent symbolic reasoning has the potential of 
helping robots perform in the first place. Then 
(Sect. 14.2), we sketch the landscape of represen¬ 
tation languages available for the endeavor. After 
that (Sect. 14.3), we present approaches and re¬ 
sults for several types of practical, robotics-related 
reasoning tasks, with an emphasis on temporal 
and spatial reasoning. Plan-based robot control 
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is described in some more detail in Sect. 14.4. 
Section 14.5 concludes. 


Historical debates about the necessity and wisdom of 
employing symbolic reasoning in autonomous mobile 
robots notwithstanding (Chap. 13), there seems to be 
agreement now that some part or layer in the control 
system of such robots should or could include rea¬ 
soning. Hybrid control architectures (Chap. 12) would 
be typical software structures for solving the difficult 
problem of amalgamating their contributions with those 
of other parts of the controller, yet guaranteeing suffi¬ 


ciently short control cycle times that the robot can act 
safely in a dynamic environment. 

Symbolic reasoning is understood here in the classi¬ 
cal sense of artificial intelligence (Al), i. e., delibera¬ 
tion based on symbols as in first-order predicate logic 
(FOPL) or Bayesian probability theory, but typically 
with restrictions and/or extensions thereof, which have 
to be traded off against each other to achieve a suitable 
combination of expressivity and inference speed. 
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14.1 Why Should a Robot Use Al-Type Reasoning? 


If robots are to accomplish tasks that are too compli¬ 
cated for specifying the appropriate behavior for all 
possible situations beforehand, they should be able to 
reason themselves about what is the appropriate way to 
perform the task - even if the task is specified vaguely. 
In this setting, reasoning means inferring the needed 
knowledge from what is spelled out explicitly in the 
available knowledge. It further means automated rea¬ 
soning using methods from AI including knowledge 
representation (KR) and inference, heuristic search, and 
machine learning. 

To perform their course of action appropriately, 
robots need to reason about the actions they intend 
to perform, their intended effects, the unwanted side 
effects, whether they are executable in the respective 
situations, the goals the actions are to accomplish, and 
so on. Consider, for example, the seemingly simple task 
of picking up an object from a table. To do so, the 
robot has to decide where to go in order to pick up the 
object, which hand(s) to use, how to reach for the ob¬ 
ject, which type of grasp to apply, where to place the 
gripper, how much grasp force to apply, how much lift 
force, how to lift the object, where to hold it, and so 
on. 

If programmers have to specify the decisions for 
every conceivable combination of object and task, the 
control program for picking up an object becomes 
very complex. But even this program would in most 
cases not suffice to generate competent robot behavior, 
because how the pickup action should be performed de¬ 
pends on the context, too - the state of the object, the 
task to be performed, the scene that the object is located 
in. If the object is a glass filled with juice, it has to be 
held upright and if it is a bottle the robot intends to fill 
a glass with, it should not grasp the top of the bottle. 
If the bottle is to be picked up in the middle of a clut¬ 
tered scene, then grasping the top of the bottle might 
be the best choice. If the scene is cluttered and the pur¬ 
pose is filling a glass, then the robot might even have to 
re-grasp after picking up the object. 

The situation gets even more complicated if the task 
is not a single action, but a compound activity such as 


cleaning the table. In this case, AI reasoning methods 
can be employed to enable programmers to specify the 
course of activity very compactly: for every object on 
the table put it where it belongs. For such a vague plan 
to be a sufficient activity prescription, the robot has to 
infer the information needed to decide on the appropri¬ 
ate parameterizations of the pickup actions and it has to 
infer where objects belong depending on whether they 
are to be used again, are dirty, perishable, etc. Compe¬ 
tently cleaning the table might also require the robot to 
think about the order in which it wants to put away the 
objects. It might decide to group the carry actions ac¬ 
cording to the place where the items have to be placed, 
or clean up perishable items first. It might reason about 
whether to stack items, get a tray and use that, and 
whether to leave doors of cupboards open. 

Thus, an important task of AI reasoning techniques 
is: given a vague instruction, infer what is the appropri¬ 
ate action, what is the appropriate object to be acted on, 
what is the appropriate order of action executions, and 
what is the appropriate way of performing every action. 

To deal with these issues, autonomous robots can 
represent and reason about aspects including the robots’ 
capabilities, their environments, the objects they are to 
act on, their actions and the effects they cause, and other 
agents in the environment. 

Important kinds of reasoning that robots should be 
capable of, therefore include the following: 

• Prediction (often called temporal projection) ', infer¬ 
ring what will (probably) happen if the intended 
course of action is executed 

• Envisioning: inferring (all) possible events and ef¬ 
fects that will happen if a plan gets executed in 
a hypothetical situation 

• Diagnosis: inferring what caused a particular event 
or effect in plan execution 

• Query answering: given some knowledge precon¬ 
ditions for plan execution (e.g., a robot has to 
know the combination of a safe in order to open 
it), inferring pieces of knowledge that satisfy these 
knowledge preconditions. 


14.2 Knowledge Representation and Processing 


Reasoning requires that the reasoner - in our case, 
a robot - has an explicit representation of parts or as¬ 
pects of its environment to reason about. Two questions 
come up immediately: what are suitable formats for 
such an explicit representation, and where does the rep¬ 
resented knowledge come from? 


The second question refers to the problem of gener¬ 
ating and maintaining in real time a symbolic descrip¬ 
tion of the robot’s environment, or at least of some 
part of it, based on, possibly, a previous symbolic de¬ 
scription and on recent environment information as by 
sensors and by communication with other agents. In full 
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generality, this problem is currently unsolved, involving 
AI fundamentals such as symbol grounding [14.1] and 
object anchoring [14.2]. So practical symbolic reason¬ 
ing in a robot is restricted to that part of its knowledge 
that can be kept sufficiently recent. This includes, obvi¬ 
ously, static knowledge about the environment, such as 
the topological elements and their relations in a build¬ 
ing; transient knowledge available in symbolic forms, 
such as in facility management data bases; and, most 
challenging, symbolic data that has to be distilled from 
sensor data. Object recognition (Chap. 33) is a crucial 
ingredient to solve this; these issues will not be treated 
here. 

This section deals with answers to the first question, 
i. e., formalisms suitable for representing knowledge. 
Suitability has two aspects here that are to be treated 
as two sides of one coin. One is epistemological ade¬ 
quacy. Does the formalism allow the targeted aspects 
of the environment to be expressed compactly and pre¬ 
cisely? The other is computational adequacy. Does the 
formalism allow typical inferences to be drawn effec¬ 
tively and efficiently? There is a tradeoff between the 
two adequacies, given that very rich, expressive, and 
hence epistemologically appealing formalisms are typi¬ 
cally accompanied by intractable or even undecidable 
inference problems, and vice versa. KR, then [14.3, 
p. xiii]: 

is the field of AI that focuses on the design of 
formalisms that are both epistemologically and 
computationally adequate for expressing knowledge 
about a particular domain. 

The plural in formalisms does not come by chance: 
there is no such thing as the KR language. The reason 
is twofold. 

First, the archetype of KR languages, FOPL, is un¬ 
decidable, i. e., any reasoner dealing with a language 
at least as expressive as FOPL is not guaranteed to 
terminate on a single query, let alone terminate fast. 
However, full FOPL expressivity is not always needed; 
for example, to represent a finite domain or to make re¬ 
stricted use of quantification may lead to decidable, and 
even tractable representation languages. Second, some 


applications of KR and reasoning, like quite a num¬ 
ber of applications in Robotics, require evidence rather 
than truth to be represented, and FOPL, in particular, 
does not come handy for doing that. So, there are good 
reasons, both for pragmatics and for epistemological 
adequacy, for having several representation languages - 
and that has happened in AI. 

Table 14.1 lists in its rows four broad types of KR 
languages, which have been in use in Robotics. De¬ 
scription logics (DLs) are in fact a specific family of 
relational languages, of which there are decidable and 
tractable members. The table gives in its columns the 
Assertions of the language types, i. e., the language el¬ 
ements used to represent some domain; the Queries, 
i. e., the type of problems that a reasoner in a particular 
language is supposed to solve; the Subject that a par¬ 
ticular language of the respective type is normally used 
for representing; and Examples of such languages. The 
remainder of this section sketches generic KR in logic 
and in a probabilistic language, including the respective 
types of inferences. Specific robotics-related instances 
of reasoning on the symbol level are described in the 
following section, in particular, reasoning about tempo¬ 
ral and spatial objects and relations, which are pervasive 
in a robot’s reasoning. 

14.2.1 Logic-Based Reasoning 

As stated above, FOPL does not qualify for a KR for¬ 
malism in the above-mentioned sense of being both 
epistemologically and computationally adequate for 
typical application domains - in many cases, it is 
neither of the two. Yet, it is the background for un¬ 
derstanding conceptually and mathematically general 
relational languages. 

In this short text, we can only introduce KR in logic 
by way of example, assuming familiarity with the basic 
notions of FOPL. For more comprehensive and formal 
treatments, we refer to the broad body of literature: ev¬ 
ery typical AI textbook introduces FOPL, and [14.4, 
5] are no exceptions; [14.6] is a principled introduction 
to logics; [14.7] a practical one, and [14.8] a concise 
mathematical one. 


Table 14.1 Abstract categorization of main relational languages as used in Robotics (see text for explanations) 


General relational 
languages 

Assertions 

Queries 

Subject 

Example 

Sentences, theories 

(Logical) consequence 

Situations, contexts, rela¬ 
tions 

FOPL, prolog, situation 
calculus 

Description logics 

A-Box and T-Box contents 

Subsumption, consistency 

Ontologies, encyclopedias 

OWL-DL 

Probabilistic 

languages 

Prior and conditional prob¬ 
abilities 

Conditional 

probabilities 

Evidence 

Bayesian 

networks 

Languages 

for temporal 

and spatial reasoning 

Temporally and/or 
spatially qualified 
sentences 

Logical consequence, 
satisfiability, minimal 
representation 

Desired or observed spa¬ 
tial/temporal state 
of affairs 

LTL, IA, TCSP, RCC, 
ARA+ 
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Representing knowledge in some variant of logic fa¬ 
cilitates to draw inferences from that knowledge using 
provably sound and/or complete calculi. Automated de¬ 
duction is a subfield of logic and AI that offers a healthy 
number of very powerful implemented deduction sys¬ 
tems that can readily be used. Reference [14.4, Chap. 9] 
gives an introduction; [14.9] is a traditional handbook 
covering all the classical topics of logical reasoning. 

Relying on logical inference, a robot could deduce 
a large number of facts that might otherwise be hard or 
impossible to get at. For example, assume a robot per¬ 
ceives by acquiring and interpreting sensor data, that 
the door D509 in an office building is currently closed: 
Closed(Dso9) expressed in some ad hoc FOPL lan¬ 
guage, assuming intuitive interpretations of symbols. 
Let us further assume that the robot’s base of static 
knowledge about the building contains the sentences 

Coimects(Dsog, C 5 , R509) , 

Connects(Dso&, Cs,Rs os) . 
Connects(D 5 o &a ,R 50 i ,R 509 ) , 

V d, /1, l2.[Connects(d, h,h) -*> 

Connects(dJ2,h )] , 

Vd.[Closed(d) -o- -^Open(d)] , 

VI.[At(l) —>■ Acessible(l)) , 

V/i, l2.[Accessible(l\) -> 

( 3 d.[Connects(d, 1 \ , L) A Open(d)\ —> 
Accessible(l 2))]. ( 14 . 1 ) 

Constants /), and variable d denote doors; /?, denote 
rooms; C, corridors; variables /, /, denote locations, i. e., 
rooms and corridors. Assume that the robot’s localiza¬ 
tion tells it its current room or corridor location as At{- ). 

Then, assuming the robot knows it is At(Cs), ob¬ 
serving Closed(Dso9) entails -'Open{D^Q 9 ), and, more 
interestingly, that Accessible{R=,Q 9 ) is true only if 
Open(Dsog) A Open(D^osa) is true. So, performing, for 
example, a delivery task to room R509, the robot may 
replan its route through R508. unless at least one of 
^ 508 . f^ 508 a is known to be closed. If the status of one or 
both of them is unknown (neither Open(-) nor Closed(-) 
is entailed by the current knowledge base), then acces¬ 
sibility of R509 can be neither proven nor disproven. 
That would leave open the option of planning the route 
through D508 and D^ a , gathering their required sta¬ 
tuses on site. 

So, as this little example may show, FOPL is a pow¬ 
erful representation and reasoning tool. Moreover, its 
theory is very well understood. In particular, it is well- 
known that consequence in FOPL is in general unde- 
cidable, that means, a sound and complete deduction 


algorithm cannot even guarantee termination for some 
particular inference attempt, let alone speedy results. 

However, that does not mean logic as a representa¬ 
tion and inference tool needs to be completely disposed 
of. Many applications do not require the full expres¬ 
sivity of FOPL. Moreover, there are many interesting 
language subsets of FOPL that are decidable and can be 
handled by algorithms that are efficient in most practi¬ 
cal cases. So considerable effort by the KR community 
has gone into identifying FOPL subsets and fitting 
inference procedures that qualify for being both episte¬ 
mologically and computationally adequate for a broad 
number of applications. We will consider two of them in 
more detail: propositional logic and description logics; 
we will then go briefly into using logics for high-level 
robot control. 

Propositional Theories 

In quite a number of practical cases, FOPL theories 
(sets of formulas) represent in fact finite domains. Log¬ 
ically speaking, they have finite Herbrand universes or 
can at least be recast in a form such that they have. In 
this case, it may still be handy to express the domain 
theory in FOPL syntax, but all that goes beyond a purely 
propositional theory is just for notational convenience. 
For example, an axiom stating that a robot can only be 
in one location (room or corridor) at a time 

VZi, h\At{h) -> (-At(/ 2 ) V h =h)] (14.2) 

can be flattened for a finite building into the, clumsy 
but equivalent, form handling all locations explicitly, 
for example in a six-storey building 

Af(Z?oot) —> [ _| A/'(/?oo 2 ) A • • • A ->Af(/?5i4)A 
-'At(Co) A • ■ ■ A -iAt(Cs)] ■ 

At(Rocn) —*■ [ _, At(Rooi) A ■ ■ ■ A -A?(R514 )a 
~'A t(Co) A • ■ • A > At(C $)] , 

( 14 . 3 ) 

where every ground (variable-free) instance of a pred¬ 
icate, such as the At instances above, are to be con¬ 
sidered as propositional variables, regarding textual 
identity. 

The good news here is that the corresponding flat 
theory to a FOPL theory over a finite Herbrand universe 
is propositional, hence, it is decidable. Moreover, under 
some practical conditions - e.g., if the variables in the 
FOPL theory are sorted (that is, the information is avail¬ 
able that the variables l\, I 2 , e.g., range over rooms and 
corridors) - it can even be generated mechanically from 
a more compact FOPL syntax. 

The potentially bad news is that the now proposi¬ 
tional theory may of course consist of a huge amount 
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of propositional sentences: in general, as all com¬ 
binations of variable substitutions in all FOPL sen¬ 
tences need to be generated, the growth is multiex¬ 
ponential in terms of the domain sizes of the FOPL 
variables. 

However, the technology for propositional satisfi¬ 
ability checking or model checking is making consid¬ 
erable progress, allowing propositional theories with 
thousands or even tens of thousands of variables to be 
handled in the order of seconds of computation time 
on regular hardware. The respective methods are par¬ 
ticularly efficient if many models exist for a satisfiable 
propositional formula or if the truth or falsity of many 
ground facts is known a priori (such as At(Cf) for 
a robot by independent localization). Both is often the 
case in practical KR. 

References [14.4, Chap. 7.6] and [14.9, Chap. 24] 
give introductions to model checking. One modern fam¬ 
ily of methods is based on the classical Davis-Putnam 
(DPLL) algorithm [14.10]. It attempts to construct sys¬ 
tematically a propositional model of a given theory, ef¬ 
ficiently propagating interpretation constraints for vari¬ 
ables. Another family of algorithms applies local search 
techniques, attempting to generate interpretations using 
random (Monte Carlo) variable assignments. Refer¬ 
ence [14.11] is a Web page maintained for the annual 
satisfiability checking competitions held together with 
the annual SAT (Theory and Applications of Satisfi¬ 
ability Testing) conferences; it gives a continuously 
updated overview over current top-performing satisfi¬ 
ability checking systems. 

Description Logics 

DL have emerged as rigorous formalizations of some¬ 
what intuitive KR forms of the AI of the 1970s, namely, 
semantic networks and frame systems. Logically speak¬ 
ing, DLs, of which there is a considerable variety, form 
a certain family of subsets of FOPL, some of which are 
decidable or even tractable. 

There are two parts of representing knowledge 
about a particular domain using a DL language. First, 
the upper ontology of the domain is formulated. It intro¬ 
duces the general domain concepts as well as relations 
between these concepts. A particularly interesting type 
of relations occurring in all ontologies is the superclass- 
subclass relation. Given that most DLs - as part of 
the means for enforcing decidability - strictly disal¬ 
low cyclic relations between concepts to be specified, 
the superclass relation imposes a hierarchical taxonomy 
on concepts, which serves for defining property inheri¬ 
tance, much like in object-oriented programming. This 
first part of a DL-based domain representation is, for 
historical reasons, often called the T-Box (or termino¬ 
logical knowledge). 


A concept in the DL language corresponds to 
a unary predicate. To give an example, an ontology 
of robot navigation domains might include concepts 
like Door, Location, and so on. The concept hierarchy 
is built by defining concept equality = or a subcon¬ 
cept property, e.g., Room C Location, and Corridor C 
Location. Concepts can be combined using concept 
conjunction, disjunction, and negation (n,U,-\ re¬ 
spectively), allowing, e.g., concept definitions like 
Location = Room U Corridor, Door = Closed U Open, 
and Open = —•Closed. 

Roles in a DL language correspond to binary pred¬ 
icates, such as leadsTo for a door and a location. 
Inversity, intersection, and union of roles are defined as 
expected, where leadsTo = leadsFrom~ l is an example 
for defining inverse roles. Roles can be composed, such 
as in defining adjacent = leadsFromoleadsTo (location 
/ is adjacent to m if and only if some door connects 
them). Finally, concepts and roles can be combined for 
defining new concepts and roles. In particular, it is pos¬ 
sible to quantify over role-fillers, i. e., the individual 
objects (see next) that can be consistently substituted 
for role arguments. For example, one could define 
BlockedLoc = Location n —>3leadsFrom.Open (assum¬ 
ing the intuitive binding rules of the operators). Differ¬ 
ent variants of DLs differ in what operators they make 
available; the set of available operators and additional 
constraints on definitions shape both the expressivity 
and the computability of the respective DL variant - 
the spectrum ranges from undecidable to tractable. 
See [14.3] for details. 

As the second part of domain representation using 
DLs, individual objects have to be introduced into the 
language of concepts and roles. This part of the domain 
representation is called A-Box (or assertional knowl¬ 
edge). For example, Room(R=,Q<j), leadsTo(Dso9,R509), 
and Closed(DsQ 9 ) could be asserted. 

DLs have a number of reasoning services to offer, 
which are based on logical inference in a given T-Box 
and A-Box. They include consistency of the concept 
definition, subsumption and disjointness of concepts, 
consistency of the A-Box wrt. the T-Box, concept and 
role instances, all of which are decidable in many DLs. 
Given the T-Box and A-Box rudiments above, it would, 
e.g., be concluded that everything is consistent and that 
BlockedLoc(Rso 9 ) (note that only door D 509 is known 
here to lead to Rsofi). These DL inferences are theoreti¬ 
cally intractable, but run efficiently in most of practical 
cases. 

Reference [14.3] provides a comprehensive 
overview of DL. In 2004, the WWW consortium 
(W3C) has defined the Web Ontology Language 
OWL [14.12] as a technical basis for the Semantic 
Web, which was followed by OWL 2 in 2009 [14.13]. 
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Part of the language is OWL-DL, a classical DL in 
the sense just described. OWL-DL ontologies are 
publicly available over the Web. Reference [14.14] 
gives a tutorial introduction. 

Logics for High-Level Robot Control 
Robot domains are dynamic by nature, including at least 
one physical agent, namely, the robot. Capturing that in 
logic-based formalisms is possible and has been under¬ 
taken, as introduced in [14.4, Chap. 12.3]. However, it 
poses some conceptual and technical problems, which 
we will describe at the end of this section on Logics. 

Very briefly, one of them comes from the require¬ 
ment of expressing concisely and reasoning efficiently 
with logical models of events in general, and actions 
in particular. Logically speaking, an individual action 
changes the truth value of a limited number of facts, 
leaving everything else as it was. For example, mod¬ 
eling on some abstract level the action of going from 
one location to another as atomic, it changes the robot’s 
current location before and after; depending on the 
modeling, it may also change the battery state and the 
mileage count; but it does not change the layout of the 
building or the name of the president. The problem of 
formalizing actions concisely in a logical language so 
that they allow facts to be inferred efficiently that may 
or may not have changed after applying a sequence of 
actions, has been termed th e. frame problem. It has re¬ 
ceived much attention in the literature; there are now 
a number of practical solutions to it. 

Another problem concerns knowledge-base update 
made necessary by independently changing facts. Con¬ 
sider, for example, a robot sitting - as it is told by its 
self-localization - in front of some door D, which is 
believed to be open, but the robot perceives a closed 
door. Logically, this is a contradiction. Now there are 
in theory several ways to make the knowledge and the 
perception consistent. One is to assume that D has been 
closed since learning that it was open - probably the 
most intuitive explanation. Logically just as good would 
be, e.g., that the perception is faulty, or that the robot 
has been teleported in front of a known closed door. 
Among these explanations, some are more intuitively 
plausible than others; logically, some would require less 
formulas of the knowledge base to be withdrawn and 
should therefore be preferred. Ideally, after replacing 
an old piece of information with a new one, one would 
have to make sure that the consequences of a retracted 
formula are no longer believed. 

Theoretically, these problems are arbitrarily diffi¬ 
cult. Practically, they can be sufficiently restricted to 
allow solutions within a neat logical framework. Typ¬ 
ical solutions would introduce some notion of state 
or holding period for formulas, following the clas¬ 


sical example of the situation calculus [14.15]. That 
allows change to be tracked. Typical solutions would 
also give up completeness of inference, resorting to 
a Prolog-like inference engine. Three examples for 
such solutions with reported applications in robot con¬ 
trol are GOLOG [14.16], event calculus [14.17] and 
FLUX [14.18]. Another family of solutions would 
model the course of time in a more fine-grained way 
than switching between states. Temporal logics are ap¬ 
propriate here, and the temporal action logic (TAL) is 
an example, which has even been integrated in a perfor- 
mant planner [14.19], It leads to a more sophisticated 
form of temporal reasoning that will be treated in 
Sect. 14.3. 

14.2.2 Probabilistic Reasoning 

KR formalisms based on logics are worth considering 
whenever factual knowledge is to be represented, from 
which consequences are to be queried. However, part of 
the knowledge that a robot might use about its environ¬ 
ment does not really have this character. 

Uncertainty is one of these traits, or rather, a whole 
family of them, as uncertainty is in itself an overloaded 
concept. Lack of knowledge is one of its aspects. Log¬ 
ics can handle this insofar as truth or falsity of some 
facts may remain undetermined. If too much is unde¬ 
termined in a knowledge base, logics will no longer be 
able to make interesting deductions though, as every¬ 
thing is possible logically. Yet, the different possibilities 
may differ considerably in likelihood, according to in¬ 
tuition. The point here is to represent and reason about 
evidence. 

The field of KR has adopted Bayesian probability 
as a means for representing and reasoning with evi¬ 
dence, using correlations between facts rather than strict 
implications. Note that this approach amalgamates dif¬ 
ferent sources of lack of precision and completeness of 
knowledge. Some piece of knowledge may be unknown 
because it is in principle unknowable, or because it was 
judged too costly to build a precise theory or determine 
all information relevant for making a sound deduction. 
In either case, working with probabilities rather than bi¬ 
nary truth values can serve as an approximation. Note 
that the notion of truth is still the same here as in clas¬ 
sical logics: objectively, a fact is supposed to be either 
true or false; probability just models the subjective evi¬ 
dence that the fact is true. 

We next describe two popular and powerful rep¬ 
resentation and processing formats for probabilistic 
knowledge: Bayesian networks (BNs) and Markov de¬ 
cision processes (MDPs). In analogy to Sect. 14.2.1, 
we here assume familiarity with the basic notions of 
probability theory. Reference [14.4, Chap. 13] gives an 
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excellent basic introduction; for a more comprehensive 
treatment, there is a large variety of introductory text¬ 
books around, [14.20] being an example. 

Bayesian Networks 

Inference in Bayesian probability theory basically 
means to infer the probability of some event of interest, 
given prior and dependent other relevant probabilities. 
Practically, an important type of probabilistic infer¬ 
ence is diagnostic reasoning from observed effects back 
to hidden causes, given rules that specify conditional 
probabilities from causes to effects. So the problem is, 
for a potential cause C and an observed effect E: Given 
prior probabilities P(C) and P(E) and the conditional 
probability P(E\C), determine the posterior P(C\E). 
The solution is of course given by the Bayes rule as 

P(E\C)P(C) 

P(C\E) = K K ’ . ( 14 . 4 ) 

v 1 > p(E) 

However, like in logical reasoning, the theoretically 
appealing principle turns out to be impractical if ap¬ 
plied naively. Consider that not just one effect E may 
be observed, but moreover, not all of them 

are conditionally independent. A generalized form of 
the Bayes rule to calculate the correct posterior is 
straightforward, but who can specify all the conditional 
probabilities involved - in the worst case 0(2") of them, 
where n may easily range in the hundreds in practical 
cases? 

Until the late 1980s, this problem was more or less 
circumvented. One way to do so was to treat the in 
bad faith as independent: simply take the n individual 
conditional probabilities P(£,]C) and use them for ap¬ 
proximating straightforward the full joint probability 
distributions. Reference [14.4, Chap. 14] reviews this 
and other alternative approaches. 

The solution used ever since it has appeared [14.21] 
is Bayesian networks (BN), more recently subsumed 
under the more general concept of graphical models. 
The idea is to represent the random variables as nodes 
in a directed acyclic graph, where a node is directly 
preceded by a set of parent nodes if and only if it 
is directly conditionally dependent on the correspond¬ 
ing parent variables. So the huge full joint probabil¬ 
ity distribution is broken down into many, typically 
very small, local joint probability distributions without 
loss of information, the trick being to use the typi¬ 
cally many known conditional independences among 
variables to reduce the representational and inferential 
effort drastically. 

Figure 14.1 shows a simple BN expressing that D 
is dependent on B and C, with probabilities given by 
a conditional probability table which specifies locally 
the joint probability distribution. In addition, the struc- 



Fig. 14.1 Structure of a simple Bayesian network. It is associated 
with conditional probability tables. (The one for D, dependent on B 
and on C, is omitted) 



Fig. 14.2 Unfolding of a simple DBN over two time slices repre¬ 
senting two successive states. Variables x denote states, u denote 
actions, and j denote measurements. The structure of each individ¬ 
ual slice (signaled by identical variable subscripts ) is invariant over 
time 

ture says that D is independent from A, given B and C 
(a node is independent from its ancestors, given its par¬ 
ents), and that B is independent from C, given A, i. e., 
P(C\A,B)=P(C\A) and P(B\A, C) = P(B\A). 

Network inference can be interpreted in both direc¬ 
tions. Bottom up, a BN enables to explain an observed 
phenomenon using the known conditional probabilities 
(diagnosis). For example, if D is observed, the likeli¬ 
hoods of its known causes (B or C) can be inferred. Top 
down, a BN enables to propagate evidence to compute 
the probability of a phenomenon, e.g., to compute the 
probability of D given A (causation). 

For systems evolving over time, for which the 
Markov property holds, i. e., a state of a system variable 
in some state depends on no variables earlier than the 
previous state, a BN representation takes on a particu¬ 
lar form, a dynamic Bayesian network, DBN. Assuming 
that the dependences among variables within a state 
remain constant over time, and assuming further that 
a variable x at different states is represented as a family 
of variables x\,... ,x t ,..., a DBN unfolded into a BN 
has a particular structure, as sketched by example in 
Fig. 14.2. The variables representing a new state corre¬ 
spond to a new BN slice, which reproduces the structure 
of the previous state representation and whose nodes 
depend only on each other and on nodes represent- 
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ing the previous state. This locality allows inference in 
DBNs to be efficient. 

Markov Decision Processes 
In the context of robot control, uncertainty is induced by 
perception, action, and incompleteness of prior knowl¬ 
edge about the environment, all of which are related. 
Perception uncertainty results from sensor noise, from 
occlusion, from interpretation ambiguities, and other 
effects. Action uncertainty results from approximate ac¬ 
tion models (e.g., wheel slippage), unsuccessful action 
(e.g., grasping failure), or other practically unavoidable 
effects. 

Figure 14.3 depicts this uncertainty laden interac¬ 
tion between the robot and its environment. 

Probabilistic representations are used to cope with 
these uncertainties. Planning being a sequential deci¬ 
sion problem, MDP and, in the case of partial observ¬ 
ability of the environment, partially observable Markov 
decision processes (POMDPs) [14.22] are widely used 
to address them. The Markov assumption, which is the 
hallmark of this approach, is that a state s n of the system 
depends only on the previous state s n — \ and the action 
that lead to s n , and not on any earlier states. 

Under this assumption, an MDP is formally defined 
by four components (S, A, T, R) where 5 is a finite set of 
states, A is a finite set of actions, T : S x A —»■ S the tran¬ 
sition function defining the probability of state change 


upon application of a given action at time t 

T(s , a, s') = P(s,+i = s' | s t = s,a, = a) . ( 14 . 5 ) 

T is known and provided as a table of transition proba¬ 
bilities. Finally, R(s, a), R : S x A — > M, is defined as the 
reward received by the system after achieving action a 
leading to state 5 . Often, the reward is associated with 
the state only ( R(s )). In the sequential process, the re¬ 
wards are supposed to be additive. Solving an MDP is 
an optimization problem in which the sum of the ex¬ 
pected rewards, called the utility, is maximized. MDPs 
suppose that the rewards are already known. When this 
is not the case, the robot can acquire them through 
experience. One technique to do so is reinforcement 
learning, as described in Chap. 15. Figure 14.4 shows 
an example of a simple MDP. 

There are two main methods to solve an MDP. In 
the value iteration (VI) method, the first step is to com¬ 
pute the utilities U(s) for all states, and the second step 
is to compute the optimal policy n* (s) which provides, 
for each state, the optimal action. Computing the utili¬ 
ties iteratively is based on the Bellman equation [14.23] 

I/;+i(s) = R(s) + ymax„ ^ T(s, a, s')Ui{s') , 

s' 

( 14 . 6 ) 



Fig. 14.3 The 

robot acts on 
its environment 
producing a new 
state with a given 
probability. 
Knowledge of the 
robot about state 
results from its 
observation and 
is also uncertain. 
Its decisions are 
taken based on its 
beliefs 
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Fig. 14.4 A representation of an 
MDP. In this example, states A, B, C, 
D have associated rewards (—1, —10, 
—50, —100). There are three possible 
actions in each state, 11 , s, d, each 
have a given probability ( shown on the 
arrows) to lead to another state 


where 0 < y < 1 is a discount factor. After computing 
the utilities, the optimal policy is the one that pro¬ 
vides the best action (i. e., the one maximizing the 
utility) for each state. This is computed by tt*(s) = 
argmax fl ^ s , T(.v, a, s')U(.s'). So the result of the MDP 
is the best local decision. A policy is an a priori compu¬ 
tation of the best sequence of actions, which is guided 
by the maximization of a global reward instead of just 
reaching a goal with minimal cost, as in classical plan¬ 
ning. So a policy is not a plan of actions that has to be 
strictly executed. Since it already takes uncertainties in 
action execution into account, the policy can be pursued 
whatever the outcome of action execution. 

The second method is called policy iteration (PI). 
Here one starts with an initial (e.g., random) policy rt 
and, through an iterative algorithm, tries to improve it 
gradually by looking for the actions that augment utility. 


In conclusion, in a deterministic setting, the solu¬ 
tion to a planning problem is a sequence of actions. In 
an observable stochastic setting, the solution is a policy 
defined by the best local decision. 

Partially Observable Markov Decision Processes 
In the (realistic) case where the robot has no full ac¬ 
cess to the environment, decision making must be made 
under partial observability. To that end, the MDP for¬ 
malism is enriched with an observation function that 
provides uncertain knowledge. A POMDP is defined 
by six components (S, A, T,R, £2,0). The first four de¬ 
scribe an MDP. £2 is a finite set of observations o. 
O : S x A —> £2 is the observation model 0(s', a, o) (de¬ 
rived for the robot sensor models) that provides the 
probability to obtain observation o after action a which 
has lead to state s'. The robot belief state is the probabil- 
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ity distribution over all states, i. e., all the probabilities 
of being in all the states. The belief to be in a given state 
is usually noted by b(s). The belief is updated after an 
action and observation by 

b'(s') = aO(s',a,o) ^ T(s,a,s')b(s) , (14.7) 

S 

where a is a normalization constant. 

Although partial observability is a typical case for 
robots acting in mundane environments, and hence 
POMDPs are the more adequate formalism for mod¬ 


eling action and observation, POMDP-based models 
are often avoided. The reason is that calculating opti¬ 
mal POMDP policies is impracticably complex. Mostly, 
solving a POMDP is based on approximate or derived 
methods, such as working with the corresponding be¬ 
lief MDP of a POMDP, which results from considering 
the (fully observable) belief function induced by the 
POMDP. 

A good example of using Markov processes for 
modelling and solving robotics decision-making prob¬ 
lems, combining task planning, and motion planning is 
provided in [14.24]. 


14.3 Reasoning and Decision Making 


The main motivation for developing reasoning capac¬ 
ities in robotics is to enable robots to make decisions 
about their future actions. This is why planning was 
one of the main early research topics in AI, within 
the SHAKEY project resulting in the 

seminal STRIPS planner [14.25]. Designing a planning 
system requires to address three main questions: 

• How to represent the world? 

• How to represent actions? 

• How to guide the plan search process? 

According to the formalism used to answer these 
questions, the planning system will cope differently 
with the constraints of the real world, and this is an 
essential issue in robotics. As mentioned above, early 
planning techniques were based on first-order predicate 
logic (FOPL) for representing knowledge, and the limi¬ 
tation of FOPL when facing the uncertainties of the real 
world has lead to adopt probabilistic representations 
and probabilistic reasoning to cope with real situations. 

World Representations. Classically, when reasoning 
at the symbolic level, knowledge about the world is 
represented as states which are sets of logical propo¬ 
sitions such as Ontable(Cup). Under the closed world 
assumption, the facts that have the value FALSE are not 
represented in the states of the world. In order to deal 
with uncertainties, instead of having TRUE/FALSE val¬ 
ues, the propositions have a probability distribution of 
being true. 

Representation of Actions. In classical planning, as 
introduced by STRIPS, actions modify the state of the 
world and are defined by three sets of predicate lists. 
The predicates that should be true for the action to 
be feasible are the preconditions. The set of predicates 
that become TRUE after the action is executed is the 
ADD list. The set of predicates that becomes FALSE 
after action execution is the DELETE list. The plan¬ 


ner includes a unification procedure to determine which 
predicates correspond to the propositions describing the 
world states. Taking into account uncertainties in action 
execution leads to including a probabilistic characteri¬ 
zation of their outcome. 

Search. Finding the most adequate action is based 
on a search algorithm in the state space. The state space 
is an implicit graph given by the initial state and a suc¬ 
cessor function that provides the states immediately 
adjacent to a given state. Each action, which represents 
the transition from one state to its successor, has a given 
cost. The well known A* algorithm [14.26] was pro¬ 
posed in 1968, again within the SHAKEY project, to 
address the search problem. The planning algorithm, 
based on this search, recursively selects the action that 
would produce the current goal state (or a subgoal 
thereof), and under certain conditions the optimal (i. e., 
least cost) solution is found. 

This classical planning scheme is not able to cope 
with uncertainties because it is based on FOPL repre¬ 
sentations. The only way it can do that is through plan 
mending or replanning (Sect. 14.4). When probabilis¬ 
tic representations are used, the plan search will adopt 
a very different scheme as explained in Sect. 14.2.2. 

However, reasoning in AI and robotics involves 
much more than sequential planning. Robots act in the 
real world in which action duration determines suc¬ 
cess or failure. They have to consider events occurring 
at given instants and situations that unfold over time. 
Temporal relations and an appropriate temporal logic 
are necessary to represent and reason about time. Sec¬ 
tion 14.3.2 discusses temporal reasoning. 

Another central issue is reasoning about space. 
What does it mean exactly when we represent a piece 
of knowledge by the predicate Ontable(Cup), or 
Near(Robot,Table)? How to express spatial relation¬ 
ships symbolically from sensor data? This involves 
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specific formalisms as well which are developed in 
Sect. 14.3.3. 

14.3.1 Using Multiple Knowledge 
Representation Formalisms 

Automated planning carries an immediate appeal for 
robotic applications: the actions a robot should perform 
to achieve a goal are obtained as a result of reason¬ 
ing in a model which can be changed to suit different 
environments, physical capabilities, and tasks. Yet rea¬ 
soning about action per se is not a silver bullet for 
robot programming. As noted earlier, obtaining com¬ 
petent robot behavior by reasoning in a model crucially 
depends on the epistemological adequacy of the model¬ 
ing language. As a case in point, let us elaborate on our 
initial example: imagine a waiter robot with two arms 
that has to serve meals to guests in a restaurant. The 
robot would have to reason about time in order to in¬ 
fer that cold beverages should be delivered before they 
get warm. Food needs to be served in front of guests, 
hence the robot should be capable of some form of spa¬ 
tial reasoning. It should be capable of reasoning about 
resources, e.g., infer that it cannot hold more than two 
parts of the meal simultaneously with its two arms, or 
that its tray can only accommodate a limited number 
of dishes. A DL ontology might inform the robot that 
all mugs in the cupboard are clean - hence, committing 
to a particular mug at planning time is not necessary - 
or that a cup or a mug both work well for serving cof¬ 
fee. Most of this knowledge is difficult or impossible 
to model in classical planning models [14.27], which 
are limited to describing the causal relations that exist 
between actions and predicates in FOPL. 

The types of knowledge a robot should possess 
ultimately depend on the application. It should be how¬ 
ever evident that - short of encoding this knowledge in 
FOPL, which is computationally inadequate - several 
models expressed in different formalisms are necessary 
in most meaningful domains. Note also that any plan 
containing actions that are to be executed by a robot 
should also be translated to actionable metric terms that 
are understandable by the robot. For instance, specific 
positions in which objects are to be placed, locations to 
navigate to, and action dispatch times should be indi¬ 
cated in the plan. Finally, the knowledge represented in 
different models a robot should use for reasoning (the 
causal model being only one of these) often presents 
nontrivial interdependences: the capacity of the robot’s 
tray (resource model) determines the number of trips 
a robot has to perform to and from the table it is 
clearing (causal reasoning), the type of meal (ontolog¬ 
ical model) may affect the spatial layout of the cutlery 
(spatial reasoning), and the time it takes for a bever¬ 


age to get cold (temporal model) will affect the order 
of goal achievement (causal reasoning). The develop¬ 
ment of problem solving algorithms that account for 
these interdependences is an active topic of research. 
An overview of recent results in this direction is pro¬ 
vided in Sect. 14.3.4. 

Of particular interest to this discussion are tem¬ 
poral and spatial KR formalisms. Several temporal 
and spatial logics possess properties that make them 
both epistemological and computationally adequate for 
a variety of robotic problems. Below, we outline the 
principal temporal and spatial KR formalisms that are 
relevant to robotics. 

14.3.2 Reasoning About Time 

Linear temporal logic (LTL) [14.28] is a decidable 
propositional logic which provides a means to describe 
how execution paths will evolve in the future. The tem¬ 
poral operators O (next), □ (always), <) (eventually), 
'll (until) and H (release) are used to represent condi¬ 
tions on the state of a system. For instance, the robot 
will eventually place the cup on the table (reachabil¬ 
ity), the robot will never ser\’e a cold coffee (safety), 
and the robot will always reach its charging station 
before its battery is discharged (liveness). A fragment 
of LTL (specifically, syntactically co-safe LTL formu¬ 
lae [14.29]) has become relevant in robotics because of 
its ability to predicate upon discrete temporal proper¬ 
ties of motions. For example, an exploration robot may 
be required to visit locations A, B and C in a prede¬ 
termined sequential order; or conditions like avoid C 
unless A or B have been visited, and always avoid D. 
Given such temporal goal specifications, it is possible 
to compute motions that satisfy these goals with proba¬ 
bilistic roadmaps (PRMs) [14.30, 31]. 

LTL formulae express qualitative temporal speci¬ 
fications, thus offering the domain expert a means to 
specify conditions on execution at a high level of ab¬ 
straction. LTL does not however consider that robot 
actions have a temporal extent, and that the temporal 
extents of different actions may be required to have 
specific relations. The qualitative temporal logics point 
algebra (PA) [14.32] and interval algebra (IA) [14.33] 
can be used to capture (and enforce) such relations. 
Specifically, they allow to represent qualitative tem¬ 
poral relations among temporal variables. Variables in 
PA represent time points, which can be used to repre¬ 
sent events, such as the robot has reached the table, 
or start/end times of actions, such as the robot starts 
to place a mug on the table. In IA, variables represent 
time intervals, e.g., the robot navigates to the table, or 
the robot places the mug on the table. The basic PA re¬ 
lations are the three temporal relations {<,>,=}. All 
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unions of these relations (<, >, the universal re¬ 
lation T, and the empty relation 0) are also valid PA 
constraints. In IA, constraints represent temporal rela¬ 
tions among temporal intervals. The basic IA relations 
5 ia are the 13 possible temporal relations between in¬ 
tervals, namely precedes (p), meets (m), overlaps (o), 
during (d), starts (s), finishes (f), their inverses (e.g., 
p 1 ), and equals (=) (Fig. 14.5). A constraint in IA 
is a disjunction of basic relations {r \,..., r n } e Bia x 
... x Bia- Going back to our example, a relevant piece 
of temporal knowledge for our waiter robot is that the 
action of picking the mug temporally overlaps the fact 
that it is holding the mug, i. e., the IA relation 

Pick(Mug) { 0 } Holding(Mug) . ( 14 . 8 ) 

This knowledge in fact represents the qualitative state 
of affairs in the presence of a successful pick-up action, 
whereas the contingency in which the mug slips from 
the robot’s gripper (an all too common occurrence with 
today’s robots!) can be expressed by the temporal rela¬ 
tion 

Pick(Mug) {d -1 } Holding(Mug) . ( 14 . 9 ) 

A set of variables and constraints in PA or IA con¬ 
stitutes a constraint satisfaction problem (CSP) [14.34], 
A fundamental reasoning problem in temporal CSPs is 
to test the satisfiability of the CSP - assessing whether 
a substitution of temporal values to variables exists that 
satisfies all constraints. A related problem is that of 
computing the minimal representation of a given CSP, 
that is, the equivalent CSP formed by the strongest im¬ 
plied constraints. Both problems are tractable for PA 
and for several fragments of IA [14.35]. A particularly 
useful example of tractable subalgebra of IA is the set 
of convex IA relations [14.36]. For example, {b, m, 0 } is 
a convex IA relation, whereas {b, 0 } is not. These good 


computational properties can be leveraged by robots 
during plan execution. Ensuring robust execution in the 
case of our waiter robot picking a mug need not be hard¬ 
coded as an ad-hoc status checking procedure into the 
implementation of the pick up action. Conversely, it can 
be modeled as a simple relation as above, and its veri¬ 
fication can be cast as a CSP containing the relations 

Pick(Mug) { 0 } Holding] Mug) , 

Pick(Mug) {d 1 } Holding(Mug) . ( 14 . 10 ) 

The first constraint represents the requirement we ex¬ 
pect to hold in nominal execution, whereas the second 
constraint results from perception and proprioception 
(more on this later). The above CSP is not satisfiable: 
the observed and executed situations are not consistent 
with the modeled temporal relation representing suc¬ 
cessful plan execution. This determination can be made 
in polynomial time (with respect to the number of vari¬ 
ables in the CSP) with a particular form of inference, 
called path consistency [14.37]. 

Although convenient for specifying high-level re¬ 
quirements on robot plans, PA and IA cannot capture 
metric information. For this, we need metric tempo¬ 
ral constraints, which allow to predicate on durations 
and relative temporal distances between variables. An 
important metric temporal problem is the temporal con¬ 
straint satisfaction problem (TCSP). As in PA, variables 
represent time points. Constraints represent disjunc¬ 
tions of bounds on the temporal distance between the 
pairs of time points: given two time points a and b, the 
constraint 

[4,“l]V[/2,M2]V...[/„,«„] 

a -► b 

states that k<b — a<Ui, for one or more i e {1 .. .n}. 
A TCSP is satisfiable iff at least one disjunct (pair of 
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Fig. 14.5 The 13 basic qualitative relations in interval algebra. [/, u] indicate metric bounds that can be attached to the 
basic relations to refine the metric semantics of the relations 
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inequalities) per constraint holds. Unlike PA and IA, 
variables in a TCSP are associated with an explicit 
set of possible assignments of time values, called do¬ 
main. Computing the minimal representation consists 
of restricting the domains of time points to the smallest 
intervals of time containing values that satisfy all con¬ 
straints. A particular restriction of the TCSP, called the 
simple temporal problem (STP), has the property that 
domains are contiguous intervals of time. The condi¬ 
tion for remaining in the STP fragment of TCSP is that 
constraints have only one disjunct, i. e., are of the form 



Whereas computational problems for the the general 
TCSP are NP-hard, the satisfiability and minimal repre¬ 
sentation problems for the STP are tractable. Both prob¬ 
lems are solved by path consistency inference [14.38], 
and highly optimized algorithms have been developed 
for the satisfiability and minimal representation prob¬ 
lems [14.39,40]. 

The semantics of basic PA and IA can be understood 
through the use of metric temporal constraints. For in¬ 
stance, the constraint 

Pick(Mug) { 0 } Holding(Mug) (14.11) 

is equivalent to the metric constraints 

Pick(Mug)~ ——-> Holding!Mug) - , 

Holding(Mug)~ ---> Pick(Mug)"^ , (14.12) 

where (-) — and (■)+ represent, respectively, the start 
and end times of the corresponding intervals. More¬ 
over, metric bounds can be used to define a metric 
extension of IA in which we can express high-level re¬ 
lations between intervals with additional metric bounds 
(Fig. 14.5). For instance, A{p[5,13]}5 states that in¬ 
terval A should end at least 5 and at most 13 time 
units before interval B starts. In addition to binary con¬ 
straints, it is also common practice to define the two 
unary constraints Release[/, u]A and Deadline[/, u\A, 
stating, respectively, that A starts between 1 and u time 
units after the origin of time, and that A ends between / 
and u time units after the origin of time. 

Overall, constraint-based temporal calculi can rep¬ 
resent both qualitative and metric time. Provided that 
appropriate fragments of IA are used in modeling, the 
reasoning tasks can be conveniently reduced to path 
consistency inference in an STP. This is an attractive 
feature for robot reasoning: it provides a means to an¬ 
chor observations and proprioception in time. In the 


example above, the robot must infer from sensing that 
the relation 

Pick(Mug) {d~'} Holding(Mug) (14.13) 

holds. An implementation of this capability can be 
achieved by representing as time points both observed 
and planned behavior, and constraining these time 
points so as to reflect the precise times in which they 
were observed. Assuming the current time is 20, that 
the robot started to pick the mug at time 5, and that 
the gripper reports that it is holding an object between 
times 10 and 18, the following STP models the robot’s 
situation 

Pick(Mug) {o[l, oo)[l, 00 )[1 00 )} Holding(Mug) , 
Release[5, 5] Pick(Mug) , 

Deadline[20, 20] Pick(Mug) , 

ReleaseflO, 10] Holding(Mug) , 

Deadline[18, 18] Holding(Mug) . 

(14.14) 

The above STP is not satisfiable, reflecting the fact that 
execution has not gone according to plan. Again, this 
state of affairs can be ascertained in low-order poly¬ 
nomial time, hence providing a simple mechanism for 
online fault detection. 

The use of temporal constraint reasoning in plan¬ 
ning and plan execution goes beyond diagnosing fail¬ 
ures. The fact that intervals tire grounded on a metric 
temporal CSP (an STP) provides a means to maintain 
knowledge about when actions should be dispatched, 
and whether they should be delayed to accommodate 
unexpected contingencies. For instance, we can express 
that the robot should start moving back to the counter 
once the mug has been held for at least 3 s with the con¬ 
straint 

Holding(Mug) {o[3, oo)[l, oo)[l, 00 )} 

Move(Table, Counter) . (14.15) 

A delay in the start time of the Holding(Mug) 
predicate will propagate to the start time of the 
Move(Table, Counter) action, progressively pushing its 
start time into the future until the robot starts holding 
the mug. Correct propagation of exogenous events is 
ensured by maintaining in the STP the requirements 
of the plan together with constraints representing when 
events are observed, when actions are dispatched, and 
when they terminate. A simple procedure for maintain¬ 
ing these constraints is the following: 

• When an action is dispatched at time t, a Re¬ 
lease^, ?] constraint is imposed on the correspond¬ 
ing interval, constraining its start time to the current 
time. 
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• The fact that the action is still executing at time 
t + m is modeled with the constraint Deadlinefr+ 
m + 1 , 00 ), modeling the fact that the action will end 
some time in the future. 

• When the lower level executive signals at time 
t + m + n that an action has finished, the constraint 
Deadline[f + m + n,t + m + n] is added to the STP, 
thus constraining the action’s end time. 

Every time a constraint is added, the minimal STP 
is recomputed - that is, the domains of the time points 
(start and end times of actions and predicates) are up¬ 
dated. In so doing, we ensure that the lower bound of 
time points representing the start times of actions will 
always correspond to the earliest admissible time at 
which actions should be dispatched to the robot’s exec¬ 
utive layer. This guarantees that execution will uphold 
all temporal constraints modeled in the temporal CSP. 
These may include additional requirements on the plan, 
such as specifications regarding necessary orderings of 
actions, actions that must occur concurrently, and syn¬ 
chronizations with exogenous events. 

The computational adequacy of temporal CSPs en¬ 
tails that reasoning about temporal models can be 
performed online during execution. Thus, reasoning 
about time is a way to enable fault diagnosis, guarantee 
timely action dispatching, and enforce temporal specifi¬ 
cations on robot plans. Mechanisms based on temporal 
constraint reasoning have been used as a tool for this 
purpose in many settings involving execution of plans 
on real physical systems [14.41,42]. Similar techniques 
have been used to account for uncontrollable events 
in plan execution [14.43], inferring context from sen¬ 
sor traces [14.44], and integrated context inference and 
planning [14.45]. A good overview of the fundamental 
principles of temporal constraint reasoning underlying 
these results is given in Dechter’s book on constraint 
processing [14.46]. 

14.3.3 Reasoning About Space 

Spatial KR formalisms serve the purpose of specify¬ 
ing desired spatial relations in a scene. As for temporal 
models, they can be used to make sense of sensor traces, 
to enforce conditions during plan execution, as well as 
to drive the planning process itself. Most work has fo¬ 
cused on the use of qualitative spatial relations for scene 
understanding (e.g., in the context of perceptual anchor¬ 
ing [14.47]). Structural pattern recognition, important 
applications of which can be found in medical domains, 
employ cognitive vision techniques to match qualitative 
spatial knowledge (representing a specified structure) 
to perceived context [14.48,49]. In many applications, 
qualitative relations do not belong to a well-defined cal¬ 


culus, rather are tailored to capture specific features 
(e.g., distance, orientation, and shape) which are use¬ 
ful for pattern specification and recognition in particular 
applications. As is the case for temporal knowledge, 
a well-defined spatial calculus is useful because of its 
provable formal properties - e.g., tractability of specific 
reasoning problems, expressiveness of the language, 
and so on. Well-founded spatial calculi permit logical 
reasoning, and, not surprisingly, they are grounded on 
similar principles as PA and IA. We illustrate here these 
principles and show simple examples of their use in 
robotic applications. 

The main entities of interest in qualitative spatial 
calculi are objects, or, rather, the regions (or points) of 
physical space that they occupy. Spatial calculi provide 
a means to represent the relations among these regions. 
There exist several well-known and well-studied quali¬ 
tative calculi. Each of these calculi focuses on one cat¬ 
egory of spatial concepts - e.g., topology, direction and 
distance. Region connection calculus (RCC) [14.50] 
is used for representing and reasoning with topologi¬ 
cal relations. Cardinal direction calculus (CDC) [14.51] 
is an approach based on directional relations. As for 
temporal calculi, these calculi use the language of con¬ 
straints to represent spatial properties, and classical 
constraint-based reasoning techniques like path consis¬ 
tency [14.52] to ascertain consistency. 

Of particular interest here is RCC, which is grounded 
on eight spatial relations describing the connectedness 
of regions: disconnected (DC), externally connected 
(EC), tangential proper part (TPP), nontangential 
proper part (NTPP), partially overlapping (PO), equal 
(=), and the inverses TPP -1 and NTPP " -1 (Fig. 14.6). 

The full algebra deriving from the eight basic RCC 
relations is called RCC- 8 . An important restriction of 
RCC -8 is RCC-5, obtained by subsuming the relations 
DC and EC into one relation, and NTPP and TPP into 
another. The satisfiability and minimal CSP problems 
are NP-hard for both RCC -8 and RCC-5 - however, 
these problems are tractable if we restrict the language 
to the basic relations (as for IA), and a large num¬ 
ber of tractable fragments of RCC -8 and RCC-5 are 
known [14.53]. A good example of where these proper¬ 
ties are useful is in the problem of anchoring modeled 
relations to observed spatial layouts. This is exemplified 
in the work by Cohn et al. [14.54] on video sequence 
analysis, where the problem is to construct qualitative 
representations of the evolution of spatial relations in 
video sequences. The qualitative descriptions are ob¬ 
tained online, due to the computational adequacy of the 
qualitative spatial calculus used (a derivative of RCC 
called CORE-9). 

Qualitative spatial reasoning can be used, much like 
qualitative temporal reasoning, to robustify plan execu- 
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Fig. 14.6 The eight basic 
qualitative relations in RCC 


tion. We may, for instance, require that mugs should be 
placed on top of saucers (Mug NTPP Saucer). As we 
have shown for temporal reasoning, the task of verify¬ 
ing the correct state of affairs can be cast to a CSP which 
is guaranteed to be feasible iff the desired spatial layout 
is observed. 

However, note that all we can express with RCC 
constraints are topological relations between objects. 
RCC cannot express notions like left-of above, and so 
on, as the only assumption made on variables is that 
they are convex regions of space. Conversely, rectangle 
algebra (RA) [14.55] expresses more than topological 
relations - at the cost of assuming that objects have 
a particular shape. Specifically, RA is an extension 
of IA to two dimensions. Its variables thus represent 
axis-parallel rectangles, and relations are pairs of IA re¬ 
lations, one for each axis. 

RA can be augmented, like IA, with metric bounds. 
The resulting calculus, called ARA + [14.56], sub¬ 
sumes both topology and cardinal relations. For in- 



Fig. 14.7 The ARA + relation B(p[5, 13],p)A, which sub¬ 
sumes RCC relation A j DC 1 H and CDC relation A North¬ 
east of B 


stance (Fig. 14.7), the relation 

B (p[5,13], p) A (14.16) 

states that B precedes A in both the x and y axes, and that 
the distance along the x axis between A and B should 
be at least 5 and at most 13. The ARA + relation sub¬ 
sumes the qualitative relation A is Northeast of B, as 
well as the RCC relation A{DC}5. Note that rectangular 
regions are compatible with the representation of seg¬ 
mented objects in most off-the-shelf perception mod¬ 
ules. This property facilitates the process of going from 
sensor data to symbols. Toward this aim, ARA pro¬ 
vides the unary constraints At[/i, u\][l 2 , M 2 ][( 3 , M 3 ] [(4, M4] 
and Size[/i, u{\[h, U 2 HI 3 , uf][U, uf\. The first constraint 
bounds the length of the sides of a rectangle, while 
the second bounds the placement of a rectangle in 2-D 
space. Note that the At constraint performs a similar 
function to the Release and Deadline constraints used 
in the metric extension of IA, which denote specified or 
perceived absolute placements in time of intervals. It is 
thus intuitive to see how the At constraint can be im¬ 
plemented with Release and Deadline constraints in the 
augmented IA CSPs used to represent the projections of 
rectangles on the two axes. 

We can employ ARA 1 constraints to represent de¬ 
sired placements of objects, both in qualitative and 
metric terms. For instance, the specification of a well- 
set table for our waiter robot could be 

Fork (d[5, +oo)[5, + 00 ), 

d[5, + 00 )[5, + 00 )) Table , 

Knife (d[5, +oo)[5, + 00 ), 

d[5, + 00 )[5, + 00 )) Table , 

Fork (p. d) Mug , 

Mug (p, d) Knife , 

Mug Size[8, 8][8, 8] , 

Fork Size[2, 2][15, 15] , 

Knife Size[2, 2][15,15] , 


(14.17) 
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that is, forks and knives should be at least 5 cm from 
the edge of the table, forks should be located on the 
left of mugs and knives on the right, the size of forks 
and knives is 2 x 15 cm 2 , and the size of mugs is 8 x 
8 cm 2 . 

The above relations can be maintained, much like 
we have illustrated for temporal reasoning, in a lower- 
level metric constraint-based representation - in this 
case, consisting of two STPs, one for each axis of 
the reference frame. It has been shown [14.56] that 
these two STPs are satisfiable iff the AR/\ + CSP is 
satisfiable, and that the minimal STPs contain all the 
admissible placements of objects. The minimal STPs 
can thus be used to extract an admissible placement of 
the objects in the scene. Technically, this is done by 
hallucinating the objects that are to be placed in the 
scene, that is, representing both observed objects and 
those that are intended to be placed into the scene in the 
spatial CSP. In the example above, supposing the robot 
must place the mug on a table which already has a fork 
and a knife on it. the hallucinated object is the mug, 
while the fork and knife would be constrained by At 
constraints representing their observed locations. The 
bounds of all variables in the minimal STPs represent 
all the admissible placements of the objects. Those of 
the fork and knife will not have been refined in the min¬ 
imal representation, as they were already fixed by the 
At constraints resulting from observation; conversely, 
the bounds of the Mug rectangle are refined in the min¬ 
imal representation to contain only those points that 
are admissible wrt. the other constraints in the spatial 
CSP. 

So far, we have shown how employing constraint- 
based temporal and spatial KR formalisms allows to 
cast the problem of fault detection to a tractable prob¬ 
lem that can be solved online. Another important aspect 
of plan execution that is facilitated by these represen¬ 
tations is fault identification and repair [14.57]. It is 
easy to see why an explicit representation formulated 
in terms of constraints facilitates these tasks: suppose 
that the fork and knife are placed on the table cor¬ 
rectly from the qualitative point of view - i. e., the 
fork is left of the knife - but that they are only 5 cm 
apart. The resulting spatial CSP would not be satisfi¬ 
able, as it would be impossible to fit the mug (whose 
bounding box is 8 x 8 cm 2 ) between the two pieces of 
cutlery, as prescribed by constraints (Fork(p. d)Mug) 
and (Mug(p, d)Knife). The spatial CSP can be used at 
this point as a tool to perform culprit analysis: the same 
CSP without the At constraint modeling the observed 
placement of the fork represents the situation in which 
we assume that the fork can be moved; if this CSP is sat¬ 
isfiable, then we know that replacing the fork is a way 
to achieve a well-set table. 


Spatial reasoning has not received as much attention 
as temporal reasoning in the context of robotic applica¬ 
tions. The AI and Robotics research communities are, 
however, quickly clustering around this important topic. 
Spatial calculi are a well studied topic in AI, albeit with 
less applications in robotics than temporal reasoning. 
As robots become more competent in manipulation and 
navigation tasks, the issue of representing and enforc¬ 
ing high-level requirements on space begins to come 
to the fore. As always, it is the combination of epis¬ 
temological and computational adequacy of significant 
fragments of spatial KR formalisms that makes them 
useful in robot applications. Indeed, the potential of 
qualitative and metric spatial reasoning goes beyond 
planning and plan execution, with active research direc¬ 
tions in human-robot interaction [14.58-60] and object 
search [14.61] 

14.3.4 Reasoning About Multiple KR 
Formalisms 

Despite their simplicity, the examples we have used to 
illustrate the basic concepts of temporal and spatial rea¬ 
soning point to the fact that reasoning about action, 
time, space, and any other knowledge possessed by the 
robot must occur jointly. For instance, the decision to 
place the mug on the table may depend on the spatial 
layout of other objects on the table, or on whether there 
is a more urgent task to be carried out before doing so. 
It is easy to see that even this simple task cannot ig¬ 
nore kinematics and geometric constraints, which may 
call for approaching the table from a different side, or 
to move another object before placing the mug. Re¬ 
cent work in integrated task and motion planning is 
a response to the need for integrating these forms of 
reasoning into planning. Today, this is very much seen 
as the next big challenge, and consistent research results 
in this specific area of hybrid reasoning are closing the 
gap between classical planning and real robotic appli¬ 
cations [14.62-67]. 

We argue, however, that kinematics and geomet¬ 
ric constraints are only two of the important types of 
knowledge necessary for achieving competent robot be¬ 
havior. Comparatively less work exists in integrating 
other types of knowledge. DL reasoing has been em¬ 
ployed to generate high-level plans [14.68], to refine 
planning domains [14.69], and Galindo et al. [14.70] 
show how to enhance the task planning process with 
spatial and ontological models. The largest volume of 
work in integrating diverse forms of reasoning into 
planning focuses on integrating planning, temporal, 
and resource reasoning - e.g., planning with metric 
temporal reasoning [14.71], qualitative temporal mod¬ 
els [14.44], combined qualitative and metric temporal 
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reasoning [14.72], and resource scheduling [ 14.73— 
75], 

Although these advancements contribute to widen¬ 
ing the expressive power of models for robot reasoning, 
they do not suggest general methods for jointly rea¬ 
soning about heterogeneous KR formalisms. As noted 
earlier, the types of knowledge a robot should pos¬ 
sess ultimately depend on the application, which makes 
this an important basic research question. A few ex¬ 
amples pointing in the direction of general solutions 
do exist, although work in the area is sparse. These 
include planning modulo theories [14.76], an exten¬ 
sion of satisfiabiliy modulo theories (SMTs) [14.77], 
which enrich classical planning models with seman¬ 
tics belonging to arbitrary first order theories. SMTs 
themselves are an example of general hybridization 
scheme, and have been employed in a robotic con¬ 
text by Nedunuri et al. [14.78] to enforce metric re¬ 
quirements on motion and manipulation tasks. The 
approach known in the literature as meta-constraint 
reasoning is similar in structure to SMTs, but is 
grounded on the more general notion of CSP. Meta¬ 
constraint reasoning has been used to develop solvers 
for highly expressive planning domains that include 
qualitative and metric time, resources, and logical 
constraints [14.79]. In the context of robotic appli¬ 
cations, the approach has been used for online plan¬ 
ning with qualitative and metric time, spatial rela¬ 
tions in ARA~*" and resources [14.57], as well as 
for online configuration planning for multirobot sys¬ 
tems [14.80]. 

14.3.5 Knowledge Representation Systems 
for Robots 

In autonomous robot control systems, the kinds of 
information that we have discussed in the previous 
subsections are not only represented and reasoned 
about but also used subsymbolically for generating 
perception-guided activity. For example, some informa¬ 
tion such as the appearance and poses of the objects on 
the table might be generated by the perception system 
of the robot through the interpretation of sensor data. 
Other information such as the position of the robot in 
the environment might be estimated using state estima¬ 
tion algorithms and stored in dedicated data structures. 
Also, symbolic action descriptions have to be translated 
into low-level parameterizations of control routines in 
order to generate behavior and cause effects. 

These aspects of embodying abstract and symbolic 
reasoning in robots are at least partly addressed by 
system-level research on KR and reasoning for robots. 
The resulting systems serve as integrated question¬ 
answering components in a robot’s control program that 


gather information from different sources and employ 
a range of inference methods for controlling and pa¬ 
rameterizing control routines. 

Examples of recent systems include the pntology- 
based unified robot knowledge (OUR-K) frame¬ 
work [14.81], mostly used for navigation actions, the 
ORO system [14.82] that focuses on knowledge for 
human-robot interaction and dialog grounding, and 
the KnowRob system [14.83] providing deep knowl¬ 
edge for robotic manipulation tasks. The semantic 
layer [14.84] of the PEIS Ecology [14.85] serves both 
autonomous robots and ambient intelligent environ¬ 
ments. In the Ke lia project [14.86], Answer Set Pro¬ 
gramming is used as representation for integrating KR, 
natural language processing, task planning and motion 
planning on a mobile platform. Also classical cogni¬ 
tive architectures such as Soar [14.87] and ACT-R also 
integrate modules for storing knowledge [14.88] with 
inference capabilities, but have only rarely been used 
on robots [14.89]. 

One field that has seen much progress in the past 
years is the creation and representation of environ¬ 
ment models, often termed semantic maps. Advances 
in object recognition allowed to detect and percep¬ 
tually interpret single objects in environments, which 
enables robots to store higher level semantic informa¬ 
tion about them in their respective maps. The term 
semantic is used for a range of different interpretation 
capabilities - from a mere segmentation and catego¬ 
rization of the environment into different region types 
to the segmentation of objects, their categorization, 
and the interpretation of their properties [14.90], or 
from the co-occurrence of objects in scenes [14.91] 
up to environment representations in description log¬ 
ics [14.92], from statistical relational environment mod¬ 
els [14.93] to an embedding of spatial information 
into an encyclopedic and common-sense knowledge 
base [14.94]. Many of the reasoning tasks that we have 
discussed in Sect. 14.3.3 have implicitly assumed ex¬ 
pressive semantic maps that can make symbolic place 
descriptions such as on the table or in the cupboard 
effective. 

Knowledge representation aspects also become in¬ 
creasingly important for the instruction of robots with 
natural language as well as for human-robot interac¬ 
tion. For understanding directives involving objects in 
the environment [14.95] and for following spoken di¬ 
rections [14.96-99], a robot might have to ground the 
words it hears or reads into its percepts and actions. 
The natural-language text may be the result of direct 
interaction with a human, but can also be obtained from 
resources on the Web, for example, for mining object 
locality knowledge [14.100] or extracting task instruc¬ 
tions from websites [14.101] (|4a>®il>l£*liM). 
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Web-enabled robotics is another branch of robotics 
that has recently become increasingly important. Mod¬ 
ern robot control systems use the worldwide web as 
an information resource [14.102], for sharing knowl¬ 


14.4 Plan-Based Robot Control 

Plan-based control is an information processing ap¬ 
proach for controlling autonomous robots that aims at 
making robot control systems more general - in at least 
three respects. First, it enables robots to successfully 
carry out multiple, diverse, and possibly interfering 
tasks. Second, it can improve the course of action by 
adapting the control software for the respective exe¬ 
cution context. Third, plan-based control can enable 
robots to perform novel tasks without requiring the col¬ 
lection of extensive experience. 

In the plan-based approach, robots generate actions 
and behavior by synthesizing, maintaining, and exe¬ 
cuting plans, where plans are robot control programs 
that a robot cannot only execute but also reason about 
and manipulate [14.107]. Thus plan-based controllers 
can manage and adapt plans in order to better achieve 
complex and changing tasks [14.108]. The use of plans 
enables robots to flexibly interleave complex and in¬ 
teracting tasks, exploit opportunities, quickly plan their 
courses of action, and, if necessary, revise their intended 
activities. 

Plans in plan-based control have two roles. They are 

(1) executable prescriptions that can be interpreted by 
the robot to generate behavior in order to accomplish its 
jobs and (2) representations that can be synthesized and 
revised by the robot to meet the robot’s criterion of util¬ 
ity. Besides having means for representing and generat¬ 
ing plans, plan-based controllers are also equipped with 
tools that enable computational processes to (1) predict 
what might happen when a robot controller gets exe¬ 
cuted and return the result as an execution scenario, 

(2) infer what might be wrong with a robot controller 
given an execution scenario, and (3) perform complex 
revisions on robot controllers. 

In most general terms robot planning can be con¬ 
sidered to be the automatic generation, refinement, 
revision, and optimization of robot plans [14.107]. As 
a computational problem it can be formulated as fol¬ 
lows: given an abstract plan P and a description of the 
current situation, find an executable realization Q that 
maximizes some objective function V. In this problem 
formulation, an abstract plan might be to go shopping 
and to clean up the apartment, to win a robot soc¬ 
cer game, to monitor the traffic in a particular area 
using an autonomous helicopter, or, for a museum tour- 


edge and skills [14.103], for telepresence applica¬ 
tions [14.104], to employ web services [14.105], and 
to use computational resources provided through the 
web [14.106] (lo*ffiEEa). 


guide robot, to inform and entertain the visitors of the 
museum. 

Robot planning algorithms aim at finding appropri¬ 
ate plans by hypothesizing possible courses of action 
and predicting what could happen when the plans get 
executed. Based on these predictions, the robot decides 
whether the plan will meet its objectives. 

In the remainder of this subsection we will first 
discuss conceptual models underlying robot planning 
and the representation of robot plans. We will then talk 
about the automated synthesis of robot plans and finally 
discuss mechanisms for revising plans and optimizing 
plans based on experience. 

14.4.1 Models of Plan-Based Control 

The generation of plans and the prediction of what will 
happen is based on models of the robot, its actions, and 
the world. The most commonly used models are state- 
transition systems (also called discrete-event systems or 
problem spaces) [14.27]. 

Using state-transition systems, the behavior and the 
effects of robot activity can be represented as a triple 
E = ( S,A , y), where: 

• S = {«i, S 2 , ■ ■ •} is a set of states 

• A = {ai, 02 , ...} is a set of actions 

• y: S x A —S is a state-transition function. 

Thus, the action repertoire of a robot can be stated 
as a graph where the nodes represent the states of the 
environment and links of the form 

A 

Si -* Sj 

represent the fact that the state .?,• can be transformed 
into the state Sj by the robot executing action A. 

Imagine a robot has to stack three different objects, 
e.g., a cup, a saucer, and a set. In the initial state, all ob¬ 
jects are placed on a table surface. The robot can pick 
an object (provided that there is no other object on top 
of it) and can place it either on the surface or on top of 
another object (provided, again, that there is nothing al¬ 
ready on top of it). If we further assume that each object 
can be placed on all other objects, and that only a single 
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object can be moved at a time (and not a stack of two of 
them), then we get the state space depicted in Fig. 14.8. 

The state-transition model abstracts away aspects of 
robot activity such as the structure of actions and the 
events, behavior, and effects that are caused during ac¬ 
tion execution. Actions are considered to cause atomic 
state transitions. 

Even with these abstractions, a number of variations 
of planning problems are investigated. These variations 
result from different trade-offs one can make with re¬ 
spect to the computational complexity of the planning 
problem, the strength of assumptions one makes about 
the capabilities of the robot, and the realism with which 
one wants to represent the behavior of the robot and the 
effects of its actions. 

An important subclass of state-transition models are 
the ones that characterize action by specifying their pre¬ 
conditions and effects [14.25, 109, 110]. Preconditions 
state the conditions under which the respective action 
is executable and has the specified effects. The effects 
represent how the world changes when the action is ex¬ 
ecuted. 

If these models are stated as axioms in a logical rep¬ 
resentation language (Sect. 14.2) planning methods can 


be particularly powerful: they can compute sequences 
or partially ordered sets of actions that are provably 
working plans from the logical point of view. This 
means that planning algorithms can find plans that are 
guaranteed to transform any state satisfying a given 
state description into a state that satisfies a given goal, 
if such a plan exists. Unfortunately, in many robot ap¬ 
plications action models that are axiomatized this way 
are too unrealistic. 

Many extensions of the state-transition model deal 
with providing the expressive power needed to state ac¬ 
tion models more realistically. The first one extends the 
state-transition model by allowing state transitions to 
be caused by a combination of an action and an event 
(y.SxAxE —> S) rather than the action alone. This 
modification allows us to represent action failures by 
stating that a failure event occurs while the action is ex¬ 
ecuted. It also allows us to approximate environments 
that are dynamically changing. The second extension 
is to change the state transition function such that it 
maps into a subset of states rather than a single state 
(y: Sx A —> 2 s ). This modification allows the robot to 
reason through the nondeterminism that can often not 
be avoided when autonomous robots perform realis- 



Fig. 14.8 Example: problem space for stacking three objects 
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tic activities. Other researchers extend models used for 
plan-based control by including aspects that allow the 
robot to reason about its resources, geometric aspects 
of manipulation actions [14.24, 111], sensing actions, 
and so on. 

Besides the state-transition model researchers also 
use hybrid system models to reason about robot ac¬ 
tivities [14.112-114]. In the hybrid system model the 
action repertoires of robots are specified as continuous 
variable dynamical systems with a phased operation. 
Within each phase, called control mode, the system 
evolves continuously according to the dynamical law 
of that mode, called continuous flow. Thus, the state 
of the hybrid system can be thought of as a pair - 
the control mode and the continuous state. The con¬ 
trol mode identifies a flow, and the continuous flow 
identifies a position in it. Also associated with each 
control mode are so-called jump conditions, specifying 
the conditions that the discrete state and the continu¬ 
ous state together must satisfy to enable a transition to 
another control mode. The transitions can cause abrupt 
changes of the discrete as well as the continuous state. 
The jump relation specifies the valid settings of the sys¬ 
tem variables that might occur during a jump. Then, 
until the next transition, the continuous state evolves 
according to the flow identified by the new control 
mode. 

The advantage of a hybrid-system-based conceptu¬ 
alization over state-based ones is that hybrid systems 
can adequately model concurrent processes with inter¬ 
fering continuous effects. They also allow for discrete 
changes in process parameterization, which we need to 
model the activation, deactivation, and reparameteriza¬ 
tion of control processes through concurrent reactive 
plans (Sect. 14.4.2). In addition, hybrid system based 
conceptualizations can model the procedural meaning 
of waiting for and reacting to asynchronous events. 

While the increased expressive power allows us to 
model the behavior that today’s autonomous manipula¬ 
tion robots generate much more faithfully, the models 
are also in most cases too complex to completely syn¬ 
thesize action plans from primitive actions. Many of 
the modelling issues to be addressed in the intersec¬ 
tion between action planning and control are discussed 
in [14.115, 116]. 

14.4.2 Representation of Plans 

When the robot is performing its activity, it is under the 
control of a plan. The most common forms of action 
plans in autonomous robot control are: 

• State-transition plans, totally or partially ordered 

sets of atomic actions. 


• Policies, mappings from (perceived) environment 

states to actions. 

• Reactive plans that specify how the robot is to re¬ 
spond to sensory events and execution failures in 

order to accomplish its jobs. 

State-Transition Plans 

Robot action plans that are generated according to 
the state-transition model are ordered sets of actions, 
often called plan steps. The assumptions of the state- 
transition model, in particular that plan steps are atomic 
and their execution can be abstracted away, are not sat¬ 
isfied in many robot applications. To execute plan steps 
successfully, actions have to include subactions such as 
detecting the object to be fetched in a scene or detect¬ 
ing that the object has been successfully grasped before 
lifting the arm. As autonomous robot activity is far from 
perfect, the robot might not be able to generate the nec¬ 
essary execution events or it can generate the events but 
fails to achieve the desired effects. For example, an ob¬ 
ject might slip out of the robot’s hand after grasping 
it. 

As a consequence, state-transition plans can of¬ 
ten only serve as guidelines for generating robust and 
flexible robot behavior [14.117]. To close the gap to 
low-level robot control, researchers have proposed mul¬ 
tilayer software architectures for autonomous robot ac¬ 
tivity, in particular 3T (3-tiered) architectures [14.118] 
(Chap. 12). 3T architectures run planning and exe¬ 
cution at different software layers and different time 
scales where a sequencing layer synchronizes between 
both layers. Each layer uses a different form of plan 
or behavior specification language. The planning layer 
typically uses a problem space plan, whereas the execu¬ 
tion layer employs feedback control routines that can be 
activated and deactivated. The intermediate layer typi¬ 
cally uses a simple form of a reactive plan language (see 
below). Even though 3T architectures have been shown 
to produce competent activity in some application do¬ 
mains, there remain important issues to be addressed for 
more complex applications. One of them is that the be¬ 
havior generated by the sequencing layer might deviate 
substantially from the action model used for planning, 
which might cause plans not to work as expected by the 
action models used by the planning component. As the 
planning layer abstracts away from control structures, it 
also lacks the expressiveness to specify robust and flex¬ 
ible behavior. 

MDP Policies 

A second category of plans are policies, which are map¬ 
pings from discretized robot states into robot actions. 
Such policies are computed as solutions of Markov de¬ 
cision problems (Sect. 14.2.2). The policies computed 
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by MDPs aim at robustness and optimizing the aver¬ 
age performance. They are less appropriate when the 
consequences of decisions are not in the near future. 
The decision of how to grasp an object might con¬ 
siderably affect the ease with which the object can 
be placed later in the activity episode. Extensions of 
MDP policies such as the combination of programming 
and reinforcement learning [14.1 19, 120] and the struc¬ 
turing of policies with so-called options [14.121, 122] 
partly address these issues. 

Concurrent Reactive Plan Languages 
Concurrent reactive plan languages are the result of 
applying programming language design methods to 
the specification of robot behavior. Concurrent reactive 
plans specify how the robot is to respond to events, 
including perception and failure events, in order to 
achieve its goals [14.123]. Such plan languages offer 
rich sets of control structures that support the struc¬ 
turing and transparent synchronization and monitoring 
of concurrent control processes, the detection and re¬ 
covery from execution failures, the context-dependent 
execution of subplans, the prompt reaction to asyn¬ 
chronous events, and data structures for the descriptions 
of objects to be manipulated [14.124, 125]. Some re¬ 
active plan languages also incorporate explicit repre¬ 
sentations of the beliefs, goals, and intentions of the 
robot [14.126-128] and even provide powerful rea¬ 
soning mechanisms to adapt the plan execution to the 
respective situation context [14.129]. 

While concurrent reactive plan languages enable 
robots to accomplish their tasks competently, they are 
too expressive for automatically generating robust, flex¬ 
ible high-performance plans from first principles. They 
typically require the use of plan libraries and transfor¬ 
mational planning techniques to deal with the compu¬ 
tational complexity caused by the expressiveness of the 
plan languages (Sect. 14.4.4). 

14.4.3 Generating Robot Action Plans 

The most common form of reasoning performed on 
plans is the synthesis of plans for given problems. In 
general, planning is concerned with reasoning about in¬ 
tended courses of action before executing them in order 
to avoid failures in achieving given goals, to prevent un¬ 
desired side effects, or to improve the performance of 
the robot in accomplishing its goals. 

There are many different variants of planning 
problems, categories of plans that can be gener¬ 
ated, and methods that can be applied to planning 
problems. Comprehensive overviews can be found in 
textbooks [14.4,27] and overview articles [14.130, 
131]. A wide range of the planning problems is ad¬ 


dressed within an international AI planning competition 
(IPC) [14.132] using standardized problem specifica¬ 
tion languages [14.109, 110]. Several implementations 
of planning algorithms participating in this competition 
are available as open-source software. 

In planning for state-transition, the task of the plan¬ 
ning system is to find an ordered set of actions that 
transform any state satisfying the initial state descrip¬ 
tion into a state satisfying the given goals. 

The first class of planning systems are systems that 
enumerate the states that can be reached by prefixes (or 
suffixes) of hypothetical action sequences until a goal 
state is reached. Planning systems that fall into this cat¬ 
egory include the STRIPS planning algorithm [14.25] 
and newer and more efficient algorithms such as the FF 
(fast forward) planning algorithm [14.133] as well as 
various variants of these algorithms. 

Another class of algorithms enumerates abstract 
plans that represent sets of plans. The idea underly¬ 
ing this approach is that by reasoning about an abstract 
plan, the planning algorithm can compactly reason 
about different plan instances at the same time by 
reasoning about their common causal structure. For ex¬ 
ample, a partial order plan represents all possible com¬ 
pletions of the plan. The POP (partial-order planning) 
algorithms is a family of algorithms that is primar¬ 
ily used for teaching action planning [14.130] while 
GraphPlan [14.134] is an algorithm that is designed for 
planning process efficiency [14.131]. 

The algorithms above are domain independent: 
they compute plans by analyzing whether precondi¬ 
tions of plan steps are achieved through the effects 
of prior actions using general algorithms. In contrast, 
domain-dependent planners use domain knowledge to 
constrain and prune the search space to be explored 
in a planning process. An example of such domain 
knowledge is that stacks of objects are built bot¬ 
tom up. Domain-dependent planners are often much 
more efficient than their domain-independent coun¬ 
terparts. Examples of domain-dependent planners are 
the TL- (temporal logic) [14.135] and the TAL plan¬ 
ners [14.19]. 

Another category of planners that are designed to 
deal with more complex planning problems are HTN 
(hierarchical task network) planners. HTN planners are 
equipped with libraries of plan schemata that specify 
how abstract plan steps can be decomposed into more 
complex subplans. Examples of such planners are the 
SHOP planners [14.136] and extensions thereof that are 
combined with description logics and applied to au¬ 
tonomous robot control [14.69]. 

The planning approaches above generate plans that 
have similar expressiveness in terms of the behavior 
they can specify. The planners mainly differ with re- 
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spect to the size of the planning problems they can 
handle and their efficiency in solving planning prob¬ 
lems. 

14.4.4 Revising and Improving Plans 

Another dimension that robot action planning meth¬ 
ods can be scaled along is the expressiveness of the 
plans they generate in terms of specifying competent 
robot behavior. A subclass of planning methods that 
is suitable for generating concurrent reactive plans is 
transformational planning. 

The underlying idea of transformational planning is 
to paste together plans from libraries of plan schemata 
and revise them for the specified task [14.137] rather 
than creating completely new plans from scratch. Trans¬ 
formational planning has also been referred to as plan 
repair or plan debugging [14.138, 139] and is often 
performed within a generate-test-debug control strat¬ 
egy [14.140]. Plan transformation is a promising ap¬ 
proach in applications where default plans that have 
a substantial chance of being successful can be con¬ 
structed easily, tested by projecting them, analyzed with 
respect to their expected execution failures, and where 
failures can be used to index the appropriate plan revi¬ 
sions. 

Transformational planning subsumes classical plan¬ 
ning methods such as partial-order planning in the 
sense that the operations of the POP algorithms can 
be simply expressed as transformations. In addition to 
the transformations that correspond to classical plan¬ 
ning, transformational planning typically employs col¬ 
lections of transformations including domain-specific 
ones, heuristic transformations, and transformations for 
plan optimization [14.141]. In cases where the planners 
employ transformations that cannot be guaranteed to 
work, they rely on powerful plan projection methods 
for testing the generated plan hypotheses [14.142]. 

Transformational planning is a promising approach 
for autonomous robots performing complex manipula¬ 
tion tasks because it makes it easy for programmers to 
specify how plans should be modified to avoid certain 
expected behavior flaws. For example, a transformation 
could tell the robot to stack items in order to transport 
them more efficiently subject to additional constraints 
such as that it should not stack too many items because 
it might drop them. Or, it might stack one kind of object 
on another one but not vice versa - typically it is safe to 
stack cups on plates but not plates on cups. 

Transformational planning can also effectively fa¬ 
cilitate the learning not only of plans [14.139] but also 
of planning capabilities by learning for a particular 
robot and environment what the robot can stack and 
carry and what not. 


14.4.5 Other Reasoning Methods 

There are a number of other reasoning methods that 
plan-based robot control needs to employ in order 
to produce competent object manipulation capabili¬ 
ties [14.143]. One of the main reasons is that the tasks 
and subtasks that a robot is to perform are typically 
underspecified: the robot might be asked to fetch the 
screwdriver from the table but when it gets there are 
several different screwdrivers. Or, the robot might be 
asked to crack an egg and has to decide where to crack 
it, where to put the yolk and egg white, what to do with 
the shell, etc. When fetching objects, it has to reason 
about abstract states such as visibility and reachabil¬ 
ity of objects. We can only give a couple of examples 
here. 

The first reasoning problem is that of object iden¬ 
tity resolution, the inference task of deciding whether 
two internal representations refer to the same object 
in the real world. When receiving the command to 
pick up the red cup from the table, the robot must 
make at some point the decision that a data structure 
returned by the robot’s perception system and the lan¬ 
guage description refer to the same object. Only by 
making this inference, the robot is able to infer that 
it can achieve the given task by picking up a partic¬ 
ular object that it sees. In logic-based representations 
this problem is even harder because symbols used in 
the planning problem specification denote objects in the 
outside world. In this setting the problem of object iden¬ 
tity resolution problem becomes the symbol grounding 
problem [14.1], i. e., the problem to determine the right 
or, at least, the most likely object in a given environ¬ 
ment for each constant and each definite description in 
the logical representation. The most powerful inference 
methods to reason about object identity resolution are 
first-order probabilistic reasoning mechanisms as de¬ 
scribed in [14.144]. 

Other important inference methods are needed to 
model some predicates used in the activity models 
more realistically. Predicates that deserve such spe¬ 
cial treatment include visibility and reachability. For 
example, the Asymov planner [14.111] reasons about 
action preconditions that include reachability condi¬ 
tions and tests reachability through the existence of 
motion plans. Other planning systems reason about the 
next best views that a robot should take in order to 
perform complex perception tasks. Sophisticated ac¬ 
tion planners that include action and motion planning 
employing more general geometric reasoning capabil¬ 
ities can be found in [14.66]. Cognition-enabled plans 
employ such reasoning methods in order to parameter¬ 
ize actions with respect to the effects that should be 
achieved and avoided [14.62], 
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The aim of this chapter was, first, to motivate the 
purpose and benefit of employing general reasoning 
methods in robot control; second, to sketch some of 
the generic reasoning methods from AI that are well- 
understood, available, and most likely of use on a robot; 
and third, to go into plan-based robot control in some 
more detail, as it is a particularly obvious form of 
employing reasoning to the end of improving robot 
action. 

Yet, the chapter has only sketched the tip of an ice¬ 
berg. Employing symbolic reasoning in an autonomous 
mobile robot requires to have and update on the robot 
a symbolic model of the world, or at least of that part 
of the world which is relevant for the current task - 
but who knows beforehand exactly what is and what 
isn’t relevant for some task? It therefore requires to be 
able to interpret the current sensor data stream on a se¬ 
mantic level to transform it into a symbolic description 
in terms of categories that are meaningful in the KR. 
Assuming that it is not practical to equip a robot with 
all knowledge that may likely be relevant for each and 
every situation it may find itself in, it requires some 
form of learning, generalization, and/or the ability to 
look up unknown knowledge from available sources. 
These and some more requirements that are needed 
for practically using AI reasoning methods on a robot 
cannot be fulfilled completely based on current state-of- 
the-art AI methods and technology; as a consequence, 
currently existing robot prototypes employing AI rea¬ 
soning methods in their control systems are limited in 
terms of generality and adaptability. 

It is no recent insight that employing AI reasoning 
methods on robots is both a chance and a challenge. 
In fact some of the early influential work in AI was 
driven by the idea of designing integrated reasoning 
robots, such as SHAKEY [14.145] as 


a consequence, early AI textbooks like [14.146] would 
naturally include robotics as a target field for using AI 
reasoning methods. The field of AI has since focused 
on applications other than Robotics. Recent efforts in 
integrating reasoning systems with autonomous robots 
have done much to bring the original robot-inspired 
vision back into focus, as witnessed by series of work¬ 
shops on AI-Based Robotics (e.g., IROS 2013) or AI 
and Robotics (AAAI 2014). It is exactly the family of 
challenges and chances that this chapter is about that is 
discussed at these events. 

14.5.1 Further Reading 

For a more complete coverage of the generic meth¬ 
ods described here, as well as the rest of AI, we 
refer to AI textbooks, of which the one by Russell 
and Norvig [14.4] is comprehensive and currently most 
widely used. Among the sources for first-hand mate¬ 
rial, let us mention the two journals Artificial Intelli¬ 
gence and Journal of Artificial Intelligence Research 
(JAIR) [14.147], which both cover well the topics ad¬ 
dressed here. The main international conference in the 
field is IJCAI ( International Joint Conference on Ar¬ 
tificial Intelligence)', ECAI ( European Conference on 
Artificial Intelligence [14.148]) and AAAI (Association 
for the Advancement of Artificial Intelligence Con¬ 
ference on Artificial Intelligence [14.149]) are other 
major international symposia on Artificial Intelligence 
in general. Regarding planning in particular, ICAPS 
(International Conference on Automated Planning and 
Scheduling [14.150]) is the main international confer¬ 
ence. 

Reference [14.151] reviews about the same topic as 
this chapter, but structures it differently along the notion 
of deliberation functions. 


Video-References 


SHAKEY: Experimentation in robot learning and planning ( 1969 ) 

available from http://handbookofrobotics.org/view-chapter/ 14 /videodetails /704 

From knowledge grounding to dialogue processing 

available from http://handbookofrobotics.org/view-chapter/ 14 /videodetails /705 
ItsswiiimKiU RoboEarth final demonstrator 

available from http://handbookofrobotics.org/view-chapter/ 14 /videodetails /706 
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15. Robot Learning 


Jan Peters, Daniel D. Lee, Jens Kober, Duy Nguyen-Tuong, J. Andrew Bagnell, Stefan Schaal 


Machine learning offers to robotics a framework 
and set of tools for the design of sophisticated 
and hard-to-engineer behaviors; conversely, the 
challenges of robotic problems provide both inspi¬ 
ration, impact, and validation for developments in 
robot learning. The relationship between disci¬ 
plines has sufficient promise to be likened to that 
between physics and mathematics. In this chapter, 
we attempt to strengthen the links between the 
two research communities by providing a survey 
of work in robot learning for learning control and 
behavior generation in robots. We highlight both 
key challenges in robot learning as well as notable 
successes. We discuss how contributions tamed the 
complexity of the domain and study the role of al¬ 
gorithms, representations, and prior knowledge 
in achieving these successes. As a result, a partic¬ 
ular focus of our chapter lies on model learning 
for control and robot reinforcement learning. We 
demonstrate how machine learning approaches 
may be profitably applied, and we note through¬ 
out open questions and the tremendous potential 
for future research. 
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In the future, robots will be part of daily life in hu¬ 
man society, providing assistance in many areas ranging 
from clinical applications, education and care giving, 
to normal household environments, disaster scenarios, 
and space exploration [15.1]. Robots will no longer 
be used to only execute the same task thousands of 
times, but rather they will be faced with thousands of 
different tasks that rarely repeat in an ever changing en¬ 
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vironment [15.2]. It will be difficult to preprogram all 
possible tasks and scenarios in such robots, and robots 
will need to be able to leam by themselves or with the 
help of humans. They will need to automatically adjust 
to stochastic and dynamic environments, as well as to 
changes due to wear and tear. Machine learning will be 
necessary to achieve this high degree of autonomy and 
enabling future robots to perceive and act in the world. 
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15.1 What Is Robot Learning 

Robot learning consists of a multitude of machine 
learning approaches in the context of robotics. The type 
of the learning problem is usually characterized by the 
type of feedback, the process of data generation, and 
the type of data. At the same time, the type of data 
will determine the robot learning approach which actu¬ 
ally can be employed. While the field of robot learning 
includes a huge number of topics ranging from learn¬ 
ing in perception, state abstraction, decision making, 
and probabilistic techniques in robotics [15.3-5], this 
chapter focuses on issues of learning action generation 
and control. As indicated in Fig. 15.1, we explain robot 
learning using the two key ingredients: (i) the setting 
in terms of data and feedback, and (ii) the learning 
approach. 

15.1.1 Robot Learning Approaches 

In this chapter, we will focus on the core robot learn¬ 
ing approaches capable of learning action generation 


and control. Particularly, we will be focusing on tech¬ 
niques of model learning and reinforcement learn¬ 
ing (RL). (Note that we omit imitation learning by 
behavioral cloning as Chap. 75 deals with it exten¬ 
sively.) These methods have enabled real-time learning 
in complex robot systems such as helicopters, legged 
robots, anthropomorphic arms, and humanoid robots. 
We first discuss some broader issues of learning con¬ 
trol before reviewing technical details and the related 
literature. 

Learning Models for Control 
Models are among the most essential tools in robotics, 
and it is widely believed that intelligent mammals 
also rely on internal models in order to generate their 
actions. A model describes essential predictive infor¬ 
mation about the behavior of the environment and the 
influence of an agent on this environment. While classi¬ 
cal robotics relies on manually generated models based 
on human insights into physics, future autonomous 
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Fig. 15.1 The main branches of robot learning for action generation and control can be characterized both in terms of 
the available data T> and the employed approach that learns from the data. A key distinction for data sets is whether 
they were generated from observed expert’s demonstration or the robot’s embodied experience. Data sets generically 
include samples i that consist at least of states s,-, actions and next states x,+i. However, in many cases, a qual¬ 
ity assessment in form of a reward r, is available as feedback. The branches in the tree illustrate the core robot 
learning problems considered here. Model learning generate a model T(x/+i,s,,a,) from data. Reinforcement learn¬ 
ing approaches require a reward in order to obtain an optimal policy n*. Model-based methods usually learn a model 
from which they subsequently construct an optimal value function and policy. Model-free methods either learn value 
functions from data without model knowledge to construct a policy while policy search methods optimize the pol¬ 
icy directly based on the interaction data. Expert data allows for reproducing observed behavior jr E . Two branches 
have had a particular importance in robot learning: imitation learning by behavioral cloning can treated by supervised 
learning and is discussed in detail in Chap. 74. Imitation learning by inverse RL reconstructs the reward function of 
the expert and uses reinforcement learning approaches to re-construct the expert’s policy (This figure was inspired 
by [15.6]) 
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robots need to be able to automatically generate mod¬ 
els from information which is extracted from the data 
streams accessible to the robot. Machine learning ap¬ 
proaches in generating these models will also need to 
be adapted to the control architectures for which they 
are intended. Such model learning for control will be 
discussed in Sect. 15.2. 

Planning with Reinforcement Learning 
Planning motor behaviors and optimizing them with 
respect to some criteria is part of every motor skill. 
Reinforcement learning offers one of the most gen¬ 
eral frameworks to address these issues. In essence, 
it assumes that the goal or intent of a movement is 
represented in terms of an optimization or reward cri¬ 
terion, such that learning algorithms can fill in the 
often nontrivial details of the plan. While RL has 
wide area of application in many domains ranging 
from computer games to elevator scheduling [15.7], 
its application to high-dimensional physical systems 
with continuous state-action spaces requires often dedi¬ 
cated approaches, algorithms, representations, and prior 
knowledge. An important issue will also be the choice 
between model-based and model-free algorithms, as 
well as between value function-based and policy search 
methods. Reinforcement learning’s different branches 
are indicated in Fig. 15.1 and will be discussed in 
Sect. 15.3. 

Learning from Demonstration 
with Inverse Reinforcement Learning 
Learning a skill starting from scratch with a complex 
robot system is often infeasible or extremely time con¬ 
suming. Taking inspiration from child teaching tech¬ 
niques, learning from demonstration has become a very 
popular tool to endow a robot with some initial rea¬ 
sonable performance, from which further learning can 
progress more rapidly. As another chapter in this hand¬ 
book will address learning from demonstration in more 
detail, we also discuss a particular subclass of learn¬ 
ing from demonstration, i. e., using inverse RL (the two 
most relevant classes of imitation learning are indi¬ 
cated in Fig. 15.1). Inverse RL addresses how to extract 
a suitable cost function from observed behavior, an idea 
going back to Kalman [15.8]. Algorithms for inverse 
RL are tightly coupled to RL which we can only briefly 
discuss in Sect. 15.3.6, Curse of Goal Specification, but 
they are also discussed in Sect. 75.4.3. 

As learning has found many applications in 
robotics, this chapter can only scratch the surface and 
extensions of several sections can be found in [15.9, 
10]. By focusing on the key approaches for teaching 
a robot new abilities, we nevertheless hope to give 
a concise overview. 


15.1.2 Robot Learning Settings 

The setting of robot learning methods is determined by 
the data. Data can either be obtained from an expert 
or generated by interaction with the system. Often data 
needs to be reproduced accurately (as e.g., for model 
learning or imitation learning by behavioral cloning) 
while in other cases a feedback signal can determine 
what data is actually relevant for the system (directly in 
RL and somewhat more indirectly in imitation learning 
by inverse RL). Also, the type of data generation will 
determine what robot learning methods are applicable 
as indicated in Fig. 15.1. For example, while expert data 
is crucial for imitation, it may simply be the wrong data 
to learn from in model learning. 

Data for Robot Learning 

In comparison to many other machine learning do¬ 
mains, robot learning suffers from a variety of complex 
real-world data problems. Real-world training time is 
limited - hence, only relatively few complete execu¬ 
tions of a task can ever be generated. These episodes 
are frequently perceived noisily, have a fair amount of 
variability in the outcome, do not cover all possible sce¬ 
narios and most often do not include all reactions to 
external stimuli. While episodes are scarce, the sam¬ 
ples executed during a trial are not (but are often 
not helpful as they contain less information for some 
types of learning). Instead, high-dimensional data is 
obtained at a fast rate (e.g., proprioceptive informa¬ 
tion at 500 Hz to 5 kHz, vision at 30—200 Hz, tac¬ 
tile at 50 Hz to 5 kHz depending on the measurement 
mode). The data the robot can learn from is usu¬ 
ally generated by the imperfect controller that needs 
to be improved. Thus, generating interesting data that 
has sufficiently useful information is a recurring prob¬ 
lem in robot learning, discussed under various names, 
like exploration/exploitation tradeoff, persistent exci¬ 
tation, etc. Last not least, for high-dimensional robot 
systems (like humanoids), there is no representative 
training data set to learn from. A small change in 
posture of a robot can radically change in which part 
of the workspace the robot operates. Rapidly learning 
on the fly is frequently more important than squeez¬ 
ing the most of information from a few data points. 
Sharing data across different robots would be desir¬ 
able, envisioning a cloud robotics approach. However, 
so far, even various instantiations of the same robot 
model can create quite different data depending on their 
wear and tear, calibration, and maintenance. Sharing 
data across different robot models has not seen a lot 
of success, as kinematic and dynamic differences can 
lead to rather different control and planning realiza¬ 
tions. Thus, machine learning techniques for robotics 
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become quickly rather specialized and domain/robotic 
specific. 

What Feedback Is Available for Learning? 

A straightforward way to categorize robot learning ap¬ 
proaches is given by the type of feedback [15.11]. 
A scalar performance score such as a reward or cost 
will often result in a RL approach. A presented de¬ 
sired action or predicted behavior allow supervised 
learning approaches such as model learning or direct 
imitation learning. Feedback in terms of an explana¬ 
tion has become most prominent in inverse RL. These 
methods will be explained in more detail below. Note 


that unsupervised learning problems where no feed¬ 
back is required also can be found in robotics [15.5, 
12 ]. 

15.1.3 Notation 

In the next section and in Fig. 15.1, we will use the fol¬ 
lowing notation. In general, we follow the tradition in 
RL [15.13]. When it comes to specific control prob¬ 
lems, we use q for a vector of states in configuration 
space (often called joint-space), u for motor commands 
in configuration space, and x for a state in task- or 
effector-space. 


15.2 Model Learning 


In recent years, methods to learn models from data 
have become increasingly interesting tools for robotics. 
There are a variety of reasons for this development. 
First, robot control - as needed in less structured dy¬ 
namic and more stochastic environments - requires 
model-based control to realize high compliance, since 
stiff control without good models exposes the robot and 
its environment to more risk of damage. Second, due 
to the complexity of modem robot systems, standard 
models like rigid body dynamics are only rough ap¬ 
proximations which do not accurately model unknown 
sources of nonlinearities from actuation (hydraulics 
and motor saturation), passive elements like wires, hy¬ 
draulic hoses, cable stretch, and/or sources of friction 
and stiction. Finally, interaction with an unstructured, 
uncertain, and human-inhabited environment may re¬ 
quire to leam models on the fly [15.14, 15]. In addition, 
wear and tear of a robot in continual use will create 
drifting models. All these issues can be addressed by 


learning models directly from measured data, ideally in 
a continual and incremental on-line fashion, using ap¬ 
propriate nonlinear function approximation techniques. 

We discuss different types of models (forward mod¬ 
els, inverse models, mixed models, and operator mod¬ 
els) in Sect. 15.2.1, along with difference architectures 
for model learning in Sect. 15.2.2. Some of the chal¬ 
lenges that arise in these models in the domain of 
robotics are discussed in Sect. 15.2.3. How models 
can be learned using machine learning techniques are 
described in Sect. 15.2.4, and examples where model 
learning has proven to be helpful for the action gen¬ 
eration in complex robot systems [15.14,16-21] are 
highlighted in Sect. 15.2.5. 

While this chapter attempts to give a fairly exhaus¬ 
tive overview on model learning in robotics, it is of 
course not complete due to the space constraints of 
a book chapter. An extended version of this section can 
be found in [15.9], 



Fig.15.2a-d Graphical models for different types of models. The white nodes denote the observed quantities, while the 
brown nodes represent the quantities to be inferred, (a) The forward model allows inferring the next state given current 
state and action. (b) The inverse model determines the action required to move the system from the current state to the 
next state, (c) The mixed model approach combines forward and inverse models in problems where a unique inverse does 
not exist. Here, the forward and inverse models are linked by a latent variable Zk- (d) The operator model is needed when 
dealing with finite sequences of future states 
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15.2.1 Model Types 


be written as 


Models represent the functional relationship between 
input and output variables. If we can observe the cur¬ 
rent state Sk of a system and the action «/ currently 
applied to the system, we can attempt to predict the next 
state Sk- pi, which is called a forward model with map¬ 
ping (Sk,Ok) —> Sk+i. We can also use the inverse model 
to infer the current action, i. e., the relation (s^, Sk+ 1 ) —»■ 
ak, if we know the current state and the desired or 
expected future state. There are also approaches com¬ 
bining forward and inverse models for prediction which 
we will refer to as mixed model approaches. However, 
for many applications the system behavior has to be pre¬ 
dicted for the next /-steps rather than for the next single 
step; models that predict a series of states will be called 
operator models. Graphical depictions of these different 
models are shown in Fig. 15.2. 

Forward Models 

Forward models predict the next state of a dynamic 
system given the current action and current state. The 
forward model directly corresponds to the state transfer 

$k-\- ! —/forward (&k'O-l) • (15.1) 

This function expresses the causal physical proper¬ 
ties of the system, and the learning of such causal 
mappings is a well-defined problem using standard re¬ 
gression techniques. While forward models are unique 
mappings in classical physics, there are several cases 
where forward models alone do not provide sufficient 
information to uniquely determine the next system’s 
state [15.22], For instance, when a pendulum is lo¬ 
cated at its unstable equilibrium point, it will tend to go 
either to the left or right rather than remain in the cen¬ 
ter. Nevertheless, the prediction of the forward model 
would be for the pendulum to remain centered. In this 
case, the state transfer function is not sufficient to cap¬ 
ture the interesting dynamics of the system [15.22,23]. 
For such cases, a probabilistic model T(sk+\,cik,Sk) = 
P(Sk+i\sk,ak) may be more appropriate as it captures 
both likely modes with a conditional density instead of 
just the mean behavior. 

An early application of forward models in classical 
control is the Smith predictor, where the forward model 
is employed to cancel out delays imposed by the feed¬ 
back loop [15.24]. Subsequently, forward models have 
been applied in the context of model reference adaptive 
control (MRAC) [15.25]. MRAC is a control system in 
which the performance of an action is predicted using 
a forward reference model. The controller adjusts the 
action based on the resulting error between the desired 
and current state. Hence, the policy n for MRAC can 


7T(s) = argmin (/forward(*.«) -*des) , (15.2) 

a 

where Sd es denotes a desired trajectory and s represents 
the observed state. MRAC was originally developed for 
continuous-time system and extended later for discrete 
and stochastic systems [15.25]. Applications of MRAC 
can be widely found for robot control, such as adaptive 
manipulator control [15.26]. 

Further applications of forward models can be 
found in the wide class of model predictive control 
(MPC) [15.27]. MPC computes optimal actions by min¬ 
imizing a given cost function over a certain prediction 
horizon N. The MPC control policy can be described by 




t+N 

argmin E F cost (/forward i$k■ ®k ). $k, des> ) > 

k=t 

(15.3) 


where F C0St (s, s des ) denotes the cost of an action a* re¬ 
sulting in a state s away from the desired state s^es- 
MPC was first developed for linear system models 
and, subsequently, extended to more complex nonlin¬ 
ear models with constraints [15.27], Forward models 
have also been essential in model-based RL approaches, 
which relate to the problem of optimal control [15.28- 
30]. Here, the forward models describe the transition 
dynamics determining the probability of reaching the 
next state given current state and action. In contrast to 
previous applications, the forward models incorporate 
a probabilistic description of the system dynamics in 
this case [15.31,32], Example applications of forward 
models for optimal control are given in the case studies 
in Sect. 15.2.5, Simulation-Based Optimization. 

Inverse Models 

Inverse models predict the action required to move the 
system from the current state to a desired future state. 
In contrast to forward models, inverse models represent 
an anticausal system. Thus, inverse models may not ex¬ 
ist or be well defined. However, for several cases, such 
as for inverse dynamics of nonoveractuated robots, the 
inverse relationship is well-defined. Ill-posed inverse 
modeling problems can be solved by introducing ad¬ 
ditional constraints, as discussed in Sect. 15.2.1 , Mixed 
Models. 

For control, applications of inverse models can be 
traditionally found in computed torque robot control, 
where the inverse dynamics model predicts the required 
torques, i. e., the action, to move the robot along a de¬ 
sired joint space trajectory. Thus, the computed torque 
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control policy can be described by 

—./inverse■ $des) T k(s -Sdes) 1 (15.4) 

where the function k(s — Sd es ) is an error correction 
term (e.g., a proportional-derivative (PD)-controller 
since both positions and velocities are normally part 
of the state in a second order dynamical system) 
needed for stabilization of the robot. If an accu¬ 
rate inverse dynamics model is given, the predicted 
torques are sufficient to obtain a precise tracking 
performance. The inverse dynamics control approach 
is closely related to the computed torque control 
method. 

Here, the feedback term acts through the inverse 
model of the system giving a control policy, 

—/inverse [s.Sdes T k(s Sdes)] • (15.5) 

If the inverse model perfectly describes the inverse 
dynamics, inverse dynamics control will perfectly com¬ 
pensate for all nonlinearities occurring in the system. 
Control approaches based on inverse models are well- 
known in the robotics community, and have gained 
increasing popularity since the late 1980s, as the in¬ 
crease in computational power and the existence of 
torque-controlled robots have allowed computing more 
complex models for real-time control. The concept of 
feedback linearization is another, more general way to 
derive inverse dynamics control laws, offering possibly 
more applications for learned models [15.33, 34]. 

Mixed Models 

In addition to forward and inverse models, there are 
also methods combining both types of models. Since 
modeling the forward relationship is well defined while 
modeling the inverse relation may be an ill-posed prob¬ 
lem, the basic idea behind the combined mixed model 
is that the information encoded in the forward model 
can help to resolve the nonuniqueness of the inverse 
model. A typical ill-posed inverse modeling problem 
is the inverse kinematics of redundant robots. Given 
a joint configuration q, the task space position* = f(q) 
can be determined exactly (i. e., the forward model is 
well defined), while there may be many possible joint 
configurations q for a given task space position *, i. e., 
the inverse model could have infinitely many solutions 
and deriving an average suitable solution is not straight¬ 
forward. The problem is that the set of inverse solutions 
may not form a convex set. Thus, when learning such 
inverse mappings, the learning algorithm will poten¬ 
tially average over the nonconvex set, giving rise to an 
invalid solution. 

The ill-posedness of the inverse model can be re¬ 
solved if it is combined with the prediction of a forward 


model, which checks whether the proposed inverse so¬ 
lution results in the desired state [15.35]. If not, the 
remaining error can be used to adjust the learned in¬ 
verse model such that it chooses a valid solution. 

The mixed model approach, i. e., the composite 
of forward and inverse models, was first proposed 
in conjunction with the distal teacher learning ap¬ 
proach [15.35], and discussed in Sect. 15.2.2, Distal 
Teacher Learning. The proposed mixed models ap¬ 
proach has subsequently evoked significant interest and 
has been extensively studied in the field of motor 
control [15.36,37]. Furthermore, the mixed model ap¬ 
proach is supported by some evidence that the human 
cerebellum can be modeled using forward-inverse com¬ 
posite models, such as MOSAIC [15.38,39]. While 
mixed models have become well known in the neuro¬ 
science community, the application of such models in 
robot control is not yet widespread. Pioneering work 
on mixed models in the control community can be 
found in [15.40,41], where mixed models are used 
for model reference control of an unknown Markov 
jump system. Despite that mixed model approaches are 
not widely used in control, with the appearance of in¬ 
creasingly more humanoid robots, biologically inspired 
robot controllers are gaining more popularity and the 
ones based on mixed models may present a promising 
approach [15.17,42,43]. 

Operator Models 

The models from the preceding sections are mainly 
used to predict a single future state or action. How¬ 
ever, in many problems, such as open-loop control, one 
would like to have information of the system for the 
next r-steps in the future. This problem is the multi-step 
ahead prediction problem, where the task is to pre¬ 
dict a sequence of future values without the availability 
of output measurements in the horizon of interest. We 
will term the models which are employed to solve this 
problem as operator models. It turns out that such oper¬ 
ator models are difficult to develop because of the lack 
of measurements in the prediction horizon. A straight¬ 
forward idea would be to apply single-step prediction 
models t times in sequence, in order to obtain a series 
of future predictions. However, this approach is suscep¬ 
tible to error accumulation, i. e., errors made in the past 
are propagated into future predictions. An alternative is 
to apply autoregressive models which are extensively 
investigated in time series prediction [15.44], in order 
to employ past predicted values to predict future out¬ 
comes. 

Combining operator models with control was first 
motivated by the need of extension of forward models 
for multi-step predictions [15.45]. In more recent work, 
variations of traditional ARX and ARMAX models for 
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Table 15.1 Overview on model types associated with applicable learning architectures and example applications 


Model type 

Forward model 

Learning architecture 

Example applications 

Direct modeling 

Prediction, filtering, learning simulations, optimization 

Inverse model 

Direct modeling (if invertible), 
indirect modeling 

Inverse dynamics control, computed torque control, feedback linearization 
control 

Mixed model 

Direct modeling (if invertible), 
indirect modeling, distal-teacher 

Inverse kinematics, operational space control, multiple model control 

Operator model 

Direct modeling 

Planning, optimization, model predictive control, delay compensation 


nonlinear cases have been proposed for control [15.46, 
47], However, operator models based on some paramet¬ 
ric structures, such as ARX or ARMAX, have shown to 
have difficulties in more complex systems. The situa¬ 
tion is even worse in the presence of noise or complex 
nonlinear dynamics. These difficulties give reasons to 
employ nonparametric operator models for multi-step 
predictions [15.48,49]. 

15.2.2 Model Learning Architectures 

In the previous section, we have presented different 
prediction problems that require different types of mod¬ 
els. A central question is how to learn and adapt these 
different models while they are being used. We will 
distinguish between direct modeling, indirect model¬ 
ing, and the distal teacher approach. Table 15.1 shows 
an overview of model types associated with applicable 
learning architectures. These architectures can be ob¬ 
served in Fig. 15.3. 

Direct Modeling 

Direct learning is probably the most straightforward 
and most frequently applied way to obtain a model 
despite that it is not always applicable. In this learn¬ 
ing paradigm, the model is directly learned by ob¬ 
serving the inputs and outputs, creating a supervised 
learning problem. Direct model learning can be imple¬ 
mented using most regression techniques, such as linear 
regression, local linear regression, recursive regres¬ 
sion [15.50,51], neural networks [15.52,53], or other 
statistical approximation techniques, currently mostly 
based on kernel methods [15.54,55]. 

An early example for direct learning control was 
the self-tuning regulator that generates a forward model 
and adapts it online [15.56]. Assuming the estimated 
forward model is correct (also called the certainty 
equivalence principle), the self-tuning regulator will es¬ 
timate an appropriate control law online. However, the 
forward model in the traditional self-tuning regulator 
has a fixed parametric structure and, hence, it cannot 
deal automatically with unknown nonlinearities [15.47, 
57]. The main reason why parametric models need to be 
used in direct modeling techniques is that such model 
parameterization is necessary for a convenient formu- 


a) Direct modeling 



b) Indirect modeling 

<r 



Fig.15.3a-c Learning architectures in model learning ap¬ 
plied to control, (a) In the direct modeling approach, the 
model is learned directly from the observations - the given 
example shows learning and inverse model, (b) Indirect 
modeling approximates the model using the output of the 
feedback controller as error signal, (c) In the distal teacher 
learning approach, the inverse model’s error is determined 
using the forward model. The resulting models will con¬ 
verge to an identity transformation 

lation of the control law and, more importantly, for 
the rigorous stability analysis. As the parametric model 
structure is often too limited for complex robot sys¬ 
tems, learning methods with more degrees of freedom 
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are needed, such as neural networks or other machine 
learning techniques [15.58,59], However, including so¬ 
phisticated learning algorithms in control makes the 
analysis more complex if not impossible. Some exam¬ 
ples of nonparameteric learning for control have been 
applied to unmanned airplane control [15.60], where 
the learning system models can prevent the airplane 
from crashing in case of changes in the airplane dynam¬ 
ics, e.g., due to failure of the mechanics. 

Inverse models can also be learned in a direct 
manner if the inverse mapping is well defined. A well- 
known example is the inverse dynamics model re¬ 
quired by computed torque and inverse dynamics con¬ 
trol [15.61,62]. If direct modeling is applicable, learn¬ 
ing becomes straightforward and can be achieved using 
standard regression techniques [15.14,63,64]. Early 
work in learning inverse models for control attempted 
to adapt a parametric form of the rigid body dynam¬ 
ics model which is linear in its parameters and, hence, 
can be estimated from data straightforwardly using lin¬ 
ear regression [15.65-67]. In practice, the estimation of 
dynamics parameters is not always straightforward. It 
is hard to create sufficiently rich data sets so that phys¬ 
ically plausible parameters can be identified [15.15], 
and when identified online, additional persistent excita¬ 
tion issues occur [15.68]. Models with fixed parametric 
structures are not capable of capturing certain nonlin¬ 
earities; hence, more sophisticated models have been 
introduced for learning inverse dynamics, such as neu¬ 
ral networks [15.64,69] or statistical nonparametric 
models [15.14,63,70]. There have also been attempts 
to combine parametric rigid body dynamics model with 
nonparametric model learning for approximating the 
inverse dynamics [15.71]. Similar as in inverse dynam¬ 
ics models, feedback linearization can also be used 
in conjunction with direct model learning. Again, the 
nonlinear dynamics can now be approximated using 
neural networks or other nonparametric learning meth¬ 
ods [15.60, 72]. Stability analysis of feedback lineariza¬ 
tion control with learned models is possible extending 
the cases where the nonlinear dynamics could not be 
canceled perfectly [15.60]. It should be noted that 
learning an inverse dynamics model (s.Sdes) —>a can 
have very slow learning performance when the initially 
learned model is rather far from reality. In this case, the 
predicted action a will create outcomes that are very far 
away from Sa es , such that the learning system, when try¬ 
ing out the bad action a, can hardly extract any useful 
information from this experience how to actually real- 
ize S'deg. 

While direct learning is mostly associated with 
learning a single type of model, it can also be applied 
to mixed models and operator models. The mixed model 
approach (e.g., combining inverse and forward models) 


finds its application in learning control for multiple- 
module systems. The basic idea is to decompose a com¬ 
plex system into many simpler subsystems which can 
be controlled individually [15.41]. The problem is how 
to choose an appropriate architecture for the multiple 
controllers, and how to switch between the multiple 
modules. Following the multiple-module idea, each con¬ 
troller module consists of a pair of inverse and forward 
models in the mixed model approach. The controller can 
be considered as an inverse model, while the forward 
model is essentially used to switch between the differ¬ 
ent modules [15.36]. Such multiple pairs of forward and 
inverse models can be learned directly using gradient- 
descend methods or expectation-maximization [15.42], 
Direct learning of operator models has been done with 
neural networks [15.73]. In more recent work, proba¬ 
bilistic methods are employed to learn such operator 
models [15.48, 49]. 

Indirect Modeling 

Direct learning works well when the functional rela¬ 
tionship is well defined. However, there can be situa¬ 
tions where the inverse relationship is not well defined, 
such as in the differential inverse kinematics prob¬ 
lem. In such cases, inverse models can often still be 
learned, but in an indirect way. One such indirect 
modeling technique is feedback error model learn¬ 
ing [15.74]. Feedback error learning relies on the output 
of a feedback controller that is used to generate the error 
signals employed to learn the feedforward controller. 
In problems, such as feedforward inverse dynamics 
control [15.61], this approach can be understood par¬ 
ticularly well: if the inverse dynamics model is perfect, 
the feedback controller is not needed and its output 
will be zero. If it is nonzero, it corresponds to the er¬ 
ror of the feedforward model [15.61]. The intuition 
behind feedback error learning is that by minimizing 
the feedback errors for learning the inverse model, the 
feedback control term will be decreasing as the model 
converges. 

Compared to the direct model learning, feedback er¬ 
ror learning is a goal-directed model learning approach 
resulting from the minimization of feedback error - in 
this error can only be zero if the goal was achieved. 
Another important difference between feedback error 
learning and direct learning is that feedback error learn¬ 
ing has to perform online, while direct model learning 
can be done both online and offline. Additionally, feed¬ 
back error learning does not create supervised learning 
data in the classical sense: the feedback error is not 
computed from an output target, but just from an error 
that represents the gradient toward better performance. 
This issue, and additionally a complex interaction be¬ 
tween PD gains and learning stability [15.75] can make 
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it more complex to set up a successful feedback error 
learning system. 

Feedback error learning is biologically motivated 
due to its inspiration from cerebellar motor con¬ 
trol [15.37]. It has been further developed for control 
with robotics applications, originally employing neu¬ 
ral networks [15.76,77]. Feedback error learning can 
also be used with various nonparametric learning meth¬ 
ods [15.75]. Conditions for the stability of feedback 
error learning control in combination with nonparamet¬ 
ric approaches have also been investigated [15.75]. 

Indirect model learning can also be used in the 
mixed model approach [15.78]. Here, the attempt has 
been made to combine the feedback error learning 
with the mixture of experts’ architecture to learn mul¬ 
tiple inverse models for different manipulated objects, 
where the inverse models are learned indirectly using 
the feedback error learning approach [15.78]. The for¬ 
ward model is used for training a gating network, as it 
is well defined. The gating network subsequently gen¬ 
erates a weighted prediction of the multiple inverse 
models, where the predictors determine the locally re¬ 
sponsible models. 

Distal Teacher Learning 

The distal teacher learning approach was motivated by 
the necessity to learn general inverse models which suf¬ 
fer from the ill-posedness problem [15.35]. Here, the 
nonuniqueness of the inverse model is resolved by the 
distal teacher which determines the error of the inverse 
model using a forward model. Unlike the feedback error 
learning approach, this insight allows directly aiming 
at a globally consistent inverse model instead of local 
on-policy optimization. The distal teacher employs two 
interacting learning process: one process where the for¬ 
ward model is learned, and the other process where the 
learned forward model is used for determining the er¬ 
ror of the inverse model. In the original distal learning 
approach, the inverse model’s output is validated by the 
forward model, as the composite of these models yields 
an identity mapping if perfectly learned [15.35]. The 
intuition behind this approach is that the inverse model 
will learn a correct solution by minimizing the error be¬ 
tween the output of the forward model and the input to 


the inverse model. Thus, the inverse model will result 
in solutions that are consistent with the forward model. 
The distal teacher approach has successfully learned 
particular solutions for multi-valued mappings, such as 
inverse kinematics of redundant robots [15.35]. Similar 
to feedback error model learning [15.74], distal teacher 
learning is also a goal-directed learning method appli¬ 
cable for various robot control scenarios. 

The distal learning approach is particularly suitable 
for control when combining with the mixed models, as 
it naturally incorporates the mixed model principle. The 
distal teacher learning approach with mixed models has 
motivated a number of follow-up projects with several 
robot control applications [15.17,43,79]. 

15.2.3 Model Learning Challenges 
and Constraints 

Given different types of models and their associated 
learning architectures, it may not be straightforward to 
apply machine learning methods [15.54, 55, 80] to these 
models in robotics. Several important problems arise 
that need to be addressed before learning methods can be 
successfully applied, and are summarized in Table 15.2. 

Data Challenge 

In order to learn a good model for applications in 
robotics, the sampled data has to cover a preferably 
large region of the model state space, and ideally cover 
it densely. Of course, in high-dimensional systems, one 
can never cover the complete state space - indeed, in 
complex robots like humanoids with state spaces of 
over 30 dimensions, only a very tiny fraction of the 
possible space can be explored in a lifetime. Thus, one 
can only hope to sample data in a task specific way, 
which is usually just in a very small part of the entire 
data space. For the purpose of generalization and accu¬ 
racy, the data has to be sufficiently rich, i. e., it should 
contain as much information about the system as possi¬ 
ble. Thus, generating large and rich data sets for model 
learning is an essential step. This step often requires 
additional excitation of the robot system during data 
generation, analogous to persistent excitation in clas¬ 
sical system identification [15.68]. 


Table 15.2 Some challenges for model learning in robotics 


Data challenges 

Algorithmic constraints 

Real-world challenges 

High-dimensionality 

Incremental updates 

Safety 

Smoothness 

Real-time 

Robustness 

Richness of data 

Online learning 

Generalization 

Noise 

Efficiency 

Interaction 

Outliers 

Large data sets 

Stability 

Redundant data 

Prior knowledge 

Uncertainty 

Missing data 

Sparse data 

In the environment 
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Many machine learning methods for regression as¬ 
sume that the model to be learned is sufficiently smooth. 
However, there are many applications in robotics where 
the approximated functions are known to be nons¬ 
mooth - one example is a stiction-friction model. Such 
nonsmooth functions are sometimes approximated us¬ 
ing kernel methods, where special kernels need to be 
defined in order to take the expected nonsmoothness in 
account [15.54,55]. Discontinuities in a function can 
also be approximated by learning how to switch dis- 
continuously between local models [15.81]. 

Difficulties arising from the large number of de¬ 
grees of freedom in robotics can also be mitigated 
using dimensionality reduction methods [15.82,83]. 
Dimensionality reduction is based on the insight that 
the useful information in the data may be represented 
by a low-dimensional manifold of the original input 
space. Dimensionality reduction methods have proven 
to be a powerful method for model learning in high¬ 
dimensional robot systems [15.63, 84]. 

As the data is sampled over a possibly long period 
of time in many robotics applications [15.85], problems 
of redundant and irrelevant data can occur. Redundant 
and irrelevant data can bias the model which severely 
degenerates the generalization performance. To over¬ 
come these issues, data needs to be filtered that is 
informative for the learning process [15.70,86]. Deal¬ 
ing with noise and outliers is always a challenging 
problem for robot learning. Recent work in the ma¬ 
chine learning community has addressed this problem 
by developing regularization frameworks based on sta¬ 
tistical learning theory. The basic idea is to constrain 
the model to be learned in order to alleviate the con¬ 
tributions made by the noisy components in the data. 
Examples of regularized model learning methods in¬ 
clude support vector regression and Gaussian process 
regression [15.87,88]. Methods that explicitly account 
for noise in the data can significantly improve their gen¬ 
eralization performance. 

Algorithmic Constraints 

In many scenarios, robot learning algorithms have 
to deal with massive amounts of data coming from 
their sensors. Thus, algorithms need to be efficient in 
terms of computation without sacrificing learning accu¬ 
racy [15.89]. Fast and real-time computation presents 
a challenging problem. Standard model learning ap¬ 
proaches, such as Gaussian process regression, for 
example, scale cubically in the number of training data 
points, preventing a straightforward usage in robotics. 
Sparse and reduced set methods smartly reduce the size 
of training data and, thus, decrease the computational 
effort for the learning and the prediction step [15.90]. 
Efficient implementations using parallel computation 


may also be used to speed up machine learning algo¬ 
rithms [15.91]. 

Online learning is also needed for continuous adap¬ 
tation to a changing world which is essential to make 
robots more autonomous [15.85]. Most machine learn¬ 
ing methods are developed for learning in batch mode, 
while online learning requires incremental approxima¬ 
tion of the model [15.14,92]. An advantage of online 
model learning is the ability to uncover interesting 
state-space regions during task execution. 

For some robots, it may be difficult or expensive 
to obtain a lot of data for learning. In these cases, 
we need algorithms which allow us to incorporate 
additional prior knowledge to improve the learning per¬ 
formance in the presence of sparse data [15.93,94], 
In kernel methods, prior knowledge can be specified 
by feature vectors which can be used to define appro¬ 
priate kernels [15.93]. In contrast, probabilistic frame¬ 
works allow specifying a priori information [15.54], 
Furthermore, prior knowledge, if given as a paramet¬ 
ric model, can be inserted into nonparametric models 
yielding semiparametric learning approaches [15.71, 
95]. Semiparametric models have shown to be capable 
in learning competitive models, when little data is avail¬ 
able [15.71], 

Real-World Constraints 

Ensuring safe interaction of robots with human beings 
in everyday life, machine learning algorithms devel¬ 
oped for robotics applications have to minimize the risk 
of injury and damage. For critical applications, such as 
medical or service robotics, robustness and reliability 
are among the most important criteria which have to be 
fulfilled by model learning. Learning becomes more ro¬ 
bust by employing feature selection as a preprocessing 
step, and has been used in several robotics application 
in vision and control [15.70, 96]. 

Robustness also requires the ability of the learn¬ 
ing algorithm to deal with missing data. Missing data 
due to measurement errors poses a difficult problem 
for model learning in robotics. Recently, probabilistic 
learning methods have been used to infer the missing 
components in the training data [15.97]. An advantage 
of probabilistic methods comes from the ability to as¬ 
sign uncertainty to the predicted values, making it easier 
to deal with missing and imprecise data. 

15.2.4 Model Learning Methods 

We consider model learning in the context of supervised 
learning, where the inputs* and outputs y are given. The 
true output data is corrupted by noise e , i. e., 

y =/(*) + €. (15.6) 
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Table 15.3 A large variety of machine learning methods have been applied in model learning for robotics. We distinguish 
between global and local methods, as well as semilocal methods which combine both approaches. The methods differ 
in the training mode and their online capabilities. For computational complexity, n denotes the total number of training 
points, m is the number of data points in a local model, and M is the number to local models 


Method 

Type 

Mode 

Online 

Complexity 

Learning applications 

Locally weighted projection regres¬ 
sion [15.80] 

Local 

Incremental 

Yes 

0(n) 

Inverse dynamics [15.63], 
Foothold quality model [15.21] 

Local Gaussian process regression [15.14] 

Local 

Incremental 

Yes 

G(ni 2 ) 

Inverse dynamics [15.14] 

Gaussian mixture model [15.98] 

Semilocal 

Batch 

No 

O(Mn) 

Human motion model [15.99] 

Bayesian committee machine [15.100] 

Semilocal 

Batch 

No 

0(m 2 n ) 

Inverse dynamics [15.54] 

Sparse Gaussian process regression [15.101] 

Global 

Incremental 

Yes 

<9(n 2 ) 

Transition dynamics [15.32], 

Task model [15.102] 

Gaussian process regression [15.103] 

Global 

Batch 

No 

0(n 3 ) 

Terrain model [15.104], 

State estimation 
model [15.105] 

Support vector regression [15.55] 

Global 

Batch 

No 

0(n 2 ) 

ZMP control model [15.106], 
Grasp stability model [15.107] 

Incremental support vector machine [15.108] 

Global 

Incremental 

Yes 

0(n 2 ) 

Inverse dynamics [15.109] 


Note that the noise is additive to f(x)\ models that 
address noise in the inputs can be developed as 
well [15.110]. The performance of a model is given by 
its generalization ability, i. e., the learned model should 
be able to provide precise predictions of the output 
on new input data. Different learning techniques make 
different assumptions on how to model the function/ 
given the data. From a function approximation point 
of view, we will distinguish between global and lo¬ 
cal techniques. Beyond the local and global types of 
model learning, there are also approaches that com¬ 
bine both ideas. An example of such a hybrid approach 
is the mixtures of experts [15.98, 111]. Here, the data 
is partitioned into smaller local models by a learned 
gating network, and prediction is accomplished by fus¬ 
ing local model predictions. Mixture of experts have 
been further embedded into a Bayesian learning frame¬ 
work, giving rise to a number of Bayesian hybrid 
approaches such as committee machines [15.100], mix¬ 
tures of Gaussian models [15.112], or infinite mixtures 
of experts [15.113]. 

Global Regression 

Global regression techniques model the underlying 
function / using all observed data [15.114], i. e., ev¬ 
ery data point is potentially adjusting every parameter 
of the learning system. A straightforward way to model 
the function / is to assume a parametric structure, such 
as linear or polynomial models or multilayer perceptron 
neural networks, and to fit the model parameters us¬ 
ing training data [15.52, 114,1 15]. However, fixing the 
model with a parametric structure beforehand may not 
suffice to explain the sampled data with sufficient ac¬ 
curacy, which motivates nonparametric model learning 
frameworks [15.54,55,115]. Nonparametric learning 


does not mean that there are no parameters, just that the 
correct number of relevant parameters is unknown, and 
that it is unlikely that a finite set of parameters would 
suffice to model the entire regression problem. 

In the last two decades, parametric and nonparamet¬ 
ric regression methods have been focussing on func¬ 
tions/ modeled as 

f(x) = 0 T <t>(x) , (15.7) 

where 0 is a weight vector and </> is a vector of nonlin¬ 
ear functions, also called features or basis-functions. (/> 
projects x into some high-dimensional space, in which 
the learning problem is assumed to become linear in 
the parameters. The basic idea behind nonparametric 
regression is that the optimal model structure should 
be obtained from the training data. Hence, the size 
of the weight vector 0 is not fixed but can increase 
with the number of training data points. In contrast 
to nonparametric statistical approaches, other popular 
function approximation methods, such as neural net¬ 
works, fix the model structure beforehand [15.52], like 
the number of nodes and their connections. However, 
there is a connection between artificial neural networks 
and probabilistic approaches [15.116,117] using the 
formalization in (15.7). 

Learning systems as formalized in (15.7) are mostly 
discussed as kernel methods [15.55]. Learning a model 
is equivalent to finding a tradeoff between the model’s 
complexity and the best fit of the model to the ob¬ 
served data. Thus, it is desirable to have a model which 
is simple but at the same time can explain the data 
well. The number of features (i. e., the length of feature 
vector (/>) has a direct correlation with the complexity 
of the regression model, as the more features one al¬ 
lows, usually the more powerful the learning system 
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becomes, but it also takes more data to constrain the 
open parameters of the model. One of the key insights 
of kernel methods [15.54, 55] was that one can actually 
work with infinitely many features using the so-called 
kernel trick, where the formulation of the regression 
problem only involves inner products of the feature 
vectors 0 (jc,-) t 0(aj) of any two inputs jty and Xj, and 
that this inner product can often be computed in closed 
form by a kernel function k(xi,Xj). This approach intro¬ 
duces a dual form of the regression learning problem 
where the number of open parameters is identical to 
the number of n training data, and not the number of 
features. In support vector regression (SVR) [15.55], 
a sparsification process will choose the minimal num¬ 
ber of parameters needed to represent the training data. 
In Gaussian process regression [15.54], a probabilistic 
approach is taken that places a prior distribution over 0. 
The posterior parameters can subsequently be obtained 
by optimizing the corresponding marginal likelihood. 

Kernel and probabilistic methods have proven to 
be successful tools for model learning over the last 
decade, resulting in a number of widely applied regres¬ 
sion methods [15.87,88,118], These methods can be 
applied to high-dimensional data. They can also deal 
well with noisy data, as the noise is taken in account 
indirectly by regularizing the model complexity. Fur¬ 
thermore, they are relatively easy to use, as several 
black-box implementations are available. However, the 
major drawback of these methods are the computa¬ 
tional complexity. For instance, for Gaussian process 
regression the complexity scales cubically in terms of 
the number of training data points. Thus, one active 
research line in machine learning is to reduce the com¬ 
putational cost of those approaches. Due to customized 
techniques for robotics, kernel and probabilistic regres¬ 
sion techniques found their ways into several robotics 
applications, such as robot control [15.70], sensor mod¬ 
eling [15.119], or state estimation [15.105]. 

Local Learning Methods 

The basic idea of local regression techniques is to 
estimate the underlying function/ within a local neigh¬ 
borhood around a query input point jc q . The data points 
in this neighborhood can then be used to predict the out¬ 
come for the query point. Generally, local regression 
models can be obtained by minimizing the following 
cost function J using n training data points 

n 

.1 = ^2w[(x k -x q ) J D(x k -x q )] \\y k -f(Xk)f • 

k=l 

(15.8) 

As indicated by (15.8), the essential ingredients for a lo¬ 
cal regression model are the neighborhood function w 


and the local model/. The neighborhood function w, 
which is controlled by a distance metric D, often sim¬ 
plified as a scalar width parameter D = 1 /hi, measures 
the distance between a query point x q to the points in the 
training data. The local model / describes the function 
structure used to approximate / within the neighbor¬ 
hood around * q [15.120, 121]. Depending on the com¬ 
plexity of the data, different function structures can be 
assumed for the local model/, such as a linear or a poly¬ 
nomial model. The open-parameters of / can be esti¬ 
mated straightforwardly by minimizing J with respect 
to these parameters. 

The choice of the neighborhood function and its 
distance metric parameter is more involved. Several 
techniques have been suggested for estimating D for 
a given w, including the minimization of the leave- 
one-out cross-validation error and adaptive bandwidth 
selection [15.122,123]. 

Due to their simplicity and computational ef¬ 
ficiency, local regression techniques have become 
widespread in model learning for robotics [15.124— 
126]. In the last decade, novel local regression ap¬ 
proaches were further developed in order to cope with 
the demands in real-time robotics applications, such 
as locally weighted projection regression [15.92, 127]. 
Inspired by local regression techniques, these meth¬ 
ods first employ a partitioning of the input space into 
smaller local regions, for which locally linear models 
are approximated. A useful metaphor is to imagine that 
the input space is partitioned into disjoint boxes, and 
for the data points falling into each of these boxes, an 
independent local model is fit, usually of very simple 
complexity, like locally linear models. 

In addition to the computational efficiency, local 
methods can deal with less smooth functions and do not 
require the same smoothness and regularity conditions 
as global regression methods. However, it has been 
shown in practice that local methods suffer from prob¬ 
lems induced by high-dimensional data, as notions of 
locality break down for sparse, high-dimensional data. 
Furthermore, the learning performance of local meth¬ 
ods may be sensitive to noise and heavily depends on 
the way how the input space is partitioned, i. e., the 
configuration of the neighborhood function w. These 
problems still present an active research topic [15.17, 
128], 

Several attempts have been made to scale local re¬ 
gression models to higher dimensional problems as 
required for many modern robotics systems. For ex¬ 
ample, locally weighted projection regression com¬ 
bines local regression with dimensionality reduction 
by projecting the input data into a lower dimen¬ 
sional space, where local regression is employed after¬ 
ward [15.92]. Other methods combine nonparametric 
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probabilistic regression, such as Gaussian process re¬ 
gression, with the local approaches while exploiting the 
strength of probabilistic methods for model learning 
in high-dimension [15.14,17]. Local learning meth¬ 
ods are very well suited to parallelization on modern 
computer hardware, as each local model can be fit 
independently. 

15.2.5 Applications of Model Learning 

Several representative examples of model learning are 
illustrated in the following sections. 

Simulation-Based Optimization 
As forward models directly describe the causal dynamic 
behavior of a system, learning such models has received 
much attention in model-based control, particularly for 
optimal control problems. In optimal control, a for¬ 
ward model is needed to predict the behavior of the 
control system over an extended time horizon in order 
to optimize the control policy. If an analytical model 
is inaccurate, learning the model offers an interesting 
alternative. 

Atkeson and Morimoto [15.29] have been among 
the first to explore this approach using differential dy¬ 
namic programming [15.129] for optimizing open-loop 
control policies. The basic idea was to use receptive 
field-weighted regression (a type of locally weighted 
regression) to learn the models of the cost function 
and the state transition. Differential dynamic program¬ 
ming locally linearizes the state transition model and 
generates a local quadratic approximation of the cost. 
These approximations are used to improve an open-loop 
policy where the linearizations are also updated after 
every policy update [15.29]. Atkeson and Morimoto 
used the method to learn the underactuated pendu¬ 
lum swing up task, where a pole is attached to with 
a passive joint to the endeffector of a robot arm. The 
goal of the robot is to bring the pole from a hang¬ 
ing to an upright position. Hence, the system needs to 
pump energy into the pendulum in order to swing it up 
(Fig. 15.4a). Subsequently, the pole needs to be sta¬ 
bilized at the upright position [15.130]. Starting from 
an unconstrained human demonstration, the robot was 
able to successfully learn the swing up and balance task 
after three trials [15.130]. The local trajectory optimiza¬ 
tion technique has been further extended to biped robot 
walking [15.131]. More recently, a related approach 
with parametric function approximation has been ap¬ 
plied by Abbeel et al. [15.132] to learn autonomous 
helicopter flight (l<s>MiliLlLliil). 

While Atkeson and Morimoto [15.29] and Abbeel 
et al. [15.132] used the forward model as an implicit 
simulator, Ng et al. [15.30] used it as an explicit sim¬ 


ulator (as originally suggested by Sutton [15.28] in 
form of the DYNA model). Here, with the help of 
the forward model, complete trajectories (or roll-outs) 
were simulated. The predictions of the forward model 
were perturbed by Gaussian noise with a repeating, 
fixed noise history. This injection of noise requires the 
learned controller to be robust to noise and modeling 
errors, while the re-use of the noise history reduces 
the variance of policy updates and results in much 
faster learning [15.133, 134]. In this way, roll-outs from 
a similar set of start-states are generated. The result¬ 
ing performance of different control policy can be 
compared, which allows policy updates both by pair¬ 
wise comparison or by gradient-based optimization. 
The approach has been able to successfully stabilize 
an helicopter in an inverted flight (Fig. 15.4b and 
lltkU). A few similar examples in model pre¬ 
dictive control (Sect. 15.2.1) exist, which employ a va¬ 
riety of different learning approaches such as statistical 
learning methods [15.48], neural networks [15.135], 
both for robot navigation [15.136] and helicopter con¬ 
trol [15.137], 



Fig. 15.4 (a) Sketch of learning a pendulum swing up task, 
and (b) inverse helicopter flight (courtesy of Andrew Y. 
Ng). In both the cases, the forward model is used to learn 
the dynamics of the system for policy optimization 
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Approximation-Based 
Inverse Dynamics Control 

Inverse models, such as inverse dynamics models, 
are frequently used in robotics for feedforward con¬ 
trol [15.62]. Inverse dynamics models characterize the 
required joint torques r =f(q , q. q) to achieve a desired 
robot acceleration q for the state q. q, i. e., the vectors of 
joint positions and velocities. In classical robot control, 
analytical dynamics models are derived in the frame¬ 
work of rigid body dynamics 

r (q,q.q) = M(q)q + F(q,q) , (15.9) 

where M(^) is the generalized inertia matrix of the 
robot, F(q, q) is a vector defined by forces/torques 
from Coriolis effects, centripetal effects, and gravity. 
This model relies on many assumptions, such as rigid¬ 
ity of the robot’s links, negligible nonlinearities from 
other sources (e.g., actuators), second-order dynamics, 
etc. [15.62]. However, modem robotics systems with 
highly nonlinear components, such as hydraulic tubes 
or elastic cable drives, can no longer be accurately mod¬ 
eled with rigid body dynamics. An inaccurate dynamics 
model can lead to severe losses in control performance 
and, in the worst case, instability. Instead of model¬ 
ing the inverse dynamics manually based on physics 
and human insight, an inverse dynamics model can 
be learned from sampled data. Such a data-driven ap- 



Fig. 15.5 The 7 degree of freedom anthropomorphic SAR- 
COS arm used in learning inverse dynamics, inverse kine¬ 
matics, and operational space control 


proach has the advantage that all nonlinearities encoded 
in the data will be approximated by the model [15.14]. 

In most robots, the inverse dynamics model is 
a unique mapping from joint space into torque space, 
and learning inverse dynamics models is a standard re¬ 
gression problem. In order to generalize the learned 
models for a larger state space and to adapt the 
models for time-dependent changes in the dynamics, 
real-time online learning becomes necessary. However, 
online learning poses difficult challenges for any re¬ 
gression method. These problems have been addressed 
by real-time learning methods such as locally weighted 
projection regression [15.80]. Nguyen-Tuong and Pe¬ 
ters [15.14,70] combined the basic ideas behind the 
locally weighted projection regression method with 
Gaussian process regression [15.54], attempting to get 
as close as possible to the speed of local learning while 
having a comparable accuracy to Gaussian process 
regression. The resulting method has shown to be ca¬ 
pable of real-time online learning of the robot’s inverse 
dynamics. Instead of using local models, data sparsi- 
fication methods can be employed to speed up kernel 
regression approaches for real-time scenarios [15.70]. 

It is worth noting that inverse dynamics model 
learning approaches can also be motivated from a bi¬ 
ological point of view. Kawato [15.37] suggested that 
the cerebellum may act as an inverse dynamics model. 
Motivated by this insight, Shibata and Schaal [15.76] 
proposed a biologically inspired vestibulo-oculomotor 
control approach based on feedback-error learning of 
the inverse dynamics model. The problem is to stabilize 
the gaze of an oculomotor system that is mounted on 
a moving body, which creates permanent perturbations 
to eye (or camera) system. The cerebellum is known 
to predict the forces required to keep the image sta¬ 
bilized on the retina, based on efferent motor signals 
and inputs from the vestibular system. Vijayakumar and 
Schaal employ the locally weighted projection regres¬ 
sion approach to learn the inverse model of the eye 
dynamics online [15.80]. The same locally weighted 
projection regression technique has also been used to 
learn a complete inverse dynamics model for a hu¬ 
manoid robot [15.63] (Fig. 15.5). 

Learning Operational Space Control 
Operational space control (OSC) allows a robot to ac¬ 
complish a task space controller, e.g., for following 
a desired trajectory in task space or for impedance 
control [15.15, 138]. The relationship between the task 
space and joint space of the robot is well defined by the 
classical forward kinematics models x =f(q), where q 
denotes a joint space configuration and x represents the 
corresponding task space position. The task space ve¬ 
locity and acceleration are given by x = J (q)q and x = 
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i(q)q + J (q)q, respectively, where J(qr) = df/dq is the 
Jacobian. To obtain the joint torques required for task 
space control, the dynamics model (as given in (15.9)) 
is needed. The direct combination of dynamics and 
kinematics model yields one possible operational space 
control law 

a = MJ+(jc- jq) + F, (15.10) 

where Jjjj denotes the weighted pseudoinverse 
of J [15.139] and a represents the desired joint torques. 
Equation (15.10) can be employed to generate the 
joint torques necessary for tracking a task space 
trajectory determined by a reference task-space accel¬ 
eration [15.15]. Note that in practical applications, such 
control laws often require further terms, such as a null- 
space control law for joint-space stabilization [15.15]. 

As discussed before, dynamics models can be hard 
to obtain and, thus, learning can be a promising alterna¬ 
tive. Learning an operational space control law corre¬ 
sponds to learning an inverse model such as (q, q.x) —>■ 
a [15.43]. However, learning such OSC models is an 
ill-posed problem, as there are infinitely many possible 
inverse models, due to the redundancy of joint space to 
task space mappings. For example, infinitely many so¬ 
lutions for a redundant robot can be derived analytically 
by varying the metric W of the weighted pseudoinverse 
in (15.10). As the space of possible solutions is not 
convex (Sect. 15.2.1, Mixed Models), such OSC mod¬ 
els cannot be straightforwardly learned using regression 
models (unless the system has no redundant degrees of 
freedom). Similar problems appear in the limited case 
of differential inverse kinematics. 

Both D’Souza et al. [15.79] and Peters and 
Schaal [15.43] noticed that local linearizations of the 
mapping in (15.10) will always form a locally convex 
space. Hence, data sets generated by such systems will 
also be locally convex. They furthermore realized that 
the predictive abilities of forward models allows deter¬ 
mining local regions, where a locally consistent inverse 
model can be learned. However, depending on how the 
learning data was collected, i. e., the local data distri¬ 
bution, learning may result in rather different solutions 
to the local inverse model. Thus, two neighboring local 
inverse models may have quite different policies how to 
resolve redundancy, resulting in a discontinuous jump 
when transitioning from one local model to the next 
during control. As a result, a globally consistent so¬ 
lution to OSC can no longer be ensured. This insight 
leads to two significantly different approaches. D’Souza 
et al. [15.79] created a differential inverse kinematics 
learning system (i. e., a limited special case of an opera¬ 
tional space control law) and chose to bias the learning 
system by selectively generating data. However, such 


an approach will generically be limited by the tradeoff 
between this intentional bias and the inverse model’s 
accuracy. Peters and Schaal [15.43] treated learning 
of complete operational space control laws. They real¬ 
ized that a re-weighting of the data using an additional 
reward function allows regularizing these inverse mod¬ 
els toward a globally consistent solution. Inspired by 
a result in analytical OSC [15.139], they suggest ap¬ 
propriate reward functions both for learning full OSC 
and differential inverse kinematics. The resulting map¬ 
ping was shown to work on several robot systems. Ting 
et al. [15.17] presented an implementation of Peters 
and Schaal [15.43] approach with modern Bayesian 
machine learning, which sped up the performance sig¬ 
nificantly. 

Instead of learning a direct OSC control law as done 
by Peters and Schaal [15.43], Salaun et al. [15.140] 
attempted to learn the well-defined differential for¬ 
ward kinematics as a first step (i. e., learning the Ja¬ 
cobian) using locally weighted projection regression. 
The corresponding weighted pseudoinverse of the Ja¬ 
cobian is subsequently computed using singular value 
decomposition (SVD). The obtained differential in¬ 
verse kinematics model is combined with an inverse 
dynamics model to generate the joint space control 
torques [15.140]. Approximating inverse kinematics 
models has also been investigated in neural network 
learning and reservoir computing architectures [15.35, 
141]. 

15.2.6 Future Directions 

A key question in robotics is how to deal with un¬ 
certainty in the environment. As probabilistic machine 
learning techniques (e.g., Bayesian inference) have 
reached a high level of maturity [15.54], it has become 
clear how beneficial Bayesian machine learning can 
be for robot learning, especially, for model learning in 
the presence of uncertainty. However, machine learning 
techniques based on Bayesian inference are known to 
suffer from high computational complexity. Thus, spe¬ 
cial approximations will be needed as illustrated by the 
problems discussed in Sect. 15.2.3. Recently developed 
approximate inference methods such as in [15.90, 142] 
may become interesting new tools for robotics. 

In order for robots to enter everyday life, they need 
to continuously learn and adapt to new tasks. Recent 
research on learning robot control has predominantly 
focused on learning single tasks that were studied in 
isolation. However, there is an opportunity to transfer 
knowledge between tasks, which is known as transfer 
learning in the field of machine learning [15.143]. To 
achieve this goal, robots need to learn the invariants 
of the individual tasks and environments and, sub- 


Part A 115.2 



Part A 115.3 


372 Part A I Robotics Foundations 


sequently, exploit them when learning new tasks. In 
this context, similarities between tasks also need to be 
investigated and how they can be exploited for general¬ 
ization [15.144], 

In most of the model learning approaches, super¬ 
vised learning methods are used. However, for many 
robot applications, target outputs are not always avail¬ 
able. In these scenarios, semisupervised learning tech¬ 
niques can be useful to learn such models [15.145], 
Semisupervised learning employs labeled as well as 
unlabeled data for model learning and can help to over¬ 
come the sparse labeling problem. It would also be 
beneficial to develop online versions of semisupervised 
approaches for real-time adaptation and learning. 

Approximation-based control often still suffers 
from a lack of proper analysis of stability and con¬ 
vergence properties. Modern statistical learning the¬ 


15.3 Reinforcement Learning 

Probably the most general frameworks for learning 
control is RL which enables a robot to autonomously 
discover optimal behaviors through trial and error in¬ 
teractions with its environment. No explicit teaching 
signal is provided, as in supervised learning, but rather 
learning is based on a scalar objective function that 
measures the overall performance of the robot. 

Consider the example of training a robot to re¬ 
turn a table tennis ball ([15.147], ) In this 

case, the robot makes observations of dynamic vari¬ 
ables specifying ball position and velocity as well as its 
internal dynamics including joint positions and veloc¬ 
ities. These measurements capture the relevant state s 
of the system. The actions a available to the robot are 
the torque commands sent to the motors. A function n 
that computes the motor commands (i. e., the actions) 
based on the current state is the policy. The RL prob¬ 
lem is to find a policy that optimizes the long term sum 
of rewards R(s.a), as already indicated in Sect. 15.1.3. 
The reward function in this example could be based 
on the number of successful hits, or on secondary cri¬ 
teria like energy consumption; RL algorithms aim to 
find (near-)optimal policies that maximize the reward 
function over a chosen time horizon. An extended dis¬ 
cussion of robot RL techniques, on which this chapter 
is based, can be found in [15.10]. 

15.3.1 Relation to Classical Control 

RL is closely related to the theory of classical optimal 
control, as well as dynamic programming, stochas¬ 
tic programming, simulation optimization, stochastic 


ory might offer new tools to derive appropriate error 
bounds [15.55]. For example, generalization bounds 
can be used to estimate the learning performance of the 
controller and, from this insight, further statements and 
conditions about stability could be made. 

Learning nonunique and ill-posed mappings is a key 
problem for several robot applications. This problem 
can be considered as a nonunique labeling problem, and 
statistical machine learning techniques, such as condi¬ 
tional random fields [15.146], may help to solve the 
problem. A conditional model specifies the probabili¬ 
ties of possible target outputs given an input observation 
sequence. As the target outputs are conditioned on 
the current input observations, nonuniqueness in the 
mappings can be resolved. It would be beneficial to in¬ 
vestigate how such models could be incorporated into 
learning control and hierarchical control frameworks. 


search, and optimal stopping [15.152]. Both RL and 
optimal control address the problem of finding an op¬ 
timal policy (often also called the controller or control 
policy) that optimizes an objective function over a time 
horizon, and both rely on the notion of a system be¬ 
ing described by an underlying set of states, controls 
and a plant or model that describes transitions be¬ 
tween states. However, optimal control assumes perfect 
knowledge of the system’s description in the form of 
a model (e.g., a model as in Sect. 15.2 that describes 
what the next state of the robot will be given the cur¬ 
rent state and action). For such models, optimal control 
ensures strong guarantees which, nevertheless, often 
break down due to model and computational approxi¬ 
mations. 

In contrast, RL operates directly on measured data 
and rewards from interaction with the environment. Re¬ 
inforcement learning research has placed great focus on 
addressing cases which are analytically intractable by 
using approximations and data-driven techniques. An 
important class of approaches to RL within robotics 
centers on the use of classical optimal control tech¬ 
niques (e.g., linear-quadratic regulation and differential 
dynamic programming) to system models learned via 
repeated interaction with the environment [15.150, 153, 
154]. A concise discussion of viewing RL as adaptive 
optimal control is presented in [15.155]. This path is 
indicated in Fig. 15.1 as optimal control with learnt 
models. 

Reinforcement learning applied to robotics 
(Fig. 15.6) differs considerably from most well-studied 
RL benchmark problems. Problems in robotics are 
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a) Obelix robot t>) Zebra Zero robot c) Autonomous helicopter d) Sarcos humanoid DB 



Fig.15.6a-d A small sample of robots with behaviors that were reinforcement learned. These cover the whole range of 
aerial vehicles, robotic arms, autonomous vehicles, and humanoid robots, (a) The OBELIX robot is a wheeled mobile 
robot that learned to push boxes (after [15.148]) with a value function-based approach (courtesy of Sridhar Mahadevan). 
(b) A Zebra Zero robot arm learned a peg-in-hole insertion task (after [15.149]) with a model-free policy gradient 
approach (courtesy of Rod Grupen). (c) Carnegie Mellon’s autonomous helicopter leveraged a model-based policy search 
approach to learn a robust flight controller (after [15.150]). (d) The Sarcos humanoid DB learned a pole-balancing task 
(after [15.151]) using forward models 


often involve high-dimensional, continuous states 
and actions (note that the 10—30-dimensional con¬ 
tinuous actions common in robot RL are considered 
large [15.152]). In robotics, it is often unrealistic to 
assume that the true state is completely observable 
and noise free. The learning system will not be able 
to know precisely in which state it is and even vastly 
different states might look very similar. Thus, RL in 
robotics is often modeled as partially observed, a point 
we take up in detail in our formal model description 
below. The learning system must hence use filters to 
estimate the true state. It is often essential to maintain 
the information state of the environment that not only 
contains the raw observations but also a notion of 
uncertainty on its estimates (e.g., both the mean and 
the variance of a Kalman filter tracking the ball in the 
robot table tennis example). 

Experience on a real physical system is tedious to 
obtain, expensive, and often hard to reproduce. Even 
getting to the same initial state is impossible for the 
robot table tennis system. Every single trial run, often 
called a roll-out or episode, is costly and, as a result, 
such applications force us to focus on difficulties that 
do not arise as frequently as in classical RL bench¬ 
mark examples that are mostly simulation studies. In 
order to leam within a reasonable time frame, suit¬ 
able approximations of state, policy, value function, 
and/or system dynamics need to be introduced. How¬ 
ever, while real-world experience is costly, it usually 
cannot be replaced by learning in simulations alone. 
In analytical or learned models of the system, even 
small modeling errors can accumulate to a substantially 
different behavior. Hence, algorithms need to be ro¬ 
bust with respect to models that do not capture all the 
details of the real system, also referred to as under¬ 
modeling, and to model uncertainty. Another challenge 
commonly faced in robot RL is the generation of appro¬ 
priate reward functions. Rewards that guide the learning 


system quickly to success are needed to cope with the 
cost of real-world experience. This problem is called 
reward shaping [15.156] and represents a substantial 
manual contribution. Specifying good reward functions 
in robotics requires a fair amount of domain knowledge 
and may often be hard in practice. 

Not every RL method is equally suitable for 
the robotics domain. In fact, many of the meth¬ 
ods thus far demonstrated on difficult problems have 
been model-based [15.50,132,157] and robot learn¬ 
ing systems often employ policy search methods rather 
than value function-based approaches [15.149,150, 
158-164]. Such design choices stand in contrast to pos¬ 
sibly the bulk of the early research in the machine 
learning community [15.13, 165]. 

15.3.2 Reinforcement Learning Methods 

In RL, an agent tries to optimize the accumulated re¬ 
ward over a time horizon, e.g., a manipulation task or 
even its entire lifetime. In an episodic setting, where the 
task is restarted at the end of each episode, the objective 
is to maximize the total reward per episode. If the task 
is on-going, a discounted accumulated reward which 
weighs earlier rewards with less influence is typically 
optimized. The agent is modeled as being in an envi¬ 
ronment with state s e S and can perform actions a e A. 
The states and actions can be either discrete or con¬ 
tinuous and are normally multi-dimensional. A state s 
contains all relevant information about the agent and 
environment needed to model future states or observ¬ 
ables. An action a is used to control the time evolution 
of the system. For example, in a navigation task, the 
state could be the position and orientation of the robot, 
whereas the action corresponds to the torques applied 
to the wheels. 

At every time step, the agent gets a reward R, 
a scalar value which typically depends on its state and 
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action. For instance, in a navigation task, rewards could 
be given for reaching the target, and penalties (neg¬ 
ative reward) would arise for expending energy. The 
goal of RL is to find a policy n that chooses ac¬ 
tion a in s such as to maximize the cumulative reward 
over time. The policy n can be either deterministic, 
which always chooses the same action for each state 
a = n(s), or probabilistic which draws a sample action 
from a distribution: a ~ jr(s,a) = P (a\s). Reinforce¬ 
ment learning algorithms need to discover the relations 
between states, actions, and rewards. Hence exploration 
is required to sufficiently sample these functional rela¬ 
tionships, which can be directly embedded in the policy 
or performed separately as part of the learning process. 

Classical RL approaches are based upon the as¬ 
sumption of a Markov decision process or MDP which 
models the dynamics of the environment through 
transition probabilities T. Given the current state s 
and action a , the conditional transition probability 
T(s',a,s) = P(s'\s,a) describes the distribution over 
the next state s' and generalize the notion of de¬ 
terministic dynamics. The Markov property requires 
that the next state s' and reward depend only on the 
previous state s and action a, and not on any other 
information [15.13]. In robotics, we are able to find 
an approximate notion of a state in a MDP for many 
situations. 

Goals of Reinforcement Learning 
The goal of RL is to discover an optimal policy n* that 
maps states (or observations) to actions so as to max¬ 
imize the expected return J, which corresponds to the 
cumulative expected reward. There are different models 
of optimal behavior [15.165] which result in differ¬ 
ent definitions of the expected return. A finite-horizon 
model only attempts to maximize the expected reward 
for the horizon H, i. e., the next I! (time-)steps h 

( H ) 

This setting can also be applied to model problems 
where it is known how many steps are remaining. 

Alternatively, future rewards can be discounted by 
a discount factor y (with 0 < y < 1) yielding a metric 

/=(l-y),E f>"R„ . 

u=o ) 

This setting is most frequently discussed in classical 
RL texts. The parameter y affects how much the future 
is taken into account and needs to be tuned manually. 
As illustrated in [15.165], this parameter often qualita¬ 
tively changes the form of the optimal solution. Policies 


designed by optimizing with small y are myopic and 
greedy, and may lead to poor performance if we actually 
care about longer term rewards. It is straightforward to 
show that the optimal control law can be unstable if the 
discount factor is too low (e.g., it is not difficult to show 
this destabilization even for discounted linear quadratic 
regulation problems). Hence, discounted formulations 
are occasionally inadmissible in robot control despite 
that they are somewhat more common in the machine 
learning community. 

In the limit when the discount factor y approaches 
1, the metric approaches what is known as the average- 
reward criterion [15.7] 



This setting has the problem that it cannot distinguish 
between policies that initially gain a transient of large 
rewards and those that do not. This transient phase, also 
called prefix, is dominated by the rewards obtained in 
the long run. If a policy accomplishes both an optimal 
prefix as well as an optimal long-term behavior, it is 
called bias optimal [15.166]. An example in robotics 
would be the transient phase during the start of a rhyth¬ 
mic movement, where many policies will accomplish 
the same long-term reward but differ substantially in the 
transient (e.g., there are many ways of starting the same 
gait in dynamic legged locomotion) allowing for room 
for improvement in practical application. 

In real-world domains, the shortcomings of the dis¬ 
counted formulation are often more critical than those 
of the average reward setting as stable behavior is of¬ 
ten more important than a good transient [15.167]. We 
also often encounter an episodic control task, where 
the task runs only for H time-steps and then reset (po¬ 
tentially by human intervention) and started over. This 
horizon H may be arbitrarily large, as long as the ex¬ 
pected reward over the episode can be guaranteed to 
converge. As such episodic tasks are probably the most 
frequent ones, finite-horizon models are often the most 
relevant. 

Two natural goals arise for the learner. In the first, 
we attempt to find an optimal strategy at the end of 
a phase of training or interaction. In the second, the goal 
is to maximize the reward over the whole time the robot 
is interacting with the world. 

In contrast to supervised learning, the learner must 
first discover its environment and is not told the opti¬ 
mal action it needs to take. To gain information about 
the rewards and the behavior of the system, the agent 
needs to explore by considering previously unused ac¬ 
tions or actions it is uncertain about. It needs to decide 
whether to play it safe and stick to well-known actions 
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with (moderately) high rewards or to dare trying new 
things in order to discover new strategies with an even 
higher reward. This problem is commonly known as 
the exploration-exploitation tradeoff. However, as state 
spaces in robotics problems are often tremendously 
large due to continuous and high-dimensional state- 
action spaces, exploring the entire state-action space is 
mostly infeasible. This problem is often referred to as 
the curse of dimensionality [15.168]. 

Off-policy methods learn independently of the 
employed policy, i. e., an explorative strategy that is 
different from the desired final policy can be employed 
during the learning process. On-policy methods collect 
sample information about the environment using the 
current policy. As a result, exploration must be built 
into the policy and determines the speed of the policy 
improvements. Such exploration and the performance 
of the policy can result in an exploration-exploitation 
tradeoff between long- and short-term improvement 
of the policy. Modeling exploration models with prob¬ 
ability distributions has surprising implications, e.g., 
stochastic policies have been shown to be the optimal 
stationary policies for selected problems [15.169, 
170] and can even break the curse of dimensional¬ 
ity [15.171], Furthermore, stochastic policies often 
allow the derivation of new policy update steps with 
surprising ease. 

The agent needs to determine a correlation between 
actions and reward signals. An action taken does not 
have to have an immediate effect on the reward but can 
also influence a reward in the distant future. The diffi¬ 
culty in assigning credit for rewards is directly related 
to the horizon or mixing time of the problem. It also in¬ 
creases with the dimensionality of the actions as not all 
parts of the action may contribute equally. 

15.3.3 Optimal Control with Learnt Models 

The classical RL setup is a MDP where, additionally, to 
the states s, actions s, and rewards R we also have tran¬ 
sition probabilities T(s ', a, s). Here, the reward is mod¬ 
eled as a reward function R(s,a). If both the transition 
probabilities and reward function are known, this can be 
seen as an optimal control problem [15.152]. If these 
functions are acquired from measured data through 
model learning methods as described in Sect. 15.2, we 
are in the scenario of optimal control with learnt models 
indicated in Fig. 15.1. 

The value function V n ( s ) provides an assessment of 
much reward will be accumulated starting from state s 
and following a given policy jr. If a policy is optimal, 
denoted as 7r*, the corresponding optimal value func¬ 
tion will be denoted a V 71 , often abbreviated as V*. 
For each state there can be one or several optimal 


actions a*, resulting in the same value V*(s). The con¬ 
sistency condition for optimality is 


V* (s) = max 

a 


R(s,a) + y ^ V* (s')T(s.a.s') 

s' 

(15.11) 


in the discounted case. This statement is equivalent to 
the Bellman principle of optimality [15.168] that states: 

An optimal policy has the property that whatever 
the initial state and initial decision are, the remain¬ 
ing decisions must constitute an optimal policy with 
regard to the state resulting from the first decision. 

Thus, we have to perform an optimal action a *, and, 
subsequently, follow the optimal policy jc * in order to 
achieve a global optimum. This principle of optimality 
has also been crucial in enabling the field of optimal 
control [15.172]. Note that for constructing the opti¬ 
mal policy, the max operator in (15.11) needs to be 
evaluated, which requires knowledge of the state transi¬ 
tion model T(s,a,s'), i. e., this approach is generically 
a model-based requiring either a predefined model or 
a learnt one using insights from Sect. 15.2. 

Dynamic programming-based methods are the clas¬ 
sical approach for solving optimal control problems in 
RL. These require, as mentioned before, a model of 
the transition probabilities T(s',a,s) and the reward 
function R(s,a) to calculate the value function. These 
functions do not necessarily need to be predetermined 
but can also be learned from data, potentially incre¬ 
mentally. Such methods are called model-based RL 
methods or optimal control with learnt models. Typical 
methods include policy iteration and value iteration. 

Policy iteration alternates between the two phases 
of policy evaluation and policy improvement. The ap¬ 
proach is initialized with an arbitrary policy. Policy 
evaluation determines the value function for the cur¬ 
rent policy. Each state is visited and its value is updated 
based on the current value estimates of its successor 
states, the associated transition probabilities, as well as 
the policy. This procedure is repeated until the value 
function converges to a fixed point, which approximates 
the true value function. Policy improvement greedily 
selects the best action in every state according to the 
value function, thus creating a policy update. The two 
steps of policy evaluation and policy improvement are 
iterated until the policy does not change any longer. 

Policy iteration only updates the policy once the 
policy evaluation step has converged. In contrast, value 
iteration combines the steps of policy evaluation and 
policy improvement by directly updating the value 
function based on (15.1 1) every time a state is updated. 
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15.3.4 Model-Free 

Value Function Approaches 

Many traditional RL approaches are based on identi¬ 
fying (possibly approximate) solutions to this equation, 
and are known as value function methods. Instead of de¬ 
riving the value functions from a transition model and 
a reward function, they first approximate the value func¬ 
tion, and use it to construct the optimal policy. 

Instead of the value function V 7r (s), these algo¬ 
rithms usually rely on the state-action value function 
Q 71 (s, a), which has advantages of determining the op¬ 
timal policy, as shown below, but also enables the use of 
model-free methods. This state-action value function is 
defined as 

Q 71 (s.a) = R(s,a) + y ^ V 71 (s')T(s, a, s'). 

s' 

In contrast to the value function V 71 (s), the state-action 
value function Q 77 (s.a) explicitly contains the informa¬ 
tion about the effects of a particular action. The optimal 
state-action value function is 

Q*(s.a ) = R(s,a) 

+ y ^ ^max<2*(.s / ,a / )^ T(s,a.s') . 

If Q* is known, finding the optimal action just requires 
finding the maximal value of Q at a given state s - 
this does not require a model, but is just a standard (al¬ 
though not necessarily trivial) root-finding problem of 
functions. Importantly, using state-action value func¬ 
tion allows a model-free learning approach, at the cost 
of learning a value function over the joint space of s 
and a. 

If the optimal value function V* (s') and the transi¬ 
tion probabilities T(s,a,s') for the following states are 
known, determining the optimal policy is straightfor¬ 
ward in a setting with discrete actions, as an exhaustive 
search is possible. For continuous spaces, determining 
the optimal action a* is an optimization problem in it¬ 
self. If both states and actions are discrete, the value 
function and the policy may, in principle, be represented 
by tables and picking the appropriate action is reduced 
to a look-up. For large or continuous spaces, represent¬ 
ing the value function as a table becomes intractable. 
Function approximation is employed to find a lower 
dimensional representation that matches the real value 
function as closely as possible, as discussed later. 

A wide variety of methods of model-free value 
function based RL algorithms that attempt to estimate 
V* (s) or Q* (s, a) have been developed and can be split 
mainly into two classes: (i) rollout-based Monte Carlo 


methods and (ii) temporal difference methods such as 
TD(A), g-learning, and SARSA (state-action-reward- 
state-action). 

Monte Carlo methods use sampling in order to 
estimate the value function. This procedure can be 
used to replace the policy evaluation step of the 
dynamic programming-based methods above. Monte 
Carlo methods are model-free, i. e., they do not need 
an explicit transition function. They perform roll-outs 
by executing the current policy on the system, hence 
operating on-policy. The frequencies of transitions and 
rewards are kept track of and are used to form estimates 
of the value function. For example, in an episodic set¬ 
ting the state-action value of a given state-action pair 
can be estimated by averaging all the returns that were 
received when starting from them. 

Temporal difference methods, unlike Monte Carlo 
methods, do not have to wait until an estimate of the 
return is available (i. e., at the end of an episode) to up¬ 
date the value function. Rather, they use temporal errors 
and update at every time step. The temporal error is the 
difference between the old estimate and a new estimate 
of the value function, taking into account the reward re¬ 
ceived in the current sample. These updates are done 
iterativley and, in contrast to dynamic programming 
methods, only take into account the sampled succes¬ 
sor states rather than the complete distributions over 
successor states. Like the Monte Carlo methods, these 
methods are model-free, as they do not use a model of 
the transition function to determine the value function. 
For example, the value function could be updated itera¬ 
tively by 

V'(s) = F(s) + a [R(s, a) + yV(s') — V(s)] , 

where V(s) is the old estimate of the value function, 
V'(s) the updated one, and a is a learning rate. This 
update step is called the TD(0)-algorithm in the dis¬ 
counted reward case. However, in order to select the 
optimal action, a model of the transition function would 
still be required, i. e., this method can only be used for 
policy evaluation in a policy iteration scenario. 

The equivalent temporal difference learning algo¬ 
rithm for state-action value functions is SARSA with 

Q' (s.a) = Q(s.a) 

+ a [R(s, a) + yQ(s', a') - Q(s, a)]. 

where Q(s.a) is the old estimate of the state-action 
value function and Q'(s,a) the updated one. This al¬ 
gorithm is on-policy as both the current action a as 
well as the subsequent action a' are chosen according 
to the current policy jr. The average-reward variant is 
called R-learning [15.173], which is closely related to 
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2-learning, with the updates 
Q'(s,a) = Q(s,a ) 


+ a 


R(s,a) + y max Q(s ', a') — Q(s,a) 


These methods do not require a model of the transition 
function for determining the deterministic optimal pol¬ 
icy jr*(s). 


15.3.5 Policy Search 


optimization methods (such as Nelder-Mead [15.150], 
cross entropy [15.188] and population-based meth¬ 
ods [15.189]), and approaches coming from opti¬ 
mal control such as differential dynamic program¬ 
ming or DDP [15.153] and multiple shooting ap¬ 
proaches [15.190], 

Let us briefly take a closer look at gradient-based 
approaches first. The updates of the policy parameters 
are based on a hill-climbing approach, which is follow¬ 
ing the gradient of the expected return 7 for a defined 
step-size a 


Value function approaches infer the optimal policy in¬ 
directly by first finding a value function, and then com¬ 
puting the corresponding optimal actions. Instead, the 
policy can be learned directly, without a detour through 
a value function. For robotics, this approach has sev¬ 
eral advantages. It allows for a natural integration of 
expert knowledge, e.g., through both structure and ini¬ 
tializations of the policy. It allows domain-appropriate 
prestructuring of the policy in an approximate form 
without changing the original problem. Optimal poli¬ 
cies often have many fewer parameters than optimal 
value functions. For example, in linear quadratic con¬ 
trol, the value function has quadratically many pa¬ 
rameters in the dimensionality of the state-variables, 
while the policy requires only linearly many param¬ 
eters. Local search in policy space can directly lead 
to good results as exhibited by early hill-climbing ap¬ 
proaches [15.172], as well as more recent successes, 
[15.149, 150,157,159-163, 174-182], Additional con¬ 
straints can be incorporated naturally, e.g., regularizing 
the change in the path distribution. As a result, policy 
search often appears more natural to robotics. 

Policy search has been considered the harder prob¬ 
lem for a long time as the optimal solution most often 
cannot directly be determined from equations as in 
Bellman principle of optimality [15.168] and dynamic 
programming. Nevertheless, in robotics, policy search 
has recently become an important alternative to value 
function based methods due to better scalability as 
well as the convergence problems of approximate value 
function methods. Most policy search methods opti¬ 
mize locally around existing policies jr, parameterized 
by a set of policy parameters 9,, by computing changes 
in the policy parameters A9 t that will increase the ex¬ 
pected return and results in iterative updates of the form 

0i +1 = 0i + Adi ■ 

The computation of the policy update is the key 
step here and a variety of updates have been 
proposed, including pairwise comparisons [15.183, 
184], gradient estimation using finite policy differ¬ 
ences [15.159,160,180,185-187], general stochastic 


&i+i = 9i + aVgiT . 

Different methods exist for estimating the gradient VqJ 
and many algorithms require tuning of the step-size a. 

Finite difference gradient estimators evaluate P pol¬ 
icy parameter perturbations to obtain an estimate of the 
policy gradient. Here, we have A J p « 7(0; + A 9 p ) 
where p = [1,... , P] are the individual perturbations, 
AJ p the estimate of their influence on the return, and 
7 re f is a reference return, e.g.. the return of the unper¬ 
turbed parameters. The gradient can now be estimated 
by linear regression 

V@7 « (A0 t A 0) H A0 t AJ , 

where the matrix A0 contains all the stacked samples 
of the perturbations A 9 P and AJ contains the corre¬ 
sponding AJ /; . In order to estimate the gradient the 
number of perturbations needs to be at least as large 
as the number of parameters. The approach is very 
straightforward and even applicable to policies that are 
not strictly differentiable. However, it is usually con¬ 
sidered to be very noisy and inefficient. For the finite 
difference approach tuning the step-size a for the up¬ 
date, the number of perturbations P, and the type and 
magnitude of perturbations are all critical tuning fac¬ 
tors. 

Likelihood ratio methods rely on the insight that 
in an episodic setting where the episodes r are gener¬ 
ated according to the distribution P 9 (r) = P (r|0) with 
the return of an episode J T = ff!h= 1 ^/i an d number of 
steps H in each episode, the expected return for a set of 
policy parameter 9 can be expressed as 

J 9 =J2 pd ( r ) jT ■ (15.12) 

T 

The gradient of the episode distribution can be written 
as 


V e P 9 (r) = P d (r)V e log PV), 


(15.13) 
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which is commonly known as the likelihood ratio or 
REINFORCE (reward increment = nonnegative fac¬ 
tor x offset reinforcement x characteristic eligibil¬ 
ity) [15.191] trick. From multivariate calculus, we have 
V e logP e (r) = N e P e (r)/P e (T). Combining (15.12) 
and (15.13) we get the gradient of the expected return 
in the form 

VoJ 8 =J2 y oP e (r) jT 

T 

= Y J P d (r)V e logP e (r)/ T 

T 

= Zt{v e logP e (r)7 T [. 

If we have a stochastic policy (a\s) that gener¬ 
ates the episodes r, we do not need to keep track of 
the probabilities of the episodes but can directly express 
the gradient in terms of the policy as Vg logP^r) = 
J2h=i ^9 log 7T^ («/, |s/,). Finally the gradient of the ex¬ 
pected return with respect to the policy parameters can 
be estimated as 

y 6 J e = e j -/ T J • 

If we now take into account that rewards at the begin¬ 
ning of an episode cannot be caused by actions taken at 
the end of an episode, we can replace the return of the 
episode J T by the state-action value function Q n (s, a) 
and get [15.161] 

y 6 J e = (1 -y) 

xE \y^,y h ^9 logn 9 (a,,\s h )Q n (si„ah)\ , 

[k= 1 ) 

which is equivalent to the policy gradient theo¬ 
rem [15.169]. In practice, it is often advisable to sub¬ 
tract a reference 7 re f, also called baseline, from the re¬ 
turn of the episode J T or the state-action value function 
Q n ( s,a ), respectively, to get better estimates, similar to 
the finite difference approach. In these settings, the ex¬ 
ploration is automatically taken care of by the stochastic 
policy. 

Initial gradient-based approaches such as finite dif¬ 
ferences gradients or REINFORCE [15.191] have been 
rather slow. The weight perturbation algorithm is re¬ 
lated to REINFORCE but can deal with non-Gaussian 
distributions which significantly improves the signal- 
to-noise ratio of the gradient [15.180]. Recent natural 
policy gradient approaches [15.161, 162] have allowed 
for faster convergence which may be advantageous for 


robotics as it reduces the learning time and required 
real-world interactions. 

Alternatives to gradient methods are currently a hot 
research topic. For example, a different class of safe 
and fast policy search methods, inspired by expectation- 
maximization from machine learning, can be derived 
when the reward is treated as an improper probability 
distribution [15.192]. Some of these approaches have 
proven successful in robotics, e.g., reward-weighted 
regression [15.161], policy learning by weighting ex¬ 
ploration with the returns [15.163], Monte Carlo ex¬ 
pectation maximization [15.193], and cost-regularized 
kernel regression [15.194]. Algorithms with closely 
related update rules can also be derived from differ¬ 
ent perspectives including policy improvements with 
path integrals [15.195] and relative entropy policy 
search [15.175]. 

Finally, the policy search by dynamic program¬ 
ming [15.196] method is a general strategy that com¬ 
bines policy search with the principle of optimality. 
The approach learns a nonstationary policy backward 
in time like dynamic programming methods, but does 
not attempt to enforce the Bellman equation and the 
resulting approximation instabilities. The resulting ap¬ 
proach provides some of the strongest guarantees that 
are currently known under function approximation and 
limited observability. It has been demonstrated in learn¬ 
ing walking controllers and in finding near-optimal 
trajectories for map exploration [15.197], 

One of the key open issues in the field is deter¬ 
mining when it is appropriate to use each of these 
methods. Some approaches leverage significant struc¬ 
ture specific to the RL problem [15.195], including 
reward structure, Markovanity, causality of reward sig¬ 
nals [15.191], and value-function estimates when avail¬ 
able [15.161]. Others embed policy search as a generic, 
black-box, problem of stochastic optimization [15.150, 
198-201]. Significant open questions remain regarding 
which methods are best in which circumstances and 
further, at an even more basic level, how effective lever¬ 
aging the kinds of problem structures mentioned above 
are in practice. Examples of resulting success can be 
observed in and I^MTEESSJ. 

Value Function Approaches 

Versus Policy Search 

The discussion whether value function approaches or 
policy search methods are more suitable for RL has 
been going on for a long time, and, so far, the answer 
is really problem specific. If a complete optimal value 
function is known, a globally optimal solution follows 
by greedily choosing actions that receive the highest 
value. However, value-function-based approaches have 
up to now been difficult to scale to high-dimensional 
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robotics as they require function approximation for 
the value function - in low dimensions, discretizations 
and lookup tables work best and have no theoretical 
issues. Unfortunately, theoretical guarantees from dis¬ 
crete state-action representations no longer hold when 
employing function approximation. Even finding the 
optimal action can be a hard problem due to the sen¬ 
sitivity of selecting the optimal action in face of even 
tiny inaccuracies in the approximated value function, 
and the cost of solving an optimization problem in 
a continuous unknown function for action selection. For 
example, the computational cost of updating a policy 
in policy search can easily be cheaper than finding just 
one optimal action for one state by searching the state- 
action value function. 

In principle, a value function requires total cover¬ 
age over the state space and the largest local error 
determines the quality of the resulting policy. A par¬ 
ticularly significant problem is the error propagation 
in value functions. A small change in the policy may 
cause a large change in the value function, which again 
causes a large change in the policy. While this may 
lead more quickly to good, possibly globally optimal 
solutions, such learning processes often prove unstable 
under function approximation [15.196,202,203] and 
are considerably more dangerous when applied to real 
robotic systems where overly large policy deviations 
may lead to dangerous motor commands. 

In contrast, policy search methods usually only con¬ 
sider the current policy and its neighborhood in order to 
gradually improve performance. The result is that usu¬ 
ally only local optima, and not the global one, can be 
found. However, these methods work well in conjunc¬ 
tion with continuous states and actions, and they scale 
to high-dimensional robots, e.g., humanoid robots. 

Policy search methods are sometimes called actor- 
only methods; value function methods are sometimes 
called critic-only methods. The idea of a critic is to first 
observe and estimate the performance of choosing con¬ 
trols on the system (i. e., the value function), then derive 
a policy based on the gained knowledge. In contrast, the 
actor directly tries to deduce the optimal policy. A set 
of algorithms called actor-critic methods attempt to in¬ 
corporate the advantages of each: a policy is explicitly 
maintained, as is a value function for the current pol¬ 
icy. The value function (i. e., the critic) is not employed 
for action selection. Instead, it observes the perfor¬ 
mance of the actor and decides when the policy needs 
to be updated and which action should be preferred. 
The resulting update step features the local convergence 
properties of policy gradient algorithms while reducing 
update variance [15.204]. There is a tradeoff between 
the benefit of reducing the variance of the updates and 
having to learn a value function as the samples required 


to estimate the value function could also be employed 
to obtain better gradient estimates for the update step. 
Rosenstein and Barto [15.205] propose an actor-critic 
method that additionally features a supervisor in the 
form of a stable policy. 

Function Approximation 

The topic of function approximation occurs repeatedly 
in RL and deserves some attention. For discrete state- 
action problems with not too many states and actions, 
look-up tables are a reasonable representation for value 
functions, rewards functions, and state-transitions (i. e., 
models). But look-up tables have no generalization to 
neighboring states, which makes them computation¬ 
ally inefficient. Moreover, for continuous state-action 
problems with many dimensions, discretization would 
create an exponential growth of the number of dis¬ 
crete states and actions, and is thus infeasible ( curse of 
dimensionality). Thus, replacing discretized representa¬ 
tions by continuous function approximators is the only 
possibility. 

Function approximation can be employed to repre¬ 
sent policies, value functions, and state-transition mod¬ 
els. Thus, the techniques discussed in Sect. 15.2.4 also 
apply to these function approximation problems. How¬ 
ever, a fundamental problem in using supervised learn¬ 
ing methods developed in the literature for function 
approximation is that most such methods are designed 
for independently and identically distributed sample 
data. However, the data generated by the RL process is 
usually neither independent nor identically distributed. 
Usually, the function approximator itself plays some 
role in the data collection process (e.g., by serving to 
define a policy that we execute on a robot.) 

Linear basis function approximators form one of 
the most widely used approximate value function tech¬ 
niques in continuous (and discrete) state spaces. This 
popularity is largely due to the simplicity of their repre¬ 
sentation as well as a convergence theory, albeit limited, 
for the approximation of value functions based on sam¬ 
ples [15.206]. Let us briefly take a closer look at a radial 
basis function network to illustrate this approach. The 
value function maps states to a scalar value. The state 
space can be covered by a grid of points, each of which 
correspond to the center of a Gaussian-shaped basis 
function. The value of the approximated function is 
the weighted sum of the values of all basis functions 
at the query point. As the influence of the Gaussian 
basis functions drops rapidly with the distance of its 
center, the value of the query points will be predom¬ 
inantly influenced by the neighboring basis functions. 
The weights are set in a way to minimize the error 
between the observed samples and the reconstruction. 
For the mean squared error, these weights can be de- 
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termined by linear regression. Kolter and Ng [15.207] 
discuss the benefits of regularization of such linear 
function approximators to avoid over-fitting. 

Other possible function approximators for value 
functions include wire fitting, which Baird and 
Klopf [15.208] suggested as an approach that makes 
continuous action selection feasible. Using Fourier ba¬ 
sis has been suggested by Konidaris et al. [15.209]. 
Even discretizing the state space can be seen as a form 
of function approximation where coarse values serve 
as estimates for a smooth continuous function. One 
example is tile coding [15.13], where the space is sub¬ 
divided into (potentially irregularly shaped) regions, 
called tiling. The number of different tilings determines 
the resolution of the final approximation. Policy search 
also benefits from compact representations of the pol¬ 
icy. 

Models of the system dynamics can be represented 
using a wide variety of techniques. In this case, it is 
often important to model the uncertainty in the model 
(e.g., by a stochastic model or Bayesian estimates of 
model parameters) to ensure that the learning algorithm 
does not exploit model inaccuracies. See Sect. 15.3.9 
for a more detailed discussion. 

15.3.6 Challenges in Robot Reinforcement 
Learning 

Reinforcement learning is generally a difficult problem 
and many of its challenges are particularly apparent in 
the robotics setting. As the states and actions of most 
robots are inherently continuous, we are forced to con¬ 
sider the resolution at which they are represented. We 
must decide how fine grained the control is that we 
require over the robot, whether we employ discretiza¬ 
tion or function approximation, and what time step we 
establish. Additionally, as the dimensionality of both 
states and actions can be high, we face the Curse of 
Dimensionality [15.168] as discussed in Sect. 15.3.6, 
Curse of Dimensionality. As robotics deals with com¬ 
plex physical systems, samples can be expensive due 
to the long execution time of complete tasks, required 
manual interventions, and the need for maintenance 
and repair. In these real-world measurements, we must 
cope with the uncertainty inherent in complex physi¬ 
cal systems. A robot requires that the algorithm runs 
in real-time. The algorithm must be capable of dealing 
with delays in sensing and execution that are inherent in 
physical systems (Sect. 15.3.6). A simulation might al¬ 
leviate many problems but these approaches need to be 
robust with respect to model errors as discussed later. 
An often underestimated problem is the goal specifi¬ 
cation, which is achieved by designing a good reward 
function. As noted in Sect. 15.3.6, this choice can make 


the difference between feasibility and an unreasonable 
amount of exploration. 

Curse of Dimensionality 

When Bellmann [15.168] explored optimal control in 
discrete high-dimensional spaces, he faced an exponen¬ 
tial explosion of states and actions for which he coined 
the term Curse of Dimensionality. As the number of di¬ 
mensions grows, exponentially more data and computa¬ 
tion is needed to cover the complete state-action space. 
Evaluating every state quickly becomes infeasible with 
growing dimensionality, even for discrete states. 

Robotic systems often have to deal with these high¬ 
dimensional states and actions due to the many degrees 
of freedom of modern anthropomorphic robots. For ex¬ 
ample, in a ball-paddling task (I'ScJtiltliliUI ) the proper 
representation of a robot’s state would consist of its 
joint angles and velocities for each of its seven degrees 
of freedom as well as the Cartesian position and veloc¬ 
ity of the ball (Fig. 15.7). The robot’s actions would be 
the generated motor commands, which often are torques 
or accelerations. In this example, we have 2 x (7 + 3) = 
20 state dimensions and seven-dimensional continuous 
actions. 

In robotics, such tasks are often rendered tractable 
to the robot engineer by a hierarchical task decom¬ 
position that shifts some complexity to a lower layer 
of functionality. Classical RL approaches often con¬ 
sider a grid-based representation with discrete states 
and actions, often referred to as a grid-world. A navi¬ 
gational task for mobile robots could be projected into 



Fig. 15.7 State-space used in the modeling of a robot RL 
task of paddling a ball. This example can also be observed 

in k3*M£ISl 
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this representation by employing a number of actions 
like move to the cell to the left that use a lower level 
controller that takes care of accelerating, moving, and 
stopping while ensuring precision. In the ball-paddling 
example, we may simplify by controlling the robot in 
racket space (which is lower dimensional as the racket 
is orientation-invariant around the string’s mounting 
point) with an operational space control law [15.15]. 
Many commercial robot systems also encapsulate some 
of the state and action components in an embedded con¬ 
trol system (e.g., trajectory fragments are frequently 
used as actions for industrial robots). However, this 
form of a state dimensionality reduction severely lim¬ 
its the dynamic capabilities of the robot according to 
our experience [15.63, 210]. 

The RL community has a long history of dealing 
with dimensionality using computational abstractions. 
It offers a larger set of applicable tools ranging from 
adaptive discretizations [15.211] and function approx¬ 
imation approaches [15.13] to macro-actions or op¬ 
tions [15.212,213]. Options allow a task to be decom¬ 
posed into elementary components and quite naturally 
translate to robotics. Such options can autonomously 
achieve a subtask, such as opening a door, which re¬ 
duces the planning horizon [15.212]. The automatic 
generation of such sets of options is a key issue in order 
to enable such approaches. We will discuss approaches 
that have been successful in robot RL in Sect. 15.3.7. 

Curse of Real-World Samples 
Robots inherently interact with the physical world and, 
thus, create their own learning data from physical sen¬ 
sors. Real physical systems create a variety of problems 
that are worthwhile to keep in mind: 

• Robot hardware is usually expensive, suffers from 
wear and tear, and requires careful maintenance. 
Repairing a robot system is a nonnegligible ef¬ 
fort associated with cost, physical labor, and long 
waiting periods. To apply RL in robotics, safe ex¬ 
ploration becomes a key issue of the learning pro¬ 
cess [15.157,214-216]. 

• The robot and its environment may continually 
change slowly, e.g., due to wear and tear, or light 
conditions. Thus, the learning process should be 
able to track these changing conditions, i. e., learn 
continually. 

• Experimentation with real robots is time consum¬ 
ing, e.g., putting the pole back on the robot’s end- 
effector during pole balancing after a failure, and 
running many trials for learning. Algorithms that 
are data efficient are often more important than 
memory and computations needed to process the 
data. 


• Real-time constraints on an actual robot impose 
constraints for how much computation can be per¬ 
formed for action generation and learning updates. 
These constraints are less severe in an episodic set¬ 
ting where the time intensive part of the learning can 
be postponed to the period between episodes. Hes¬ 
ter et al. [15.217] have proposed a real-time archi¬ 
tecture for model-based value function RL methods 
taking into account these challenges. 

• Time-discretization, inevitable in computer imple¬ 
mentations, can generate undesirable artifacts, e.g., 
the distortion of distance between states, or smooth¬ 
ing effect by temporal aliasing. 

• Delays in information processing with physical sen¬ 
sors and actuators can negatively affect learning. 
This effect can be addressed by putting some num¬ 
ber of recent actions into the state, at the cost of 
significantly increasing the dimensionality of the 
problem. 

Curse of Under-Modeling 
and Model Uncertainty 

One way to offset the cost of real-world interaction is 
to use accurate models as simulators as suggested in 
Sect. 15.2.5. In an ideal setting, this approach would 
render it possible to learn the behavior in simulation and 
subsequently transfer it to the real robot. Unfortunately, 
creating a sufficiently accurate model of the robot and 
its environment is often impossible. As small modeling 
errors accumulate over time, the simulated robot can 
quickly diverge from the real-world system, such that 
the policy will not transfer without significant modi¬ 
fications as experienced by [15.218]. For tasks where 
the system is self-stabilizing (i. e., where the robot does 
not require active control to remain in a safe state or 
return to it), transferring policies often works much bet¬ 
ter [15.219]. Learning in simulation, however, becomes 
quickly useless when contact dynamics and friction are 
too hard to model. For example, in the ball-paddling 
task the elastic string that attaches the ball to the racket, 
and the contact dynamics of the bouncing ball have such 
modeling problems. 

Curse of Goal Specification 
In RL, the desired behavior is implicitly specified by the 
reward function. While often dramatically simpler than 
specifying the behavior itself, in practice, it can be sur¬ 
prisingly difficult to define a good reward function, such 
that manual design and tuning become very important. 
In many domains, it seems natural to provide rewards 
only upon task achievement - for example, when a table 
tennis robot wins a match. This view results in an ap¬ 
parently simple, binary reward specification. However, 
a robot may receive such a reward so rarely that it is un- 
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likely to ever succeed in its lifetime. Instead of relying 
on simpler binary rewards, we frequently need to in¬ 
clude intermediate rewards in the scalar reward function 
to guide the learning process to a reasonable solution, 
a process known as reward shaping [15.156]. Addition¬ 
ally, physical constraints of the robot in terms of range 
of motion and torque saturation need to be taken into ac¬ 
count. Often, a change in coordinates, i. e., the question 
in which space (e.g., task space, joint space) the cost 
function should be specified, can determine success or 
failure. In general, RL algorithms are also notorious for 
exploiting the reward function in ways that are not an¬ 
ticipated by the designer. 

Inverse RL, also known as inverse optimal con¬ 
trol [15.220], is a promising alternative to specifying 
the reward function manually. It assumes that a re¬ 
ward function can be reconstructed from a set of expert 
demonstrations as indicated in Fig. 15.1. This reward 
function does not necessarily correspond to the true 
reward function, but provides guarantees on the re¬ 
sulting performance of learned behaviors [15.221, 222]. 
Inverse RL was initially studied in the control com¬ 
munity [15.8] and in the field of economics [15.223]. 
The initial results were only applicable to limited 
domains (linear quadratic regulator problems) and re¬ 
quired closed form access to plant and controller, hence 
samples from human demonstrations could not be used. 
Russel [15.220] brought the field to the attention of the 
machine learning community. Abbeel and Ng [15.221] 
defined an important constraint on the solution to the 
inverse RL problem when reward functions are linear 
in a set of features: a policy that is extracted by observ¬ 
ing demonstrations has to earn the same reward as the 
policy that is being demonstrated. Ratliff et al. [15.222] 
demonstrated that inverse optimal control can be un¬ 
derstood as a generalization of ideas in machine learn¬ 
ing of structured prediction and introduced efficient 
subgradient-based algorithms with regret bounds that 
enabled large scale application of the technique within 
robotics. Ziebart et al. [15.2] extended the technique 
developed by Abbeel and Ng [15.221] by rendering 
the idea robust and probabilistic, enabling its effec¬ 
tive use for both learning policies and predicting the 
behavior of suboptimal agents. These techniques, and 
many variants, have been recently successfully applied 
to outdoor robot navigation [15.224—226], manipula¬ 
tion [15.227], and quadruped locomotion [15.224,227, 
228]. An example of a resulting success can be ob¬ 
served in kaMSEEISI. 

More recently, the notion that complex policies 
can be built on top of simple, easily solved optimal 
control problems by exploiting rich, parametrized re¬ 
ward functions has been exploited within RL more 
directly. In [15.229,230], complex policies are derived 


by adapting a reward function for simple optimal con¬ 
trol problems using policy search techniques. Zucker 
and Bagnell [15.230] demonstrate that this technique 
can enable efficient solutions to robotic marble-maze 
problems that effectively transfer between mazes of 
varying design and complexity. These works highlight 
the natural tradeoff between the complexity of the re¬ 
ward function and the complexity of the underlying RL 
problem for achieving a desired behavior. 

15.3.7 Tractability Through Representation 

Much of the success of RL methods has been due 
to the clever use of approximate representations. The 
need of such approximations is particularly pronounced 
in robotics, where table-based representations (as dis¬ 
cussed in Sect. 15.3.4) are rarely scalable. Common 
themes are to reduce the dimensionality of states and 
actions, to choose representations that are robust in face 
of the data generation process of RL, or to find compact 
parameterizations of policies and value functions. The 
following outlines a list of items that are relevant: 

• A variety of authors have manually developed dis¬ 
cretizations so that basic tasks can be learned on 
real robots. For low-dimensional tasks, we can gen¬ 
erate discretizations straightforwardly by splitting 
each dimension into a number of regions. The 
main challenge is to find the right number of re¬ 
gions for each dimension that allows the system 
to achieve a good final performance while still 
learning quickly. Example applications include bal¬ 
ancing a ball on a beam [15.231], one degree of 
freedom ball-in-a-cup [15.232], two degree of free¬ 
dom crawling motions [15.233], and gait patterns 
for four-legged walking [15.234]. Complex tasks 
need much more hand crafting, e.g., as in naviga¬ 
tion [15.235], vision-based processing [15.236], or 
RoboCup scenarios [15.237], 

• Adaptive representations can be very useful, e.g., as 
in cooperative task achievement [15.238], or com¬ 
puter vision [15.239]. 

• Automatic construction of meta-actions (and the 
closely related concept of options) has fascinated 
RL researchers. The idea is to have more intelligent 
actions that are composed of a sequence of com¬ 
mands and that in themselves achieve a simple task. 
A simple example would be to have a meta-action 
move forward 5 m. For example, in [15.240], the 
state and action sets are constructed in such a way 
that repeated action primitives lead to a change in 
the state to overcome problems associated with the 
discretization. Huber and Grupen [15.241] use a set 
of controllers with associated predicate states as 
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a basis for learning turning gates with a quadruped. 
Fidelman and Stone [15.242] use a policy search 
approach to learn a small set of parameters that 
controls the transition between a walking and a cap¬ 
turing meta-action in a RoboCup scenario. A task 
of transporting a ball with a dog robot [15.243] can 
be learned with semiautomatically discovered op¬ 
tions. Using only the subgoals of primitive motions, 
a humanoid robot can learn a pouring task [15.244]. 
Other examples include foraging [15.245] and 
cooperative tasks [15.246] with multiple robots, 
grasping with restricted search spaces [15.247], 
and mobile robot navigation [15.248]. If the meta¬ 
actions are not fixed in advance, but rather learned 
at the same time, these approaches are hierarchical 
RL approaches. Konidaris et al. [15.249,250] pro¬ 
pose an approach that constructs a skill tree from 
human demonstrations. Here, the skills correspond 
to options and are chained to learn a mobile manip¬ 
ulation skill. 

• In a relational representation, the states, actions, and 
transitions are not represented individually. Enti¬ 
ties of the same predefined type are grouped and 
their relationships are considered. This representa¬ 
tion may be preferable for highly geometric tasks 
(which frequently appear in robotics) and has been 
employed to learn to navigate buildings with a real 
robot in a supervised setting [15.251] and to manip¬ 
ulate articulated objects in simulation [15.252]. 

• Function approximation has always been the key 
component that allowed value function methods to 
scale into interesting domains. Unfortunately, the 
max-operator used within the Bellman equation and 
temporal-difference updates can theoretically make 
most linear or nonlinear approximation schemes 
unstable for either value iteration or policy itera¬ 
tion. Quite frequently such an unstable behavior 
is also exhibited in practice. Linear function ap¬ 
proximators are stable for policy evaluation, while 
nonlinear function approximation (e.g., neural net¬ 
works) can even diverge if just used for policy 
evaluation [15.206]. A large number of approaches 
have been suggested to facilitate value function ap¬ 
proximation. For instance, creating physics-inspired 
features as nonlinear basis functions for linear ap¬ 
proximator [15.151,253] can be promising. Generic 
neural networks can be successful [15.254,255], 
in particular the CMAC (cerebellar model articu¬ 
lation controller) neural network [15.256]. Local 
model function approximators often provide more 
stable learning behavior [15.151,257,258]. Proba¬ 
bilistic nonparametric function approximators like 
Gaussian process regression has recently shown in¬ 
teresting successes in robot RL [15.259-262]. 


• Similar to choosing a good structure in value func¬ 
tion approximation, choosing a good structure and 
parameterization for the policy can make a big dif¬ 
ferent for success or failure or RL. Finding sparse 
parameterization essentially reduces the dimension¬ 
ality of the learning problem, as only a few pa¬ 
rameters need to be learned, and not an action for 
every time step. Among the most popular have 
been spline-based parameterizations [15.158, 180, 
199], linear parameterizations based on physics- 
inspired features [15.263], parameterized motor 
primitives [15.161,162,264,265], Gaussian Mix¬ 
ture Models [15.157, 177,266], generic neural net¬ 
works [15.149,176,185], and nonparametric ap¬ 
proaches [15.175, 194]. 

15.3.8 Tractability Through Prior Knowledge 

Prior knowledge can dramatically help guide the learn¬ 
ing process. It can be included in the form of initial 
policies, demonstrations, initial models, a predefined 
task structure, or constraints on the policy such as 
torque limits or ordering constraints of the policy pa¬ 
rameters. These approaches significantly reduce the 
search space and, thus, speed up the learning pro¬ 
cess. Providing a (partially) successful initial policy 
allows a RL method to focus on promising regions 
in the value function or in policy space. Prestructur¬ 
ing a complex task such that it can be broken down 
into several more tractable ones can significantly re¬ 
duce the complexity of the learning task. Constraints 
may also limit the search space, but often pose new, 
additional problems for the learning methods. For ex¬ 
ample, policy search limits often do not handle hard 
limits on the policy well. Relaxing such constraints 
(a trick often applied in machine learning) is not fea¬ 
sible if they were introduced to protect the robot in the 
first place. 

As indicated in Fig. 15.1, observed expert data can 
be used for constructing useful policies. Using demon¬ 
strations to initialize RL has become rather popular and 
provides multiple benefits. Perhaps the most obvious 
benefit is that it provides supervised training data of 
what actions to perform in states that are encountered. 
Such data may be helpful when used to bias policy ac¬ 
tion selection. The most dramatic benefit, however, is 
that demonstration - or a hand-crafted initial policy - 
removes the need for global exploration of the pol¬ 
icy or state-space of the RL problem. The student can 
improve by locally optimizing a policy knowing what 
states are important, making local optimization meth¬ 
ods feasible. Of course, if the demonstration was not 
close to the globally optimal behavior, only local op¬ 
tima can be discovered. 
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Fig. 15.8 Boston Dynamics LittleDog 
jumping (after [15.178], courtesy of 
Zico Kolter) 



Demonstrations by a teacher can be obtained in 
two different scenarios. In the first, the teacher demon¬ 
strates the task using his or her own body; in the 
second, the teacher controls the robot to do the task. 
The first scenario is limited by the embodiment issue, 
as the movement of a human teacher usually cannot be 
mapped directly to the robot due to different physical 
constraints and capabilities. For example, joint angles 
of a human demonstrator need to be adapted to account 
for the kinematic differences between the teacher and 
the robot. Often it is more advisable to only consider 
task-relevant information, such as the Cartesian posi¬ 
tions and velocities of the end-effector and the object. 
Demonstrations obtained by motion capture have been 
used to learn a pendulum swingup [15.174], ball-in-a- 
cup [15.267], and grasping [15.260]. 

The second scenario obtains demonstrations by 
a human teacher directly controlling the robot. Here 
the human teacher first has to learn how to achieve 
a task with the particular robot’s hardware, adding valu¬ 
able prior knowledge. If the robot is back-drivable, 
kinesthetic teach-in (i. e., by taking it by the hand 
and moving it) can be employed, which enables 
the teacher to interact more closely with the robot. 
This method has resulted in applications including 
T-ball batting [15.161,162], reaching tasks [15.177, 
268], ball-in-a-cup [15.163] and laaMMfrU-U, flipping 
a light switch [15.269], playing pool and manipulating 
a box [15.270], and opening a door and picking up ob¬ 
jects [15.271]. A marble maze task can be learned using 
demonstrations by a human player [15.272], 

Often a task can be decomposed hierarchically into 
basic components or into a sequence of increasingly dif¬ 
ficult tasks. In both cases the complexity of the learning 
task is significantly reduced. For example, hierarchical 
2-learning has been used to learn different behavioral 
levels for a six-legged robot: moving single legs, lo¬ 
cally moving the complete body, and globally moving 


the robot toward a goal [15.273]. A stand-up behavior 
considered as a hierarchical RL task has been learned 
using (7-learning in the upper-level and a continuous ac¬ 
tor-critic method in the lower level [15.274]. Navigation 
in a maze can be learned using an actor-critic architec¬ 
ture by tuning the influence of different control modules 
and learning these modules [15.275]. Huber and Gru- 
pen [15.241] combine discrete event system and RL 
techniques to learn turning gates for a quadruped. Hart 
and Grupen [15.213] learned to bi-manual manipula¬ 
tion tasks by assembling policies hierarchically. Daniel 
et al. [15.276] learn options in a tetherball scenario 
and Muelling et al. [15.147] learn different strokes in 
a table tennis scenario (|4S>M2D3$I23i). Whitman and 
Atkeson [15.277] show that the optimal policy for some 
global systems (like a walking controller) can be con¬ 
structed by finding the optimal controllers for simpler 
subsystems and coordinating these. 

As discussed in Sect. 15.3.2, balancing exploration 
and exploitation is an important consideration. Task 
knowledge can be employed to guide to robots curiosity 
to focus on regions that are novel and promising at the 
same time. For example, a mobile robot learns to direct 
attention by employing a modified Q-learning approach 
using novelty [15.278]. Using corrected truncated re¬ 
turns and taking into account the estimator variance, 
a six-legged robot employed with stepping reflexes can 
learn to walk [15.279]. Offline search can be used to 
guide Q-learning during a grasping task [15.280]. Using 
upper confidence bounds [15.281] to direct exploration 
into regions with potentially high rewards, grasping can 
be learned efficiently [15.262]. 

15.3.9 Tractability Through Models 

In Sect. 15.3.2, we discussed robot RL from a model- 
free perspective where the system simply served as 
a data generating process. Such model-free reinforce- 
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ment algorithms try to directly learn the value function 
or the policy without any explicit modeling of the tran¬ 
sition dynamics. In contrast, many robot RL problems 
can be made tractable by learning forward models, i. e., 
approximations of the transition dynamics based on 
data. Such model-based RL approaches jointly learn 
a model of the system with the value function or the 
policy and often allow for training with less interac¬ 
tion with the the real environment. Reduced learning 
on the real robot is highly desirable as simulations are 
frequently faster than real-time while safer for both 
the robot and its environment. The idea of combining 
learning in simulation and in the real environment was 
popularized by the Dyna-architecture [15.282], prior¬ 
itized sweeping [15.283], and incremental multi-step 
Q-learning [15.284] in RL. In robot RL, the learning 
step on the simulated system is often called mental re¬ 
hearsal. 

A few remarks concerning model-based RL are im¬ 
portant. It is essentially impossible to obtain a forward 
model that is accurate enough to simulate a complex 
real-world robot system without error. Reinforcement 
learning approaches exploit such model inaccuracies 
if they are beneficial for the reward received in simu¬ 
lation [15.174]. The resulting policies may work well 
with the forward model (i. e., the simulator) but poorly 
on the real system. This effect is known as simulation 
bias. 

Nevertheless, simulation biases can be addressed 
by introducing stochastic models or distributions over 
models even if the system is very close to deter¬ 


15.4 Conclusions 

We focussed on the two key branches of robot learn¬ 
ing, i. e., supervised learning of internal models with 
regression methods, and trial-and-error with reinforce¬ 
ment learning. While both topics are traditional topics 
of machine learning, inserting these learning topics into 
robot learning adds a variety of complexities and in¬ 
tricacies, which are not immediately obvious - indeed 
many issues have been revealed only through experi¬ 
ments with physical robots. A large number of issues 
are control theoretic, i. e., due to dealing with a closed- 
loop physical system that needs to be stable. Other 
issues are due to the inherent constraints that a phys¬ 
ical system imposes, e.g., actuator saturation and the 
limits of the work space. High-dimensional continuous 
state-action space causes yet another set of problems, 
and particularly affects feasibly and efficiency of robot 
learning. Model-errors and senor noise can have ad¬ 


ministic. Artificially adding a little noise will smooth 
model errors and avoid policy over-fitting [15.153, 
285]. Model learning methods that maintain probabilis¬ 
tic uncertainty about true system dynamics allow the 
RL algorithm to generate distributions over the per¬ 
formance of a policy. Such methods explicitly model 
the uncertainty associated with the dynamics at each 
state and action. For example, when using a Gaus¬ 
sian process model of the transition dynamics, a policy 
can be evaluated by propagating the state and associ¬ 
ated uncertainty forward in time. Such evaluations in 
the model can be used by a policy search approach 
to identify where to collect more data to improve 
a policy, and may be exploited to ensure that con¬ 
trol is safe and robust to model uncertainty [15.150, 
214]. When the new policy is evaluated on the real 
system, the novel observations can subsequently be 
incorporated into the forward model. Bagnell and 
Schneider [15.150] showed that maintaining model un¬ 
certainty and using it in the inner-loop of a policy 
search method enabled effective flight control using 
only minutes of collected data, while performance was 
compromised by considering a best-fit model. This ap¬ 
proach uses explicit Monte Carlo simulation in the 
sample estimates. By treating model uncertainty as 
if it were noise [15.214] as well as employing ana¬ 
lytic approximations of forward simulation, a cart-pole 
task can be solved with less than 20 s of interac¬ 
tion with the physical system [15.157]; a visually 
driven block-stacking task has also been learned data- 
efficiently [15.164]. 


verse outcome on the learning process. And the fact that 
a robot needs to generate its own learning data, i. e., 
the exploration-exploitation tradeoff and ever chang¬ 
ing sampling distributions, creates a large number of 
problems. 

We note that there is no clear receipe for robot 
learning, but rather a large number of ingredients that, 
when used appropriately, can achieve excellent learn¬ 
ing results. As a caveat, even minor incorrect choices 
can completely prevent any success of learning. That 
robot learning has yet not reached maturity is obvi¬ 
ous from the fact that learning algorithms are rarely 
used on robots on a daily basis - most results are 
feasibility studies and not robust enough for daily 
use. Many results are also overtuned to a particu¬ 
lar physical robot, and do not generalize to others 
easily. 
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Some of the open questions for the future of robot 

learning include: 

• How to choose appropriate representations automat¬ 
ically for model learning, value function learning, 
and policy learning? Perhaps, there is one partic¬ 
ular choice that robustly works for many different 
robots, or maybe one could find a small set of pos¬ 
sible approaches that can easily be compared. 

• How to create useful reward functions? This topic 
connects to inverse RL, and on a higher level to how 
to understand the intentions of observed behavior. 

• How much can prior knowledge help? How much is 
needed? How should it be provided? 

• How to integrate more tightly with perception? 
Robot learning is largely action-centric and as¬ 
sumes that perception is provided. In reality, there 
is a perception-action-learning loop where the dif¬ 
ferent components interact significantly, and need 
to be jointly developed. 

• How to reduce parameter sensitivity? Manual tun¬ 
ing of hyperparameters such as gradient rates, for- 

Video-References 


getting rates, exploration rates, etc., are a common 
curse for the robot learning practitioner, and often, 
a small change in a parameter determines success or 
failure. 

• How to robustly deal with modeling errors? Mod¬ 
els are great when they work, but disastrous if they 
are not accurate enough. Probabilistic and robust 
control methods may help, but can also degrade per¬ 
formance due to very conservative learning results. 

This list is by no means exhaustive, but captures 
some of the key issues. In the end, there is strong 
need for researchers and scientists who are willing to 
tackle a complex mixture of theoretical problems and 
experimental problems. Often, just setting up an exper¬ 
imental robot environment with accurate and efficient 
debugging and visualization tools is a formidable ef¬ 
fort. Then, finding the right experiment and the right 
data trace that allows for error diagnosis in robot learn¬ 
ing is still a bit of an art and requires fairly deep insights 
into physics, algorithms, software architectures, and 
technology. 
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Inverted helicopter hovering 

available from http://handbookofrobotics.org/view-chapter/15/videodetails/352 
Inverse reinforcement 

available from http://handbookofrobotics.org/view-chapter/15/videodetails/353 
Machine learning table tennis 

available from http://handbookofrobotics.org/view-chapter/15/videodetails/354 
Learning motor primitives 

available from http://handbookofrobotics.org/view-chapter/15/videodetails/355 
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The chapters contained in Pa rt B, Design, are concerned 
with the design and modeling of the actual physical 
realizations of a robot. Some of the more obvious me¬ 
chanical structures that come to mind are arms, legs, 
and hands. To this list we can add wheeled vehicles 
and platforms; snake-like and continuum robots; robots 
capable of swimming and flying; and robot structures 
at the micro- and nanoscales. Even for that most ba¬ 
sic robotic device, the arm, an incredibly diverse set 
of structures is possible, depending on the number and 
types of joints and actuators, and the presence of closed 
loops in the kinematic structure, or flexibility in the 
joints and links. Constructing models, and planning and 
control algorithms for these diverse structures repre¬ 
sents an even greater set of challenges. 

The topics addressed in these chapters are essential 
to creating not only the physical robot itself, but also to 
creating and controlling movements, and manipulating 
objects in desired ways. As such the connections with 
the chapters on Robot Foundations (Part A) - particu¬ 
larly the chapters on Kinematics (Chap. 1), Dynamics 
(Chap. 2), and Mechanisms and Actuation (Chap. 3) - 
are self-evident. What ultimately distinguishes robotics 
from other disciplines that study intelligence is that, 
by definition, robots require a physical manifestation, 
and by extension must physically interact with the en¬ 
vironment. In this regard the topics addressed in these 
chapters can be said to constitute the most basic layer 
of this endeavor. 

Just as it is difficult to examine human intelligence 
from a purely abstract perspective, remotely detached 
from the physical body, so it is difficult to separate the 
contents of the remaining parts without including in the 
discussion the actual medium of interaction with the 
physical world, the (physical) robots themselves. For 
example, the question of how to coordinate sensing and 
perception with action (Part C), how to grasp and ma¬ 
nipulate objects (Part D), and how to teach robots to 
move in the world (Part E), must inevitably consider 
the physical structure of the robot. Robots specialized to 
various applications and environments (Part F), partic¬ 
ularly those intended for direct interaction with humans 
(Part G), naturally must also consider the robot’s phys¬ 
ical structure. 

With this overview of Part B, we now provide a brief 
synopsis of each chapter. 

Chapter 16, Performance Evaluation and Design 
Criteria, provides a concise overview of the robot de¬ 
sign process, and surveys some of the criteria and tools 
used in the mechanical design and performance evalua¬ 
tion of robots. Criteria such as workspace volume, local 
and global dexterity, and elastostatic and elastodynamic 


performance are not only applicable to determining the 
topological structure and physical dimensions of the 
robot, but can also be useful for, e.g., workpiece place¬ 
ment and kinematic redundancy resolution. 

Chapter 17, Limbed Systems, discusses the myriad 
issues involved in the design, analysis, and control of 
robots with limbs. Defining a limbed system as a robot 
consisting of a body and at least one limb such that it 
is able to support and propel itself, the chapter begins 
with an overview of the design process for limbed 
systems,from conceptual to detailed, and the basic dy¬ 
namics of passive and controlled walking. The chapter 
also examines numerous case studies illustrating the 
diversity of limbed robot designs, and schemes for their 
actuation and locomotion. Multi-legged robots, such as 
dynamic quadrupeds inspired by mammals and behav¬ 
ior-based multilegged robots, are also discussed, as are 
hybrid leg-wheel-arm robots, tethered walking robots, 
and even legged robots capable of climbing walls. 

Chapter 18, Parallel Mechanisms and Robots, 
presents an introduction to the kinematics and dynam¬ 
ics of parallel mechanisms such as the well-known 
Stewart-Gough platform. Parallel mechanisms contain 
closed loops in their kinematic structure, and as such 
methods for their analysis differ considerably from 
those for their serial counterparts. This chapter dis¬ 
cusses topics ranging from type synthesis and forward 
and inverse kinematic solutions of parallel mecha¬ 
nisms, to an investigation of their singularity behavior, 
workspace characterization, static and dynamic analy¬ 
sis, and practical issues in their design. 

Chapter 19, Robot Hands, investigates the princi¬ 
pal issues behind the design, modeling, and control of 
robot hands. Beginning with a discussion of levels of 
anthropomorphism, and the characterization of robot 
hand dexterity, the chapter investigates the relevant de¬ 
sign issues for robot hands, actuation and transmission 
architectures, and available sensing technologies. The 
dynamic modeling and control of robot hands are made 
challenging not only by the complex kinematic struc¬ 
ture, but also by the flexible transmission elements, and 
the chapter devotes particular attention to these issues. 

Chapter 20, Snake-Like and Continuum Robots, 
begins with a history of snake robots, starting with the 
pioneering work of Shigeo Hirosein in the early 1970s. 
While snake-like and continuum robots have very simi¬ 
lar exterior appearances, there is considerable diversity 
in their mechanical design, and the ways in which they 
are actuated.This chapter describes the mechanical de¬ 
sign, actuation, modeling,motion planning, and control 
of such robots. The chapter also provides case studies 
of a wide range of existing snake-like and continuum 
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robots that illustrate the diversity of designs as well as 
applications. 

Chapter 21, Soft Robots, begins with the premise 
that robots of the future will not resemble the bulky 
rigid machines found in today’s factory floors, but will 
be compliant and adaptable, and able to safely inter¬ 
act with humans - in other words, soft. This chapter 
discusses the design, modeling, and control of actua¬ 
tors for this new generation of soft robots.The chapter 
surveys the different principles and technologies that 
can be used to design and implement actuators for soft 
robotics. Many of the concepts are organized so as to 
allow a direct analogy with natural muscles. Variable 
impedance actuators are examined in some detail, from 
their mathematical modeling to motion and force plan¬ 
ning and control. 

Chapter 22, Modular Robots, provides an overview 
of the design of modular robots. The chapter begins 
with a discussion of the concept of modularity, and 
a definition and classification of modular robots. The 
chapter then examines reconfigurable modular manip¬ 
ulators, from the earliest designs that were conceived 
in an industrial automation setting, to more recent self- 
reconfigurable modular robots. Issues related to the 
design of modules and the interfaces between them, and 
the determination of optimal configurations, are also 
discussed in this chapter. 

Chapter 23, Biomimetic Robots, broadly examines 
the ways in which biological principles can be applied 
to the design of robotic mechanisms. The challenges 
of biomimetic design include developing a deep under¬ 
standing of the relevant natural system, and translating 
this understanding into a set of engineering design 
rules; this often entails the development of novel fab¬ 
rication and actuation to realize the biomimetic design. 
This chapter discusses the basic design principles un¬ 
derlying biomimetic robots and their contrast with 
bio-inspired robots, and the fundamental components 
for developing a biomimetic robot. The chapter also 
provides detailed reviews of biomimetic designs that 
have been developed for flapping-wing flight, jumping, 
crawling, wall climbing, and swimming, as well as the 
enabling material and fabrication technologies for these 
biomimetic designs. 

Chapter 24, Wheeled Robots, provides a gen¬ 
eral and comprehensive description of wheeled mobile 
robots. The chapter begins with a discussion of robot 


mobility based on the types of wheels and the nature 
of the kinematic constraints, followed by a classifi¬ 
cation of wheeled robot structures according to the 
number and type of wheels and how they are arranged. 
Omnimobile robots and articulated robot realizations 
are described, and wheel-terrain interaction models 
for computing contact forces are also presented. The 
chapter concludes with a classification of wheel-terrain 
interaction cases depending on the relative stiffnesses of 
the wheel and terrain, and the structure and dyamics of 
suspension systems that enable movement of wheeled 
robots over uneven surfaces. 

Chapter 25, Underwater Robots, examines the de¬ 
sign issues for underwater robots, with a focus on 
remotely operated vehicles and autonomous underwa¬ 
ter vehicles. The major components of an underwater 
robot - from the mechanical elements and subsystems 
including any attached manipulators, to power sources, 
actuators and sensors, and architectures for comput¬ 
ing and communications and control, are discussed in 
this chapter. Aspects of the mathematical modeling and 
control of underwater robots is covered in a separate 
chapter later in this handbook. 

Chapter 26, Flying Robots, provides an overview 
of the core elements of flying robots. The reader will 
be guided through the design process of aerial robots, 
beginning with a qualitative characterization of the dif¬ 
ferent types of flying robot. Design and modeling are 
particularly closely intertwined in flying robots, and the 
chapter provides an overview of the underlying aerody¬ 
namics and tools for their analysis. The chapter then 
shows how these tools can be applied to the design 
and analysis of various types of flying robots, including 
fixed-wing, rotary-wing, and flapping wing systems, 
with case studies illustrating these design principles. 

Chapter 27, Micro/Nanorobots, provides an over¬ 
view of the state of the art in micro- and nanorobotics. 
The former entails robotic manipulation of objects 
with dimensions in the millimeter to micrometer range, 
as well as the design and fabrication of autonomous 
robotic agents within this size range (nanorobotics is 
defined in the same way, but for dimensions smaller 
than a micrometer). The chapter outlines scaling 
effects, actuation, and sensing and fabrication at 
these scales, and also applications to microassembly, 
biotechnology, and the construction and characteriza¬ 
tion of micro- and nano-electromechanical systems. 
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In this chapter we survey some of the tools and 
criteria used in the mechanical design and perfor¬ 
mance evaluation of robots. Our focus is on robots 
that are (a) primarily intended for manipulation 
tasks and (b) constructed with one or more serial 
kinematic chains. The kinematics of parallel robots 
is addressed in detail in Chap. 18; their elastostat- 
ics is the subject of Sect. 16.5.1. Wheeled robots, 
walking robots, multifingered hands, and robots 
intended for outdoor applications, i. e., those en¬ 
compassing what is known as field robotics, are 
studied in their own chapters; here we provide 
an overview of the main classes of these robots as 
relating to design. 
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The most obvious application of the criteria and tools 
described in this chapter is in the mechanical design 
of a robot. Robot design differs from the design of 1- 
degree-of-freedom (DOF) machinery in that the latter is 
intended for one specific task, e.g., picking up a work- 
piece from a belt conveyor and placing it on a magazine. 
Moreover, the conveyor is synchronized with the ma¬ 
nipulating machine and the magazine is stationary, with 
well-defined locations where each workpiece is to be 
placed. Manipulation robots, in contrast, are not in¬ 


tended for one specific task, but rather for a family of 
tasks falling within one class of workpiece motions, 
e.g., planar, spherical, translational, or motions pro¬ 
duced by systems of the selective compliance assembly 
robot arm (SCARA) type, also known as Schonflies dis¬ 
placements [16.1]. The challenge that robot designers 
face is therefore one of uncertainty in the specific task 
that the robot will be required to execute. Design cri¬ 
teria have been devised to help the designer cope with 
uncertainty, as discussed herein. 


Part B 116 
























Part B 116.1 


400 Part B 


Design 


16.1 The Robot Design Process 

Given a family of tasks that constitute the func¬ 
tional requirements in the design process, besides 
more-detailed design specifications, the role of the de¬ 
signer consists in producing a robot that will meet 
all the requirements and specifications. The various 
stages in the robot design job at hand are intended 
to: 

1. Determine the topology of the kinematic chain un¬ 
derlying the mechanical structure. Under this item 
we consider first the robot type: serial, parallel or 
hybrid. Then, a decision is to be made on the lay¬ 
out of the various subchains in terms of the type of 
joints, most commonly, revolute and prismatic. Re¬ 
cently, one additional type has been recognized to 
be equally useful, the 77-joint, coupling two links 
under relative translation by means of two other 
links undergoing identical angular displacements, 
although about different parallel axes. The four 
links form a parallelogram four-bar linkage [16.2]. 

2. Determine the geometric dimensions of the various 
links defining the robotic architecture, as required 
to fill a table of Denavit-Hartenberg (DH) parame¬ 
ters [16.3] so as to satisfy workspace requirements. 
Although these parameters are usually understood 
to include the joint variables, these variables do not 
affect the robot architecture; they determine instead 
the robot posture. 

3. Determine the structural dimensioning of the vari¬ 
ous links and joints, as needed to meet static load 
requirements, where load includes both forces and 
moments - wrenches - under either the most de¬ 
manding or the most likely operation conditions, 
depending on the design philosophy adopted at the 
outset. 

4. Determine the structural dimensioning of the vari¬ 
ous links and joints, as needed to meet dynamic load 
requirements, where loads are inertia effects of links 
and manipulated object. 

5. Determine the elastodynamic dimensioning of the 
overall mechanical structure, including the actuator 
dynamics, to avoid a specific spectrum of excitation 
frequencies under either the most demanding or the 
most likely operation conditions. 

6. Select the actuators and their mechanical transmis¬ 
sions for the operation conditions adopted at the 
outset to cope with task uncertainty. 

The above stages can be performed sequentially, 
in the order given above: (i) first, the topology is de¬ 
termined based on the family of tasks specified at the 
outset and the shape of the workspace, as discussed in 


Sect. 16.2.2; (ii) the link geometry is defined based on 
the workspace requirements, which include the maxi¬ 
mum reach, and the topology defined in stage 1; (iii) 
with the link geometry thus defined, the structural 
dimensioning of links and joints (unless the robot un¬ 
der design is parallel, which does not fall within the 
scope of this chapter, all joints are actuated) is un¬ 
dertaken, so as to support the static loads assumed at 
the outset; (iv) with the links and joints dimensioned 
for static-load conditions, the link centers of mass and 
link inertia matrices are determined for a preliminary 
evaluation of the motor torque requirements (this eval¬ 
uation is preliminary in that it does not consider the 
dynamic load brought about by the actuators; this load 
can be significant, even in the case of parallel robots, 
which can have all their motors fixed to the robot 
base); (v) with the links assumed rigid, joint stiffness 
is assumed, based on experience or using data from 
a similar robot, which then leads to an elastodynamic 
model whose natural modes and frequencies can be de¬ 
termined at a selected set of robot postures (dynamic 
behavior of the structure is dependent on robot pos¬ 
ture) by means of scientific code such as Matlab or 
computer-aided engineering (CAE) code such as CA- 
TIA, Siemens PLM, Pro/Engineer or ANSYS; and (vi) 
if the frequency spectrum of the robot structure is ac¬ 
ceptable, the designer can continue to motor selection; 
otherwise, a redimensioning is required, which means 
returning to stage 3. 

Even though a design cycle can be completed as 
outlined above, the designer must now incorporate into 
the elastodynamic model the structural and inertial data 
provided by the motor manufacturer. This requires a re¬ 
turn to stage 5 and a new elastodynamic analysis. It 
is thus apparent that the robot design process has one 
element in common with engineering design in gen¬ 
eral: both are iterative and open-ended [16.4]. Remark¬ 
ably, however, the various items driving each design 
stage are, to a large extent, independent of each other, 
e.g., topology and geometry can be determined inde¬ 
pendently from motor selection. Obviously, all issues 
interact in the overall design process, but, within certain 
design specifications, the various items do not contra¬ 
dict each other, as to warrant a multiobjective design 
approach. That is, the optimum design of serial robots 
can be accomplished fairly well by means of a sequence 
of single-objective optimization jobs. Again, the results 
of the last stage, motor selection, must be integrated 
into an overall mathematical model to test the overall 
performance. One reference addressing practical opti¬ 
mization issues in the conceptual design of industrial 
robots is [16.5]. 
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Only when the physical limits of components 
have been exhausted may a radical redesign requir¬ 
ing a return to stage 1 be warranted. This is the 
case with SCARA systems. Current industrial robots 
of this class bear topologies that are largely of the 
serial type, but some parallel SCARA systems have 
now appeared in the market. Indeed, the quest for 
shorter cycle time, as pertaining to an industry test 
cycle - described in Sect. 16.2.1 - has prompted the 
industry to look for alternatives to serial architec¬ 
tures. This is how ABBRobotics developed a hybrid 
parallel-serial robot, the FlexPicker, built upon Clavel 's 
Delta robot [16.6], to which a fourth axis has been 
added in series with the first three. The latter are 
laid out in a symmetric, parallel architecture that en¬ 
ables Delta to produce pure translations of its moving 
platform. Adept Technology’s Quattro s650H is re¬ 
portedly capable of three cycles/s for the same test 
cycle. 


16.2 Workspace Criteria 

The most obvious consideration in designing a robot is 
that its workspace has a set of required characteristics. 
This is a fundamental problem in classical mechanism 
design, and raises the obvious question of how a user 
can specify those characteristics. 

Issues to consider here pertain, mostly, to what Vi- 
jaykumar et al. [16.7] termed the regional structure of 
a manipulator. This applies to manipulators with a de¬ 
coupled architecture, whose last three revolutes have 
concurrent axes, thereby forming a spherical wrist, the 
point of concurrency being the wrist center. The ma¬ 
nipulation task of architectures of this kind thus allows 
for a decoupling of the positioning and the orientation 
subtasks: the regional structure, consisting of the first 
three joints, is first postured so as to locate the center 
of its wrist at a specified point C(x, y, z); then, the lo¬ 
cal structure, i. e., the wrist, is postured so as to make 
the end-effector (EE) attain a specified orientation with 
respect to a frame fixed to the base, given by a rotation 
matrix. 

Most algorithms reported in the literature to de¬ 
termine the workspace of a given robot refer to the 
workspace of the regional structure. Here, we should 
distinguish between the workspace of the kinematic 
chain, regardless of the physical implementation of the 
chain, and that of the physical robot. In the former, 
all revolute joints are capable of unlimited rotations 
about their axes; in the latter, joint limits are needed, 
for example, to avoid wire entanglement. In the early 
stages of robot design, joint limits need not be con¬ 


This chapter is organized according to the various 
stages of the robot design process outlined earlier. Not¬ 
ing that topology selection and geometric dimensioning 
are tightly coupled in the kinematic design process, we 
first begin with an examination of workspace criteria: 
we review methods for determining the topology of the 
kinematic chain, followed by the geometric dimensions 
so as to satisfy workspace requirements. We then review 
in detail the various criteria developed for characteriz¬ 
ing a robot’s manipulating capability, focusing on quan¬ 
titative notions of dexterity based on both kinematic and 
dynamic models. We then examine methods for struc¬ 
tural dimensioning of the links and joints so as to meet 
both static and dynamic load requirements. Finally, we 
discuss elastodynamic dimensioning, and actuator and 
gear sizing, taking into account properties such as the 
natural frequency of the robot, and force and accelera¬ 
tion capability requirements. We end the chapter with 
an overview of mobile, aquatic, and flying robots. 


sidered, the workspace thus exhibiting symmetries that 
are proper of the type of joints of the regional struc¬ 
ture. If the first joint is a revolute, the workspace has 
an axis of symmetry, namely, the axis of this revo¬ 
lute joint; if the first joint is prismatic, the workspace 
has an extrusion symmetry, with the direction of ex¬ 
trusion given by the direction of motion of this joint. 
As prismatic joints are infinitely extensive, so is the 
kinematic workspace of a robot with a prismatic joint. 
The kinematic workspaces of robots with prismatic 
joints are usually displayed for a finite portion of this 
workspace. 

In the case of parallel robots, to be studied in 
full detail in Chap. 14, the regional structure is elu¬ 
sive, in general. The usual practice when displaying 
the workspace for these robots is to assume a constant 
orientation of the moving plate, the counterpart of the 
EE of serial robots [16.8]. A common architecture of 
parallel robots, which arises quite naturally in the de¬ 
sign process, entails identical legs symmetrically placed 
both on the base platform and on the moving plat¬ 
form. Each leg is, in turn, a serial kinematic chain with 
one or two active joints, all others being passive. The 
workspace of this kind of robots also exhibits certain 
symmetries, but no axial symmetry. The symmetries are 
dictated by the number of legs and the types of actuated 
joints. 

Coming back to serial robots, the workspace can 
be defined by an envelope that is essentially of one 
of two types, either a manifold or a surface that is 
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smooth almost everywhere, i. e., smooth everywhere ex¬ 
cept for a set of points of measure zero in the Lebesgue 
sense [16.10]. Broadly speaking, a set of measure zero 
on a surface is a curve, e.g., a meridian on a sphere, 
or a set of isolated points on a line, e.g., the set of ra¬ 
tional numbers on the real line. A paradigm for this 
second kind of workspace is that of the Puma robot, 
whose kinematic chain is displayed in Fig. 16.1. In 
this figure, the regional and the local structures are 
clearly distinguished, the former being fully extended. 
The workspace of this robot is obtained upon lock¬ 
ing all joints but the second, when the robot is in the 
posture shown in Fig. 16.1. Then, the second joint is 
fully rotated about its axis, the center C of the wrist 
then describing a circle of radius R equal to the dis¬ 
tance of C from the line Ln. the plane of the circle 
being normal to this line and lying a distance b 3 from 
the axis L\ of the first joint. This distance is known as 
the shoulder offset. Now, with all joints locked again, 
but this time with the first joint unlocked, the robot 



Fig. 16.1 A Puma robot in a fully stretched posture (af¬ 
ter [16.9]) 


is turned as a rigid body about L\. The result is the 
toroid of Fig. 16.4. Notice that the solid enclosed by 
this surface is the result of the Boolean operation S—C, 
where S is the sphere of radius R centered at point O 2 
of Fig. 16.1, while C is the infinite cylinder of ra¬ 
dius 63 and axis L\, which appears as Z\ in Fig. 16.2. 
It is noteworthy that, although this workspace can be 
readily generated by a simple Boolean operation, it 
cannot possibly be generated by an implicit function 
of the form fix, y, z) = 0 because the surface is not 
a manifold. 

Robots with manifold workspaces are not common 
in industry. We display in Fig. 16.3 an architecture for 
the regional structure of a six-axis decoupled robot, 
with its neighboring axes mutually orthogonal and 
at the same distance a from each other. The com¬ 
mon normals to the two pairs of axes, Xj and X 3 , 
also lie at the same distance a, as do X 4 from Xj 
and C from Z 3 . Point C is the center of the spherical 
wrist, the latter not being included in the figure. The 
workspace of this robot admits a representation of the 
form f(x, y, z) = 0 [16.9], which produces the mani¬ 
fold workspace of Fig. 16.4. The shaded internal region 
of the workspace includes all points admitting four real 
inverse-kinematics solutions, all other points admitting 
only two. 

Given that any point of the workspace boundary 
represents a positioning singularity - different from an 
orientation singularity - manipulators with workspace 
boundaries that are not manifolds exhibit double sin¬ 
gularities at the edges of their workspace boundary, 
which means that at edge points the rank of the robot 
Jacobian becomes deficient by two. At any other point 
of the workspace boundary the rank deficiency is by 
one. 
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Fig. 16.2 The workspace of a Puma robot (after [16.9]) 




Fig. 16.3 An orthogonal three-revolute robot (af¬ 
ter [16.9]) 
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Design rules based on the shape of the workspace 
can now be drawn: 

1. If the workspace required is axially symmetric and 
finite, use a serial robot with a regional structure 
composed of revolute joints only. 

2. If the workspace required has a symmetry of ex¬ 
trusion and is unbounded, use a serial robot with 
regional structure having one first joint of the pris¬ 
matic type. Here, unbounded is used in a restricted 
sense, meaning much larger in one direction than 
the others. Moreover: 

• If one direction is required to be much larger 
than the others, then practical implementations 
of prismatic joints are available in the form 
of rails either overhead, thereby giving rise to 
gantry robots, or on the floor. 

• If two directions are required to be much larger 
than the other, then use a wheeled mobile robot 
carrying a manipulator on top. A famous re¬ 
alization of this concept is the National Aero¬ 
nautical and Space Agency’s (NASA) Sojourner 
used in the Pathfinder mission to Mars in 1997. 

3. If axial symmetry is not required, but rather 
a workspace with various coplanar axes of sym¬ 
metry, similar to those of regular polygons, use 
a parallel robot. 

16.2.1 Reaching a Set of Goal Frames 

Closely related to the problem of workspace spec¬ 
ification is that of task specification. In mechanism 
design it is customary to specify a set of coordinate 
frames in space, and to design a mechanism with an 
a priori specified topology that can visit these frames. 
An order in which the frames must be reached may 
be given. In the event that not all of the frames 
are reachable, then one may seek a mechanism that 

Regions Number of solutions 



Fig. 16.4 The workspace of the orthogonal robot of 
Fig. 16.3 (after [16.9]) 


comes closest, in some suitable sense, to the specified 
frames. The literature on this classical mechanism de¬ 
sign problem is vast [16.1, 11, 12] and the references 
cited therein. Some further remarks in connection with 
this goal-frame approach to robot dimensioning are 
noteworthy: 

1. Reaching exactly the desired frames may not al¬ 
ways be desired or possible: in some cases it is 
better to use an optimization approach that allows 
for solutions that will visit the desired poses within 
a minimum error (provided that an error norm can 
be suitably engineered, of course, as suggested 
in [16.13]). 

2. It has been claimed [16.8] that interval analysis 
allows not only a discrete set of desired poses 
but also a full six-dimensional (6-D) workspace to 
be met while taking into account manufacturing 
errors. 

3. The branching problem occurring in 1-DOF mech¬ 
anisms may also occur in robot design: a design 
solution based on via points may indeed visit the 
prescribed poses, but not all of these may be reach¬ 
able within the same assembly mode. This problem 
is exacerbated in the design of serial robots, as a 6- 
DOF, revolute-coupled robot may admit up to 16 
distinct postures - branches - for one given EE 
pose [16.14, 15]. 

4. While a robot designed to visit a set of prescribed 
poses via its EE will be able to visit that set, we 
should not forget that the purpose of using robots 
is first and foremost to be able to perform not one 
single task, but rather a family of tasks. In this 
light, the set of poses for which a robot is designed 
might as well be a task that is representative of that 
family. 

In connection with remark 4 above, we can cite the 
design or evaluation of SCARA systems. A SCARA 
system is a 4-DOF serial robot capable of tasks that 
lie within the Schonflies subgroup of the group of 
rigid-body displacements [16.16,17], namely the set of 
three-dimensional (3-D) displacements augmented with 
a rotation about an axis of fixed direction. In these 
systems, the task at hand is given by two vertical seg¬ 
ments joined by one horizontal segment. Moreover, the 
length of the vertical segments is 25.0 mm, that of the 
horizontal segment being 300.0 mm. While the EE is 
traversing the horizontal segment, moreover, it should 
rotate about a vertical axis through an angle of 180°. 
This task specification, which has been adopted by 
SCARA manufacturers, does not indicate how to ne¬ 
gotiate the corners, which is left to the imagination of 
the robotics engineer. 
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16.2.2 Workspace Volume and Topology 

Reachable and Dexterous Workspace 
Beginning with the early work of Roth [16.18], there 
have been many studies on the relationship between 
manipulator kinematic geometry and its workspace. 
Most studies have focused on a classification of the 
workspace into two components, the reachable and the 
dexterous workspace [16.19]. Given a reference point P 
attached to a manipulator EE, such as the center of 
the spherical wrist, or a point on the EE, the reach¬ 
able workspace is defined to be the set of points in 
physical space that can be reached by P. The dexterous 
workspace, on the other hand, is the set of points that 
can be reached by P with arbitrary EE orientations. 

The early literature on workspace focuses on nu¬ 
merical and algebraic methods to characterize these 
workspaces. Reachable and dexterous workspaces have 
been analyzed using numerical techniques by Kumar 
and Waldron [16.19], Yang and Lee [16.20], and Tsai 
and Soni [16.21], among others. The advantage of these 
schemes over algebraic approaches is that kinematic 
constraints can be readily included. More-general de¬ 
sign principles or insights, however, are more difficult 
to come by using these techniques. Among the alge¬ 
braic approaches to workspace characterization, a topo¬ 
logical analysis of robot workspace is given by Gupta 
and Roth [16.22] and Gupta [16.23]; here the concept 
of workspace holes and voids is defined, and condi¬ 
tions for their existence are identified. The shape of the 
reachable and dexterous workspaces is also analyzed as 
a function of P. 

Further studies of workspace analysis were re¬ 
ported by Freudenstein and Primrose [16.24] and by 
Lin and Freudenstein [16.25], where precise relation¬ 
ships between kinematic and workspace parameters are 
developed, and a class of three-joint manipulators is 
optimized for workspace volume. A more general anal¬ 
ysis of workspace optimization is given in Vijaykumar 
et al. [16.7]. Performance criteria for manipulators are 
defined here in terms of the dexterous workspace; given 
that a manipulator satisfies certain constraints on its 
Denavit-Hartenberg parameters, it is shown that the op¬ 
timal six-revolute (6R) design is the elbow manipulator. 

A typical design of robot regional architecture is of 
the orthogonal type , consisting of one revolute of the 
vertical axis and two revolutes of the horizontal axes, 
one of which intersects the vertical axis. Moreover, 
the most common architecture includes intermediate 
and distal links of identical lengths. The workspace 
of this architecture is thus a sphere of radius equal 
to twice that common link length. The volume of this 
workspace is thus determined by the length in ques¬ 
tion. As shown by Yoshikawa [16.26], the workspace of 


the two-link planar manipulator defined by the last two 
links of the foregoing regional architecture is of maxi¬ 
mum area for a prescribed reach and equal link lengths. 
As a result, the volume of the same regional architec¬ 
ture is similarly of maximum volume for a prescribed 
reach. 

Differential-Geometric Workspace 
Characterization 

Workspace can also be approached from a differential- 
geometric perspective, by regarding the configuration 
space of a robotic EE frame as a subset of the special 
Euclidean group SE(3). An important physical consid¬ 
eration in defining the workspace volume of spatial 
mechanisms is that it should not depend on the choice 
of fixed reference frame. Less obvious but just as im¬ 
portant is the requirement that the volume should not 
depend on which point of the last link the EE frame is 
fixed to. This last condition has the following physical 
significance: if the EE were enlarged or shrunk, then 
the robot would have the same workspace volume. The 
workspace volume of a robot therefore depends only on 
the joint axes. 

The workspace volume of a robot is defined by re¬ 
garding SE(3) as a Riemannian manifold, so that the 
workspace volume is simply the volume of the im¬ 
age of the forward kinematic map / with respect to 
the volume form on SE(3). It is known that SE(3) has 
a bi-invariant volume form, that is, the notion of vol¬ 
ume is invariant with respect to the choice of both 
the fixed (base) and moving (EE) frames - see, e.g., 
Loncaric [16.27]. Paden and Sastry [16.28] provide 
the following visualization for this volume form: Sup¬ 
pose an airplane is restricted to move within a cube 
of airspace of length 1 km on a side. At each point 
within this cube the airplane can point itself anywhere 
in a 4 tc solid angle and roll 2 n about the direction it is 
pointing. The orientation volume at such an airplane is 
4n x 2;r = 8 it 2 rad 3 . Multiplying by the positional vol¬ 
ume one obtains 8jr 2 rad 3 km 3 for the volume of the free 
configuration space of the aircraft. 

This depiction is the notion of workspace volume 
used for robots; it has the advantage of being able 
to trade off orientation freedom for positional free¬ 
dom smoothly, unlike the popular notion of dexterous 
workspace. Note that the actual numerical value one 
obtains will depend on the choice of length scale for 
physical space; this in itself does not pose a serious 
problem for workspace volumes, as long as the same 
length scale is maintained when comparing different 
workspaces. 

In [16.28] Paden and Sastry show that the op¬ 
timal 6R manipulator that maximizes the workspace 
volume subject to a kinematic length constraint is the 
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elbow manipulator. This result is consistent with the 
earlier finding of Vijaykumar et al. [16.7], but the au¬ 
thors employ the geometric framework outlined above. 


Moreover, the results are obtained without some of the 
a priori assumptions on the kinematic structure made by 
Vijaykumar. 


16.3 Dexterity Indices 

16.3.1 Local Dexterity for Open Chains 

Dexterity can be defined as the ability to move and 
apply forces and torques in arbitrary directions with 
equal ease, the concept thus belonging to the realm 
of kinetostatics, which is the study of the interplay 
between feasible twists and constraint wrenches in 
multibody mechanical systems under static consen’a- 
tive conditions. Here, twist is the 6-D array of velocity 
variables of a rigid body, involving three components of 
a landmark-point velocity and three of angular velocity; 
wrench, in turn, is the 6-D array of static variables act¬ 
ing on a rigid body, three accounting for the resultant 
force applied at the same landmark point and three for 
the concomitant moment acting on the same body. 

Salisbury and Craig [16.29] introduced the concept 
of dexterity when working on the design of articulated 
hands. At issue is the way in which input joint velocity 
errors propagate to the output velocities of each fin¬ 
gertip. To illustrate this concept, let J(0) denote the 
Jacobian of the forward kinematic map, i. e., 

t = J(0)0 , (16.1) 

in which 6 and 6 denote the vectors of joint variables 
and joint rates, respectively, while t is the EE twist, de¬ 
fined, in turn, as, 

with a) denoting the angular velocity of the EE and p the 
velocity of the operation point P of the EE, at which the 
task is specified. 

Assuming the Jacobian J is m x n, the singular value 
decomposition of J can be expressed in the form 

J = UXV T , (16.3) 

where U and V are respectively m x m and n x n unitary 
matrices, and E is a m x n matrix with zeros every¬ 
where, except for its ( i , i) entries, for i = 1 .... n if 
m> n\ otherwise, for i = 1 .m. The non-zero ele¬ 
ments are the (non-negative) singular values of J. At 
nonsingular postures, the Jacobian is invertible and we 
can write 


(16.4) 


Furthermore, if we assume that all the components of 
both t and 6 have the same physical units, which is the 
case for purely positioning or purely orienting manip¬ 
ulators with only revolute joints, then we can take the 
Euclidean norm of both sides of (16.4), thereby obtain¬ 
ing 


||0 || 2 = f T U(EE T ) 'u T t. (16.5) 

If we let v = U T f then the above expression for ||0 || 2 
becomes 

t; T (EE T ) _1 v = \\9 \\ 2 . (16.6) 


Now, if the i-th component of v is denoted by 17 , 
for i = 1 ,...,«, and we look at the mapping of the unit 
ball in J, ||0 || 2 = 1, (16.6) leads to 


V\ V2 v n 

— -|—- -|- 1 —- = 1 

(if ct 2 - a- 


(16.7) 


which is the canonical equation of an ellipsoid of semi¬ 
axes { 07 }'j in the g-space, i. e., the space of Cartesian 
velocities, or twists of the EE. Notice that the ellipsoid 
in question takes its canonical form when represented 
in a coordinate frame of axes oriented in the directions 
of the eigenvectors of U. 

In summary the unit ball in joint space is mapped by 
the Jacobian-inverse J 1 into an ellipsoid whose semi¬ 
axes are the singular values of J. That is, J distorts the 
unit ball in the joint-rate space into an ellipsoid in the 
EE-twist space. Hence, a measure of the quality of mo¬ 
tion and force transmission of the robotic architecture 
from the joints to the EE is given by the above distor¬ 
tion; the smaller the distortion, the higher the quality of 
the transmission. 

A measure of the Jacobian-induced distortion can 
thus be defined as the ratio of the largest <7 m to the 
smallest a m singular values of J, which is nothing 
but the condition number k 2 of J based on the matrix 
2-norm [16.30], i. e., 


O'M 

k 2 = — ■ 


°m 


(16.8) 


Actually, (16.8) is only one possibility of comput¬ 
ing the condition number of J, or of any m x n matrix 


6 = VE _1 U T f. 
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for that matter, and certainly not the most economical. 
Notice that this definition requires knowledge of the 
singular values of the Jacobian. However, computing 
the singular values of a matrix is as computationally in¬ 
tensive as computing eigenvalues, with the added cost 
of a polar decomposition [16.31]; the combined oper¬ 
ation is slightly less expensive than the singular-value 
decomposition [16.32]. The most general definition of 
condition number, for n x n matrices, is [16.30] 

k(A) = ||A||||A -1 1| . (16.9) 

The expression (16.8) is obtained when the matrix 2- 
norm is adopted in (16.9). The matrix 2-norm is defined 
as 

IIA|| 2 = max{cr, } . (16.10) 

i 

If, on the other hand, the weighted matrix Frobenius 
norm is adopted, which is defined as 


The significance of the condition number in robot 
design and control can be best understood if we re¬ 
call that the concept stems from numerical analysis in 
connection with the solution of the linear system of 
equations (16.1) for 0. Given that J is a function of 
both the architecture parameters and the posture vari¬ 
ables 0, J is known only to within the error with 
which those quantities are known. Further, let the ar¬ 
chitecture parameters, namely, the constant values in 
the Denavit-Hartenberg parameter list, be stored in an 
array p. Both p and 6 are known up to measurement 
errors Sp and SO , respectively. Moreover, the twist t is 
input into the control software of the robot with an un¬ 
avoidable error St. 

In solving (16.1) for 9 with floating-point arith¬ 
metic, the computed value is contaminated with 
a roundoff error 80. The relative error in the com¬ 
puted 0 is related to the relative errors in the archi¬ 
tecture parameters and posture variables by the rela¬ 
tion [16.30] 


II A|| f = J- tr(AA T ) = J - tr(A T A) , (16.11) 

V 11 V n 

then, apparently, the computation of the singular val¬ 
ues can be obviated. When the weight \/n is omitted 
in the above definition, the standard Frobenius norm 
is obtained. Notice, however, that the weighted Frobe¬ 
nius norm is more significant in engineering, for it does 
not depend on the number of rows and columns of the 
matrix at hand. The weighted Frobenius norm, in fact, 
yields the root-mean-square (mis) value of the set of 
singular values. 

The Frobenius condition number K-p of the Jaco¬ 
bian J, based on the matrix Frobenius norm, is then 

*f(J) = - V tr(JJ T ) y/ tr[(JJ T )~ 1 ] 
n 

= - AtruAK-FJ) -1 ] • (16.12) 

n 

One more important difference between the two forms 
of computing the matrix condition number is worth 
pointing out: /^(O is not an analytic function of its 
matrix argument, while Kp( ■) is. Hence, the condition 
number based on the Frobenius norm can be applied to 
great advantage in robot architecture design, as /cf(-) 
is differentiable and lends itself to gradient-dependent 
optimization methods, which are much faster than di¬ 
rect methods based only on function evaluations. In 
robot control, which requires real-time computations, 
atf( -) also offers practical advantages, for its computa¬ 
tion obviates the knowledge of the singular values, only 
requiring a matrix inversion, which is a rather simple 
task. 



Ilfrll IIMII 

Up II ll«ll 11*11 ) ' 


(16.13) 


where p and 0 represent the (unknown) actual values of 
these vectors and t the nominal value of the twist. 

Nevertheless, the foregoing discussion applies to 
tasks involving either positioning or orientation, but 
not both. Most frequently, robotic tasks involve both 
positioning and orientation, which leads to Jacobian 
matrices with entries bearing disparate physical units, 
the consequence being that the Jacobian singular values 
have disparate units as well. Indeed, singular values as¬ 
sociated with positioning bear units of length, whereas 
those associated with orientation are dimensionless. As 
a consequence, it is impossible to either order all singu¬ 
lar values from smallest to largest or to add them all. 

To cope with the issue of disparate units in the Ja¬ 
cobian entries, and to allow for the computation of the 
Jacobian condition number, the concept of characteris¬ 
tic length has been proposed [16.9]. The characteristic 
length L has been defined as the length by which the 
entries of the Jacobian with units of length are to be 
divided to render the Jacobian condition number a min¬ 
imum at an optimum posture. This definition, while 
sound, lacks an immediate geometric interpretation, 
which has made its adoption in the robotics community 
difficult. In order to provide for a geometric interpreta¬ 
tion, the concept of homogeneous space was recently 
introduced [16.33]. Using this concept, the robot ar¬ 
chitecture is designed in a dimensionless space, with 
points whose coordinates are dimensionless real num¬ 
bers. In this way, the six Pliicker coordinates [16.34] 
of lines are all dimensionless, and hence the columns 
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of the robot Jacobian matrix, comprising the Pliicker 
coordinates of the revolute axes, are dimensionless as 
well. As a consequence, the Jacobian singular values 
are all dimensionless, and the Jacobian condition num¬ 
ber can be defined. Once the robotic architecture has 
been designed for minimum condition number, under 
geometric constraints on link-length ratios and angles 
between neighboring revolute axes, for example, the 
maximum reach of the robot can be calculated. The 
maximum reach r will thus be a dimensionless quan¬ 
tity. When this quantity is compared with the prescribed 
maximum reach R, with units of length, the characteris¬ 
tic length is computed as the ratio L = R/r. 

16.3.2 Dynamics-Based Local Performance 
Evaluation 

Since motions are caused by forces and torques act¬ 
ing on rigid bodies, it seems reasonable to formulate 
performance indices that take into account the iner¬ 
tial properties of the mechanism. Asada [16.35] defines 
the generalized-inertia ellipsoid (GIE) as the ellipsoid 
described by the product G = J~ T MJ — where M 
denotes the inertia matrix of the manipulator. This el¬ 
lipsoid is that with semiaxes given by the singular 
values of the foregoing product. Yoshikawa [16.36], 
in turn, defines a corresponding dynamic manipulabil- 
ity measure as det[JM~'(JM - 1 ) T ], Physically these 
concepts represent two distinct phenomena. Suppose 
the robot is viewed as an input-output device, which, 
given a joint torque, produces an acceleration at the 
EE. Yoshikawa’s index measures the uniformity of this 
torque-acceleration gain, whereas Asada’s GIE charac¬ 
terizes the inverse of this gain. If a human operator were 
holding the EE and attempting to move it about, the GIE 
would measure the resistance of the robot to this EE 
motion. 

Other measures that attempt to capture robot per¬ 
formance as a function of the dynamics can be cited: 
Voglewede and Ebert-Uphoff [16.37] propose perfor¬ 
mance indices based on joint stiffness and link inertia, 
with the aim of establishing a distance to singularity for 
any robot posture, while Bowling and Khatib [16.38] 
propose a general framework for capturing the dynamic 
capability of a general robot manipulator that includes 
the velocity and acceleration characteristics of the EE, 
taking into account factors such as torque and the ve¬ 
locity limits of the actuators. 

16.3.3 Global Dexterity Measures 

The above measures are local in the sense that they 
characterize the dexterity of a robot at a given pos¬ 
ture. Local measures are useful for applications ranging 


from redundancy resolution to workpiece positioning, 
but for design applications a global measure may be 
more desirable. One straightforward way of extending 
local measures to global ones is to integrate them over 
the allowable joint space. In [16.39] Gosselin and An - 
geles integrate the Jacobian condition number over the 
workspace to define a global measure, thereby produc¬ 
ing a global conditioning index. For the simpler cases 
of planar positioning and spherical manipulators, the 
global conditioning index was found to coincide with 
its local counterpart. 

16.3.4 Closed-Chain Dexterity Indices 

The formulation of dexterity for closed chains presents 
a number of subtleties. The first obvious difference 
is that the joint configuration space of a closed chain 
will no longer be a flat space; in the general case it 
will be a curved multidimensional surface embedded 
in a higher-dimensional (typically flat) space. Also, 
dual to the open chain case, the forward kinematics for 
closed chains is generally more difficult to solve than 
the inverse kinematics, with the possibility of multi¬ 
ple solutions. Another important difference is that only 
a subset of the joints may be actuated, and that the num¬ 
ber of actuated joints may even exceed the mechanism’s 
kinematic degrees of freedom. 

Several coordinate-based formulations for closed- 
chain dexterity have been proposed for specific mecha¬ 
nisms [16.40] and for cooperating robot systems whose 
joints are all assumed to be actuated [16.41,42], at 
least some of which have led to apparently paradoxical 
results [16.43,44]. Because of the nonlinear charac¬ 
teristics unique to closed-chain mechanisms discussed 
above, particular care must be exercised in formulating 
coordinate-based dexterity measures. 

Another recent line of work has attempted 
a coordinate-invariant differential-geometric formula¬ 
tion of dexterity for closed chains. In this framework 
the joint and EE configuration spaces are regarded as 
Riemannian manifolds with an appropriate choice of 
Riemannian metric, with the choice of joint space met¬ 
ric reflecting the characteristics of the joint actuators. 
The previous notions of ellipsoid developed for se¬ 
rial chains can be extended to general closed chains, 
containing both passive and active joints, and possibly 
redundantly actuated [16.45,46]. 

16.3.5 Alternative Dexterity-Like Measures 

The various definitions of dexterity discussed above all 
refer to the same qualitative feature of the ability of 
a robot to move and apply forces in arbitrary direc¬ 
tions. A different viewpoint is taken in the work of 
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Liegeois [16.47] and Klein and Huang [16.48], where 
dexterity is quantified in terms of joint-range availabil¬ 
ity. The driving motivation here lies in that most robots 
have joint limits; therefore, one should minimize the 
possibility of a joint reaching a stop. 

Hollerbach [16.49] takes an alternative approach to 
designing a redundant seven revolute 7R manipulator, 
by considering: (a) elimination of internal singularities, 
(b) ability to avoid obstacles in the workspace, (c) the 
solvability of the kinematic equations, and (d) mechan¬ 
ical constructability. Based on these four criteria, he 
derived a particular 7R design with the morphology of 
the human arm, i. e., composed of two spherical joints 
defining the shoulder and the wrist plus an intermedi- 

16.4 Other Performance Indices 

16.4.1 Acceleration Radius 

Another measure that attempts to capture the dynamic 
capabilities of a robotic manipulator is the acceleration 
radius. Initially proposed by Graettinger and Krogh 
in [16.51], the acceleration radius measures the min¬ 
imum acceleration capability of the EE, in arbitrary 
directions, for the given torque bounds on the actuators. 
Specifically, given the dynamics equations for a serial 
chain in the form 

t=M(0)0 + C (6,6)0, (16.14) 

in which M is the robot mass matrix - also known as the 
inertia matrix - in joint space, and C(0 . 0 ) is the matrix 
mapping the joint-rate vector to the vector of Coriolis 
and centrifugal forces in the same space. Moreover, the 
actuators are assumed to have joint torque limits of the 
form 

t min — C t max > (16.15) 

where the lower and upper limits t^, T max e R" are 
constant or functions of the manipulator posture 0. The 
EE twist rate, denoted by t, can be expressed as 

t= J(0)0 + J{0,0)0 , (16.16) 

where J (0,0) is the Jacobian time-derivative. The Ja¬ 
cobian J was introduced in (16.1). 

Under the assumption that ,\(0) is nonsingular, one 
can write 

(16.17) 


ate revolute playing the role of the elbow. Now, while 
a redundant architecture should remain fully capable of 
performing 6-DOF tasks upon locking one joint, the 
architecture reported in this reference may end up by 
losing this capability if the elbow joint is locked. 

From a control perspective, Spong [16.50] shows 
that, if the inertia matrix of a manipulator has a van¬ 
ishing Riemannian curvature, there exists a set of co¬ 
ordinates in which the equations of motion assume 
a particularly simple form. The curvature of the iner¬ 
tia matrix also reflects the sensitivity of the dynamics 
to certain robot parameters. Minimizing the curva¬ 
ture, therefore, is another possible criterion for robot 
design. 


Substituting the above expression into the dynamic 
equations (16.14) leads to 

T(0,t,i) = M'(0)i + C'(0,t)t, (16.18) 

where 

M'(0) = M(0)J(6>r 1 

C'(0,t) = [C(0,t)-M(0)J(0r 1 kO,t)]r 1 (O). 

For a given state (0.0), the linear torque bounds 
(16.15) now define a polytope in the twist-rate space. 
Graettinger and Krogh [16.51] define the acceleration 
radius to be the largest sphere centered at the origin that 
is contained in this poly tope; its radius reflects the min¬ 
imal guaranteed EE acceleration in arbitrary directions. 
This concept is applied to measure the EE accelera¬ 
tion capability of a manipulator, as well as to determine 
the actuator sizes for achieving a desired acceleration 
radius. Bowling and Khatib [16.38] generalize this con¬ 
cept to capture both force and acceleration capabilities 
of the EE, with a view to quantifying the worst-case dy¬ 
namic performance capability of a manipulator. 

16.4.2 Elastostatic Performance 

Elastostatic performance refers to the robotic system’s 
response to the applied load - force and moment, i. e., 
wrench - under static equilibrium. This response may 
be measured in terms of the stiffness of the manip¬ 
ulator, which determines the translation and angular 
deflections when the EE is subjected to an applied 
wrench. 

Robot deflections have two sources: link and joint 
deflection. In the presence of long links, as in the space 


0 =J(0)- v t-J(0)- 1 j(O,t)0 . 
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robot Canadarm2, link compliance is a major source 
of deflection. However, in the majority of today’s se¬ 
rial robots, the main source of deflection occurs at the 
joints. 

In this chapter, we assume that the manipulator 
links are rigid, and model the joints as linearly elastic 
torsional springs. The more complex problem of link 
flexibility is studied in detail in Chap. 11. That is, for 
the elastostatic model we base our analysis on the as¬ 
sumption that, under a positioning task, the joints are 
locked at a certain posture 0 0 , while the EE is subject to 
a perturbation wrench Aw that is balanced by an elas¬ 
tic joint torque Ar. Under these conditions, A 0 and At 
obey the well-known linear relation 

KA0 = At , (16.19) 

in which K is the stiffness matrix in joint space at the 
given posture. Moreover, the matrix K is diagonal, with 
its entries equal to the torsional stiffnesses of the cor¬ 
responding joints, K is therefore posture independent , 
i. e., constant throughout the whole robot workspace. 
Moreover, since all joints exhibit a finite, nonzero stiff¬ 
ness, K is invertible, its inverse C being termed the 
compliance matrix. We can thus express the inverse re¬ 
lation of (16.19) as 

AO = CAt . (16.20) 

It should be apparent that A6 and At have an incre¬ 
mental nature, for both are measured from the equilib¬ 
rium posture, at which At and AO vanish. 

On the issue of stiffness matrix, Griffis and 
Duffy [16.52] proposed a mapping from an incre¬ 
mental rigid-body displacement Ax into an incremen¬ 
tal wrench Aw that turned out to be nonsymmetric. 
The concept behind this mapping was formalized by 
Howard , Zefran and Kumar [16.53] by means of Lie 
algebra. However, in the foregoing papers. Ax and Aw 
turn out to be incompatible in the sense that their re¬ 
ciprocal product does not yield incremental work - the 
point at which Ax is defined is distinct from that at 
which Aw is applied. For this reason, the array rep¬ 
resentation of the same mapping cannot be, properly 
speaking, termed a stiffness matrix. 

For a constant magnitude of At, the deflection 
attains its maximum value in the direction of the eigen¬ 
vector associated with the maximum eigenvalue of C 
or, equivalently, with the minimum eigenvalue of K, 
denoted by In terms of elastostatic performance, 
we aim to (a) make the maximum deflection a mini¬ 
mum, i. e., we want to maximize /fmin, and (b) make the 
magnitude of the deflection 11A011 as insensitive as pos¬ 
sible to changes in the direction of the applied load At . 


This can be done by rendering as close as possi¬ 
ble to /c max . The first aim is associated with the stiffness 
constants, i. e., the higher the constants the lower the 
deflections. The later, however, is associated with the 
concept of isotropy, the ideal case being when all the 
eigenvalues of K are identical, which means that K is 
isotropic. Due to the pyramidal effect of serial robots, 
in which downstream motors carry their upstream coun¬ 
terparts. the joint stiffness is bound to be largest for the 
proximal joints. Hence an isotropic stiffness matrix is 
impossible for serial robots. 

Notice that (16.19) and (16.20) may also be formu¬ 
lated in the task space, i. e., 

K c Ar= Ait), (16.21) 

where Ax = tAt, with At denoting a small time interval 
producing a correspondingly small change Ax in the 
pose of the EE. That is. 

Ax = JO At = JAO , (16.22) 

which is a linear transformation of the joint-vector in¬ 
crement into the pose-increment vector. We show next 
that the stiffness matrix is not frame invariant. This 
means that, under the linear transformation from joint to 
Cartesian coordinates, the stiffness matrix does not obey 
a similarity transformation. For quick reference, we re¬ 
call below the definition of similarity transformation: 
if y = Lx is a linear transformation of R" into itself, 
and we introduce a change of vector basis, x' = Ax, 
y' = Ay, then L changes to I/, which is given by 

if = ALA -1 . (16.23) 

The above transformation of any vector of R" into an¬ 
other one of the same space, and of a matrix L into If, 
as given by (16.23), is termed a similarity transforma¬ 
tion. Notice that A is guaranteed to be invertible, as it 
represents a change of coordinate frame. 

Now, under the change of coordinates given 
by (16.22), (16.21) leads to 

K c JA0=J _t At, (16.24) 

where we have used the kinetostatic relation [16.9] 

J t Aw = At . 

The exponent —T denotes the transpose of the inverse 
or, equivalently, the inverse of the transpose. Upon mul¬ 
tiplication of both sides of (16.24) by J T from the left, 
we end up with 

J t K c JA0 = At . 


(16.25) 
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If we now compare (16.19) with (16.25), we can 
readily derive the relations between the stiffness ma¬ 
trix K in joint space and the stiffness matrix Kc in 
Cartesian space 

K = J t K c J and K c = J _T KJ _1 , (16.26) 

which apparently is not a similarity transformation be¬ 
tween K and Kc. What this means is that the two 
matrices do not share the same set of eigenvalues 
and their eigenvectors are not related by the linear 
relation (16.22). As a matter of fact, if the robot is 
revolute-coupled, the entries of its stiffness matrix K all 
have units of Nm, i. e., of torsional stiffness, while the 
entries of Kc have disparate units. To show this, the Ja¬ 
cobian matrix, the inverse Jacobian and the two stiffness 
matrices are partitioned correspondingly into four 3x3 
blocks, i. e., as 



Furthermore, in light of the definition of the 
twist, (16.2), the two upper blocks of J are dimen¬ 
sionless, while its two lower blocks have units of 
length [16.9]. As a consequence, the two left blocks 
of J~ 1 are dimensionless, while its two right blocks 
have units of inverse length. Now, the blocks of Kc are 
computed from the corresponding relation in (16.26), 
thereby obtaining 

K C ii =J^(K M 7n +K I2 J',) 

+ Jn( K^Jn TK 2 2J21) 

Kci 2 = j£(K„/ 12 + K 12 J' 2 ) 

+ J? l (K{ 2 J' l 2 +K 22 y 22 ) 

Kc2i = Kj 12 

K C 22=Jg(K u / 12 + K 12 j' 2 ) 

+ J? 2 (K T l 2 J' l 2 +K 22 J' 22 ). 

It is now apparent that: Ken has entries with units 
of Nm, i. e., of torsional stiffness; K 02 and Kc 2 i have 
entries with units of N; and Kc 22 has entries with units 
of N/m, i. e., of translational stiffness. 

The outcome of the foregoing discussion is that 
a norm for K is possible, but not one for Kc, unless of 
course a characteristic length is introduced to render all 
the entries of Kc dimensionally homogeneous. A norm 
of a matrix is useful as it indicates how large the entries 
of the matrix are. We would like to characterize how 


stiff a robot is in both joint and Cartesian spaces. In joint 
space we could adopt any norm, but notice that the 2- 
norm, introduced in (16.8), would be misleading, as this 
norm would assign the value of the strongest joint to 
the overall robot stiffness. A more suitable norm would 
be the weighted Frobenius norm, introduced in ( 16.11 ), 
which would assign the rms value of the joint stiffnesses 
to the overall robot stiffness. 

To design a robot optimally, we would therefore aim 
to maximize the Frobenius norm of its stiffness matrix 
in joint space, while observing constraints on the robot 
weight, as stiffer joints lead to heavier joints if the same 
material is used for all joints. 

16.4.3 Elastodynamic Performance 

For a general design problem, not only the kinetostatic 
and elastostatic performances, but also the elastody¬ 
namic performance have to be considered. In this re¬ 
gard, we introduce the assumptions of Sect. 16.4.2, with 
the added condition that inertia forces due to the link 
masses and moments of inertia are now taken into con¬ 
sideration. 

The linearized model of a serial robot at the posture 
given by Oq, if we neglect damping, is 

MA0 + KA0 = Ar , (16.27) 

in which M is the nxn positive-definite mass ma¬ 
trix introduced in Sect. 16.4.1, while K was defined 
in Sect. 16.4.2 as the nxn positive-definite stiffness 
matrix in joint space. Both K and M were defined in 
joint-space coordinates, AO, representing the vector of 
joint-variable elastic displacements, as in Sect. 16.4.2. 
These displacements are produced when, as the joints 
are all locked at a value 6 0 and thereby become ideal 
linear springs, the robot is subject to a perturbation At, 
to nonzero initial conditions, or to a combination of 
both. 

Under free vibration, i. e., under a motion of the sys¬ 
tem (16.27) caused by nonzero initial conditions and 
a zero excitation Ar, the foregoing equation can be 
solved for AO 

AO =-DAO, D = M _1 K, (16.28) 

in which the matrix D is known as the dynamic ma¬ 
trix. This matrix determines the behavior of the system 
under consideration, as its eigenvalues are the nat¬ 
ural frequencies of the system and its eigenvectors 
the modal vectors. Let {cn, }j ! and {/, }j ! denote the 
sets of eigenvalues and the corresponding eigenvectors 
of D, respectively. Under the initial conditions [AO (0), 
A0(O)] t , in which A0(O) is proportional to the i-th 
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eigenvector of D and A 0(0) = 0, the ensuing motion 
is of the form A 0(t) = AO (0) cos cog [16.54]. 

Furthermore, under the change of variables given 
by (16.22). the model (16.27) changes to 

M.F 1 Ax + KJ~'Ax = J t Aw . 

If we now multiply both sides of the above equation 
by 7 _t , we obtain the elastodynamic model (16.27) in 
Cartesian coordinates, namely 

J~ T MJ~ 1 Ax + J _T KJ~ 1 Ax = Aw , 

in which the first matrix coefficient is the mass ma¬ 
trix Me in Cartesian coordinates, and the second is 
identified as Kc, which was introduced in (16.26), i. e., 

M c = J _t MJ _1 . (16.29) 

The elastodynamic model in Cartesian coordinates 
thus takes the form 

MeAx + KcAx = Aw . (16.30) 

Again, by virtue of the transformation (16.29), it is 
apparent that the mass matrix, like its stiffness coun¬ 
terpart, is not invariant under a change of coordinates. 
Moreover, in a revolute-coupled robot, all the entries 
of M have units of kg m 2 ; however, the entries of Me 
have disparate units. An analysis similar to that con¬ 
ducted in Sect. 16.4.2 for the stiffness matrix in Carte¬ 
sian space reveals that, if Me is partitioned into four 
3x3 blocks, then its upper-left block has units of mo¬ 
ment of inertia, its lower-right block has units of mass, 
while its off-diagonal blocks have units of kg m. 

Correspondingly, the dynamic matrix in Cartesian 
coordinates becomes 

D c = Mc 1 K c . (16.31) 

It is now a simple matter to prove that the dynamic ma¬ 
trix is invariant under a change of coordinates. Indeed, 
if we substitute transformations (16.26) and (16.29) 
into (16.31), we obtain 

D c = JM -1 J t J -t KJ -1 = JM K.T 1 , 


16.5 Other Robot Types 

Serial robots were fully developed, to a great extent, in 
the nineties, with SCARA systems capable of executing 
the standard manipulation task defined in Sect. 16.2.1 at 


in which the dynamic matrix D in joint coordinates can 
be readily identified in the final expression, and hence, 

D c = JDJ~' , (16.32) 

which shows that Dc is indeed a similarity transfor¬ 
mation of D. As a consequence, the dynamic matrix is 
indeed invariant under a change of coordinates, the two 
matrices thus sharing the same sets of eigenvalues, their 
eigenvectors being related by the same similarity trans¬ 
formation. That is, if the modal vectors in joint space - 
the eigenvectors of D - are denoted by {/,}" and the 
modal vectors in Cartesian space are denoted by {gi )■", 
then these two sets are related by 

gi = 3fi, i= 1,- n. (16.33) 

Therefore, the natural frequencies of the elastody¬ 
namic model are the same, regardless of the space in 
which they are calculated, while their natural modes of 
vibration change under a similarity transformation. 

Under an excitation of the form At = 0 0 cos tot and 
zero initial conditions, the system is known to respond 
with a harmonic motion of frequency to and magni¬ 
tude that depends on both to and the system frequency 
spectrum { &>, }j' [16.54], When to equals one of the nat¬ 
ural frequencies of the system, the response magnitude 
grows unbounded, a phenomenon known as resonance. 
For this reason, when designing a robot, it is imperative 
that its frequency spectrum does not involve any of the 
expected operation frequencies, which can be achieved 
by designing the robot with stiffness and mass matrices 
that yield a frequency spectrum outside of the frequency 
range of the operation conditions. 

This design task is not straightforward. Indeed, 
while the stiffness matrix in joint space is constant, 
the mass matrix is posture dependent, i. e., M = M (0 ). 
Because of this feature, the elastodynamic design of 
a robot is bound to be iterative: the design can be con¬ 
ducted for a straw-man task , i. e., a typical task in the 
target applications, thus defining a set of postures that 
lead in turn to a set of mass-matrix values. Then, the fre¬ 
quency spectrum for all these postures can be designed 
to lie above the frequency range of the straw-man task. 
Since the robot will eventually be commanded to exe¬ 
cute other tasks than the straw-man task, simulation of 
alternative tasks is required to ensure that the design is 
safe from a resonance viewpoint. 


the rate of two cycles per second. Other, more sophis¬ 
ticated robots started appearing in the 1980s. Parallel 
robots were a natural extension of serial robots. They 
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presented new challenges to researchers, as their for¬ 
ward kinematics turned out to be quite more complex 
than the inverse kinematics of serial robots. Nowadays 
a few parallel and hybrid, i. e., serial-parallel robots, 
are found in applications in industry and other envi¬ 
ronments. A remarkable type of robot that emerged in 
the 1990s is humanoids, with their own design and con¬ 
trol challenges. Flying and swimming robots are also 
developing fast. An outline of the design challenges 
brought about by these new types of robots is included 
below. 

16.5.1 Robots Mounted on a Fixed Base: 
Parallel-Kinematics Machines 

Under the umbrella parallel-kinematics machines 
(PKMs) are included robots comprising one base plate 
(BP), one moving plate (MP) and a set of limbs that 
couple the two. The earliest robot of this kind that at¬ 
tracted the attention of researchers is what is known as 
the Gough-Stewart platform (GSP), consisting of six 
limbs, each of the SPU type, where S, P and U stand for 
spherical, prismatic and universal joint, respectively. In 
fact, the first two belong to what is known as the six 
lower kinematic pairs [16.3] - the other four are R, H, 
C, and F, which are termed revolute, helical, cylindri¬ 
cal and planar pairs. The U joint is a concatenation of 
two R pairs with axes intersecting at right angles. Three 
of these six pairs, R, P and H, allow for 1-DOF motion 
between the two links that they couple; C allows for 2- 
DOF, while S and F allow for three. Each limb of the 
GSP thus allows for 6-DOF of the MP with respect to 
the BP. 

The first and foremost task when designing a PKM 
is the choice of both the type and the number of limbs. 
As to the latter, it varies from two to six. Apparently, the 
DOF of a limb should be at least that of the PKM under 
design. The early PKMs studied, mainly in the 1980s, 
were invariably 6-DOF systems. Clavel [16.6] patented 
in 1990 what may be the first PKM with reduced mo¬ 
bility, i. e., with a DOF = 3, namely, a robot that allows 
pure translations of its MP. Spherical robots, like Gos- 
selin’s Agile Eye [16.55], are also of reduced mobility. 
Different from Clavel’s Delta, all links of spherical link¬ 
ages undergo spherical motion, rotating about the same 
point - the center of the sphere. Delta is supplied with 
links that undergo other motions than those of their MP, 
purely translational. 

Other PKMs with reduced mobility appeared later, 
the best known being the H4, a 4-DOF system, patented 
in 2001 [16.56], and designed to implement Schonflies 
displacements (Sect. 16.2.1) of its MP. The forego¬ 
ing patent came as a response to the need of ever 
faster pick-and-place robots for the packaging and the 


electronics industry sectors, when it was apparent that 
the SCARA systems of the mid eighties had reached 
their limits because of their serial architecture. The H4 
is manufactured by Adept Technology, Inc. under the 
name Adept Quattro s650HS parallel robot. 

Motivated by the growing interest in fast robots 
with reduced mobility, two approaches to the design 
of PKMs with reduced mobility have been put forth: 
screw theory [16.57] and group theory [16.58]. By far, 
the most favored approach is the former, as screw the¬ 
ory is well established within the community of robot 
mechanicists. However, group theory offers features not 
found in screw theory, which is local in that its scope is 
limited to the velocity level. Group theory is applicable 
at the displacement level. What is at stake in this con¬ 
text is the topology of the robot, namely, the number of 
joints, links, limbs and the type of joints, what Harten- 
berg and Denavit [16.3] term the number synthesis 
and the type synthesis. These two activities correspond 
to the qualitative synthesis underlying the design pro¬ 
cess, which appears at the conceptual stage, as opposed 
to its quantitative counterpart; the latter appears at 
the embodiment and the detail stages of the design 
process [16.4]. Recently, a design paradigm based on 
the concept of complexity has been proposed [16.59], 
aimed at assessing the complexity of a linkage design 
variant based on the types of its joints. 

As an extension of the Delta robot, ABB Robotics 
developed the Flexpicker IRB360, a hybrid system that 
comprises a Delta robot in series with a revolute joint at 
its MP that adds one fourth DOF to the purely transla¬ 
tional motion of the Delta. The result is, thus, a hybrid 
Schonflies motion generator. ABB’s robot is used in the 
food-processing industry. 

A major bottleneck in the development of fast 
robots with reduced mobility is their structural stiffness. 
With the packaging and electronics industry requiring 
ever faster pick-and-place robots, capable of speeds 
greater than 3 cycles/s, high excitation frequencies are 
generated, which calls for structures much stiffer than 
those found in other applications areas. The ideal fast 
robot should have a parallel structure whose mass is 
negligible when compared to that of its payload and, 
at the same time, highly stiff. As the robots of interest 
are intended for the production of Schonflies displace¬ 
ments, any motion of their MP outside of this subgroup, 
namely, any tilt of the MP, is considered parasitical and 
should be of as small an amplitude as possible. This 
calls for a structure that exhibits a high rotational stiff¬ 
ness about any horizontal axis - under the assumption 
that the only rotational motion of the MP is about a ver¬ 
tical axis, which is usually the case. The challenge here 
is how to characterize this stiffness. A simple elasto- 
dynamic model of the structure is derived by assuming 
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that the limbs provide a linearly elastic suspension for 
the MP, which is assumed to be rigid. The structure can 
be regarded as a generalized spring [16.60], namely, 
a linearly elastic suspension that is fixed to one station¬ 
ary base at one end and carries a moving rigid body 
at another end. Moreover, the suspension allows full 6- 
DOF small-amplitude displacements of the body. What 
is meant by the latter is rotations that entail a small an¬ 
gle - as small as to allow for an approximation of the 
type sin 9 9, with 9 denoting the angle - and transla¬ 

tions small with respect to the dimensions of the body. 
The motion of the MP can then be represented using 
a unit screw [16.57] multiplied by a small angle 0. Un¬ 
der these conditions, the Cartesian stiffness matrix is of 
6x6, given that the screw is of dimension 6. 

In characterizing the rotational stiffness of PKMs, 
one may think of resorting to the eigenvalues and eigen¬ 
vectors of their stiffness matrix. There are some issues 
with this approach, however, for, in light of the different 
units of the four blocks of this matrix (Sect. 16.4.2) the 
eigenvalue problem associated with the matrix is mean¬ 
ingful only in a generalized sense [16.61], as explained 
presently. Let us recall the 6x6 Cartesian stiffness ma¬ 
trix Kc, introduced in Sect. 16.4.2, and let us denote 
with kj its i-th eigenvector, a unit screw, and with at ,- its 
corresponding eigenvalue. The generalized eigenvalue 
problem associated with Kc now can be stated as 

K C A:, = k,T kj, ki = ^ j , i = 1, ... 6 , (16.34) 

in which e, is the unit vector denoting the direction of 
the axis of the i-th eigenscrew, /<,, is the moment of the 
axis with respect to the origin and T is a permutation 
matrix that exchanges the top and the bottom blocks of 
a screw array, these two items being defined as 

r * qJ , p-i = e, X Pi + p,e, , (16.35) 


which means that A,- must be singular, thereby leading 
to the characteristic equation of K c , namely. 


det 


( t K " 

\K-ki 


K rt — Kl\ 

K„ ) 


= 0, 


(16.37) 


where subscript i has been deleted from at, for it is no 
longer needed. In light of the linear homogeneous form 
in k of the foregoing 6x6 matrix, its determinant is 
a sextic polynomial in k, the eigenvalue sought. 

It is worth mentioning that the set of screws does 
not form, properly speaking, a vector space, for screws 
do not have a zero - they include a three-dimensional 
unit vector, which hence cannot vanish. By the same 
token, the set of screws does not admit a scalar prod¬ 
uct. Indeed, if two screw arrays are dot-multiplied, their 
product is meaningless, because it involves additions 
of quantities with disparate units. The pertinent con¬ 
cept is reciprocity, the reciprocal product of two unit 
screws, si and .v? is defined via matrix T, introduced 
in (16.35), namely, sjTs 2 , which rightfully has units of 
length. When the reciprocal product of two unit screws 
vanishes, the screws are said to be reciprocal to each 
other. The physical significance of the concept lies in 
that a unit screw multiplied by an amplitude F with 
units of force produces a wrench ; one multiplied by 
an amplitude Q with units of frequency yields a twist. 
If the wrench happens to be a pure force with line of 
action intersecting the axis of the twist - the case of 
a force applied at a door whose line of action intersects 
the axis of the hinges - then wrench and twist are recip¬ 
rocal to each other in that the power developed by the 
wrench on the door undergoing a feasible twist is zero. 

Now we have a result that parallels one applicable 
to symmetric n x n matrices: 


Theorem 16.1 

The eigenvalues of K are real and the product at,/?, is 
non-negative, while the eigenvectors { k ,}® are mutually 
reciprocal. 


for i = 1, ... 6, O and 1 denoting the 3x3 zero and 
identity matrices, respectively, while p , is the position 
vector of a point on the axis of the eigenscrew k t , while 
Pi is the pitch of k t . The moment of a line £ can be 
interpreted as that of a unit force whose line of action 
coincides with £. The generalized eigenvalue problem 
of (16.34) now leads to 

** l k ,, V 1 ) • (16J6) 

for i = 1, ... 6. In light of its definition, (16.34), k t ^ 
0, and hence, (16.36) must admit a non-trivial solution. 


its proof being found in [16.62] and [16.63]. 

Further, decoupling of the stiffness matrix means 
finding a new coordinate frame in which its two off- 
diagonal blocks vanish. Such a decoupling is not al¬ 
ways possible, as shown below (See the corrigenda 
in [16.63]). It is known that the entries of the Carte¬ 
sian stiffness matrix change according with a similarity 
transformation applicable to screws, which involves 
both a shift of origin and a change of the orientation of 
the original frame [16.64]. If the latter is labeled JA and 
the new frame H, then J4 is carried into 'B by means 
of a shift of the origin given by vector d and a rota¬ 
tion matrix Q. In the sequel, the cross-product matrix 
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of a vector v e R 3 will be needed. This is known to be 
a skew symmetric matrix V with the property 

\x = v xx, V x e l 3 , V = CPM(w). 

(16.38) 

Now let D = CPM(rf), with d defined above as the 
displacement that takes the origin of JA into that of 2>. 
The components of a unit screw s in ® change to those 
in SA according to the similarity transformation 

[sU=S[s] 3 , S=(£ (16.39) 

the inverse of the 6 x 6 matrix S being 

S ' = (_qt d qt) • (16 ' 40) 

Accordingly, the components of the stiffness ma¬ 
trix change, under the same similarity transformation, 
as [16.63] 

[K]ji = rsr[K] s s _1 , (16.41) 

which, were it not for matrix T, looks like the similar¬ 
ity transformation of a n x n matrix. The presence of T 
is essential by virtue of the definition of the reciprocal 
product. Let, now, 


Theorem 16.2 

The stiffness matrix can be decoupled if and only if its 
off-diagonal block is singular. When this is the case, 
decoupling is achieved by a similarity transformation 
involving only a shift of origin. 


The design criterion to apply here is, thus, to aim at 
a singular coupling block of the Cartesian stiffness ma¬ 
trix. This can be achieved by means of finite elements, 
but these work for one single robot posture. However, 
with the help of the concept of generalized spring, Tagh- 
vaiepour et al. [16.65] showed that it is possible to 
obtain the stiffness matrix of deformable links using 
that obtained with finite element analysis (FEA) at one 
particular posture, then transforming this matrix to ob¬ 
tain its counterpart at an alternate posture. The details 
on how to compute the displacement vector d that helps 
decouple the stiffness matrix are given in [16.63]. 

Once the stiffness matrix is decoupled, the eigen¬ 
values of the rotational and translational blocks can be 
readily found, and assess the performance of the robot 
from the stiffness viewpoint. As a matter of fact, in the 
presence of a decoupled matrix, the characteristic equa¬ 
tion (16.37) reduces to 

d “( K ii K,, 1 )" 0 ' 116441 


[KU 


( Kr k ;,\ 
U;,) T k;J 



(16.42) 


The distinct blocks of the two matrices are related 
by 


Upon expansion, using the expression for the deter¬ 
minant of a block-matrix [16.66], the foregoing equa¬ 
tion leads to 

det(K„-)det(K„ - r 2 K“') = 0 , 
the characteristic equation in k readily following 


Kr = - K„D)Q t + DQ(K 2 - K I»Q t 

k;, = (QK, ( + DQK„)Q t 

K = QK„Q T 

the decoupling condition following upon setting K'., = 
O, which leads to 

Q t DQK„ = —K„ , (16.43) 


P(k) = det(K ff — K'K.K) = 0 , (16.45) 

which is apparently a cubic equation in k 2 , the outcome 
being that the eigenvalues of the decoupled 6x6 Carte¬ 
sian stiffness matrix occur in symmetric pairs, i. e., the 
six eigenscrews of the matrix occur in pairs of oppo¬ 
site chirality, while the absolute values of each pair are 
identical [16.62,63], 


Now, since D is a cross-product matrix, it is singu¬ 
lar - its null space is spanned by vector d. Based on 
Sylvester’s Theorem [16.31], which states that the rank 
of the product of two matrices is at most the smaller of 
the matrix ranks, the left-hand side of (16.43) is singu¬ 
lar, and hence, the right-hand side must also be singular, 
which is the decoupling condition sought. If block K rt 
is singular, then decoupling can be achieved by means 
of a shift of the origin, without changing the orientation 
of the frame, i. e., upon choosing Q = 1 in (16.43). This 
result is summarized below: 


16.5.2 Mobile Robots 

Mobile robots bear an architecture similar to PKMs. 
Different from the latter, their limbs are not attached 
to a base plate, but free to move. In fact, their links 
proximal to the ground end by either feet or wheels, 
thereby leading to weheeled or, correspondingly, legged 
mobile robots. Mobile robots termed hybrid have been 
designed with a combination of wheeled and legged 
limbs. These robots take on various morphologies, as 
outlined below. 
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Wheeled Robots 

The earliest of these robots were termed autonomous 
guided vehicles (AGV). They were provided with 
a chassis and two motors, to implement the 2-DOF 
proper of any terrestrial vehicle equipped with conven¬ 
tional - as opposed to omnidirectional - wheels. These 
systems were capable of following a trajectory printed 
on the flat ground of a manufacturing facility. They had 
thus a rather limited mobility, but their load-carrying 
capacity was virtually unlimited. 

Robots with conventional wheels are generally 
equipped with two pairs of pneumatic tires, that al¬ 
low them to adapt themselves to irregular surfaces. 
Moreover, these robots can either bear steerable wheels, 
like MDA’s Rover Chassis Prototype (RCP), or change 
course by means of maneuvers involving different an¬ 
gular velocities of their tires mounted on axles fixed to 
the chassis, like the Khepera robot. 

Extraterrestrial exploration has prompted the devel¬ 
opment of rovers, namely, wheeled robots intended for 
roaming on unstructured terrain, either soft, like the 
Moon’s or Mars’s regolith, or hard, like rocks. Design 
issues here pertain mainly to the wheels, as these must 
adapt themselves to a rich variety of conditions and ter¬ 
rain constitution. Rovers are currently designed to travel 
at very low speeds, sometimes of a few meters per day, 
the case of NASA’s Curiosity (This rover landed on 
Mars on August 6, 2012.). The main issue here is for 
the wheels to provide enough traction and prevent spin¬ 
ning without advancing. The latter is guaranteed by the 
use of individual motors on each wheel. Traction is en¬ 
hanced by means of grousers on the wheel periphery. 
Curiosity and RCP feature metal wheels with grousers. 

Legged Robots 

This kind of robots has evolved tremendously since the 
earlier systems developed in the eighties: Ohio state 
university (OSU) hexapod and the OSU adaptive sus¬ 
pension vehicle (ASV), among others. Current legged 
robots range from those with six legs, X-RHEX Lite, 
with a planar symmetry and three identical legs on each 
side of the body, or the multi-appendage robotic system 
(MARS), with its legs distributed along the vertices of 
a regular hexagon, to four-legged robots with an archi¬ 
tecture similar to that of mammals, like Big Dog and 
Cheetah. Special mention deserve bipeds, of which the 
best known are humanoids, with the architecture of the 
human body, namely, ASIMO (advanced step in inno¬ 
vative mobility) and its competitor, QRIO (quest for 
curiosity). Design issues vary, depending on the num¬ 
ber of legs. Hexapods offer a combination of static 
stability and maneuverability, which their four-legged 
counterparts do not. By far, humanoids are the highest 


demanding in terms of static and dynamic stability, but 
offer the highest mobility. 

Hybrid Robots 

Hybrid robots stem from the combination of the two 
foregoing types, wheeled with legged. These are sys¬ 
tems that look like wheeled robots, but that feature 
wheels instead of feet. They offer the advantages - and 
the disadvantages - of the two foregoing types. A recent 
example of this kind of robots is NASA’s ATHLETE 
(all-terrain hex-legged extra-terrestrial explorer) robot, 
that features six articulated legs stemming from the 
edges of a regular hexagon defining the robot body and 
ending in wheels. Its design is intended for service to 
the planned Lunar base. According to NASA, the 15- 
ton lunar habitat would be [16.67] 

mounted on top of the six-legged robot. The habi¬ 
tat could walk right off of the lunar lander, and 
then proceed to any desired location. Wheeled lo¬ 
comotion would be used for level ground; more 
challenging terrain could be negotiated with the full 
use of the flexible legs. 

16.5.3 Aquatic Robots 

Aquatic robots are currently being marketed for opera¬ 
tion on water, whether on the surface or underneath. An 
example of a surface robot is the Mantra, intended to 
act as a lifeguard and help swimmers in distress. Some 
aquatic robots have been designed with the morphol¬ 
ogy of fish [16.68]. One swimming robot designed with 
a hexapod morphology, featuring six flippers in lieu of 
legs is Aqua, developed at McGill University, and de¬ 
picted in Fig. 16.5. 

By far, the aquatic robot with the highest profile is 
the robotic submarine Blue fin, which became a celebrity 



Fig. 16.5 Aqua, an amphibious robot 
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in the search for the debris of Flight MHS370 (Flight 
went missing on March 7, 2014 at 16:41 GMT.). 

16.5.4 Flying Robots 

This field is becoming quite active, with some robots 
mimicking the morphology of insects, and falling 
into the category of micro-robots [16.69]; other fly¬ 
ing robots are designed as unmanned aerial vehicles 
(UAV) [16.70]. Intensive research is now being re¬ 
ported in the design and control of two novel types 
of UAVs, namely, drones and quadrotors. The con- 

16.6 Summary 

This chapter is devoted to the design of robots, 
with a focus on serial architectures. In this regard, 
we started by proposing a stepwise design proce¬ 
dure; then, we recalled the main issues in robot 
design. These issues pertain to workspace geome¬ 
try, and to the kinetostatic, the dynamic, the elasto- 
static and the elastodynamic performances. In doing 
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17. Limbed Systems 


A limbed system is a mobile robot with a body, 
legs and arms. First, its general design process is 
discussed in Sect. 17.1. Then we consider issues of 
conceptual design and observe designs of vari¬ 
ous existing robots in Sect. 17.2. As an example in 
detail, the design of a humanoid robot HRP-4C 
is shown in Sect. 17.3. To design a limbed system 
of good performance, it is important to take into 
account of actuation and control, like gravity com¬ 
pensation, limit cycle dynamics, template models, 
and backdrivable actuation. These are discussed 
in Sect. 17.4. 

In Sect. 17.5, we overview divergence of limbed 
systems. We see odd legged walkers, leg-wheel 
hybrid robots, leg-arm hybrid robots, tethered 
walking robots, and wall-climbing robots. To com¬ 
pare limbed systems of different configurations, we 
can use performance indices such as the gait sen¬ 
sitivity norm, the Froude number, and the specific 
resistance, etc., which are introduced in Sect. 17.6. 
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In this chapter, we refer a limbed system as a mobile 
robot which consists of a body, legs and arms. Its gen¬ 
eral concept is illustrated in Fig. 17.1. 

Let us define that a limbed system must have a body and 
at least one leg (lower limb) to support and to propel 
itself. A leg interacts with the environment by its end 
effector (foot). It can also have an arbitrary number of 


arms (upper limb) to manipulate target objects by their 
end effectors (hands). 

From a body-centered point of view, legs can be re¬ 
garded as special arms designed for manipulating the 
earth. This justifies us to treat both of the legs and the 
arms as limbs in a unified manner, thus we chose the 
word limbed systems as the title of this chapter. 
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17.1 Design of Limbed Systems 

General robot design process has already been dis¬ 
cussed in the previous 16, therefore, we will concentrate 
on the specific issues on limbed systems. A design 
process of actual limbed system can be seen in liter¬ 
atures [17.1-3]. A typical development process may 
takes the following steps: 

• Step 1 : Basic requirements. A designer must specify 
the class of tasks performed by the limbed system 
and its requirements, target velocity, payload, ex¬ 
pected environment, etc. 

• Step 2: Conceptual design. The number of legs and 
arms, their topology, and associated gait are de- 


Target object 



Fig. 17.1 General concept of a limbed system 


cided simultaneously. They must be designed at 
the same time, because the leg structure and gait 
are tightly coupled, thus they should not be con¬ 
sidered separately. Geometric dimensions of links 
are also determined based on the basic require¬ 
ments. 

• Step 3: Detailed design. A designer selects actua¬ 
tors, sensors, reduction gears, and other mechani¬ 
cal/electrical components to realize the conceptual 
design. Then the structural elements are designed to 
integrate the devices. 

• Step 4\ Evaluation. Evaluate the design by building 
a prototype model or performing computer sim¬ 
ulations. In our experience, the first design can 
never meet the basic requirement, and we are 
forced to return to Step 3 to change the motor 
type or the reduction ratio. Sometimes, the simu¬ 
lation results suggest us reconsidering the concep¬ 
tual design or even rethinking the basic require¬ 
ments. 

Figure 17.2 overviews the development process of 
a limbed system. Note that a development of a limbed 
system is an extremely dynamic process, rather than 
a static top-down development. The detail of each step 
will be discussed in the following subsections. 


Retargetting the design goal 



- class of tasks 

- target speed 

- payload 

- terrain profile 
etc. 


- number of legs 

- gait 

- number of arms 

- topology 

- geometric dimensions 


- actuators 

- reduction gears 

- sensors 

- computers 

- power source 


- total mass 

- joint speed 

- joint torque 

- power consumption 


Fig. 17.2 Typical develop¬ 
ment process of a limbed 
system 


17.2 Conceptual Design 

At the beginning, a designer must have a class of tasks 
in his mind, for example: 

• Enter a damaged nuclear power plant, climb stairs, 
and close specified valves 

• Explore a planet to seek a new lifeforms 

• Carry a person and climb stairs 

• Perform dances to entertain its audience, etc.. 


Note that it must be a class of tasks and should not 
be a single task, because the latter can be realized by 
a single degree-of-freedom machinery as expressed in 
the former chapter. 

A class of tasks should be compiled into more 
specific goals. For example, target speed of traveling, 
maximum step length, maximum payload, or expected 
terrain profiles (ex. maximum height of obstacles). 
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Those figures give a solid guide line for a design pro¬ 
cess. 

17.2.1 Conceptual Design Issues 

The goal of conceptual design is to decide the basic 
kinematic property of the robot, the number of legs and 
arms, limb topology, and geometric dimensions of the 
links. A robot designer should take into account the fol¬ 
lowing issues: 

• Gait type: Leg motion pattern for walking or running 
is called gait and it greatly affects limbed system de¬ 
sign. For example, a robot for static gait should be 
designed to have sprawled posture to maximize its 
stability. Whereas, a robot for dynamic gait is de¬ 
signed to have erect posture with high center of mass 
to get a longer time constant of its falling. The gait 
generation is discussed in 75. 

• Biomimesis: Some robots are designed to mimic the 
mechanical structure of a living creature as accurate 
as possible. This is beneficial for deeper under¬ 
standing of biological systems. For example, Quinn 
and Ritzmann have built a hexapod robot with 
kinematics that are remarkably similar to the cock¬ 
roach Blaberus discoidalis [17.4] (IdaMUESjEIl). 
Biomimesis is also meaningful in terms of enter¬ 
tainment applications. Hirukawa et al. developed 
biped walking dinosaur robots of Tyrannosaurus 
Rex and Parasaurolophus for an exposition [17.5]. 
Those robots were built on 30 % scale of the 
real dinosaurs due to safety issues; nevertheless, 
they could attract many audience by their realis¬ 
tic outlooks at the 2005 World Exposition, Aichi, 
Japan. 

• Bioinspired dynamics: To reproduce the robust and 
versatile locomotion of animals, some designers 
pay their attention on essential dynamics of the 
locomotion, rather than the mechanical similarity. 
Koditschek et al. emphasized this standpoint on the 
development of their hexapod robot RHex, which 
equipped with only six actuators and yet can run on 
various irregular terrain [17.6]. 

• Mechanical simplicity: It is preferred to realize the 

target tasks with a mechanism as simple as possi¬ 
ble. Therefore, the design is always biased to use 
fewer number of actuators. shows an 

example of robot design with minimum numbers of 
actuators. 

• Limb workspace: A limb must have at least three 
degrees of freedom (DOF) to freely locate the tip of 
a limb in 3-D space. It must have also at least six 
DOF for the arbitrary orientation of the end effector 
in 3-D space. 


• Load bearing: Proper joint assignment can reduce 
the joint torque to support the body weight. In 
ks&XZEEDSai, we can see a unique hexapod leg de¬ 
sign for efficient load bearing. 


Let us observe how abovementioned issues are reflected 
in the conceptual design of actual limbed systems. 

Underactuatedness Versus Redundancy 
We compare two monopods which were designed un¬ 
der different policies on actuation. Figure 17.3a is the 
3-D one-legged hopping machine developed by Raib- 
ert [17.1]. This robot is driven by one pneumatic 
cylinder in its leg and two hydraulic actuators in its hip 
joints. Therefore, it has only three actuated DOF while 
3-D rigid body motion requires 6. Despite its underac¬ 
tuatedness, it can freely travel in 3-D space keeping its 
body upright thanks to its balance control. 

Figure 17.3b shows the one-legged robot developed 
by Tajima and Suga [17.7]. It has seven actuated DOFs 
in total, three at the hip, one at the knee, two at the 
ankle, and one at the toe. Therefore, this robot is a re¬ 
dundant system having one extra DOF in addition to the 
six required for 3-D motion. This redundancy can help 
the robot to reduce the excessive joint speed required 
at hopping. As a result, successful hopping motion con¬ 
trol was realized by using servo motors with reduction 
gears for all joints. 

Serial Versus Parallel 

Legs of a limbed system can be configured either as 
a serial or parallel structures. In most cases, each leg 
of a biped robot is designed as serial chain of six ac- 



Fig.17.3a,b Monopod robot designs, (a) MIT 3-D One-Leg Hopper 
(1983); (b) Toyota monopod (2006) 


17.2.2 Case Studies 
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tuated DOF, where three are used in the hip, one in 
the knee and two in the ankle. HRP-2 L developed by 
Kaneko et al. [17.8], as shown in Fig. 17.4a, is one of 
such examples. For such robots, the most difficult part 


a) _| b) 



Fig.17.4a,b Biped robot designs, (a) HRP-2L (2001); 
(b) WL-16R (2003) 



Fig.17.5a,b Quadruped designs, (a) TITAN III (1984); 
(b) BigDog (2005) 


is the hip joint design. A hip joint mechanism must con¬ 
tain three axis of rotation (roll/pitch/yaw) bearing high 
torque caused by the upper body weight. This is serious 
especially for a robot to carry heavy loads. 

Takanishi et al. solved this problem by introducing 
legs of the Stewart platform for their WL-16R, a human 
carrying biped locomotor (Fig. 17.4b). Each leg con¬ 
sists of six linear actuators driven by DC-geared motors 
and ball screws. In this design, their robot can carry 
a human of 50 kg weight [17.9]. 

Static Gait Versus Dynamic Gait 
Figure 17.5a is TITAN III developed by Hirose and 
his colleagues [17.10]. This is one of the successful 
quadruped robots in 1980s. Each leg is a 3-D panto¬ 
graph mechanism driven (PANTOMEC) by three linear 
actuators (ball screw with DC motor) equipped on its 
body, thus it has 12 DOF. To perform statically bal¬ 
anced walk, this robot was designed to have a low center 
of mass and widely spread legs. 

Figure 17.5b is BigDog, an energy autonomous hy¬ 
draulic quadruped robot of 1 m tall. 1 m long, and 90 kg 
developed by Buehler et al. [17.11]. Each leg has one 
passive linear pneumatic compliance in the lower leg, 
three active joints for knee, hip pitch, and roll. There - 



Fig.17.6a,b Hexapod designs, (a) ASV (1986); (b) Gen¬ 
ghis (1989) 
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fore it has 12 actuated DOF and four passive DOF. 
This robot can demonstrate remarkable walking per¬ 
formance in an outdoor environment, as well as can 
maintain its balance even when kicked by a human. This 
ability of dynamic balancing is helped by its design of 
high center of mass with the leg assignment with nar¬ 
row lateral separation. 

Big Versus Small 

Design concept is deeply affected by the robot size. Fig¬ 
ure 17.6a shows one of the most famous hexapods, the 
adaptive suspension vehicle (ASV) developed by Wal¬ 
dron et al. [17.2, 12], The ASV is a hydraulically driven 
hexapod robot which can carry a parson over rough 
terrain. Its length and height are 5.2 m and 3.0m, re¬ 
spectively, and weighs 2700 kg. To walk around in 3-D 
space, each leg requires 3 DOFs, thus the robot has 18 
DOF in total. 

Figure 17.6b shows a small hexapod robot Gen¬ 
ghis (35 cm length and 1 kg weight) developed by 
Brooks [17.13]. This robot could also demonstrate ro¬ 
bust walking in 3-D space, but its weight is less than 
a thousandth of the ASV. As a result, Genghis does not 
need precise 3-D foot positioning for its walking con¬ 
trol. Each leg has only two DOF and the robot has 12 
DOF in total. Standing with three legs, the body posi¬ 
tion and orientation can be fully controlled as long as 
the feet is allowed to slip on the ground. 

Degree of Biomimesis 

We can increase the degree of biomimesis by adding 
DOF; however, fewer DOF is preferable in terms of en- 



b) 


Fig.17.7a,b Humanoid designs, (a) ASIMO (2000); (b) HRP-4C 
(2009) 



gineering. The first version of ASIMO has 26 DOF in 
total, 6 for each leg, 5 for each arm, one for each hand, 
and two for the head (Fig. 17.7a) [17.14]. As a biped 
walking humanoid robot, this is a reasonable configura¬ 
tion. 

Cybernetic human HRP-4C (Fig. 17.7b) was de¬ 
signed to be as close as a human in consideration of 
applications in entertainment industry [17.15, 16]. It has 
44 DOF in total, 7 for each leg, 6 for each arm, 2 for 
each hand and, three for its waist, 3 for its neck, and 
8 for its face. Its design detail is discussed in the next 
section. 


17.3 Whole Design Process Example 

In this section, we explain the design process of a hu¬ 
manoid robot HRP-4C. We do this to give the read¬ 
ers an example of the whole development process of 
a limbed system. The readers are also recommended 
to see the comprehensive development report for the 
humanoid robot LOLA written by Lohmeier [17.3]. 
Also please watch another successful humanoid de¬ 
signs in and l<E&M < JI>H»>-7t< Note that 

a general mechanical design and construction pro¬ 
cess have already been discussed in this handbook 
(Chap. 4). 

17.3.1 Conceptual Design of HRP-4C 

At the beginning of our project, we defined Cybernetic 
human as a humanoid robot with the following features: 


1. Have the appearance and shape of a human being. 

2. Can walk and move like a human being. 

3. Can interact with humans using speech recognition 
and so forth. 

Such robots can be used in the entertainment indus¬ 
try, for example, exhibitions and fashion shows. It can 
also be used as a human simulator to evaluate devices 
for humans. 

As the successor of our previous humanoid robots 
HRP-2 and HRP-3 [17.17, 18], we call our new hu¬ 
manoid robot HRP-4C, C stands for cybernetic human. 

To determine the target shape and dimensions of 
HRP-4C, we used the anthropometric database for 
lapanese population, which was measured and com¬ 
piled by Kouclii et al. [17.19], The database provides 
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the dimensions of different Japanese groups of ages and 
sex. Considering the entertainment applications like 
fashion show, we picked the average young female data 
(age from 19 to 29). Figure 17.8 shows part of the di¬ 
mensions provided by the database. 

To reproduce women’s graceful motion, we asked 
a professional walking model to perform walking, turn¬ 
ing, sitting on chair, and other motions. The perfor¬ 
mance was motion-captured and used to evaluate possi¬ 
ble joint configurations proposed for HRP-4C structure. 
From the captured data, the required joint workspace 
was estimated. In addition, we estimated the motor 
power during biped walking to design the appropriate 
leg joint configuration. 

Figure 17.9 shows the joint configuration we finally 
decided for HRP-4C. To reproduce the human motion, 
three DOFs (roll, pitch, yaw) were assigned for each of 
the waist and the neck joints. In addition, we put active 
toe joints to realize biped walking with wide stride. 

17.3.2 Mechanical Design 

Figure 17.10 shows the mechanism to drive the knees 
of FIRP-4C. The rotation of the servo motor is trans- 


Upper arm circumference: 253 



Fig. 17.8 Anthropometric data of average young Japanese 
female (after [17.19], all measures in mm) 


mitted to the input of the harmonic drive gear by way 
of the pulleys and the timing belt, and the output of the 
harmonic gear is connected to the lower leg link. By this 
way, we can obtain high torque output without backlash 
which is required for dynamic biped walking. Note that 



Fig. 17.9 Joint configuration of HRP-4C body (joints for 
the face and hands are omitted) 
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the pulleys and the timing belt offer us a flexible choice 
of the motor placement as well as the total reduction ra¬ 
tio. The belt compliance becomes negligible thanks to 
the high reduction ratio of the harmonic gear at the final 
stage. 

The same mechanisms of Fig. 17.10 are used for 
most of the joints of HRP-4C; however, there are some 
exceptions. The first example is the ankle joint. Fig¬ 
ure 17.11 illustrates the mechanism of the left lower leg 
link and the ankle. The ankle pitch and roll joints are 
driven by two servomotors (Motor #1 and Motor #2) 
embedded in the shin link. Motor #1 rotates the ball 
screw to create the linear motion via timing belt and 
pulleys. Then the ball screw’s nut pushes or pulls the 
connecting rod that finally drives the ankle pitch joint. 

Motor #2’s rotation is transmitted to the bevel gear 
#1, timing belt #2, and the second bevel gear. The bevel 
gear #2 rotates the harmonic drive gear and it finally 
rotates the ankle roll joint. This elegant yet compli¬ 
cated mechanism was necessary to realize the slim outer 
shape of the HRP-4C ankle which must have close di¬ 
mensions of human ankle. 

Figure 17.12 is the toe joint mechanism of HRP-4C. 
To avoid sole plate separation which may catch obsta¬ 
cles between the plates, the toe joint axis should locate 
as close as possible to the floor surface. For this pur¬ 
pose, a four bar linkage is used to drive the toe sole 
plate by the harmonic drive gear mounted on the heel 
sole plate. 

As already mentioned, harmonic drive gears are 
mainly used because they are backlash free. On the 
other hand, for the mechanical parts, which do not 
require backlash-less motion, we can use alternative de¬ 
vices. For this reason, servo motors with planetary gear 
heads were used as the actuators for facial expressions 
and hands. 

Figure 17.13 is the final mechanical design of 
HRP-4C (Fig. 17.13a) and its exterior (Fig. 17.13b). 
The covers for the arms, legs and body are made of fiber 
reinforced plastics (FRP) and the facial skin and hand 
skin are made of silicone rubber. 

17.3.3 Electrical System Design 

Figure 17.14 shows the electronics architecture of HRP- 
4C. The entire system is controlled by a single board 
computer of the PCI-104 form factor with Intel Pen¬ 
tium M 1.6 GHz. Two peripheral boards, the 10 ch 
controller area network CAN interface board and the 
force sensor interface board are stacked on the CPU 
board. 

To realize a human-like slender outlook, we used 
compact motor drivers which were developed in our 
previous project [17.18]. This device is an intelligent 




driver for a single servo motor, which takes an in¬ 
cremental encoder signal and provides motor current 
by proportional-integral-derivative (PID) control. The 
driver is small enough to be embedded in leg/arm links 
nearby the target motor. Since it has a CAN interface 
which allows cascade connection from the CAN inter¬ 
face board, the wiring can be simplified and reduced. 

In addition, we used another CAN-interfaced in¬ 
telligent drivers for small motors for the face and 
the hands. This driver can drive multiple motors by 
handling each incremental encoder, PID feedback, 
and the current control. We developed this type of 
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Fig.17.13a,b Mechanism of (a) HRP-4C and its (b) exterior 


Table 17.1 Principal specifications of HRP-4C 


Height 

1,600 mm 

Weight 

48 kg (with batteries) 

Total DOF 

44 DOF 

Face 

8 DOF 

Neck 

3 DOF 

Arm 

6 DOF x 2 

Hand 

2 DOF x 2 

Waist 

3 DOF 

Leg 

7 DOF x 2 

CPU 

Intel Pentium M 1.6 GHz 

Sensors Head 

CCD camera 

Body 

Inertial measurement unit (IMU) 

Sole 

6-axis force sensor X 2 

Batteries 

NiMH 48 V 


driver by modifying the device for our multifingered 
hand [17.20]. 

By using these motor drivers and the CAN interface 
board, we can control 44 servo motors in total. Further¬ 
more, a commercial inertial measurement unit (IMU) is 
connected via CAN to measure and control of the body 
posture during biped walk. 

The feet of HRP-4C are equipped with six-axis 
force sensors to measure the zero-moment point (ZMP) 
for its balance control. These sensor signals are inter¬ 
faced by the force sensor interface board. 

Table 17.1 shows the principal dimensions and 
specification of the final design of HRP-4C. It becomes 



■N 


^ 44 motors 
in total 


Fig. 17.14 Electronics architecture of HRP-4C 
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a humanoid robot with 1.6 m height, 46 kg weight, and 
44 actuated DOF. The on-board nickel metal hydride 
battery (NiMH) provides power for 20 min operation 
time without an external cable. HRP-4C’s human-like 


walking demonstration and a quick slip-turn experi¬ 
ment are shown in l<s>JEI3l!E3i and I^M’ll'Jttm, 
respectively. 


17.4 Model Induced Design 

The design of multilimbed robotic systems is a mecha- 
tronic challenge involving expertise in diverse fields 
such as mechanical design, electrical design, commu¬ 
nication and control, as well as software engineering. 
A task-oriented perspective on the overall system al¬ 
lows in some cases to incorporate a desired function 
directly into the system design rather than enforcing 
this behavior by active control. This idea can be found 
in many robotic designs. The most prominent exam¬ 
ple might be the use of counterweights for passive 
gravity compensation employed in robot manipulators. 
A clever mechanical design thus leads to a smaller 
power consumption of the actuators and consequently 
leads to a simplified system design. Passive gravity 
balancing has been realized via several approaches 
involving counterweights, springs (Fig. 17.15), and par¬ 
allel mechanisms [17.21]. Such a gravity compensation 
based on springs is for example employed in the robot 
PR2 from Willow Garage (Fig. 17.16) [17.22,23]. In 
this section, we will highlight several examples of 
legged robots where a desired function or property of 
the final system is realized at least partly via clever me¬ 
chanical design. 

17.4.1 Dynamic Walking 

Passive Dynamic Walking 

The concept of passive dynamic walking was pio¬ 
neered by McGeer (l<s&m'ii']jii-TtrM) i n the seminal 
paper [17.24], a class of purely passive mechanical 



Fig. 17.15 Passive gravity compensation by a spring: For 
Kab = Mgl the torques around the hinge produced by grav¬ 
ity and by the spring counter-balance each other 


systems on an inclined surface were analyzed which ex¬ 
hibit a periodic gait that is intrinsically, i. e., open loop, 
stable. The resulting periodic motion is characterized 
by the fact that the increase of energy due to the slope 
is exactly balanced by the energy loss at the impact. 
McGeer proposed to analyze these systems in terms of 
orbital stability using Poincare maps. 

A detailed analysis of simple walking models with 
point feet, which are based on this principle, can be 
found for instance in [17.25] or [17.26], Figure 17.17 
shows a conceptual model of a passive walking ma¬ 
chine which is able to walk down on an inclined slope. 
For analyzing the dynamics of this system, the fol¬ 
lowing two components are needed: 1) The open-loop 
dynamics of the swing phase: Under the assumption 
that the stance leg is standing firmly on the ground 
(without lift-off or slipping) until the next touch-down, 
the open loop dynamics can be derived from standard 
Lagrangian multibody dynamics. 2) An impact model 



Fig. 17.16 PR2 robot from Willow Garage using springs 
for passive gravity compensation (after [17.22,23]) 
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at the time of touch-down of the swing leg, which de¬ 
termines the state variables after the impact based on 
the state variables before the impact. Often, a perfectly 
inelastic impact is assumed. 

In a fully passive mechanism with given dynam¬ 
ics parameters, the motion is completely determined by 
the initial condition of the system. The mapping from 
the initial state to the state at the beginning of the next 
step is called stride function or return map. A periodic 
gait can be characterized by a fixed point of the stride 
function, and the search for a periodic gait can accord¬ 
ingly be formulated as an optimization problem on the 
stride function. In general, optimization may involve 
the initial condition as well as the dynamics parameters 
including the inclination of the slope. 

In order to analyze the stability properties of a peri¬ 
odic gait, McGeer proposed to interpret the stride func¬ 
tion as a Poincare map of the system. Local (orbital) 
stability can then be decided based on the lineariza¬ 
tion of the Poincare map. Since computing the Poincare 
map (i. e., computing the stride function) involves solv¬ 
ing the open loop dynamics of the system until the 
next touch-down, this often can only be implemented 
numerically. 

Figure 17.18 shows technical realizations of pla¬ 
nar passive dynamic walkers. The walker shown in 
Fig. 17.18a from Technical University of Delft has the 
simplest possible configuration with straight legs and 
point-feet. Premature impact of the swing leg with the 
ground is avoided by a stepping pattern on the in¬ 
clined slope. The mechanism in Fig. 17.18b was built 
at Cornell University based on ideas from McGeer’s 
seminal paper. The arc shape of the foot implements 
a rolling motion during stance. The ground clearance of 
the swing leg was realized by a passive knee joint. 



t \ 



Fig. 17.17 Conceptual model of a simple passive dynamic 
walker 


Figure 17.19 shows a fully passive walking machine 
in 3-D [17.27] (l<s>]KM3!l2£fl). It employs soft heels for 
improving the contact transition. The swinging motion 
of the arms allows for reducing the angular momentum 
around the vertical axis as well as side-to-side rocking. 
Knee extension is limited to a straight configuration by 
a locking mechanism. 

Semipassive Limit Cycle Walker 
Purely passive dynamic walkers can produce a sta¬ 
ble limit cycle when walking down a shallow slope. 
Thereby, the energy loss at each impact is compensated 



Fig.17.18a,b Examples of planar passive dynamic walkers 
with (a) point feet, (b) arc feet, and knees 
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by the energy gain from the decent along the slope, i. e., 
these systems are powered by gravity. 

Several attempts have been made for increasing 
the versatility and robustness of limit cycle walkers 
by adding active power sources. Figure 17.20 shows 
some successful designs reported in [17.28]. In these 
systems, simple actuation is used for substituting grav¬ 
itational power in order to allow for walking on level 
ground. While these systems are not fully passive, 
their energy consumption is much less than com¬ 
pared to fully actuated walking robots. The Cornell 
biped, as shown in Fig. 17.20a, utilizes actuation only 
in the ankle. The actuation is used such that each 
ankle joint extends when the opposite foot hits the 
ground. The Delft biped, as shown in Fig. 17.20b, 
instead is powered at the hip and utilizes passive 
ankles. 

The superior energy efficiency which can be 
achieved by dynamic limit cycle walkers has been 
demonstrated in the Cornell Ranger [17.29] that suc¬ 
ceeded to walk over a distance of 65 km without 
recharging or direct human intervention except for the 
remotely controlled steering. 



Fig. 17.19 A 3-D passive walker with upper body (af¬ 
ter [17.27]) 


Actuated Limit Cycle Walkers 
Passive and semipassive limit cycle walkers strongly 
rely on the passive dynamics for realizing a specific 
gait. In actuated limit cycle walkers the control of some 
joints is used for imposing a limit cycle on the remain¬ 
ing dynamics. The systems shown in Fig. 17.21 are 
fully actuated, except for some unactuated degrees of 
freedom, e.g., at the foot-ground contact, and/or under¬ 
actuation due to elasticity. 

In the framework of the hybrid zero dynam¬ 
ics [17.32], a cyclic variable like the virtual leg angle 
(angle from the hip to the stance foot) is introduced and 
the active degrees of freedom are controlled via partial 
feedback linearization to implement a virtual constraint 
which describes how the active degrees of freedom are 
coupled to the cyclic variable. The shape of the vir¬ 
tual constraint is then optimized aiming at a stable limit 
cycle for the remaining two-dimensional (2-D) zero dy¬ 
namics. A comprehensive treatment of this approach 
can be found in [17.33]. Such control approaches 
were implemented on the robots RABBIT [17.34] and 
MABLE [17.35]. While RABBIT employed position 
control in the active degrees of freedom, MABLE was 
implemented with series elastic actuators allowing for 
torque-based control (l<s>M5ES!EJl). 

The design of the 3-D walking machine Flame in 
Fig. 17.21c was also motivated by the aim of combin¬ 
ing limit cycle walking with active control [17.36]. The 
actuation of this robot was also based on series elastic 



Fig.17.20a,b Underactuated biped robots: (a) 3-D passive walker 
(Cornell Biped, 2003) developed at Cornell Univ. (after [17.30]); 
(b) Denise (2004) (after [17.31]) 
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Fig.17.21a-c Actuated limit cycle 
walkers (a) RABBIT (2002); (b) MA¬ 
BEL (2008); (c) Flame (2007) 


actuation and it was controlled by an event-based state 
machine in combination with active foot placement for 
gait stabilization. 

17.4.2 Template Models 

Conceptual models play an important role as tem¬ 
plates for the design and control of legged walking 
machines. Template models can also be used for gener¬ 
ating hypotheses about the neuro-mechanical control in 
biological systems [17.37]. In this context, they provide 
a possible answer to Bernstein’s DOF problem, how the 
redundancy lying in the human neuro-muscular system 
is coordinated. We will discuss two conceptual models 
which have proven useful for legged locomotion. 

The linear inverted pendulum (LIP) model has been 
proposed as a template model for bipedal walking 
(|«t>MMSE3). Figure 17.22a shows the main forces 
acting on the center of mass (COM) of a legged robot. 
If all the mass is concentrated in one single point, which 
implies mass-less legs, the acceleration forces of the 



Fig. 17.22 (a) The linear inverted pendulum model (af¬ 
ter [17.38]) (b) Definition of the centroidal moment pivot 
(after [17.39,40]) 


COM are counter balanced by the ground interaction 
force F acting at the ZMP p. If the COM is kept at 
a constant height, i. e., z = 0, one immediately obtains 
the dynamical equations of the linear inverted pendu¬ 
lum model 

x = -(x—p) . (17.1) 

z 

This model has been the starting point for many 
successful methods aiming at trajectory generation and 
feedback control of bipedal humanoid robots [17.41— 
43], More details on the related control issues can be 
found in 48. By using this model, the implicit assump¬ 
tion is made that the change of angular momentum 
around the COM, e.g., due to the swing leg dynamics. 



Fig.17.23a,b Bipedal robots (a) LOLA (2009); (b) HUBO 
(2005) 
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can be neglected. This assumption plays an important 
role also for the mechanical design. The bipedal robot 
LOLA (Fig. 17.23a) developed at Technical Univer¬ 
sity of Munich (TUM) is optimized for low inertia 
in the legs by moving the knee and ankle actuators 
into thigh [17.44]. In the robot HUBO from KAIST 
(Fig. 17.23b) the design aimed at concentrating the 
main inertia into the torso while keeping the mass of 
the moving parts in the arms and legs small [17.45]. 

Aiming at a better representation of the change of 
angular momentum the centroidal moment pivot (CMP) 
(Fig. 17.22b) was proposed as an additional ground ref¬ 
erence point [17.39, 40]. For the more general model in 
Fig. 17.22b the force balance results in 

8 + z , . r , 

x= - {x-p)-\ -. (17.2) 

z mz 

The CMP is defined as the hypothetical location of the 
ZMP for the case t = 0, such that the distance between 
the ZMP and the CMP is proportional to the change of 
angular momentum, i. e., 

c = P~ > (12.3) 

Fz 

where F z = m(g + z). In undisturbed human walking it 
was observed that the distance between ZMP and CMP 
is kept small [17.40], while the change of angular mo¬ 
mentum plays a stronger role for push recovery [17.46]. 

An alternative way of modeling the effect of the 
limb motion (e.g., swing foot dynamics) on the over¬ 
all dynamics is to consider a three-mass model, which 
has been utilized in the control of LOLA [17.44] and 
the Honda humanoid robot ASIMO [17.47]. 

While the LIP model has been introduced as a con¬ 
ceptual model for walking, it cannot describe running 
motions in which a flight phase is present. In biome¬ 
chanics, the spring-loaded inverted pendulum (SLIP) 
model (Fig. 17.24) was introduced in which the leg is 
modeled as a linear spring [17.48]. 

During the stance phase, the dynamics is given by 

mr = mg + k(- —- —--l)(r-r F ), (17.4) 

\\r-r F \ ) 



Fig. 17.24 The spring loaded inverted pendulum (SLIP) 
model 


where Iq and k are the rest length and stiffness of the 
spring, respectively, and r F is the location of the stance 
foot. During the flight phase, one simply has r = g, 
and it is assumed that the leg angle instantaneously ad¬ 
justs to an angle a for preparing the next touch-down. 
The model contains no damping and impact phenomena 
are eliminated by assuming mass-less legs. Therefore, 



Fig.17.25a,b Hopping robots developed in the MIT LegLab (a) 3-D 
Biped (1989); (b) Quadruped (1984) (after [17.1]) 



Fig.17.26a-c Robotic arms for implementing torque control: 
(a) WAM arm (Barrett Technologies) utilizing direct drive actua¬ 
tion in combination with cable transmissions, (b) the DLR light- 
weight-robot-III with integrated joint torque sensors, (c) the robot 
arm A2 (Mekka Robotics) based on series elastic actuators 
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Fig.17.27a,b Fully torque controlled humanoid robots: (a) DLRs 
robot TORO (2013); (b) Humanoid robot CB based on hydraulic 
actuation (CB-i, 2008) 



Fig.17.28a,b Humanoid upper body robots which allow whole 
body control based on joint torque: (a) DLRs torque controlled 
robot Justin and (b) Twendy-one from Waseda Univ. using 
SEA 


this model is conservative. It allows to generate peri¬ 
odic hopping trajectories representing running motions. 
Moreover, the contact force profile resulting from these 
motions shows similarity to data from human running 
motions. An extension of the SLIP dynamics to bipedal 
walking including double support phases was analyzed 
in [17.49]. 


The SLIP model can be considered as the simplest 
model of one-legged hopping and bipedal running. Its 
compliant leg function can be of interest for the design 
and control of future legged robots, which aim at highly 
dynamic motions like fast running, hopping, and jump¬ 
ing. 

Gait generation and control of compliant hopping 
robots have been studied in the 1980s by Raibert in 
the MIT LegLab. A detailed treatment of these de¬ 
velopments is summarized in [17.1]. The basic idea 
of Raibert’s dynamic locomotion approach can be ex¬ 
plained by considering a planar one-legged hopping 
robot. It can be shown that this system can be con¬ 
trolled in a surprisingly simple way by the combination 
of three separate components: 

1. Altitude control which is implemented by a fixed 
thrust during stance. 

2. Control of the forward velocity by placing the foot 
at a fixed distance from the hip at touch-down. 

3. Control of the body attitude during stance using ac¬ 
tuation at the hip. 

The same principle was applied to a one-legged 
hopper in 3-D, as well as to bipedal and quadrupedal 
hopping robots (Fig. 17.25). In the quadrupedal case, 
the combined effect of alternating pairs of legs was 
summarized by considering a single virtual leg. 

17.4.3 Robots for Control Based 
on Backdrivable Actuation 

A large number of algorithms for model-based control 
of robotic systems assume a rigid-body model of the 
robot in which the joint torques act as the control in¬ 
put. The capability of controlling the torque in practice 
is heavily influenced by the properties of the actuator 
transmission. While direct drive actuation allows a pre¬ 
cise feed-forward control of the actuator torque via the 
motor current, this leads to rather heavy drive units. The 
WAM arm from Barrett Technologies (Fig. 17.26a) uti¬ 
lizes direct drive actuation in combination with cable 
transmissions in order to store the leading four drives 
into the base of the robot. In this way, a large inertia 
of the moving elements is avoided. The use of gears 
with high transmission ratios allows to reduce the over¬ 
all weight, but leads to slower drive units which are 
hardly or not at all backdrivable. Joint torque sens¬ 
ing and serial elastic actuation present two hardware- 
oriented approaches for improving the torque controlla¬ 
bility of highly geared actuators. 

Joint torque sensing assumes a dedicated sen¬ 
sor for measuring the link side torque r and was 
applied in a series of lightweight robot arms de- 
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Fig.17.29a-c Bipedal robots using 
serial elastic actuators: (a) Spring 
Flamingo (1996) (after [17.50]); 

(b) M2V2 (2008) (after [17.51]); 

(c) COMAN (2011) developed at IIT 
utilizing SEA in selected joints (hip, 
knee, ankle) 


veloped at DLR, including the light-weight robot 
in Fig. 17.26b [17.52] and the upper body robot 
Justin as shown in Fig. 17.28a [17.53]. Recently, the 
same drive technology has been integrated in the de¬ 
sign of the torque-controlled humanoid robot TORO 
(Fig. 17.27a) [17.54] (l<otmJ21l). Torque sensing 
and control have also been employed in the hydraulic 
humanoid CB developed by Sarcos (Fig. 17.27b). 

In series elastic actuators (SEAs), the joint torque 
sensing is implemented indirectly by measuring the de¬ 
flection of an elastic element which is introduced into 
the drive train [17.55]. Early robots with compliant ac¬ 
tuators were the two armed systems COG [17.56] from 
MIT and Wendy [17.57] from Waseda university, which 
even allowed the adaptation of the compliance. More 
recently, series elastic actuators are used in the compli¬ 
ant arm A2 from Mekka (Fig. 17.26c) and the humanoid 
upper body robot Twendy-one from Waseda university 
(Fig. 17.28). They have been also applied to bipedal 
systems in the robots as shown in Fig. 17.29 (also watch 
ka MEEirai and |43>KM2EIil). 

While the mathematical models of torque controlled 
robots and robots with SEA have the same structure 
it should be mentioned that the source of elasticity is 


Motor inertia T Link inertia 



Fig. 17.30 Conceptual model of an elastic joint: A propor¬ 
tional feedback of the joint torque reduces the effective 
motor inertia and motor friction 


different in these approaches. In SEA, the deflective el¬ 
ement for the torque sensing represents the main source 
of elasticity and its role is to decouple the motor and 
link side dynamics for high frequency disturbances like 
impacts. A well-designed torque sensor instead should 
not introduce a big effect on the overall joint stiffness 
(in this case usually the gear introduces the main source 
of elasticity). Despite their conceptual similarity, the lit¬ 
erature of SEA and elastic joint robots also differs in 
the way that in the context of SEA usually the linear 
dynamics of a single compliant actuator is considered 
for the controller design, while in the context of elas¬ 
tic joint robots the full nonlinear multibody dynamics 
is considered. 

From a modeling and control point of view, the use 
of torque sensors in the joints leads to a robot model 
with elastic joints. For actuators with a high transmis¬ 
sion ratio, a common modeling assumption is that the 
kinetic energy of each rotor depends only on its own 
spinning motion and not on the rigid body motion of the 
other joints [17.58]. Let the motor and link side joint an¬ 
gles be denoted by 9 G R" and q e R", then the reduced 
flexible joint model (Chap 11 for a detailed derivation) 
is given by 

M(q)q+ C(q,q)q + g(q) = T + tf, q , (17.5) 

B0 + T = T m + Xf Q , (17.6) 

where ML/), CL/, q)q. and g(q) represent the link side 
inertia matrix, the centrifugal and Coriolis forces, and 
the gravity term of the rigid body dynamics. The motor 
inertia is given by the diagonal matrix B. The vector r 
represents the joint torques. In the case of an elastic 
joint with linear joint stiffness k, the torque is given 
by t j = kj(6j — qi). The terms and represent 
friction terms at the link side and the motor side, re¬ 
spectively. 
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Fig. 17.31 The torque sensing based joint technology of DLR’s light 
weight robot arm 


A purely proportional feedback of the joint torques 


design of torque-controlled robots. Since the effec¬ 
tive motor side friction can be reduced by a simple 
proportional torque feedback but link side friction is 
not affected, we conclude that the mechanical design 
should aim at low friction at the link side and the torque 
sensing should be implemented at the power output 
side of the gear. Since the assembly of gear and torque 
sensor is assumed to behave like an ideal spring, any 
masses of moving parts in the transmission should be 
kept low in order to minimize unmodeled dynamics 
which would affect the validity of (17.7). As an exam¬ 
ple of a successful design, Fig. 17.31 shows a cross- 
section of a joint of the torque-controlled DLR-light- 
weight-robot-III [17.52] in which a harmonic drive 
transmission is combined with a strain gauge-based 
torque sensor. 

Instead of the simple proportional control of (17.7), 
a complete torque controller can be designed based on 
the torque dynamics 

BK 1 i + r = r m + Zf_ g + B q , (17-9) 

resulting from (17.6) and r = K(9—q). Torque tracking 
is achieved via 


r m = —(K r — 7)r + K t u (17.7) 

has the effect that the motor dynamics equation (17.6) 
shows a controlled behavior as 

K^BG + r = u + K- 1 T t '0 . (17.8) 

Let us assume that K T is chosen as a diagonal matrix. 
Then we see that the matrix K~*B corresponds to an ef¬ 
fective (i. e., virtual) motor inertia matrix. By choosing 
the gains in K T larger than unity, we can thus reduce the 
effective motor inertia (Fig. 17.30). Moreover also the 
effects of motor friction are reduced such that a torque 
controlled joint ultimately exhibits a backdrivable be¬ 
havior even if the underlying gear mechanism is not or 
only hardly backdrivable. 

From this interpretation of torque feedback, we 
can derive some general guidelines for the mechanical 


On = hi + + Tf,e + K q Bq + u( r d - r) , 

(17.10) 

where u {Td — r) represents a proportional-derivative 
(PD) or PID control law and the nonnegative fac¬ 
tor K q < 1 is chosen less than 1 in order to avoid 
an overcompensation of the link side acceleration. In 
the context of SEA such a controller was proposed 
based on a linear system representation of a single joint 
in [17.55], Cascaded torque control of a single SEA- 
based joint using an inner loop velocity controller has 
been analyzed in [17.59, 60], also using a linear model. 
In the context of torque-controlled elastic joint robots, 
in [17.61] a combination with a Cartesian impedance 
controller was proven to be stable for the full nonlinear 
dynamics based on the stability theory of cascaded sys¬ 
tems (using K q = 1). Passivity-based control has been 
analyzed in [17.62]. 


17.5 Various Limbed Systems 

In this section, we explore the divergence of the limbed 
system design. 

17.5.1 Odd Legged Walkers 

A robot can be designed by pure engineering. For 
example, the tripodal walking robots STriDER (self- 


excited tripodal dynamic experimental robot) 1 and 
2 (Fig. 17.32) can demonstrate a unique locomotion 
which have never developed by evolution [17.63] and 
It has three identical legs each of them 
has four DOF, three at the hip and one at the knee. 
Supporting the body with two legs, the robot takes a dy¬ 
namic step by swinging the rest leg. Since the body flips 
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with 180 o about the aligned pelvis j oints connecting the 
support legs, the robot can continue its dynamic walk. 

Five legged walkers have been developed by some 
groups. For example, Besari et al. developed a five¬ 
legged robot inspired from starfish, and applied rein¬ 
forcement learning to obtain its optimal gait [17.64]. 

17.5.2 Leg-Wheel Hybrid Robots 

Unlike animals or insects, a robot can be designed 
to have wheels which can rotate infinitely. By mixing 
the efficiency of wheels and the flexibility of legs, we 


can expect a robot of maximum terrain adaptivity with 
minimum power consumption. Figure 17.33 shows ex¬ 
amples of such a design concept. 

Figure 17.33a shows a stair climbing of a biped leg¬ 
wheeled robot developed by Matsumoto et al. [17.65]. 
The robot is a planer biped with telescopic legs, but the 
tip of each leg is equipped with a powered wheel. Dur¬ 
ing a single leg support phase, the robot is controlled 
as a wheeled inverted pendulum. In addition, a con¬ 
troller was developed to realize the smooth transient 
between single support and statically stable double sup¬ 
port phase. 

Figure 17.33b is RollerWalker developed by Hi- 
rose et al. [17.66,69]. RollerWalker is a 12 DOF 
quadruped robot equipped with passive wheels on the 
tip of legs. It uses roller-skating mode on a flat floor, 
while it can walk on an uneven terrain by retracting the 
passive wheels (l<s>®iliLi!lS£l). 

Figure 17.33c is RHex developed by Buehler, 
Koditchek et al. [17.67]. Although it was originally in¬ 
spired from the locomotion of cockroach, RHex has 
only six active DOF, that is, one actuator for each hip. 
Moreover, the legs can rotate full circle around the 
pitch axis. By this unique design, RHex can walk and 
run over rugged, broken, and obstacle-ridden ground 
(|4S®ZES!EH). Recently, it also demonstrated biped 
running with its rear legs [17.70]. 

Figure 17.33d is Whegs II, another cockroach- 
inspired robot developed by Allen et al. [17.68]. This 
robot has only 4 active DOFs, one for propulsion, 
two for staring and one for body flexion. Each leg is 
equipped with three spring-loaded spoke and driven by 



Fig.17.33a-d Leg-wheel hybrid 
robots, (a) Biped leg-wheel robot 
(1998); (b) Roller walker (1996); 

(c) RHex (2001); (d) Whegs II (2003) 
(after [17.65-68]) 
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Fig. 17.35 Dante II (1994) (after [17.71]) 


Fig.17.34a,b Leg-arm hybrid robots (a) MELMANTIS-1 
(1996); (b) Yanbo3 (2003) (after [17.72,73]) ◄ 

the same actuator (l<s3>M3Bl!EEi). Whegs II can realize 
comparable mobility to RHex, while it uses fewer actu¬ 
ators. 

17.5.3 Leg-Arm Hybrid Robots 

Another design concept is a leg-arm hybrid robot. Since 
legs have inherently many degree of freedom, it is pos¬ 
sible to use them as manipulators. By this way, we 
can minimize total DOF, complexity, weight and power 
consumption of the walking robot. MELMANTIS-1 
(Fig. 17.34a) developed by Koyachi et al. is a hexapod 
walker of 22 DOF, which can transform its legs into ma¬ 
nipulators [17.72]. The robot can manipulate an object 
by two legs while standing with four other legs, after 
traveling by six legs with the maximum stability. 

Yanbo3 is a biped walker of 8 DOF developed by 
the group of Ota et al. [17.73]. It is designed to have 
minimum DOF necessary for a biped robot as well as 
for a manipulator when it is in the single support. In 
Fig. 17.34b, the robot is pressing the elevator button by 
its foot. 

17.5.4 Tethered Walking Robots 

Figure 17.35 shows Dante II, an eight-legged tethered 
walking robot developed by CMU Field Robotics Cen¬ 
ter in 1994. It was used at an Alaskan volcano for 
scientific exploration. To descend down steep crater 
walls in a rappelling-like manner, the robot uses tether 
cable anchored at the crater rim [17.71], Hirose et al. 
also developed tethered quadruped for construction 
work [17.74], 

17.5.5 Wall-Climbing Robots 



Wall-climbing robots are characterized by their foot 
mechanisms and leg configurations. The vital part is the 
foot mechanism to generate pulling force and the use of 
vacuum suction cups, electromagnets (for steel wall), 
adhesive materials, or miniature spine array has been 
proposed. 

Figure 17.36a shows a wall-climbing quadruped 
NINJA-1 developed by Hirose et al. [17.75] Each foot 
of NINJA-1 is equipped with a specially designed 
suction pad which can minimize its vacuum leakage. 
Another reliable wall-climbing robot with suction cups 
was developed by Yano et al. [17.76] 

Fig.17.36a,b Wall-climbing robots, (a) NINJA-1 (1991); 
(b) RiSE (2005) (after [17.75,77]) ◄ 
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Figure 17.36b shows a wall-climbing hexapod RiSE 
developed by Kim et al. [17.77, 78]. Each foot of RiSE 
is equipped with arrays of miniature spines observed in 
some insects and spiders. The robot can reliably climb 


on a wide variety of outdoor surfaces including concrete, 
stucco, brick, and dressed sandstone. As a wall-climbing 
robot more recently developed, watch Stickybot III in 
l<gf' and Waalbot in Idgf'ii »»»!••»■ 


17.6 Performance Indices 

In this section, we introduce useful performance in¬ 
dices, that can be used to evaluate legged robots of 
different configurations. 

17.6.1 Stability Margins 

The stability margin was originally proposed for a de¬ 
gree of stability of the statically walking multilegged 
robots by McGhee and Frank [17.79]. By neglecting the 
inertial effects caused by body and leg acceleration, we 
can guarantee the robot keep balance if the projection of 
center of mass (COM) exists inside the support polygon 
as shown in Fig. 17.37. (Note: In multilegged robot re¬ 
search, the word support pattern is frequently used for 
support polygon [17.2, 79, 80].) 

For a given configuration of a walking robot, the 
stability margin S m is defined as the minimum distance 
of the vertical projection of COM to the boundaries of 
the support pattern in the horizontal plane as illustrated 
in Fig. 17.38a. 

In addition, an alternative index was proposed to 
obtain the optimal gait analytically. That is the lon¬ 
gitudinal stability margin Si which is defined as the 
minimum distance from the vertical projection of COM 
to the support pattern boundaries along the line parallel 
to the body motion. 

For a dynamic walking robot, we can define a stabil¬ 
ity margin as the minimum distance of the ZMP to the 
boundaries of the support polygon, since ZMP is the 
natural extension of a projected COM on the ground. 



Fig. 17.37 Support polygon (support pattern) of multi¬ 
legged robot 


This fact has already been mentioned in the original 
work of ZMP [17.81], and has been implicitly used 
by many researchers. Explicit definition of this ZMP 
stability margin can be seen, for example, in Huang 
et al. [17.82], 

For a legged robot on a rough terrain, Messuri and 
Klein defined the energy stability margin as the mini¬ 
mum potential energy required to tumble the robot as. 


Se = mm(Mghi) , (17.11) 

i 

where /z, is the COM height variation during the tumble 
around the z-th segment of the support polygon and M 
is the total mass of the robot [17.83]. This concept is 
widely accepted, and there are some proposals for im¬ 
provement [17.14, 84], 

17.6.2 Gait Sensitivity Norm 

Hobbelen and Wisse proposed a gait sensitivity norm 
(GSN) as a disturbance rejection measure of limit cycle 
walking machines [17.85]. GSN is defined as 


GSN = 



(17.12) 


where e is the set of disturbances and g is the gait in¬ 
dicator which characterizes the failure mode , motion 
finally end up a tumble. For a 2-D limit cycle walker, 
they took floor irregularities (step height to walk over) 
and step time as e and g , respectively. 

For real robots, GSN can be experimentally ob¬ 
tained from the response of gait indicators g to a single 




Fig.17.38a,b Definition of stability margins, (a) Stability margin; 
(b) Longitudinal stability margin 
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disturbance eo as 
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(17.13) 


where g k (i) is the z-th gait indicator value at k steps after 
the disturbance eo has applied, q is the number of gait 
indicators. GSN has a good correlation with the basin of 
attraction, hence it can be used for a design of a robust 
walking robot and its controller. 

17.6.3 Duty Factor 

and Froude Number 


One of the useful indices for walking machines is 
a duty factor fi defined as 

(support period) 

(cycle time) 

Duty factors can be used to make the distinction 
between walks and runs, since we have /J > 0.5 for 
walking and /) < 0.5 for running [17.87], 

Froude number is used in fluid mechanics to explain 
the behavior of surface waves. Since both of surface 
waves and legged locomotion are dynamic motion in 
gravity, Alexander used it to characterize animal lo¬ 
comotion [17.87,88]. He calculated a Froude number 
by 


Throughout this chapter, we have observed various 
walking robots which might fit best for certain en¬ 
vironment and purpose. In some cases, however, we 
need to compare walking robots which have different 
masses, sizes, and leg numbers by using a certain per¬ 
formance index. Such an index should be dimensionless 
like a Mach number or a Raynords number in fluid 
mechanics. 


V 2 

Fr 2 = — , (17.14) 

gh 

where V is the walking or running speed, g is the 
gravity acceleration, and h is the height of hip joint 
from the ground. He showed animals of different 
sizes uses similar gaits when they travel with equal 
Froude numbers. Especially, most animals changes 


Specific resistance £ 



Fig. 17.39 Gabrielli-von Karman diagram (after [17.86]) 
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their gait from walking to running at the speed of 
Fr 2 = 1. 

Froude number is also defined as 

Fn = , (17.15) 

Vgh 

which is the square root of Fr 2 and can be used non 
dimensional speed of animals or legged robots. 

17.6.4 Specific Resistance 


The specific resistance is another important dimen¬ 
sionless number, which is used to evaluate the energy 
efficiency of a mobile robot. 

Gabrielli and von Karman discussed the perfor¬ 
mance of various vehicles using the power consumption 
per unit distance. That is 


E 

~Mgd ’ 


(17.16) 


where E is the total energy consumption for a travel of 
distance d, M is the total mass of the vehicle and g is 
gravity acceleration [17.89]. Note that when we push 
a box of mass M for a distance cl on a floor with a fric¬ 
tion coefficient p, we consume the power Mg fid and the 
specific resistance becomes e = p. Therefore, we can 
say that the specific resistance indicates how smooth the 
locomotion is. 

In the original work, Gabrielli and von Karman 
plotted the specific resistance as the function of speed 
for various vehicles (Fig. 17.39). This is called the 
Gabrielli—von Karman diagram and it was used to 
compare various styles of locomotion by Umetani and 
Hirose [17.90]. Gregorio et al. also showed the specific 
resistance of recent walking robots including their ef¬ 
ficient hopping robot, ARL monopod [17.86]. Collins 
et al. developed robots based on passive-dynamics with 
small actuators and achieved energy efficiency close 
to human walking [17.91]. In the paper, the specific 
resistance is referred to as specific energetic cost of 
transport. 
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Linear inverted pendulum mode 

available from http://handbookofrobotics.org/view-chapter/17/videodetails/512 
Hexapod robot Ambler 

available from http://handbookofrobotics.org/view-chapter/17/videodetails/517 
Hexapod ParaWalker-ll 

available from http://handbookofrobotics.org/view-chapter/17/videodetails/520 
Cockroach-like hexapod 

available from http://handbookofrobotics.org/view-chapter/17/videodetails/521 
Bipedal humanoid robot: WABIAN 

available from http://handbookofrobotics.org/view-chapter/17/videodetails/522 
Cybernetic human HRP-4C walking 

available from http://handbookofrobotics.org/view-chapter/17/videodetails/524 
Cybernetic human HRP-4C quick turn 

available from http://handbookofrobotics.org/view-chapter/17/videodetails/525 
Development of a humanoid robot DARwin 

available from http://handbookofrobotics.org/view-chapter/17/videodetails/526 
Passive dynamic walking with knees 

available from http://handbookofrobotics.org/view-chapter/17/videodetails/527 
Intuitive control of a planar bipedal walking robot 

available from http://handbookofrobotics.org/view-chapter/17/videodetails/529 
IHMC/Yobotics biped 

available from http://handbookofrobotics.org/view-chapter/17/videodetails/530 
Torque controlled humanoid robot TORO 

available from http://handbookofrobotics.org/view-chapter/17/videodetails/531 
3-D passive dynamic walking robot 

available from http://handbookofrobotics.org/view-chapter/17/videodetails/532 
Biped running robot MABEL 

available from http://handbookofrobotics.org/view-chapter/17/videodetails/533 
STriDER: Self-excited tripedal dynamic experimental robot 
available from http://handbookofrobotics.org/view-chapter/17/videodetails/534 
Roller-Walker: Leg-wheel hybrid vehicle 

available from http://handbookofrobotics.org/view-chapter/17/videodetails/535 
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RHex rough-terrain robot 

available from http://handbookofrobotics.org/view-chapter/17/videodetails/536 
Whegs II: A mobile robot using abstracted biological principles 
available from http://handbookofrobotics.org/view-chapter/17/videodetails/537 
Stickybotlll climbing robot 

available from http://handbookofrobotics.org/view-chapter/17/videodetails/540 

Waalbot: agile climbing with synthetic fibrillar dry adhesives 

available from http://handbookofrobotics.org/view-chapter/17/videodetails/541 
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18. Parallel Mechanisms 


Jean-Pierre Merlet, Clement Gosselin, Tian Huang 


This chapter presents an introduction to the kine¬ 
matics and dynamics of parallel mechanisms, also 
referred to as parallel robots. As opposed to classi¬ 
cal serial manipulators, the kinematic architecture 
of parallel robots includes closed-loop kinematic 
chains. As a consequence, their analysis differs 
considerably from that of their serial counterparts. 
This chapter aims at presenting the fundamen¬ 
tal formulations and techniques used in their 
analysis. 
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18.1 Definitions 


A closed-loop kinematic chain is one in which the links 
and joints are arranged such that at least one closed loop 
exists. Furthermore, a complex closed-loop kinematic 
chain is obtained when one of the links - other than 
the base - has a degree of connectivity greater than or 
equal to 3, i. e., one link other than the base is connected 
through joints to at least three other links. A parallel 
manipulator can be defined as a closed-loop mechanism 
composed of an end effector having n degrees of free¬ 
dom (DOF) and a fixed base, linked together by at least 
two independent kinematic chains. 

An example of such a structure was patented in 
1928 by Gwinnett [18.1] to be used as a platform for 
a movie theatre. In 1947, Gough [18.2] established the 
basic principles of a mechanism with a closed-loop 
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kinematic structure (Fig. 18.1) that allows the position¬ 
ing and the orientation of a moving platform so as to test 
tire wear and tear. He built a prototype of this machine 
in 1955. 

For this mechanism, the moving effector is a hexag¬ 
onal platform whose vertices are all connected to a link 
by a ball-and-socket (spherical) joint. The other end of 
the link is attached to the base by a universal joint. 
A linear actuator allows the modification of the to¬ 
tal length of the link; this mechanism is, therefore, 
a closed-loop kinematic structure, actuated by six lin¬ 
ear actuators. 

In 1965, Stewart [18.3] suggested the use of such 
a structure for flight simulators and the Gough mecha¬ 
nism is sometimes referred to as the Stewart platform 
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or the Gough-Stewart platform. The same architecture 
was also concurrently proposed by Kappel as a motion 
simulation mechanism [18.4], Nowadays, the Gough 
platform and its variants are the platform of choice 
for many applications, for example, flight simulators. 
This application provides a convincing illustration of 
the main advantage of parallel robots namely, their 
load-carrying capacity. Indeed, while the ratio of the 
mass of the payload over the mass of the robot is typ¬ 
ically smaller than 0.15 for serial 6R industrial robots, 
this ratio can be larger than 10 for parallel structures. 
Another advantage of the Gough platform is its very 
good positioning accuracy (although it has been ques¬ 
tioned [18.5]). This high accuracy arises from the fact 
that the legs work essentially in tension/compression 
and are subjected to virtually no bending - thereby 



leading to very small deformations - and from the 
fact that the errors in the internal sensors of the 
robot (measurement of the lengths of the legs for the 
Gough platform) are mapped into very small errors 
of the platform position. Parallel robots are also al¬ 
most insensitive to scaling (the same structure can 
be used for large or micro robots) and they can be 
built using almost any type of joint, actuator, or trans¬ 
mission, including exotic ones such as tape spring 
and flexure joints [18.6] or binary actuators [18.7]. 
The main drawbacks of parallel robots are their small 
workspace and the singularities that can appear within 
this workspace. However, large workspaces may be ob¬ 
tained using wire transmissions (see, for example, the 
Robocrane [18.8]): such robots will be specifically ad¬ 
dressed in Sect. 18.10 as their analysis is somewhat 
different. Apart from the Gough platform, the most suc¬ 
cessful designs of parallel robots are the Delta robot 
proposed by Clavel [18.9] (Fig. 18.2) and some planar 
parallel robots. The most common planar parallel robots 
have three identical legs each having an RPR or RRR 
architecture, where the underlined joint is the actuated 
one (such robots are often denoted as 3—RPR (3 —RRR\ 
Fig. 18.3). 

The geometric arrangement of the joints and links 
of the Delta structure provides three translational de¬ 
grees of freedom at the platform. Numerous other 
types of parallel robots have been proposed in the re¬ 
cent years. Although most existing architectures are 
based on the intuition of their designer, the synthesis 
of parallel mechanisms can be dealt with systemati¬ 
cally. An overview of the main approaches to the type 
synthesis of parallel mechanisms is given in the next 
section. 
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18.2 Type Synthesis of Parallel Mechanisms 


Determining all potential mechanical architectures of 
parallel robots that can produce a given motion pattern 
at the moving platform is a challenging problem. Sev¬ 
eral authors have addressed this problem, referred to as 
type synthesis. The approaches proposed in the litera¬ 
ture can be divided into four groups: 

• Approaches based on graph theory. The enumera¬ 
tion of all possible structures having a given number 
of DOFs can be performed by considering that there 
is only a finite set of possible kinematic pairs, and 
hence a very large, but finite, set of possible struc¬ 
ture combinations (e.g., [18.10]). Classical mobility 
formulae - for example, the Chebychev-Griibler- 
Kutzbach formula - are then used to determine the 
number of DOFs in the structure. Unfortunately, 
these formulas do not take into account geometric 
properties of the mechanism that can change the 
number of DOFs of the platform. Although many 
parallel robots have been successfully invented and 
developed for industrial uses, based on graph the¬ 
ory - the Neos Tricept [18.11] and the Sprint Z3 
head of DS Technologie Gmbh, for example - the 
synthesis approaches based strictly on graph theory 
can produce only limited nonoverconstrained paral¬ 
lel robots and have been largely superseded by the 
other methods described below. 

• Approaches based on group theory: The motion of 
a rigid body has the special structure of a group, the 
displacement group. Subgroups of the group of dis¬ 
placements, such as the spatial translations or all the 
translational and rotational motions about all axes 
that are parallel to the axis defined by a given vec¬ 
tor (Schonflies motion [18.12]), play an important 
role as they can be combined through the intersec¬ 
tion operation [18.13] when elements of subgroups 
act on the same rigid body. Type synthesis con¬ 
sists in determining all the possible subgroups to 
which the different kinematic chains that will con¬ 
stitute the legs of the robot may belong to so that 
their intersection leads to the desired motion pattern 
for the platform. The synthesis approaches based 
on group theory led to the discovery of numer¬ 
ous possible architectures [18.14]. Nevertheless, the 
displacement group has special properties that are 
not reflected by its group structure alone. Also, 
the approach is limited to motion patterns that can 
be described by a subgroup of the displacement 
group. 

• Approach based on differential geometry: As 
a full extension and completion of the Lie-group- 
algebraic method, the differential geometry method 


can be used to cover the motion patterns that cannot 
be modeled by subgroups but regular submanifolds, 
a plane-hinged parallelogram, for example [18.15]. 
The additional merit of the geometric method lies 
in that it enables to link the finite motion with 
the instantaneous motion of the platform thanks to 
the integral/differential relationship between the Lie 
group and Lie algebra. 

• Approaches based on screw theory: In this ap¬ 
proach, the first step is to determine the wrench 
system S that is reciprocal to the theoretically acces¬ 
sible instantaneous twist (or velocity twist) of the 
moving platform. Then, the wrenches of each limb 
whose union spans the system S (and that determine 
all the possible structures of the kinematic chains 
that will generate the corresponding wrenches) are 
enumerated. Although this method is very effective 
at providing simple and intuitive descriptions of the 
instantaneous motions of the platform, it is neces¬ 
sary to verify that the mobility of the platform is full 
cycle and not only instantaneous as all considered 
twists and wrenches are instantaneous. Thus, tech¬ 
niques to ensure the consistency between the finite 
and instantaneous motions have also been devel¬ 
oped. A systematic implementation of this approach 
is provided in [18.16]. 

The above synthesis methods have been used to 
generate a large number of architectures that cannot 
be presented in this book, but the website [18.17] 
presents a comprehensive description of a large num¬ 
ber of mechanical architectures. Among others, several 
remarkable architectures were proposed in the recent 
years to generate Schonflies motions using the innova¬ 
tive concept of articulated plates, leading to a brand new 
pick-and-place parallel robot known as Adept Quat- 
tro [18.18]. Also, other similar architectures allowing 
unlimited rotations of the end effector while avoiding 
all singularities have recently been proposed (see for 
instance [18.19]). 

It should also be pointed out that although these 
synthesis methods can be used to generate a large num¬ 
ber of architectures [18.20], there is a lack of a metric 
to evaluate which one is better than the other even if 
they have completely identical motion patterns. Fur¬ 
thermore, it is hard to link the models for type synthesis 
with those for kinematic, static, and dynamic analyses 
and design. Open issues, therefore, include: 

• To develop suitable performance metrics for each 
motion type to identify suitable topological struc¬ 
tures for specific applications. 
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• To develop a complete theoretical package that 
enables type synthesis and parametric design to 
be integrated into a unified framework that takes 

18.3 Kinematics 

18.3.1 Inverse Kinematics 

The solution of the inverse kinematic problem is usually 
straightforward for parallel robots. This can be illus¬ 
trated with the Gough platform. 

The solution of the inverse kinematic problem con¬ 
sists in determining the vector of leg lengths q for 
a given pose of the platform defined by the position vec¬ 
tor/; of a given point on the platform in a fixed reference 
frame and a rotation matrix R representing the orien¬ 
tation of the platform with respect to the fixed frame. 
Let a, denote the position vector of the anchor point 
of the i-th leg on the base given in the fixed reference 
frame and b , denote the position vector of the anchor 
point of the i-th leg on the platform, given in a moving 
frame attached to the platform. The length of the i-th 
leg is the norm of the vector connecting the two anchor 
points. This vector, noted s,, can be written as 

s, = p + Rbf — cij , i=l.6. (18.1) 

Given the pose of the platform (vector p and matrix R), 
vectors, is readily calculated using (18.1) and, hence, 
the leg lengths can be obtained. 

18.3.2 Forward Kinematics 

The solution of the forward kinematic problem consists 
in determining the pose of the platform for a given set 
of actuated joint coordinates (a given vector q). This 
solution is needed for control purposes, calibration, and 
motion planning. 

The forward kinematic problem of a parallel robot is 
usually much more complex than the inverse kinematic 
problem. Indeed, the loop closure equations (18.1) are 
typically highly nonlinear expressions of the pose vari¬ 
ables. They form a nonlinear set of equations that 
generally admits multiple solutions (e.g., the Gough 
platform can have up to 40 solutions [18.21-23], while 
a table with the number of solutions for Gough plat¬ 
forms with special geometries is provided in [18.24]). 
The forward kinematic problem arises in two different 
contexts. In the first case, no estimate of the current 
pose of the platform is available (e.g., when starting 
the robot), while in the second case, a relatively precise 
estimate of the pose is known (e.g., in real-time con- 


into account the unavoidable uncertainties in the 
manufacturing process, an ultimate objective of 
design. 


trol when the forward kinematics has been solved at the 
previous sampling time of the controller). In the first 
case, the only known approach is to determine all the 
solutions of the inverse kinematic equations, although 
there is no known algorithm to sort the set of solu¬ 
tions. It is often possible to determine an upper bound 
on the number of real solutions. Consider, for exam¬ 
ple, the case of the planar 3 — RPR robot (Fig. 18.3). 
If the joint at point S 3 is disassembled, two separate 
mechanisms are obtained, namely, a 4-bar linkage and 
a rotary lever. From the kinematics of 4-bar linkages, 
it is known that point S 3 moves on a sixth-order alge¬ 
braic curve. Moreover, it is also known that point S 3 
of the lever moves on a circle, that is, a second-order 
algebraic curve. For a given set of actuated joint coordi¬ 
nates, solutions to the forward kinematic problem arise 
when the two curves intersect, that is, when the mech¬ 
anism can be assembled. Bezout’s theorem states that 
algebraic curves of order m, n intersect in nm points, 
counting their order of multiplicity. In the above case, 
this means that the curves intersect in 12 points. How¬ 
ever, these 12 points include the two circular imaginary 
points that belong to the coupler curve of the 4-bar 
linkage and to any circle, thereby to their intersection. 
These points are counted three times in Bezout’s the¬ 
orem and, hence, the forward kinematic problem will 
have at most six real solutions, corresponding to the in¬ 
tersection points. 

Various methods have been proposed for the 
solution of the forward kinematic problem: elim¬ 
ination [18.25], continuation [18.23], Grobner 
bases [18.26], and interval analysis [18.27]. Elim¬ 
ination is usually not very stable numerically (i. e., 
it can produce spurious roots and miss solutions) 
unless special care is taken in the implementation of 
the resulting univariate equation and the elimination 
steps. For instance, the transformation of polynomial 
solutions into eigenvalue problems can be used [18.28]. 
Polynomial continuation, on the other hand, is much 
more stable numerically since mature algorithms can 
be found in the literature [18.29]. The fastest methods - 
although not appropriate for a real-time use - are 
Grobner bases and interval analysis. They also have the 
advantage of being numerically certified (no roots can 
be missed and the solution can be computed with an 
arbitrarily prescribed accuracy). 
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However, in the simplest cases, elimination can usu¬ 
ally be used to produce stable implementations as, for 
example, for the 3 —RPR robot [18.30,31]. Finally, it is 
pointed out that the forward kinematic problem solved 
for one type of robot may often be used for another ar¬ 
chitecture: for example, the one developed above for the 
3 — RPR robot may be used for the 3 — RRR. This is typ¬ 
ical for parallel robots. 

When a priori information (an initial guess of the 
solution) is available, the forward kinematics is usually 
solved using the Newton-Raphson or the Newton- 
Gauss iterative scheme. Consider the solution to the 
inverse kinematic problem, written as 

q=f(x). (18.2) 

Iteration k of the Newton-Raphson procedure is written 
as 

x k +i=x k +A(q-f(x k )), (18.3) 

where q is the vector of prescribed joint variables. Ma¬ 
trix A is usually chosen as (df / dq)~ l (x k ) (the inverse 
matrix may not be computed at each iteration or even be 
chosen as constant). The iterative scheme stops when 
the magnitude of the difference vector (q—f(x k )) is 
smaller than a chosen threshold. 

Provided that a good initial estimate of the solution 
is available, the Newton-Raphson algorithm is usually 
very fast. However, the procedure may not converge or, 
even worse, it may converge to a solution that is not the 
correct pose of the platform, that is, it may converge 
to another assembly mode. Such a situation can occur 
even if the initial guess is arbitrarily close to the correct 
pose. If the result is used in a control loop, the conse¬ 
quences can be catastrophic. Fortunately, mathematical 
tools, such as the Kantorovitch theorem combined with 
interval analysis can be used to determine if the solution 


18.4 Velocity and Accuracy Analysis 

Similarly to serial robots, the actuated joint velocity 
vector of parallel robots q is related linearly to the 
vector of translational and angular velocities of the 
platform (for simplicity, the latter vector is denoted 
here as p although the angular velocity is not the time 
derivative of any set of angular parameters). The linear 
mapping between the two vectors is provided by the Ja¬ 
cobian matrix J 

p = i(p)q. (18.4) 

However, for parallel robots, a closed-form expression 
is usually available for the inverse Jacobian J^ 1 but this 


found by the Newton-Raphson scheme is the correct 
pose of the robot, that is, to certify the result at the cost 
of a longer computation time, yet still compatible with 
real time [18.27]. 

Furthermore, the choice of the inverse kinematics 
equations plays an important role in the convergence 
of this method [18.27]. For example, for a Gough 
platform, a minimal set of equations (six equations in¬ 
volving six unknowns: three for the translation and 
three orientation angles) can be used but other rep¬ 
resentations are possible. For instance, the position 
coordinates of three of the platform anchor points - in 
the fixed reference frame - can be used as unknowns 
(the coordinates of the remaining three points are then 
readily computed upon convergence). Using this rep¬ 
resentation, nine equations are needed. Six equations 
are obtained based on the known distances between 
the anchor points on the base and platform and three 
additional equations are obtained based on the known 
distances between all combinations of the three selected 
platform anchor points. 

Another possible approach for the solution of 
the forward kinematic problem is either to add sen¬ 
sors on the passive joints (e.g., on the U joints 
of a Gough platform) or to add passive legs with 
sensed joints. The main issue is then to determine 
the number and location of the additional sensors that 
lead to a unique solution [18.32-34] and to deter¬ 
mine the effect of the sensor errors on the position¬ 
ing error of the platform. For example, Stoughton 
mentions that for a Gough platform with sensors in 
the U joints, it is still necessary to use the Newton- 
Raphson scheme to improve the accuracy of the so¬ 
lution because the solutions obtained with the ad¬ 
ditional sensors are very sensitive to measurement 
noise [18.35]. 


is much more difficult for J (more precisely, the closed- 
form for most 6-DOF robots will be so complex that 
it cannot be used in practice). For example, a simple 
static analysis can be used to show that the i-th row Jr 1 
of matrix J~ 1 for a Gough platform can be written as 

= Ci x rii) , (18.5) 

where m represents the unit vector defined along leg i 
and Ci is the vector connecting the origin of the mobile 
frame attached to the platform to the i-th anchor point 
on the platform. 
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The effect of the joint sensor errors A q on the posi¬ 
tioning error A/7 follows the same relationship, namely 

Ap = 3{p)Aq. (18.6) 

Since matrix J is difficult to be obtained in a closed 
form, accuracy analysis (i. e., finding the maximal po¬ 
sitioning errors over a given workspace for bounded 
joint sensor errors) is much more difficult than for serial 
robots [18.36, 37]. Apart from the measurement errors, 
there are other sources of inaccuracy in parallel robots, 
namely, clearance in the passive joints, manufacturing 
tolerances, thermal errors, gravity induced and dynamic 
errors [18.38, 39]. The effect of joint clearances on tra¬ 
jectories followed by serial and parallel robots was 
studied in [18.40-42]. According to these works, it is 
impossible to determine trends for the effect of the ge¬ 
ometric errors: a case-by-case study must be performed 
since the effect is highly dependent on the mechanical 
architecture, dimensioning, and workspace of the robot. 
Thermal effects are sometimes mentioned as possible 
sources of inaccuracy, although few works substantiate 
this claim [18.43] and cooling may slightly reduce their 
effects [18.44,45]. 


18.5 Singularity Analysis 

The analysis of singularities of parallel mechanisms 
was first addressed in [18.54]. In this formulation, the 
kinematic equations were reduced to the input-output 
relationship between the actuated joint coordinate vec¬ 
tor q and the platform Cartesian coordinate vector p, 
namely, 

f(q,p) = 0. (18.7) 

Differentiating (18.7) with respect to time leads to 

B<jr + A/) = 0 . (18.8) 

Three types of singularities can then be defined: i) 
when matrix B is singular (termed serial singularity), 
ii) when matrix A is singular (termed parallel singular¬ 
ity) and iii) when the input-output equations degenerate 
(termed architecture singularity) in which case both ma¬ 
trices A and B can be singular. In a serial singular 
configuration, the joints can have a nonzero velocity, 
while the platform is at rest. In a parallel singularity, 
there exist nonzero platform velocities for which the 
joint velocities are zero. In the neighborhood of such 
a configuration, the robot can undergo an infinitesimal 
motion, while the actuators are locked. Since the mobil¬ 
ity of the end effector should be zero when the actuators 


Calibration is another means of improving the ac¬ 
curacy of parallel robots. This issue was addressed in 
Chap. 3. The methods and procedures used for parallel 
robots differ slightly from the ones used for serial robots 
since only the inverse kinematic equations are available 
and the positioning of the platform is much less sensi¬ 
tive to the geometric errors than in serial robots [18.46, 
47], Hence, the measurement noise occurring during 
calibration has a significant impact and may lead to sur¬ 
prising results. For example, the classical least-squares 
method may lead to parameters that are such that some 
constraint equations are not satisfied even when the 
measurement noise is taken into account [18.48] or 
even to parameter values that are unrealistic. It has also 
been shown with experimental data that classical paral¬ 
lel robot modeling leads to constraint equations that do 
not have any solution irrespective of the measurement 
noise [18.49]. Moreover, calibration is very sensitive to 
the choice of the calibration poses [18.50]: it seems that 
the optimal choice is to select poses on the workspace 
boundary [18.51,52]. We may also mention an origi¬ 
nal calibration approach based not on the proprioceptive 
sensors of the robot but on leg observation with a vision 
system [18.53]. 


are locked, it is also said that in such a configuration, the 
robot gains some DOFs. As a consequence, certain de¬ 
grees of freedom of the platform cannot be controlled, 
and this is a major problem. 

A more general study of singularities was pro¬ 
posed by Zlatanov et al. [18.55]. The latter analysis 
uses the velocity equations involving the full twist of 
the end effector and all joint velocities (passive or 
actuated). This approach led to a more detailed clas¬ 
sification of singularities and was also used later to 
reveal special singularities (referred to as constraint sin¬ 
gularities) [18.56] that could not be found with the 
analysis presented in [18.54]. Also, the singularity anal¬ 
ysis of parallel mechanisms was revisited more recently 
in [18.57], where hierarchical levels in which different 
critical phenomena originate are recognized. 

The singularity analyses discussed above are of the 
first order. Secondhand higher-)order singularity anal¬ 
yses can also be performed, although these analyses are 
much more complex [18.42, 58]. 

18.5.1 Parallel Singularity Analysis 

This type of singularity is especially important for par¬ 
allel robots because it corresponds to configurations in 
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which the control of the robot is lost. Furthermore, very 
large joint forces can occur in the vicinity of singular 
poses that may lead to a breakdown of the robot. The 
main issues to be addressed in this context are: 

1. The characterization of the singularities. 

2. The definition of performance indices representing 

the closeness to a singularity. 

3. The development of algorithms that are capable of 

detecting the presence of singularities over a given 

workspace or trajectory. 

Parallel singularities arise when the 6 x 6 full inverse 
Jacobian Jf (i. e., the matrix that maps the full twist 
of the platform into active - and eventually passive - 
joint velocities) is singular, that is, when its determi¬ 
nant det(Jf) is equal to 0. It is pointed out that passive 
joint velocities sometimes have to be included because 
restricting the analysis to the active joint velocities may 
not allow the determination of all singular configura¬ 
tions of the mechanism (see the example of the 3— UPU 
robot [18.59]). Usually, a proper velocity analysis al¬ 
lows one to establish this matrix in a closed form. But 
computing the determinant of this matrix may be dif¬ 
ficult even with symbolic computation tools (see the 
example of the Gough platform [18.60,61]) and it does 
not provide an insight into the geometric conditions as¬ 
sociated with the singularities. 

An alternative approach is to use line geometry: in¬ 
deed, for several parallel robots (although not all), a row 
of Jf corresponds to the Pliicker vector of some line de¬ 
fined on the links of the robot. For example, for a Gough 
platform, the rows of Jf are the normalized Pliicker vec¬ 
tors of the lines associated with the legs of the robot. 
A singularity of matrix Jf, therefore, implies a linear 
dependency between these vectors (they then constitute 
a linear complex), a situation that may occur only if the 
lines associated with the vectors verify particular geo¬ 
metric constraints [18.62]. These geometric constraints 
have been identified by Grassmann for every set of 3, 

4. 5, or 6 vectors. Singularity analysis is, thus, reduced 
to determining conditions on the pose parameters for 
which these constraints are satisfied, giving geometric 
information on the singularity variety. A variant of this 
approach is to use the Grassmann Cayley algebra that 
in certain cases may lead directly to the geometric sin¬ 
gularity conditions [18.63]. 

Measuring closeness between a pose and a sin¬ 
gular configuration is a difficult problem: there exists 
no mathematical metric defining the distance between 
a prescribed pose and a given singular pose. Hence, 


a certain level of arbitrariness must be accepted in the 
definition of the distance to a singularity and none of 
the proposed indices is perfect. For example, a possi¬ 
ble index is the determinant of Jf: unfortunately, when 
the platform is subjected to both translational and rota¬ 
tional motion, the Jacobian matrix is not dimensionally 
homogeneous and, hence, the value of the determinant 
changes according to the physical units used to describe 
the geometry of the robot. Dexterity indices, as defined 
in Chap. 10 (although they are much less relevant for 
parallel robots than for serial robots [18.37]), can also 
be used, and special indices for parallel robots have also 
been proposed [18.64-66], For example, it may be in¬ 
teresting for a Gough platform to define a workspace as 
singularity-free if for any pose the absolute value of the 
joint forces is lower than a given threshold [18.67] or 
to use a physically meaningful index like the kinematic 
sensitivity [18.68]. 

Most of the analyses based on the above indices 
are local, that is, valid only for a given pose, while in 
practice the problem is to determine if a singular config¬ 
uration can occur over a given workspace or trajectory. 
Fortunately, an algorithm exists that allows this veri¬ 
fication, even if the geometric model of the robot is 
uncertain [18.69]. However, it should be pointed out 
that a singularity-free workspace may not always be 
optimal for parallel robots. Indeed, other performance 
requirements may impose the presence of singularities 
in the workspace or the robot may have part of its 
workspace singularity-free (for example its working re¬ 
gion) while exhibiting singularities outside this region. 
Therefore, a motion planner producing a trajectory that 
avoids singularities while remaining close to a desired 
path is advisable and various approaches have been pro¬ 
posed to address this problem [18.70,71]. A related 
problem is to determine if two solutions of the forward 
kinematics may be joined without crossing a singular¬ 
ity: this has been proved for planar robots [18.72,73] 
but also for 6-DOF robots [18.74]. 

Finally, it is noted that being close to a singular 
configuration may be useful in some cases. For exam¬ 
ple, large amplification factors between the end-effector 
motion and the actuated joint motion may be inter¬ 
esting for fine positioning devices with a very small 
workspace or for improving the sensitivity along some 
measurement directions for a parallel robot used as 
a force sensor [18.75]. It should also be mentioned that 
parallel robots that remain permanently in a singular 
configuration may be interesting since they are capable 
of producing complex motions with only one actua¬ 
tor [18.76-78]. 
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18.6 Workspace Analysis 

As mentioned earlier, one of the main drawbacks of par¬ 
allel robots is their small workspace. Moreover, their 
workspace analysis is usually more complex than for 
serial robots, especially if the platform has more than 
3-DOFs. Indeed, the platform motion capabilities are 
usually coupled, thereby prohibiting simple graphical 
representations of the workspace. Decoupled robots 
have also been proposed [18.79-81] (l-g&M'll'ittUJl 
and but they do not provide the load¬ 

carrying capacity of conventional parallel robots. In 
general, the workspace of parallel robots is restricted 
by the following: 

• The limitations on the actuated joint variables: for 
example, the lengths of the legs of a Gough platform 
have a minimum and maximum value. 

• The limitations on the range of motion of the pas¬ 
sive joints: for example, the LJ and S joints of 
a Gough platform have restricted ranges of angular 
motion. 

• The mechanical interferences between the bodies of 
the mechanism (legs, base, and platform). 

Different types of workspaces can be defined such 
as, for example, the dextrous workspace (the set of plat¬ 
form locations in which all orientations are possible), 
the maximal workspace (the set of platform locations 
that may be reached with at least one orientation), or 
the orientation workspace (the set of orientations that 
can be reached for a given location of the platform). 

A possible approach for representing the workspace 
of parallel robots with n > 3 DOFs is to fix the value 
of n — 3 pose parameters in order to be able to plot the 
workspace over the remaining 3-DOFs. Such plots can 
be obtained very efficiently using geometric methods 
if the plotted variables involve only translational mo¬ 
tions because a geometric approach usually allows one 
to establish directly the nature of the boundary of the 
workspace (e.g., [18.82,83] for the calculation of the 
workspace of a Gough platform when its orientation is 
fixed). Another advantage of this approach is that it al¬ 
lows the computation of the surface and volume of the 
workspace. Possible alternatives include: 

• Discretization methods in which all poses of an n- 
dimensional grid are checked with respect to all 
types of kinematic constraints. Such methods are 
usually simple to implement but are computation¬ 
ally intensive because the computation time in¬ 
creases exponentially with the resolution of the 
mesh. Also, they require a large storage space. 

• Numerical methods that allow the determination of 
the workspace boundary [18.84, 85]. 


• Numerical methods based on interval analysis that 

allow the determination of an approximation of 

the workspace volume up to an arbitrary accu¬ 
racy [18.86,87], This representation is also appro¬ 
priate for motion planning problems. 

Singularities can also split the workspace calculated 
from the above kinematic constraints into elementary 
components, called the aspects by Wenger and Chab- 
lat [18.88] and which are separated by a singularity 
variety. Some very good results have been obtained for 
planar robots [18.89] but determining the aspects for 
spatial robots is still an open problem. A parallel robot 
may not always be able to move from one aspect to an¬ 
other (at least without considering the dynamics of the 
robot [18.90,91]) and hence the useful workspace can 
be reduced. 

A problem related to workspace analysis is the 
motion planning problem, which is slightly different 
from that encountered with serial robots. For parallel 
robots, the problem is not to avoid obstacles within 
the workspace but either to determine if a given tra¬ 
jectory lies entirely inside the workspace of the robot 
and is singularity-free (for which an algorithm is avail¬ 
able [18.92]) or to determine such a trajectory between 
two poses which is much more difficult [18.93]. In¬ 
deed, classical serial robot motion planners work in the 
joint space and assume that there is a one-to-one re¬ 
lationship between the joint space and the operational 
space. With this assumption, it is possible to determine 
a set of points in the joint space that is collision-free 
and to use this knowledge to build a collision-free 
path between two poses. However, this assumption is 
not valid for parallel robots because the mapping be¬ 
tween the joint space and the operational space is not 
one to one: a point in the joint space may either cor¬ 
respond to multiple points in the operational space 
or have no correspondence because the closure equa¬ 
tions of the mechanism are not satisfied. For parallel 
robots, the most efficient motion planner seems to be an 
adaptation of a probabilistic motion planner that takes 
into account - to a certain extent - the closure equa¬ 
tions [18.94, 95]. 

Another motion planning problem is related to tasks 
where a limited number of DOFs of the robot are used, 
while the others may be used to increase the machine’s 
workspace, to avoid singularities or to optimize some 
performance index of the robot [18.96,97]. Along the 
same line, it is possible to define the part positioning 
problem [18.98] as determining the pose of a rigid body 
with respect to the workspace of the robot so that this 
pose satisfies some constraints. 
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18.7 Static Analysis 

Similarly to serial robots, the static analysis of paral¬ 
lel robots can be readily performed using the Jacobian 
matrix. Indeed, the mapping between the vector of ac¬ 
tuated joint forces/torques r and the external wrench / 
exerted by the platform can be written as 

r = J T / , ( 18 . 9 ) 

where J T is the transpose of the Jacobian matrix of the 
robot. Equation (IS.9) can be used for various purposes, 
namely: 

• During the design process in order to determine 
the maximal actuator forces/torques over the robot 
workspace (for actuator selection). This is a com¬ 
plex issue since J T is not known in a closed 
form. 

• In applications where the robot is used as a force 
sensor: If r is measured and the pose of the plat¬ 
form is known, then ( 18 .9) can be used to calculate/ 
so that the robot can be used both as a motion and 
sensing platform [18.99-101]. 

The stiffness matrix K of a parallel robot is defined 
similarly to that of serial robots as 

K = J _T ^J _1 , ( 18 . 10 ) 

where K,- is the diagonal matrix of actuated joint stiff¬ 
ness. Duffy [18.102] notes that this derivation is, in 
general, incomplete. For example, for a Gough platform 
this derivation assumes that there is no initial load on 
the elastic element of the link. Assuming that the length 
of the unloaded link is q ? leads to 

i=6 

A/ = kAqm + kj(qi - q?) An,-, 

1=1 

1=6 

A m = ^2 kA qiCi x n i + k i(qi - £?) A(c,- x n i) . 

1=1 

where k t denotes the axial stiffness of the leg, rep¬ 
resents the unit vector defined along the i-th leg, c,- is 
the vector connecting the origin of the moving refer¬ 
ence frame to the i-th anchor point of the platform, 
and/, m are the external force and moment vectors that 
are applied by the platform. Consequently, the stiffness 
matrix as defined in (18.10) is valid only if q ,• = q 9, and 
is coined the passive stiffness. 

Moreover, it was pointed out in the litera¬ 
ture [18.103] that the formulation of (18.10) is valid 


only when the external wrench is zero. Indeed, the 
above formulation does not account for the fact that the 
Jacobian matrix is configuration dependent and there¬ 
fore changes with the applied loads. The formulation 
proposed in [18.103] and termed consecutive congru¬ 
ence transformation (CCT) takes these changes into 
account and leads to a stiffness matrix defined as 

K c = J- t K,J-' + - (18.11) 

where p is the vector of Cartesian coordinates and r 
is the vector of actuated joint forces/torques. When¬ 
ever possible, the formulation of (18. 1 1) should be used 
instead of that of (18.10) because it is mechanically 
consistent. 

Another interesting static problem is the static bal¬ 
ancing of parallel robots. The static balancing of mech¬ 
anisms, in general, has been an important research topic 
for several decades (e.g., [18. 104] for an account of the 
state of the art and many recent new results). Parallel 
mechanisms are said to be statically balanced when the 
weight of the links does not produce any torque (or 
force) at the actuators under static conditions, for any 
configuration of the manipulator or mechanism. This 
condition is also referred to as gravity compensation. 
The gravity compensation of parallel mechanisms was 
addressed in [18.105] where the use of counterweights 
was suggested to balance a 2-DOF parallel robot used 
for antenna aiming, and in [18.106] for planar robots. 
This analysis leads to simple and elegant balancing 
conditions. 

In general, static balancing can be achieved us¬ 
ing counterweights and/or springs I<b>M'1I > H i JUI. When 
springs are used, static balancing can be defined as the 
set of conditions for which the total potential energy in 
the mechanism - including gravitational energy and the 
elastic energy stored in the springs - is constant for any 
configuration of the mechanism. When no springs - or 
other means of storing elastic energy - are used, then 
static balancing conditions imply that the center of mass 
of the mechanism does not move in the direction of the 
gravity vector, for any motion of the mechanism. 

Sufficient conditions under which a given mech¬ 
anism is statically balanced may be obtained. The 
problem of statically balancing spatial parallel robots is 
usually complex [18.107, 108]). Among other results, 
it was pointed out that it is not possible to balance 
a 6 -UPS robot in all poses by using only counter¬ 
weights [18.109] ([/ stands for a Hooke joint, P stands 
for a prismatic joint and S stands for a spherical joint). 
Alternative mechanisms (using parallelograms) that can 
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be balanced using only springs were suggested [18.107, 

110 , 111 ], 

A natural extension of static balancing is dynamic 
balancing. The dynamic balancing problem consists 
in finding parallel mechanisms that do not produce 
any reaction force or torque on their base for ar- 

18.8 Dynamic Analysis 

The dynamic model of parallel robots has a form similar 
to that of serial robots (Chap. 3), namely, 

M(x)p + C(p,q,p,q) + G(p,q) = r , ( 18 . 12 ) 

where M is the positive-definite generalized inertia ma¬ 
trix, G is the gravitational term, and C is the centrifugal 
and Coriolis term. However, this equation is more dif¬ 
ficult to obtain for parallel robots because the closure 
equations have to be satisfied. A classical method for 
calculating the dynamic models of closed-chain mech¬ 
anisms is to consider first an equivalent tree structure, 
and then to enforce the kinematic constraints using La¬ 
grange multipliers or d’Alembert’s principle [18.113, 
114]. Other approaches include the use of the principle 
of virtual work [18.115-118], the Lagrange formal¬ 
ism [18.119-122], Hamilton’s principle [18.123], and 
Newton-Euler equations [18.124-129]. 

The main drawback of dynamic models obtained for 
parallel robots is that they are usually rather complex, 
they require the determination of dynamic parameters 
that are often not well known (dynamic identification 
of closed-loop mechanisms being not trivial [18.130]), 
and they usually involve solving the forward kinemat¬ 
ics. Therefore, their calculation is computer intensive 
while they must be used in real time. 

18.9 Design 

The design of a robot can be decomposed into two main 
phases: 

• Structural synthesis: Finding the general mechani¬ 
cal structure of the robot. 

• Geometric synthesis: Determining the value of the 
geometric parameters involved in a given structure 
(here geometric parameters has a loose sense: for 
example, mass, inertia may also be involved). 

The problem of structural synthesis (type synthesis) 
was addressed in Sect. 18.2. However, performance re¬ 
quirements other than the motion pattern of the robot 
have to be taken into account in the design of a robot. 


bitrary trajectories. This problem was addressed for 
planar and spatial parallel mechanisms. In [18.112], it 
was shown that dynamically balanced parallel mecha¬ 
nisms with up to 6-DOFs can be synthesized, although 
the resulting mechanical architectures may be complex 
(ksi>MM£EEB). 


Using dynamic models for control purposes was 
proposed [18.131,132], usually in the context of an 
adaptive control scheme, in which the tracking errors 
are used on-line to correct the parameters used in the 
dynamic equations [18.133,134]. Control laws were 
proposed mainly for planar robots and for the Delta 
robot [18.135,136], although some implementations 
have been proposed for general 6-DOF robots [18.137, 
138] or vibration platforms [18.139]. However for 6- 
DOF robots, the benefits of using dynamic models for 
high-speed motions are difficult to establish because the 
computational burden of the dynamic model somewhat 
reduces the potential gains. An original approach is to 
work directly in the joint space, using information ob¬ 
tained on the legs’ orientation through a vision system, 
allowing to get rid of the forward kinematics [18.140]. 

Dynamic control is an important and open issue 
since parallel robots can operate at velocities and ac¬ 
celerations much larger than those of serial robots. For 
example, some of the Delta robots have reached accel¬ 
erations of the order of 500 m/s 2 (with an objective of 
reaching 100G [18.141]) while wire-driven robots can 
probably exceed this value. 

Finally, the optimization of the dynamic properties 
of parallel mechanisms can be addressed by minimizing 
the variation of the inertia over the workspace [18.142]. 


Serial robots have the advantage that the number of pos¬ 
sible mechanical architectures is relatively small and 
that some of these architectures have clear advantages 
in terms of performance compared to others (for exam¬ 
ple the workspace of a 3 R structure is much larger than 
the workspace of a Cartesian robot of similar size). 

Unfortunately, no such rules exist for parallel 
robots, for which there are furthermore a large number 
of possible mechanical designs. Additionally, the per¬ 
formances of parallel robots are very sensitive to their 
geometric parameters. For example, the extremal stiff¬ 
ness of a Gough platform over a given workspace can 
change by 700% for a change of only 10% of the plat¬ 
form radius. Consequently, structural synthesis cannot 
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be dissociated from the geometric synthesis. In fact, it is 
conjectured that a well-dimensioned robot of any struc¬ 
tural type will, in general, perform better than a poorly 
designed robot with a structure that may seem more ap¬ 
propriate for the task at hand. 

Usually, the design process is treated as an opti¬ 
mization problem. To each specified performance re¬ 
quirement is associated a performance index whose 
value increases with the level of violation of the per¬ 
formance requirement. These performance indices are 
summed in a weighted real-value function, called the 
cost function, which is a function of the geometric 
design parameters, and then a numerical optimization 
procedure is used to find the parameters that minimize 
the cost function (hence, this approach leads to what 
is called an optimal design) [18.143-145]. There are, 
however, numerous drawbacks to this approach: the 
determination and effect of the weights in the cost func¬ 
tion are difficult to ascertain, imperative requirements 
are difficult to incorporate and complexify the opti¬ 
mization process, and the definition of the performance 
indices is not trivial, only to name a few. The main is¬ 
sues can be stated as follows: 

• It is quite difficult to ensure that the global minima 
is reached in spite of using relatively new opti¬ 
mization methods such as neural network or genetic 
algorithm. 

• The robustness of the design solution obtained with 
the cost function approach with respect to the uncer¬ 
tainties in the final design. Indeed, the real instanti- 

18.10 Wire-Driven Parallel Robots 

Parallel mechanisms naturally lead themselves to an¬ 
other paradigm that exploits the structural properties 
of wires (cables). Cables are mechanical components 
that can withstand large tensile loads relative to their 
weight. When applied to parallel mechanisms, a plural¬ 
ity of cables can be attached to a moving platform and 
driven by actuated reels mounted on the base frame. 
Because of the parallel architecture, it is possible to 
ensure that cables can be maintained under tension 
within a given workspace. Wire-driven parallel mech¬ 
anisms have been studied by several researchers over 
the last 20 years (e.g., [18.8, 147] for early work). The 
advantages of these mechanisms are numerous. When 
wound on a spool, cables allow motion ranges that are 
much larger than that produced by conventional artic¬ 
ulated systems. Also, because they can only resist to 
tensile forces, cables are, as mentioned above, much 
thinner and lighter than most conventional mechani¬ 
cal components, for similar payloads. Thus, they have 


ation of a theoretical solution always differs from 
the theoretical solution itself because of the manu¬ 
facturing tolerances and other uncertainties that are 
inherent to a mechanical system. 

• Performance requirements may be antagonistic 
(e.g., workspace and accuracy), and the approach 
only provides a compromise between these require¬ 
ments that is difficult to master through the weights. 

An alternative to optimal design is referred to as 
appropriate design, in which no optimization is con¬ 
sidered, but the objective is to ensure that desired 
requirements are satisfied. This approach is based on 
the definition of the parameter space in which each 
dimension is associated with a design parameter. Per¬ 
formance requirements are considered in turn and the 
regions of the parameter space which correspond to 
robots satisfying the requirements are calculated. The 
design solution is then obtained as the intersection of 
the regions obtained for each individual requirement. 

In practice, only an approximation of the regions is 
necessary since values close to the boundary cannot be 
physically realized due to the manufacturing tolerances. 
For that calculation, interval analysis was successfully 
used in various applications [18.36, 146]. 

The appropriate design approach is clearly more 
complex to implement than the cost function approach 
but has the advantage of providing all design solutions, 
with the consequence that manufacturing tolerances 
may be taken into account to ensure that the real robot 
also satisfies the desired requirements. 


very low inertia and are particularly suitable for systems 
in which large ranges of motion are required. Indeed, 
long-reach articulated robots are heavy and involve the 
motion of significant mass, whereas cable-driven robots 
remain light even when the range of motion is very 
large. This property is clearly exemplified by aerial 
video camera systems such as the SkyCam [18.148] 
or in which shows an application in 

the scanning of artifacts for the generation of digi¬ 
tal three-dimensional (3-D) models. and 

l<t3»WiTDT<m show another application in material han¬ 
dling. 

Because wires can only resist tensile loads, wire- 
driven parallel mechanisms are generally designed 
based on either one of the two following approaches, 
namely: (1) the weight (gravity) of the moving plat¬ 
form is used to ensure that all cables are under tension 
or (2) the number of cables is larger than the number 
of degrees of freedom of the platform, thereby allowing 
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a pre-stressing of the cables that can be used to ensure 
that all cables remain under tension. The wire-driven 
parallel mechanisms that are designed based on the first 
approach are referred to as cable-suspencled parallel 
mechanisms, while the mechanisms that are based on 
the second approach are referred to as fully constrained 
wire-driven parallel mechanisms. 

18.10.1 Cable-Suspended Parallel 
Mechanisms 

Most cable-suspended parallel mechanisms are not re¬ 
dundantly actuated: they typically include as many 
actuators as DOFs. These mechanisms have been pro¬ 
posed in the literature as potential candidates for appli¬ 
cations that require very large workspaces or mecha¬ 
nisms that can provide effective payload to mass ratios. 
One of the first cable-suspended mechanisms that was 
built is the Robocrane [18.8], developed by NIST al¬ 
most two decades ago. This 6-DOF robot was intended 
for crane-type operations in which the pose of the pay- 
load can be fully controlled. Other cable-suspended 
mechanisms have also been studied and prototypes 
were built to validate their performance (e.g., [18.149] 
and [18.150]). More recently, cable-suspended parallel 
mechanisms including more actuators than DOFs were 
proposed, and it was shown that their workspace-to- 
footprint ratio can be much larger than that of nonredun- 
dant cable-suspended parallel mechanisms [18.151], 
This is convincingly illustrated in i' w '™ni t | IH and 
|43>M2miiE*, where a cable-suspended parallel mech¬ 
anism with eight cables is demonstrated. 

Most of the cable-suspended parallel mechanisms 
proposed or built in the past were assumed to work 
under static or quasi-static conditions. Under this as¬ 
sumption, the workspace can be determined based on 
the static equilibrium of the moving platform. The 
workspace is defined as the set of platform poses for 
which static equilibrium can be obtained while main¬ 
taining tension in all cables. Techniques to determine 
the static workspace of cable-suspended robots were 
proposed in the literature, for instance, in [18.152], 
The dynamic trajectory planning and control of cable- 
suspended parallel mechanisms were also investigated 
in the recent literature (e.g., [18.153-156]), leading to 
cable-suspended parallel mechanisms that can be op¬ 
erated beyond their static workspace. The concept of 
dynamic workspace then arises [18.157], which can be 
defined as the set of poses that the platform can reach 
with at least one kinematic state (position, velocity, and 
acceleration). 

Finally, it should be mentioned that the determi¬ 
nation of the static pose of an underactuated cable- 
suspended parallel mechanism - that is, one with fewer 


cables than DOFs - is a very complex problem that has 
not yet been completely solved [18.158, 159]. In fact, 
for a given platform and given cable lengths, it is pos¬ 
sible that an equilibrium pose can be reached, although 
some of the cables are not under tension. Therefore, the 
determination of the static pose involves not only solv¬ 
ing for the actual number of cables, n, but also for all 
subrobots with n— 1 cables. 

18.10.2 Fully Constrained Wire-Driven 
Parallel Mechanisms 

Fully constrained wire-driven parallel mechanisms 
have also attracted a great deal of attention. Fully con¬ 
strained mechanisms include a number of cables larger 
than the number of degrees of freedom of the mech¬ 
anism. If a proper geometry is used, they generally 
exhibit a wrench-closed workspace within which the 
platform can resist arbitrary external wrenches, while 
maintaining the cables in tension. 

The statics of wire-driven robots can be described 
by 

Wr = / , (18.13) 

where r is the vector of tensions in the wires, / is 
the wrench at the platform, and W is the so-called 
wrench matrix, which is configuration dependent. Since 
fully constrained wire-driven parallel mechanisms in¬ 
clude more cables than DOFs, their wrench matrix W 
contains more columns than rows. Based on the above 
equation, the wrench-closed workspace can then be de¬ 
fined as the set of poses for which there exists at least 
one vector in the nullspace of W that has all its compo¬ 
nents of the same sign. When this condition is satisfied, 
the mechanism is said to be tensionable [18.160] and 
can theoretically resist arbitrary external wrenches sim¬ 
ply by increasing the pre-stress in the cables. 

The determination and the optimization of the 
wrench-closure workspace is a challenging problem 
that has attracted significant attention. Although the 
early work presented in [18.149, 161, 162] was mainly 
applicable to planar wire-driven robots, it paved the 
way to more general analyses by providing a mathe¬ 
matical modelling of the problem. Determining whether 
a given pose of the platform is located within the 
wrench-closure workspace is a rather straightforward 
problem. However, discretizing the workspace and ap¬ 
plying such a criterion to a mesh of points does not 
solve the problem satisfactorily because the wrench- 
closed workspace is generally not convex and may 
even be composed of disconnected components. There¬ 
fore, more powerful techniques are needed in order 
to obtain reliable and useful results. In [18.163], in- 
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terval analysis is used to determine the wrench-closed 
workspace of fully constrained wire-driven parallel 
robots while providing definite answers. l<s>M3H3DEEB 
and l<Et>»'IPH'HiM show an example of fully constrained 
wire-driven parallel mechanisms. 

18.10.3 Design and Prototyping 

Synthesizing a wire-driven parallel mechanism for 
a prescribed wrench-closed workspace is a very chal¬ 
lenging issue [18.164]. Also, in addition to the 
workspace, several other issues must be considered 
in the design of wire-driven parallel robots, for ex¬ 
ample, the determination of the potential interferences 

18.11 Application Examples 

Parallel robots have been successfully used in many ap¬ 
plications, a few of which are now briefly mentioned. 
Almost all recent land-based telescopes use parallel 
robots, either as a secondary mirror alignment sys¬ 
tem (e.g., the University of Arizona MMT or the ESO 
VISTA) or as a primary mirror pointing device [18.6]. 
A wire-driven parallel robot flew in the space shuttle 
mission STS-63 in February 1999 while an octopod 
(a parallel robot with eight legs) was used to isolate 
the space shuttle payload from vibration. All flight 
simulators use a parallel structure as a motion plat¬ 
form. They are nowadays used for driving simulators 
as well (e.g., the NADS-driving simulator [18.167]). 
In industry, numerous machine tools based on paral¬ 
lel structures have been designed. Some of them have 
found a niche market (e.g., the Tricept) and it can be 
expected that more will be used in the future. Ultra ac- 


between the wires. An elegant geometric solution to this 
problem is proposed in [18.165] but handling transla¬ 
tions and rotations simultaneously remains a challenge. 
Other design problems include the handling of special 
configurations and assembly mode changes as well as 
the accuracy and the practical implementation of the 
spools [18.166]. 

Several prototypes of wire-driven parallel robots 
were built and reported in the literature. Early proto¬ 
types include the NIST Robocrane [18.8], the Falcon 
robot [18.147], and the skycam system [18.148]. Many 
others were built, including those shown in the videos 
accompanying this chapter and the potential applica¬ 
tions are numerous. 


curate positioning devices based on parallel robots are 
proposed by companies, such as Physik Instrumente, 
Alio, Micos, and Symetrie. In the food industry, the 
Delta robot proposed by ABB and Demaurex is widely 
used for fast packaging. Other companies such as Adept 
are also developing fast parallel robots with 3- and 
4-DOFs. Medical applications are also growing, for 
example, in rehabilitation (after the pioneering work 
of the Rutgers ankle [18.168] and the platform Caren 
of Motek), in surgery (for example for spine surgery 
with the Renaissance robot of Mazor or for vitreoreti- 
nal surgery [18.169]) or for training and monitoring 
the motion of human joints [18.170], a topic in which 
cable-driven robots play an important role, as they do 
for aerial robots [18.171] or rescue operation (a version 
of the SkyCam robot has been used to fight oil fires dur¬ 
ing the Iraqi War). 


18.12 Conclusion and Further Reading 


The analysis of a parallel mechanism may be partly 
based on methods that are presented in other chapters 
of this handbook: 

• Kinematics'. Kinematics background is covered in 
Chap. 2. 

• Dytunnies: General approaches are presented in 
Chap. 3 while identification of the dynamic param¬ 
eters is addressed in Chap. 6. 

• Design: Design methodologies are covered in 
Chap. 16. 

• Control. Control issues are presented in Chaps. 7, 8, 
and 9 although the closed-loop structure of parallel 
robots may require some adaptation of the control 
schemes. 


It must also be emphasized that efficient numeri¬ 
cal analysis is a key point for many algorithms related 
to parallel robots. A system solving with Grobner 
basis, continuation method, and interval analysis is 
essential for kinematics, workspace, and singularity 
analysis. Further information and up-to-date exten¬ 
sive references and papers related to parallel robots 
can be found on the following two websites [18.38, 
100]. Useful complementary readings on parallel robots 
are [18.140, 147,164]. Parallel robots are slowly find¬ 
ing their way into various applications, not only in 
industry but also in field and service robotics, as dis¬ 
cussed in Part F of this handbook. Still, compared to 
their serial counterpart, their analysis is far from being 
complete. 
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Video-References 


3-DOF high-speed 3-RPS parallel robot 

available from http://handbookofrobotics.org/view-chapter/18/videodetails/43 
6-DOF cable-suspended robot 

available from http://handbookofrobotics.org/view-chapter/18/videodetails/44 
CoGiRo 

available from http://handbookofrobotics.org/view-chapter/18/videodetails/45 
Parallel 5R robot 

available from http://handbookofrobotics.org/view-chapter/18/videodetails/46 
Diamond 

available from http://handbookofrobotics.org/view-chapter/18/videodetails/47 
6-DOF statically balanced parallel robot 

available from http://handbookofrobotics.org/view-chapter/18/videodetails/48 
3-DOF dynamically balanced parallel robot 

available from http://handbookofrobotics.org/view-chapter/18/videodetails/49 
IPAnema 

available from http://handbookofrobotics.org/view-chapter/18/videodetails/50 
Par2 robot 

available from http://handbookofrobotics.org/view-chapter/18/videodetails/51 
Quadrupteron robot 

available from http://handbookofrobotics.org/view-chapter/18/videodetails/52 
R4 robot 

available from http://handbookofrobotics.org/view-chapter/18/videodetails/53 
Tripteron robot 

available from http://handbookofrobotics.org/view-chapter/18/videodetails/54 
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19. Robot Hands 


Claudio Melchiorri, Makoto Kaneko 


Multifingered robot hands have a potential ca¬ 
pability for achieving dexterous manipulation of 
objects by using rolling and sliding motions. This 
chapter addresses design, actuation, sensing and 
control of multifingered robot hands. From the 
design viewpoint, they have a strong constraint in 
actuator implementation due to the space limi¬ 
tation in each joint. After briefly introducing the 
overview of anthropomorphic end-effector and its 
dexterity in Sect. 19.1, various approaches for ac¬ 
tuation are provided with their advantages and 
disadvantages in Sect. 19.2. The key classification 
is (l) remote actuation or build-in actuation and 
(2) the relationship between the number of joints 
and the number of actuator. In Sect. 19.3, actu¬ 
ators and sensors used for multifingered hands 
are described. In Sect. 19.4, modeling and con¬ 
trol are introduced by considering both dynamic 
effects and friction. Applications and trends are 
given in Sect. 19.5. Finally, this chapter is closed 
with conclusions and further reading. 


Human hands have great potentialities not only for 
grasping objects of various shapes and dimensions, but 
also for manipulating them in a dexterous manner. It 
is common experience that, by training, one can per¬ 
form acrobatic manipulation of stick-shaped objects, 
manipulate a pencil by using rolling or sliding motions, 
perform precise operations requiring fine control of 
small tools or objects. It is obvious that this kind of dex¬ 
terity cannot be achieved by a simple gripper capable of 
open/close motion only. A multifingered robot hand can 
therefore provide a great opportunity for achieving such 
a dexterous manipulation in a robotic system. More¬ 
over, we have also to consider that human beings do not 
use hands only for grasping or manipulating objects. 
Exploration, touch, perception of physical properties 
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(roughness, temperature, weight, just to mention a few) 
are other fundamental tasks that we usually are able to 
perform thanks to our hands. We expect this type of 
capabilities also from robotic end-effectors and there¬ 
fore, by adding quite advanced sensing equipments and 
proper control strategies, we may improve the interac¬ 
tion capabilities with the environment, achieving for 
example active exploration, detection of sensing sur¬ 
face properties (local friction, impedance, and so on), 
tasks that are usually very hard or impossible for sim¬ 
ple grippers. For these and other reasons the study of 
multifingered robot hands has strongly interested the re¬ 
search community since the early days of Robotics. 

It was in late 1970s that Okada developed a mul¬ 
tifingered robot hand with a tendon driving system 
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and demonstrated a nut opening motion [19.1]. In 
early 1980s, two major projects on multifingered robot 
hands have been launched: the Stanford/Jet Propul¬ 
sion Laboratory (JPL; hand and the 

Utah/Massachusetts Institute of Technology (MIT) 
hand [19.2,3]. These two robot hands still represent 
a milestone and a term of comparison for the de¬ 
sign of new devices. Since then, several multifingered 
hands have been designed and developed in a num¬ 
ber research institutes all over the world. Among the 
most known, one can mention the Deutsches Zentrum 
filr Luft- und Raumfahrt (DLR) hand(s) (|43>M2ESIH1, 
|<Ef'jpjyma, and |4S>MaH3tE51), Mechanical Engi¬ 
neering Laboratory (MEL) hand, Electro-Technical 


19.1 Basic Concepts 

Before illustrating the main issues involved in the de¬ 
sign and use of a robotic hand, it is necessary to discuss 
some basic concepts and definitions often encoun¬ 
tered when dealing with these devices. In particular, 
terms like dexterity and anthropomorphism must be de¬ 
fined, and their implications on robotic hand design 
specified. 

19.1.1 Anthropomorphic End-Effectors 

The term anthropomorphism denotes the capability of 
a robotic end-effector to mimic the human hand, partly 
or totally, as far as shape, size, consistency, and gen¬ 
eral aspect (including color, temperature, and so on) are 
considered. As the word itself suggests, anthropomor¬ 
phism is related to the external perceivable properties, 
and is not, itself, a measure of what the hand can do. On 
the contrary, dexterity is related to actual functionality 
and not to shape or aesthetic factors. In this sense an¬ 
thropomorphism and dexterity are orthogonal concepts, 
whose reciprocal dependance (at least in the robotic 
held) has been not proved yet. 

As a matter of fact, we can find in the literature 
anthropomorphic end-effectors with very poor dexter¬ 
ity level, even though they are called hands , as the 
tasks they can perform are limited to very rough grasp¬ 
ing procedures [19.11]. Similarly, we can find smart 
end-effectors, capable of sophisticated manipulation 
procedures, without any level of anthropomorphism, 
e.g., the DxGrip-II [19.12]. Anthropomorphism itself is 
neither necessary nor sufficient to achieve dexterity, al¬ 
though it is quite evident that the human hand achieves 
a very high level of dexterity and represents a preferen¬ 
tial paradigm for dexterous robotic manipulation. 


Laboratory (ETL) hand, Darmstadt hand, Karlsruhe 
hand, University of Bologna (UB) hand 
|<s>M3EliaEai), Barrett hand (I^MSESE!!), Yasukawa 
hand, Gifu hand, U-Tokyo hand, Hiroshima hand. Soft 
Pisa/IIT hand I4S&M2H5EE1), and many 

others [19.4-10]. 

When designing a multifingered hand, on the basis 
of its utilization, one should first define the following 
key issues: number and kinematic configuration of the 
fingers, anthropomorphic or nonanthropomorphic as¬ 
pect, built-in or remote actuation, transmission system 
(in case of remote actuation), sensor assignment, inte¬ 
gration with a carrying device (robot arm), control. All 
these aspects are considered in this chapter. 


Anthropomorphism is a desirable goal in the design 
of robotic end-effectors mainly for the following rea¬ 
sons: 

• The end-effector can operate in a human-oriented 
environment (e.g., servicing robots), where tasks 
may be executed by robots or men as well. 

• The end-effector can be tele-operated by a human 
operator, by means of special-purpose interfaces 
(e.g., a data-glove), directly reproducing the oper¬ 
ator’s hand behavior. 

• For purposes of entertainment, assistance, and so 
on, a human-like aspect and behavior may be specif¬ 
ically required, like for humanoid robots. 

• For prosthetic devices anthropomorphism is a quite 
evident design goal. The development of end- 
effectors for prosthetic purposes [19.13-15] has 
recently produced so advanced devices that they can 
be fully considered robotic systems. 

While it is difficult to quantify the effective degree 
of dexterity of a robotic system, its anthropomorphism 
can be defined in a precise and objective way. In partic¬ 
ular, the aspects that mainly contribute to determine the 
anthropomorphism level of a robotic hand are: 

• Kinematics', concerning the presence of the main 
morphological elements (principal upper fingers, 
secondary upper fingers, opposable thumb, palm). 

• Contact surfaces: extension and smoothness of the 
contact surfaces, aspect that reflects on the capa¬ 
bility to locate contacts with objects all over the 
surface of the available links and on the presence 
of external compliant pads [19.16]. 
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• Size: i. e., the size of the robotic hand both referring 
to the average size of a human hand and the correct 
size ratio between the links. 

19.1.2 Dexterity of a Robotic Hand 

Besides the geometrical reproduction of the human 
hand, the main research target remains the emulation 
of those functionalities which make it such a versatile 
end-effector. 

Two are the main capabilities of a human hand: 

• Prehension, i. e., the hand’s ability to grasp and hold 
objects of different size and shape 

• Apprehension, or the hand’s ability to understand 
through active touch. 

In this sense, the human hand is both an output 
and input device [19.17]. As output device, it can ap¬ 
ply forces in order to obtain stable grasps or perform 
manipulation procedures. As input device, it is capable 
to explore an unknown environment providing informa¬ 
tion about the state of the interaction with it. The same 
features are desirable in robot hands. As a matter of 
fact, the application of robotic systems in unknown en¬ 
vironments requires dexterous manipulation abilities to 
execute complex operations in a flexible way. 

A widely accepted definition states that the dex¬ 
terity of a robotic end-effector is a measure of its 
capability of changing the configuration of the manipu¬ 
lated object from an initial configuration to a final one, 
arbitrarily chosen within the device workspace. Gen¬ 
erally speaking, with the term dexterity we intend the 


19.2 Design of Robot Hands 

The mechanical design of an articulated robotic hand 
can be performed according to many possible design 
concepts and options, even if a kinematical architecture 
has already been defined and size and shape specifi¬ 
cations imposed. One of the main issues is the design 
of a proper actuation and transmission system. This as¬ 
pect is crucial because space and dimensions are usually 
limited, being in general an anthropomorphic aspect 
and dimension a design goal to be pursued. Another 
aspect that is relevant for the design is the adoption 
of compliant structures (Fig. 19.1), in place of con¬ 
ventional mechanical joints, e.g., rolling pairs [19.20, 
21 ]. 

Note that, since many solutions and operating con¬ 
cepts can be adopted, what is presented here aims only 
at illustrating the most significant solutions, and does 


capability of the end-effector, operated by a suitable 
robotic system, to autonomously perform tasks with 
a certain level of complexity. An exhaustive review 
of scientific work developed so far about dexterity of 
robotic hands, with a quite complete and updated list of 
references, can be found in [19.18]. 

Even though the word dexterity itself has a very 
positive meaning, it may be useful to consider different 
levels of dexterity, associated with growing complexity 
and criticality of performable tasks. The dexterity do¬ 
main for robotic hands can be roughly divided in two 
main areas, i. e., grasping and internal manipulation. 

Grasping is intended as the capability of constrain¬ 
ing objects with a constraint configuration that is sub¬ 
stantially invariant with time (the object is fixed with 
respect to the hand). 

Internal manipulation is a controlled motion of the 
grasped object in the hand workspace, with the con¬ 
straint configuration changing with time. 

Further subdivisions of these two domains have 
been widely discussed in the literature (different 
grasp topologies [19.19], different internal manipula¬ 
tion modes based on internal mobility and/or contact 
sliding or rolling [19.18]). 

Although the notion of dexterity is well settled, the 
way to achieve it remains debated. Factors affecting 
the actual capabilities of a robotic end-effector are so 
many that often the analysis and above all the synthesis 
of dexterous hands do not take into proper consider¬ 
ation some of these elements, namely: morphological 
features; sensory equipment; control algorithms; task 
planning strategies; and so on. 


not pretend to be a complete discussion of all the possi¬ 
ble choices. 

19.2.1 Actuators Placement 

and Motion Transmission 

In order to actuate the joints of a robot hand, two basic 
approaches for the placement of the actuators are possi¬ 
ble, i. e.: 

• Placing the motors as close as possible to each joint, 
directly in the fingers and sometimes integrating 
them within the joint itself. 

• Placing the motors into the palm or in the forearm; 
in this case motion is transmitted to each joint by 
means of (complex) kinematic chains. 


Part B 119.2 



Part B 119.2 


466 Part B 


Design 



Fig.19.1a-c 

Three robotic 
fingers based on 
compliant joints. 

(a) The finger is 
obtained in a sin¬ 
gle teflon piece; 

(b) joint compli¬ 
ance is achieved 
with metallic 
springs; (c) fast 
prototyping al¬ 
lows for different 
compliant mech¬ 
anisms as joints 


In-site actuation can be defined as the case in which 
the actuator is hosted inside one of the two links con¬ 
nected by the actuated joint or is placed directly inside 
the joint: 

• Direct-drive actuation : the actuator is placed di¬ 
rectly on the joint, without transmission elements. 

• Link-hosted actuation : the actuator is placed inside 
one of the two links constituting the actuated kine¬ 
matic chain. 

In-site actuation simplifies the mechanical config¬ 
uration of the joint, reducing the transmission chain 
complexity. In particular, it has the great advantage 
that the motion of the joint is kinematically indepen¬ 


dent with respect to other joints. Usually, the size of 
the finger is imposed by the dimension of the actuators, 
and for technological reasons it is quite difficult to ob¬ 
tain both an anthropomorphic size and the same grasp 
strength of the human hand. Furthermore, the motors 
occupy a large room inside the finger structure, and it is 
a serious problem to host other elements, like sensors or 
compliant skin layers. A further negative aspect is that, 
since the mass of the actuators is concentrated inside 
the finger, the dynamic behavior of the system and its 
response bandwidth are reduced. 

Nevertheless, the recent advancement of actuator 
technology enables us to directly implement a quite 
powerful actuator with reasonable size in each joint. 
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This built-in actuation has been adopted, e.g., for DLR 
hand [19.4,22], ETL hand, Karlsruhe hand, Yasukawa 
hand, Barrett hand, Gifu hand, U-Tokyo hand, and Hi¬ 
roshima hand. Since this actuation does not include 
compliant element like tendons, we can keep a stiff 
transmission system, which leads to a stable control 
system even under a high gain (Sect. 19.4). An issue 
is the routing of wires for both power and signal ca¬ 
bles. This issue is more serious in distal joints than for 
the base joint, since the cables in distal joints produce 
a relatively large torque disturbance on the first joint, 
and therefore it is difficult to achieve a precise torque 
control for this joint. 

Remote Actuation 

Remote actuation is an alternative solution to in-site 
actuation. In remote actuation, the joint is driven by ac¬ 
tuators placed outside the links connected by the joint 
itself. Remote actuation requires a motion transmis¬ 
sion system, that must pass through the joints between 
the motor and the actuated joint. In some way, re¬ 
mote actuation must consider the problem of kinematic 
coupling between the actuated joint and the previous 
ones. Remote actuation is prevalent in biological struc¬ 
tures (e.g., in human hand), where the finger joints are 
moved by muscles placed on the palm or in the fore¬ 
arm. This human-like approach has been adopted in 
projects of robotic hands like the UB hand or the Na¬ 
tional Aeronautics and Space Agency (NASA) Robo- 
naut hand [19.23, 24], 

Remote actuation systems can be classified accord¬ 
ing to the type of adopted transmission elements, i. e., 
flexible- or rigid-link transmission. 

Flexible Link Transmission. Flexible link transmis¬ 
sion is based on deformable connections, either flexible 
or rotational, that can adapt to variations of configura¬ 
tion by changing the transmission path. Linear flexible 
transmissions are based on flexible elements with trans¬ 
lating motion, subject to tension (more frequently) 
or tension and compression. Two further subcate¬ 
gories can be identified: pulley-routed flexible elements 
(tendons, chains, belts) or sheath-routed flexible ele¬ 
ments (mainly tendon-like elements). Rotational flex¬ 
ible transmissions are based on flexible rotary shafts, 
that can transmit rotational motion inside the finger 
structure to the joint, where a final transforming mech¬ 
anism (a bevel gear or a worm gear) can be used to 
actuate the joint. 

Rigid Link Transmission. Rigid link transmission is 
mainly based on articulated linkages or on rolling 
conjugated profiles (mainly gear trains). A further sub¬ 
division can be made between parallel and nonparallel 


axes gear trains, like bevel gears, worm gears, and so 
on. 

19.2.2 Actuation Architectures 

Both in-site and remote actuation can be applied ac¬ 
cording to different types of organization, i. e., by using 
one ore more actuators for each joint and by making 
these actuators work in different ways. 

In general, we can consider an overall number N 
of joints for the robotic hand (the wrist joints are not 
considered) and a number M of actuators that are used 
to drive, directly or indirectly, the joints. According 
to different concepts of actuation and transmission, 
three main categories of actuation schemes can be 
identified: 

• M < N: some joints are passive, coupled, or under¬ 
actuated. 

• M = N: each joint has its own actuator and there are 
no passive, coupled or underactuated joints. 

• M > N: more than one actuator is operating on 
a single joint. 

These architectures strongly depends on the type of mo¬ 
tors. In particular, it is possible to recognize two main 
actuation modalities: 

• Single-acting actuators - each motor can generate 
a controlled motion in one direction only: return 
motion in opposite direction must be obtained by an 
external action, that can be a passive (e.g., a spring) 
or an active system (e.g., an antagonistic actuator); 
this is the case of tendon-based transmission sys¬ 
tems. 

• Double-acting actuators - each motor can generate 
a controlled motion in both directions and can be 
used alone to drive the joint or to cooperate with 
other actuators; in this case the functional redun¬ 
dancy can allow sophisticated drive techniques, like 
push-pull cooperation. 

Each category can be further subdivided. In the 
following, a brief description of the most frequently 
adopted schemes is presented. 

Single-Acting Actuators 
with Passive Return Elements 
Passive elements, like springs, can store energy dur¬ 
ing the actuation phase, restituting it during the return 
stroke (Fig. 19.2a). This mechanism leads to a simplifi¬ 
cation of actuation scheme, but requires mechanically 
backdrivable actuators. Other possible drawbacks are 
related to the loss of available power for the grasp and 
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the limited response bandwidth in case of low spring 
stiffness. 

Agonistic-Antagonistic 
Single-Acting Actuators 

Two actuators drives the same joint, acting in op¬ 
position in different directions (agonistic-antagonistic 
couple) (Fig. 19.2b). This solution leads to an N joints- 
2N actuators scheme and is quite complex since a large 
number of actuators must be placed in the hand. On the 
other hand, it may allow sophisticated control proce¬ 
dures, as both actuators can pull at the same time, with 
different intensity, generating a driving torque on the 
joint and a preloading of the joint itself (cocontraction, 
typical of tendon-driven joints): 

• Pros: cocontraction strategies, possibility to change 
the joint stiffness according to the grasping phase 
and therefore to limit the influence of friction 
during fast approaching motions; independent po¬ 
sition/tension control on each actuator can allow 
compensation of different path length in case of re¬ 
mote transmission; it is the most flexible solution 
for driving a joint. 

• Cons: back-drivability of actuators is required; high 
difficulty in hosting two actuators for each joint, 
both in in-site and in remote location; higher con¬ 
trol complexity; higher cost. 

Single-Acting Actuators Organized According 
to the Concept of Actuation Net 
This is a very interesting case, mimicking biological 
systems, but has not been implemented yet in robotic 
hands, except for some preliminary studies. N joints are 
driven by M actuators, being N < M < 2N. Each actua¬ 
tor cooperates in moving more than one joint, thanks to 
proper net-shaped transmissions: 



Fig.19.2a,b Single-acting actuator with an antagonist pas¬ 
sive element (a) and in an agonist-antagonist configura¬ 
tion (b) 


• Pros: cocontraction strategies, possibility to change 
the joint stiffness according to the grasping phase 
and therefore to limit the influence of friction dur¬ 
ing fast approaching motions; reduced number of 
actuators with respect to the 2N actuators scheme. 

• Cons: back-drivability of actuators is required; high 
complexity of the kinematic scheme and therefore 
high complexity in control. 

The simplest case of actuation net is represented 
by the so called N + 1 actuation (being N in this case 
the number of joints of a finger), frequently adopted in 
practice (Fig. 19.3). In this case, all actuators are cou¬ 
pled, and therefore a damage of any of them will result 
in a general failure. 

Double Acting Actuators with M < W 
In this case, the number of actuators is less than the 
number of joints. With reference to a single motor and 
several joints, two main subcases can be defined: 

1. The joints are kinematically coupled, in a fixed or 
variable way, so that the number of degrees of free¬ 
dom of this subsystems is reduced to one. 

2. The joints are selectively actuated by the motor, ac¬ 
cording to an active or passive selection subsystem. 

The former case can be further subdivided: 

• Joints kinematically coupled in a fixed way: In this 
kind of kinematical configuration, each motor can 
move more joints connected by rigid mechanisms 
with fixed transmission ratios. A typical application 
is obtained with the use of a gear train: the first 
link is directly actuated by a motor, while a gear 
transmission between a wheel fixed to the frame 
and a final wheel connected to the joint generates 
the relative motion of the second link (Fig. 19.4a). 
Should the motion of two parallel fingers be re¬ 
quired, their connection could be easily obtained 
mounting two gear wheels on the same shaft. An¬ 
other very common way to obtain this kind of 
kinematical linkage is to use tendon driven devices 


a) b) c) 



Fig.19.3a-c Remote actuation, (a)A-type, (b) 2/V-type, (c) 
N + 1-type 
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as shown in Fig. 19.4b. In artificial hand design, the 
main advantage when using joints driven by fixed- 
ratio mechanisms is the possibility to know and 
control the position of the second link. A disadvan¬ 
tage is that this kind of mechanisms does not adapt 
to the shapes of the grasped objects, and this may 
cause grasp instability. 

• Joints coupled in a noil-fixed way: This is the case of 
underactuated mechanisms and deformable passive- 
driven joints. A mechanism is said to be under¬ 
actuated when the number of actuators is smaller 
than the number of degrees of freedom. When ap¬ 
plied to mechanical fingers, this concept may lead 
to shape adaptation, i. e., underactuated fingers can 
envelope the objects to be grasped and adapt to their 
shape even with a reduced number of actuators. 
In order to obtain a statically determined system, 
elastic elements and mechanical limits must be in¬ 
troduced in underactuated systems (simple linear 
spring are often used). In the case of a finger clos¬ 
ing on an object, for instance, the configuration of 
the finger is determined by the external constraints 
associated with the object. An example of an un¬ 
deractuated two-degrees of freedom finger is shown 
in Fig. 19.5 [19.25]. The finger is actuated through 
the lower link, and a spring is used to maintain the 
finger fully extended. A mechanical limit is used 
to keep the phalanges aligned under the action of 
this spring when no external forces are applied on 
the phalanges. Since the joints cannot be controlled 
independently, the behavior of the finger is deter¬ 
mined by the design parameters (i. e., the geometric 
and the stiffness properties). Hence, the choice of 
these design parameters is a crucial issue. 

Another approach consists in coupling the motion 
of two adjacent joints by means of deformable link¬ 
ages. This feature introduces in the kinematical 
chain the needed compliance to fit to the shapes of 
the grasped objects. A very simple mechanism of 
this category is reported in Fig. 19.6. Structurally it 
is similar to the mechanisms based on a fixed cou¬ 
pling, the only important difference is the addition 
of a spring to give extensibility to the tendon. This 
spring allows to decouple the motion between the 
first and second link when an external force is ap¬ 
plied to the distal one. This solution is widely used: 
a well known example is the DLR hand. The bene¬ 
fits of this solution are mainly due to the possibility 
to fit to the shapes of objects. A design problem is 
the choice of the stiffness of the deformable element 
in order to achieve at the same time a strong grasp 
and a good shape adaptability. 

• Joints selectively driven by only one motor: With 
this solution, the motion generated by only one 


(large) motor is transmitted and distributed to sev¬ 
eral joints. Actuation and control of each joint is 
obtained by means of insertion-disinsertion devices 
like self-acting or commanded clutches. 

Double-Acting Actuation, with M = N 
This is a very common case: each joint is driven in both 
directions by the same actuator. The achievable perfor¬ 
mances are therefore similar (equal) in both directions, 
but particular attention must be paid to backlash, and it 
is usually necessary to preload the transmission system. 
In particular, preload is mandatory in case of trans¬ 
mission by means of flexible elements like tendons 
(Fig. 19.3a). Furthermore, the adoption of a closed- 



Fig. 19.4 Double-acting actuator with N = M based on 
gears (a) and tendons (b) 



Fig.19.5a-d Grasping sequence performed by a finger based on 
underactuated mechanism 
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loop tendon transmission requires that the overall length 
of the tendon route must be kept constant, accord¬ 
ing to the concept that winded and unwinded parts 
of the tendon on the motor pulley have the same 
length; this involves the need of length-compensating 
mechanisms (e.g., pulley-trains, cams) every time that 
changes in the geometry of the finger cause a dif¬ 
ferential displacement of the tendons. In spite of this 


required complexity, this actuation scheme has been 
widely used, with simple pulley routing (UB hand, 
Okada hand, ...), or sheath-routing (Salisbury hand, 
Dipartmento di Informatica Sistemica e Telematica 
(DIST) hand), that has a simpler mechanical structure 
but must face the problem of sheath-tendon friction 
(application of high preload is not convenient in this 
case). 


19.3 Technologies for Actuation and Sensing 


In this Section, a brief description of the main issues 
related to technological aspects of actuation and sensing 
for robot hands is reported. A more general presentation 
and detailed description of these aspects is given in Part 
A (Chaps. 4 and 5) and Part C (Chap. 28). 

19.3.1 Actuation 

Electrical actuators are without doubt the most com¬ 
mon choice for actuating robot hands. As a matter 
of fact, electric motors have very good performance 
in terms of position/velocity control, have a reason¬ 
able mass/power ratio, and are a very common tech¬ 
nology, that does not require external devices (as for 
hydraulic or pneumatic actuators). However, there are 
several other possibilities. For example: ultrasonic mo¬ 
tors (Keio hand [19.26]), chemical actuators, pneu¬ 
matic actuators (McKibben in the Shadow hand [19.27]; 
|<£t>Kii!if!UfJI), spring based actuators (as for the 100G 
Capturing Robot [19.28] l<s%KJJl!£Ii£l), twisted string 
actuators (Fig. 19.7) [19.29] (Dexmart hand [19.30]; 
and others. 

In particular, for pursuing quick responses, either 
pneumatic or spring-based actuators may be good solu¬ 
tions, although it should be noted that a braking system 


with quick response is essential for achieving good po¬ 
sition controllability for this type of actuators. 

19.3.2 Sensors 

In robot hands, as in other robotic devices, sensors can 
be classified in two main categories: proprioceptive and 
exteroceptive sensors. The first type of sensors mea¬ 
sures physical information related to the state of the 
device itself (e.g., position, velocity, and so on), while 
the second one is devoted to the measurement of data re¬ 
lated to the interaction with objects/environment (e.g., 
applied forces/torques, friction, shape, and so on). 

Joint Position/Velocity Sensors 
For control purposes, there is the obvious necessity 
of measuring position/velocity of the actuated joints. 
A major problem consists in the limitation of the avail¬ 
able space, both for the sensors and for the wires. 
Different technological solutions can be adopted, but 
a rather common choice is based on Hall-effect sensors, 
that are sufficiently small, precise and reliable for this 
type of application. In case of remote actuation, there 
is the possibility of having two position/velocity sen¬ 
sors for each joint: one located in the actuator (e.g.. 



Fig. 19.7 The twisted string concept: 
by twisting the string, its twisted 
string concept length is reduced 
transforming a rotational motion into 
a linear one 
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an encoder) and one placed in the joint itself, often 
necessary because of the non linearities introduced by 
the transmission system (elasticity, friction, and so on). 
Quite often, this latter sensor is specifically designed 
and implemented for the given hand, being commer¬ 
cially available sensors too large and not suitable for 
installation in the joints. 

Tendon Tension Sensor 
and Joint Torque Sensor 

It is well known that humans can control finger tip 
compliance as well as finger tip force by controlling 
voluntary muscles. In remote actuation, it is essential 
to measure the tendon tension for two main reasons: for 
compensating the friction existing in the transmission 
system, and for measuring the external contact force. 
Figure 19.8 shows a way for measuring the tendon ten¬ 
sion where the tendon is pressed by an elastic plate 
with a strain gauge. When a tension is applied to the 
tendon, the sensor measures a force composed of axial 
and bending force components. The displacement of the 
elastic plate due to the axial force component is negligi¬ 
bly small compared with that due to the bending force 
component. As a result, the bending force component 
generates a bending deformation for the elastic plate. 
This deformation is transformed in an electric signal 
by means of proper transducers, such as strain gauges 
attached on the surface of the plate or optoelectronic 
components [19.31,32]. Now, suppose /V-type actua¬ 
tion with two tension sensors, as shown in Fig. 19.9, 
where joint torque r is given. Note that x = r(T\ — 73) 



Fig. 19.8 Tendon tension sensing 


Amplifier 


Fig. 19.9 Tension sensor based torque sensing 


where r, 7j, and 73 are the pulley radius and ten¬ 
don tensions, respectively. Since we can measure e\ 
and ei corresponding to 7j and 73, x can be obtained 
by feeding both e\ and e 2 into the differential circuit. 
This approach, however, includes a couple of issues. 
The main problem is the plastic deformation of the 
sensor plate under an extreme large pretension. Once 
such a plastic deformation has happened, the sensor 
will never work appropriately anymore. Another mi¬ 
nor issue is are that two sensors are always necessary 
for measuring a joint torque. To cope with these is¬ 
sues, the tension-differential-type torque sensor [19.28] 
can be used as shown in Fig. 19.10. The sensor is de¬ 
signed with just a single body and it partially includes 
an elastic part where at least one strain gauge is at¬ 
tached. The working principle of the sensor, shown in 
Fig. 19.10a, supposes that a torque is applied to the 
joint. This means that 7) and 73 have different val¬ 
ues. This difference causes a bending force around the 
strain gauge. The key is that the bending force is kept 
to zero even under an extremely large tension as far 
as no joint torque is given. Therefore, we are com¬ 
pletely released from the plastic deformation of the 
elastic plate due to pretension. Furthermore, the sen¬ 
sor is constructed with just a single body. There are 
couple of variations in this type of torque sensor. As 
decreasing the pulley distance in Fig. 19.10a, the sen¬ 
sor eventually results in the single-pulley-version with 
zero distance, as shown in Fig. 19.10b. The single- 
pulley-version has been implemented into Darmstadt 

a) 



Fig.19.10a-c Tension-differential type (TDT) sensor, 
(a) Double pulley version, (b) single pulley version, 
(c) pulley-less version (after [19.28]) 
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hand [19.33] and MEL hand [19.34]. Furthermore, if 
the sensor is built in the finger link connected by the 
concerned tendon, there is no relative motion between 
the sensor and the tendon. As a result, we can remove 
the pulley, as shown in Fig. 19.10c. This is called as 
the pulley-less version and has been implemented into 
Hiroshima hand. The tension-differential-type torque 
sensor will be a powerful tool for measuring a tendon 
drive joint. 

Finger Tip Tactile (or Force) Sensors 
Most robot manipulation and assembly tasks would 
benefits of the utilization of tactile sensory information. 
When lifting an object, tactile sensing could detect the 
onset of slip in time for corrective action to be taken. In 
addition to the contact point between the finger tip and 
the object, several objects properties, such as friction 
coefficient of the object surfaces, surface texture, and 
weight can be determined by utilizing a finger tip tac¬ 
tile (or force) sensor. A six-axis force sensor allows us 
to detect contact point as well as contact force between 
finger and environment, if a single contact is assumed. 
For the finger model as shown in Fig. 19.11, the follow¬ 
ing relationship between the sensor output and contact 
force may be defined 

F s =f, (19.1) 

M s =x c xf, (19.2) 

where/ e R 3 , F s e R 3 , M s e R 3 , and jc c e R 3 are the 
external force vector, the force vector measured by the 
six-axis force sensor, the moment vector measured by 
the six-axis force sensor, and the position vector in¬ 
dicating the contact position, respectively. From the 
first equation, we can directly obtain the contact force. 
Putting F s into the second equation leads to M s = x c x 
F s . x c is determined in such a way that M s = x c xF s 
may be satisfied. For a finger with convex object, we 
have always two mathematical solutions as shown in 
Fig. 19.12a where the meaningful solution is the one 
satisfying /*« < 0, n being the outward normal direc¬ 
tion to the finger’s surface (a finger can only push the 



object). However, for a finger with concave shape, we 
have at least four mathematical solutions, as shown in 
Fig. 19.12b where two of those are physically possible. 
A finger with the six-axis force sensor located in the 
fingertip. Fig. 19.12. c, can avoid multiple solutions. On 
the other hand, only forces applied to the fingertip can 
be detected, and if more links are in contact with the ob¬ 
ject it would be necessary to have a force/torque sensor 
placed in each of them. 

This type of solution, i. e., a multiaxis sensor for 
measuring not only forces and torques but also the posi¬ 
tion of the contact point, is known in the literature as the 
intrinsic tactile (IT) principle [19.35]. In general, with 
respect to the use of traditional tactile sensors, see later, 
it leads to a simplification in the design since it requires 
less wires and connections for the sensor. 

Tactile Sensors 

Another important class of sensing devices consists of 
tactile sensors, which are used for several purposes, 
such as shape recognition, contact point determination, 
pressure/force measurement. A number of tactile sen¬ 
sors have been proposed in the literature, with several 
different solutions concerning the implementation fea¬ 
tures: optical, piezoresistive, piezoelectric, and so on. 
References [19.36,37] give an overview on technolo¬ 
gies and applications. 

a) 




Fig. 19.12 Interpretation of solutions, (a) Convex shape, 
(b) finger with a concave shape, (c) sensor located in the 
fingertip 
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b) 



Fig.19.13a,b A Tactile sensor, (a) Scheme of a tactile sen¬ 
sor, (b) example of data from a tactile sensor 


goal of this class of sensors is to measure the map of 
pressures over the sensing area. 

In general, the types of information that may be ob¬ 
tained from a tactile sensor are: 

• Contact. This is the most simple information given 
by the sensor, concerning the presence or absence 
of a contact. 

• Force : Each sensing element provides an informa¬ 
tion related to the amount of locally applied force, 
which can be used in several manners for successive 
elaborations. 

• Simple geometrical information , i. e., position of the 
contact area, geometrical shape of the contact itself 
(planar, circular, and so on). 

• Main geometrical features of the object. By proper 
elaborations of the data of the taxels, it is possible 
to deduce the type of object in contact with the sen¬ 
sor, for example a sphere, a cylinder and so on (data 
relative to the 3-D (three-dimensional) shape). 

• Mechanical properties, such as friction coefficient, 
roughness, and so on. Also thermal properties of the 
object may be measured by a tactile sensor. 

• Slip condition, i. e., the relative movement between 
the object and the sensor. 

Several technologies have been adopted for the de¬ 
sign of tactile sensors, ranging from piezoresistive to 
magnetic, to optical effects, and so on. Among the most 
common, one can mention: 


Tactile sensors have been introduced in robotics 
since the late 1970s. Nowadays, as the force sensors, 
also tactile sensors are commercially available devices. 
Probably, they represent the most commonly adopted 
sensorial class for industrial grippers, even though they 
are often used as advanced on/ojf devices to check 
whether a grasp or contact condition occurs. 

Usually, they consist in a matrix (array) of sensing 
elements. Each sensing element is usually referred to 
as a taxel (from tactile element), and the whole set of 
information is called a tactile image. Fig. 19.13. Main 


• Resistive and conductive effect 

• Electromagnetic effect 

• Capacitive effect 

• Piezoelectric effect 

• Optical effect 

• Mechanical methods. 

Each of these technologies has positive and negative 
aspects. Common drawbacks, however, are the size of 
these sensors, usually quite large in comparison with 
the available space, and the necessity of a high number 
of electrical connections. 


19.4 Modeling and Control of a Robot Hand 


The dynamic model of a robot hand with in-site actua¬ 
tion is very similar to the model of a traditional (indus¬ 
trial) robot, and the hand can be considered as a collec¬ 
tion of robot manipulators. On the other hand, remote 
actuation introduces some peculiar features that have to 
be carefully considered. In particular, the problems tied 
to nonlinear phenomena (e.g., friction and backlash), 


compliance of the transmission system, and noncolo¬ 
cation of sensors and actuators are very critical for the 
design of the control. Moreover, the use of single-acting 
actuators, such as tendon based actuation systems, re¬ 
quires the adoption of proper control techniques, which 
allow the imposition of the desired torque at each joint 
of the hand, despite the coupling among them. 
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Fig.19.14a,b Model of a robot joint (a) with transmission flexibility, and (b) with tendon based transmission 


19.4.1 Dynamic Effects of Flexible 
Transmission Systems 

The transmission system of robot hands with remote 
actuation is usually characterized by an high level of 
friction and non negligible dynamic effects which com¬ 
plicate the control problem. A simple representation 
considers a single axis motion with two inertial ele¬ 
ments linked by an elastic transmission. This is the 
typical representation of elastic joints in which the for¬ 
mer element represents the motor inertia, while the 
latter is related to the inertial properties of the ac¬ 
tuated joint/link (Fig. 19.14a). More complex models 
assume a dynamic model for the transmission system, 
i. e., the classical representation of tendons based on the 
serial repetition of masses linked by springs/dampers, 
reported in Fig. 19.14b. These simple models are par- 



Fig. 19.15 Root contour of the transfer function (19.3) 
with variable k t 


ticularly useful to understand some drawbacks and 
limitations due to the fact that actuation system and ac¬ 
tuated element are located in two different places and 
the motion is transmitted by a nonideal (that is not 
purely static) element. If we consider the capability of 
the fingers’ joint of applying a force on the environ¬ 
ment, the effect of the transmission system on the open 
loop response of the system modeled as in Fig. 19.14a, 
are a noticeable reduction of the bandwidth, and an 
important phase delay between the input Fj, (the force 
applied by the motor) and the output F c (the force ex¬ 
changed at the contact). As shown in Fig. 19.15, the 
open-loop transfer function 

Fa = 

F c 

(b c s + k c )(b t s+kt) 

[/A 2 + (F t + b c )s + k t + k c ] (j m s 2 + btS + k t ) - (b t s + k t ) 2 

(19.3) 

is characterized by four poles that, for growing values 
of the transmission stiffness k s , move from their initial 
locations (that depend on the values of physical param¬ 
eters j\, j m , etc., although for k s = 0 at least one pole 
is in the origin of the Gauss plane) towards the poles 
of a system with an infinitely rigid transmission (for k c 
tending to oo, two poles go to infinity) (and a total iner¬ 
tia given by the contributions of both the motor and the 
link) whose transfer function is 

El = _ (b cS + k c ) _ (ig 4) 

Fc (A +jm)s 2 + b c S + k c 

As a consequence, the bandwidth of the system with 
flexible transmission, that for high values of k s approx- 
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Fig. 19.16 Bode plots of the open loop transfer function 
(19.3) for low stiffness values (continuous line ) high stiff¬ 
ness value (dashed line ) and with no transmission (dotted 
line ) ► 

imates those of (19.4), decreases when the compliance 
of the transmission is not negligible, see the Bode plots 
reported in Fig. 19.16. The bandwidth of the system 
with flexible transmission is strongly affected by the lo¬ 
cation of the speed reducer: when the reducer is placed 
at the joint the bandwidth is K r times ( K r is the reduction 
ratio) higher than the one achievable with the reduc¬ 
tion applied directly on the motor [19.38] (Fig. 19.17). 
Moreover, it is worth to notice that for low level of 
the stiffness k t a sharp phase drop occurs at the fre¬ 
quency of the flexible mode. Therefore, there are some 
frequencies (relatively low) at which the force applied 
by the motor and the one measured with a sensor in 
the finger’s joint are completely out of phase. These 
effects, which may cause the instability of the overall 
system under force control (or impedance control) are 
referred to as noncolocation. In general, when actuators 
and sensors are physically located at different points of 
a flexible structure (or a structure with flexible transmis¬ 
sion), there will be unstable modes in the closed-loop 
system [19.39]. 

From the control viewpoint, the problem of me¬ 
chanical transmission flexibility is further exasperated 
by the non linear frictional phenomena that inevitably 
affect remote actuation and motion transmission. As 
a matter of fact, the linear viscous friction, represented 
in Fig. 19.14 by the damping coefficient b t , is accompa¬ 
nied by stiction and Coulomb friction, both of which are 
discontinuous at zero velocity (Fig. 19.18). These non- 
linearities may cause limit cycles and input-dependent 
stability, and must be accurately taken into account 
in the design of the robot hand structure as well as 
of its control architecture [19.40]. For instance, in the 
design of the Utah/MIT dexterous hand, depicted in 
Fig. 19.19, in order to reduce static friction, the idea 
of using tendon sheaths was abandoned in favor of pul¬ 
leys [19.3]. In order to find an optimal trade-off between 
complexity and reliability of the mechanical arrange¬ 
ment and achieved friction level, a number of solutions, 
which combine sheaths and pulleys for routing the ten¬ 
don from actuators to fingers’ joints has been adopted 
in the design of robot hands, e.g., the Stanford/JPL 
hand (|43>XU>H!U1B) and the UB hand 3 reported in 
Fig. 19.20. This device is characterized by an extremely 
simple structure, with the tendons completely routed 
within sheaths, but on the other hand the friction can¬ 
not be absolutely neglected and a precise modeling of 
the interaction between the tendons and the tube is nec¬ 
essary for control purposes [19.41]. 


Magnitude (dB) 




Fig. 19.17 Reducer location on the medial joint of the DLR 
hand II 



Reducer 


Motor 



Fig.19.18a,b Frictional phenomena: viscous friction (a) and stic¬ 
tion and Coulomb friction (b) 
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19.4.2 Transmission Model 

of Tendon-Outer-Tube System 


where K t and <p% are the total stiffness and the equiva¬ 
lent backlash, respectively, and those are given by, 


This system can be modeled as shown in Fig. 19.21 
where T m , T out , Tq, £j n , f out , R u x, and L are the tension 
at the input side, the tension at the output side, the initial 
pretension, the displacement at the input side, the dis¬ 
placement at the output side, the local radius of routing, 
the coordinate system along the wire, and the length of 
the tendon, respectively. The relationship between the 
tension at output and the input displacement is given 
by [19.42], 

Tout-To = ^(^6), (19.5) 



Fig. 19.19 The Utah/MIT robotic hand 



Fig.19.20a,b Tendon based robot hands: Stanford/JPL hand (a) and 
UB hand 3 (b) 


1 _ 1 1 1 

Kt - Y e + Y & + K^ 

A 

ap w exp(A) — 1 

TqL exp(A) — A — 1 
“ ~EA A 

A = Xli^ i i /xsgn ^ n 

where K e , K s , K v , K zp , fx, E, A and ^ are the stiffness of 
environment, the stiffness of force sensor, the apparent 
stiffness of the tendon, the friction coefficient, Young’s 
modulus, the cross sectional area, and the bending angle 
of each segment of tendon, respectively. For example, 

Z>i= 2jr 

for the case given in Fig. 19.21. As can be seen from 
this example, the friction related parameter A increases 
dramatically when the tube is heavily bent. While we 
have a big advantage of choosing a free route for the 
power transmission, it brings a large nonlinearity for 
the transmission system. We would note that while both 
the apparent stiffness of the tendon and the equivalent 
backlash vary depending upon A which is the function 
of the curvature of the route as well as the friction coef¬ 
ficient, (pB and K ap result in </)% = 0 and K^ v = K w under 
/x = 0. From the view point of control, such hysteresis 
is, of course, not desirable. To cope with these issues, 
each tendon should be designed as short as possible, so 
that we may keep high stiffness and small backlash in 
the transmission system. 

19.4.3 The Control Through Single-Acting 
Actuators 

The use of single-acting actuators (i. e., standard mo¬ 
tors with tendinous transmission), which are commonly 


(19.6) 

(19.7) 

(19.8) 

(19.9) 
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assumed in the design of robot hands with remote 
actuation, requires the adoption of specific control tech¬ 
niques in order to guarantee the desired torques at 
the joints and to maintain at each time a positive ten¬ 
sion on the tendons. To this purpose, the tendons are 
treated like inelastic frictionless elements and the prob¬ 
lem is coped in a way completely decoupled from 
the issue of the device stability, discussed in previous 
sections. 

A tendon, routed in the finger structure through 
sheaths or/and pulleys, can be modeled by means an 
extension function k{6) [19.43] relating the joints’ con¬ 
figuration with the tendon elongation. In the case of the 
tendon network represented in Fig. 19.3, the extension 
functions of the three tendons have the form 

h(0) =l Oi ±R0i±RO 2 , 

where R is the radius of the pulleys and 

0 = [M 2 ] t 

is the vector of the joint variables. Once the extension 
function has been determined, it is straightforward to 
derive the relationship between tendons forces and re¬ 
sulting joints torques. As a matter of fact, the relation 
between the joint speeds 0 and tendon speeds l can be 
deduced by simply differentiating the expression of ex¬ 
tension functions 

1= ^(0)0 =P(0)0 ■ (19.10) 

du 

Because of the conservation of the power, from (19. 10), 
one can achieve 

r = P T (0)f, (19.11) 

where r are the torques exerted on the joints, and/ are 
the force applied by tendons. From ( 19.1 1) it results that 
the force transmitted by a tendon may affect (and, in 
general, will affect) more than one joint. 

In order to guarantee the possibility of exerting joint 
torques in every direction under the constraint of pure 
tensile forces, for any r G M" it must exist a set of 
forces /i G R™ (n and m are respectively the number of 


19.5 Applications and Trends 

In the industrial environment, simplicity and cost are 
the main guidelines for the design of end-effectors, 
and therefore simple devices, as open-close grippers, 


joints and the number of tendons) such that 

r=P T (0)f and /; > 0, i = 1,. .., m . (19.12) 

In this case the tendon network is said force closure. 
If the condition expressed by (19.12) is verified, given 
a desired torque vector r it is possible to compute the 
force that the actuators must provide to the tendons ac¬ 
cording to 

/ = pt(0) T +/ N , (19.13) 

where 

Pi = PIP 1 ?)- 1 

is the pseudo-inverse of the coupling matrix P T and 
/n e ^f(P T ) 

is a vector of internal forces that insures that all tendon 
tensions are positive. In general, internal forces will be 
chosen as small as possible, so that the tendons are al¬ 
ways taut but are not subject to excessive strains. 

19.4.4 Control of a Robot Hand 

The modeling and control aspects described in the 
previous Sections, although very important and fun¬ 
damental, can be considered as a sort of low level 
problems in the control of a robot hand, in the sense 
that they are related to the specific physical properties 
of the device. 

There are also other problems that must be faced 
and solved in order to operate in a profitable manner 
with a multifingered hand. These problems are solved 
by a proper design of a high level control for the hand, 
that must take into account the interaction of the hand 
with the objects and more in general with the envi¬ 
ronment. In this context, general aspects that must be 
considered are: the control of forces/torques applied 
at the contact points, the necessity to model contact 
compliance/friction effects, the type of mobility both 
for the fingers and at the contact (rolling, sliding, ...), 
a suitable planning algorithm for grasping and/or ma¬ 
nipulating the objects, and so on. 

These problems are illustrated in detail in Part C, 
Chaps. 37-39. 


are very common and widely used. This situation has 
led during the years to the development of a num¬ 
ber of special-purpose devices, optimized for single 
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specialized operations but not suitable for other tasks. 
At the moment, dexterous multifingered hands have 
not really been applied to any major application, 
mainly because of problems of reliability, complexity, 
cost. 

On the other hand, more and more operations are 
currently envisaged for robots working in environments 
designed for, and utilized by, human operators. Enter¬ 
tainment, maintenance, space/underwater applications, 
help to disable persons are just a few examples of use of 
robotic systems in which interaction with tools and ob¬ 
jects designed for human beings (or directly with them) 
is implied. In all these circumstances, the robot must 
be able to grasp and manipulate objects (different in di¬ 
mension, shape, weight, ...) similarly to humans, and 
therefore a robot hand, with a proper number of fingers 
and joints and also with an anthropomorphic appear¬ 
ance, seems to be the most adequate solution. 

There are several projects aiming at developing an¬ 
thropomorphic robots. Among others, one can mention 



Fig. 19.22 The NASA/JPL Robonaut 


the NASA/JPL Robonaut [19.24], Fig. 19.22, the de¬ 
vices developed at the DLR, the several projects on 
humanoid robots currently under development. 


19.6 Conclusions and Further Reading 


The design of multifingered robot hands has attracted 
the interest of the research community since the early 
days of robotics, not only as a challenging technical 
problem itself but, probably, also because of anthro¬ 
pomorphic motivations and the intrinsic interest for 
a better knowledge of the human beings. In the last 
decades, has previously discussed, several important 
projects have been launched, and important examples 
of robot hands developed. Nevertheless, the current sit¬ 
uation is that reliable, flexible, dexterous hands are still 
not available for real applications. For these motiva¬ 
tions, it is easy to foresee also for the future a consistent 
research activity in this fascinating field, with develop¬ 
ments at the technological (sensor, actuator, material, 
...) and methodological (control, planning, ...) level. 


Important connections with other scientific fields are 
also expected, as for example with cognitive science. 

Being this research area so wide, it is not simple 
to suggest to interest readers further readings, except 
for quite classical books such as [19.43-45]. As a mat¬ 
ter of fact, depending on the specific research area, 
many publications are available, although often not or¬ 
ganized as reference books, but mainly as technical 
papers published in journals or presented at interna¬ 
tional conferences. Moreover, since hundreds of new 
papers are published every year covering the different 
aspects of this robotic field, it is really quite difficult, 
and also not fair, to give at the moment specific sug¬ 
gestions for further readings. We can only refer to the 
citations already provided in the references. 
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The PISA-1 IT SoftHand 

available from http://handbool<ofrobotics.org/view-chapter/19/videodetails/749 
The PISA-1 IT SoftHand 

available from http://handbookofrobotics.org/view-chapter/19/videodetails/750 
The Salisbury Hand 

available from http://handbookofrobotics.org/view-chapter/19/videodetails/751 
The Barrett Hand 

available from http://handbookofrobotics.org/view-chapter/19/videodetails/752 
The Shadow Hand 

available from http://handbookofrobotics.org/view-chapter/19/videodetails/753 
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The DLR Hand 

available from http://handbookofrobotics.org/view-chapter/19/videodetails/754 
A high-speed Hand 

available from http://handbookofrobotics.org/view-chapter/19/videodetails/755 
The UBH2, University of Bologna Hand, ver. 2 (1992) 

available from http://handbookofrobotics.org/view-chapter/19/videodetails/756 
The Dexmart Hand 

available from http://handbookofrobotics.org/view-chapter/19/videodetails/767 
DLR Hand 

available from http://handbookofrobotics.org/view-chapter/19/videodetails/768 
The DLR Hand performing several task 

available from http://handbookofrobotics.org/view-chapter/19/videodetails/769 
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20. Snake-Like and Continuum Robots 


Ian D. Walker, Howie Choset, Gregory S. Chirikjian 


This chapter provides an overview of the state 
of the art of snake-like (backbones comprised 
of many small links) and continuum (continu¬ 
ous backbone) robots. The history of each of these 
classes of robot is reviewed, focusing on key hard¬ 
ware developments. A review of the existing theory 
and algorithms for kinematics for both types of 
robot is presented, followed bya summary of mod¬ 
eling of locomotion for snake-like and continuum 
mechanisms. 


20.1 Snake Robots - Short History 

The study of snake robots began in earnest with the pio¬ 
neering work of Shiego Hirose, a professor at the Tokyo 
Institute of Technology. He called his snake robots these 
active chord mechanisms, or ACMs for short. The first 
successful locomotive snake was ACM III (Fig. 20.1a), 
which he developed between 1972 and 1975; the ACM 
III robot had 20 segments and used passive wheels to 
form a constraint between the mechanism and the en¬ 
vironment. Each segment was able to yaw with respect 
to each other and the combined motion of these degrees 
of freedom propelled the entire robot forward [20.1]. 
This type of coordinated control resulting in locomo¬ 
tion is called a gait, more formally, a gait is a cyclic 
motion in the mechanisms internal degrees of freedom 
that result in net motion for the snake robot. The next 
step in the development of the Hirose family of snake 
robots were the OBLIX (Fig. 20.1b) (1978-1979) and 
MOGURA (1982-1984), two robots that could achieve 
three-dimensional movements and lift their heads by 
rotating their oblique joints. The benefit of a single 
actuator operating an oblique joint over a true three- 
dimensional joint is simplicity, light weight, and ease 
to control because each segment only needed to be ac¬ 
tuated by one motor [20.2]. 
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Hirose’s lab then began exploring possible appli¬ 
cations of snake robots with Koryu I and II (1985- 
present). Each one of Koryu’s load-bearing modules 
uses actuated joints to lift itself, lower itself, and in 
more recent incarnations, angle its wheel base to fit 
the ground. With such forms of actuations, these large 
snake robots were able to traverse uneven terrain and 
even climb stairs, all while carrying a load, distributed 
along the entire robot [20.2]. Inspired by the ability of 
real snakes to traverse uneven terrains, Hirose’s lab de¬ 
veloped a smaller mechanism, called Souryu I and II 
(1975-present) for the purposes of search and rescue 
(Fig. 20.2). The Souryu robots were able to climb over 
rubble because of their low center of mass and treads 
all along their bodies [20.3, 4], 

In 1999, Hirose returned to the ACM with Slim 
Slime, a robot that was pneumatically actuated and 
formed a smooth curve instead of the usual sectioned, 
angular curve of other snake robots [20.5]. After Slim 
Slime, the lab began working on ACM-RI, R2, R3, and 
R4 (2001-present.) These robots are much closer to the 
design of the first ACM, with passive wheels and many- 
sectioned bodies. These robots, unlike the first ACM, 
can lift their bodies and move in three dimensions. Each 
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Fig. 20.1 Hirose’s snake robots starting (a) with the active chord 
(ACM III) and (b) Oblix/Mogura mechanisms (after [20.2]) 


Fig.20.2a,b The Koryu I, II robot drving outside (a) (after [20.2]) 
and (b) the Souryu I. II robot maneuvering over rubble (after [20.4]) 

section has wheels that are set 90° from each other in 
an alternating fashion. They were created for use in 
search and rescue [20.1]. The most recent ACM is R5, 
a fully amphibious robot that uses passive wheels to roll 
along the ground and paddles to move through the wa¬ 
ter [20.6] (Fig. 20.3). 

Snake robot research began in the United States in 
the late 1980s. Joel Burdick and his then-student Greg 
Chirikjian at Caltech developed a continuum theory to 
model the shape of snake robots in general. Burdick 
and Chirikjian coined the term hyper-redundant ma¬ 
nipulator, which means a mechanism that has many 
more degrees of freedom that are necessary to position 
and orient its end-effector. They built Snakey in 1992; 
Snakey consisted of three degree of freedom bays or ac¬ 
tuated trusses that could both manipulate and locomote. 
Their focus was to create a strong, precise mechanism 
that focused less on snake-like movement and more on 
the difficulties of programming a mechanism with many 
degrees of freedom. The Caltech team considered ap¬ 
plications such as retrieval of satellites and search and 


rescue [20.7,8]. Burdick and his group also designed 
and built snake robots with the Jet Propulsion Labora¬ 
tory and the NEC Corporation with Nobuaki Takanashi. 
These were based on a three-dimensional joint proto¬ 
type built by Takanashi. This smaller snake was able to 
execute several movement gaits, and was intended to be 
used in search and rescue. 

Chirikjian moved on to Johns Hopkins University 
where he created a comprehensive research program 
on a broad class of hyper-redundant mechanisms, not 
limited to snake robots. One such nonsnake robot was 
the metamorphic shape-changing modular robot sys¬ 
tem comprising hexagonal elements. His group also 
developed snake robots based on binary actuators, 
which have two states, say fully elongated and con¬ 
tracted [20.9]. 

Snake and reconfigurable modular robot research 
also co-developed at Xerox Parc, and then at the Uni¬ 
versity of Pennsylvania under the direction of Mark 
Yim. Beginning with PolyPod [20.10] in 1994 as part 
of Mark Yim’s dissertation while he was at Stanford, 
snake configurations of modular robots were perform¬ 
ing many of the conventional snake gaits as well as 
some innovative slinky-like and rolling gaits [20.11]. 
This research continued after Yim’s move to Xerox’s 
Palo Alto Research with PolyBot in 2000, a more 
robust version of PolyPod with more abilities for sens¬ 
ing [20.12]. When Yim began Modlab at the University 
of Pennsylvania, PolyBot evolved into CKbot ([20.13], 
2009). CKbot can now reconfigure from an exploded 
position, run on four legs, walk on two, and perform 
most basic snake gaits [20.14, 15]. They also developed 
foamBot (2011), a robot that can create other robots, 
including snake robots, out of CKbot modules and hard¬ 
ening spray foam [20.16]. 

Hirose, Chirikjian, Burdick, and Yim had a strong 
influence on Howie Choset’s work at Carnegie Mel¬ 
lon University (CMU). Choset’s snake robot research 
began in the CMU Biorobotics Lab as an undergrad¬ 
uate research project which adapted Yim’s modular 
design specifically for snake robot use (Fig. 20.7). It¬ 
erating on this modular design, this lab has created 
a series of snake robots over 15 years [20.17, 18]. In 





Fig.20.3a-d Slim Slime (a) and the new generation of ACM-R snake robots: (b) ACM-R3; (c) ACM-R4; (d) ACM-R5 
(after [20.1]) 
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Fig. 20.4 Snakey, the 
snake robot showing 
also Joel Burdick, Greg 
Chirikjian, Howie Choset, 
Jimbo Ostrowski (af¬ 
ter [20.8]) 


addition to the mechanism, one of the major innova¬ 
tions of this work is gaits, the controllers that prescribe 
motion for the snake robots. These gaits generalized 
the gait motion, based on Hirose’s serpenoid curve and 
Chirikjian’s modal functions, to perform a large variety 
of behaviors including climbing up poles, swimming 
in a pond, maneuvering through pipes, and breaching 
through a fence [20.19]. Working with Robin Murphy’s 
group at Texas A&M, the snake robots have been fine 
tuned for urban search and rescue use. This created 
an ideal platform for infrastructure inspection such as 
networks of pipes in power plants. In 2012, Choset 
sent his robots to Egypt for archaeology in confined 
spaces [20.20]. 

In Norway, 0yvind Stavdahl in the Gemini Cen¬ 
tre for Advanced Robotics created several snake robots, 
many of which were in collaboration with the Norwe¬ 
gian University of Science and Technology. The first 
of these major research projects in snake robotics was 
the Anna Konda (2005-present), a firefighting snake 
concept. Based on hydraulic actuation, this robot is 
powered entirely by the pressure of a fire hose, and is 
designed to reach flames in areas that are either too dan¬ 
gerous or too small for firefighters to reach [20.21]. 

This team also created AIKO (2006-2010), a snake 
robot created to experiment with snake locomotion that 
could later be applied to Anna Konda [20.22]. Mostly 
focused on object-aided locomotion, AIKO evolved 


into KULKO (2010-present), a robot with a force¬ 
sensing body that uses its simple understanding of the 
obstacles it is in contact with in order to locomote 
more effectively. These two robots have produced sev¬ 
eral innovative movement algorithms for object aided 
locomotion that could be applied to Anna Konda as well 
as other non wheeled snake robots [20.23]. 

The snake robots described above rely on internal 
motions to propel themselves forward. Johann Boren- 
stein at the University of Michigan created the Omni- 
Tread (2004) robot [20.24] that propels itself with 
moving threads situated along side of the robot. Its 
strong joints are actuated by pneumatic bellows. The 
bellows can be proportionally controlled, allowing its 
body to be fully compliant while crossing rubble to have 
more of its body touching the ground, or strong enough 
to lift half of its body weight, which is necessary for 
ledges and gaps. The Omni-Tread excelled at moving 
over rubble [20.24]. 

David Anhalt and James McKenna at SAIC took 
the tread concept to the limit; they designed and built 
a toroidal skin drive, a drive mechanism that has re¬ 
ciprocating skin that consists of an elongated toroidal 
skin that covers the entire length of the robot, and 
a drive unit which propels it. The outer (tubular) layer 
of the skin slides axially from the head to the tail of 
the snake robot, and wraps inside itself over a cap¬ 
tured ring at the tail. The skin then recirculates from the 
tail to the head (through the center of the outer tubu¬ 
lar layer), and changes direction again (over a second 
captured ring at the head) to again become the outside 
layer. In this configuration, the skin forms a continu¬ 
ous toroidal loop. The SAIC team collaborated with 
Choset’s group to build a snake robot using this drive 
mechanism [20.25]. 

Finally, we describe some snake robots that use 
wheels to locomote. Hisashi Date, who heads a re¬ 
search team at the Robotics Lab for National Defense 
Academy of Japan, built a snake robot in 2007 to trace 
lines using lateral undulations and a line-tracing sensor 
arm. Date later developed in 2009 a mechanism that has 
passive wheels; it is very similar to Shigeo Hirose’s first 
ACM robots, except it is powered by hydraulics. Date’s 
robot also implements touch sensing much like being 
planned for Anna Konda [20.26]. 



Fig. 20.5 (a) Yim’s original Polypod 
robot (after [20.10]) and (b) current 
CK Robot in a wheeled snake 
configuration (after [20.13]) 
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Fig. 20.6 Choset’s Modsnake consisting of 16 DOFs (de¬ 
gree of freedom) each orthogonally placed with respect to 
each other (after [20. 17, 18]) 



Fig. 20.7 Anna Konda snake robot (after [20.21]) 


Another wheeled snake robot is HAUBOT (2009- 
present), developed by Akio Ishiguro at Tohoku Uni¬ 
versity. It is being used to study the autonomous de¬ 
centralized control mechanisms underlying the adaptive 
locomotion of animals. It uses real-time tunable rub¬ 
ber springs to actuate each joint. Each joint generates 


torque proportional to the difference between the joint 
angle and the target angle which is given by the ante¬ 
rior joint angle. By using just this method of simple 
sensing, the robot can locomote in two dimensions, 
without requiring a centralized control hub. More re¬ 
cently, HAUBOT has been successfully paired with 
head tracking software so that it can be driven intu¬ 
itively [20.27], 

At Kyoto University, Fumitoshi Matsuno has also 
developed several snake robots. Matsuno has made 
many innovations in the control of snakes with differ¬ 
ent modes of propulsion. In 2002, Matsuno and his 
lab built KOHGA, a treaded snake robot made for 
search and rescue. Its joints can be passive or ac¬ 
tive, allowing it to move through rough terrain and 
also lift parts of its body [20.28]. Another robot they 
have developed is a snake robot that propels itself 
using screw-inspired modules. Built in 2010, it uses 
many of the same control algorithms as KOHGA ex¬ 
cept besides moving forward, backward, and turning, it 
can also roll sideways perpendicular to its head direc¬ 
tion [20.29]. 

These labs together have created an incredible va¬ 
riety of snake-like robot locomotion. Snake robots can 
now traverse nonflat fields, climb poles, swim under¬ 
water, and even rebuild themselves. They can access 
incredibly small spaces and transition between swim¬ 
ming to slithering to climbing without having to be 
reconfigured. Advancing far beyond slithering across 
flat ground, these robots are nearly ready to begin tack¬ 
ling spaces in the field that no other robot can reach. 




Single drive motor 
in center segment 


Four pneumatic 
bellows per 2-DOF joint 


Drive shaft spine 
through all 
segments 


Fig.20.8a,b The Omni-Tread treaded 
snake robot (a) and an (b) illustration 
of its design (after [20.24]) 



Fig. 20.9 The SAIC skin drive mech¬ 
anism followed by the SAIC/CMU 
snake robot powered by the skin drive 
mechanism (after [20.25]) 
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Soon these robots may be found in the pipes of oil re- front line of search and rescue operations, fighting fires 

fineries, crawling through nuclear power plants, at the or exploring underwater environments. 


20.2 Continuum Robots - Short History 


The term continuum manipulator, first coined by 
Robinson and Davies in [20.30], which is equiva¬ 
lent to the continuous morphology manipulator defined 
in [20.31], indicates the snake/hyper-redundant robot 
concept taken to the extreme. In other words, concep¬ 
tually this category of robots have backbone structures 
taken to the limit, with their number of joints tending 
to infinity, but with their link lengths tending to zero, 
which is also the approach taken in Chirikjian and Bur¬ 
dick’s work on backbone curves. While this concept 
initially appears idealistic, complicated and impossible 
to implement, the limiting case (a smooth continuum 
curve, with the ability to bend at any point along the 
backbone) is quite easy to realize in hardware. 

Numerous continuum designs have been proposed 
since the early 1960s beginning with the Tensor 
Arm [20.32] developed based on the original Orm con¬ 
cept of Leifer and Scheinman [20.33]. With a flexible 
backbone bent by remotely driven tendons, the Ten¬ 
sor Arm anticipated numerous later designs. However - 
a key issue with later continuum robot designs until the 
1990s - coordination of the inputs to take advantage of 
the shapes achievable by the robot proved difficult. This 
lack of enabling underlying models for motion planning 
(now much less of an issue, see the following section) 
hindered the development of the field for many years. 

Nevertheless, some researchers, beginning with Shi- 
geo Hirose [20.2], continued to explore the problem 
of constructing continuum robots. Many of these ef¬ 
forts were inspired by an evolving knowledge of the 
morphology and functionality of biological continuum 
structures [20.34—36] particularly the trunks of ele¬ 
phants [20.37-39] and the arms of octopuses [20.40- 
42]. The key robot design issues revolve around how to 
actuate (bend and possibly extend/contract) the back¬ 
bone. Two basic design strategies have emerged [20.43, 
44]: (1) extrinsic actuation, where the actuators are sep¬ 
arate from the backbone structure, e.g., [20.45-49]; and 
(2) intrinsic actuation, where the actuators represent 
a fundamental part of the backbone, e.g., [20.50-53]. 

Extrinsically actuated designs have the advan¬ 
tage that by locating the actuators outside the robot 
workspace, the backbone itself can be simplified and 
streamlined. Tendons have proved a popular and gen¬ 
erally successful choice for extrinsically actuated con¬ 
tinuum designs [20.38,54,55]. Groups of tendons are 
terminated at specific points down the backbone, and 


combine to bend the backbone to produce a series 
of (usually constant curvature) sections. The tendons 
bend a backbone that is nominally straight (realized 
either via an elastic rod [20.56], the incorporation of 
springs [20.38], or pneumatic chambers [20.57,58]). 
A good example of the ability of this design strategy 
to produce a long, thin backbone with extrinsic actua¬ 
tion is given by NASA Johnson Space Center’s Tendril 
([20.49], Fig. 20.10), intended to penetrate through 
small orifices and explore in tight spaces for Space 
applications. 

In recent years, a completely different form of ex¬ 
trinsically actuated continuum robot, based on concen¬ 
tric tubes, has appeared. These concentric tube [20.59, 
60] (sometimes termed as active cannula [20.61]) 
robots have been primarily motivated by medical ap¬ 
plications [20.62]. Research has focused on applica¬ 
tion (e.g., as active endoscopes [20.63-66]) within the 
human body [20.67, 68], on various types of mini¬ 
mally invasive surgical procedures [20.69-71], and the 
associated development of steerable needle technolo¬ 
gies [20.72-74], 

Concentric tube robots, as the name suggests, form 
the robot backbone from several (typically two or three) 
hollow tubes, with tubes of smaller radius inserted 
through those of larger radius. Each smaller tube can 
be pushed through the tube immediately outside it to ex¬ 
tend the backbone length. The tubes can also be rotated, 
to give 2 n degrees of freedom for an n-tube backbone. 
The tubes are preformed to have a constant curva¬ 
ture, so can be extended and bent (albeit with amount 


Fig. 20.10 NASA’s Tendril, an extrinsically actuated con¬ 
tinuum manipulator (after [20.49]) 
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Fig. 20.11 Extrinsically actuated concentric tube continuum manip¬ 
ulators 



Fig. 20.12 The OC Robotics Snake-Arm robot - an ex¬ 
trinsically actuated continuum-like robot with segmented 
backbone (after [20.75]) 



Fig. 20.13 The intrinsically actuated Octarin pneumatic continuum 
manipulator and its development team (after [20.42]) 


of bending preset) in arbitrary directions (Fig. 20.11). 
This design arrangement is well-suited to the medi¬ 
cal applications which have motivated its development. 
Numerous prototypes have been developed in the last 
several years, and the results of initial trials are highly 
promising. This work is being pioneered by the groups 
of Pierre Dupont at Boston University [20.59, 73,74, 


76], Michael Zinn et al. at the University of Wiscon¬ 
sin [20.62,77], and Robert Webster, III et al., at Van¬ 
derbilt University [20.44, 61,67,78]. 

Note that the backbone for extrinsically actuated 
robots does not have to be continuous to produce the 
continuum effect [20.47,79,80]. A commercially suc¬ 
cessful group of extrinsically actuated continuum-like 
snake-arm manipulators are currently manufactured 
and retailed by OC Robotics (OCR) in the United 
Kingdom [20.75]. Remotely tendon-driven (designed 
by Rob Buckingham and Andy Graham of OCR), but 
with a snake-like segmented backbone (the number and 
form of segments can be changed) these snake-arms 
give a continuous backbone effect. OCR’s manipula¬ 
tors (Fig. 20.12) are currently the only general purpose 
continuum-like robot commercially available, and have 
been deployed in nuclear reactors and inside airframes, 
among other applications for largely noncontact opera¬ 
tions in tight, cluttered spaces. This ability to penetrate 
and slither through complex environments (mirroring 
NASA’s goals for Tendril) is one of the main motiva¬ 
tions and anticipated practical advantages of continuum 
robots. 

Intrinsically actuated continuum robots have their 
actuators within the continuum backbone. Indeed, the 
backbones of most intrinsically actuated continuum 
robots are largely formed by their actuators. Artifi¬ 
cial muscle technologies, notably pneumatic McKibben 
muscles, have proved most effective in intrinsically ac¬ 
tuated hardware realizations [20.50,81]. A particular 
advantage of intrinsic actuation is the ability to produce 
extensible backbones [20.43,44]. 

For example, Ian Walker’s group at Clemson led 
the development of the octopus-arm inspired Octarm 
instrinsically actuated extensible continuum manipula¬ 
tors ([20.42], Fig. 20.13, l<a>lMH=l'IIf). The backbone 
of the Octarm is formed from its actuators: sets of 
(3) McKibben muscles arranged in series to give three 
or four sections. Several generations of Octarm were 
tested in field trials, mounted on a Foster-Miller Talon 
mobile base [20.81] (l<s>MtEE!lEBi). The robots were 
shown to be able to grasp and manipulate objects over 
a wide range (order of magnitude) of sizes and weights, 
and of widely varying shapes and textures [20.82]. This 



Fig. 20.14 The intrinsically actuated 
bionic handling assistant by Festo 
(after [20.50]) 
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ability to adapt shape to perform whole-arm manip¬ 
ulation [20.83] with the backbone is another prime 
motivation for continuum robots [20.84]. 

Another good example of an intrinsically actuated 
continuum robot is the Bionic Handling Assistant re¬ 
cently developed by Festo [20.50] (Fig. 20.14). This 
continuum robot is unusual in that it is preshaped for 
its sections to be nonconstant curvature. The curvature 
is biologically inspired. 

Currently, another biologically inspired (once again 
by the octopus) large project, under EU funding, led by 
Darwin Caldwell, and featuring researchers in Greece, 
Israel, Italy, Switzerland, and the UK, is working to¬ 
ward creation of a European Octopus robot [20.40, 
85] (Fig. 20.15). This robot features multiple contin¬ 
uum limbs [20.85]. The European group has considered 
both intrinsic [20.86] and extrinsic actuation [20.40] 
for their limbs. 

Many snakelike/continuum robotic devices have 
found applications in the medical area. This includes 
very flexible steerable needles [20.87-91], and the 



Fig. 20.15 The European Octopus continuum-limbed 
robot (after [20.40]) (currently under development) 


concentric-tube robots mentioned earlier [20.73, 74,92- 
97], Other recent medical snake robots include novel 
articulated designs [20.98, 99], and stiff-continuum- 
filament cable-driven designs [20.100-103]. 


20.3 Snake-Like and Continuum Robot Modeling 


While continuum manipulators are seen to be the ex¬ 
treme case of snake-like or hyper-redundant structures, 
it was quickly realized that they represent a funda¬ 
mentally new class of manipulators [20.43,44,104]. 
A continuous flexible backbone features, at least in 
theory, an infinite number of degrees of freedom. As 
such, the traditional tools in robotics for modeling 
(a finite number of) serial rigid links no longer ap¬ 
ply. Additionally, it is obviously not possible to actuate 
an infinite number of degrees of freedom in practice. 
Continuum manipulator hardware universally features 
a finite number of actuators, applying forces/torques 
to the backbone at a fixed and preselected set of lo¬ 
cations. This causes significant complexity in their 
analysis. However, significant progress has been made, 
and understanding resulting from analysis of continuum 
kinematics has been applied, as discussed in the follow¬ 
ing section. 

Clearly, to correctly model continuum manipula¬ 
tors, models based on continuous backbones are re¬ 
quired. Interestingly, such models have also proved to 
be key theoretical resources in the motion planning 
of rigid-link hyper-redundant systems. While in the¬ 
ory discrete-link hyper-redundant manipulators can be 
modeled (and their planning based on) traditional ma¬ 
nipulator modeling techniques, it was quickly found 
that the computational complexity of such approaches 
made them difficult to implement, and the correspond¬ 


ing models hard to visualize. An alternative strategy 
(and a more successful one in practice) has been to 
adopt an approach based on continuum manipulator 
kinematics. In the following, we overview the recent 
development of continuum manipulator kinematics. We 
then discuss how the existence of both discrete-link and 
continuum models have enhanced motion planning for 
each category of hyper-redundant system. 

The lack of distinct links in continuum systems 
makes the standard robot manipulator modeling strat¬ 
egy of a finite number of coordinate frames (each fixed 
in one link) inappropriate for their modeling. Instead, 
the natural approach is to model continuum kinematics 
via a frame which evolves along a continuous back¬ 
bone, parameterized by arc length .v. Local motion of 
the backbone at point j is modeled in terms of the local 
frame at s. This strategy allows computation of for¬ 
ward kinematics, and construction of continuum Jaco- 
bians, analogous to those for rigid-link systems [20.3 1, 
105]. 

Numerous alternatives for selection of the back¬ 
bone frames have been proposed [20.31, 106-109]. In 
all such approaches, positions along the backbone curve 
can be expressed relative to the unit tangent, u(s), as 

t 

x{t ) = J [1 + £(i)]z<(5)di . 

0 
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In this formulation, e(s) is the local extensibility, or 
stretch, of the curve. If this quantity is equal to zero, 
then t becomes arc length. Otherwise, equal increments 
of t can correspond to different lengths along the curve, 
with a positive value of e(j) corresponding to a stretch, 
and negative values corresponding to a compression. 
While most continuum robots have a backbone that is 
essentially inextensible, the reason for introducing this 
variability is that allowing certain sections to stretch or 
compress affords greater freedom to hyper-redundant 
manipulators with discrete stages, making it easier to fit 
the physical robot to the curve. This fitting process takes 
as an input not only the relative positions of the distal 
and proximal ends of each stage, but also orientations. 
If the tangent is taken as one of the axes of a reference 
frame, a choice for the other two axes remains and can 
be made in a number of different ways. Once this choice 
is made, a reference frame 

g(t) = (R(t),x(t)) 

is established at each value of t and relative rigid-body 
displacements are computed for stages as 


shape, the total amount of motion of the evolving refer¬ 
ence frame required to traverse the curve is not minimal 
subject to end constraints and the requirement that one 
axis of the reference frame is the tangent. Variational 
calculus can be used to obtain the optimal framing 
of curves, as reviewed in [20.31] in the context of 
coordinate-dependent optimization, and in [ 20 . 1 10 ] us¬ 
ing a coordinate-free approach. 

Moreover, there are multiple approaches to defin¬ 
ing the evolution of the tangent, and hence the shape of 
the curve. Consider an inextensible curve whose shape 
has not yet been specified, and let the unit tangent to 
the curve be defined relative to an a priori-unknown 
arclength-dependent orientation as 

u(s) = R(s)e 3 . 

The goal is to find R(s) such that the positional con¬ 
straint 

L 

x(L) = / w(s)ds, 

0 


k(f;)] l °g(ti+i) 

= ([R(ti)] T R(t i+ i), [R(ti)] T (x(t i+ i) - x(ti))) . 

In the inextensible case where the curve parameter 
becomes arclength s one natural choice for defining 
the orientation R(t) is the well-known Serret-Frenet 
frame [20.31, 108,109] which evolves along the back¬ 
bone according to the following 

At/ As = Kn 
An/ ds = — Kt + r b 
Ab/ ds = —r n . 

In the above, the frame origin is given by x, where 
the unit tangent to the curve 1 = Ax/ As (denoted as t 
rather than n), and forms one of the frame axes. The 
other axes are defined as the normal ( t-n = 0) and 
the binormal b = t x n. The curvature k and torsion r 
dictate the shape of the curve. The axes of the Frenet- 
Serret frame provide an intuitive visualization of local 
movement: two possible dimensions of bending, cor¬ 
responding to rotations about the normal and binormal 
axes, and one of extension/contraction (present and 
controllable in several continuum robots), correspond¬ 
ing to translation along the tangent axis. 

The Frenet-Serret model has the advantage of be¬ 
ing a well-established means for modeling continuous 
spatial curves. The drawback of this formulation is that 
it is not optimal in the sense that for a given curve 


and the orientational constraints on R(L) are satisfied 
when R( 0) = /. Here, I is the identity matrix and the 
manipulator has length L. 

Treating .v as time, then corresponding to R(s) is 
an angular velocity a> = (R J AR/ d.?)'', where the V op¬ 
eration extracts the dual vector of a skew-symmetric 
matrix. We can then ask what R(s ) is optimal in the 
sense of minimizing a cost of the form 

L 

I = - J [ft>(j) — h] T S[<y(s) — b] ds . 

0 


When b = 0 and B is the moment of inertia of a cross 
section of the manipulator, this is akin to kinetic energy 
of rotation, and minimizing it provides an evolution of 
reference frames that minimally rotate. Variational cal¬ 
culus provides the conditions 


Bib + co x (Bco — b ) 


( -X T Re 2 s 
X J Rei 

V 0 


that can be integrated numerically and the rotation ma¬ 
trix R(s) is obtained from the angular velocities by 
integrating 


R = R 


- 3 
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The three-dimensional vector of Lagrange multipli¬ 
ers A and the a priori unspecified initial value of 
the three-dimensional vector co at 5 = 0 together pro¬ 
vide six degrees of freedom with which to match the 
end constraints on position and orientation. In this 
way, curves of fixed length that vary in shape as lit¬ 
tle as possible while meeting end constraints can be 
generated. 

Alternative frame representations in the contin¬ 
uum robot literature also have selected the axes of 
the frame to align with the controlled motion axes 
of particular hardware realizations [20.47, 107]. In ei¬ 
ther case, given a frame, the key problem is how 
to use the resulting model to plan motions. The ba¬ 
sic issue is that the continuum kinematics models 
in their basic form feature infinite degrees of free¬ 
dom (as necessary to model arbitrary spatial curves). 
However, hyper-redundant robots (discrete-joint or con¬ 
tinuum) can be controlled in only a finite number 
of ways, thus admitting a reduced set of physical 
solutions. Therefore, research in continuum manip¬ 
ulators has concentrated on how to constrain con¬ 
tinuum kinematics models to best represent robot 
hardware. 

The initial major breakthrough in motion planning 
for hyper-redundant arms was made in a landmark se¬ 
ries of publications by Chirikjian and Burdick [20.31, 
105,111] introducing continuum kinematics and us¬ 
ing them to approximate rigid-link hyper-redundant 
systems [20.105]. Basically, the philosophy is to use 
a (theoretical) curve to model the physical backbone 
of a hyper-redundant robot. Motion planning is per¬ 
formed for the curve, and the (discrete) robot back¬ 
bone then fit to the resulting (continuous) solution 
curve. This approach has proved quite effective, and 
the associated research introduced several key theo¬ 
retical concepts to the area. In particular, the use of 
a modal approach [20.105, 112] (restricting the classes 
of allowable solutions to shapes generated via simple 


linear combinations of modes), arose from this ap¬ 
proach to redundancy resolution for hyper-redundant 
manipulators. This concept can be viewed as a top- 
down approach of building a general model, and 
adapting it (via mode selection and curve-fitting) to 
hardware. 

In the planar case, the angle of the rotation ma¬ 
trix R(s) is simply the integral of the curvature of the 
curve. Expressing this curvature as a weighted sum of 
two functions, or modes, then restricts the manipula¬ 
tor to act as if it only has two degrees of freedom. As 
an example of this modal approach, a closed form so¬ 
lution can be obtained for inextensible manipulators of 
unit length by restricting the curvature to be a linear 
combination of cos2jrs and sin 2ns. If the coefficients 
in front of these functions are, respectively, 2na\ and 
2na 2 , then 

x ee = sm(a 2 )J 0 [(a; + a\) 3 j , 

y ee = cos(a 2 )Jo [(a) + a|)*] , 

where Jo is the zeroth-order Bessel function. 

And inverse kinematics can be achieved in closed 
form by computing 

Cl\ Uq ( Xee ) 

l 

- [Atan2 (x ee ,y ee )] 2 ^j 

a 2 tt 2 (x ee ) Atan2(x w ,, y^fi ). 

In this planar example the convention that the tangent 
to the curve at the base of the manipulator points along 
the y axis has been used. 

This closed-form solution has also been used to gen¬ 
erate closed-form solutions for spatial manipulators, as 
well as to define waves that travel along the manipula¬ 
tor for the purpose of obstacle avoidance, locomotion 
and manipulation. This is illustrated in Figs. 20.16- 
20.18 [20.31] with a variable-geometry truss superim¬ 
posed on the backbone curve. In the context of obstacle 
avoidance, the modal solution given above is scaled in 
order that the part of the manipulator outside of the 
obstacle field terminates at o position to allow entry, 
and the section inside of the obstacle field has curva¬ 
ture that is piecewise constant to maneuver around the 
obstacles. 

In the context of locomotion, the modal approach 
is used to ensure that the end conditions on a traveling 
wave match those of terrain. In the context of grasping 
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and manipulation, the modal approach is used in two 
ways. First, it defines the shape of a locomotion-like 
traveling wave that traverses the grasped object. This 
extends the reach of the manipulator around the object. 
Then, the bottom part of the manipulator contracts to 
form a shape defined by the modes while the part of the 
manipulator that is closest to the base yet in contact with 



Fig.20.16a-d Using the modal approach to position part 
of a manipulator at the entry of an obstacle field (af¬ 
ter [20.31]) 
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Fig. 20.17 Net motion to the left by wave propagation (af¬ 
ter [20.31]) 


the grasped object unwinds. The result is a combina¬ 
tion of waves and unwinding that produce net rotation 
of grasped objects without ever letting go, as described 
in [20.31, 113]. 

More recent research has concentrated on the notion 
dual to that of the previous section, using the physi¬ 
cal constraints imposed on the backbones of specific 
continuum robot hardware types to construct contin¬ 
uum kinematics. This can be viewed as a bottom-up 
approach, focused on particular hardware classes, with 
the main aim of sufficiently modeling the hardware to 
avoid the use of approximations in the motion plan¬ 
ning. Examples of this approach are given in [20.44, 
107] where the key constraint imposed on a general 
continuum kinematics model arises from the observa¬ 
tion that many continuum hardware implementations 
resolve into a finite number of constant curvature sec¬ 
tions. (This arises naturally from the imposition of 
a finite number of input force/torques on a stiff con¬ 
tinuum backbone). In [20.106], it is noted that the 
resulting constant curvature models can be viewed 
as a particular case of the general modal approach 
of [20.105], 

The approach shown [20.44, 107] involves several 
transformations [20.1 14], to transition the map between 
task and actuator space via a section space. As key 
part of this transformation, a conventional (theoretical) 
rigid-link model can be used to model the kinemat¬ 
ics of each section. Thus, rigid-link kinematics have 
proven key to resolving redundancy for continuum ma¬ 
nipulators, as continuum kinematics play a key part 
in redundancy resolution for rigid-link hyperredundant 
arms. 

Finally, a series of Jacobians are formed. The differ¬ 
ence here is that the continuum Jacobian is a function of 
locally meaningful variables (bending angle, curvature, 
and extension) defining section shape or the direct val¬ 
ues of actuators which determine these variables. Given 
the existence of continuum Jacobians, the redundancy 
is usually resolved as 

i = Jp'tE + — JeJe^ Io 

in the same way as for conventional redundant ma¬ 
nipulators (and with the same general corresponding 
advantages and issues), as discussed in Chap. 10. 

Recent efforts have extended the progress in con¬ 
tinuum robot kinematics to develop dynamic mod¬ 
els for continuum robots [20.84, 104], New models 
based on the Lagrangian [20.108,115] and Newton- 
Euler [20.116] formulations have been introduced. 
Progress is impeded by the complexity of the dynam¬ 
ics of continuous backbones with large deformations, 
and many models remain too complex for practical 
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Fig. 20.18 Net rotation of an object by wave initiation at 
base and wave propagation (after [20.31]) ► 

implementation. Hence, recent work has focused on im- 
plementable dynamic models based on simple, lumped- 
parameter models [20.1 17]. 

The new kinematic and dynamic models are in 
turn now enabling research in continuum robot motion 
planning [20.118, 119] and control [20.120, 121]. New 
models for force estimation [20.122], handling [20.76, 
123], and sensing [20. 124] using continuum robots have 
been introduced. Both model-based [20.54] and non¬ 
model based [20.125] controllers have been proposed. 
Controllers have been proposed in both configuration 
and task [20.54, 126] space. However, at this time, im¬ 
plementation of model-based controllers is rare. 





20.4 Modeling of Locomotion for Snake-Like 
and Continuum Mechanisms 


Snake and continuum robots have the ability to use their 
entire bodies to interact with the environment in order to 
produce purposeful motion [20.127-136]. Locomotion 
is a challenging motion planning and control problem 
because a planner/controller must contend with many 
degrees of freedom at once, even if they were somehow 
reduced with approaches like the backbone method, dis¬ 
cussed above. Many researchers achieve this by model¬ 
ing the snake robot’s motion with a gait , a cyclic motion 
in the internal degrees of freedom of a mechanism that 
produces net motion. It makes sense to implement gaits 
because all locomoting animals exhibit gaits: horses 
gallop, trot, and walk; snakes sidewind, laterally undu¬ 
late, and linearly progress. 

The benefit of the gait is that one can reduce a com¬ 
plex system down to a few set of parameters that 
describe the gait and then by cycling through the phase 
of the gait, a planner can achieve locomotion. So, if 
the gait were just a sinusoid, the user specifies ampli¬ 
tude, frequency, and phase and the mechanism is then 
fit to the sinusoid, which requires very little computa¬ 
tion as was seen with the backbone method. To achieve 
motion in three dimensions, we describe gaits for the 
snake robot with two sinusoidal waves lying in two mu¬ 
tually perpendicular planes, a horizontal plane parallel 
to the ground and a vertical plane perpendicular to the 
ground. Recall that some of the mechanisms, described 
above, consists of single degree of freedom joints, each 
placed orthogonally to its neighboring joints. For the 
sake of notation, we ascribe the even joints to modules 
that bend in the vertical plane and odd joints to modules 
that bend in the horizontal plane. So, if a vertical wave 


perpendicular to the ground is sent through the mech¬ 
anism, only the even modules participate, while the 
odd modules remain stationary. Likewise, only the odd 
modules participate in sending lateral waves parallel to 
the ground through the robot. Should the snake robot 
fall onto its side, then even modules simply become 
the lateral modules and the odd become the vertical 
modules. 

Given that the even modules are responsible for 
propagating vertical waves and the odd for propagating 
lateral waves, we use the terms offset vert i ca i, offseti atera i, 
amplitude ve rticab and amplitude groun d to denote the pa¬ 
rameters affecting the two orthogonal waves. Finally, 
we assume that both offset parameters are set to zero, 
unless otherwise noted. All of our differentiable gaits 
can be described by an angle of the n-th module at time t 
as 


angle («, t) 

offseteven + am plitude even * sin(0) , 
n = even, 

offset 0 dd + amplitude odd * sin(0 + 5) , 
n = odd, 

, (d0 d 9 

1 = — * n -1-* t 

\ d n df 


( 20 . 1 ) 


The offset is generally used to aim the robot when 
locomoting on the ground. For instance, when the snake 
is straight or climbing a pipe, the offset is zero. The am- 
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plitude terms describe the amplitudes of the mutually 
perpendicular waves. In most cases, the amplitude and 
speed of the robot are directly correlated and as both in¬ 
crease, the robot can better reach over large obstacles. 
However, larger amplitude comes at the cost of reduced 
stability. Intuitively, given a fixed number of modules, 
the fewer waves in the robot, the larger each wave can 
be. Thus, smaller values of Ad/ d/r allow more modules 
to participate in any one wave, which in turn allows for 
larger waves. 

The frequency of the sine wave is determined by 8 , 
which increases with time and along the length of 
the snake robot. The rate at which 8 changes is con¬ 
trolled by the Ad/ An and A8/ At terms. Note that the 
sine wave moving through the robot has a spatial and 
temporal component: 1) A8/ An determines the spatial 
component, meaning if we freeze time (fix t), it tells 
us what the robot as a whole looks like, and 2) Ad/ At 
determines the frequency of the temporal component, 


meaning that if we examine the sinusoidal trajectory 
of one individual actuator (fix n ), we determine how 
it moves. A zero value for Ad/ An causes identical an¬ 
gles on all of the modules on one axis, generating an arc 
along that axis, and a zero value for Ad/ At has the effect 
of freezing time. Finally, S is simply an offset to control 
the timing between the motions of the two orthogonal 
waves. These parameters can be modified to change the 
type of and nature of the gait performed by the robot. 

Current work includes combining the gait-equation- 
based approach with backbone curves. We are devel¬ 
oping techniques that take an animated sequence of 
backbone curves and infer the gait parameters from 
that sequence. The backbone curve approach is easy to 
visualize and therefore geometrically manipulate. The 
benefit of the gait approach is that it contains an an¬ 
alytic expression which permits evaluation, and hence 
optimization. With this new current approach, we can 
obtain the best of both worlds. 


20.5 Conclusion and Extensions to Related Areas 


The field of snake-like and continuum robots has blos¬ 
somed over the past 40 years. The work has taken many 
different directions including novel designs for locomo¬ 
tion, manipulation, and medical applications, as well as 
different avenues for applying the resulting modeling 
techniques including nonholonomic planning and the 
conformational flexibility and statistical mechanics of 
biological macromolecules. 

It is worth noting that other related works in 
nonrobotic fields and application areas have devel¬ 
oped either in parallel or as a consequence of the 
efforts mentioned before. For example, continuum 


devices were developed essentially outside the field 
of robotics in [20.137,138] from the perspective of 
mechanics. Continuum curve approaches applied to 
variable-geometry trusses were developed in [20.139- 
141], essentially in parallel with [20.9]. In recent 
work, wormlike robots [20.142] have been designed 
that execute peristaltic waves, motivated by biology 
akin to those described in [20.113]. Finally, analy¬ 
sis for understanding curves and workspace formula¬ 
tions for snake and continuum robots have been used 
to study deoxyribonucleic acid (DNA) [20.110, MS- 
MS], 
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available from http://handbookofrobotics.org/view-chapter/20/videodetails/169 
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available from http://handbookofrobotics.org/view-chapter/20/videodetails/171 
Modsnake sidewinding 

available from http://handbookofrobotics.org/view-chapter/20/videodetails/174 
CMU medical snake robot 

available from http://handbookofrobotics.org/view-chapter/20/videodetails/175 
Active compliant insertion 

available from http://handbookofrobotics.org/view-chapter/20/videodetails/244 
Automatic insertion implant calibration 

available from http://handbookofrobotics.org/view-chapter/20/videodetails/245 
IREP tagging spikes 

available from http://handbookofrobotics.org/view-chapter/20/videodetails/246 
RDP experimental results 

available from http://handbookofrobotics.org/view-chapter/20/videodetails/247 
Stenting deployment system 
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Bimanual dissection 
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Shoe decoration using concentric tube robot 

available from http://handbookofrobotics.org/view-chapter/20/videodetails/251 
Concentric tube robot at TEDMED 2010 

available from http://handbookofrobotics.org/view-chapter/20/videodetails/252 
Aiko obstacle-aided locomotion 

available from http://handbookofrobotics.org/view-chapter/20/videodetails/253 
Aiko sidewinding 
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Anna Konda - Motion 
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21. Actuators for Soft Robotics 


Although we do not know as yet how robots of 
the future will look like exactly, most of us are 
sure that they will not resemble the heavy, bulky, 
rigid machines dangerously moving around in old- 
fashioned industrial automation. There is a grow¬ 
ing consensus, in the research community as well 
as in expectations from the public, that robots of 
the next generation will be physically compliant 
and adaptable machines, closely interacting with 
humans and moving safely, smoothly and effi¬ 
ciently - in other terms, robots will be soft. 

This chapter discusses the design, modeling 
and control of actuators for the new generation 
of soft robots, which can replace conventional ac¬ 
tuators in applications where rigidity is not the 
first and foremost concern in performance. The 
chapter focuses on the technology, modeling, and 
control of lumped parameters of soft robotics, that 
is, systems of discrete, interconnected, and com¬ 
pliant elements. Distributed parameters, snake¬ 
like and continuum soft robotics, are presented 
in Chap. 20, while Chap. 23 discusses in detail the 
biomimetic motivations that are often behind soft 
robotics. 
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The chapter starts by analyzing the main motivations 
that are behind the development of actuators for soft 
robotics: these are mainly safety of operation, espe¬ 
cially in interaction with humans; resilience of the robot 
itself in interaction with harsh, unpredictable environ¬ 
ments, and performance enhancement in dynamic tasks. 

We then briefly survey the different principles and 
technologies that can be used to implement actuators for 
soft robotics. Using natural muscles as a term of com¬ 
parison, we consider different elements constituting an 
actuator, such as the prime movers, energy storage and 
dissipation elements, and their different possible ar¬ 
rangements. 

We then move to consider a class of actuators for 
soft robotics that includes many of the recent devel¬ 
opments in this field, that is, the class of variable 

21.1 Background 

21.1.1 Motivations 

The intrinsic properties of the biological muscles and 
their advantageous arrangement within the muscu¬ 
loskeletal system are a key aspect to the motion perfor¬ 
mance of many animal species. The flexibility, force- 
to-weight ratio, viscous damping, and fast reacting 
properties provide animals with an actuator having 
properties well suited for the tasks they are faced with 
day to day. Taking the human as an example, our mus¬ 
cles allow us to do very fine manipulation, precisely 
exerting forces and torques at the millinewton level, 
but also lift objects at a 10: 1 force-to-weight ratio. 
Through the interplay of these physical properties with 
the neural sensory-motor control, motion is accom¬ 
plished in a very energy-efficient, safe, and effective 
way. 

Despite the diverse range of actuation techniques 
developed over the past years, the lack of an actuator 
which can reach the functional performance of the bio¬ 
logical muscle and its neuro-mechanical control system 
still remains one of the most significant barriers for the 
development of machines, which can match the motion 
capabilities of natural systems. This observation has 
motivated many researchers in recent years to work to- 
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impedance actuators (VIA) and the subclass of variable 
stiffness actuators (VS As), and consider how a math¬ 
ematical model of these systems can be provided in 
general but precise terms. With the objective of help¬ 
ing the reader navigate the space of the many existing, 
and many more conceivable, designs we consider these 
actuators from a general point of view, discussing how 
their performance envelope and specifications could be 
given in a unified, comprehensive way. 

In the final part of the chapter, we consider the new 
problems and opportunities arising from the control of 
soft robots, including the need to estimate the variable 
physical compliance of the actuators, to plan its vari¬ 
ations to adapt to tasks and environments, and to do 
real-time control of soft robots exploiting their physi¬ 
cal properties. 


ward developing and exploiting actuation technologies 
for a new generation of soft robots, which can co-exist 
and co-operate with people and reach or even surpass 
their performance. 

21.1.2 Safety 

One of the main motivations for revisiting traditional 
robotic design is the concern about their safety: it 
is to be expected that interacting with robot assis¬ 
tants or collaborators should not constitute (or even 
just be perceived as) a higher injury risk to humans 
than the interaction with another cautious human. At¬ 
tention toward safe operation of robots near to, and 
even in contact with, humans is rather recent, although 
some of the pioneering contributions date back to the 
1990s [21.1], Robot-specific risk assessment methods 
have been studied in recent years [21.2,3], and are dis¬ 
cussed in Chap. 69. 

The inherent danger to humans of conventional 
arms can be mitigated by retro-fitting their design by 
increasing their sensorization (using, e.g., proximity- 
sensitive skins), or by increasing the energy-absorbing 
properties of protective layers (adding enough soft and 
compliant coverings or placing airbags around the arm). 
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Starting in the early 1990s, researchers have gradually 
abandoned the retro-fitting approach and started de¬ 
signing robots with interaction safety in mind from the 
beginning: notable examples are MIT’s WAM (whole 
arm manipulator) [21.4] and DLR’s LWR (light-weight 
robot) [21.5]. Arms in this class are primarily char¬ 
acterized by low inertia of their moving parts (links 
and motors) and backdrivability. Advanced sensing and 
control can realize a soft behavior via software. 

Another approach to increase the safety level of 
robot arms interacting with humans is soft robotics, 
that is, to intentionally introduce mechanical compli¬ 
ance and damping in design. By this researchers tend 
to replace the sensor-based computation of a behavior 
and its error-prone realization using active actuator con¬ 
trol, by its direct physical embodiment, as in the natural 
example. Having compliance and damping in the robot 
structure is by no means sufficient to ensure its safety, 
as it might indeed be even counterproductive for the 
elastic energy potentially stored. Just like a human arm, 
a soft robot arm will need intelligent control to make it 
behave softly as when caressing a baby, or strongly as 
when punching. 

21.1.3 Resilience 

Physical interaction of a robot with its environment can 
also be dangerous to the robot itself. Indeed, while for¬ 
tunately the number of accidents involving robots and 
humans is very limited, the number of times a robot 
is damaged because of impacts or force overexer¬ 
tion is rather large, and clearly the more so the less 
structured is the task and/or the environment. A typ¬ 
ical cause of breakdown for traditional robots is the 
failure of reduction gears (especially harmonic drives 
are quite sensitive to shock loads) and motor bear¬ 
ings, but large accelerations can easily damage sensors 
and electronics. Although seldom considered in the 
scientific literature, resilience to shocks is not only in¬ 
strumental to achieve viable applications of robots in 
everyday life, but would also be very useful in indus¬ 
trial environments, substantially enlarging the scope of 
applicability of robot technology. Soft robotics tech¬ 
nologies can provide solutions that are extremely ef¬ 
fective in absorbing shocks and reducing accelerations: 
soft materials can be used as coverings or even as 
structural elements in robot limbs, but the main tech¬ 
nological challenge remains with soft actuators and 
transmissions. 

21.1.4 Performance and Energy Efficiency 

A further reason to search for a new generation of actu¬ 
ators with controllable compliance is for their dynamic 


behavior, which can afford high-performance, natural¬ 
looking motion while being more energy-efficient than 
rigid robots. 

Indeed, adaptable compliance can be used to adjust 
the natural dynamics of the robot in tune with those 
of its environment, so that the intrinsic physical be¬ 
havior of the resulting system is close to the desired 
motion. In these circumstances, actuators would only 
have to inject and extract energy into and out of the sys¬ 
tem for small corrective actions, thus reducing energy 
consumption. 

The idea of embodying desirable dynamics in the 
physical properties of soft robots is of particular im¬ 
portance, for example, in walking/running robots and 
prostheses: indeed, biomechanical studies reveal that 
compliance adaptation is important for human walk¬ 
ing and running. Although most humanoid robots (such 
as HRP-4 [21.6,7] and ASIMO [21.8,9]) use rigid 
robotics technologies, soft actuators are more and more 
often adopted in humanoid robots: pneumatic muscles 
are used, for example, in [21.10] and [21.11], while 
several legged systems have appeared that use series- 
elastic actuators [21.12], What is probably the first pro¬ 
totype of fully physically compliant humanoid has been 
recently presented in [21.13]. 

The fact that natural systems change the compli¬ 
ance of their muscular system depending on the gait 
and environmental conditions, and even during the dif¬ 
ferent phases of the gait, seems to indicate the potential 
usefulness of variable impedance actuators (VIA) for 
locomotion - yet, because of the technical and theoret¬ 
ical challenges posed, only early prototypes of bipedal 
walking systems with variable stiffness actuation have 
been reported so far [21.14, 15], and no humanoid robot 
has been reported (to the best of our knowledge) mak¬ 
ing full use of VIAs. 

Besides their role in reducing energy consumption 
in cyclic motions, soft robotics can provide definite per¬ 
formance advantages in many other tasks. One notable 
class are those tasks where performance is measured 
in terms of peak velocity reached at the end of an ex¬ 
plosive-type motion, such as for example, in throwing, 
hitting, jumping, kicking, etc. Compliance in the robot 
structure indeed allows to store elastic energy in a build¬ 
up phase, and restitute it in the subsequent explosive 
phase. Furthermore, it permits us to have joints that, at 
least for a short time duration, move at a substantially 
higher velocity than the motor shaft [21.16, 17]. 

21.1.5 The Third Way 

of Robotic Control and A 

Novel actuators for soft robotics introduce one further 
interesting innovation for what concerns their control 
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philosophy and architecture. Indeed, it can be roughly 
but fairly said that most existing robotic systems are 
controlled according to either one of two dominant ar¬ 
chitectures: position-based controller and torque-based 
controller. Position-based controllers, which prevail in 
traditional industrial robotics, are rather closed con¬ 
troller architectures built around an inner servomech¬ 
anism loop, which basically provide the user with 
the ability to set joint position references, with very 
limited access to inner variables such as velocities, 
gains, or torques. Torque-controlled robots, on the other 
hand, provide a completely open control architecture, 
whereby the user can specify the motor currents or 
joint torques directly as a function of all accessible 
states. While the position-based control is still by far the 
most widely used architecture, because it requires less 
costly sensors, actuators and processors, and is more 
robust, serviceable, and fool-proof, researchers have 
always had a preference for the versatility and open¬ 
ness of the torque-based control. The introduction of 
relatively large physical compliance in soft actuators 
changes the picture considerably. This is particularly 
true in VIA systems, where the user is provided with 
the control of neither the joint positions nor torques di¬ 


21.2 Soft Robot Design 

In this section, we provide a concise presentation of the 
main elements and technologies that may contribute to 
the development of novel actuators for soft robotics. 

21.2.1 Natural Muscles 

For comparison and inspiration, it is natural to consider 
first the characteristics of natural muscles. A typical 
muscle consists of many thousands of muscle fibers 
working in parallel, organized in motor units, that is, 
one or several fibers with a motor neuron activating 
them. A single muscle fiber contains several myofibrils, 
made of sarcomere separated by Z disks. In a sarcomere, 
thin (actin) and thick (myosin) filaments are arranged in 
a hexagonal lattice, such that one thick filament inter¬ 
acts with six thin filaments. Cross-bridges linking actin 
and myosin pull toward sacromere center from both 
ends resulting in muscle shortening. Actin and myosin 
filaments slide against each other byrepeated attach¬ 
ment and detachment of cross-bridges, thus resulting in 
muscle contraction. 

As a consequence of their structure, natural muscles 
have several mechanical properties which differ from 
conventional actuators. Muscles (and tendons) have 
spring-like properties, with stiffness and damping in¬ 


rectly. Rather, one can specify the neutral equilibrium 
position of the joint, and the stiffness (and damping) 
by which the joint is attracted toward it, when exter¬ 
nal or inertial loads displace it from such equilibrium. 
To illustrate the difference, consider building a joint 
for a fully dynamic task, such as jumping on one leg: 
while position- or torque-control architectures would 
hardly be able to do the job (unless using very high-per¬ 
formance components), two relatively slow, low-cost 
servomotor units connected through springs in agonist- 
antagonist arrangement can already provide a very 
effective solution. 

It should be noted here that such VIA control 
model is very similar in philosophy to the X version 
of the equilibrium-point hypothesis in human motor 
control [21.18-20]. According to the equilibrium point 
hypothesis, movements arise as a consequence of shifts 
in the equilibrium of the motor system. This equi¬ 
librium is dependent upon the interaction of control 
signals, reflexes, muscle properties and loads, but it is 
under the control of central arm movement [21.21], In 
this sense, soft robotics also provide a very interesting 
arena for exchanging ideas and insights with the re¬ 
search community in motor neuroscience. 


creasing with activation, and are capable of storing and 
releasing energy, but also of dissipating it. Muscles are 
equipped with sensors which can provide stretch (e.g., 
spindles) and force (e.g., Golgi tendon organs) infor¬ 
mation to the nervous system. Muscle tension increases 
nonlinearly with activation, and - for fixed activation - 
depends on both length and velocity. Muscles are most 
often arranged in antagonistic pairs (such as the bicep 
and tricep in the elbow) or groups (such as in the shoul¬ 
der): this fact, together with the nonlinearity of their 
force-stretch relationship, is used to modulate the ef¬ 
fective impedance at the joints. 

21.2.2 Breaking Down Artificial Muscle 
Systems 

Even a nonexhaustive but reasonably broad analysis 
of technologies potentially contributing to the develop¬ 
ment of novel actuators for soft robotics could not fit 
this handbook chapter. However, we will attempt here 
to provide a simple categorization of the subject mat¬ 
ter, along whose lines a more complete survey could be 
organized. To this purpose, we will break down a soft 
robot system into a network of simpler elements, whose 
behavior is simpler and more easily described. 
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Like other systems, a soft actuator is comprised of 
storage elements that keep energy inside the same en¬ 
ergetic domain, and of transducers that allow a flow of 
energy from one domain to another. Transducers may 
dissipate energy via heat production (energy sinks), or 
can instead inject energy inside the system (energy 
sources). Sources can, in turn, be divided into flow 
sources, effort sources, and hybrid, or real sources. 
A flow source determines the amount of flow (linear 
or angular velocity, heat flow, volume or mass flow,...) 
going through it, without caring for the effort (force, 
torque, temperature, pressure, ...) needed to keep such 
flow. The converse holds for an effort source. A hybrid 
source, instead, is a device that offers a generic con¬ 
straint between the effort/flow pair. 

Storage and transducer elements can be connected 
in different arrangements, which may result in com¬ 
pletely different behaviors. VIA, an important class of 
actuators for soft robotics, have the characteristic that 
their overall behavior can be changed under active con¬ 
trol between that of an effort and a flow source. 

21.2.3 Sources for Soft Actuators 

Technologies used as sources for soft actuators vary 
broadly in nature and scope. In this section, we limit 
ourselves to provide a list with comments on the range 
of applicability, while we refer the reader to spe¬ 
cific literature for more detailed analysis: for instance, 
the author of [21.22] compares several active mate¬ 
rials in a volumetrically normalized fashion, includ¬ 
ing stress (force-generating capability), strain (stroke 
length), speed of actuation, and power and energy met¬ 
rics. 

21.2.4 Pneumatic and Hydraulic 

Pneumatic and hydraulic actuators [21.23,24] include 
classical tie-rod or rod-less cylinders and rotary motors. 
Pneumatic actuators are usually low-impedance effort 
sources, while hydraulics tend to be high-impedance 
sources, but compliant behaviors can be actively con¬ 
trolled, or obtained through combination with other 
elements in the actuator system arrangement. 

Of specific interest for soft robotics are the so- 
called pneumatic artificial muscles (PAMs), a name 
used to describe contractile linear-motion engines op¬ 
erated by gas pressure. The actuator’s core element is 
a flexible membrane or shell attached at both ends to 
fittings along which mechanical power is transferred 
to a load. As the membrane is inflated, it bulges out¬ 
ward and expands radially, and contracts axially thus 
exerting a pulling force on its load. The force and 


motion thus generated by this type of actuator is typ¬ 
ically unidirectional in tension (different from bellows, 
which work in compression). Pneumatic artificial mus¬ 
cles (PAM) became popular in robotics through the 
work of McKibben, who used braided sleeves around 
an elastic bladder as an orthotic actuator in the late 
1950s [21.25], and have been widely used since then 
in robotics in various forms [21.26-32]. 

PAMs have an intrinsically nonlinear characteris¬ 
tic (force decreases with contraction) and, due to gas 
compressibility, they exhibit a natural low impedance. 
As they act one way, two PAM are needed to generate 
bidirectional motion in an antagonistic arrangement. By 
a pressure differential on the two muscles, a net joint 
torque is applied. On the other hand, due to nonlinear¬ 
ity of the stretch/force characteristic, an increase in the 
common-mode pressure varies the joint stiffness, while 
leaving torque unaffected. The above characteristics 
of PAMs are somehow similar to those natural mus¬ 
cles, although many important differences exist. From 
a practical viewpoint, the main drawbacks of pneumatic 
muscles are the encumbrance, noise, and slow response 
related to use of pressurized gas, and their relatively 
limited energy density. 

21.2.5 Electromagnetic and Electrostatic 

Electromagnetic motors use Lorentz-type forces act¬ 
ing on electric charges moving in a magnetic field. 
Examples are solenoids, that is, electromagnetically in¬ 
ductive coils wound around a movable steel or iron 
slug (the armature) [21.33], and rotating or linear in¬ 
duction motors [21.34], Electromagnetic sources are 
typically effort sources, although often they have to 
be matched to the system impedance through gears, 
which may transform their behavior substantially. On 
the other hand, stepper motors, that is, brushless syn¬ 
chronous electric machines that divide a full rotation 
into a large number of steps, can be considered ide¬ 
ally as position sources. Stepper motors are mainly of 
three main types, that is, permanent magnet, hybrid syn¬ 
chronous, and variable reluctance steppers. 

Electrostatic actuators are based on the attraction 
and repulsion of electric charges [21.35]. They typ¬ 
ically require high-driving voltages at low currents 
(dual in a sense to electromagnetic motors). Linear 
electrostatic motors can be realized using comb-drive 
arrangements. While electrostatic drives at macroscopic 
scales are impractical due to large voltages, they have 
excellent potential at micro or nanoscale, where mi¬ 
croelectromechanical systems (MEMSs) with moving, 
charged plates are far easier to fabricate than coils and 
iron cores. 


Part B | 21.2 



Part B | 21.2 


504 Part B 


Design 


21.2.6 Piezoelectric 

The reverse piezoelectric effect , that is the production of 
stress and/or strain in a piezoelectric material when an 
electric field is applied can also be used for actuating 
robots. Most used inorganic materials for piezoelec¬ 
tric transducres consist of ceramics with perovskite 
or tungsten-bronze structures, such as barium titanate 
(BaTiOs), lead titanate (PbTiO,) and lead zirconate ti¬ 
tanate (PbtZr^Tii—JOs, 0 < x < 1), commonly known 
as lead zirconate titanate (PZT) [21.36], while the direct 
effect of piezoelectric polymers such as polyvinylidene 
fluoride (PVDF) is mostly used for sensors. 

Elementary piezoelectric actuators are best suited 
for microscale applications, where they can reach 
very high accuracy. Piezoelectric motors are some¬ 
times used in resonant mode with rectification mecha¬ 
nisms that transform oscillations in net displacements 
(e.g., traveling-wave, inch-worm and stick-slip step¬ 
ping motors). Multilayer (stacks) actuators, piling up 
several layers (usually thinner than 100 |im), are used 
to increase the actuator stroke, while retaining contin¬ 
uous and reversible operation. Stacked actuators are 
sometimes used within a mechanical amplifier struc¬ 
tures, which can reach millimeter strokes (Fig. 21.1) 
Amplified piezoelectric stack actuators are commer¬ 
cially available from Dynamic Structures and Materials 
(Franklin, TN, USA) and Cedrat (Meylan, France). 
In essence, the compliant mechanism accomplishes 
a force-displacement tradeoff from input to output at 
the cost of storing energy within the compliant mech¬ 
anism. This principle has been used in several mecha¬ 
nisms, such as in [21.37]. 

Ueda et al. [21.38] envision amplified piezoelectric 
stack actuators as modular units in a cellular actua¬ 
tor, drawing a parallel with muscles in that the overall 
actuation for a given joint is the sum of a number of dis¬ 
crete actionable units connected by compliant material. 
In the same way, muscles are composed of individual 
fibers grouped into motor units whose effects are cou¬ 
pled by elastic tendons. In order to function as modular, 
muscle-cell like units, the compliant mechanism in the 



Fig. 21.1 Example of an amplified piezo actuator with mul¬ 
tilayer stacks 


amplified piezoelectric stacks need to have very ag¬ 
gressive force-displacement tradeoffs while fitting into 
a compact volumetric envelope. This is accomplished 
by using a hierarchical set of strain amplifiers, where 
the output of each stage in the amplification is the input 
to the subsequent stage. Each stage fits inside the subse¬ 
quent stage; hence, this is referred to as a nested strain 
amplification mechanism. Schultz and Ueda [21.39, 
40] present a framework to describe and specify the 
geometry of a nested configuration that results in a fa¬ 
vorable tradeoff that meets design specifications. The 
work of Schultz and Ueda [21.41] draws yet another 
parallel with muscle, the idea that the central nervous 
system has authority over each motor unit in the mus¬ 
cle and activates each one in an on-off manner. This, 
in essence makes the control of muscles or muscle-like 
actuators a switching, rather than analog interface, and 
circumvents the problem of hysteresis inherent in posi¬ 
tioning piezoelectric material with an analog voltage. 
This actuator type also incorporates this idea of re¬ 
dundancy; a lost motor unit (dead piezoelectric stack) 
does not result in total failure, but rather a loss of 
performance. 

21.2.7 Electro- and Magneto-Strictive 

The electro- and magnetostrictive effects exhibited by 
some materials, that is, the tendency to change their 
shape under electric or magnetic fields, can also be 
exploited to build microscale actuators. Electrostric- 
tion differs from piezoelectricity because it is exhibited 
by all dielectric materials, and is proportional to the 
square of the polarization (instead of linearly). Re¬ 
versal of the electric field does not reverse the direc¬ 
tion of the deformation. Certain engineered ceramics, 
such as lead lanthanum zirconate titanate (PLZT) have 
high-electrostrictive constants and can produce strains 
of the order of 0.1% at field strengths of the or¬ 
der of 1 MV/m. Electrostrictive actuators are typically 
structured and used like piezoelectric actuators. Sim¬ 
ilarly, magnetostrictive materials deform under mag¬ 
netic fields and in proportion to their magnetostrictive 
coefficient [21.42]. One of the strongest effects is ex¬ 
hibited by Terfenol-D, with strain of the order of 
0.1% in a field of the order of lOOkA/m at room 
temperature. 

The pronounced thermal expansion of some mate¬ 
rials (e.g., doped single-crystal silicon or polysilicon) 
can be used to transform heat in mechanical energy and 
generate motion at micro scales. Temperature is typi¬ 
cally controlled by Joules’s effect, or through local heat 
injection, while mechanical amplification is often intro¬ 
duced with MEMS techniques (e.g., symmetric bent- 
beam or asymmetric bimorph structures). 
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21.2.8 Shape Memory Alloys and Polymers 

Macro-scale thermal actuators can otherwise be built 
using shape memory materials, which include al¬ 
loys (shape memory alloys, SMA) and polymers 
(shape memory polymers SMP). Alloys like copper- 
zinc-aluminum-nickel, copper-aluminium-nickel, and 
nickel-titanium (NiTi) alloys [21.43] exhibit shape 
memory when transitioning from martensitic to 
austenitic phase (one-way effect) and, under particular 
conditions, also on the opposite phase change (two-way 
effect). SMA can exhibit up to 8% recoverable strain, 
and can be formed in virtually any shape. Energy in¬ 
efficiency, slow response times, and large hysteresis are 
the main limitations. Torres-Jara et al. [21.44] have cre¬ 
ated an SMA muscle-like module in a convenient form 
factor by combining laser-cut SMA sheets with plas¬ 
tic hubs so that the hubs move away from one another 
when energized. These modules can be connected to¬ 
gether in series or parallel to make an overall actuator 
with the desired properties. 

SMPs are relatively new shape memory materials, 
which can be triggered by temperature but also by elec¬ 
tric or magnetic field, light or solution [21.45,46]. The 
shape change happens with transitions to glass phase, 
crystallization, or melting. While mechanical proper¬ 
ties of SMPs are not yet comparable to SMAs, SMPs 
have an enormous potential for their versatility, as they 
range from stable to biodegradable, from soft to hard, 
and from elastic to rigid polymers. 

21.2.9 Polymeric Actuators 

Shape memory polymers represent only an example of 
viscoelastic materials which can be used for actuation. 
Very intense research has been pursued in recent years 
on actuators based on several different phenomena ex¬ 
hibited by polymeric materials [21.47]. Electroactive 
polymers (EAPs) are divided into ionic EAPs (based 
on diffusion of ions and solvents) and electronic EAPs 
(based electronic charging of the material). Ionic EAPs 
include polyelectrolyte gels [21.45,46], ionic polymer 
metal composites (IPMC, such as Naftion by DuPont), 
conducting polymers (such as polypyrrole (PPy) and 
polyaniline (PANi)). Electronic EAPs include piezo¬ 
electric [21.48], electrostrictive, and dielectric poly¬ 
mers and elastomers (such as acrylic polymers and 
silicones [21.49]), and flexoelectric polymers, such as 
liquid crystal elastomers [21.50]. 

21.2.10 Carbon Nanotubes 

Carbon nanotubes (CNTs) have a nonpolymeric 
macromolecular structure consisting of a graphite 


monoatomic sheet rolled to form tubes whose length 
can be about 1000 times their diameter. CNT actuators 
can be realized by using sheets of single- or multiwalled 
nanotubes as electrolyte-filled electrodes of a super ca¬ 
pacitor [21.51], developing up to 0.75 MPa (cf. 0.3 MPa 
of the human muscle tissue) at low voltages, though 
still with limited stroke. A recent breakthrough in CNT 
actuators has been the use of highly ordered aerogel 
sheets, which allowed fabrication of actuators with gi¬ 
ant strokes (ca. 180% actuation along the width) with 
fast response (5 ms delay) [21.52], which is slightly bet¬ 
ter than the human muscle. The artificial CNT muscle 
is stronger than steel in one direction, and more flexible 
than rubber in the other two directions [21.53]. When 
large strokes will be obtained at low voltages, the poten¬ 
tial of CNT actuators for the design of artificial muscles 
is clearly enormous. 

21.2.11 Energy Storage 

Although energy storage elements in artificial muscles 
use a variety of materials and structures, due to the 
need of having mechanical energy readily available in 
dynamic phases, storage in the form of elastic energy 
prevails. This can be done by either pressurized gas sys¬ 
tems (mainly in connection with hydraulic or pneumatic 
actuation systems), of by deformation of solid elements, 
that is, springs. Materials for storing a large amount 
of elastic energy in a small volume, and with lim¬ 
ited losses, include special metal alloys (e.g., medium- 
and high-carbon steel, beryllium copper, and phosphor 
bronze). Composite materials, in the three main types 
available (polymer matrix composites (PMCs), metal 
matrix composites (MMCs) and ceramic matrix com¬ 
posites (CMCs)) offer a wider versatility in obtaining 
different stiffness, which depends not only on the con¬ 
stituent materials but also on the fiber design. Advanced 
needs of soft robotics applications motivate research in 
using sophisticated materials for energy storage, such 
as, for example, carbon nanotubes, elastomers, and su¬ 
perelastic materials. 

Mechanical energy can also be stored in kinetic 
form through the use of inertial elements, similar to 
the kinetic energy recovery system (KERS) used in 
FI racing cars. However, the mismatch with typical 
slow speeds of robotic joints introduces the need for 
gears. Conceptual design of CVT (continuous vari¬ 
able transmission)-based VIAs have been proposed, 
although research in this direction is still in its infancy. 

21.2.12 Energy Dissipation and Dampers 

Energy sinks, that is, dissipation systems and dampers, 
are another important element in the construction of 
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a soft robot actuator. Indeed, underdamped behaviors 
are both nonnatural looking and potentially dangerous. 
Damping can be obtained through real-time control, 
or directly in hardware via electrical shunting or with 
mechanical devices. The latter have been extensively 
applied to a diverse range of cases, from the control of 
excessive structural response to transient environmental 
disturbances in civil structures, to automotive suspen¬ 
sion systems, and more recently in robotics. 

Dampers are often classified as either passive, semi¬ 
active or active systems, depending on the amount of 
external power required to perform its functionality. 
Dampers can be realized using different operating prin¬ 
ciples, which we briefly summarize here. 

A friction damper (FD) is essentially composed of 
an actuator that applies a normal force on the output 
shaft, and generates friction as a consequence of rela¬ 
tive motion ([21.54] for modeling). The control of the 
compression force between the frictional surfaces con¬ 
trols friction and can simulate damping: this has been 
effectively used in robotics to implement variable phys¬ 
ical damping [21.55]. Drawbacks include the possibility 
for hysteresis and dead zone effects, due to static fric¬ 
tion [21.56]. 

Electrorheological (ER) and magnetorheological 
(MR) dampers are based on liquids whose physical 
behavior depends on the application of electric or mag¬ 
netic fields, respectively [21.56]. These fluids follow the 
Bingham models: after a yielding point, they behave as 
viscous liquids. The property that can be changed is the 
yield stress itself. The MR operating principle has been 
used to realize VDAs in vehicles [21.57] and also in 
robotics [21.58]. A more accurate model and a com¬ 
parison between MR and FD can be found in [21.54], 
where it is pointed out that MR dampers, like the FDs, 
present high hysteresis. 

Eddy current dampers (ECDs) are magnetic devices 
composed of a conductive material moving through 
a magnetic field. Eddy currents are induced and cre¬ 
ate a damping force that is proportional to the relative 
velocity between the material and the magnetic field. 
These devices can be realized with both permanent 
magnets and electromagnets. In both cases, there is 
the possibility of designing a device whose damping 
can be adjusted [21.59,60]. In one case, the damping 
coefficient can be controlled by varying the intensity 
of the magnetic field, in the other case, by modifying 
the geometry of the conductor, or the gap between the 
conductor and the magnets (the effectiveness is shown 
in [21.60]). Electromagnetic ECDs have the advantage 
of not requiring mobile parts, but have the disadvantage 
of consuming power for maintaining a fixed damping 
value. Being fluid-free and contact-free, ECD dampers 
are clean and wear-free. However, they typically ex¬ 


ert low damping torque at high velocity, thus requiring 
gearboxes for typical robotic applications. 

Fluidic dampers are probably the most widely used 
in robotics and general machinery and can be divided 
into two main categories: turbulent flow dampers (high 
Reynolds number) which produce a damping force pro¬ 
portional to the square of the relative speed; and lam¬ 
inar flow dampers (low Reynolds number), where the 
damping force is proportional to relative speed. Simple 
dampers are largely used in the automotive industry us¬ 
ing an orifice through which the viscous fluid flow is 
turbulent. Such device generates, at a given frequency, 
high damping for high amplitudes, but lower damping 
for lower amplitudes, and thus, has the drawback of pre¬ 
senting long lasting residual oscillations [21.56]. 

21.2.13 Artificial Muscles Arrangements 

The basic technologies and elements reviewed above 
can be arranged in many different ways to achieve ac¬ 
tuators for soft robotics. While a detailed taxonomy of 
arrangements is impossible here, the main distinction 
is between actuators that have a constant mechanical 
compliance characteristic, and those which can vary 
stiffness. In the first class are series elastic actuators 
(SEA), which have been developed since the 1990s 
for applications in humanoid locomotion and manip¬ 
ulation [21.61]. An SEA may include a conventional, 
rigid actuator connected to a compliant element, which 
is in turn connected on the other end to the moving link. 
The compliant element can exhibit a nonlinear compli¬ 
ance (load-deformation) characteristic, which however, 
is constant in time. SEA actuators, however, may use 
torque and position sensing at both ends of the elastic 
element and modify its natural mechanical impedance 
by virtue of active control of the motor. The impedance 
range to which an SEA can be effectively controlled is 
centered around the natural (passive) impedance of the 
elastic element and is limited in span by motor torque 
limits and overall control bandwidth. 

As opposed, VS A can vary their natural impedance 
directly at the physical level, so that active control 
can be superimposed and ultimately achieve larger 
ranges of effective impedance than SEA. The idea of 
varying the mechanical impedance of actuation comes 
directly from natural musculoskeletal systems, which 
often exhibit this feature. VSA systems can change in¬ 
dependently the equilibrium point of the moving body 
part, and the stiffness of the elastic force between the 
equilibrium point and the displaced body position (thus 
effectively implementing Feldman’s equilibrium point, 
or A-model for human movement control [21.62]). 
Accordingly, VS As always use two prime movers. 
A rough distinction can be made between VSA ar- 
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Fig. 21.2 Explicit stiffness 
variation and agonist- 
antagonist arrangements 



Fig.21.3a-c Three basic antagonist 
arrangements: (a) Simple, (b) cross- 
coupled and (c) bidirectional 


rangements where the motors are explicitly dedicated 
to either equilibrium or stiffness control, and arrange¬ 
ments where the roles are mixed (Fig. 21.2). Explicit 
stiffness variation arrangements include, for example, 
the MIA (mechanical impedance adjuster) [21.63]), the 
AMASC (actuator with mechanically adjustable series 
compliance) [21.64], the MACCEPA (mechanically ad¬ 
justable compliance and controllable equilibrium posi¬ 
tion actuator) [21.65]), and the variable stiffness joint 
(VS-Joint) [21.66]. Other VSA designs use instead 
agonist-antagonist (AA) arrangements, either in its 
simplest version, directly inspired to biological models, 
with two actuators pulling against each other [21.67, 
68 ], or in other variants such as the cross-coupled 
AA [21.69] and bidirectional AA [21.70] arrangements 
(Fig. 21.3) It is a matter of simple calculations [21.71] 
to appreciate that, in an AA arrangement, the elastic 
elements must exhibit a nonlinear characteristic in or¬ 
der for the joint stiffness to be variable. If each tendon 
has a quadratic force-deformation relationship, then 
the overall joint behaves as a linear spring with vari¬ 
able spring constant K. A potential disadvantage of 
AA arrangements using unilateral (e.g., tendon) cou¬ 
plings between the prime movers and the driven link 
is that, to be able to exert a maximum torque r at 
the link, two motors each capable of r are needed. To 
alleviate this problem, the cross-coupled and bidirec¬ 
tional design introduce the possibility for an agonist 
actuator to also work in favour of the antagonist when 
required. 

Recent work in soft robotics has introduced phys¬ 
ically variable damping on the rotor dynamics in con¬ 


junction with series elastic or VSAs [21.72]. Physically 
variable damping based on the ECD effect has been 
demonstrated in haptic interfaces [21.73]. Some au¬ 
thors introduced physical dampers in robotic design, 
both in the so-called series damping actuator configura¬ 
tion [21.58], or as a variable friction damper connected 
to an SEA [21.55], and as a variable diaphragm laminar 
damper module connected to a variable stiffness actua¬ 
tor [21.74], 

21.2.14 Specifications and Performance 

The increasing number of available actuators for soft 
robotics and the widening scope of applications are 
rapidly creating a need for a simple language for ap¬ 
plication developers and actuator providers to talk to¬ 
gether. Just like the perspective user of conventional 
hydraulic pistons or DC motors looks at spec sheets 
for a concise description of the performance guaran¬ 
teed by the product, a soft robot designer will need to 
find salient facts about different artificial muscles pre¬ 
sented in a clear and homogeneous way. The European 
project VIACTORS has dedicated a substantial effort to 
investigate what the crucial parameters of different de¬ 
signs are and has described a data sheet template that 
organizes these contents both from the designer and the 
user’s points of view [21.75-77]. The interested reader 
is referred to these publications for details: it is inter¬ 
esting to note here that, besides the characteristics that 
are shared with conventional actuators (stall and peak 
motor torque, max velocity, power, electrical and me¬ 
chanical interfaces, etc.), there are few data and curves 
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that are specific of soft actuators: for instance, the range reach from minimum to maximum stiffness, or vice 
of stiffness variability, and the settling time needed to versa. 


21.3 Modeling Actuators for Soft Robotics 


To approach the analysis and control of novel ac¬ 
tuators for soft robotics, it is necessary to describe 
their characteristics in terms of a mathematical model 
which accounts for their static and dynamic behaviors. 
We begin here by reviewing the familiar concept of 
mechanical impedance in linear time-invariant single 
degree of freedom (DOF) systems. Hence, in view of 
the fact that actuator softness control often introduces 
nonlinearity and time variance of the model, we pro¬ 
ceed to generalize the concept. Finally, we move from 
consideration of a single actuator to modeling of multi- 
DOF, articulated soft robots. 

21.3.1 The Mass-Spring-Damper Paradigm 

The simplest example of mechanical impedance is a lin¬ 
ear spring, that is, a sample of material which exhibits 
a proportional relation between applied force / and 
displacement at equilibrium y, y = cf. The proportion¬ 
ality constant c is the spring compliance. The inverse 
relation, that is, / = ky introduces the spring stiffness 
k = cW 1 . A spring can store an elastic energy V, whose 
first derivative with respect to the displacement y is the 
force 

, _ dV(y) 

y 

and the second derivative is the spring stiffness constant 

_ d 2 V(y) 
dy 2 ' 

The concepts of compliance and stiffness are gener¬ 
alized from static (equilibrium) to dynamic cases by 
introducing the notion of admittance and impedance, 
respectively. 

Consider a damper made of viscous material that 
opposes to displacements proportionally to how quick 
they are, that is, / = by, where b is the damping coef¬ 
ficient, and the inertial effects of accelerating an equiv¬ 
alent mass m of material under deformation, / = my. 
In most materials, the three effects occur at the same 
time, and we have that the force/displacement relation 
is described by an ordinary differential equation. In the 
simple case of a single-DOF mass-spring-damper sys¬ 
tem, we have 

( 21 . 1 ) 


that is, a relation between two functions of time, f(t) 
and y(t). Such a differential equation describes a dy¬ 
namical system. This can be regarded as a causal 
system if displacement (appearing with the highest-or- 
der derivative) is regarded as the effect (or output), 
and force as the cause (or input). Indeed, the knowl¬ 
edge of initial conditions on displacement, and the time 
course of force, determine all subsequent evolution of 
displacement. 

Insofar as the dynamical system (21.1) is linear 
and time-invariant, its input-output behavior is conve¬ 
niently studied via Laplace transforms from functions 
in the domain of time t to functions in the domain of the 
complex variable s. By denoting F(s), T(s), Vj(i) the 
transforms of/(f), y(t), and Vy(t ) = y(t), respectively, 
one has immediately 

F(s) = (ms 2 + bs + k)Y(s) 

= (ms 1 + bs + k)s~ 1 V y (s) . (21.2) 

The operator Z(s) := (ms 2 + bs + k)s~ l is called (me¬ 
chanical) impedance of the spring-damper-mass sys¬ 
tem. The reciprocal operator of impedance is called 
admittance A(s). Notice that the admittance operator is 
causal, while impedance is not. 

21.3.2 Nonlinear Mechanical Systems 

Real materials and systems deform under applied forces 
in more complicated ways than (21.1): in general, the 
dynamics between forces and deformations include 
nonlinearities, and can be of an order higher than 
two. How do we extend to these cases the concept of 
impedance (21.2), and specifically the notions of mass, 
stiffness and damping ? 

There is no easy clearcut solution to this problem. 
A straightforward approach is to find a linear, second- 
order model of type (21.1), which approximates well 
enough the real dynamics, and take its parameters to 
define those of the system. In this section we briefly 
review some of the implications of this approach. 

Consider the relation between a force / e F (F here 
is the set of allowed forces) applied at some point in 
a material sample, and the displacement y e Y mea¬ 
sured at equilibrium at the same point and along the 
same direction of the force (Y the set of allowed dis¬ 
placements). By ideally collecting all pairs (f(t),y(t )) 
corresponding in an infinite set of experiments (indexed 


/ = my + by + ky , 
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by t for convenience), one can describe the relation by 
its graph G C F xY. Notice that this relation does not 
need to be a function: for hysteretic materials, for ex¬ 
ample, there are more admissible values of one variable 
corresponding to the same value of the other (Fig. 21.4). 

Considering an analytic description of the graph G 
given by the locus G(f, y) = 0, at regular points of the 
graph a force function f(y) is defined for values of y in 
a neighborhood of yo- The stiffness associated with such 
a nonlinear force function can then be defined as 



_d f(y) _ f dG(f,y) \-' ZG(f,y) 

dy V 9 f ) dy 


Fig. 21.5 A link subject to an external load and actuated by 
two antagonistic actuators. Nomenclature refers to actuator 
models used in (21.3) and (21.5), respectively 


This can be generalized to a second-order, non¬ 
linear dynamic setting by considering the relation 
between forces, displacements, and first- and sec¬ 
ond-order derivatives of displacements and its graph 
G C F x 7 x DY x D 2 Y, comprised of 4-tuples d(f) : = 
( f(t),y(t),y(t),y(t)) corresponding to an idealized, infi¬ 
nite set of experiments. (If hysteretic nonlinearities are 
considered, it is necessary to describe the relation in 
an extended space where hysteresis state variables are 
introduced [21.78].) If G(f,y,y,y ) = 0 is an analytical 
description of the graph, and do is a regular point, then 
a force function f(y, y, y) is defined in a neighborhood 
of do- More generally, when the system includes time- 
varying parameters u(t) - such as, for example, a stiff¬ 
ening action in VSAs - one has G(f,y,y,y,u(t)) = 0 
and f(y,y,y,u). Following the linearization approach 
set out above, we can define admittance along a given 
(nominal) trajectory as the linear operator mapping 
small changes of the external force with respect to its 
nominal course, to changes in the resulting motion. 
To do this, consider the nonlinear ordinary differential 
equations (ODE) obtained by solving G(f , y, y, y, t) = 0 


Force 



at a regular point with respect to y as 

y = g(y,y,f,u) ■ 

and its state space form, with x e R 2 , x\ = y, x 2 = y, 
that is, 

d /xA _ / x 2 \ 

At \X 2 j \g(x 1 ,X 2 ,f,u)J 

For given initial conditions x(0) = xo and a given nom¬ 
inal course in time of the force f(t), let x(t ) be the 
nominal solution obtained. The first-order approxima¬ 
tion of the dynamics of the perturbed motion x(t ) = 
x(t)—x(t) corresponding to a change in force/(r) = 
f(t) —f(t) is the linear time-varying system 

U 0 1 h + (° 

\~K(t) ~P(t)J \/l(t ) 

For the given system and the given motion, hence, we 
can define instantaneous mass, stiffness, and damping 
coefficients as 

m(t) = /x 1 (?) ; 
b(t) = m(t)f(t ); 
k(t) = m(f)K(t) . 


Example 21.1 

Consider the link in Fig. 21.5, actuated by two antag¬ 
onistic actuators (with a role vaguely similar to that 
of the biceps and triceps muscles at the elbow), with 
a quadratic damping and subject to gravity and to an 
external torque load r e . The system dynamics are 



16 + f 9\6 1 — Tb + r t — mglsin 6 — r e = 0 . 


Fig. 21.4 Elastic hysteresis of a rubber-like material 
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Time (?) Time (7) 

Fig.21.6a,b Generalized stiffness (a) ( dashed is without gravity term) and generalized damping for the example with 
actuators as in (21.3), subject to a unit step in external torque at t = Is, and (b) with time-varying activation u\, (t) = u t (t) 
linearly increasing from 0 at t = 1 s to 1 at t = 15 s. Numerical values used in simulation: I = 0.05 Nm s 2 , mgl = 0.1 Nnr, 
fi = 1 Nm s 2 , a = 1 Nm, r max = 2Nm 


a) K (N m/rad) b) B (N m s/rad) 



Fig.21.7a,b Generalized stiffness (a) ( dashed is without gravity term) and generalized damping for the example with 
actuators as in (21.5), subject to a unit step in external torque at t= Is, and (b) with time-varying reference angle 
Ub(t) = u t {t) linearly decreasing from n/3 at t = 1 s to 0 at t = 15 s. Numerical values used in simulation as in Fig. 21.6, 
except for a = 0.3 Nm 


Assume first that the two actuators generate torques ac¬ 
cording to the model [21.67] 

Tb — (Anax dfdb) . (213) 

Tt — ( hnax tt 0i) U{ , 

where 0\, = (jr/2 + 6), 9 t = (n/2 — 6), r max is the max¬ 
imum isometric torque, u^, u t are the normalized con¬ 


traction parameters (0 < u < 1, u = u^, w t ), and a is 
a constant assumed to be equal for the two actuators. We 
easily obtain m = I for the generalized mass, b(6) = 
2/316? | for the generalized damping, and 

k(6, u) = a(wb + « t ) — mglcosf#) (21.4) 

for the generalized stiffness. In the latter expression, the 
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role of a gravity-induced term and that of a co-activation 
stiffness term are apparent. 

If a different actuator model is adopted, 
namely [21.79] 

r b = —a(Ob — iib) 2 , 

, ( 21 . 5 ) 

r t = —a(0 t ~ Ut) , 

where n b , u t are now interpreted as the rest lengths of 
the actuators, one has 

k(8 ) = 2ci!(jr — A b — A t ) — mglcos($) . 

The values of stiffness and damping for the two 
examples above, corresponding to time-varying values 
of the control parameters, are reported in Figs. 21.6 
and 21.7, respectively. ■ 


So far, we have considered only impedance for 
second-order dynamical systems - basically modeling 
actuators with a single moving element that domi¬ 
nates the system inertia. In practice, one may encounter 
higher order systems because of either (i) more complex 


actuator systems, or (ii) the connection of actuators and 
links in an articulated robot structure. The latter case 
will be discussed in Sect. 21.4. 

Examples of actuators with higher order dynam¬ 
ics are the arrangements in Fig. 21.3: if the agonist 
and antagonist prime movers as well as the driven 
link have nonnegligible inertia, the dynamics are at 
least of sixth order. If the system is linear, admit¬ 
tance can still be defined as the sixth-order transfer 
function from link torque to link displacement: how¬ 
ever, there is no natural way of associating an equiv¬ 
alent single mass, spring or damper coefficient. To 
recover these notions, an approach could be to ap¬ 
ply some form of model reduction of the higher order 
dynamics to second order [21.80]. However, model 
reduction is all but an obvious procedure, as the ap¬ 
proximate heavily depends on how the similarity of 
response between dynamics is measured. With nonlin¬ 
ear systems, the problem is clearly even more com¬ 
plicated. Overall, the topic is definitely worth further 
investigation, which should take into account what 
are the ultimate practical goals of defining impedance 
parameters. 


21.4 Modeling Soft Robots 

In this section, a brief introduction of dynamic mod¬ 
els for soft robots is given, starting with a quite general 
lumped-parameter formulation and successively intro¬ 
ducing several simplifying assumptions. 

A soft robot with lumped elasticities can be imag¬ 
ined as a set of directly actuated rigid bodies with 
configuration 0 e R" ! , connected to the indirectly ac¬ 
tuated rigid bodies (with configuration vector q e R") 
through viscoelastic forces, as exemplary sketched in 
Fig. 21.8. The entire configuration space of the system 
is denoted by x = (0,q), jcgR (, ‘+" i) . A quite gen¬ 
eral abstraction of a compliant robot, which can be 
used for the generic design of controllers, is given 
by 


M(x)x + c(x,x) + ^ ^ j +</(j:.x) = 

( 21 . 6 ) 

with M(x) being the inertia matrix, c(x,i) the Cori¬ 
olis and centrifugal vector, V the potential energy of 
the elastic element and of the gravity forces, t„£ M m 
the actuator generalized forces acting as control inputs, 
and text 6 R" the external torques acting on the robot as 
a disturbance. The most relevant property of this struc¬ 



ture is its underactuation, meaning that the system has 
less control inputs (m) than its configuration space di¬ 
mension (u + m). However, in contrast to other purely 
inertially coupled underactuated systems (for example 
multiple pendulums such as the Acrobot [21.81].), for 
the considered robots V(x) is [21.82] positive defi¬ 
nite, implying that a unique equilibrium point exists for 
each external torque with actuators in a fixed configu- 



Fig. 21.8 A soft robot can be imagined as a set of directly 
actuated rigid bodies with configuration 0, connected to 
the indirectly actuated rigid bodies (with configuration co¬ 
ordinates q through elastic forces). Note that in general, 
inertial couplings, not depicted in the figure, might exist as 
well 
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Stiffness (N m/rad) 



8-q (rad) 

Fig. 21.9 Typical shapes of elastic torque characteristics, 
denoted by nonlinear deflection dependency and a family 
of linear curves, parameterized by <r 


ration 0 = 0q and that the linearization of the system 
around an equilibrium point {x = * 0 , x = 0} is con¬ 
trollable. Typically, V(jc) = VgM + Vr (x ), that is, the 
potential function is the sum of a gravity potential 
Vg and an elastic potential V z . The elastic poten¬ 
tial function V T (jt) is a convex function (Fig. 21.9), 
increasing strongly enough to compensate for the desta¬ 
bilizing effects of Vg, such that y(jr) is convex as well. 
Furthermore, the system contains in general a dissi¬ 
pative friction force d(x, x) with x T d(x,x) <0. Note 
that between the directly and indirectly actuated states, 
there can exist also inertial couplings, as depicted in 
Fig. 21.10. Therefore, M(x) is in general a fully cou¬ 
pled matrix. 

While at first sight looking pretty general and sim¬ 
ple, model (21.6) is actually quite complex due to the 
full, parallel inertia and stiffness coupling. This can be 
seen by rewriting it in the less compact form 


(M(q,0) 

W(q.O) 


Q (q.0)\ 

B(q,0)) 



+ c(q.0,q, 0) +g(q.0) 


f dV(q,0) T \ 

3 q 

dV(q,0) J 

V d0 / 


+ d(q, 0) 



( 21 . 7 ) 


The above formulation emphasized the inertial cou¬ 
pling terms Q(q,0) as well as possible dependen¬ 
cies of the inertial properties on both motor and link 
configuration. 

Although (21.6) is the only formulation general 
enough to describe most soft robot designs with lumped 
elasticities encountered in the literature so far, it might 
pose substantial difficulties in controller design, that 
is, for the fact that it is not necessarily feedback-lin- 
earizable. (See, for example, the full dynamic model 



Fig. 21.10 In this example, the directly and the indirectly 
actuated states are both inertially and elastically coupled. 
This leads to a fully coupled inertia matrix in the dynamics 
equations 


of flexible joint robots from [21.83,84], which is not 
static state-linearizable.) Therefore, several simplifying 
assumptions are encountered in the literature, justified 
by particular robot design choices: 

• A1 : The system has no inertia coupling between 
the directly and the indirectly actuated stated, but 
only elastic couplings. This assumption includes 
the usual neglecting of motor gyroscopic effects 
for high-gear actuators [21.85] as well the neglec- 
tion of small inertial couplings usually appearing in 
the VIA mechanisms [21.86] and imposes some re¬ 
strictions related to mechanical design (Fig. 21.10 
for a counter-example). The assumption leads to 
a block-diagonal structure of the mass matrix. 

• A2: The gravity potential energy as well as the vari¬ 
ation of the mass matrix is only due to the link-side 
variables q. This implies that also the Coriolis and 
centrifugal torques are only depending on q and q. 

• A3: The dynamics of the stiffness adjusting mecha¬ 
nism is negligible; the system dynamics is mainly 
described by the states q and a directly actuated 
state vector 0 e R" of the same dimension as q, 
corresponding to one principal motor per joint. The 
remaining m — n motor coordinates of the stiffness 
adjusters are treated as parameters and are usually 
denoted by a. In this case, the elastic potential 
reduces to V z (q — 0 , <r). This is already a quite re¬ 
stricting assumption, applying mainly for SEA-type 
actuators with a dedicated stiffness adjuster, as typ¬ 
ically used in the design of large joints of arms and 
legs. The model is practically valid only if stiff¬ 
ness is changed off-line or at low rate. Classical 
antagonistic actuators including two equally sized 
motors, as used, for example, in some robotic hand 
designs [21.87] do not fulfill this assumption. 

• A4: A further particular choice of V z (q — 0, or) is 
that of a quadratic function in q — 0 such that the 
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joint torque is 


3 V(0,(r) T 

30 


K(c)0 , 


( 21 . 8 ) 


with 0 = q — 0. This implies that the stiff¬ 
ness of the actuator depends only on the stiff¬ 
ness adjuster and not on the deflection of the 
springs [21.82]. This assumption introduces ad¬ 
ditional simplifications in controller design, ap¬ 
plies however only to a restricted subclass of soft 
actuators. 


• The classical, fully antagonistic case with m = 2 n. 

• The quite popular n + 1 tendon design for robot 
fingers which leads to m = n + 1. More generally, 
2 n > m> n tendon designs are encountered. 

• Synergy hands with m < n, where each synergistic 
actuator acts over springs on several links. 

• Multi-articular muscles, as mostly existing in bio¬ 
logical systems. 

Under assumption A3 the size of 0 reduces to the 
size of q(m = n) and the model becomes 


The model, under assumptions Al, A2 reduces to 


/M (q) 

V 0 



dq 0)\ + /W)j 


( dV(q,0) T \ 

3 q 

3 V(q.0) T 

V 30 / 


+ rf(i) 



( 21 . 9 ) 


Note that the model above does not make any fur¬ 
ther specification regarding the dimensions m and n, 
thus covering a broad range of robots, including as par¬ 
ticular cases: 



with t 1 = 3V(0,<r)/30. It is then quite close to the 
flexible joint models treated in detail in Chap. 11. The 
similarity becomes even stronger, when the torque is de¬ 
scribed by (21.8) under assumption A4. Note that the 
dimension of 0 in (21.10) is half of the dimension of 
the same variable in (21.9) since the dynamics of the 
stiffness adjuster is neglected. 


21.5 Stiffness Estimation 

Strictly speaking, impedance is a differential operator 
relating physical quantities (forces and displacements). 
Rather than a direct measurement, the process of char¬ 
acterizing impedance of a system is therefore a process 
of dynamical system identification. 

In typical procedures for impedance measurement 
of material samples, such as, for example, organic 
tissue samples, the measured system is put into me¬ 
chanical contact with another mechanical system, the 
instrument. The resulting compound dynamical system 
is then subject to a dynamical excitation of different 
nature, and data are collected. Experiments are typi¬ 
cally conducted by touching the sample with a small, 
hard, vibrating probe. On the basis of measurements 
on the probe motion (displacement y, velocity y or ac¬ 
celeration y) and the applied force /, the mechanical 
impedance of the sample is inferred. A vast literature 
is available on the metrology of impedance of mate¬ 
rials, and the reader is referred there for a detailed 
survey [21.88]. 

The problem of measuring impedance in articulated 
systems is obviously more complex than for material 
samples, but is of great relevance to several fields - 


including notably human movement science and neu¬ 
roscience, who strive to understand how humans vary 
and control impedance in their movements. When the 
hand is slightly perturbed during arm movements, it 
tends to return to the undisturbed trajectory [21.20, 
89], as if the hand would be connected to a spring 
along the planned trajectory. This spring-like property 
stems mainly from muscle elasticity and the stretch re¬ 
flex, which produce - through the complex leverage 
of muscles at different limb configurations - a restor¬ 
ing force toward the undisturbed trajectory. Methods to 
characterize the mechanical impedance of human limbs 
have been the subject of extensive research [21.90-98]. 
Current protocols for identifying impedance in human 
motion typically recur to experiments in which per¬ 
turbations are purposefully injected in the system, and 
their effects are measured. In artificial robotic systems, 
impedance parameters can also be calculated on the 
basis of a precise description of the model (wherever 
this is available), or obtained through accurate calibra¬ 
tion procedures. In both natural and artificial systems, 
it would be of great utility to have a method which 
could identify impedance in real time, without per- 
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turbing the normal execution of the task, and robustly 
to inaccurate modeling and to time-varying parame¬ 
ters altering calibration results. Of particular interest 
in robotics is the application of such methods to VIAs 
for soft robotics. As discussed, these can be imple¬ 
mented in a variety of ways and arrangements, but all 
designs share a fundamentally unavoidable nonlinear 
behavior, which makes stiffness measurement a tough 
problem. 

According to the description of impedance as a dif¬ 
ferential operator, its characterization can be cast as 
an identification problem of a dynamical system. To 
date, the literature on this subject is not abundant, and 
it can be subdivided in two main approaches, that is, 
parametric and nonparametric identification. Paramet¬ 
ric approaches postulate that a model of the actuator 
generating variable stiffness is known in its structure, 
but that some of its parameters are unknown. Non¬ 
parametric approaches do not use such an assumption, 
and are thus more general; however, as we will see, 
nonmodel-based observers have the disadvantage that 
they cannot go to sleep after reaching convergence, and 
require persistent excitation to stay tuned to variable 
impedance. 

21.5.1 Observers for Linear Impedance 

To introduce the problem, let us first consider the case 
of constant linear impedance identification in the ele¬ 
mentary system ( 21 . 1 ) 

/ = my + by + ky , 

where we assume that m, b, k are unknown but con¬ 
stant, while y and / are available from position and 
force measurements at one of the ends of the compli¬ 
ant elements. 

Before embarking in building an algorithm to es¬ 
timate impedance, it is important to establish that the 
problem is well posed, that is, that a solution exists and 
is unique. To this purpose, consider an extended state 
vector 


The identification of the impedance parameters can thus 
be cast as a nonlinear observability problem, that is, 
from the knowledge of the input / and output y, es¬ 
timate the initial state z( 0 ), and in particular its three 
last components which completely determine the lin¬ 
ear impedance. The observability codistribution for this 


system is 

«(z) 
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It is easy to check that, for m, b, k > 0, the ob¬ 
servability codistribution has codimension zero for 
all states, except if Zi = Zi = 0. Hence, whenever 
the system is not in stationary equilibrium, the 
three linear constant impedance parameters can, in 
principle, be reconstructed from position and force 
measurements. 

To actually estimate the impedance in this case, dif¬ 
ferent methods can be adopted. These include standard 
off-line identification techniques (which exploit the lin¬ 
ear nature of the regressor for the unknown parameters), 
such as, for example, in [21.99], or on-line nonlinear 
state observers (e.g., extended Kalman filters) applied 
to system ( 21 . 11 ) [ 21 . 100 ], 

21.5.2 Variable Stiffness Observers 

Unfortunately, the generalization of the above straight¬ 
forward approach to the case when impedance is non¬ 
linear and/or time-varying is not trivial. To convince 
oneself, it is sufficient to consider the case that, instead 
of the linear spring term ky, a nonlinear and time-vary¬ 
ing spring function s(y,u(t)) is introduced in ( 21 . 1 ), 
that is 


z =(y y ~7n 

_b_ 

m 

h) 

and rewrite the dynamics ( 2 1 . T 

cal system 
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/. 


( 21 . 11 ) 


y = h(z) = zi . 


f = my + by + s(y,u) . ( 21 . 12 ) 

In the s(y, u(t )) term, u indicates an exogenous in¬ 
put that commands changes in stiffness. For instance, 
disregarding gravity in Example 21.1 earlier, u would 
represent the co-activation term Mb + u t in (21.4). 

21.5.3 Parametric Estimation 

A first approach to identify stiffness can be based on the 
assumption that the arrangement of the variable stiff¬ 
ness actuator is known (Fig. 21.2), and that a parametric 
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description of the spring function is available, for ex¬ 
ample, in terms of a finite basis expansion. In this case, 
parametric stiffness observers have been proposed that 
work on the motor side, that is, considering the dynam¬ 
ics of each motor as 

r =J9 + b6-f(4>), (21.IS) 


is on the link side, that is trying to estimate the joint 
stiffness looking only at the link dynamics, without as¬ 
suming any structure for the spring stiffness function. 
A nonparametric method was proposed in [21.104] to 
measure stiffness in a system such as (21.12), which 
can be easily derived from the idea of differentiating 
(21.12) once with respect to time 


where r is the torque of the motor, J and b are the motor 
inertia and damping, respectively, and <j> = 0 — q is the 
displacement angle with q being the link angle. These 
parameters and signals being known or measured, the 
problem is to estimate the stiffness a(cp) = as¬ 
suming it can be approximated by the truncated Taylor 
expansion 


Oi(<pi) : 


^ MiY 


In [21.101], a parametric stiffness observer is presented 
to solve this problem. The idea is to first obtain an esti¬ 
mation of the spring force function f(<p) by combining 
a modified kinematic Kalman filter for the estimation 
of the first derivative of q and a first-order filter. Then 
f((p) is approximated by a power series whose co¬ 
efficients are estimated by a least-squares algorithm. 
Finally, a is obtained by analytically differentiating 
the Taylor expansion. The method has been demon¬ 
strated to work well in simulation, and encouraging 
experimental results have been reported. The method is 
somewhat sensitive to the choice of several parameters, 
which require careful tuning. Subsequently, [21.102, 
103] proposed algorithms based on differential algebra 
and modulating functions, respectively, that improved 
convergence and robustness avoiding taking any deriva¬ 
tives and are very easy to tune. 

21.5.4 Nonparametric Estimation 


When no information on the structure of / or on the 
variable u(t) is available, the only possible approach 


/ = my + by + ay + s„u , 


where 

ds(y, u ) 

s u :=-• 

u 

The method is based on the continuous update of an 
estimate a of stiffness. Based on a best effort prediction 
for the link torque/ as 

/ = my + by + ay , 

it was shown that, under mild assumptions, the update 
law 

a = a(f-/)sgn(y) , (21.14) 

with a > 0 makes d(t) converge to the true stiffness 
value a(t) within a uniformly ultimately bounded error. 
The error bound is affected by errors in obtaining fil¬ 
tered approximations of derivatives of the position y(t) 
and of the applied force/(t), and in estimating mass and 
damping coefficients m and b, respectively. Conversely, 
the error can be reduced by increasing the gain factor a. 
As opposed to parametric methods, no assumptions are 
made on the function s(y, u), except that it is smooth 
and bounded in both arguments. The method has been 
experimentally demonstrated to work well, practically 
converging to the true stiffness value in all circum¬ 
stances except when stiffness is varied without moving 
the link. In this case (where it is indeed theoretically 
impossible to observe stiffness from the link side), the 
method of [21.104] simply stops updating the stiffness 
estimate, to resume as soon as a sufficiently rich dy¬ 
namic excitation is again applied to the link. 


21.6 Cartesian Stiffness Control 

21.6.1 Cartesian Impedance Control 

The Cartesian impedance control paradigm is one of 
the most widely used manipulation control approaches 
in modern robotics to handle contact situations [21.90, 
105,106]. It combines the advantages of compliant ma¬ 
nipulation with the ease and intuitiveness of a task 
description in Cartesian coordinates. Indeed, this con¬ 


trol technique has been one of the motivations for the 
research of variable-impedance-actuated robots. There¬ 
fore, it is a natural aim to analyze and transfer Cartesian 
impedance control concepts to soft robots. 

The goal addressed in this section is to realize a de¬ 
sired Cartesian impedance behavior at the tool center 
point (TCP) of a variable- impedance- actuated robot. 
Following a central paradigm of VIA, the mechanical 
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robot parameters need to be tuned such that the desired 
behavior is naturally achieved on a mechanism level to 
the largest extent possible. 

With the model from (21.10), the VIA joints shall 
be adjusted to realize a Cartesian stiffness. The stiff¬ 
ness behavior is described by a constant stiffness matrix 
Kc = 3//9* e R mX '" as the relation between the Carte¬ 
sian wrench / and the Cartesian displacement x. The n 
passive and adjustable joint stiffness components pro¬ 
vide the diagonal matrix Kj = dx/dq e R' iX '' with the 
joint torques r and the joint positions q. Decoupled joint 
compliance is assumed. The mapping from the Carte¬ 
sian stiffness space to the joint stiffness space is given 
by T : Kj = T(Kc). This transformation can be written 
as 


the stiffness in a wider range, and the elastic elements 
are capable of absorbing impacts and increase the en¬ 
ergy efficiency. By combining the two concepts, one 
can benefit from the individual advantages. The serial 
interconnection of the active stiffness K act j ve and the 
passive one K pass j ve results in an overall stiffness K res 
(Fig. 21.11) 

Kres' = K active + K passive ■ (21.18) 

To compute the active and passive stiffness com¬ 
ponents, a two-step optimization algorithm can be 
used [21.109]. It achieves first a passive stiffness as 
close as possible to the desired one and secondly de¬ 
signs the active stiffness to minimize the residual. 


_ 3r 3(J(<jf) T Kc Ax) 
Kj “3^- 3 q 


= J(<jf) T K c J(<z) 


3J(g) T K c Ax 

dq 


(21.15) 


J(q) = is the manipulator Jacobian, where f(q) 
is the forward kinematics mapping. Ax = Xd — x is the 
Cartesian position error between the desired and the 
actual position. The first part of (2 1.1 5) reflects the stiff¬ 
ness around the equilibrium point. The second part of 
(2 1.15 ) is due to the change of the Jacobian [21. 1 07]. 

The Cartesian stiffness at the equilibrium position 
(Ax = 0), resulting from a specific joint stiffness, can 
be obtained by solving the inverse problem of (21.15), 
Kc = ‘T~ l ( Kj). Note that at the equilibrium position 
the second term in (21.15) vanishes. Using compliance 
matrices C. = K~', it results from 

C c = J(<7)CjJ(<7) T (21.16) 


that 


K c = (JO/jK-'JO/) 7 ) -1 . (21.17) 

In [21.108], it was investigated that an arbitrary 
Cartesian stiffness matrix with three translational and 
three rotational stiffnesses (Kc € M 6x6 ) can hardly be 
reached by a 7-DOF robot, even exploiting the null 
space. Bounds are imposed by the technical realization, 
namely the diagonal joint stiffness matrix or mechanical 
VSA stiffness limits. Furthermore, the transformation 
of the joint stiffness into the Cartesian space is strongly 
determined by the robot kinematics and pose. 

To overcome these restrictions and improve the 
stiffness tracking performance, one can use the pas¬ 
sive stiffness in combination with an active impedance 
controller [21.109], which widely extends the achiev¬ 
able Cartesian stiffness range. Active control adapts 


21.6.2 Independent Position 
and Stiffness Control 

Variable stiffness robots are well suited for manip¬ 
ulation tasks involving the interaction with an envi¬ 
ronment. Such tasks are often specified by a robot 
pose trajectory and an associated stiffness trajectory. 
As VSA robots allow us to vary joint position and 
stiffness in a mechanical way, a direct approach is to 
adjust the respective position or stiffness inputs. De¬ 
pending upon the joint setup, these inputs are reflected 
by distinct actuators [21.66] or can be achieved by 
a simple coordinating transformation arising from the 
coupling of actuators [21.69]. The underlying idea is to 
embed and thereby achieve the desired robot behavior 
on a mechanical level. Additional control goals, espe¬ 
cially important in the presence of disturbances, are the 
minimization of deflections and the suppression of vi¬ 
brations. 

The solutions to achieve independent control of link 
motion and stiffness can be classified in two categories: 

• One class of controllers exploits the knowledge of 
system models. 

In [21.110] and [21.111], feedback linearization ap¬ 
proaches are used to transform the robot dynamics 



Fig. 21.11 Combination of active impedance controller 
{brown springs) with passive compliance (black springs) to 
achieve a desired Cartesian stiffness 
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into an equivalent model of simpler form. A de¬ 
coupled chain of integrators can be achieved, as 
long as system inversion is possible. The simple 
structure of the equivalent dynamics allows for si¬ 
multaneous decoupling and accurate tracking of 
motion and stiffness reference profiles. However, 
the abstraction of the robot dynamics hinders the 
implementation of performance criteria. 

Another model-based approach aims to achieve 
a reduced order model [21.112], Therefore, separate 
dynamics are identified in the robot system, namely 
the arm, the positioning actuators, and the stiffness 
actuators. The independence of these dynamics is 
shown by a singular perturbation analysis. A cas¬ 
caded control structure is based upon this analysis. 
The abstraction that this class of controllers pro¬ 
vides, allows for theoretical simplicity and design 
flexibility. On the other hand, robot performance 
and robustness in the presence of model uncer¬ 
tainty are not guaranteed. High-model accuracy and 
high derivatives of states are often required. Fur¬ 
thermore, to conform with the idea of embodiment 
additional effort is necessary, as the controllers of¬ 
ten introduce a different robot behavior. 

• The second class of controllers are energy-shaping- 
based controllers. 

One of the first controllers in this category was 
presented in [21.7]. The controller acts on motor po¬ 
sition and uses a transformation to independently 
control the joint position and stiffness. The con¬ 
troller is validated on a one-DOF VS A joint. 

An extension and a generalization have been pre¬ 
sented in [21.113], The control design formulation 
is valid for a quite general form of underactu¬ 
ated Euler-Lagrange systems including variable 
impedance robots. Herein, the controller action can 
be interpreted as shaping of the potential and kinetic 
robot energy ensuring system passivity. 

A general task is described as to control k indepen¬ 
dent output variables given by q = h (jc) to desired 
constant values q d e R*'. x e R" is the vector of 
generalized coordinates, where 11 = 2k is the usual 
case for VS A robots. Given the structure of VS A 
robots (21.10), a new variable q can be found. This 
is a collocated (directly actuated) variable, which 
is statically equivalent to the noncollocated (indi¬ 
rectly actuated) q. Using this collocated variable 
for a passive feedback ensures stability and can be 
interpreted as shaping the potential energy of the 
system. 

The variable q is achieved by solving the static so¬ 
lution of the link side equation 

T+g(4r) = 0 (21.19) 


for q. Except for very simple cases, this equation has 
to be solved numerically. Due to the convex nature, 
this is a fast and numerically robust task in practice. 
Consequently, the controller 

« = g(q) ~ j|(^)K p (^ - q d ) - K d 0 (21.20) 

stabilizes the desired position q d \g(q) is a feed for¬ 
ward term, compensating for the gravity. The use 
of q enables arbitrarily low controller gains even 
for large displacements from the equilibrium. J ( y 
is a Jacobian mapping the collocated variable on 
the statical equivalent q. Global asymptotic stabil¬ 
ity based on LaSalle’s invariance theorem can be 
shown. 

The approach can be extended in order to include 
also the feedback of torque and torque deriva¬ 
tive, with the effect of reducing the apparent ac¬ 
tuator inertia and friction. This allows improving 
the transient performance while remaining within 
the passivity framework. A torque feedback of the 
form 

r,„ = BBq 1 u + (I-BBgV , (21.21) 

with a new control input u leads to a new subsystem 
with scaled motor inertia 

B e £ + T = u. (21.22) 

The torque controller can therefore be interpreted as 
a scaling of the kinetic energy of the rotors in order 
to reduce the vibrations caused by the joint flexi¬ 
bility. The controlled system allows again a passive 
representation. 

The energy shaping approach provides excellent 
performance in the static case and for well damped 
systems. Some joints show low intrinsic damping to 
enable joint torque estimation and energy efficiency. 
In this case it is desirable to add additional damp¬ 
ing via control or to include a physical variable 
damping element. Several damping control struc¬ 
tures can be considered. A simple gain scheduling 
approach for a one-DOF system has been presented 
in [21.1 14]. Local linear sub problems are identified 
on which a linear quadratic regulator (LQR) state 
feedback controller is designed. The nonlinearity of 
the robot dynamics requires to adapt the controller 
poles dependent on the system state, which is espe¬ 
cially hard for the multi-joint robots. 

A physically motivated state feedback control ap¬ 
proach for multi-DOF VS A robots has been pre¬ 
sented in [21. 11 5] using an eigenvalue-based modal 
decomposition. 
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21.7 Periodic Motion Control 

Robotic tasks, such as walking, jumping, running, and 
some pick-and-place motions, are essentially periodic 
motions. During these tasks, the natural oscillatory dy¬ 
namics of compliant actuators can be ideally exploited 
to increase performance and energy efficiency. It is 
well known that fast running animals store and recover 
a substantial amount of energy during a running gait pe¬ 
riod in their tendons and muscles. Also human running 
strongly relies on elastic energy storage. 

Exciting periodic motions in a highly nonlinear 
multibody system by using nonlinear elastic actuators 
is, however, a nontrivial task. While modal oscillations 
and their control are very well understood for linear sys¬ 
tems, the nonlinear generalization is a very active topic 
of research. Since we consider that the ability to per¬ 
form (quasi) periodic motions with high velocity and 
force at low weight will be one of the major benefits 
of soft robots, we dedicate an extensive section to this 
topic. 

The topic of exploiting intrinsic elasticity for pe¬ 
riodic motions was addressed in robotics by [21.1 16— 
120] among others. Two basic approaches can be dis¬ 
tinguished: 

• Tracking an arbitrary desired periodic trajectory, or 
enforcing an arbitrary oscillatory closed-loop be¬ 
havior, by control while simultaneously adjusting 
the compliance of the actuator such that the control 
effort is minimal. 

• Exciting and sustaining intrinsic oscillatory dynam¬ 
ics of the compliantly actuated robotic system by 
control. Completion of a given task implies that this 
intrinsic oscillation modes correspond to the task by 
the initial mechanical design and by online adjust¬ 
ment of the robot configuration and of its intrinsic 
actuator stiffness. 

Depending on the design of the compliantly actu¬ 
ated robotic system, the above control approaches have 
to cope with underactuation, model nonlinearities, and 
multiple-DOF structures. 

21.7.1 Periodic Motion Tracking 
and Stiffness Adaptation 

The idea of tracking a periodic trajectory in the link- 
side variables q with minimal torque is inspired by 
classical rigid-robot control. The approach [21.117] 
considers, for example, robotic systems with elastic el¬ 
ements in parallel to the main actuators 

M(q)q+ (C(q,q) + T>)q+g(q) + K(q-6) = r . 

(21.23) 


Since 6 is assumed to be constant here, the model re¬ 
duces in this case to a fully actuated robot dynamics 
with parallel elasticity, being thus simpler than (21.6). 
The objectives of the controller are: 

1. Tracking a periodic trajectory q d (t + T) = q d (t) 
with period T such that q(t) —» q d {t ) 

2. Minimizing the actuator torque r by optimizing the 
stiffness K. 

Considering the actuator torque required to generate 
the desired motion 

r d = M(<jr d )ij d + [C( 9 d , q d ) + D] q d 
+ g(q d ) + K(q d -0) , 

a stiffness matrix K opt is defined to be optimal if it min¬ 
imizes the cost function 

iT+T 

J(K) = J r d (K, ?) T r d (K, t)df . (21.24) 

iT 

In the case of the single-DOF system 

m'q + dq + kq= r, (21.25) 

with constant system parameters m, d, k > 0, the 
desired, sinusoidal motion q d (t) = a sin cot obviously 
minimizes the cost function (21.24), when the stiffness 
is k opt = mar. With the optimal stiffness, the sinusoidal 
motion with an arbitrary amplitude a > 0 corresponds 
to a resonance motion, since the angular frequency co = 
y/kopt/ni of the desired trajectory corresponds to the 
resonance frequency of the system (21.25). 

The method proposed in [21.117] extends this sim¬ 
ple concept of resonance to the multi-DOF system 

(21.23) . Various online or offline optimization algo¬ 
rithms can be defined to minimize the cost function 

(21.24) . For example, [21.117] uses the parameter ad¬ 
justment law 

k = rdiag(</ — 6) [§ +K B s(?)] 

to adapt the stiffness K = diag(k). The controller gain 
matrix K b is positive definite and diagonal and the 
adaptation gain T is a positive definite matrix. The 
rationale behind this law is to increase the stiffness 
whenever the elastic deflection (0 —q) has the oppo¬ 
site sign to the tracking error. In this case, a stiffness 
increase reduces the tracking error. The stiffness is re¬ 
duced whenever the elastic torque increases the tracking 
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error. In particular, the control error q = q — q& is satu¬ 
rated by functions Ji(r?i) satisfying certain boundedness 
constraints. These saturated functions are important for 
the stability analysis in [21.117], 

The paper [21.121] considers dynamic systems with 
elastic elements either in parallel with the main actuator 
as in (21.23), or in series as in 

f(q,q,q,t) = -K(q-6 ) (21.26) 

B0 =K(q-6) + T , (21.27) 

where B represents the constant inertia matrix of 
the main motor part of the model. The objectives 
of [21. 11 7] were extended to compute the optimal value 
of a constant stiffness for minimizing energy consump¬ 
tion in a periodic task for series configuration, and both 
the optimal stiffness and spring preload values for the 
parallel case. 

Assuming that K = diag(Xj) and B = diag(Bj) (the 
second assumption is required in the serial case) it is 
shown how to derive expressions of the optimal ac¬ 
tuation parameters for minimizing energy-related cost 
functions such as (21.24) and 

iT+T 

J(K) = I io d (K, f) T iu d (K, t) dt. (21.28) 

iT 


where t) = r d (K, t)* 0 d (K, t), with ■* denot¬ 

ing element-wise multiplication. Rather surprisingly, 
it is shown in [21.121] that it is possible to save 
lengthy iterations and simulations by computing ana¬ 
lytical expressions for the optimal stiffness and preload 
as integral functions of any given desired joint trajecto¬ 
ries: for example, the series stiffness that minimizes the 
cost (21.24) is given by the formula 


K* = 


f^ +T Bjfj(q d ,q d ,q d ,t)dt 


WT+T 

JiT 


u\q d, <7<i. </d. q.i- q,\. t))dt 

(21.29) 


21.7.2 Controlled Oscillatory Dynamics 

The basic idea to generate periodic motions is here 
to control the output of the joint such that it be¬ 
haves like a desired, oscillatory dynamic system rather 
than imposing a predefined, time-dependent trajec¬ 
tory [21.118]. The resulting motion will be therefore 
depending on initial conditions and possible perturba¬ 
tions, but will eventually converge to the periodic orbit. 
For simplicity, here we describe the concept for the 


model of a single variable stiffness actuated joint, satis¬ 
fying 


m'q + 


dVr (O.q) 
dq 


= 0 . 


(21.30) 


The output position of the joint is denoted gel and 
the velocities 9 = u e R 2 of the two actuators of the 
VIA joint are assumed as control inputs. Note that this 
model corresponds to the first row in (21.9) with m = 2 
and n = 1. Let us assume a generic family of desired 
oscillatory dynamics 


q + a(q) = 0, 


(21.31) 


with a(q) chosen such that a periodic orbit results. In 
particular, a{q) can be the derivative of a desired po¬ 
tential function V&(q). The goal is to control the system 
(21.30) such that 


,, a , , , 1 dV z (0,q) n 

h(9 , q) = a(q )--- = 0 

m dq 


(21.32) 


holds. With initial conditions (<j(0), c/(0), $i(0), 
02(0)) G h( 0) —1 the constraint (21.32) can be fulfilled, 
if the control u satisfies the constraint on velocity level, 
that is, h(9, q) = 0. Since 9 = u is the control input, 
this implies 


h(9, q) = —A (9, q)u + b(6, q, q) = 0 , 


with 


, 19 2 VA9,q) 

A (9,q) := -—-— 

m dqdO 


b(9,q, q) := 


(da(q) 1 d 2 V z (9,q)' 


V dq 


dq 2 


I q ■ 


(21.33) 

(21.34) 

(21.35) 


Using a generalized right inverse A^ of the matrix A, 
the control 


u = A(9, q)^b(9, q, q) , 


(21.36) 


fulfills the velocity constraint (21.33). 

The solution of (21.33) with regard to u is not 
unique, as the problem is under-constrained. By choos¬ 
ing a particular generalized pseudoinverse of A of the 
form A ! = B~ 1 A t (AB~ 1 A t ) _1 , the solution (21.36) 
minimizes the norm ||u||^ with respect to a certain met¬ 
ric tensor B. If, for example, B is the matrix containing 
the motor inertias then the kinetic energy of the actua¬ 
tors is minimized. The under-constrained situation can 
be interpreted such that the desired motion might be 
achieved by both adjusting the stiffness and the motor 
position of the actuators. Choosing a certain metric B 
implicitly defines a weighting of the two components. 
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21.7.3 Exciting Intrinsic Oscillatory Dynamics design 


Exciting the intrinsic oscillatory behavior of mechani¬ 
cal systems, that consist of kinetic and potential energy 
storages, is inspired by the concept of resonance from 
linear oscillation theory. The idea is to move the actu¬ 
ators of the compliant system such that the trajectory 
of link variables is periodic and thereby the energy in¬ 
put due to the motor motion ideally counteracts only 
the loss of dissipated energy. Instead of tracking an 
arbitrary trajectory, here we are interested in exciting 
intrinsic resonant motions of the system. 

Consider the dynamical system of the form (21.10). 
The objectives in this section are finding a control r,„ 
such that for a certain period T, the trajectories of link 
variables fulfill q(t + T) = q(t), and the intrinsic oscil¬ 
latory dynamics are exploited to the maximally possible 
extent. Assuming that the motion q(t) should take place 
on one of several one-dimensional manifolds Zj C R" 
(referred to as oscillation modes), the problem can be 
further refined: 

• Cl: Given the mappingz = z(q). If Zj represents the 
coordinate of the desired oscillation mode, then the 
control must ensure that oscillations excited in all 
other modes decay, that is, for i j, zi —> 0. 

• C2: The motion of zj has to be controlled to sustain 
a periodic motion with a desired oscillation ampli¬ 
tude. 


21.7.4 Model-Based Approach 

To derive a coordinate transformation z = z(q) and to 
fulfill the above conditions, the free motion (r ex t = 0) 
link-side dynamics of (21.10) can be modified by first 
considering 0 as control input, that is, 

0 d = q* + Kp‘(Cfar, q)q + g(q) - K D q + y) , 

(21.37) 

with y being a decoupling term to be specified later. 
This leads to the closed-loop dynamics 

M(#)<j + Kp(</ —</*) + K D <jr = 0 . (21.38) 

In order to excite oscillations which are quite close 
to the initial oscillation modes, consider coordinates zj 
which correspond to the instantaneous modal coordi¬ 
nates z = Q (q)-\q-q*). Q (q) is defined on the ba¬ 
sis of double diagonalization of M and K such that 
Q(9) _T Q(?) _I = Mfar) and Q(^) T diag(A(qr))Q(^r) —1 
= K P , where X(q) e ®>o are generalized eigenvalues, 
as proposed in [21.120]. By considering a damping 


K d = 2Q(<jr) —T diag Q(?)"' . 

and by choosing y as 

Y = [MfarJQfar) + A D Q(?)]z + 2M(q)Q(q)z . 

(21.39) 


the resulting closed-loop dynamics 


Zi + 2fj V A i(q)zi + A i(q)zi = 0 


fulfills the condition Cl, if for i ^ j, > 0. Then, the 
nonlinear damping term 

§ = - + \ JW> k nl H <?> 40 - #d] 

4A i(q)-2 2V 

produces a periodic motion Zj(t+T) = Zj (f). In partic¬ 
ular, the resulting limit cycle corresponds to a level H d 
of the energy function H(q,Zj,Zj) = 5 ( 1 ^)^+^) as 
proposed in [21.122]. 

Note that applying the control (21.37) to the sys¬ 
tem (21.10) requires a controller that ensures a track¬ 
ing 0 (f) —> 0 d (t) of the desired motor position 0 d (t). 

21.7.5 Model-Free Approach 


Exciting oscillations in multi-DOF mechanical systems 
without changing at all the original inertia and stiff¬ 
ness properties of the plant requires either an ideally 
separated excitation on a certain oscillation mode or 
intrinsic physical damping properties that ensure the 
decay of oscillations excited in the other modes cf. 
condition Cl. Note that in most cases fully decou¬ 
pled modes do not even exist. On the basis of the 
latter assumption, which is valid in biological systems 
and some robotic examples [21.123], the control law 


A0 Z 


sign(r Z )6 Z if|T ; |>e Tz 
0 otherwise 


(21.40) 


excites a modal oscillation in a simple way. Thereby, the 
linear transformation Zj = w T q approximates the non¬ 
linear mapping <p: R" —> Zj, such that the generalized 
modal force can be transformed by the vector of weights 
w e R n , that is, 


r z = w T (i/f(0 — q) — f(0* - q*)) , (21.41) 


where 0 * and q* satisfy the equilibrium conditions 
g(q) + \j/ (q— 0 ) = 0. When the modal force r z exceeds 
a certain threshold e Xz , the controller switches by an 
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amplitude 6 This results in steps of desired motor 
positions 

o d = o* + wao z . 

Since the motor positions 9 are not the control input 
of the system (21.10), a motor PD controller should be 
considered to ensure that 0(t) —> 0 d (t). 

Note that the oscillation controller requires neither 
derivatives of measured signals nor model knowledge 

21.8 Optimal Control of Soft Robots 

Robot joints with intrinsic elasticity have some major 
advantages over their rigid counterparts: they are more 
robust, versatile, and can store energy. These properties 
enable robots to realize motions which would not be 
possible using stiff actuators of the same size. However, 
the question remains open as to how elastic elements in 
soft robots can be best controlled to exploit their po¬ 
tential. Optimal control (OC) theory is being regarded 
as one of the most promising tools to clarify such 
questions: as far as an appropriate choice of the cost 
functional is available, optimal control techniques can 
provide design guidelines for both the physical design 
and the control of soft robots. 

There are at least two main reasons why OC is 
fundamental to study soft actuators. On one side OC 
provides an absolute performance reference that fac¬ 
torizes the control design out of the equation, hence 
it furnishes a principled basis to compare the perfor¬ 
mance of different system designs. On the other side, 
OC is a key element in understanding planning and 
control methodologies for soft actuators. A careful anal¬ 
ysis of OC results obtained through either analytic or 
numerical techniques, allows to distillate laws summa¬ 
rizing control policies that can be applied to classes of 
tasks. 

General analytic solutions could be developed so far 
only for one-DOF joints. Physical bounds on states and 
inputs, the strong nonlinear structure of the robots and 
the high number of DOF allow only numerical solutions 
for general robotic systems. Nevertheless, choosing an 
advantageous parametrization for numerical solutions 
of the OC problem strongly benefits from insights given 
by the low-dimensional analytical solutions. Therefore, 
the analytical concepts are explained for some basic ex¬ 
amples in this section. 

21.8.1 OC Theory 

We consider the dynamics of a system described 
in standard form (x=f(u,x,t)), with the cost func¬ 
tional J consisting of a terminal cost 9 and an 


of the plant. Therefore it is simple and robust, and 
can be easily applied to different types of compli¬ 
antly actuated systems. The control law is, at least 
for the one-DOF case, the optimal solution for max¬ 
imizing the oscillation amplitude in minimum time. 
Furthermore, the control law defined by (21.40) and 
(21.41) is plausible to have a biological equivalent, 
since it strongly resembles the computation function of 
neurons. 


integral cost L 

if 

J(u) = &(x(tf), tf) + J L(x,u,t)dt. (21.42) 

to 

According to Pontryagin’s minimum princi¬ 
ple [21.124], the optimal control u* is known 
to minimize the Hamiltonian function 31 = 
A T f(u,x,t) + L, along the optimal trajectory, where A 
are the costates. For an OC problem with constraints on 
only the control u £ U, differential equations of these 
costates are given by A = — 9H/9x. Furthermore, if the 
final state is not explicitly stated, Minimum Principle 
also provides boundary conditions for A at the final 
time 



In order to make use of the Minimum Principle, one 
has to first solve these differential equations for x 
and A and then find the control u *, which minimizes 
the Hamiltonian: H(jc*, A*, u*) < H(jc*, A*, u) for all 
u e U, where x* and A* denote the optimal states and 
costates. 

21.8.2 SEA Optimization 

Naturally, the performance achievable with a given soft 
actuator under optimal control depends on the physics 
of the actuator itself. It is therefore very interesting to 
study the nested optimization problem of how to design 
soft robotics actuators which can produce the best per¬ 
formance under optimal control. 

From the analysis of a one-DOF SEA joint, it can 
be observed [21.17] that there exists an optimal value 
of the linear stiffness that maximizes the (optimally 
controllable) peak speed. The optimal stiffness value 
depends on the motor (e.g., maximum acceleration and 
speed) and on the task (e.g., terminal time). This result 
is illustrated in Fig. 21.12. 
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21.8.3 Safe Brachistochrone OC Problem 

One of the first instances of use of OC for soft (vari¬ 
able stiffness) robots is the study of the so-called safe 
brachistochrone problem [21.125]. The goal was to 
find control laws for the equilibrium position and vari¬ 
ables transmission stiffness such that a point-to-point 
link motion is achieved in minimum time subject to 
a limit on the worst-case impact force that the link could 
exchange with the environment during the trajectory. 
Limiting the impact force during motion can be use¬ 
ful both for increasing the safety of a robot operating 
in proximity of humans, and for reducing the risk of 
damaging the robot itself in accidental collisions with 
a stiff environment. The main results were to show that 
VSAs can perform better (that is, faster within bounds 
on a measure of impact risk) than both conventional 
(rigid) actuators and SEAs, and that the optimal stiff- 
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Fig. 21.12 Optimal motor velocity (co = 5rad/s, |i/imin| = 
ill max = 1 rad/s) 


ness law is inversely related to the link velocity, in 
a fashion roughly described by the easy-to-remember 
rule fast and soft, stiff and slow. 

21.8.4 Max Speed OC Problem 

A complementary example of application of OC theory 
concerns the problem of maximizing the velocity of an 
elastic robot joint at some desired final time or position. 
This goal is relevant to impulsive tasks, such as jumping 
on feet, hitting a ball or hammering a nail. Conversely 
to the previous example, in such tasks it might be desir¬ 
able that impact is maximized: indeed, it turns out that 
elastic actuators are neither intrinsically more or less 
safe than rigid joints, but can be both, under suitable 
control laws. 

The differential equations for the states and co¬ 
states of this OC problem are summarized in Table 21.1. 
Note that the functional J to be minimized consists 
merely of a terminal cost J(ui) = —q(tf). For this par¬ 
ticular problem, studied in [21.126,127), the condition 
of minimizing the Hamiltonian HI on the optimal trajec¬ 
tory becomes u* A* < u\X *, for all u\ e [u\ min, u\ max ]. 
Consequently, u* will have its minimum or maximum 
value depending on the sign of the first costate A*. Us¬ 
ing Table 21.1 , this first costate can be obtained as A * = 
—co sin[®(rf— f)], where co denotes the natural frequency 
of the system. The optimal control u* will thus be peri¬ 
odic with the eigenfrequency of the system co and take 
its maximum (minimum) value, whenever sin[o>(rf — f)] 
is positive (negative). Note that the switching of the op¬ 
timal control occurs with the eigenfrequency starting 
from the final time backward. For the particular case 



Fig. 21.13 Diagram (after [21.17]) 
representing the maximum speed 
reached by a one-DOF SEA after 
a single swing at varying the spring 
stiffness (co = y/k/m). Dashed 
curves show terminal speed q(T) 
under ideal motor position control 
(case P), speed control (case S) 
and acceleration control (case A). 

The continuous red curve shows 
terminal speed in the realistic 
condition of motor acceleration 
control (|0(f)| < lOrads -2 ), with 
bounds on position (|0(t)| < n rad) 
and velocity (|0(f)| < 5 rads -1 ). The 
small charts show optimal control and 
state patterns in time 
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Table 21.1 OC problem for the elastic joint ( u> = 


Excitation 

States x 

Cost J 

X = /(*,«l) 

i=-“ 

dx 

A (4) 

li\ = 6 

(1) 

-*2 (tf) 

till -* 2 \ 

\ ) 

tt 2 ) 



when the final time is a multiple of the oscillation period 
tf = nT = 2itn/a>, switching can be equivalently done 
in feed-forward after every half period. For an elastic 
joint starting from the rest position, this particular con¬ 
trol law states then to switch the motor velocity when 
the angular deflection </> or similarly the acceleration of 
the system changes its sign 

0* = sign(§)Mi max . (21.43) 


switching function 


k* 


Amin tf 77 < 0 
Amax tf q'q > 0 


(21.44) 


while for qq = 0 the stiffness value can be set arbi¬ 
trarily. This indicates the rather intuitive notion be¬ 
tween link velocity and accelerations summarized by 


If an additional quadratic integral cost in the con¬ 
trol, L = cu\ with a positive scalar c, is considered 
in (21.42), a continuous control law will follow. For 
tf = nT a classical resonance excitation with a sinu¬ 
soidal input then follows, corresponding to intuition and 
knowledge of linear systems. It is possible to extend 
these analytical solutions for nonlinear stiffness charac¬ 
teristics and damped systems [21.127]. For example, for 
an underdamped robotic link a similar optimal control 
is found, which is periodic with the damped eigenfre- 
quency. Different motor models can also be investigated 
analytically. 

21.8.5 VSA Optimization 

The very fact that for different tasks the optimal value 
of stiffness is different implies that the adaptability of 
the physical stiffness of actuators can be very useful in 
improving performance over a set of tasks. 

One further interesting question is raised by the 
possibility offered by VSA technologies to vary the ac¬ 
tuator stiffness in time. The problem becomes that of 
finding the optimal time course of both the actuator 
position 8(t) and stiffness k(t) so as to maximize per¬ 
formance. This problem can be again investigated using 
OC theory [21.17, 128]. Without going in the details of 
OC computations, results show that under the simplify¬ 
ing assumption that the actuator position and stiffness 
are bounded but can be changed instantaneously, the op¬ 
timal control for a single swing and free terminal time 
(within this swing) is bang-bang: the optimal reference 
position is 

8* = sign (q)8 mm , 

while the optimal stiffness k*(t) takes its minimum 
value k m ; n or maximum value £ max depending on the 


a) 




Fig.21.14a-c Throwing experiment with the DLR hand- 
arm system (after [21.87]). (a) Experimental setup, (b) 
velocity of joint 1 , (c) velocity of joint 2 
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the rule stiff speed-up, soft slow-down. More general 
cases of VSA optimization are discussed in [21. 1 7,128] 
and [21.126]. It is important to notice that even for one- 
link robots analytical solutions are not always straight¬ 
forward to find, especially if state constraints exist or 
system properties have nonlinearities. 

Under some conditions, it may be possible to ap¬ 
ply efficient numerical solutions to such problems. For 
instance, if the soft actuator dynamics system are lin¬ 
ear (as e.g., in the SEA case) but there exist state 
and control constraints (e.g.. the torque-speed char¬ 
acteristic of a dc motor), it is possible to translate 
the optimal control problem into a convex optimiza¬ 
tion problem [21.129], for which there exist fast and 
reliable methods to obtain the global solution. For 
robots with multiple-links, differential equations re¬ 


sulting from OC theory are nonlinear and cannot in 
general be solved analytically. Consequently, numeri¬ 
cal methods such as pseudospectral methods [21.130] 
or iterative linear quadratic regulator (ILQR) meth¬ 
ods [21.131, 132] are employed for these systems. One 
example of how to use these methods to outperform 
rigid robots is described in [21.133], where a throw¬ 
ing motion is optimized using the gauss pseudospectral 
method [21.134]. The reported experiments verify the 
system’s ability to use the elastic energy for explo¬ 
sive motions. Figure 21.14a illustrates the experimental 
setup, while the motor and link velocity are depicted on 
Fig. 21.14b,c. Note that the link velocities q reach more 
than double the maximum motor velocities 0 max during 
the motion, which would be clearly not possible with 
stiff actuators. 


21.9 Conclusions and Open Problems 

In this chapter, we have reviewed some of the technolo¬ 
gies and methods for analysis and control underpinning 
the use of novel actuators for soft robotics. The field 
has rapidly expanded in recent years, and it is very dif¬ 
ficult to provide an exhaustive up-to-date account of all 
innovations being continuously brought forward: hope¬ 
fully, this chapter provides at least a faithful picture of 
some of the most important current research trends in 
soft actuation. 

When comparing current robots with biological 
systems in terms of actuator energy density, energy ef¬ 
ficiency of motion, resilience, and sensitivity during 
interaction, the importance of intrinsic compliance be¬ 
comes striking. We are still far from having found so far 
the optimal technical realization of compliant, muscle¬ 
like actuators regarding design simplicity, integration, 
and energy density. Nevertheless, in this chapter foun¬ 
dations have been described both by properly defining 
basic notions such as nonlinear impedance and ad¬ 
dressing parametric identification, and in analyzing and 
standardizing the main physical properties to be used 
for the description of elastic actuators. A widely agreed 
set of characteristic parameters summarized in a stan¬ 
dardized data sheet format [21.75-77] help comparing 
the very heterogeneous design approaches and choosing 
the best concept for a given application. We proposed 
a quite general dynamic model for flexible robots with 
lumped elasticities, covering all designs proposed to our 
knowledge so far. The model is obviously related to 
the models of flexible robots from Chap. 11 and can 
be regarded as a generalization thereof. The control of 
flexible robots was focused on two main challenges. 
First, the simultaneous control of the position and in¬ 


trinsic impedance of the devices has to be solved. This 
is a challenge for systems with highly nonlinear and 
very compliant behavior. This aspect was addressed on 
joint and the Cartesian level. 

Second, control to push flexible robots to their limit 
and maximally exploit their potential benefits. If robust¬ 
ness comes mainly by the design process, that is not 
true for energy efficiency and performances maximiza¬ 
tion (even under safety constraints). The achievement of 
these features requires optimization and optimal control 
algorithms, such as those exposed in the present chap¬ 
ter. Moreover, OC algorithms can be used as a tool for 
comparison among different actuator designs extracting 
their inherent (i. e., control-independent) performance 
boundaries, and to derive easy-to-understand control 
rules for peculiar DOFs (e.g., stiffness) of soft robots. 

Important new directions are open in front of the 
research community. On one side, new active materi¬ 
als could provide a boost to the development in soft 
robotics actuators, especially in the small scale; on 
the other hand, it can be expected that some of the 
promising advanced material research in the past will 
eventually make their way in practically viable tech¬ 
nologies. New functional architectures for actuators 
that can adapt their physical behavior to the task are 
also potentially very fruitful. An important future trend 
is the extension from variable stiffness to VIAs, capa¬ 
ble of more closely mimicking the functionalities of 
muscular structures where not just the elasticity, but 
also the damping (and perhaps the effective inertia) can 
be physically tuned. Much work remains also to be 
done in the understanding of the theoretical underpin¬ 
nings and in practical methods for the identification and 



Actuators for Soft Robotics I 21.9 Conclusions and Open Problems 


control of nonlinear, time varying, multidimensional 
impedance. 

The very fact that research in novel actuators for 
soft robotics has produced a wealth of ideas and so¬ 
lutions poses a challenge of different nature. Indeed, 
a huge potential for innovation is already available 
to researchers and practitioners, which has been only 
superficially tapped so far in applications. Soft actu¬ 
ators could change the landscape not only in robust 
and safe interaction or in locomotion: there are prob¬ 
ably many, more specific application niches where the 
new technologies could have an impact - possibly 
also outside the scope of robotics narrowly defined, 
in the larger domain if automation. To foster tech¬ 
nological awareness of soft robotics potential in the 
community of those interested in machines that can 


move and interact more naturally, several initiatives are 
being undertaken, including an IEEE technical commit¬ 
tee on soft robotics [21.135] and several workshops, 
special issues and even specialized journals on the 
topic. Recently, a natural machine motion initiative 
(NMMI) [21.136] has been launched with the aim of 
diffusing the ideas of soft actuation to achieve nat¬ 
urally moving robots with human-like grace, power, 
and dexterity. One of the objectives of the NMMI is 
to make the technology of soft actuators widely avail¬ 
able to the general public, by publishing as open-source 
the design and all instructions needed to build and use 
low-cost actuators, thus building a community of de¬ 
velopers interested in sharing advances toward simple, 
affordable, yet smooth, strong, and accurate muscles for 
robots. 
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Variable impedance actuators: Moving the robots of tomorrow 

available from http://handbookofrobotics.org/view-chapter/21/videodetails/456 

Petman tests camo 

available from http://handbookofrobotics.org/view-chapter/21/videodetails/457 
Introducing Wildcat 

available from http://handbookofrobotics.org/view-chapter/21/videodetails/458 
VSA CubeBot - peg in hole 

available from http://handbookofrobotics.org/view-chapter/21/videodetails/460 
DLR Hand Arm System smashed with baseball bat 

available from http://handbookofrobotics.org/view-chapter/21/videodetails/461 
Safety evaluation of lightweight robots 

available from http://handbookofrobotics.org/view-chapter/21/videodetails/463 
Hammering task with the DLR Hand Arm System 

available from http://handbookofrobotics.org/view-chapter/21/videodetails/464 

Dynamic walking of whole-body compliant humanoid COMAN 

available from http://handbookofrobotics.org/view-chapter/21/videodetails/465 

Dynamic walking of whole-body compliant humanoid COMAN 

available from http://handbookofrobotics.org/view-chapter/21/videodetails/466 

Maccepa System 

available from http://handbookofrobotics.org/view-chapter/21/videodetails/467 
AMASC - changing stiffness 

available from http://handbookofrobotics.org/view-chapter/21/videodetails/468 
VSA-Cube: Arm with high and low stiffness preset 

available from http://handbookofrobotics.org/view-chapter/21/videodetails/470 
CompAct™ robotics technology 

available from http://handbookofrobotics.org/view-chapter/21/videodetails/471 
VSA-Cube arm: Drawing on a wavy surface (high stiffness) 
available from http://handbookofrobotics.org/view-chapter/21/videodetails/472 
Arm drawing on a wavy surface (low stiffness) 

available from http://handbookofrobotics.org/view-chapter/21/videodetails/473 
Arm drawing on a wavy surface (selective stiffness) 

available from http://handbookofrobotics.org/view-chapter/21/videodetails/474 
Intrinsically elastic robots: The key to human like performance (Best Video Award) 
available from http://handbookofrobotics.org/view-chapter/21/videodetails/475 
DLR Hand Arm System: Punching holes 

available from http://handbookofrobotics.org/view-chapter/21/videodetails/546 

DLR Hand Arm System throwing a ball and Justin catching it 

available from http://handbookofrobotics.org/view-chapter/21/videodetails/547 
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Active damping control on the DLR Hand Arm System 

available from http://handbookofrobotics.org/view-chapter/21/videodetails/548 
Throwing a ball with the DLR VS-Joint 

available from http://handbookofrobotics.org/view-chapter/21/videodetails/549 
DLR Hand Arm System: Two arm manipulation 

available from http://handbookofrobotics.org/view-chapter/21/videodetails/550 
Full body compliant humanoid COMAN 

available from http://handbookofrobotics.org/view-chapter/21/videodetails/698 
AWAS-II 

available from http://handbookofrobotics.org/view-chapter/21/videodetails/699 
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22. Modular Robots 


l-Ming Chen, Mark Yim 


This chapter presents a discussion of modular 
robots from both an industrial and a research 
point of view. The chapter is divided into four 
sections, one focusing on existing reconfigurable 
modular manipulators typically in an indus¬ 
try setting (Sect. 22.2) and another focusing on 
self-reconfigurable modular robots typically in 
a research setting (Sect. 22.4). Both sections are 
sandwiched between the introduction and con¬ 
clusion sections. 

This chapter is focused on design issues. Rather 
than a survey of existing systems, it presents some 
of the existing systems in the context of a dis¬ 
cussion of the issues and elements in industrial 
modular robotics and modular robotics research. 
The reader is encouraged to look at the references 
for further discussion on any of the presented 
topics. 
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Modularity in design engineering refers to a compart- 
mentalization of elements. Most often modularity in 
complex systems occurs as a result of taking a com¬ 
plex system and dividing it into pieces in order to 
better understand the simpler elements and parallelize 
the design efforts. Modularity also facilitates the re¬ 
placing of elements either for repair or upgrading new 
functionality. The alternative to a modular approach is 
an integrated approach where systems are designed as 


a whole. While integrated approaches tend not to be 
as easy to repair, upgrade or reconfigure, they do have 
fewer constraints on element design and therefore can 
be made more optimal. Integrated approaches can fo¬ 
cus on lowering cost or having higher performance. 
In mechanical devices, the choice between modular 
or integrated architectures can have a large impact on 
the range of application as well as cost or perfor¬ 
mance [22.1]. 



22.1 Concepts and Definitions 

For example, a hand drill with modular attachments 
can expand itsrange of functionality from drilling holes 


to screwing bolts or buffing surfaces. For robotics, the 
same impact applies. However, robots have an inher- 
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ent complexity that lends itself to modularity,such as 
actuator modules, sensor modules, and sometimes com¬ 
putation modules. In the following sections, definitions 
will be given for different kinds of modularity fre¬ 
quently seen in robotics. 

22.1.1 Concept of Modularity 

The product design literature can be looked at as 
encompassing robotics - which can be considered 
as an industrial or research product. The architec¬ 
ture of product modularity can be categorized into 
three subtypes: slot, bus, and sectional modular¬ 
ity [22.1]: 

• Slot architecture: Each of the interfaces between 
components is of a different type from the others, 
so that the various components in the product can¬ 
not be interchanged. 

• Bus architecture: There is a common bus to which 
the other physical components connect via the same 
type of interface. 

• Sectional architecture: All interfaces are of the 
same type and there is no single element to which 
all the other components attach, i. e., there is no 
base component. The assembly is built up by con¬ 
necting the components to each other via identical 
interfaces (Fig. 22.1). 

Such classifications provide a good definition of 
modular robots as follows: 

• If a complex robotic system adopts a slot- and 
bus-modularity design approach for its internal 
structure and architecture, not the external con¬ 
figuration, it can be called a modularly designed 
robotic system benefiting from design paralleliza¬ 
tion. Such a robot may have a unified and inte¬ 
grated configuration that cannot be changed from 
outside. 

• If a robot adopts a bus- and sectional-modular¬ 
ity design approach for both internal structure and 
external configuration, it can be called a modular 
robot. The users can reconfigure the compartmen- 
talization and interchange functional modules with 
some level of effort. 


22.1.2 Definition and Classification 
of Modular Robots 

Any system can be reconfigured by destructing and re¬ 
constructing it, for example in the worst case, using 
a blowtorch and milling machine. The key element we 
must define is the level of effort required to reconfig¬ 
ure. We propose here three levels from the lowest to the 
highest level of effort of the user: 

1. The system reconfigures itself. It is self-reconfig- 
urable. 

2. The system is reconfigured by a lay user with or 
without special tools typically in matter of seconds 
to minutes. 

3. The system is reconfigured only by an expert with 
specialized tools. 

This chapter will focus on modular robots that give 
rise to plug-and-play reconfiguration of the system for 
task and function changes, levels 1 and 2. Level 1 
systems are self-reconfigurable systems with sectional 
modularity. Level 2 systems in this chapter focuses on 
reconfigurable modular manipulators with a finite set of 
modules of different functions. 

The modular manipulator type of robotic systems 
are natural evolution of industrial robot manipulators 
that consist of a number of specific functional modules, 
such as actuator modules, link modules, and end-effec¬ 
tor modules. Subsequently, robots with the serial and 
branching topology, such as humanoid robots, legged 
robots, mobile manipulators adopt a similar approach 
for modularity as these functional modules form the ba¬ 
sis of a robotic system. 

The self-reconfigurable modular robots grew out of 
the concept of self-evolution and self-configuration of 
biological cells with identical units. Such a robot nor¬ 
mally consists of a large number of a small set of types 
of mechatronic units that possess actuation, connection, 
communication, and computing capability that can be 
assembled together in arbitrary forms and also recon¬ 
figure itself. 

Although the two types of modular robots origi¬ 
nated from different fundamental concepts, the goals to 
provide a large number of possible robot configurations 
for different tasks with the same set of basic robot mod¬ 
ules are the same. 


a) 


b) 


c) 



Fig.22.1a-c Modular ar¬ 
chitecture types: (a) slot 
architecture (b) bus architec¬ 
ture (c) sectional architecture 
















Modular Robots I 22.2 ReconfigurableModular Manipulators 533 


22.2 Reconfigurable Modular Manipulators 


Reconfigurable modular manipulators are robot arms 
that have elements that can be rearranged. 

22.2.1 Background of Modular 
Manipulator Systems 

The simplest form of modular manipulator comes in 
the form of automatic tool changers also called quick- 
change end-effectors. These are optional tools that 
can be attached to the end of a robotic arm. Auto¬ 
matic tool changers are standard equipment for many 
CNC (computer numerically controlled) milling ma¬ 
chines and lathes since the 1960s. They enable the 
machines to drill holes of different sizes, or cut dif¬ 
ferent shapes. Although CNC machines are not often 
considered industrial robots, they share the same ele¬ 
ments (actuators, sensors, and computation) and their 
function is more limited. Most industrial robot arms can 
be equipped with automatic changing end-effectors by 
adding a wrist that has the compatible interface for a va¬ 
riety of grippers and end-effectors. These devices are 
available commercially from Schunk (Germany), ATI 
industrial automation (USA), Destaco, Amatrol, RE2 
(USA), RAD, and others. 

In the modularization of industrial robots, the gran¬ 
ularity of the components is usually based on their basic 
functions, i. e., motion actuation and tooling. Thus, the 
design of modules is highly differentiated into actuator 
modules, passive joint modules, and tooling modules, 
etc. Several prototype modular robotic systems have 
been built and demonstrated, including the reconfig- 
urable modular manipulator system (RMMS) [22.2], 
several generations of the cellular robotic system (CE- 
BOT) [22.3], and modular manipulator systems devel¬ 
oped by University of Toronto [22.4], University of 
Stuttgart [22.5], University of Texas at Austin [22.6], 
and Toshiba Corp. [22.7]. 

Basically, these systems have serial-type (or open- 
chain) geometry with large working envelopes. These 
serial-type modular robots are well suited for assembly, 
trajectory tracking, welding, and hazardous material 
handling. Parallel modular robots have also been de¬ 
veloped for light-machining tasks [22.8]. As indicated 
in [22.8], modular design can reduce the development 
cycle of the parallel robots significantly. Furthermore, 
it allows a trial-and-error approach for the construction 
of parallel robots that is impossible with the integrated 
design approach. 

With globalization of world manufacturing, the con¬ 
cept of modular manipulators has quickly gained in¬ 
dustrial attention. A full-scale reconfigurable robotic 
system workcell consisting of three modular robots 


with a total 15 axes was successfully showcased in 

1999 [22.9] (Fig. 22.2). The modular robot workcell has 
a serial type 6-degrees-of-freedom (DOF) robot for the 
pick-and-place action of the work piece, a 2-DOF robot 
for work piece transfer, and a 6-DOF parallel robot for 
milling operations on the work piece. All the robots are 
built from the same set of modular components, includ¬ 
ing actuator modules, link modules, and tool modules. 

The German company Amtec, later acquired 
by Schunk, developed the first commercially avail¬ 
able modular manipulator system called PowerCube. 
Schunk has subsequently developed its industrial ma¬ 
nipulators and automation systems based on Power- 
Cube with some success [22.10]. Today there are many 
robotic systems with a wide spectrum of applications 
that are built around modular robot components. 

The modular robot concept also proliferated in the 
hobby and educational robot sectors around the year 

2000 by the introduction of well-packaged self-con¬ 
tained servo motor modules into inexpensive robotic 
devices, such as Robotis (Korea) and Kondo (Japan), as 
well as Lego (Denmark) and other toy companies mak¬ 
ing educational robots. 

Mobile robots with legs, wheels, and tracks also 
belong to this class of modularity where they are 
configured for different task requirements such as 
those needed for disaster relief, rescue, and surveil¬ 
lance purposes. Two tracked modular mobile robots 
designed with multiple track segments [22.11] and 
reconfigurable tracks allowing serial and parallel con¬ 
nections [22.12] have been demonstrated. The work 
in [22.13] contains an in-depth review of the develop¬ 
ment in modular mobile robots. 



Fig. 22.2 15-axis Reconfigurable Robotic Workcell (after 
[22.9]) 
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22.2.2 Module Design Issues 

A modular robot consists of the two main features 
found in a modular product: 1) a one-to-one mapping 
from functional elements to the physical components 
of the product and 2) decoupled interfaces between the 
components of different modules [22.1], For modular 
manipulators, the essential components are the base, 
positioning, and orienting mechanisms composed by 
actuator modules and link modules of different dimen¬ 
sions and geometry, and the end-effector module. For 
legged and wheeled mobile robotic systems, the motion 
generation mechanism modules are essential. 

The actuator modules normally use DC or AC 
motors as a 1-DOF rotate or pivot joint module typ¬ 
ically with compact high reduction ratio transmission 
mechanisms [22.2,4,6,7,9]. Some modular systems 
also adopt 1-DOF linear modules for larger workspace 
envelope [22.4, 9] and 2-DOF joint modules for com¬ 
pact dexterous motions [22.10]. The actuator module 
shown in Fig. 22.3 has two independent linear and 
rotary motion capabilities suitable for compact assem¬ 
bly tasks [22.9]. The actuator modules are typically 
designed with similar geometry but with different di¬ 
mensions and power ratings for different application 
requirements. 

The link modules connecting units in between the 
actuators function as reachable workspace extenders. 
Some systems adopt a standard fixed-dimension con¬ 
nection module [22.3,4,6] and some use variable di¬ 
mension modules that can be customized to satisfy arbi¬ 
trary design constraints [22.9]. In some systems [22.2], 



Fig. 22.3 A 2-DOF translate-turn module with ball-screw 
and ball-spline mechanism 


the link module becomes part of the actuator module so 
that the module acts as an actuator as well as the con¬ 
necting structure. 

22.2.3 Interface Design Issues 

The mechanical connecting interface between modules 
in a modular manipulator needs to satisfy three basic 
requirements: 

1. Stiffness 

2. Fast reconfiguration 

3. Interchangeability. 

Thus, the design of mechanical connections or 
docking mechanisms is a critical issue. In a fully or 
semisupervised robotic system, like some modular ma¬ 
nipulator systems [22.2-9], the connecting mechanism 
is designed to be manually operated for reliability and 
safety reasons. In a fully autonomous system, the con¬ 
necting mechanism needs to be designed typically with 
an extra actuator and locking mechanism for carrying 
out the connection automatically. This is the case for 
most of the self-reconfigurable modular robots. 

In order to meet the requirement of interchangeabil¬ 
ity, the electronic and communication interface for the 
modular system normally adopts common communica¬ 
tion network architecture with plug-and-play capability 
similar to local area network (LAN). There are a num¬ 
ber of existing industrial standard network protocols for 
real-time robot control suitable for such applications, 
like CAN-bus, RS485, and IEEE 1394. The progressive 
development of industrial automation protocols will 
facilitate the implementation of modular robot com¬ 
munications. While many of the early systems use 
wired multidrop bus architectures for communications, 
multirobot systems have used fast local message for¬ 
warding [22.14] and wireless networking. 

22.2.4 Modeling of Modular Manipulators 

Challenges to model the modular manipulator systems 
come from the lack of uniform formalisms of the un¬ 
fixed robot configuration and geometry and the errors 
accumulated from assembly and dis-assembly of the 
modules. Hence, the first effort in modular robot mod¬ 
eling was the introduction of a graph-based technique 
with additional module assembly information for the 
representation of a modular robot configuration [22.15]. 
This work introduced a modular robot representation 
scheme, termed assembly incidence matrix (AIM) for 
distinct modular robot configurations. There are sev¬ 
eral subsequent extensions and variations of AIM for 
broader categories of modular robots including modu¬ 
lar mobile manipulators [22.16-18]. 
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Once the modular robot configuration can be dis¬ 
tinctly defined with the type of modules, the connection 
sequence, module orientation, kinematic, and dynamic 
models of the robot can be obtained through an auto¬ 
matic generation algorithm [22.19]. Kinematic model 
generation can be achieved through the conventional 
Denavit-Hartenberg (DH) parameterization [22.2-4, 
17] or the coordinate-free local product-of-exponential 
(POE) approach [22.8, 9, 19]. However, the DH method 
does not provide a clear distinction between the ar¬ 
ranging sequence of modules in a robot chain, and it 
is an initial position-dependent representation. The lo¬ 
cal POE formulation of the kinematics and dynamics 
based on the theory of Lie groups and Lie algebras for 
rigid motion in se(3) and SO( 3) can avoid this prob¬ 
lem. Furthermore, the POE representation can avoid the 
singularity conditions that frequently occur in the kine¬ 
matic calibration formulated by the DH method [22.20], 
Thus, POE representations provide a uniform and well- 
behaved method for handling the inverse kinematics of 
both calibrated and un-calibrated robot systems. In local 
POE modeling, the joint axes are described in the local 
module (body) coordinate systems, it is progressive in 
constructing the kinematic models, so it conveniently 
resembles the assembling action of the physical modu¬ 
lar robot components. 

The machining tolerance, compliance, and wear of 
the connecting mechanism due to frequent module re¬ 
configuration may introduce errors in positioning the 
end effector. Hence, kinematic calibration is a must for 
modular robots. In the POE calibration model, the robot 
errors are assumed to be in the initial positions of the 
consecutive modules because the local POE model is 
a zero reference method. Based on linear superposi¬ 
tion and differential transformation, a 6-parameter error 
model can be established for serial-type robots [22.19], 
This model can be obtained through the automatic 
generation process. An iterative least-square algorithm 
employed to find the error parameters to be corrected. 
The corrected kinematic model is then updated in the 
robot controller for operation. The simulation and ex¬ 
periment have shown that the proposed method can 
improve the position accuracy up to two orders of mag¬ 
nitude, or to the nominal repeatability of the robot after 
calibration with measurement noise. A typical 6-DOF 


articulate-type modular robot can reach a position accu¬ 
racy of 0.1 mm compared to an accuracy of 1 mm before 
the calibration [22.20] 

A formulation of the dynamic model of modu¬ 
lar manipulators starts from a recursive Newton-Euler 
algorithm [22.21,22]. The generalized velocity, accel¬ 
eration, and forces can be expressed in terms of linear 
operations on se(3) [22.23]. Based on the relation¬ 
ship between the recursive formulation and the closed- 
form Lagrangian formulation for serial-robot dynam¬ 
ics discussed in [22.24, 25], the AIM can assist in the 
construction of the closed-form equation of motion of 
a modular robot in any generic topology with redundant 
and non-redundant configurations [22.19]. 

22.2.5 Configuration Optimization 

Due to the modular design, the modular manipulator 
can be optimal at the component level, but may not 
obtain optimal performance at the system level. Task- 
driven robot configuration optimization becomes nec¬ 
essary to establish locally optimal performance for the 
overall robotic system. Typically, the problem of robot 
configuration optimization can be stated as finding an 
assembly of robot modules that can achieve a certain 
task requirement based on an inventory of modules. 
The configuration of a modular robot can be treated as 
a compound entity with finite number of constituents. 
Finding the most suitable task-oriented robot config¬ 
uration then becomes a discrete design optimization 
problem using a task performance related objective 
function. Discrete optimization techniques, such as ge¬ 
netic algorithms (GAs), the simulated annealing (SA) 
method, and other artificial intelligence techniques have 
been employed to find solutions [22.26-28]. 

The criteria used in selecting the optimal configu¬ 
ration depend largely on the task requirements, which 
mostly describe the necessary robot trajectories or key 
postures. Yang and Chen proposed a reduced DOF 
approach to minimize the total number of actuator mod¬ 
ules employed in a serial-type modular robot for a given 
task [22.29]. With fewer modules, the robot can carry 
more payloads instead of carrying distal modules. Fur¬ 
thermore, the robot can be operated at higher speed with 
better dynamic response. 


22.3 Self-Reconfigurable Modular Robots 


Self-configurable systems can rearrange their own 
topology. An example is shown in |<ss>KlLlH!KS. There 
are dozens of research groups who have constructed 
many versions of self-reconfigurable robots [22.3, 
30-46], with many approaches for programming 


them [22.36,47-61]. As of 2012, over 800 papers and 
a book [22.31] have been written. 

These systems are characterized by many identi¬ 
cal modules that can be rearranged into a variety of 
shapes and configurations and by being highly seal- 
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able with simulated systems having hundreds, thou¬ 
sands, or millions of modules. These systems have three 
promises: 

1. Low cost from batch fabrication of repeated mod¬ 
ules 

2. High robustness from redundancy and the ability to 
self-repair 

3. High versatility from the ability to reconfigure and 
adapt to changing situations [22.62]. 

Practically speaking, none of the promises have 
been proven, though the promise of versatility is getting 
close. These systems have exhibited a wide variety of lo¬ 
comotion and manipulation including: legged walking 
with between 2 and 14 legs; riding a tricycle [22.63]; 
rolling like a tread [22.14]; snake-like locomotion (lat¬ 
eral and rectilinear undulation, concertina, sidewind¬ 
ing) [22.64]; manipulation of large objects with mul¬ 
tiple arms/fingers [22.65]; manipulation of small ob¬ 
jects; climbing stairs, fences, poles, in pipes; self¬ 
reconfiguration between dozens of shapes and many 
others [22.31]. Figure 22.4 and I<SWII > 11 I 1M show a 
self-reconfigurable modular robot SMORES [22.66]. 
Other self-reconfigurable robots like ATRON [22.40] 
is shown in l<s>)KZIin!Hi and M-blocks is shown in 
ktbVEim. 

22.3.1 Types of Self-Reconfigurable Modular 
Robots 

Self-reconfiguring systems can be classified into three 
types based on the style of reconfiguration: chain, 
lattice, and mobile [22.62]. The chain systems recon¬ 
figure by using chains of modules that form and break 
loops [22.36, 62]. They tend to be well suited for work 
on the environment, as they can form articulated limbs. 
The lattice systems have modules which have nominal 
positions sitting on a regular lattice and tend to be better 
at self-reconfiguration as moving to neighboring lattice 
positions makes collision checking easy [22.33, 34,39, 
43]. The mobile systems have modules that individu¬ 
ally maneuver on terrain and reconfigure by moving on 
the environment to relocate themselves in a conglomer¬ 
ate [22.38,67], 


Of the systems that have been implemented to date, 
some that have been shown to be most capable (judg¬ 
ing by number of demonstrations) are the hybrid chain- 
lattice systems: Superbot [22.14], MTRAN III [22.68] 
and CKbot [22.69]. Recent additions to this group are 
Johns Hopkins University [22.70], iMobot [22.71], and 
SMORES [22.66] which are hybrid in all three areas, 
chain-lattice-mobile systems. 

22.3.2 System and Module Design Issues 

There is an interesting phenomenon called second sys¬ 
tem syndrome [22.72], where designers include many 
features in the second version of a system they de¬ 
sign. They often include many more features than the 
system may need. This is especially problematic in 
modular reconfigurable systems with repeated modules; 
any feature added to one module has the possibility 
of its effect multiplied by n, the number of modules. 
For example, an increase of d in computational pro¬ 
cessing power (e.g., microprocessor without interlocked 
pipeline stages, MIPS) in one module will result in 
a system increase of dn. 

Another similar phenomenon for designers is called 
feature creep , where more and more features are added 
as the system is being designed. Often this has a cu¬ 
mulative linear effect on cost, but worse, there is an 
exponential effect on reliability. 

The simplest form of robustness analysis assumes 
independent probabilities that a module may fail dur¬ 
ing a specified function. If one module succeeds during 
that function with probability p then the system with 
^identical module has a probability of success of //', un¬ 
der the assumption that all modules must be functioning 
to succeed. Clearly this is problematic for large n. 

As systems scale up in number, a key to make things 
work is to be sure that the system does not depend on 
every module working perfectly. Indeed, if only X num¬ 
bers are needed, one would wonder why one would use 
more than X, especially if the only impact is reduced re¬ 
liability. Here, one strategy is to ensure that the solution 
is devised in a way where the performance improves 
with the number of modules. If modules fail, then the 
system can gracefully degrade. In systems with tightly 
coupled actions between modules, this can be difficult 



Fig. 22.4 (a) One SMORES module 
with four main actuators and four 
docking faces, (b) Three SMORES 
modules attached together, (c) Two 
modules moving on a lattice 
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to implement. In tasks where only binary metrics exist, 
(success or failure) using the optimal X modules, setting 
aside the extras may be the best solution. 

One of the aspects that researchers find most in¬ 
teresting about self-reconfigurable robots is examining 
what happens when modules scale up in number (and 
also typically, scaled down in size). As the numbers in¬ 
crease, the number of shapes/configurations increases 
and concomitantly, the types of activities that are possi¬ 
ble increase as well. 

Control and planning get very complicated quickly 
though. Simply enumerating the number of isomor¬ 
phic configurations has resulted in a PhD thesis without 
completely solving the problem [22.15]. 

While researchers have simulated hundreds, thou¬ 
sands, and even millions of modules, no physical 
system has been demonstrated with more than a few 
hundred to date. The largest single system so far is 
Kilobot [22.73], which has very simple mobile mod¬ 
ules that swarm together and actually do not connect 
rigidly. The modules communicate wirelessly in a one- 
to-many (broadcast) fashion. For rigidly connected sys¬ 
tems the largest number of modules demonstrated in 
one connected system was a 14-legged 48-module Poly- 
bot system [22.74]. 

Main Actuation 

Every module has some form of actuation that enables 
the modules to move from one position to another or 
to do some work on the environment. We call it the 
main actuator. By far the most common main actua¬ 
tor is a DC electric motor as it is the lowest cost and 
easiest to implement. 

In chain systems, the main actuator typically drives 
a revolute joint so that a chain of modules forms an 
articulated arm. In lattice systems, the main actuator 
typically moves the module (or a neighboring module) 
along a constrained 1-DOF path that can be translation 
or rotation. In mobile reconfiguration, the main actuator 
enables locomotion, usually through wheels or treads. 

The main actuator is typically the largest compo¬ 
nent in a module and so has been the focus when trying 
to make modules smaller. To date, the smallest mod¬ 
ule with onboard actuators, that both attaches and de¬ 
taches uses shape memory actuated module that was 
2 cm [22.75]. An even smaller 1 mm module was created 
at Carnegie Mellon University [22.76] using electrostat¬ 
ics as an actuator, but it did not attach to other modules. 

On the other extreme, the largest module is 
GHC [22.77], an 8 m 3 helium-filled cube with shape 
memory actuators on the edges to rotate these float¬ 
ing balloons about edges, attaching electrostatically. 
DARPA is also sponsoring a project to look at reconfig- 
urable maritime craft constructed in a 20 / ISO container 


form factor. University of Pennsylvania has demon¬ 
strated scale models at 1/12 scale, with main actuators 
being thrusters to maneuver the parallelepipeds in wa¬ 
ter [22.78]. 

22.3.3 Interface Design Issues 

The main component of modular systems differentiat¬ 
ing it from an integrated system is the interface between 
modules. When talking about modular systems, the 
amount of modularity can be measured by the num¬ 
ber of these interfaces. For self-reconfigurable modular 
robotic systems in this chapter the number of interfaces 
in one connected component can be as small as six or 
as large as millions. 

In the most general case, every interface must do 
two things: 1) attach and 2) detach. When they attach 
they must do two things: 1) form a physical coupling 
and 2) allow a flow through the interface for supply 
power and information (often done electrically). 

The interfaces can be gendered [22.36], that is two 
mating interfaces are not identical, one has male fea¬ 
tures sticking out and the other female features to 
receive the male features. They can be ungendered 
with no protruding or receding mating features. Or they 
can be hermaphroditic [22.39] where interfaces con¬ 
tain both male and female features. Ungendered and 
hermaphroditic interfaces can have identical interfaces 
on both sides. This increases the number of possible 
arrangements over gendered modules; however, un¬ 
gendered and hermaphroditic components are typically 
more complex than gendered interfaces. 

Each module can have multiple interfaces. If we 
consider the number of configurations possible with m 
interfaces per module assuming identical modules and 
ignoring any physical constraints such as self-collision, 
we obtain m" possible configurations. The number of 
non-isomorphic configurations is much less, but is very 
difficult to enumerate in general. However, one special 
case to consider is two interfaces per module. In this 
case, topologically, there are only two different configu¬ 
rations, a single chain, and a loop. For this reason, most 
systems have three or more interfaces. 

In addition, each interface can have multiple sym¬ 
metries that allow different kinematic relationships for 
the same adjacency relationship. For example many 
systems have either a 2-way connector (modules may 
be optionally be rotated 180° to each other) or a 4-way 
connector (90° rotations.) This p -way symmetry leads 
to (pm)" total possible configurations. 

Mechanical Interfaces 

Mechanical interfaces rigidly attach two modules to¬ 
gether with the ability to detach as well. Strategies to 
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enable this capability include using nuts and bolts, mag¬ 
netic bonding [22.39], electrostatic bonding [22.77], 
probe and drogue [22.79], and a variety of physical 
hook-type mechanisms. Each mechanical interface has 
three performance metrics: bond strength, acceptance 
range, and interface precision/stiffness. 

Bond strength can be characterized by the amount 
of force required to separate two modules in its weakest 
direction [22.80]. The acceptance range is the tol¬ 
erance to position errors as two docks approach to 
dock [22.81]. Interface precision/stiffness indicates the 
amount of position error (deflection) that can occur con¬ 
tributed by the interface under load. This can come from 
elastic deflection, slop at a joint in a mate, or misalign¬ 
ments in non-self-guiding docks. 

For self-reconfiguring systems, attaching and de¬ 
taching is automatic and the area of acceptance is 
required to be much larger than typical manually assem¬ 
bled modular systems. This is in large part because the 
cost of precision is high and bond precision/stiffness is 
typically small. The ideal mechanical interface has high 
bond strength, high acceptance range and high bond 
stiffness, all of which comes with increased weight, 
size, and cost. However, different styles of reconfigu¬ 
ration emphasize different aspects. 

Lattice reconfiguration systems do not need high ac¬ 
ceptance range as modules move in highly constrained 
manner when reconfiguring, usually with one degree of 
freedom from one lattice position to neighboring one. 
However, high bond stiffness is emphasized, as these 
modules do not often have actuated degrees of freedom 
to compensate for errors in position. For example, a se¬ 
quence of modules in a lattice that are forming a loop 
might not close properly because one section of mod¬ 
ules may sag under gravity. This sag was noticeable in 
the 3-D lattice module [22.82], 

Chain reconfiguration modules can deal with lower 
bond stiffness since they usually have degrees of free¬ 
dom that can compensate for errors in deflection. How¬ 
ever, docking even with this compensation is not trivial 
and most systems require as large an acceptance range 
as possible. 

Bond strength for systems that reconfigure tend to 
be much lower than the one would find in a manu¬ 
ally assembled system. For example, magnetic bonding 
methods are easy to implement, typically have good 
precision and wide range of acceptance, but are weak. 
Manual systems that are bolted together have high bond 
strength, but are very complicated if they are made 
automatic with a very small acceptance range. The vari¬ 
ability in bond strength versus the stiffness in joints 
can be used to vary the overall stiffness of a con¬ 
glomerate system [22.80]. This variable stiffness can 
be used to make a compliant material that can conform 


to surfaces or a stiff material that will not bend under 
load. 

Depending on the task, bond strength may not 
be required to be high. For example, in mobile re¬ 
configuration systems that primarily move on the 
ground, e.g., forming trains such as millibot [22.67] or 
swarms [22.38], the worst case situation is when the 
conglomerate tries to cross a gap with modules can¬ 
tilevered over the gap. Otherwise, the conglomerate is 
nominally supported at all times and the bond only 
sees dynamic friction and inertial forces. But in general, 
higher bond strength does not hurt and keeps a system 
from falling apart from static or dynamic loads. 

Power and Communications Interface 
Power and communications interfaces are typically 
electrical, though in the case of the related work in 
quick-change end effectors, pneumatics is often sup¬ 
plied too. In any case, the primary parameters of con¬ 
cern are the number of lines that must be transmitted 
between the interface, and the type of lines (e.g., pneu¬ 
matic. high voltage, high current, fiber optic, etc.). The 
type of line will determine the size (which usually limits 
the number of lines) as well as the precision alignment 
required at the interface. 

In self-reconfigurable systems, the typical interface 
passes electric power and a separate electronics com¬ 
munications bus on which all modules talk, though they 
are sometimes combined [22.77]. A key consideration 
for electronic lines is not only that the correct lines 
have good contact when mated, but also that wrong 
contacts do not touch during the docking process due 
to position error. For most self-reconfiguring systems 
maximizing the tolerance to position errors is the most 
important [22.56]. 

Communication between modules is essential for 
any self-reconfigurable system, whether the control 
is distributed or centralized. There are primarily two 
forms of communication: a global communications bus 
(any module talks to any other module) or a local 
communications medium (modules only talk to their 
neighbors through the modular interface.) 

These two forms have different implications. Global 
buses are typically much faster than local methods of 
equivalent cost. There are no issues of varied latency 
between modules as with local methods. One problem 
with global buses is that there is no mechanism for 
modules communicating with each other to know the 
relative physical position of each module. Local meth¬ 
ods get this for free. Local methods, can also emulate 
global architectures by message passing and routing. 
Local methods are also more robust to physical errors 
(such as shorting a communications line to power) con¬ 
taining failures to locally. However, they are just as 
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Table 22.1 Characteristics of used communication protocols 


Method 

Speed (kbs) 

Range(m) 

Power (mW) 

Address space 

Rel. cost 

Notes 

Ethemet/CAT 

lk-lm 

100's 

500 

billion 

$$$$ 

Complex stack (IPV4) 

CANbus 

10-1000 

100 

200 

^2000 

$$$ 

Robust (base frame) 

Bluetooth 

lk-3k 

10 

KS 100 

~ 1 million 

$$$ 

3 s wake-up 

BLE (wibree) 

lk 

10 

10 

~ 1 million 

$$ 

Low power Bluetooth 

Zigbee 

20-250 

10-75 

^30 

64 bit 

$$ 

Wakeup 15 ms 

RS485 

100-10 k 

1200 

5 

256 

$ 

Robust elec, protoc 

SPI 

1—12k 

10 

5 

Out of band 

$ 

Simple sync 5 wire 


vulnerable to software errors such as flooding a network 
with garbage messages. 

There are many choices for global communica¬ 
tion protocols falling into two categories: wired and 
wireless. Wired is the most common and has many 
possible protocols, mostly requiring that the bus is 
multidrop. Popular wired protocols include Ethernet, 
EtherCAT, RS-485, SPI, and CANbus. Popular wire¬ 
less protocols include Zigbee, Bluetooth, and 802.11. 
Important characteristics of communication protocols 
include the speed, real-time aspects, address space, and 
cost. A comparison is shown in Table 22.1 with typical 
values of common implementation. 

The importance of the speed depends (or often dic¬ 
tates) the architecture of distributed control. Communi¬ 
cations to individual modules can range from 100 times 


a second (for direct remote control) to several times 
per minute for higher level behavior control (e.g., hor¬ 
mone control [22.50]). Many protocols have automatic 
recovery from bad packets, which has obvious im¬ 
portance. Ethernet uses random backoff retransmission 
after a collision, which makes the protocol non-deter- 
ministic. This makes it difficult to have guarantees of 
real-time performance. The protocol is very fast so it 
is possible to ignore this as messages will get through 
with small latency with high probability. EtherCAT is 
Ethernet for Control Automation Technology that is 
better suited for real-time control. CANbus (Control 
Area Network bus) is a well-established bus used in 
the automotive technology that can be real-time, and 
robust, though it is typically slower than Ethernet or 
EtherCAT. 


22.4 Conclusion and Further Reading 

After more than a quarter century of research and de¬ 
velopment in modular robotics, a number of modular 
robotic systems have successfully entered the indus¬ 
try automation, education and entertainment markets. 
These successful modular robotic systems indicate that 
modular design offers advantages in the areas of prod¬ 
uct variety, application variety and creativity. However, 
the cost of such systems, no matter at the module level 
or at the system level, has room to be reduced to help 
mass adoption. 

The cost structure of a modular robot is closely 
linked with the design of the individual modules and 
the systemic architecture to be conceived as well as 
the market demand and expectation. From the develop¬ 
ment history of LEGO bricks to the Schunk Powercube 
modules, it is clear that module design functions are be¬ 
coming simpler (reducing cost) and increasing variabil¬ 
ity (increasing the user base). Hence, for reconfigurable 
modular manipulators, mobile systems, and self-recon- 
figurable modular robots, focusing on system designs to 
meet the expectation of the end user is the current trend. 

Besides system design, standardization of mechani¬ 
cal and communication interfaces is critical. Like other 
electronic and industrial products, electronic and com¬ 


munication interfaces for modular robots could adopt 
existing industry standards depending on the form 
factors, connecting performance, reliability, etc. The 
mechanical interface design normally does not have 
industry standard to follow due to variety in mechan¬ 
ical specifications on the form factor, loading capacity, 
rigidity of the joints. Hence, many novel mechani¬ 
cal connection designs can be explored for modular 
robots. 

Future research continues to explore increasing the 
number of modules. As they approach hundreds and 
thousands, there are interesting questions that arise as 
modules become more tightly coupled than current 
efforts [22.73]. Issues include how to deal with the in¬ 
creased likelihood that some modular elements are not 
functioning completely correctly. Biologically inspired 
mechanisms such as intentional cell-death and cell-re- 
placement may become a required part of very large 
systems. 

As the modular approach has increasingly large 
numbers of modules, there are many more configura¬ 
tions and resulting capabilities from those configura¬ 
tions. Future research will need to address the problem 
of determining appropriate or optimal configurations 
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for arbitrary tasks. This may lead to a better understand¬ 
ing of robotic tasks in general. 

The most recent review article on modular mo¬ 
bile robots can be found in [22.13]. In [22.30], the 


technical challenges and future of self-reconfigurable 
modular robots are reviewed. Interested readers may 
find further information on self-reconfigurable robots 
in [22.31]. 
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23. Biomimetic Robots 


Biomimetic robot designs attempt to translate 
biological principles into engineered systems, re¬ 
placing more classical engineering solutions in 
order to achieve a function observed in the natu¬ 
ral system. This chapter will focus on mechanism 
design for bio-inspired robots that replicate key 
principles from nature with novel engineering 
solutions. The challenges of biomimetic design 
include developing a deep understanding of the 
relevant natural system and translating this un¬ 
derstanding into engineering design rules. This 
often entails the development of novel fabri¬ 
cation and actuation to realize the biomimetic 
design. 

This chapter consists of four sections. In 
Sect. 23.1, we will define what biomimetic de¬ 
sign entails, and contrast biomimetic robots with 
bio-inspired robots. In Sect. 23.2, we will dis¬ 
cuss the fundamental components for developing 
a biomimetic robot. In Sect. 23.3, we will review 
detailed biomimetic designs that have been de¬ 
veloped for canonical robot locomotion behaviors 
including flapping-wing flight, jumping, craw¬ 
ling, wall climbing, and swimming. In Sect. 23.4, 
we will discuss the enabling technologies for 
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these biomimetic designs including material and 
fabrication. 


Biomimetics is a broad field that covers all ranges of 
robotics including robot structure and mechanics, actu¬ 
ation, perception, and autonomy. This section will focus 
on robots that mimic structure and movement principles 
found in nature to perform desired tasks in unstructured 
environments. 

Nature frequently inspires engineers to adopt solu¬ 
tions from biology for application to human challenges. 
Robots are built to perform certain tasks, and many tasks 
include moving from one place to another. The various 


modes of locomotion in nature have inspired robots to 
mimic these locomotion with a goal to overcome var¬ 
ious obstacles in the environment, and move around 
with the extreme agility similar to that found in nature. 
Mankind has engineered various modes of transporta¬ 
tion on ground, air, and water. On ground, wheeled ve¬ 
hicles are the most popular choice. In air, fixed wing 
airplanes and helicopters with rotating blades dominate, 
and in water, ships, and submarines propelled by similar 
rotating elements are most common. In contrast, nature 
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has different solutions to locomotion involving mov¬ 
ing legs, undulating fins, and flapping wings. Many of 
these biological locomotion mechanisms have analogies 
to engineered systems. Instead of using wheels, nature 
uses legs of different sizes, numbers, and impedances; 
humans and birds are bipedal, many mammals and rep¬ 
tiles move on four legs, and insects have six legs, and 
other arthropods have eight or more legs. These legs are 
coordinated to move on different surfaces in a stable 
manner. Snakes and worms move around without legs 


23.1 Overview 

The main purpose of locomotion in manmade machines 
is to deliver large payloads across long distances in 
the shortest possible time or with the minimal energy 
expenditure. There are roads, airports, shipyards, and 
other foundations that support this transportation, en¬ 
abling the structural design of cars, trains, planes, and 
ships to focus on transportation tasks. On the other 
hand, locomotion in nature is primarily for survival. 
Animals and insects have evolved to survive in various 
environments. Each species uses their mode of locomo¬ 
tion to hunt for food, find mates, and escape danger all 
in unstructured natural environments. Therefore, the re¬ 
quirements for biological locomotion are much more 
complicated than human transportation systems. 

Biomimetic robots try to mimic the structural char¬ 
acteristics and the principles of movement to be able 
to move around places where conventional machines or 
robots are unable to perform as needed. Animals can 
crawl on a rugged terrain at a high speed, can climb 
walls without a tether, can fly in cluttered environments 
and hover and perch as needed. 

Depending on the size of the species the optimal 
mode of locomotion and the underlying structure is dif¬ 
ferent. For example, jumping is found frequently in 
small insects to escape danger, since their small size 
makes it hard for them to escape quickly using other 
forms of locomotion. Jumping by a large animal is dif¬ 
ferent from jumping by small insects like a flea. Larger 
insects or animals tend to use their legs to run or crawl 
to escape the danger. Large birds flap their wings at 
a much lower frequency, and use gliding mode whereas 
bees and flies beat their wings continuously at a high 
frequency during flight. The structure is very differ¬ 


by creating waves with their body. Instead of fixed wings 
with jets and rotary propellers, birds, bats, and flying in¬ 
sects flap their wings. Fish create undulatory motions 
with their bodies to swim with agility far beyond con¬ 
ventional boats and submarines. Each of these biologi¬ 
cal locomotion modes are based on the nature’s funda¬ 
mental actuator: muscle. Muscles create linear motion 
that is coupled to the structures that generate locomo¬ 
tion. Therefore, these structures are closely related to the 
morphology of the muscles that move the structures. 


ent, where the wings of a bird have bones, muscle, and 
feathers versus the muscle-less and lightweight wings 
of insects. Therefore, when developing biomimetic 
robots, the size of the target species should be taken 
into account. 

Since nature has evolved to survive in extreme 
environments, there are many examples of extreme 
locomotion that are typically not possible with con¬ 
ventional engineering designs. For example, the wall 
climbing robot Stickybot uses the directional adhesion 
principle of a gecko to climb up smooth vertical walls. 
However, directional adhesion alone is not enough. The 
design has to enable even pressure distribution on the 
pad to make sure the pads are well in contact with the 
wall. These small details can be important to perfor¬ 
mance of biomimetic robots and should be considered 
carefully. 

Since robots are commonly built to perform tasks 
too tedious or dangerous for humans, it is natural to 
adopt designs found in nature for functionality where 
nature has already found a solution. In fact, many robot 
designs in this handbook, such as legged robots and 
robot hands and many examples of flying robots, un¬ 
derwater robots, and micro robots, are biomimetic. This 
chapter will focus on areas of biomimetic robot designs 
that have been recently expanding. Legged robots and 
robot hands have already formed large communities and 
are covered in separate chapters. There are various other 
forms of locomotion, e.g., crawling, jumping, swim¬ 
ming, and flapping, that are being researched to mimic 
the principle of movement and the biological structure. 
The structure and the components of these robots will 
be reviewed and discussed in this chapter. 


23.2 Components of Biomimetic Robot Design 

Biomimetic robot design begins with an understanding For the design of a biomimetic robot, the components 
of the underlying principles at work in a natural system, used in building the robot are important; the structural 



Biomimetic Robots I 23.3 Mechanisms 545 


design is constrained or enabled by the actuator system, 
the materials and the fabrication methods used to build 
the robot. 

Understanding of the principles of movement found 
in nature initiates the design of biomimetic robots. This 
can be studied by observing the kinematics and measur¬ 
ing forces in moving animals, or by understanding and 
modeling the dynamics. Substantial research on biolog¬ 
ical locomotion analyzes the kinematics of the animal. 
However, it is important to understand the dynamics 
behind the kinematics to be able to implement the 
dynamic principles into the robot design, which will en¬ 
able the robot to perform like an animal, instead of just 
looking like one. Full et al. proposed clock-driven, me¬ 
chanically self-stabilizing, compliant sprawled-posture 
mechanics to explain a cockroach racing seemingly ef¬ 
fortlessly over a rough surface [23.1, 2]. Ellington built 
a system that can measure the force generated by flap¬ 
ping wings to understand the thrust generated by insect 
wings [23.3]. These studies in biology have influenced 
robotics researchers, and inspired them to build hexa¬ 
pod and flapping-wing robots. 

To implement biological principles in engineered 
systems, components that can create performance sim¬ 
ilar to that of nature is required. Nature’s biological 
structures are composed of various materials, e.g., tis¬ 
sues, bones, cuticles, flesh, and feathers. These ma¬ 
terials are replaced with engineered materials, e.g., 
metal, plastics, composites, and polymers. New en¬ 
gineered materials are being developed that exhibit 
similar properties to natural materials, which will en¬ 
able life-like robots. However, the performance of the 
biomimetic robot can be similar to nature without ac¬ 
tually mimicking the exact structure. Therefore, it is 
important to decide to what level biomimicry is re- 

23.3 Mechanisms 

Developing multilegged robot has been achieved by 
using specific design which enables some function or 
biomimetic model inspired by multilegged insects. Es¬ 
pecially, the ability of a cockroach enabling to run on 
a rough surface at a high speed has inspired to develop 
a series of multilegged robots. They are capable of 
maintaining stability during locomotion at a high speed 
(relative to their body length). 

23.3.1 Legged Crawling 

Some types of cockroaches can achieve speed up to 
50 body length per second and can crawl on un¬ 
even terrain, overcoming obstacles far higher than their 
height [23.2], RHex (Fig 23.1a) [23.4] is one of the 


quired. Large robots tend to be built with conventional 
mechanical components such as motors, joints, and 
linkages made of metal. The challenge is in building 
meso-scale robots, where the conventional mechani¬ 
cal components become ineffective, due to friction and 
other inefficiencies. As the robot becomes smaller, it 
is beneficial to mimic not just the kinematics seen in 
nature, but also the structure at the component level. 
Another challenge for small scales is actuation - the ac¬ 
tuator must be as effective as the muscle even at small 
scales. 

A number of new manufacturing methods have be¬ 
come available which also enable new designs that 
were not previously possible by more classical machin¬ 
ing methods and nuts-and-bolts assembly techniques. 
Examples include shape deposition manufacturing and 
smart composite microstructures. In many cases, these 
new fabrication processes facilitate the construction of 
novel biomimetic robots beyond what was possible by 
conventional means. 

Levels of mimicry can vary depending on the com¬ 
ponents used in the design. The robot can simply look 
like an animal, but not perform similarly to the mim¬ 
icked animal. On the other hand, once the principle is 
mimicked the robot does not have to look like the ani¬ 
mal, but it can still perform a task in a similar manner 
as the animal. The different ways of mimicking allows 
each robot to have its own characteristic. For example, 
a flea uses what is called a torque reversal for jump¬ 
ing, which is a unique method of storing and releasing 
large amount of energy. In nature, this method is possi¬ 
ble with muscles and a lightweight-legged structure. To 
mimic the flea’s locomotion, we need an actuator that 
is comparable to muscles and a fabrication method that 
allows us to build a small-scale rigid structure. 


first robots to implement cockroach-like characteris¬ 
tics. It is a hexapod crawling robot with C-shaped 
legs, so it is suitable for walking on uneven terrain 
or large obstacles, as shown in l-E&MFlilitTiLl. Mini- 
Whegs (Fig 23.1b) [23.5] also have unusual wheel with 
three spokes. Because of the wheel-spoke structure, the 
gait passively adapts to the terrain similar to climb¬ 
ing cockroaches, as shown in l-s&MlliiilEliiJi [23.15]. 
Sprawlita (Fig 23. lc) [23.6] is a hexapod crawling robot 
that uses pneumatic actuators on each leg and passive 
rotary joints so that it can achieve dynamic stability. 
It weighs 27 kg and its speed is over 3 body length/s 
or 550mm/s. iSprawl (Fig 23. Id) [23.7] is hexapod 
crawling robot that uses extension of each legs during 
crawling. This robot is driven by electric motors and 
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Fig.23.1a-d Crawling robots inspired by cockroach (a) RHex (after [23.4]), (b) Mini-Whegs (after [23.5]), (c) Sprawlita 
(after [23.6]), (d) iSprawl (after [23.7]) 



Fig.23.2a-g Crawling robots inspired by cockroach, (a) DASH (after [23.8]), (b) RoACH (after [23.9]), (c) DynaRoACH 
(after [23.10]), (d) OctoRoACH (after [23.11]), (e) HAMR3 (after [23.12]), (f) centipede-like modular robot (af¬ 
ter [23.13]), (g) crawling robot through the integration of microrobot technologies (after [23.14]) 


flexible push-pull cables. Rotation of the motor causes 
the legs to extend or contract. It weighs 300 g and is able 
to crawl 15 body length/s or 2.3 m/s, as can be seen in 

Cockroach-based designs would improve the per¬ 
formance of millimeter or centimeter scale crawl¬ 
ing robots which face inefficiency with conventional 
mechanisms. DASH (Fig 23.2a) [23.8] is a hexapedal 
crawling robot fabricated using SCM (Smart compos¬ 
ite manufacturing) process. This robot has only one 
electric motor coupled to a four-bar linkage pushing 
the legs in an elliptical crawling motion, as shown 
in DASH weighs 16.2 g and achieves 



Fig.23.3a,b The latest version of crawling robots which shows im¬ 
proved performance (a) HAMRV (after [23.16]), (b) VelociRoACH 
(after [23.17]) 


speed up to 15 body length/s or 1.5 m/s. RoACH 
(Fig 23.2b) [23.9] is a hexapod crawling robot that 
imitate cockroach’s alternating tripod gait. A typical 
cockroach gait involves the ipsilateral front leg, hind 
leg, and contralateral middle leg moving simultane¬ 
ously. Two sets of three legs tread on a surface in 
turn - generating the alternating tripod gait. RoACH 
uses two shape memory alloy (SMA) wire actuators 
to contract the body in two orthogonal directions. 
Contraction of the two actuators results in motion 
of legs through a four-bar linkage. The legs repeat¬ 
edly operate swing and stance motions according to 
the sequential stimulation of the two actuators and it 
can be seen in It weighs 2.4 g and is 

able to crawl 1 body length/s or 3cm/s. DynaRoACH 
(Fig 23.2c) [23.10] has six legs driven by one DC 
motor. It has passive dynamics similar to RoACH to 
achieve better locomotion performance. Lift motion 
is achieved by a slider crank mechanism and swing 
motion by a four-bar mechanism. Like RHex, it uses C- 
shape legs so that the robot has lower vertical stiffness, 
lateral collapsibility for obstacle climbing, and more 
distributed ground contact. It weigh 24 g and is 100 mm 
long and is capable of speeds up to 14 body length/s 
or 1.4m/s. OctoRoACH (Fig 23. 2d) [23.11] is quite 
similar to DynaRoACH, but it has two motors driv- 
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ing the legs on each side. It has eight legs so that 
it maximizes pitch stability using two motors. Oc- 
toRoACH weigh 35 g and is 100 mm long. HAMR3 
(Fig 23. 2e) [23.12] uses nine piezoelectric actuators. 
Each leg performs swing and lift motion through 
two decoupled piezoelectric actuator through four-bar 
slider-crank mechanisms and spherical five-bar mecha¬ 
nisms (l<s>M3H2!E33l). This robot weighs 1.7 g and has 
speeds up to 0.9 body length/s or 4.2 cm/s (4.7 cm long 
robot). Especially, the HAMR is manufactured using 
a method inspired by pop-up books that enables fast 
and repeatable assembly. Multilegged robots inspired 
by centipede using mechanism similar to alternative 
tripod gaits is also developed (Fig 23. 2f) [23.13]. Its 
several gait patterns that differ in gait frequency and 
phase are described in Using microrobot 

technologies developed until 2006, integrated structure 
was developed although the structure was not yet tested 
(Fig 23. 2g) [23.14], 

With more experiments and research about mo¬ 
tion of cockroaches, the cockroaches like robots 
can extremely improve their performance by revi¬ 
sion using biomimetic model. HAMRV, which is the 
most recent version of HAMR [23.16], can move 
10.1 body length/s (44.2 cm/s) which is remarkably 
improved compared to preceded versions are capable of 
running at 0.9 body length/s. It is even capable of ma¬ 
neuverability and control at both low and high speeds. 
VelociRoACH (Fig 23.3b) [23.17] which is the latest 
version of RoACH can run 2.7 m/s extremely higher 
speed relative to previous version (l^sfcMSQlSEiEI). 

23.3.2 Worm-Like Crawling 

Worm-like crawling motion can be separated into two 
categories: peristaltic crawling and two-anchor crawl¬ 
ing. Worms exhibiting peristaltic crawling locomotion - 
such as earth worms - can move through small tunnels 
with limited space. Therefore, mimicry of this locomo¬ 
tion imposes similar characteristics to the robot and it 
has the potential to be used in small and harsh environ¬ 
ment such as collapsed disaster site or inside a pipe line, 
which A schematic of peristaltic locomotion is shown in 
Fig. 23.4. By sequentially changing the volume of the 
body, the whole structure generates the moving motion. 
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Fig. 23.4 Peristaltic locomotion (after [23.19]) 


The key design issue in mimicking peristaltic motion 
is how to create sequential volume change. Many re¬ 
searchers have tried various creative methods to solve 
this problem. Boxerbaum et al. built a robot with a mesh 
structure, and using a single motor and wire, made 
a partial volume change of the robot which realized 
a crawling motion (Fig. 23.5a) [23.18]. Seok et al. used 
a SMA coil spring actuator to change the segmented 
volume, and also used a mesh tube as the body structure 
(Fig. 23.5b) [23.19]. Menciassi et al. also used a SMA 
coil spring actuator, but they used a micro hook to en¬ 
hance the friction force (Fig. 23.5c) [23.20]. 

Two-anchor crawling is a locomotion method used 
by inchworms. This locomotion mode is not fast, but 
it can overcome nearly any complicated topology. With 
an appropriate gripping method, it not only can climb 
vertical walls, but also can cross gaps. There are two 
key design issues for generating a two-anchor crawling 
motion: the first is how to change the shape of the waist 
and the second is how to anchor and unanchor to the 
surface. Kotay and Rus simply used an electric motor to 
articulate the waist motion and used an electromagnetic 
pad as the anchoring method (Fig. 23.6a) [23.21]. Us¬ 
ing an electromagnetic pad, the robot can climb a steel 
structure. Cheng et al. used a tendon-driven mechanism 
with a compressible body and an anisotropic friction 
pad to generate motion (Fig. 23.6b) [23.22]. Using sym¬ 
metrical or asymmetrical winding of the wire attached 
on the both side, the robot can make forward or steering 
movements. Koh and Cho used SMA coil spring actu¬ 
ators to control the waist motion (Fig. 23. 6d) [23.23]. 



Fig. 23.5 (a) Robot with peristaltic 
motion (after [23.18]). (b) Meshworm 
robot (after [23.19]). (c) Biomimetic 
miniature robotic crawler (af¬ 
ter [23.20]) 
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Fig.23.6a-e Two anchor crawl¬ 
ing robot, (a) The inchworm robot 
(after [23.21]). (b) The soft mo¬ 
bile robot with thermally activated 
joint (after [23.22]). (c) Treebot 
(after [23.24]). (d) Omega shaped 
inchworm inspired crawling robot (af¬ 
ter [23.23]) (e) GoQBot (after [23.25]) 


The body of the robot is made by a single sheet design 
with glass fiber composite, and a folding pattern de¬ 
signed to enable steering motion even though the robot 
was built by a single sheet. Lin et al. realized a robot 
with two-anchoring motion, but he also added a rolling 
locomotion to solve the speed limitation of previous 
two-anchor motion demonstrations (Fig. 23. 6e) [23.25]. 
Lam and Xu realized another method to generate the 
waist motion. The robot used a backbone rod coupled 
to an electrical motor. By controlling the length of the 
backbone using the motor, the robot can control the po¬ 
sition of the anchoring point (Fig. 23.6c) [23.24]. 



Fig.23.7a-d Three snake-like robots, (a) AMC-III, Shigeo Hi- 
rose, Fukushima Robotics Lab, Tokyo Institute of Technology 
(after [23.26]), (b) Slim Slime Robot II (SSR-2), Shigeo Hi- 
rose, Fukushima Robotics Lab, Tokyo Institute of Technology 
(after [23.27]), (c) Modular snake robot, Howie Choset, Biorobotics 
Lab, Carnegie Mellon University (after [23.28]), (d) AMC-R5, 
Shigeo Hirose, Fukushima Robotics Lab, Tokyo Institute of Tech¬ 
nology (after [23.29]) 


23.3.3 Snake Robots 

Studies on the locomotion of snakes began in middle 
of the 20th century [23.30-32]. Snakes are limbless, 
slender, and flexible [23.33]. Their locomotion gives 
them adaptability and mobility through land, uneven 
ground, narrow channel, pipes, and even water, and 
even flying between trees [23.34, 35]. An advantage of 
snake-like locomotion is the great versatility and free¬ 
dom of movement with numerous degrees of freedom 
[23.36]. Additionally, snake locomotion could be ef¬ 
ficient compared to legged animals, because there is 
no lifting of the center of gravity or limb accelera¬ 
tion [23.37]. In 1970s, Hirose developed a continuous 
locomotion model and a snake-like robot called the 
Active Cord Mechanism (ACM) [23.38]. After the Hi- 
rose’s ACM robot, snake-like robots have been widely 
studied. In 1972, ACM-III (Fig. 23.6) was developed 
and it was the first robot that mimics the serpentine mo¬ 
tion of real snakes [23.38]. The recent versions of ACM 
are in 

Locomotion of snake-like robots can be categorized 
into the following different types: serpentine motion, si¬ 
nus lifting, pedal wave, side-winding, spiral swimming, 
lateral rolling, lateral walking, mixture lean serpentine, 
and lift rolling motions. 

The first generation of snake-like robots could 
only achieve motion on planar surfaces. These designs 
quickly evolved and current snake-like robots can go 
upward within narrow pipes and can climb and hold on 
to trees like [23.39]. To facilitate travers¬ 

ing large obstacles, some robots have added actuated 
articulation between each joint [23.40]. In addition, 
there are some snake-like robots that can swim in water. 
With a firmly waterproofed body, these robots can swim 
with spiral and sinusoidal locomotion patterns [23.38]. 

Today, mechanism design of snake-like robots can 
be classified with following five different types: active 
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Fig. 23.8 (a) Configuration of the CPG model (after [23.41]). (b) Salamandra robotica I, Auke Jan Ijspeert, Biorobotics 
Lab, Ecole Polytechnique Federale de Lausanne (after [23.41]). (c) Salamandra robotica II, Auke Jan Ijspeert, 
Biorobotics Lab, Ecole Polytechnique Federale de Lausanne (after [23.42]) 


bending joint type, active bending and elongation joint 
type, active bending joint and active wheel type, passive 
bending joint and active wheel type, and active bend¬ 
ing joint, and active crawler type [23.43]. Each of these 
types of snake-like robots consist of a number of seri¬ 
ally connected joints. Therefore, snake-like robots are 
easy to modularize with their joints [23.43]. 

Most snake-like robots are equipped wheels that are 
actively or passively driven. Recently, wheelless snake¬ 
like robots have been studied [23.32]. These robots 
move with undulatory motion, especially lateral un¬ 
dulation that can be observed in real snakes. Some 
snake-like robots are actuated with smart materials, 
such as shape memory alloys and IPMCs, rather than 
motors [23.44]. Snake-like robots roll to avoid obstacles 
and interact with environment. They make waveforms 
with their body for propulsion. Therefore, it is impor¬ 
tant to define the configuration of the body of the robots. 
Measuring yaw (pitch) and roll angles are important for 
controlling the snake robots. Tilt sensors, accelerome¬ 
ters, gyroscope, and joint angle sensors are typically 
leveraged to control the robot [23.37]. Tactile sensors 
attached to the contact area or outside of the body of 
the snake-like robots have been used to measure surface 
contact forces at each joint, providing more information 
to the controller. Additionally, measuring contact forces 
could be useful for active and adaptive grasping of the 
snake-like robot [23.45]. 

However, due to the high degrees of freedom of the 
snake-like robots, designing controllers is not easy even 
for flat surface locomotion. Efforts without complex 
sensors and controllers are in Because of 


this, contrasting with the advantages of locomotion of 
real snakes - which can move in uneven environments - 
most existing snake-like robots are developed based on 
flat surface movement [23.46]. 

A major motivating application for snake-like 
robots is the exploration of hazardous environment that 
are inaccessible to humans. In particular, industrial 
inspection of pipes and ventilation tubes, and chem¬ 
ical channels are key operating environments. There 
is also the potential for snake-like robots as medical 
devices, such as minimal invasive surgery device and 
laparoscopy and endoscopy [23.37]. For these appli¬ 
cations, snake-like robots require their outer skin to 
hermetically seal the internal components [23.40]. 

There are numerous challenges for snake-like 
robots. For greater reliability, robustness and controlla¬ 
bility, the mechanisms and configuration of the snake¬ 
like robots could be simplified. To use the snake-like 
robots for exploration and inspection applications, rout¬ 
ing of external wires for electronics and power to the 
robot is an important consideration. Finally, with a large 
number of degrees of freedom typical of snake-like 
robots, designing efficient control strategy is large is¬ 
sue [23.37], 

Similar to undulatory locomotion of snakes, body 
of salamanders makes S-shape standing waves. They 
are capable of rapidly switching between swimming 
and walking locomotion. Their locomotion in aquatic 
and terrestrial environments is generated by a central 
pattern generation (CPG) and stimulation of a mesen¬ 
cephalic locomotor region (MLR) located in the mid¬ 
brain (Fig. 23.7 and l43®!ihl£ii£l) [23.41]. There were 
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some efforts to produce similar swimming and walking 
gaits to real salamander with robotic salamanders. The 
salamander robots with the mathematical CPG model, 
DC motors, and oscillators could produce similar kine¬ 
matics to real salamanders [23.42] (Fig. 23.8). 

23.3.4 Flapping-Wing Flight 

Flapping-wing flight is a common inspiration for 
biomimetic aerial robots. This is due to the agility of 
natural flyers such as birds, bats, and insects. Advances 
in the understanding of the aerodynamics of flapping¬ 
wing based on hydrodynamic theories and experimental 
results have provided insights into thrust production in 
flapping-wing animals [23.47]. From observation of the 
flapping motion exhibited by insect flight, the transla¬ 
tional, and rotational motions of the wing produce lift 
forces at high angles of attack - beyond what is typ¬ 
ical for fixed wing aircraft. The formation of a large 
vortex at the leading edge of the wing and the recaptur¬ 
ing of shed vortices by properly timing of the swing 
enhance the resultant lift force [23.48]. Figure 23.9 
shows the representative motion of an insect wing dur¬ 
ing flight and the vector formation of hydrodynamic 
forces. Such characteristics learned from nature inspires 
the design of wing-driving systems in flapping-wing 
flying robots. 

Most flapping mechanisms are constructed from 
electromagnetic rotary motors driving crank-rocker 
linkages to flap the wings. Examples include the 
DelFly II [23.49], Robot dragonfly [23.50] and Nano 
Hummingbird [23.51] as shown in Fig. 23.10. As can 
be seen in losBHiEiEEl, DelFly II employs four-wing 
morphology where two wings on each side perform 


a clapping motion during each period. This contributes 
to lower power consumption and the low rocking ampli¬ 
tude of the fuselage. The DelFly II model used a crank 
mechanism such that the gear axis is perpendicular to 
the flying direction, for overcoming phase differences 
between the two sets of wings and this difference in¬ 
duced rotational movement on the fuselage [23.49]. In 
related work, the Robot Dragonfly [23.50] is inspired 
from a dragonfly’s hovering capabilities and utilizes 
a tandem wing model. Each transmission link is at¬ 
tached to gear mechanisms that can make wing motions 
similar to those of a dragonfly. The Nano Humming¬ 
bird uses single pair of flapping wings as with real 
hummingbirds. The flapping mechanism is a dual lever, 
string-based flapping mechanism. Wing rotation modu¬ 
lation for control of the wing attack angle is achieved by 
the two adjustable stops which limit how far each wing 
can rotate [23.51]. Alternative mechanisms use oscillat¬ 
ing actuators and flexure-based transmissions systems 
(these mechanisms are discussed in Sect. 23.4.2). 

As the robot scale decreases, previous flapping 
mechanisms have been difficult to adapt due to man¬ 
ufacturing challenges and the physics of scaling. The 
Harvard robotic fly shows promising fabrication and 
design processes to build the small scale flapping air 
vehicle as shown in Fig. 23.11 [23.52]. Previous ver¬ 
sions of the Harvard robotic fly had three degrees of 
freedom, only one of which was actuated. The angle 
of attach of two the wings are passively controlled by 
a compliant flexure joint connected to the transmis¬ 
sion. The wing beat frequency is tuned at 110 Hz in 
resonance. Through advances in meso-scale manufac¬ 
turing methods, the capabilities of the robotic fly have 
been dramatically extended and unconstrained flight 


Total 



Fig. 23.9 Diagram of wing motion 
indicating magnitude and orien¬ 
tation of the aerodynamic force 
(after [23.48]) 



Fig. 23.10 (a) Deify II (after [23.49]), 
(b) Robot Dragonfly [23.50], (c) Nano 
Hummingbird (after [23.51]) 
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Fig. 23.11 (a) Harvard robotic fly, (b) Illustrations of thorax mechanisms, (c) and transmission (after [23.52]) 


has been demonstrated like in an 80 mg 

robot as discussed in Sect. 23.4.2. 

Flapping-wing locomotion can also be extended 
to multimodal locomotion in combination with 
other mechanisms. In the DASH+Wings shown in 
Fig. 23.12 [23.53], the combination of wing flapping 
and crawling compliments each other for improving 
agility and stability. This hybrid robot improves per¬ 
formance of the maximum horizontal running speed 
in a factor of two and the maximum climbing incline 
angle by a factor of three. 

23.3.5 Wall Climbing 

Climbing and maneuvering on vertical surfaces present 
a difficult challenge. However, this locomotion mode 
is needed in many areas such as shipping, construction, 
and terrestrial locomotion in natural environment. Early 
attempts involved the use of suction cups, magnets, 



Fig. 23.12 DASH+Wings (after [23.53]) 


or pressure-sensitive adhesives to implement climbing. 
More recently, claw, spines and sticky pads inspired by 
nature have been used. Climbing insects and animals 
inspired many researchers. Insects and reptiles em¬ 
ploy small spines that catch on fine asperities. Geckos 
and some spiders employ large numbers of very fine 
hairs that achieve adhesion based on van der Waals 
interaction. 

Early in the 1990s, nonbiomimetic wall climbing 
robots, i. e., the Ninja-1, RAMR, and Alicia were de¬ 
veloped using suction cup. Ninja-1 attaches to a wall 
making use of a suction mechanism (Fig. 23.13a). The 
main mechanism consists of a three-dimensional (3-D) 
parallel link, a conduit-wire-driven parallelogram, and 
valve-regulated multiple sucker that enabled the robot 
to attach the surface with grooves [23.54]. RAMR used 
underactuation to remove the redundant actuators to 
drive the small two-legged robot [23.59]. The Alicia 
robot was developed for a variety of applications such 
as maintenance, building inspection, and safety in pro¬ 
cess and construction industries. An aspirator is used to 
depressurize a suction cup, so the whole robot can ad¬ 
here to the wall like a standard suction cup. The Alicia3 
robot use three of the Alicia II modules, allowing the 
whole system to better deal with obstacles on the target 
surface [23.60]. REST is an exceptional case that ap¬ 
plies electromagnets instead of suction cup. It climbed 
only ferromagnetic wall using electromagnetic four legs 
with 12-DOF [23.61]. 

The effective wall climbing mechanisms of animals 
and insects have inspired development of biomimetic 
wall climbing robots. Typical robots utilizing bio¬ 
inspired spines found in climbing insects and cock¬ 
roach are Spinybot and RiSE. Spinybot in 
climbs hard vertical surfaces including concrete, brick, 
stucco, and masonry with compliant microspine arrays 
(Fig. 23.13b). It can exploit small asperities (bumps 
or pits) on the surface. The sequence of motions is 
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Fig. 23.13 (a) Ninja-1 (after [23.54]), 

(b) Spinybot II (after [23.55]), 

(c) RiSE (after [23.56]), (d) Gecko 
adhesive system (after [23.57]), 

(e) Stickybot (after [23.58]) 


accomplished using an under-actuated mechanism con¬ 
sisting of a single rotary RC servo motor and several 
spines independently engaging asperities on the con¬ 
crete surface [23.55]. RiSE is a hexapod robot capable 
of locomotion on ground and vertical surfaces such as 
brick, stucco, crushed stone, and wood, as shown in 
(Fig. 23.13c) [23.56]. To climb a vertical 
wall, it uses microspines inspired by cockroach’s tarsus 
structure. In addition, it held its center of mass close to 
surface to minimize the pitch-back moment. RiSE also 
employed a static tail to reduce disparity from the pull- 
in forces experienced by the different legs [23.56, 62]. 

Insects and geckos can provide inspiration for novel 
adhesive technology and for the locomotion mecha¬ 
nisms employed during climbing. Geckos are able to 
climb rapidly up smooth vertical surfaces and biologists 
reveal that a gecko’s foot has nearly five hundred thou¬ 
sand keratinous hairs or setae. And measured adhesive 
force values show that individual seta operate by van 
der Waals forces. The gecko’s toe uncurling and peel¬ 
ing suggests that two aspects of setal function increase 
effectiveness [23.63]. The subsequent study shows that 
the linear relation between adhesion and shear force is 
consistent with a critical angle of release in live gecko 
toes and isolated setal arrays (Fig. 23.13d). And the 
frictional adhesion model provides an explanation for 
the very low detachment forces observed in climbing 
geckos that does not depend on toe peeling [23.57]. 

Stickybot, Mini-Whegs, Geckobot, and Waalbot are 
prototypical robots that leverage biomimetic dry adhe¬ 
sives. Stickybot climbs smooth vertical surfaces such 
as glass, plastic, and ceramic tile at 4 cm/s, as shown 
in (Fig. 23.13e). The robot employs sev¬ 

eral design principles adapted from the gecko including 
a hierarchy of compliant structures, directional adhe¬ 


sion. The undersides of Stickybot’s toes are covered 
with arrays of small, angled polymer stalks. They read¬ 
ily adhere when pulled tangentially from the tips of 
the toes toward the ankles. When pulled in the oppo¬ 
site direction, they release [23.58]. Mini-Whegs uses 
wheel-legs with compliant, adhesive feet for climb¬ 
ing. The foot motion mimics the foot kinematics of 
insects, in order to test new bio-inspired adhesive tech¬ 
nologies and novel, reusable insect-inspired polymer 
(polyvinylsiloxane) [23.67]. Geckobot has kinematics 
similar to a gecko’s climbing gait. It uses a novel peel¬ 
ing mechanism of the elastomer adhesive pads, steering 
mechanisms and an active tail for robust and agile 
climbing [23.68,69). Waalbot used two actuated legs 
with rotary motion and two passive revolute joints at 
each foot. The robot has ability to climb on nonsmooth 
surfaces as well as on inverted smooth surfaces using 
gecko-like fibrillar adhesives and passive peeling. It is 
also capable of plane-to-plane transitions and steering 
to avoid obstacles [23.70]. 

Different approaches for climbing include elec¬ 
troadhesion. Electroadhesives use a novel clamping 
technology called compliant electroadhesion - a form 
of electrically controllable adhesion. This involves in¬ 
ducing electrostatic charges on a wall substrate using 
a power supply connected to compliant pads situated on 
the moving robot. This generates high clamping forces 
that are around 0.2— 1.4 N supported for a one square 
centimeter clamp area, depending on the substrate. 
Regarding power considerations for electroadhesion, 
assuming 50% conversion efficiency, in the worst-case 
scenario, two AAA batteries weighing 7.6 g each can 
hold up a robot in perch mode for almost one year. 
Electroadhesion combined with a conventional wheeled 
robot results in inchworm-style wall climbing [23.79]. 
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Fig. 23.14 (a) DynoClimber (after [23.64]). (b) A dynamic template for climbing (after [23.65]). (c) CLASH with gecko 
pad (after [23.66]) 


Dynamic wall climbing is the next challenge for 
wall climbing robots because previous robots were slow 
and in most instances restricted to targeted surfaces. 
For dynamical climbing originating in biology, pendu¬ 
lous climbing model was proposed (Fig. 23.14b). This 
model abstracts remarkable similarities in dynamic wall 
scaling behaviour exhibited by radically different ani¬ 
mal species such as cockroaches and geckos [23.65]. 
The findings suggest that animals employ large lateral 
in-pulling forces and body rotations to achieve fast, 
self-stabilizing gaits. DynoClimber displays the feasi¬ 
bility of adapting the dynamics to robot that runs verti¬ 
cally upward (Fig. 23.14a). A novel bi-pedal dynamic 
climber can scale a vertical wall fast accompanying 
while achieving dynamic stability. For dynamic climb¬ 
ing, this robot consists of a DC motor, a crank slider 
mechanism and passive-wrist springs so its climbing 
at speeds 0.67m/s (1.5 body lengths/s) [23.64]. The 
climbing robot CLASH has modified DASH platform 
but it actuates in horizontal direction to reduce height 
(7 mm from robot bottom). One of the key points is 
passive foot mechanism. When climbing upward, the 
foot hangs its spines on the surface and then retracts 
passively. This increases the shear and normal forces 
and enables climbing on loose cloth at 15 cm/s speed, 
as shown in loatffiEiEH [23.80]. The next version 
of CLASH has a foot that consists of an 18 x 15 mm 
pad of microfabricated PDMS (polydimethylsiloxane) 
ridges inspired by gecko feet (Fig. 23.14c). The ankle 
is an isosceles-trapezoid four-bar that creates a remote 
center-of-motion. This mechanism allows the foot to 
make coplanar contact with the surface and reduces roll 
peeling moments [23.66]. 

23.3.6 Swimming 

Underwater vehicles have been made to achieve marine 
explorations, surveillance, and environmental monitor¬ 
ing. Most underwater vehicles employ propellers for 


propulsion and these vehicles have shown great perfor¬ 
mance with respect to the cost of transport. However, 
efficiency and manoeuvrability in confined areas is 
problematic for most surface or underwater vehicles. 
Moreover, propeller-driven vehicles risk tangling when 
moving thourgh environments with debris and vegeta¬ 
tion. To resolve these issues, researchers have tried to 
replace the conventional rotary propellers with undula- 
tory movement inspired by fish (Fig. 23.15). 

The undulatory movement of fish provides two 
main advantages - manoeuvrability in confined areas 
and high propulsive efficiency. The main difference be¬ 
tween existing propeller and undulatory movement is 
turning radius and speed. Fish can turn with a radius 
1/10 of their body length, while propeller-driven ships 
require a much larger radius. Accordingly, the turn¬ 
ing speed of fish is much faster than ships. Beyond 
manoeuvrability, the driving efficiency in biological 
swimmers also show more improvement over man¬ 
made systems [23.81]. 

To achieve fish-like swimming motion, various 
mechanisms have been employed such as linkage mech¬ 
anisms and compliant mechanisms. Barrett first pro¬ 
posed a RoboTuna by using six servo motors and 
eight linkages [23.71]. Morikawa et al. built a robotic 
fish mimicking the caudal musculo-skeletal structure of 
a tuna with two rubber pneumatic artificial muscles and 
a multijoint bending mechanism [23.82]. A robotic dol¬ 
phin was designed with four links and six servo motors 
to mimic the dorsoventral movement of a real dol¬ 
phin [23.83]. Low developed a fish robot to generate ar¬ 
bitrary undulating waveforms, by connecting ten servo 
motors in series by linking them with sliders [23.74]. 
Liu and Hu developed a robotic fish mimicking the body 
motion of carangiform fish by using three servo mo¬ 
tors on each joint, as shown in [23.72]. 

Yang et al. presented Ichthus V5.5 by using 3-DOF se¬ 
rial link-mechanism with servo motors on each joint for 
propulsion, as shown in |43>M2ES!E3I. Ichthus V5.5 has 
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Fig.23.15a-h Design of robotic fin (a) Robo Tuna (after [23.71]), (b) G9 serise robotic fish (after [23.72]), (c) Ichthus 
(after [23.73]), (d) Undulating robotic fin rays (after [23.74]), (e) Soft robotic fish (after [23.75]), (f) Robotic mata ray 
(after [23.76]), (g) Turtle-like swimming robot (after [23.77]), (h) Robotic fin rays (after [23.78]) 


several sensors to navigate autonomously in the real en¬ 
vironment such as rivers [23.73]. 

Beyond linkage mechanisms, several researchers 
have employed compliant materials in their designs to 
make the undulatory motion without complicated link¬ 
age structures. Salumae and Kruusmaa implemented 
swimming kinematics of a trout by simply adjusting 
the compliance of a flexible fin with a single ac¬ 
tuator [23.84]. Alvarado and Yoiicef-Toumi designed 
a robotic fish with a simple and robust mechanism, 
using a compliant body that was approximated by 
a continuous cantilever beam to generate a fish-like os¬ 
cillating motions [23.85]. This simple design achieved 


biomimetic locomotion using only one servomotor, 
while most other robotic fish use several motors to 
achieve biomimetic modes of swimming. Marchese 
et al. employed a compliant body with embedded ac¬ 
tuators and used a novel fluidic actuation system that 
drives body motion, as shown in loKUHiZISi [23.75], 
Park et al. presented a guideline for optimizing the fin 
to maximize the thrust generated by a compliant fin. 
The half-pi phase delay condition describes the con¬ 
dition that the thrust is maximized regardless of the 
shape of the fin, driving frequency, and amplitude. They 
also presented a variable-stiffness flapping mechanism 
to improve the performance of a compliant fin while 
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Fig.23.16a,b Escapement cam mechanism (a) Grillo (Ver. 1) (after [23.86]) and (b) 7 g jumping robot (after [23.87]) 


the operating conditions change [23.88]. Tendons were 
used to vary the stiffness and the attachment point is 
determined based on the anatomy of a dolphin’s fluke. 
Several fish robots that use smart actuators to create 
undulating motion have also been investigated [23.89]. 
Wang et al. embedded shape memory alloy (SMA) wire 
actuator to create flexible bending and investigated the 
musculature of a cuttlefish fin to aid the design of the 
biomimetic fin [23.90]. Chen et al. mimicked manta 
ray by using ionic polymer-metal composite (IPMC) 
as artificial muscles. They embedded an IPMC mus¬ 
cle in each pectoral fin and a passive PDMS membrane 
to lead to an undulatory flapping motion on the fin, 
as shown in [23.76]. Kim et al. [23.77] 

used a smart soft composite (SSC) structure to generate 
bending and twisting motions in a simple, lightweight 
structure. Lauder et al. designed a robotic fish caudal 
fin with six independently moving fin rays based on the 
anatomy of bluegill sunfish and presented that the cup¬ 
ping motion produced greater thrust than others such as 
W-shaped, undulation, and rolling [23.78]. They used 
five different sets of fin rays and measured thrust by 
varying the motion program. In addition, Lauder et al. 
used a flexible plastic foil to explore the effects of 
changing swimming speed, foil length, and shape of the 
foil-trailing edge on undulatory locomotion [23.91]. 

23.3.7 Jumping 

In nature, many animals use jumping as a locomotion 
strategy. Jumping has the advantages of overcoming 
large obstacles and avoiding predators quickly and in¬ 
creasing the chances of survival. Robots also experience 
challenges in overcoming obstacles larger than the char¬ 
acteristic dimension of the robot. To find solutions for 
this, many researchers have developed jumping robots 
inspired by nature. 

The jumping process requires large amounts of en¬ 
ergy to be released instantaneously. However, muscle 
has limited reaction speed - achieving a maximum 
acceleration of 15 ms -2 . Therefore, many small crea¬ 
tures, such as insects, have adapted special elastomers 
for energy storage to generate large accelerations in¬ 


stead of using muscles. On the other hand, most large 
creatures, such as human that have relatively long legs, 
primarily use large muscles that can generate sufficient 
large force to swing long legs quickly. 

In small-scale jumping, to achieve large instanta¬ 
neous acceleration, the jumping process has two steps: 
1) slow energy storage and 2) rapid release of the stored 
energy. Escapement cam mechanism is widely used to 
achieve these two steps. It consists of a spring, a ra¬ 
dius varying cam, a motor, and a gearbox for torque 
amplification. The motor rotates the cam slowly, but 
powerfully, toward the direction of compressing or ex¬ 
tending the spring. At the final portion of the cycle, the 
cam’s radius returns to the initial state instantly, releas¬ 
ing the spring causing the robot to jump. Grillo (Ver.l) 
(in l<g&»'ii>n»iwt [23.86] and a similar 7 g jumping 
robot (in [23.87] use this escapement 

cam mechanism (Fig. 23.16). 

A toothless gear mechanism is similar to the es¬ 
capement cam mechanism. The main differences are 
that the toothless gear mechanism uses an incomplete 
gear instead of a change in the cam shape. The mo¬ 
tor actuates the incomplete gear, and the gear actuates 
a transmission to compress or extend a spring. When the 



Fig.23.17a,b Toothless gear mechanism (a) Grillo (Ver.2) 
(after [23.92]), (b) Mini-Wheg (after [23.93]) 
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Fig.23.18a-c Other escapement 
mechanism (a) MSU Jumper (af¬ 
ter [23.94]), (b) MSU Jump-runner 
(after [23.95]), (c) Jollbot (af¬ 
ter [23.96]) 



Fig. 23.19 (a) Torque-reversal mech¬ 
anism, (b) Flea-inspired catapult 
mechanism (after [23.97]), (c) Simpli¬ 
fied flea-inspired catapult mechanism 
(after [23.98]), (d) Jumping robotic 
insect (after [23.99]) 


transmission passes the toothless gear part, the trans¬ 
mission returns to the initial position and the stored 
energy is released instantly. Examples of robots using 
toothless gear mechanisms include Mini-Whegs [23.93] 
and Grillo (Ver.2) [23.92] (Fig. 23.17). 

These two click mechanisms, the escapement 
cam mechanism and the toothless gear mechanism, 
are commonly used for jumping mechanisms, but 
other methods have also been developed (Fig. 23.18 
and l<s&M'ii>uiy<:M) The catapult mechanism of MSU 
Jumper [23.94] and MSU Jump-Runner [23.95] is sim¬ 
ilar to the escapement cam mechanism, except for the 
absence of a cam. Instead of using a cam, it uses a one¬ 
way bearing. This mechanism can be separated in two 


parts based on the critical point in the jump cycle. 
Before passing the critical point, the one-way bearing 
cannot rotate freely, so it rotates to the direction of en¬ 
ergy storage. On the other hand, after passing that point, 
it can rotate freely and release the stored energy for 
jumping. The catapult mechanism in Jollbot [23.96] is 
similar to the mechanism in the MSU Jumper. Its struc¬ 
ture’s slit acts like a one-way bearing. 

A flea-inspired catapult mechanism [23.97] is 
different from aforementioned catapult mechanisms 
(Fig. 23.19 and l^ailUl’hlil). It consists of three SMA 
coil springs: (1) Flexor, (2) Extensor, and (3) Trigger. 
The SMA is activated by heat induced from applied cur¬ 
rent. The flea catapult mechanism begins by activating 
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Fig. 23.20 An asymmetric robotic catapult jumping robot (after [23.100]) 


the flexor that folds the leg. Then, the extensor is ac¬ 
tivated. Because the direction of the torque generated 
by the extensor force is in the folding direction, the leg 
does not move and energy is stored in the extensor SMA 
coil spring. After energy is storage, activation of the 
trigger attached to the extensor pulls the extensor until 
it passes the joint. As a result, the direction of the torque 
generated by the extensor is reversed and the robot 
starts to jump. This mechanism uses muscle-like ac¬ 
tuators to create the torque reversal mechanism, which 
enables simple design. Variation of the torque reversal 
mechanisms have been developed with lesser number of 
actuators but maintaining the same biological principle: 
the simplified flea-inspired catapult mechanism [23.98] 
and the jumping robotic insect [23.99]. 

An asymmetric robotic catapult jumping 
robot [23.100] also has a unique catapult mecha¬ 
nism (Fig. 23.20). It utilizes buckling in a compliant 
beam to jump. It consists of a main frame, an elastic 
strip and a motor. The elastic strip is connected to 
the main frame with a free rotational joint that can 
rotate from 0° to 180° and is connected to a motor 
that can control the rotation angle. When one of the 
elastic strip’s ends is fixed to the main frame, adjusting 
the angle of the other end can be in a snap-through 
buckling shape. This can produce bidirectional jumps, 
but the buckling modes are different on either side. 

Jumping mechanisms for microrobots [23.101] 
have been built using microelectromechanical systems 


(MEMS) manufacturing methods to create a silicon 
body, silicon leg, and a series of PDMS springs 
(Fig. 23.21a). This mechanism consists of two rigid 
bodies connected by PDMS springs and is activated 
by an external force. The robot includes only the 
mechanisms required to demonstrate a jump. The ac¬ 
tuation combined elastomer mechanism is shown in 
Fig. 23.21b [23.101], It used chevron actuators. These 
actuators are used to linearly pull and release the PDMS 
spring embedded into an etched silicon structure for 
jumping. The mechanism’s PDMS springs are designed 



Fig. 23.21 (a) Microrobot (after [23.101]), (b) a colored SEM im¬ 
age of the actuated mechanism (after [23.101]) 
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Fig. 23.22 (a) Pneumatic artificial 
muscles arranged like bi-articular 
muscle, (b) Mowgli (after [23.102]) 


to be similar to resilin - the elastomer that appears in 
insects [23.101]. 

In large-scale jumping, to overcome muscle’s lim¬ 
ited speed, large animals use their long legs or spe¬ 
cial arrangements of muscles and bones such as bi- 
articular muscles. They are muscles that work on two 
joints. In a mechanical linkage that is composed of 
bi-articular muscle and bones, the two joints affect 
each other. During the jumping process, these condi¬ 
tions can be helpful for generating the optimum force. 
The large jumping robot Mowgli uses long legs and 
pneumatic artificial muscles that are arranged like bi- 
articular muscle and it can be seen in Fig. 23.22 and in 
loMEEsEa [23.102], 

Specifications of the jumping robots are summa¬ 
rized in Table 23.1. 

23.3.8 Gripping and Perching 

In nature, insects and animals climb various kinds of 
terrains - from flat and smooth to wavy and rugged 
surfaces. Some animals evolved in a way that pas¬ 
sively adapts to unstructured environments to reduce 
the energy consumption and control complexity. From 
the view point of robotics, these properties have the 
potential to increase energy efficiency and reduce sys¬ 
tem complexity. Therefore, many researchers have 
employed such mechanism to gripping and perching 
devices. 


Hawkes et al. developed a mechanism that allows 
large patches of directional dry adhesives to conform 
to the topology of the surfaces they are in contact 
with [23.103]. The mechanism uses a rigid tile sup¬ 
ported by a compliant material loaded by an inexten- 
sible tendon - inspired by the tendon system and the 
fluid-filled sinus in gecko toes. This mechanism permits 
the adhesive to make full contact with the surface and 
have uniform loading despite significant errors in align¬ 
ment. Hawkes et al. also developed a gasper for landing 
of microair vehicles and grappling objects in space us¬ 
ing gecko-inspired directional adhesives, as shown in 
[23.104] (Fig. 23.23). 

There are several devices and robots that employ mi¬ 
crospines for grasping rough surfaces easily seen in na¬ 
ture. Kim et al. proposed arrays of miniature spines that 
catch opportunistically on surface asperities [23.55]. 
Desbiens et al. proposed a small and unmanned air¬ 
craft that can land, perch and take off from vertical sur¬ 
faces, as shown in l<S£>aiL!lllLiU [23.105] inspired by 
squirrels that reduce their horizontal velocity up to 60% 
prior to impact to distribute impact over all four limbs. 
Spenko et al. developed a hexapedal climbing robot us¬ 
ing rows of toes having microspine [23.56]. Parness 
et al. also employed 16 carriages, each of which con¬ 
tains 16 microspines that conform to mm-scale and be¬ 
low, as shown in l^failillilitl [23.106] (Fig. 23.24). 

Trimmer et al. employed a passive gripping method 
found in caterpillars [23.107]. Caterpillars use their re- 


Table 23.1 Specifications of the jumping robots 


Robot 

Actuator 

Length 

Weight 

Jumping height 

Initial velocity 

7 g jumping robot [23.87] 

Motor 

5 cm 

7 g 

1.4 m 

5.9m/s 

Grillo(Ver.l) [23.86] 

Motor 

5 cm 

15g 

- 

1.5 m/s 

Grillo(Ver.2) [23.92] 

Motor 

3 cm 

lOg 

- 

3.6 m/s 

Mini-Wheg [23.93] 

Motor 

9— 10cm 

90-190 g 

0.18m 

- 

MSU Jumper [23.94] 

Motor 

6.5 cm 

23.5 g 

0.87 m 

- 

MSU Jump-runner [23.95] 

Motor 

9 cm 

25 g 

1.43 m 

- 

Jollbot [23.96] 

Motor 

30 cm 

465 g 

0.218 m 

- 

Flea-inspired catapult mechanism [23.97] 

SMA 

2 cm 

11 g 

0.64 m 

4.4m/s 

Simplified flea-inspired jumping mechanism [23.98] 

SMA 

3 cm 

2.3 g 

1.2 m 

7 m/s 

Jumping robotic insect [23.99] 

SMA 

2 cm 

0.034 g 

0.3 m 

2.7 m/s 

An asymmetric robotic catapult jumping robot [23.100] 

Motor 

17 cm 

30 g 

0.2 m 

- 

Microrobot [23.101] 

None 

0.4 cm 

0.008 g 

0.32 m 

3 m/s 

Mowgli [23.102] 

Pneumatic 

lm 

3 kg 

0.4 m 

- 
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Fig.23.23a,b Graspers based on Gecko-inspired adhesives, (a) Gecko-inspired toe used in RiSE robot {left) and cross 
section of the gecko foot (right) (after [23.103]), (b) Collapsing truss grasper (left) and pivot linkage grasper (right) 
(after [23.104]) 



Fig.23.24a-d Microspine-based robots, (a) Landing and perching UAV (after [23.105]), (b) RiSE robot (after [23.56]), 
(c) sample acquisition tool (after [23.106]), (d) Spinybot (after [23.55]) 


tractor muscles to release the grip, which means that 
they do not consume any energy during gripping. Like 
the caterpillar. Trimmer et al. designed the grippers so 
that gripping is released when the SMA spring actua¬ 
tor is activated. Jung et al. proposed an underactuated 


mechanism based on flexural buckling [23.108]. The 
flexural buckling mechanism is inspired by the soft 
cuticle of a caterpillar’s feet, which largely deforms de¬ 
pending on the shape of the contacting surface. The 
large deformation in the engineered device, flexural 
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c) 



Gripping angle ( 8 ) Gripping angle ( 8 ) 


buckling with an adequately selected length, provides 
wide gripping range with a narrow range of force varia¬ 
tion. This provides a sufficient number of contacts with 
even contact forces, enabling adaptive gripping on var¬ 
ious surfaces. In addition, design of the gripper can 


Fig. 23.25 (a) Passive gripping system (after [23.107]), 
(b) caterpillar-inspired underactuated gripper and (c) con¬ 
stant force region by flexural buckling (after [23.108]) ◄ 

easily be scaled up or down depending on required 
scale. shows the small and large scale 

gripper that can achieve adaptive grasping (Fig. 23.25). 

The octopus performs crawling movements with 
the same limbs used for grasping and manipulation, 
as shown in lojEEHUU. Calisti et al. proposed an 
octopus-inspired solution for movement and manipula¬ 
tion [23.109]. To implement octopus-like motion, they 
employed a steel cable for elongating and shortening 
and fiber cables for bending, which is inspired by the 
longitudinal muscles found in an octopus shown in 
Fig. 23.26. 

Kim et al. developed flytrap-inspired high-speed 
gripper, as shown in [23.110]. Flytraps 

achieve fast capturing by using the bistable structural 
characteristic of its leaf. To achieve similar bistability, 
Kim et al. used asymmetrically laminated carbon fiber 
reinforced prepregs (CFRP). They also utilized a de¬ 
velopable surface having kinematic constraints, which 
constrain the curvature of the artificial leaf. Therefore, 
the curved leaf can be actuated by bending the straight 
edge orthogonal to the curve, a process called bending 
propagation. 

Doyle et al. developed an avian-inspired passive 
perching mechanism for quadrotors for perch-and- 
stare, as shown in l<£a*iLitiLlita [23.111], Songbirds 
had evolved to sleep while perching. When they perch 
on a branch, the tendon connected from the ankle and 




Fig. 23.26 (a) Octopus-inspired 
manipulation {left) and tendon-driven 
mechanism (right) (after [23.109]), 
(b) Flytrap-inspired gripper (left) and 
Orthogonally laminated CFRP (right) 
(after [23.110]) 
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Fig. 23.27 (a) Avian-inspired perching mechanism with UAV (left) anatomy and mechanism design (right) (after [23.111]), 
(b) perching mechanism and process (after [23. 1 12]) 


the rear side of toe automatically cause the toe to grip 
the branch. This allows the songbirds to tightly grip the 
branch without any muscular effort. Inspired by this, 
Doyle et al. used a four-bar mechanism and a tendon 
connected from the knee to the ankle and foot to couple 
landing motion with grasping. Kovac et al. presented 


a 4.6 g perching mechanism for microaerial vehicles 
(MAVs) to make them perch on various walls such 
as tree and concrete buildings [23.112], as shown in 
l<ta>M r iDt<LFFl To achieve high impact force, the nee¬ 
dles snap through as the trigger collides with the target 
surface (Fig. 23.27). 


23.4 Material and Fabrication 

23.4.1 Shape Deposition Manufacturing 

The fundamental concept of shape deposition manufac¬ 
turing (SDM) is layered molding manufacturing with 
CNC machining process. It not only create complex 
3-D shapes rapidly, but also enables high precision 
finishing and large design flexibility. This concept is ini¬ 
tially proposed by Weiss et al. [23.113]. Figure 23.28a 
shows the steps of the SDM process and Fig. 23.28b 
shows fabrication result [23.114]. After deposing sup¬ 
port material, the support was fabricated by CNC ma¬ 
chining to make high precision surface. Li et al. shows 


that it is possible to embed a various functional ma¬ 
terial such as sensors to the structure [23.115] and 
Marra et al. show that this process can be used in 
the fabrication of scaffolds for bone tissue engineer¬ 
ing [23.116], 

In 1999, the SDM process was first used for robot 
design by Bailey et al. [23.114]. Robot design meth¬ 
ods based on the SDM process have advantages that it 
does not need complex assembly or connecting meth¬ 
ods, and it can embed sensors and actuators directly 
into the body structure. In other words, the structure 
is build and assembled at the same time, and this 



Fig.23.28a, b Shape deposition manufacturing (a) Process, (b) result of SDM process for robot mechanism design (af¬ 
ter [23.114]) 


561 


Part B | 23.4 



































Part B | 23.4 


562 Part B 


Design 



2.5 cm 


Actuators and 
wiring embedded 
inside structure 


Legs with 
compliant 
flexures 


Fig. 23.29 Hexapedal robot built by SDM process (af¬ 
ter [23.117]) 


process for mechanism design, making more multi¬ 
farious applications possible [23.119]. Figure 23.29 
shows Sprawlita, a hexapedal robot fabricated based 
on the SDM process. Using the SDM process, actu¬ 
ators and wires are embedded in the body structure, 
resulting in robust performance and minimal manual as¬ 
sembly operations [23.117]. Robotic grippers can also 
be made by the SDM process (Fig. 23.30) [23.118], 
All of joints, links, and sheath for the actuating wire 
were fabricated and assembled simultaneously. Embed¬ 
ded sensors are also possible. Force sensors embedded 
in the fingertips of a robotic SDM gripper have been 
demonstrated [23.120]. Using a soft base material for 
the robot, it is also possible to design a human-friendly 
SDM robot [23.121], 



Fig. 23.30 Robotic gripper built by SDM process (af¬ 
ter [23.118]) 


characteristic leads to enhanced manufacturability in 
small-scale applications. To build a desired mechanism, 
the author used multiple cycles of material deposi¬ 
tion and shaping as shown in Fig. 23.28a. Multiple 
materials were used in fabrication to create variable 
characteristic to each functional part. More flexible 
material was used for joints and rigid material was 
used for links. The flexible components are not only 
used as articulated joints, but also as dampers and 
springs to control the impedance of a mechanism. Bin- 
nard et al. suggested a design framework for the SDM 


23.4.2 Smart Composite Microstructures 

In the late 1990s, researchers at the University of Cal¬ 
ifornia, Berkeley began a project to create a robotic 
insect capable of sustained autonomous flight - the mi¬ 
cromechanical flying insect was born [23.122]. Among 
the many challenges for this project, how to manufac¬ 
ture and what materials should be used for structures, 
mechanisms, and actuators were primary concerns. 
Recognizing the lack of a viable meso-scale manu¬ 
facturing method, the team lead by Fearing attempted 
multiple techniques including folded triangular stain¬ 
less steel beams [23.123] and eventually settled on 
multilayer composites [23.124]. In this paradigm, later 
called smart composite microstructures (SCM) [23.50], 
layers of materials are machined, aligned, and lam¬ 
inated to form a quasi-two-dimensional (quasi-2-D) 
laminate. The choice of materials, 2-D layer geome¬ 
tries, and order of the layup allows the user to create 
an array of rigid components separated by compliant 
flexures. This composite laminate, later called a stan¬ 
dard linkage layer can then be folded into 3-D shapes 
and mechanisms. 

The SCM process presents a new paradigm of de¬ 
sign and fabrication for developing small-scale robots. 
Planar fiber-reinforced prepreg (FRP) and flexure 
hinges replace conventional links and joints in the 
robot mechanism. The composite laminating process is 


Polymer - 


Composite 


Link 

(rigid) 


Flexure 

(compliant) 


Link 

(rigid) 



Fig. 23.31 Schematic illustration 
of a single joint unit in SCM and 
the layup of laminating process 
(after [23.50]) 
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Fig.23.32a-c Spherical five-bar linkage structure created by the SCM fabrication process (after [23.52]). (a) The pattern design of 
a rigid face sheet, (b) laser-cut and cured microcomposite sheet before folding, (c) SCM spherical five-bar linkage for transmission 
of MFI 




Actuators 


Fourbars 


Differential 



Fig.23.33a-d Various generations 
of MFI. (a) Wing transmission made 
of steed face sheet (after [23.123]). 
(b) Thorax mechanism with five- 
bar spherical linkage SCM (after 
[23.124]). (c) The Harvard robotic 
fly is capable of flapping-wing liftoff 
(after [23.50]). (d) The Harvard 
robotic fly is the first robotic insect 
capable of unconstrained controlled 
flight (after [23.125]) 


adopted instead of conventional machining and assem¬ 
bly processes which are difficult to apply to building 
a small-scale robot mechanism. Figure 23.31 shows 
a single unit of the links and the flexure hinge joints 
which are the key building blocks of the SCM pro¬ 
cess [23.50]. The flexure joint is a polymer film that 
can be bent easily and eliminating friction losses that 
are the dominant cause of reduced efficiency in small- 
scale robot mechanisms. 

Rigid face sheets of fiber-reinforced composites 
sandwich polymers and joints are created at the gap 
between face sheets. The resulting quasi-2-D sheets 
are then folded into 3-D structures. The face sheets 
of rigid composite materials are carefully patterned to 
create robot mechanisms. For example. Fig. 23.32a is 
a 2-D pattern designed for creating a spherical five- 
bar linkages as shown in Fig. 23.32c. This is the wing 


transmission linkage in the early versions of microme¬ 
chanical flying insect (MFI) [23.52], 

Figure 23.33 shows the various versions of the 
MFI developed using the SCM fabrication process. 
Figure 23.33a is the early version of a transmission 
that uses a steel face sheet [23.123], Figure 23.33b 
is the next generation of the MFI fabricated using 
a carbon fiber composite (CF) sheet and polyester 
film. It has 26 joints, 4-DOF, four actuators, and two 
wings. CF composites materials improve the perfor¬ 
mance of MFI thorax structure by reducing the iner¬ 
tia by a factor of three and increasing the resonant 
frequency by 20% [23.124]. A later version with a 3- 
DOF transmission (two passive DOF) and a single 
bimorph PZT actuator as shown in Fig. 23.33c was able 
to produce sufficient thrust to achieve liftoff [23.52]. 
Passive dynamics in the thorax structures simpli- 
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Fig. 23.34 DASH (after [23.8]) 



Fig.23.35a,b 1:900 scale 1903 Wright Flyer model 

14 mm in wing span (after [23.126]). After laminating 
layup (a), model after folding (b) 

fies the design and reduces undesired coupling be¬ 
tween the degrees of freedom. Controlled flight is 
accomplished by separated actuator design shown in 
Fig. 23.33d. Tethered but unconstrained stable hov¬ 
ering and basic controlled manoeuvres are demon¬ 
strated [23.125]. 


The SCM fabrication process can be extended to 
larger scales by using various sheet materials. Fig¬ 
ure 23.34 shows centimetre scale crawling robots that 
are built using the SCM fabrication process with card¬ 
board and adhesive films in place of the composites 
used in the MFI. This new paradigm is fast and inex¬ 
pensive - both in the materials used and the required 
infrastructure. Furthermore, novel bio-inspired robot 
mechanism that produce high performance can be cre¬ 
ated easily as shown in Fig. 23.34. Figure 23.34a 
is a small, lightweight, power autonomous running 
robot, DASH [23.8]. It is capable of running at speeds 
up to 15 body lengths per second and surviving 
falls from large heights, due to the unique compli¬ 
ant nature of its structures. The design can be modi¬ 
fied easily and achieve high performance with regard 
to stability, speed, and maneuverability as shown in 
Fig. 23.34b [23.10], 

23.4.3 Pop-Up Book MEMS 

In order to streamline the development of high per¬ 
formance and economical robots, many assembly tools 
have been developed to assist with robot construc¬ 
tion at various scales. However, in millimetre-scale 
robots, many challenges arise in the fabrication pro¬ 
cess and handling many individual parts for assembly. 
Pop-up books and paper folding inspire the solution 
for eliminating the onerous assembly process with 
small individual parts in the fabrication process of 
the millimetre-scale robotic structures. Monolithic fab¬ 
rication using pop-up book-inspired designs enables 
efficient batch processing starting from multiple layered 
composites similar to the basic elements used in SCM. 
The carefully designed layers are interconnected and 
allow folding mechanisms of high complexity. Once 
folding ensues, the complex 3-D structures are cre¬ 
ated by parallel mechanisms created in the thickness 




Fig. 23.36 (a) Mobee, monolithic design of MFI [23.127], (b) HAMR-VP, a 1.27 g quadrupedal microrobot manufac¬ 
tured using the PC -MEMS and pop-up assembly techniques (after [23. 16]) 
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Fig. 23.37 Young’s modulus for 
various materials (after [23.128] which 
was also adapted by Autumn et al. 
in [23.63]) 


of the laminate. Figure 23.35 presents a pop-up struc¬ 
ture of 1:900 scale 1903 Wright Flyer model, 14 mm 
in wing span [23.126]. The model consists of six rigid 
CF composite layers, seven adhesive layers, and two 
polymer flexure layers. Multiple rigid-flex folding lay¬ 
ers are stacked and selectively bonded. The idea is 
previously applied to microelectric mechanical systems 
(MEMS) process. Combining this idea and pop-up book 
designs, the fabrication process is developed for build¬ 
ing micro robotic systems including robot structures 
and actuators. 

Figure 23.36 shows two examples that are designed 
and fabricated by the pop-up book MEMS process. 
Figure 23.36a is the Mobee (Harvard robotic fly us¬ 
ing the monolithic popup book MEMS design meth¬ 
ods) [23.127], In I<E6JCES!E11, it demonstrates how 
the pop-up book MEMS process enables mass produc¬ 
tion by parallel manufacturing on a single sheet and 
reduces entire fabrication time by eliminating oner¬ 
ous assembly tasks. Figure 23.36b is a small-scale 
quadruped crawling robot designed by pop-up book 
MEMS techniques. This device demonstrates how com¬ 
plex millimeter-scale robot structures are capable of 
pop-up assembly [23.16]. Twenty-three material layers 
are cut by precision laser machining and laminated with 
selective adhesion. After popup, the body frame is cre¬ 
ated and other components such as a circuit board and 
actuators are bonded on the frame. 

23.4.4 Other Fabrication Methods 

In nature, animals use soft parts of their body for 
generating locomotion, morphing configuration, and 
adapting to the environment. In order to maximize the 
utility of their compliance, some bio-inspired robots 
are primarily composed of soft materials such as flu¬ 
ids, gels, granules, and soft polymers. With respect to 
compliance, soft materials vary with a wide range of 
elastic (Young’s) modulus [23.128] (Fig. 23.37). There¬ 
fore, bio-inspired robots should be built by different 
fabrication methods depending on their constituent ma¬ 
terials. 
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Fig. 23.38 Manufacturing process of directional adhesion 
pad (after [23.58]) 


Soft Lithography 

Soft lithography was originally proposed for micro- and 
nanostructure manufacturing in 1998 [23.129]. Early 
soft lithography used an elastomeric stamp with pat¬ 
terned relief structures on its surface to generate desired 
patterns and structures. In bio-inspired robotics, for ex¬ 
ample, a directional adhesion pad inspired by a gecko 
foot’s microscale adhesion spines was developed by 
a similar fabrication method to soft lithography [23.58] 
(Fig. 23.38). 

Advances of convenient, effective and low-cost soft 
lithography allowed it to be utilized not only in the 
micro and nano scale in bio-chemical fields, but also 
in macro-scale robotics. Moreover, development and 
popularization of additive manufacturing process us¬ 
ing 3-D printers which can design molds with various 
shapes easily and rapidly have helped soft lithography 
to be applied in robotics. Soft lithography has become 
a representative manufacturing method for bio-inspired 
soft robots since soft lithography was first applied to 
manufacturing bio-inspired soft robots [23.130-133] 
(Fig. 23.39). In soft lithography for bio-inspired soft 
robots, uncured elastomeric polymers, such as PDMS 
or EcoFlex, are poured into a mold designed with 
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configuration of the structure. After curing, the soft 
materials form a structure containing multiple air cham¬ 
bers and pneumatic channels. Such kind of structure is 
so-called a pneumatic network (PneuNet) or bending 
fluidic actuator (BFA). 

This structure allows the robots to generate so¬ 
phisticated locomotion such as gripping, walking, and 
crawling as shown in Fig. 23.40. In addition, extra 
embedded channels can control the flow of a dye¬ 
ing solution so that a soft robot can camouflage by 
changing body color to match the color of the surround¬ 
ings [23.135]. The actuation or locomotion of robots 
manufactured by soft lithography depends on the elas¬ 
tic modulus of materials and the geometry and position 
of air chambers [23.136]. 

Furthermore, soft lithography in robotics has ad¬ 
vanced by mixing different kind of materials. By em¬ 
bedding sheets or fibers into elastomers, an actuator 
has asymmetric compliance which allows the structure 
to be flexible but not extensible, so that the actuator 
can generate a wide range of motions such as bending, 
extension, contraction, twisting, and others [23.137] 
(Fig. 23.41). By embedding magnets into elastomers, 
a soft robot can attach, detach, and easily align modules 
that have a unique function per each module depending 
on the task [23.138]. 
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Fig. 23.39 Soft lithography of pneumatic network (Pne¬ 
uNet) or bending fluidic actuator (BFA) (after [23.130, 
133, 134]) 



Fig. 23.40 Bio-inspired soft robots and actuators manu¬ 
factured by soft lithography (after [23.130, 131,135,136]) 


To overcome a slow actuation, which is a typi¬ 
cal issue for soft robots, the design of segmented air 
chambers divided by slits was introduced as an alterna¬ 
tive soft actuator [23.139]. As a result, a fast PneuNet 
structure had high rate of actuation, improved 25 times 
relative to the slow PneuNet actuators. Also, a reduced 
change of volume minimizes fatiguing the materials, 
and thus the durability improves to a level that the actu¬ 
ator does not fail within a million cycles of full bending. 

Actuator Embedded Molding 
Bio-inspired soft robots potentially have infinite de¬ 
grees of freedom and the nonlinearity of soft materials 
creates difficulty in generating desired postures and mo¬ 
tion. Actuator-embedded molding is commonly used to 
manufacture the soft structure for bio-inspired robots. 
In actuator-embedded molding, the design considera¬ 
tions are the type of the actuator and the actuator’s 
location and direction in the soft structure. For an ex¬ 
ample of a bio-inspired robot using actuator-embedded 
molding, a turtle-like swimming robot was created us¬ 
ing a smart soft composite (SSC) structure to create 
bending and twisting deformation [23.77] (Fig. 23.42). 
The SSC structure consists of an actuator-embedded 
layer as an active component, a patterned layer as a pas¬ 
sive component, and a soft matrix as the body as shown 
in Fig. 23.42. The angle of the patterned layer deter¬ 
mines the bending direction passively, and the actuator- 
embedded layer generates deformation. The soft matrix 
helps to deform the structure continuously. Widely used 



Fig.23.41a-e Programmable paper-elastomer compos- 
ites-based pneumatic actuators by using soft lithography 
(after [23.137]) 
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Fig. 23.42 Actuator-embedded molding for the SSC structure actuator (after [23.77]) 


actuators in the actuator embedded molding are wires 
including common wires connected with servo motors 
and shape memory alloys (SMA) [23.25,77, 140]. 

Additive Manufacturing 

Additive manufacturing or 3-D printing is a rapid pro¬ 
totyping (RP) process of manufacturing a 3-D solid 
structure of any configuration through sequential lay¬ 
ering from a digital CAD model. Since the 1980s, 
various types of additive manufacturing have been de¬ 
veloped: solid-based processes such as fused deposi¬ 
tion modeling (FDM), photo curable liquid polymer- 
based processes such as stereolithography (SLA), and 
powder-based processes such as selective laser sinter¬ 
ing (SLS). 


Evolving the technology of 3-D printers, soft mate¬ 
rial deposition and even hybrid deposition of different 
materials with different stiffnesses are available. Thus, 
soft structures can be built at once, and products using 
hybrid deposition can have features of rigidity and soft¬ 
ness at the same time. For example, a highly deformable 
3-D printed soft robot was developed [23.141]. This 
robot body was printed by using a multimaterial print¬ 
able 3-D printer (Objet Connex 500TM 3-D printer) 
with two materials: one is a soft rubber-like Objet Tan- 
goBlackPlusTM and another is a hard polypropylene¬ 
like Objet VeroWhitePlusTM. These two materials have 
different friction coefficients; thus, the robot can switch 
friction with the ground on its edge by bending the 
body. 


23.5 Conclusion 

Biomimetic robotics attempt to create devices that are 
capable of various types of effective interaction with 
natural environment, e.g., locomotion and manipula¬ 
tion, by using the principles of nature. Nature is full of 


surprising types of movements that enables insects and 
animals to survive by escaping from danger or hunting 
for food. In this chapter, we have presented robots that 
attempt to recreate these feats by understanding the un- 
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derlying principles, deriving an engineering design, and 
fabricating them with novel methods. In recent years, 
engineers have succeeded in recreating insects and ani¬ 
mals that show amazing capabilities such as climbing 
walls like a gecko, hovering like a fly and climbing 
trees like a snake. These robots have been developed 
to understand nature, and also to be used as tools for 
surveillance, information gathering, and rescue opera¬ 
tions. However, many of these robots are still in the 
basic research phase and are not yet ready for everyday 
use. 

The key open research issues remain in the broad 
areas of materials, fabrication, actuation, and power. 
Composites and polymers, together with novel fabrica¬ 
tion methods, have enabled various novel biomimetic 
robots. Development of these new material and fabri¬ 
cation has been one of the key enabling technologies, 
and further development of these technologies will cer¬ 
tainly contribute to more mature biomimetic robots. 
Actuation and power still remain the bottleneck of 
many biomimetic robots. DC motors are the actua¬ 
tors of choice for many biomimetic robots, and with 
novel transmission design, DC motors can create mo¬ 
tions required by the robots. However, DC motors 
are inefficient for small-scale biomimetic robots. Al¬ 
though artificial muscle actuators that are based on 


shrinkage or expansion of material, e.g., shape mem¬ 
ory alloys, IPMC, electro-active polymers and shape 
memory polymers, promise to give robots capabilities 
similar to creatures that use biological muscles even 
at small scales. However, many issues related to ro¬ 
bustness, efficiency, and power limit the capabilities of 
robots that use these actuators. Limitations of these ac¬ 
tuators should be carefully considered to match the de¬ 
sired application. Development of new artificial muscle 
actuator that can emulate biological muscles - with¬ 
out the drawbacks of current actuators - is needed to 
open up a new era for biomimetic robots. Batteries also 
limit the capabilities of current biomimetic robots com¬ 
pared to their biological counterparts in terms of size 
and operation time. Development of energy-harvesting 
technologies together with new battery chemistries and 
manufacturing methods will enable longer operation 
time, which will contribute to realizing wider applica¬ 
tions for biomimetic robots. Overall, biomimetic robot 
design is one of the most challenging areas of robot 
design since it requires development of various tech¬ 
nologies to mimic the structure and function of natural 
systems. Therefore, development of biomimetic robots 
can have broader impact in many areas of engineering 
and science, and should be seen as a platform for vari¬ 
ous convergent technologies. 
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The long-jumping robot 'Grillo' 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/278 
A miniature 7gjumping robot 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/279 

A single motor actuated miniature steerable jumping robot 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/280 

The Flea: Flea-inpired light jumping robot using elastic catapult 

with active storage and release mechanism 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/281 
Jumping & landing robot 'MOWGU' 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/285 
RoACH: A 2.4g, untethered crawling hexapod robot 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/286 
A new form of peristaltic locomotion in a robot 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/287 
Meshworm 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/288 
Treebot: Autonomous tree climbing by tactile sensing 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/289 
Omegabot: Inchworm inspired robot climbing 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/290 
GoQBot: Insanely fast robot caterpillar 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/291 

Spinybotll: Climbing hard walls with compliant microspines 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/388 

Smooth vertical surface climbing with directional adhesion 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/389 
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Biologically inspired climbing with a hexapedal robot 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/390 
CLASH: Climbing vertical loose cloth 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/391 
Torque control strategies for snake robots 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/392 
Snake robot climbs a tree 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/393 
Snake robot in the water 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/394 
Salamandra Robotica II robot walking and swimming 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/395 
ACM-R5H 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/397 

Pop-up fabrication of the Harvard monolithic bee (Mobee) 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/398 

Controlled flight of a biologically-inspired, insect-scale robot 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/399 

Rhexthe parkour robot 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/400 
Mini whegs 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/401 
Robot dragonfly DelFly explorer flies autonomously 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/402 
Standford Sprawl and iSprawl 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/403 
DASH: Resilient high-speed 16g hexapedal robot 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/405 
HAMR3: An autonomous 1.7g ambulatory robot 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/406 
Undulatory gaits in a centipede millirobot 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/407 
VelociRoACH 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/408 
Underactuated adaptive gripper using flexural buckling 
available from http://handbookofrobotics.org/view-chapter/23/videodetails/409 
Flytrap-inspired bi-stable gripper 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/410 
An octopus-bioinspired solution to movement and manipulation for soft robots 
available from http://handbookofrobotics.org/view-chapter/23/videodetails/411 
Landing and perching UAV 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/412 
Dynamic surface grasping with directional adhesion 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/413 
Gravity-independent rock-climbing robot and a sample acquisition tool 
with microspine grippers 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/414 
Avian-inspired perching mechanism with UAV 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/415 
A perching mechanism for micro aerial vehicles 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/416 
G9 series robotic fish 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/431 
Ichthus 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/432 
Autonomous, self-contained soft robotic fish 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/433 
Robotic Ray takes a swim 

available from http://handbookofrobotics.org/view-chapter/23/videodetails/434 
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24. Wheeled Robots 



The purpose of this chapter is to introduce, ana¬ 
lyze, and compare various wheeled mobile robots 
(WMRs) and to present several realizations and 
commonly encountered designs. The mobility of 
WMR is discussed on the basis of the kinematic 
constraints resulting from the pure rolling condi¬ 
tions atthe contact points between the wheels and 
the ground. Practical robot structures are classified 
according to the number of wheels, and features 
are introduced focusing on commonly adopted de¬ 
signs. Omnimobile robotand articulated robots re¬ 
alizations are described. Wheel-terrain interaction 
models are presented in order to compute forces at 
the contact interface. Four possible wheel-terrain 
interaction cases are shown on the basis of relative 
stiffness of the wheel and terrain. A suspension 
system is required to move on uneven surfaces. 
Structures, dynamics, and important features of 
commonly used suspensions are explained. 
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24.1 Overview 

Wheeled robots have been widely used to achieve mo¬ 
bility because there are many advantages including the 
simple structure, energy efficiency, fast speed, low fab¬ 
rication cost, and so forth. The purpose of this chapter 
is to provide a general description of wheeled mobile 
robots, to discuss their properties from the mobility 
point of view, to describe the most commonly en¬ 
countered realizations of such robots, and to explain 
wheel-terrain interaction models and suspension sys¬ 
tems. 
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The chapter is organized as follows: Sect. 24.2 is de¬ 
voted to the characterization of the restriction of robot 
motion induced by these pure rolling conditions. We 
first describe different types of wheels used in the con¬ 
struction of mobile robots and derive the corresponding 
kinematic constraints. This allows us to characterize the 
mobility of a robot equipped with several wheels of 
these different types, and we show that these robots can 
be classified only into five categories, corresponding to 
two mobility indices. 
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In Sect. 24.3, we present several realizations of 
wheeled mobile robots by the use of various types of 
wheels. Structures and features of widely used wheel 
mechanisms are explained. Design issues of wheeled 
robots include capability of omnidirectional move¬ 
ment, fabrication cost, difficulty of control, and terrain 
traversability. 

Section 24.4 explains wheel-terrain interaction 
models. It is shown that four possible wheel-terrain in¬ 
teraction cases can be obtained according to the relative 
stiffness of the wheel and terrain. The presented analyti¬ 
cal models are broadly suitable for analysis, simulation, 
design, and control purposes. 


Section 24.5 presents suspensions for wheeled 
robots. A suspension is a system of linkages, springs, 
dampers, and actuators that governs the relative mo¬ 
tion between a robot’s wheels and body. Suspen¬ 
sion mechanisms are useful when driving over un¬ 
even surfaces. Common methods for modeling passive 
and semiactive suspensions are presented. The anal¬ 
ysis of these models allows for principled selection 
of springs, dampers, and active or semiactive actua¬ 
tors that form the basis of nearly all wheeled robot 
suspensions. 

Finally, some concluding remarks are given in 
Sect. 24.6. 


24.2 Mobility of Wheeled Robots 

In this section, we describe a variety of wheels and 
wheel implementations in mobile robots. We discuss 
the restriction of robot mobility implied by the use of 
these wheels and deduce a classification of robot mo- 





Fig.24.1a-c The general design of a standard wheel, 
(a) Side view, (b) front view, and (c) top view 


bility allowing one to characterize robot mobility fully, 
whatever the number and type of the wheels. 

24.2.1 Types of Wheels 

In order to achieve robot locomotion, wheeled mobile 
robots are widely used in many applications. In general, 
wheeled robots consume less energy and move faster 
than other locomotion mechanisms (e.g., legged robots 
or tracked vehicles). From the viewpoint of control, less 
control effort is required, owing to their simple mech¬ 
anisms and reduced stability problems. Although it is 
difficult to overcome rough terrain or uneven ground 
conditions, wheeled mobile robots are suitable for 
a large class of target environments in practical appli¬ 
cations. When we think of a single-wheel design, there 
are two candidates: a standard wheel or a special wheel. 
A standard wheel can be understood as a conventional 
tire. Special wheels possess unique mechanical struc¬ 
tures including rollers or spheres. Figure 24.1 shows the 
general design of a standard wheel. Three conditions 
should be defined for a standard wheel design: 

1. Determination of the two offsets d and b 

2. Mechanical design that allows steering motion or 
not (i. e., to fix the wheel orientation or not) 

3. Determination of steering and driving actuation 
(i. e., active or passive drive). 

Condition 1 is the kinematic parameter design prob¬ 
lem for a single standard wheel. The parameter d can 
be either 0 or some positive constant. Parameter b is 
the lateral offset of the wheel and is usually set to 
zero. In a special design, a nonzero b may be selected 
to obtain pure rolling contact between the wheel and 
ground without causing rotational slip at the contact 
point. However, this is rarely used and we mainly con¬ 
sider the case of zero lateral offset b. 
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Condition 2 is a design problem for whether the 
wheel orientation can be changed or not. If the steering 
axis is fixed, the wheel provides a velocity constraint 
on the driving direction. Condition 3 is the design prob¬ 
lem of whether to actuate steering or driving motion by 
actuators or to drive steering or motion passively. 

If steering motion is allowed, the offset d plays 
a significant role in the kinematic modeling. For a con¬ 
ventional caster wheel (i. e., an off-centered orientable 
wheel), there is a nonzero offset d. Point A in Fig. 24.1 
indicates the location of the joint connecting the wheel 
module to the robot chassis. Two orthogonal linear ve¬ 
locity components at point A are obtained by a caster 
wheel, which results from the steering and driving mo¬ 
tions of the wheel module. This implies that a passive 
caster wheel does not provide an additional velocity 
constraint on the robot’s motion. If a caster wheel is 
equipped with two actuators that drive steering and 
driving motions independently, holonomic omnidirec¬ 
tional movement can be achieved because any desired 
velocity at point A can be generated by solving the in¬ 
verse kinematics problem. 

If the offset d is set to zero, the allowable velocity 
direction at point A is limited to the wheel orientation. 
In such a case, the steering motion should not be pas¬ 
sive because the wheel orientation cannot be changed 
passively. However, the driving velocity can be de¬ 
termined passively by the actuation of other wheels. 
Wheel orientation should be actively steered to the 
desired velocity direction due to the nonholonomic ve¬ 
locity constraint. This implies that the wheel orientation 
should be aligned before movement. 

In summary, four types of standard wheels are com¬ 
monly used. First is a passively driven wheel with 
a fixed steering axis. Second is a passive caster wheel 
with offset d. Third is an active caster wheel with 
offset d, where the steering and driving motions are 
controlled by actuators. An example of caster wheels 
is shown in IdSf'il’JVKU. The fourth is an active ori¬ 
entable wheel with zero offset d , where steering and 
driving motions are driven by actuators. The structures 
of each wheel type are shown in Fig. 24.2. The kinemat¬ 
ics and constraints of those wheels will be explained in 
detail in Sect. 24.2.2. 

Although standard wheels are advantageous be¬ 
cause of their simple structure and good reliability, 
the nonholonomic velocity constraint (i. e., no side¬ 
slip condition) limits robot motion. On the other hand, 
special wheels can be employed in order to obtain 
omnidirectional motion of a mobile robot (omnimo- 
bile robot), i. e., to ensure three degrees of freedom 
for plane motion. We consider two typical designs 
of special wheels: the Swedish wheel and the spher¬ 
ical wheel. Figure 24.3a and show the 



Fig.24.2a-c Structures of standard wheels, (a) Passive 
fixed wheel, (b) passive or active, off-centered orientable 
wheel, and (c) active orientable wheel without offsets 


Swedish wheel. Small passive free rollers are located 
along the outer rim of the wheel. Free rollers are em¬ 
ployed in order to eliminate the nonholonomic velocity 
constraint. Passive rollers are free to rotate around the 
axis of rotation, which results in lateral motion of 
the wheel. As a result, a driving velocity should be 
controlled, while the lateral velocity is passively deter¬ 
mined by the actuation of the other wheels. A similar 
design can be seen in I^M'iNVKM. 

A spherical wheel is shown in Fig. 24.3c. The ro¬ 
tation of the sphere is constrained by rollers that make 
rolling contact with the sphere. The rollers can be di¬ 
vided into driving and supporting rollers. The sphere is 
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b) X 2 



Fig. 24.3 (a) Swedish wheel, (b) attachment of a Swedish 
wheel, and (c) spherical wheel (after [24.1]) 

driven by the actuation of the driving rollers, whereas 
the rolling contacts provide nonholonomic constraints, 
and the resultant motion of the sphere module becomes 
holonomic. This implies that the robot can be moved 



with any desired linear/angular velocities at any time. 
By using the spherical wheel, a holonomic omnidirec¬ 
tional mobile robot can be developed and the robot 
achieves smooth and continuous contact between the 
sphere and the ground. However, the design of the 
sphere-supporting mechanism is difficult and the pay- 
load must be quite low due to the point contact. Another 
drawback is that the surface of the sphere can be pol¬ 
luted when traveling over dirty ground and it is difficult 
to overcome irregular ground conditions. These draw¬ 
backs limit the practical application of the spherical 
wheel. An example of the use of spherical wheels can 
be found in [24.1] and [24.2]. The spherical structure 
can also be applied to special robotic transmissions; ex¬ 
amples include the nonholonomic manipulator in [24.3] 
and the passive haptic system in [24.4]. 

24.2.2 Kinematic Constraints 

We assume, as a first step, that the mobile robot under 
study is made up of a rigid cart equipped with nonde- 
formable wheels, and that it is moving on a horizontal 
plane. The position of the robot on the plane is de¬ 
scribed, with respect to an arbitrary inertial frame, by 
the posture vector £ = (x y 6 ) T , where x and y are 
the coordinates of a reference point P of the robot cart, 
while 9 describes the orientation of a mobile frame at¬ 
tached to the robot, with respect to the inertial frame 
(Fig. 24.4). 

We assume that, during motion, the plane of each 
wheel remains vertical and the wheel rotates around 
its horizontal axle, whose orientation with respect to 
the cart can be fixed or varying. We distinguish be¬ 
tween the two basic classes of idealized wheels, namely 
conventional and the Swedish wheels. In each case, 


k 



Fig. 24.4 The posture definition of a mobile robot on 
a plane 
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it is assumed that the contact between the wheel and 
the ground is reduced to a single point. The kinematic 
constraints result from the fact that the velocity of the 
material point of the wheel in contact with the ground 
is equal to zero. 

For a conventional wheel, the kinematic constraints 
imply that the velocity of the center of the wheel is 
parallel to the wheel plane (nonslip condition) and is 
proportional to the wheel rotation velocity (pure rolling 
condition). For each wheel, the kinematic constraints 
therefore result in two independent conditions. For 
a Swedish wheel, due to the relative rotation of the 
rollers with respect to the wheel, only one of the veloc¬ 
ity components of the wheel contact point is zero. The 
direction of this zero component is fixed with respect 
to the wheel plane and depends on the wheel construc¬ 
tion. For such wheels, the kinematic constraints result 
in only one condition. 


The projections of this vector onto the direction of 
the wheel plane, i. e., onto the vector (cos(a + fi — n/2), 
sin(a + yS — tt/2)) and the vector of the wheel axle 
(cos(a + /)), sin(a + /))), are r<j> and 0, respectively, cor¬ 
responding to the pure rolling and nonslip conditions. 

After some manipulations, these conditions can be 
rewritten in the following compact form. 

Pure Rolling Condition. 

(—sin(a + /l) cos (a + fi) Zcos /J)R(60£ + r<p = 0 . 

(24.1) 

Nonslip Condition. 

(—cos(a + fi) sin(a + /l)d+/sin/l)R(60£-|-<f/j = 0 . 

(24.2) 


Conventional Wheels 

We now derive the general form of the kinematic con¬ 
straints for a conventional wheel. 

As shown in Fig. 24.2, there are several variations 
of the conventional wheel design. First, we focus on the 
off-centered orientable wheel in Fig. 24.2b. The center 
of the wheel, B, is connected to the cart by a rigid rod 
from A (a fixed point on the cart) to B, aligned with 
the wheel plane. The rod, whose length is denoted by d, 
can rotate around a fixed vertical axle at point A. The 
position of A is specified by two constant polar coor¬ 
dinates, / and a, with respect to the reference point P. 
The rotation of the rod with respect to the cart is rep¬ 
resented by the angle ft. The radius of the wheel is 
denoted by r, and its angle of rotation around its hor¬ 
izontal axle is denoted by tp. The description therefore 
involves four constant parameters: a, /, r, and d, and 
two variables: ip(t) and /3(f). 

With these notations the kinematic constraints are 
derived as follows. 

We make the derivation explicit for the general situ¬ 
ation corresponding to a caster wheel (Fig. 24.2b). For 
fixed or steering wheels one just has to consider either 
the case d = 0 and constant f) (fixed wheels), or d = 0 
and variable /J (steering wheels). 

First, we evaluate the velocity of the center of 
the wheel, which results from the following vector 
expression j t OB = j t OP+ jfi'A + jpAB. The two com¬ 
ponents of this vector in the robot frame are expressed 
as 

icos 9 + y sin# — 19 sin a + (9 + / 3 ) dcos(a + fi) 


and 

—x sin 9 +y cos 9 — 19 cos a + (9 + /l)d sin(a + /)) . 


In the earlier given expressions, R((9) is the orthog¬ 
onal rotation matrix expressing the orientation of the 
robot with respect to the inertial frame, i. e.. 


R(0) 


( cost? 
— sin 9 


sin 9 
cos 9 


0 \ 

0 


V 0 01/ 


(24.3) 


As said before, these general expressions can be 
simplified for different types of conventional wheels. 

For fixed wheels, the center of the wheel is fixed 
with respect to the cart and the wheel orientation is 
constant. This corresponds to a constant value of /) and 
d = 0 (Fig. 24.2a). The nonslip equation (24.2) then re¬ 
duces to 


(005(01-1-/1) sin(ct' + yS) / sin/J)R(0)| = 0 . (24.4) 

For steering wheels, the center of the wheel is also 
fixed with respect to the cart (i. e., d = 0), with /J time- 
varying, so the nonslip equation takes the form (24.2). 
This structure was already introduced in Fig. 24.2c. 

The situation described by (24.1) and (24.2), with 
a nonzero-length rod AB and time-varying orientation 
angle /3 corresponds to caster wheels. 

Swedish Wheels. The position of a Swedish wheel 
with respect to the cart is described, as for fixed wheels, 
by three constant parameters: a, /l, and 1. An additional 
parameter is required to characterize the direction, with 
respect to the wheel plane, of the zero component of 
the velocity at the contact point of the wheel. This pa¬ 
rameter is y, which is the angle between the axle of the 
rollers and the wheel plane (Fig. 24.3b). 
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The kinematic constraints now impose only one 
condition 

[— sin(a + f + y) cos(a + f) + y) l cos(j 6 + y)] 
x R(0)| + rcos y <p = 0 (24.B) 

24.2.3 Robot Configuration Variables 

We now consider a wheeled robot equipped with N 
wheels of the earlier described types. We use the fol¬ 
lowing subscripts to identify quantities related to these 
four types: f for fixed wheels, s for steering wheels, c for 
caster wheels, and sw for Swedish wheels. The number 
of wheels of each type are denoted by Nf, N s , N c , and 
N sw , with N = Nf + N s T N c + /V sw . 

The configuration of the robot is fully described by 
the following generalized coordinate vector: 

• Posture coordinates', the posture vector | (f) = 
(x(t) y(t) 6»(t)) T 

• Orientation coordinates : the N s + N c orientation an¬ 
gles of the steering and caster wheels, i. e., y3(?) = 
08.(0 &( 0 ) T 

• Rotation coordinates: the N rotation angles of the 
wheels, i.e., <p(t) = (. <p f (t ) (pjt) <p c (t) <p sw (f)) T . 

This whole set of coordinates is termed as the set of 
configuration coordinates. The total number of config¬ 
uration coordinates is Nf + 2N S + 2N C + N sw + 3. 

24.2.4 Restriction on Robot Mobility 

The pure rolling conditions for fixed, steering, and 
caster wheels, as well as the constraints relative to the 
Swedish wheels, can be written in the following com¬ 
pact form 

Ji(j8 8 ,i8 c )R(0)|+J 2 * = O, (24.6) 

with 


JiO8..0 c ) 


< J if ^ 

JlsOSs) 
JlcOSc) 
V Jlsw / 


In this expression, J lf , Ji s (/3 s ), Jic(/S c ), and J isw are, re¬ 
spectively, (IVfx3), (A s x 3), (N c x3), and (A sw x 3) ma¬ 
trices, whose forms derive directly from the kinematic 
constraints, while J 2 is a constant (NxN) diagonal ma¬ 
trix whose entries are the radii of the wheels, except for 
the radii of the Swedish wheels which are multiplied 
by cos y. 


The value y = jr/2 would correspond to the di¬ 
rection of the zero component of the velocity being 
orthogonal to the plane of the Swedish wheel. Such 
a wheel would be subject to a constraint identical to the 
nonslip condition for a conventional wheel; hence, los¬ 
ing the benefit of implementing a Swedish wheel. This 
implies that y 7 ^ f and that J 2 is a nonsingular matrix. 

The nonslip conditions for caster wheels can be 
summarize as 

C lc (;6 c )R(^ + C 2c/ 6 c = 0, (24.7) 

where Ci c (/3 c ) is a (N c x 3) matrix, whose entries de¬ 
rive from the nonslip constraints (24.2), while C 2c is 
a constant diagonal nonsingular matrix, whose entries 
are equal to d. 

The last constraints relate to the nonslip conditions 
for fixed and steering wheels. They can be summarized 
as 


cr(/uR((^=o, 

where 


cros.) 


c lf \ 
c l8 (/? 8 )j’ 


(24.8) 


where Cif and Ci s (yS s ) are, respectively, (IVfx3) and 
(N s x 3) matrices. 

It is important to point out that the restrictions on 
robot mobility result only from the conditions (24.8) in¬ 
volving the fixed and the steering wheels. These condi¬ 
tions imply that the vector R(0)£ belongs to N[C* (/J s )], 
the null space of the matrix C*(/3 S ). For any R(0)| 
satisfying this condition, there exists a vector <p and 
a vector /J c satisfying, respectively, conditions (24.6) 
and (24.7), because J 2 and C 2c are nonsingular matrices. 

Obviously rank[C* (/S s )] < 3. If it is equal to 3 
then R(60(j = 0 , which means that any motion in the 
plane is impossible. More generally, restrictions on 
robot mobility are related to the rank of C* (P s ), as will 
be discussed as follows in detail. 

It is worth noticing that condition (24.8) has a di¬ 
rect geometrical interpretation. At each time instant 
the motion of the robot can be viewed as an instanta¬ 
neous rotation about the instantaneous center of rotation 
(ICR), whose position with respect to the cart can be 
time varying. At each instant the velocity of any point 
of the cart is orthogonal to the straight line joining this 
point and the ICR. This is true, in particular, for the cen¬ 
ters of the fixed and steering wheels, which are fixed 
points of the cart. On the other hand, the nonslip con¬ 
dition implies that the velocity of the wheel center is 
aligned with the wheel plane. These two facts imply that 
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Fig.24.5a,b The instantaneous center of rotation, (a) A car-like 
robot; (b) a three-steering-wheels robot 


the horizontal rotation axles of the fixed and steering 
wheels intersect at the ICR (Fig. 24.5). This is equiva¬ 
lent to the condition that rank[C* (/J s )] < 3. 

24.2.5 Characterization of Robot Mobility 

As said earlier, the mobility of the robot is directly 
related to the rank of C* (/),), which depends on the 
design of the robot. We define the degree of mobility 8 m 
as 

8 m = 3 — rank[C* (/? s )] . (24.9) 

Let us first examine the case rank(Cif) = 2, which im¬ 
plies that the robot has at least two fixed wheels. If there 
are more than two fixed wheels, their axles intersect at 
the ICR, whose position with respect to the cart is then 
fixed in such a way that the only possible motion is a ro¬ 
tation of the cart about this fixed ICR. Obviously, from 
the user’s point of view, such a design is not acceptable. 
We therefore assume that rank(Cif) < 1. 

Moreover, we assume that 

rank[C*(/J s )] = rank(Ci f ) + rank[C| s (/3 s )] < 2 . 

These two assumptions are equivalent to the following 
set of conditions: 

1. If the robot has more than one fixed wheel, they are 
all on a single common axle. 

2. The centers of the steering wheels do not belong to 
this common axle of the fixed wheels. 

3. The number rank[Ci s (/J s )] is equal to the number of 
steering wheels that can be oriented independently 
in order to steer the robot. 

We call this number the degree of steerability 

8 S = rank[C ls (/? s )] . (24.10) 

If a robot is equipped with more than 8 S steering 
wheels, the motion of the extra wheels must be coordi¬ 
nated in order to guarantee the existence of the ICR at 
each instant. 

We conclude that, for wheeled mobile robot of prac¬ 
tical interest, the two defined indices, <5 m and (5 S , satisfy 
the following conditions: 

1. The degree of mobility satisfies 1 < 5 m < 3. The up¬ 
per bound is obvious, while the lower bound means 
that we consider only cases where motion is possi¬ 
ble. 

2. The degree of steerability satisfies 0 < 8 S < 2. The 
upper bound can be reached only for robots without 
fixed wheels, while the lower bound corresponds to 
robots without steering wheels. 

3. The following is satisfied: 2 < 5 m + <5 S £ 3. 


The case <5 m + <5 S = 1 is not acceptable because it 
corresponds to the rotation of the robot about a fixed 
ICR. The cases 8 m > 2 and 8 S = 2 are excluded because 
according to the assumptions, S s = 2 implies 8 m = 1. 
These conditions imply that only five structures are of 
practical interest, corresponding to the five pairs (<5 m , 8 S ) 
satisfying the aforementioned inequalities, according to 
the following array 

<5 m 3 2 2 1 1 

8 S 0 0 1 1 2 

In the following, each type of structure will be des¬ 
ignated by using a denomination of the form type (5 m , 
8 S ) robot. 


We now briefly describe the five classes of wheeled 
robot structures, pointing out the mobility restriction in¬ 
herent to each class. Details and examples can be found 
in Sect. 24.3 and in [24.5]. 

Type (3,0) Robots 

These robots have no fixed and no steering wheels 
and are equipped only with Swedish or caster wheels. 
Such robots are called omnimobile, because they 
have full mobility in the plane, which means that 
they are able to move in any direction without any 
reorientation. 

Type (2,0) Robots 

These robots have no steering wheels, but either one 
or several fixed wheels with a common axle. Mo¬ 
bility is restricted in the sense that, at a given pos¬ 
ture £(?), the velocity £(t) is constrained to belong to 
a two-dimensional distribution spanned by the vector 
fields R t (0)si and R T (0)«2, where si and S 2 are two 
constant vectors spanning A(Cif). A typical example 
of such a robot is the wheelchair. 


24.2.6 Five Classes 

of Wheeled Mobile Robots 
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Type (2,1) Robots 

These robots have no fixed wheels and at least one 
steering wheel. If there is more than one steering 
wheel, their orientations must be coordinated in such 
a way that rank[Ci s (/J s )] = 5 S = 1. The velocity £(;) 
is constrained to belong to a two-dimensional dis¬ 
tribution spanned by the vector fields R T (0)si(/3 s ) 
and R t (0)s 2(/? s ), where s 1 (/3 S ) and sojySs) are two vec¬ 
tors spanning A(Ci s (/3 s )). 

Type (1,1) Robots 

These robots have one or several fixed wheels on a sin¬ 
gle common axle, and also one or several steering 
wheels, with the conditions that their centers are not lo¬ 
cated on the common axle of the fixed wheels, and that 


their orientations are coordinated. The velocity § ( t ) is 
constrained to belong to a one-dimensional distribution 
parameterized by the orientation angle of one arbitrarily 
chosen steering wheel. Mobile robots built on the model 
of a conventional car (often called car-like robots) be¬ 
long to this class. 

Type (1,2) Robots 

These robots have no fixed wheels, but at least two 
steering wheels. If there are more than two steering 
wheels, then their orientation must be coordinated in 
order to satisfy the condition rank[Ci s (/? s )] = 8 S = 2. 
The velocity |(?) is constrained to belong to a one¬ 
dimensional distribution parameterized by the orienta¬ 
tion angles of two arbitrarily chosen steering wheels. 


24.3 Wheeled Robot Structures 

There are many design alternatives for wheeled mobile 
robots. Design problems of a single-body mobile robot 
include the selection of wheel types, the placement of 
wheels, and the determination of the kinematic parame¬ 
ters. Design objectives should be specified according to 
the target environments and tasks, as well as the ini¬ 
tial and operational costs of a robot. In this section, 
robot structures are classified according to the number 
of wheels, and then features will be introduced focusing 
on commonly adopted designs. 

24.3.1 Robots with One Wheel 

A robot with a single wheel is basically unstable with¬ 
out dynamic control in order to maintain its balance of 
the body. A typical example is a unicycle. As a variation 
of a unicycle, a robot with a rugby-ball-shaped wheel 
can be used in order to improve stability in the lateral 
direction, as studied in [24.6]. 

A spherical robot can also be considered as a single¬ 
wheel robot. A balancing mechanism such as a spinning 
wheel is employed to achieve dynamic stability. This 
approach has advantages including high maneuverabil¬ 
ity and low rolling resistance. However, single-wheel 
robots are rarely used in practical applications, because 
additional balancing mechanisms are required, control 
is difficult, and pose estimation by pure dead reckoning 
is not available. An example of a spherical robot can be 
found in [24.7]. 

24.3.2 Robots with Two Wheels 

In general, there are two types of two-wheel robots, 
as shown in Fig. 24.6. Figure 24.6a shows a bicycle- 
type robot. It is common to steer a front wheel and 


to drive a rear wheel. Since the dynamic stability of 
a bicycle-type robot increases with its speed, a bal¬ 
ancing mechanism is not necessarily required. The 
advantage of this approach is that the robot width can be 
reduced. However, a bicycle type is rarely used because 
it cannot maintain its pose when the robot stands still. 
Figure 24.6b shows an inverted-pendulum-type robot. 
It is a two-wheel differential drive robot. 

It is possible to achieve static stability by accurately 
placing the center of gravity on the wheel axle. How¬ 
ever, it is common to apply dynamic balancing control, 
which is similar to the conventional control problem for 
an inverted pendulum. The size of a robot can be re¬ 
al 

Robot 
chassis 


Active fixed wheel Passive, actively 

steerable wheel 

Active fixed wheel 


Robot 

chassis 


Fig. 24.6 (a) Bicycle-type robot and (b) inverted-pendu- 
lum-type robot 



a 


Active fixed wheel 
























Wheeled Robots I 24.3 Wheeled Robot Structures 583 


duced by using two-wheel robots, when compared with 
robots with more than three wheels. A typical applica¬ 
tion of a pendulum-type robot is to design a structure as 
a four-wheel robot, consisting of two pendulum robots 
connected. Then, the robot can climb stairs by lifting its 
front wheels while the robot reaches the stair. A major 
disadvantage is that control effort is always required for 
dynamic balancing. Examples of inverted-pendulum- 
type robots can be found in [24.8] and [24.9]. 

24.3.3 Robots with Three Wheels 

Since a robot with three wheels is statically stable and 
has a simple structure, it is one of the most widely used 
structures for wheeled robots. There are a large number 
of designs according to the choice of individual wheel 
types. Every wheel introduced in Sect. 24.2.1 can be 
used to construct three-wheel robots. In this section, 
five popular design examples are described (Fig. 24.7): 

1. Two-wheel differential drive 

2. Synchronous drive 

3. An omnimobile robot with Swedish wheels 


4. An omnimobile robot with active caster wheels 

5. An omnidirectional robot with steerable wheels. 

Two-Wheel Differential-Drive Robot 
A two-wheel differential-drive robot is one of the most 
popular designs and is composed of two active fixed 
wheels and one passive caster wheel. The robot can be 
classified as a type (2,0) robot in the nomenclature of 
Sect. 24.2.6. It is possible to extend the robot to a four- 
wheel robot by adding passive caster wheels. The major 
advantages of the robot can be summarized as follows: 

• A simple mechanical structure, a simple kinematic 
model, and low fabrication cost. 

• A zero turning radius is available. For a cylin¬ 
drical robot, the obstacle-free space can easily be 
computed by expanding obstacle boundaries by the 
robot radius r. 

• Systematic errors are easy to calibrate. 

On the other hand, its drawbacks are: 

• Difficulty of moving irregular surfaces. When the 
robot goes over uneven surfaces, its orientation 


Passive caster wheel 






Fig. 24.7 (a) Two-wheel differential drive, (b) synchronous drive, (c) omnimobile robot with Swedish wheels, (d) omn¬ 
imobile robot with active caster wheels, and (e) omnidirectional robot with active steerable wheels 


Part B | 24.3 



































Part B | 24.3 


584 Part B 


Design 


might change abruptly if one of the active wheels 
loses contact with the ground. 

• Only bidirectional movement is available. 

Synchronous-Drive Robot 
A synchronous-drive robot can be built by using cen¬ 
tered or off-centered orientable wheels. The steering 
and driving motions of each wheel are mechanically 
coupled by chains or belts, and the motions are actuated 
synchronously, so the wheel orientations are always 
identical. The kinematic model of a synchronous drive 
robot is equivalent to that of the unicycle, a type (1,1) 
robot. Therefore, omnidirectional motion, i. e., motion 
in any direction can be achieved by steering the wheel 
orientations to the desired velocity direction. How¬ 
ever, the orientation of the robot chassis cannot be 
changed. Sometimes a turret is employed to change the 
body orientation. The most significant advantage of the 
synchronous-drive robot is that omnidirectional move¬ 
ment can be achieved by using only two actuators. Since 
the mechanical structure guarantees synchronous steer¬ 
ing and driving motions, less control effort is required 
for motion control. Other advantages include that 
odometry information is relatively accurate and driving 
forces are evenly distributed among all the wheels. The 
drawbacks of this approach can be summarized as: 

• Complicated mechanical structure. 

• If backlash or loose coupling is present in the chain 
transmission, velocity differences between wheels 
may occur. 

• In order to achieve omnidirectional movement, the 
wheel orientations should be aligned to the desired 
velocity direction before movement, due to the non- 
holonomic velocity constraints. 

Omnimobile Robot with Swedish Wheels 
The omnimobile robot with Swedish wheels corre¬ 
sponds to type (3,0) in the nomenclature of Sect. 24.2.6. 
At least three Swedish wheels are required to build 
a holonomic omnidirectional robot. A major advantage 
of using the Swedish wheel is that omnidirectional mo¬ 
bile robots can be easily constructed. At least three 
Swedish wheels are required to build a holonomic om¬ 
nidirectional robot. Since omnidirectional robots can be 
built without using active steering of wheel modules, 
the mechanical structures of actuating parts can have 
simple structures. However, the mechanical design of 
a wheel becomes slightly complicated. One drawback 
of the Swedish wheel is that there is a vertical vibra¬ 
tion because of discontinuous contacts during motion. 
In order to solve this problem, a variety of mechani¬ 
cal designs have been proposed; examples can be found 
in [24.10] and [24.11]. Another drawback is its rela¬ 


tively low durability when compared to conventional 
tires. An example of a robot using Swedish wheels can 
be found in [24.12]. 

Omnimobile Robot 
with Active Caster Wheels 
A holonomic omnidirectional robot can be constructed 
by using at least two active caster wheels, and the robot 
also belongs to type (3,0). The robot can be controlled 
to generate arbitrary linear and angular velocities re¬ 
gardless of the wheel orientations. Since the robot 
uses conventional tires, the disadvantages of Swedish 
wheels, for example, vertical vibrations or durability 
problems, can be solved. An example can be found 
in [24.13]. The disadvantages of this robot can be sum¬ 
marized as follows: 

• Since the location of the ground contact point (i. e., 
footprint) changes with respect to the robot chassis, 
instability can take place when the distance between 
the wheels is too short. 

• If the robot switches its movement to the reverse di¬ 
rection, an abrupt change of wheel orientations may 
take place. This is called the shopping-cart effect, 
which may result in instantaneous high steering ve¬ 
locities. 

• If a driving motor is directly attached to the wheel, 
wires to the motor will be wound due to steering 
motions. In order to avoid this, a gear train should 
be employed to transmit the input angular veloc¬ 
ity from the driving motor, which is attached to the 
robot chassis. In this case, the mechanical structure 
becomes quite complicated. 

• If a robot is equipped with more than two active 
caster wheel modules, more than four actuators are 
used. Since the minimum number of actuator to 
achieve holonomic omnidirectional motion is three, 
this is an overactuated system. Therefore, actuators 
should be accurately controlled in a synchronous 
way. 

Omnidirectional Robot 
with Active Steerable Wheels 
Centered orientable wheels are also employed to build 
omnidirectional robots; at least two modules are re¬ 
quired. A significant difference between the active 
caster wheel and the centered orientable wheel is that 
the wheel orientation should always be aligned with the 
desired direction of velocity direction, as computed by 
inverse kinematics. This fact implies that this robot is 
nonholonomic and omnidirectional: it is a type (1,2) 
robot. The control problem is addressed in [24.14]. The 
mechanical drawbacks are similar to those of using 
active caster wheels (i. e., many actuators and compli- 
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cated mechanical structures). Since the driving motor is 
directly attached to the driving axis in many cases, al¬ 
lowable steering angles are limited in order to prevent 
wiring problems. 

There are a lot of design candidates for three-wheel 
robots, other than the five designs described earlier. 
They can be classified and analyzed according to the 
scheme presented in Sect. 24.1. The aforementioned 
designs can be extended to four-wheel robots to im¬ 
prove stability. Additional wheels can be passive wheels 
without adding additional kinematic constraints. Active 
wheels can also be added and should be controlled by 
solving the inverse kinematics problem. Four-wheeled 
robots require suspension to maintain contact with the 
ground to prevent wheels from floating on irregular 
surfaces. 

24.3.4 Robots with Four Wheels 

Among the various four-wheel robots, we focus on 
the car-like structure. The car-like structure has been 
called as the Ackermann steering geometry that was 
shown in Fig. 24.3c. The front two wheels should be 
synchronously steered to keep the same instantaneous 
center of rotation. It is clear that the orientations of two 
front wheels are slightly different because the curva¬ 
tures of rotation are different. As a result, this solution 
is kinematically equivalent to a single orientable wheel 
and the robot can be classified as a type (1,1) robot. 
A major advantage of a car-like robot is that it is 
stable during high-speed motion. However, it requires 
a slightly complicated steering mechanism. If the rear 
wheels are actuated, a differential gear is required to 
obtain pure rolling of the rear wheels during the turning 
motion. If the steering angle of the front wheel cannot 
reach 90°, the turning radius becomes nonzero. There¬ 
fore, parking motion control in a cluttered environment 
becomes difficult. 

24.3.5 Special Applications 
of Wheeled Robots 

Articulated Robots 

A robot can be extended to an articulated robot, which 
is composed of a robot and trailers. A typical exam¬ 
ple is the luggage-transporting trailer system at airports. 
By exploiting trailers, a mobile robot obtains vari¬ 
ous practical advantages. For example, modular and 
reconfigurable robots can change their configuration ac¬ 
cording to service tasks. A common design is a car with 
multiple passive trailers, which is the simplest design 
of an articulated robot. I<S3>3BE5E53I shows another ex¬ 
ample of a trailer robot. From the viewpoint of control, 
some significant issues have been made clear, including 



Fig. 24.8 An active trailer system (after [24.16]) 



Fig. 24.9 A mobile robot for rough terrain (after [24.17]) 


a proof of controllability and the development of open- 
and closed-loop controllers using canonical forms such 
as the chained form. The design issues for trailer sys¬ 
tems are the selection of wheel types and decisions 
regarding the link parameters. In practical applications, 
it is advantageous if trailers can move along the path of 
the towing robot. Passive trailers can follow the path of 
a towing robot within a small error by using a special 
design of passive steering mechanism for trailers; see, 
for example, [24.15], 

On the other hand, active trailers can be used. There 
are two types of active trailers. A first approach is to 
actuate wheels of trailers. The connecting joints are 
passive, and two-wheel differential-drive robots can be 
used as active trailers. By using this type of active trail¬ 
ers, accurate path-following control can be achieved. 
The second approach is to actuate connecting joints. 
The wheels of the trailer are passively driven. By appro¬ 
priate actuation of the connecting joints, the robot can 
move without wheel actuation, by snake-like motions. 
As an alternative design, we can use an active prismatic 
joint to connect trailers, in order to lift the neighboring 
trailer. By allowing vertical motion, a trailer system can 
climb stairs and traverse rough terrain. Examples of ac¬ 
tive trailers can be found in [24.16]. 
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Hybrid Robots 

A fundamental difficulty of using wheels is that they 
can only be used on flat surfaces. To overcome this 
problem, wheels are often attached to a special link 
mechanism. Each wheel is equipped with independent 
actuators and a linkage mechanism enables the robot 
to adapt its configuration to irregular ground condi¬ 
tions (l-g&M'il'Jt'HfM ). A typical design can be found 
in [24.17] and can be understood as a hybrid robot 


that is a combination of a legged robot and a wheeled 
robot. Another hybrid example is a robot equipped with 
both tracks and wheels. Wheels and tracks have com¬ 
plementary advantages and disadvantages. Wheeled 
robots are energy efficient; however, tracked robots 
can traverse rough terrain. Therefore, a hybrid robot 
can selectively choose its driving mechanism according 
to environmental conditions, although fabrication cost 
increases. 


24.4 Wheel-Terrain Interaction Models 


A wheeled robot’s mobility properties are governed 
by forces generated at the wheel-terrain contact in¬ 
terface. The ability to accurately model wheel-terrain 
interaction forces is therefore an important aspect of 
robot design, simulation, and control. These forces 
are strongly influenced by the relative stiffness of the 
wheel and terrain. Generally, there are four possible 
wheel-terrain interaction cases. The first case is that of 
a rigid wheel traveling on rigid terrain (Fig. 24.10a). 
The second case is that of a rigid wheel traveling on 
deformable terrain (Fig. 24.10b). The third case is that 
of a deformable wheel traveling on deformable terrain 
(Fig. 24.10c). The fourth case is that of a deformable 
wheel traveling on rigid terrain (Fig. 24.10d). Models 
for these four cases are presented as follows. It should 
be noted that while many different types of models (i. e., 
finite element, discrete element, empirical) have been 
developed for each of these four cases, the focus here is 
on analytical models that are broadly suitable for anal¬ 
ysis, simulation, design, and control purposes. 

24.4.1 Rigid Wheels on Rigid Terrain 

Many robots employ wheels made of metal, stiff rubber, 
or other materials that deform very little when subject 
to loading experienced during operation. When oper¬ 
ated on stiff surfaces such as indoor flooring, pavement, 
or stone, wheel-terrain interaction can be reasonably 
approximated as a point contact. An interaction model 
based on classical Coulomb friction can then be em¬ 
ployed to describe bounds on available tractive forces, 
F x , and lateral forces, F y , as a function of the load on 
the wheel, W, for a robot with n wheels traveling on 
a surface with coefficient of friction fi 

F xi <fi x Wi, i = 1... n , ( 24 . 11 ) 

Fyi < /J.yWj , i = 1 ... n . ( 24 . 12 ) 

Since the frictional force can be generated in any 
direction, and its magnitude is limited, a bound on the 
norm of the frictional and lateral forces can be ex¬ 


pressed as 



Equation (24.13) represents a concept known as the 
friction ellipse (Fig. 24.1 1). When the effective friction 
is equal in all directions the ellipse becomes a circle. 

24.4.2 Rigid Wheels on Deformable Terrain 

Robotic locomotion in outdoor, off-road terrain fre¬ 
quently results in deformation of the terrain, especially 
when the terrain is composed of a deformable mate¬ 
rial such as sand, silt, loam, or clay. When the wheel 
is constructed of a rigid material such as metal or stiff 
rubber, or the tire inflation pressure is high enough such 
that tire deformation is small, wheel-terrain interaction 
occurs along an arc of the wheel, rather than at a sin¬ 
gle point (Fig. 24.10b). In such scenarios, the Coulomb 
friction model described in Sect. 24.4.1 does not accu¬ 
rately represent the relationship between the wheel load 




Fig.24.10a-d Four cases of wheel-terrain interaction me¬ 
chanics: (a) rigid wheel traveling over rigid terrain, 
(b) rigid wheel traveling over deformable terrain, (c) de¬ 
formable wheel traveling over deformable terrain, and 
(d) deformable wheel traveling over rigid terrain 
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Fig. 24.11 Illustration of friction ellipse concept to repre¬ 
sent constraint on longitudinal and lateral friction 


and tractive forces. This is because the mechanism for 
force generation derives primarily from shearing along 
failure planes in the terrain, rather than from frictional 
contact at a point on the wheel-terrain interface. 

Models of the interaction of rigid wheels on de¬ 
formable surfaces were developed by Bekker in the 
1950s and 1960s in the context of large military ve¬ 
hicles [24.18]. Bekker’s research gave rise to the dis¬ 
cipline of terramechanics, i. e., the study of vehicle- 
terrain interaction phenomena. Bekker’s terramechanics 
models have subsequently been applied to robotic sys¬ 
tems at a range of sizes [24.19] and [24.20]. However, 
Bekker performed his original analyses in the context 
of large military vehicles, which were typically several 
meters in length and with gross vehicle weights exceed¬ 
ing 500kg. The accuracy of such models when used to 
analyze the performance of small robotic systems re¬ 
mains an open research question [24.21] and [24.22]. 

A description of terramechanics models for the case 
of rigid wheels on deformable surfaces is presented 
in Chap. 55, in the context of Space Robotics. How¬ 
ever, Chap. 55 assumes that the wheel is constructed 
of a rigid material such as metal, which is typical for 
planetary exploration rovers but rare for robots oper¬ 
ating on Earth. As noted earlier, a pneumatic tire will 
behave as a rigid wheel if the average tire ground pres¬ 
sure, which is a function of the tire carcass stiffness and 
inflation pressure, exceeds some critical ground pres¬ 
sure. The average tire ground pressure can be obtained 
experimentally by measuring the tire contact patch area 
while the tire is subjected to a series of known vertical 
loads and inflation pressures. The critical ground pres¬ 
sure is defined as 

(ko \ <5TFIT / 31V \2^FT 

+ 4 J V ( 3 — n)bu\/T)) ’ 

( 24 . 14 ) 


where k c , k$, n are the soil-dependent parameters, b is 
the smallest dimension of the (typically rectangular) 
wheel-terrain contact patch, b t [ is the tire width, TV is 
the vertical load on the tire, and D is the tire diameter. 

If the average tire ground pressure exceeds the crit¬ 
ical ground pressure, a pneumatic tire can be modeled 
as a rigid wheel interacting with a deformable surface. 
This scenario is governed by the equations presented 
in Chap. 55. If the average tire ground pressure does 
not exceed the critical ground pressure, a pneumatic tire 
can be modeled either as a deformable wheel interact¬ 
ing with deformable terrain, or as a deformable wheel 
interacting with rigid terrain. Models of these scenarios 
are presented in Sects. 24.4.3 and 24.4.4. 

24.4.3 Deformable Tires 

on Deformable Terrain 

When the average tire ground pressure does not exceed 
the critical ground pressure, both the tire and the terrain 
surface may undergo deformation. Accurate modeling 
of this interaction thus requires consideration of phe¬ 
nomena related to both soil deformation (most notably, 
soil compaction, and shear deformation) and tire defor¬ 
mation. Several semiempirical models for this scenario, 
based on Bekker theory, have been developed in recent 
years. 

A simple approach to modeling in this interaction 
scenario was initially proposed by Bekker, and further 
developed by Harnisch [24.23]. This approach relies 
on the observation that tire deformation serves to in¬ 
crease the size of the wheel-terrain contact patch, and 
thus reduce the average ground pressure and minimize 
sinkage. Bekker proposed the substitute circle concept, 
which embodies the notion that the deformable tire can 
be equivalently represented by a rigid tire of increased 
diameter, to yield identical sinkage (Fig. 24.12). The di¬ 
ameter of the larger rigid tire can be computed via the 
following method: 

1. Assuming zero slip, sinkage for the original tire di¬ 
ameter D, assumed to be rigid, is calculated for the 
vertical load IV and soil parameters. 

2. Knowing the tire load-deflection characteristics, for 
the given inflation pressure the tire deflection S can 
be calculated. 

3. The diameter of the larger substitute circle D' will 
correspond to the diameter of the circle that passes 
through Pi, P 2 , and P 3 . 

Once the outer circle diameter is known, all other 
quantities will be calculated following the standard 
methodology presented in Chap. 55, substituting D' for 
the original wheel diameter D. 
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An alternative approach, based on a more physi¬ 
cally realistic model of the deformed tire geometry, was 
proposed by Wong [24.24]. Here, the tire geometry is 
represented as a circular arc with a perfectly flat (i. e., 
horizontal) base (Fig. 24.13). The pressure distribution 
under the flat section BC is assumed to be uniform, and 
equal to the average ground pressure of the tire p g . Dis¬ 
regarding shear effects and soil elastic rebound (section 
CD), it is possible to calculate the tire deflection 8 by 
balancing vertical forces 


W = bp g k + 1T ab , 


(24.15) 


where b is the tire width, l, is the length of the flat sec¬ 
tion BC, and Wab is the vertical reaction exerted by the 
soil along the undeformed tire section AB 


W AB 



Vd^o + S )"- 1 


(3-«)(z 0 + <$)2 -(3 -n)Si -3zoVS 


3 


(24.16) 



Fig. 24.13 Diagram of deformable wheel traveling over 
deformable terrain 


where D is the undeformed tire diameter and zo is the 
static sinkage that can be calculated through standard 
pressure-sinkage relations 


i 



f Ps \ 


\ (.kc/ld+kcj, ) 


( Ps 'l 




(/ t < b) 
(k > b) 


(24.17) 


where / t = 2 ~JD8 — S 2 is the contact patch length. Al¬ 
though not strictly correct, the tire ground pressure 
p g can be approximated by the tire inflation pressure, 
a known variable. Following this assumption, it is pos¬ 
sible to solve (24.16) for the tire deflection 8. 

Once the deformed tire shape has been calculated, 
all other relevant quantities (i. e., sinkage, drawbar 
force, and torque) can be calculated. Since the tire pro¬ 
file is nonsmooth, the normal and tangential stresses are 
described by piecewise expressions. The vertical load 
balance can be written as 


"c 

7 


W = br a cos 9 — r sin 0)d$ + bDp g sin 9 C 



cos 9 + r sin 8)d6 . 


(24.18) 


The aforementioned equation can be solved for the 
entry angle 9{. The angle 9 C can be derived from the tire 
deflected shape. Once the tire-terrain geometry is de¬ 
termined, the sinkage can be found. Note that this new 
sinkage will differ from the static sinkage zo- It is then 
possible to calculate drawbar pull F x and driving torque 


9 C It 

F x = br I (a cos 9 + r sin 9)d9 + b rdv 


7 < 

e. 

Of 

/< 


+ br / (— a cos 9 + r sin 9)d9 , (24.19) 


6 C l\ Of 

T = br 2 / rd9 + br cos 9 C j rcbr + br 2 J td9 . 


0, 


(24.20) 


Another flexible tire model developed by Senatore 
and Sandu is similar to the approach proposed by Wong, 
however it calculates the tire deflection as a function 
of inflation pressure, and also models multi-pass ef¬ 
fects [24.25]. Generally speaking, all of the flexible tire 
models proposed to date have various advantages and 
drawbacks, and none have gained universal approval in 
the research community. 
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24.4.4 Deformable Tires on Rigid Terrain 

In scenarios where the tire stiffness is significantly 
lower than the terrain stiffness, only the tire will expe¬ 
rience significant deformation. This scenario has been 
studied extensively in the automotive community, since 
it is the typical interaction scenario for passenger and 
commercial vehicles operating on paved roads. Here, 
two common models of steady-state tire-terrain inter¬ 
action are briefly described: the brush model and the 
magic formula model [24.26]. 

The brush model models the tire contact patch as 
a row of elastic bristles in contact with the ground. Car¬ 
cass, belt, and tread element compliance is captured by 
a lumped bristle compliance. For the case of pure lon¬ 
gitudinal slip, the tractive force can be calculated as 

F x = 2c px a 2 ^, (24.21) 

where c px is the longitudinal tread element stiffness 
per unit length and other quantities are illustrated in 
Fig. 24.14. 

In a similar manner, the lateral force can be calcu¬ 
lated as follows 

F y = 2c py a 2 a, (24.22) 

where a is the slip angle. As previously discussed for 
the case of rigid wheels on rigid terrain, combined slip 
operation will limit the availability of longitudinal and 
lateral thrust [24.26]. 

The magic formula model is an empirical model 
that has found widespread application due to its 
flexibility and reduced computational burden. The 
model is based on a combination of trigonometric 
functions that generate a curve that (usually) passes 
through the origin, reaches a peak value, and then tends 
to a horizontal asymptote: this behavior is typical of 
force/moment versus slip characteristics of modern 
tires, regardless of tire size, construction, inflation 
pressure, and other characteristics. The general form of 
the magic formula model is 

y = D sin{C arctan[/ft — E(Bx — arctan Bx)]} , 

(24.23) 

24.5 Wheeled Robot Suspensions 

A suspension is a system of linkages, springs, dampers, 
and actuators that govern the relative motion between 
a robot’s wheels and body. These mechanisms include 
a degree of freedom allowing the wheel to spin, and op¬ 
tional degrees of freedom for steering and vertical trans¬ 
lation. Suspension mechanisms are useful when driving 



Fig. 24.14 Brush tire model for analyzing interaction of 
deformable tire and rigid terrain 



Fig. 24.15 Diagram illustrating influence of parameters of 
magic tire model 


with Y(X) = y(x) + Sy and x = X + 5h- Here, Y repre¬ 
sents the output variable, which may be F x , F y or M : , 
and X represents the input variable, which may be tan a 
or k (i. e., the lateral or longitudinal slip). The relation 
also contains several empirically determined values, as 
follows: B represents a stiffness factor; C a shape fac¬ 
tor; D a peak value; E a curvature factor; .S’n a horizontal 
shift; and Sy a vertical shift. These factors are illustrated 
in Fig. 24.15. 


over uneven surfaces for several reasons. First, a sus¬ 
pension allows the wheels to maintain contact with the 
ground despite disturbances from the uneven ground 
surface, which allows generation of traction, braking, 
and cornering forces. Additionally, suspensions provide 
load-bearing and dissipative elements (i. e., springs, 
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dampers, and active or semiactive actuators) to mitigate 
the effect of disturbances on robot body motion, and 
thus allow for improved obstacle negotiation and sen¬ 
sor or payload stabilization. Key issues in suspension 
design include the design of suspension linkage kine¬ 
matics and selection of load-bearing and dissipative el¬ 
ements. 

Suspension linkage kinematics are designed to al¬ 
low vertical displacement of the wheels. Typical sus¬ 
pension mechanisms include simple prismatic joints, 
rotational joints with a wheel attached to a trailing arm, 
and 4-bar linkages. For car-like wheeled robots, sub¬ 
stantial inspiration can be drawn from the design of 
passenger vehicle suspensions, which has received vast 
research attention [24.27]. A comprehensive treatment 
of the design of suspension linkage kinematics is be¬ 
yond the scope of this chapter. Interested readers are 
referred to [24.28]. 

The selection of load-bearing and dissipative ele¬ 
ments is typically based on analysis of low-order mod¬ 
els of the suspension subject to expected operational 
conditions. Common methods for modeling passive and 
semiactive suspensions are presented in the following. 
Analysis of these models allows for principled selection 
of springs, dampers, and active or semiactive actuators 
that form the basis of nearly all wheeled robot suspen¬ 
sions. 

24.5.1 Passive Suspension Dynamics 

Arguably the most common passive suspension design 
consists of a parallel spring and damper mounted be¬ 
tween the robot body and wheels. A simple model for 
analyzing the vertical dynamics of this suspension is 
commonly referred to as the quarter car model [24.29] 
and is illustrated in Fig. 24.16. The fraction of body 
mass supported by the suspension element, m b , is con¬ 
nected to a wheel with mass m w by a spring element 
with stiffness k s and a damping element with damping 



Fig.24.16a,b Quarter car model for passive suspension 
modelling (a) and quarter car model with ideal sky-hook 
semi-active suspension control (b) 


coefficient b s . The wheel stiffness is denoted by k w , and 
transmits vertical excitation from the uneven terrain sur¬ 
face. The heights of the body mass, wheel mass, and 
terrain surface are given by Zb, Zw. and z u , respectively. 

A state vector x for this model is composed of 
the suspension displacement d s = z b ~ z w , wheel spring 
displacement <7 W = 7 W —z u , and velocities Zb, z w , as 
shown in (24.24), along with a linear state space 
model (24.25)-(24.27). Note that the input to the model 
is the vertical terrain velocity z u . This is a convenient 
formulation for linear systems analysis since z u can be 
modeled as white noise with intensity proportional to 
the vehicle’s forward speed and a road roughness pa¬ 
rameter [24.29]. 
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(24.26) 

(24.27) 


To investigate the properties of this model, transfer 
functions G(s) = Y(s)/U(s) are computed for outputs 
of interesty = Cx as 

G(s) = CQI —A) -1 B . (24.28) 


With a white noise input, all frequencies are equally 
represented. Thus, the magnitude of the transfer func¬ 
tion \G(ja>)\ at a given frequency w represents the 
frequency response of each output. Two outputs are of 
particular interest for suspension design purposes: the 
wheel displacement d^ and body acceleration Zb- This 
is because the wheel displacement corresponds to vari¬ 
ations in the wheel contact force, and thus informs the 
road holding and maneuvering capability of the robot. 
The body acceleration represents the amount of vibra¬ 
tion experienced by the sensors and payloads mounted 
on the robot body. 

The output matrix for the wheel displacement dm is 
C = (0 1 0 0), (24.29) 


and the output matrix for the body acceleration Zb is 
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(24.30) 
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Fig.24.17a,b Wheel displacement frequency response for passive suspension (a) and semiactive suspension. Wheel dis¬ 
placement frequency corresponds to wheel contact force variation and road holding capability 
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Fig.24.18a,b Body acceleration frequency response for passive suspension (a) and semiactive suspension (b). Body 
acceleration frequency corresponds to the vibration of sensors and payloads mounted on the robot body 


The frequency response of wheel displacement and 
body acceleration are shown in Figs 24.17 and 24.18, 
respectively. These plots are shown with the follow¬ 
ing representative parameter values for a passenger 
vehicle: the ride frequency 04 , = y/k s /m \, = 2 jt rad/s, 
the wheel hop frequency <y w = y/k^/m^ = 20 it rad/s, 
the mass ratio p = m w //Mb = 0.1, and the damping ra¬ 
tio £ = b s /(2-y/kJn^) = (0.2,0.7,1.2). Various damp¬ 
ing ratios are shown to illustrate the behavior of 
underdamped, critically damped, and over-damped 
suspensions. 

For the underdamped case, where £ = 0.2, reso¬ 
nant peaks (which are generally undesirable) exist in 


the frequency response near < 24 , and <y w for both the 
wheel displacement and body acceleration. For the 
over-damped case, where £ = 1.2, the peak response 
occurs between the resonant peaks. For the (approxi¬ 
mately) critically damped case, where £ = 0.7, the peak 
responses are reduced, and for this reason many suspen¬ 
sion are designed to exhibit critical damping. 

24.5.2 Semiactive Suspension Dynamics 

The introduction of controllable actuators to the sus¬ 
pension allows for shaping of the system’s dynamic 
response, and can improve road holding performance 
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and reduce body vibration. Suspension systems inte¬ 
grated with controllable actuators are referred to as 
active or semiactive suspensions. Active suspensions 
refer to systems with powered rotational or linear actua¬ 
tors capable of both supplying and absorbing significant 
amounts of energy, such that the actuator effectively 
replaces the passive components in a traditional suspen¬ 
sion. Such systems are rare due to their complexity and 
cost. A simpler and more inexpensive (and therefore 
more common) approach to controllable suspensions 
is to employ semiactive elements. This refers to ele¬ 
ments that can absorb energy in a controlled manner, 
such as active (e.g., magnetorheological) fluid dampers. 
Semiactive suspensions can be modeled in a manner 
similar to passive suspensions with a time-varying, pos¬ 
itive damping coefficient b s (t) > 0. 

Semiactive suspension control schemes attempt 
to control the damping coefficient b s (t ) to emulate 
dampers with coefficients b, b acting on the absolute ve¬ 
locities of the body and wheels, in order to reduce the 
effect of wheel vibration on the robot body. A common 
technique known as a sky-hook controller is illustrated 
in Fig. 24.16. The technique can be approximated by 
applying the damper force F given in the following. 
Note that a semiactive system can only approximate this 
control, as F is constrained as F(z — z) < 0. 

F=—b'z+bz (24.31) 


24.6 Conclusions 

Wheels are the most commonly employed running 
gear for mobile robots due to their relative simplic¬ 
ity, robustness, and low cost. The number of possible 
wheeled mobile robot realizations is almost infinite, 
depending on the number, type, implementation, geo¬ 
metric characteristics, and motorization of the wheels. 
This chapter has described several such realizations. 
Notwithstanding this variety, it is possible to classify 
WMRs into only five generic categories. This catego- 

Video-Refe rences 


To illustrate the potential improvement achievable 
by the use of a semiactive suspension with a sky-hook 
controller, frequency responses of the semiactive sus¬ 
pension quarter car model with F given earlier are 
plotted in Figs 24.17 and 24.18. The model parame¬ 
ters are identical to the values from the previous section 
for critical damping f = 0.7, with damping coefficients 
b = b and b = ( b , 0.5 b, 0). Note that the case b = b is 
identical to the passive, critically damped case. 

With no wheel damping, b = 0, both frequency 
responses are improved for co <£ co, and the body ac¬ 
celeration response is improved for 6)»a). Near co, 
however, there exist substantial resonant peaks for both 
outputs. With moderate wheel damping b = 0.5 b, the 
amplitude of the resonant peaks near co are reduced and 
both peak responses are reduced, though performance 
improvements of b = 0 are not matched for frequencies 
far from co. 

While the sky-hook and other suspension control 
algorithms can improve the dynamic response of the ve¬ 
hicle, it is important to note some inherent trade-offs in 
semiactive and active suspension control. It has been 
proven that road holding and attenuation of body vi¬ 
bration can be concurrently improved through active 
control for frequencies below the wheel hop frequency, 
co [24.29]. At frequencies above co , however, any at¬ 
tempts to improve one of these quantities necessarily 
causes the other to degrade. 


rization aids understanding of wheel structures through 
simplification. Practical robot structures have been clas¬ 
sified according to the number and type of wheels. 
Wheel-terrain interaction models have been presented 
to allow the analysis of tractive force generation ca¬ 
pability, for design and simulation purposes. Finally, 
a brief description of common suspension systems has 
been presented, including a presentation of structures 
and dynamic models. 
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An omnidirectional mobile robot with active caster wheels 

available from http://handbookofrobotics.org/view-chapter/24/videodetails/325 

Articulated robot - a robot pushing 3 passive trailers 

available from http://handbookofrobotics.org/view-chapter/24/videodetails/326 
An omnidirectional robot with 4 Mecanum Wheels 

available from http://handbookofrobotics.org/view-chapter/24/videodetails/327 
An omnidirectional robot with 4 Swedish wheels 

available from http://handbookofrobotics.org/view-chapter/24/videodetails/328 

An innovative space rover with extended climbing abilities 

available from http://handbookofrobotics.org/view-chapter/24/videodetails/239 
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25. Underwater Robots 



Covering about two-thirds of the earth, the ocean 
is an enormous system that dominates processes 
on the Earth and has abundant living and non¬ 
living resources, such as fish and subsea gas and 
oil. Therefore, it has a great effect on our lives 
on land, and the importance of the ocean for 
the future existence of all human beings can¬ 
not be overemphasized. However, we have not 
been able to explore the full depths of the ocean 
and do not fully understand the complex pro¬ 
cesses of the ocean. Having said that, underwater 
robots including remotely operated vehicles (ROVs) 
and autonomous underwater vehicles (AUVs) have 
received much attention since they can be an ef¬ 
fective tool to explore the ocean and efficiently 
utilize the ocean resources. This chapter focuses 
on design issues of underwater robots including 
major subsystems such as mechanical systems, 
power sources, actuators and sensors, computers 
and communications, software architecture, and 
manipulators while Chap. 51 covers modeling and 
control of underwater robots. 
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25.1 Background 

The ocean covers about two-thirds of the earth and 
has a great effect on the future existence of all human 
beings. Underwater robots or unmanned underwater ve¬ 
hicles (UUVs) can help us better understand marine and 
other environmental issues, protect the ocean resources 
of the earth from pollution, and efficiently utilize them 
for human welfare. 

Most UUVs commercially available in the market 
are tethered and remotely operated, referred to as re¬ 
motely operated vehicles (ROVs). The operators on 
the mother vessel control the ROV by sending control 


signals and power to the vehicle and receiving video, 
status, and other sensory data via an umbilical cable. 
Most work-class ROVs are equipped with one or mul¬ 
tiple video cameras and lights, sonars, a still camera, 
one manipulator, one grabber, and a wide range of sam¬ 
pling devices. The large deep-sea ROVs use a tether 
management system (TMS) that allows the vehicle to 
be deployed using a strong but heavy umbilical ca¬ 
ble and then flown out from the TMS using a lighter, 
more flexible cable. ROVs must have high bandwidth 
for high-resolution video and data transmission. 
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The US Navy funded the early ROV technology 
development in the 1960s. They were used to recover 
underwater ordnance. ROVs have been used for various 
applications including military operations, environmen¬ 
tal systems, scientific missions, ocean mining, and gas 
and oil industry [25.1]. Visiongains analysis indicates 
that total spending on ROVs in the energy industry will 
reach about $2.7 billion in 2022 as ROVs are required 
for more inspection, maintenance, drill support, and de¬ 
commissioning activities than ever before. 

However, since most commercial ROV systems 
currently have some operational issues such as very 
high operational costs with mother vessels and opera¬ 
tor fatigue, the demand for advanced underwater robot 


25.2 Mechanical Systems 

Every 10 m adds approximately 1 atm of water pres¬ 
sure on a robot body and the chemical environment of 
the seawater is highly corrosive. Therefore, mechani¬ 
cal systems that have excellent structural strength, and 
excellent chemical and environmental resistance are 
needed. 

An underwater robot’s mechanical structure can be 
divided into three functional subsystems: frame, pres¬ 
sure vessel, and fairing. Frame is the primary skeleton 
of the robot. It defines the robot’s shape, supports its 
weight both in and out of the water, provides mechani¬ 
cal support, resists hydrodynamic forces of motion and 
additional force due to operation of tools, and pro¬ 
tects against any damages [25.3]. Its design must be 
based on structural analysis. A pressure vessel should 
be a waterproof structure and it must be strong enough 
to withstand water pressure at a target design depth. 
Typically, a cylindrical or spherical shape is the most 
effective way to maximize strength at the expense of 
low space efficiency. A pressure vessel with a larger 
diameter would result in increased cost of materials 
and machining since it requires increasing the thickness 
of the pressure vessel for the same strength. Conse¬ 
quently, the pressure vessel would become heavier. 
Therefore, selecting appropriate materials for a pres¬ 
sure vessel is very important. Fairing is like a skin 
of the robot. A fairing that covers all or parts of an 
underwater robot offers some advantages, such as an 
aesthetically pleasing exterior, a decreased drag force, 
and protection. 

25.2.1 Frames and Pressure Hulls 

The frame must be designed considering the following 
operational conditions: 


technologies that would lead to next generation work- 
class ROV systems or autonomous underwater vehicles 
(AUVs) has been growing. A survey paper on AUVs 
was published by Yuh [25.2] in 2000. The paper de¬ 
scribes key areas, such as dynamics, control systems, 
navigation, sensors, communications, power systems, 
pressure hulls, fairing, and mechanical manipulators. 
It also presents existing AUV systems and components 
commercially available in the market then. 

This chapter focuses on design issues with each ma¬ 
jor subsystem of underwater robots, such as mechanical 
systems, power sources, actuators and sensors, com¬ 
puters and communications, software architecture, and 
manipulators. 


1. On the ground without motion 

2. Underwater in motion 

3. Hanging from a crane for launch and recovery. 

The major points of acting force for each condi¬ 
tion are totally different. The static gravitational force is 
enough to be considered for the first case. For the second 
case, we should consider buoyancy force, gravitational 
force as well as all thruster forces in a dynamic state. 
During the launch and recovery stages, the enormous 
snap force, due to the mother ship’s vertical motion can 
be concentrated on a lifting point of the robot. 

The structural design and analysis must be con¬ 
ducted to take advantages of shapes offering high 
strength and stiffness for their weight. At the same time, 
materials should be chosen to give the maximum struc¬ 
tural performance with the minimum weight [25.4]. An 
important point is that weights in both air and wa¬ 
ter must be carefully calculated. One should note that 
buoyancy materials will also increase the overall weight 
of the robot in air. A factor of safety is one of crucial 
parameters in designing mechanical systems. Incorpo¬ 
rating a factor of safety may result in 50% overdesign 
for a manned system and 10% overdesign for an un¬ 
manned system [25.5]. Figure 25.1 shows a deep-sea 
ROV, HEMIRE’s open frame that is made of aluminum. 
HEMIRE was developed by Korea Research Institute of 
Ships and Ocean Engineering (KRISO) (l<sz>lKM£I&l). 
On the other hand, a closed frame or a fairing is desir¬ 
able for AUVs, due to limited on-board power sources 
for required hydrodynamic performance. 

The first and ultimate goal in designing a pressure 
vessel is to provide a watertight environment, where 
on-board computers, sensors, or normal batteries are in¬ 
stalled. Therefore, two important design factors are wa- 
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terproof sealing design and water pressure-resistant de¬ 
sign. When given specifications, such as an inside diam¬ 
eter (ID) or an outer diameter (OD), length, thickness, 
and materials, we must analyze buckling stress and 
yield stress. They are the dominant factors for the deep- 
sea (high pressure) environment [25.6], A spherical- 
or cylindrical-shaped pressure vessel would be good 
to withstand the hydrostatic pressure; however, a pres¬ 
sure vessel of cylindrical shape is considered a better 
alternative as it provides a better placement for the com¬ 
ponents. Hydrodynamic design would not be needed 
for pressure vessels placed inside a frame with fair¬ 
ing. However, if the body of an underwater robot is 
designed as a pressure vessel, hydrodynamics in design¬ 
ing a pressure vessel must be considered to reduce the 
drag, cavitation, and acoustic noise [25.7]. 

Frames and pressure vessels must have the struc¬ 
tural analysis using the finite element method (FEM) 
software, and nondestructive tests and/or pressure test¬ 
ing to avoid any mechanical defects. Pressure testing 
is the best way to reveal hidden defects in materials 
and in machining process even though it requires addi¬ 
tional time, cost and nontrivial facilities. It is a critical 
environmental test because a hydrostatic pressure test, 
cyclic pressure test for fatigue test due to repeated de¬ 
ployments, and failure mode test to compare a designed 
crush depth with its real depth are all involved [25.5], 
Pressure testing facilities provide not only a cham¬ 
ber for external pressure testing but also various types 
of measurement sensors and systems to record any 
changes during the testing. 

25.2.2 Materials and Buoyancy 


Besides high strength and high stiffness, one should 
consider good corrosion resistance and machinability 



Fig. 25.1 Aluminum frame of an ROV, Hamire, developed 
by KRISO, Korea 


in choosing materials for frames and pressure ves¬ 
sels [25.7]. 

A typical underwater robot for shallow water 
tends to use aluminum, polycarbonate (PC), and poly- 
oxymethylene (POM), while high strength steel, tita¬ 
nium, and composites are considered for a deep-sea 
underwater robot. The hybrid ROV, Nereus, is for 
a depth of 11 000 m and employs two pressure vessels 
composed of 96% alumina ceramic because its high 
compressive strength-to-weight ratio allows for near 
neutrally buoyant pressure vessels capable of going to 
these extreme depths [25.8]. Basic properties of materi¬ 
als are described later. 

Steel is commonly used because of its high tensile 
strength and low cost. For marine applications, surface 
treatment of steel must be done because it is suscepti¬ 
ble to corrosion or rusting. It tends to distort magnetic 
fields. Type 316 stainless steel is usually regarded as 
the standard marine grade and it is typically used for 
hydraulic, pneumatic and fasteners, but it is difficult to 
machine and weld. The aluminum alloy is lightweight 
and has high strength. It is of reasonable value and not 
related to magnetic effects. Since it is vulnerable to 
corrosion, surface treatment is also required. High heat 
conductivity is another advantage for a pressure vessel 
of aluminum alloy. The titanium alloy has a very high 
strength-to-weight ratio, excellent corrosion resistance 
without any surface treatment, low electric conductiv¬ 
ity, and no magnetic field distortion. Drawbacks include 
low heat conductance, difficulties in machining and 
welding, and high cost. 

The most commonly used composite for marine 
applications is glass-fiber reinforced plastic (GFRP), 
which has a very high strength-to-weight ratio, excel¬ 
lent corrosion resistance, and a competitive price tag. 
Carbon fiber reinforced plastic (CFRP) performs bet¬ 
ter, but it is more expensive. For deep-sea applications, 
CFRP would bring lots of advantages. The pressure ves¬ 
sel of the Seaglider for 6000 m was made of CFRP 
because of its ability to achieve weight to displacement 
ratios of less than 0.5 [25.9]. However, it has not been 
popular in the market mainly because of difficulties in 
machining. Metal matrix composites (MMC) and ce¬ 
ramic alloy have many advantages over other materials. 
In particular, they have high elastic modulus and hard¬ 
ness, high melting points, low thermal expansion, and 
good chemical resistance. 

Instead of metals and composite materials, acrylic 
(plexiglas), POM (polyoxymethylene, acetal, del- 
rin), PC (polycarbonate lexan), ABS (acrylonitrile- 
butadiene-styrene), PVC (polyvinyl chloride), and 
plastic are alternatives for underwater robots operat¬ 
ing in shallow-water environments. The main advan¬ 
tages of the abovementioned materials are excellent 
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Table 25.1 Properties of materials for an underwater robot 


Specification 

Density (kg/dm 3 ) 
(typical) 

Tensile strength (MPa), 

Ultimate (typical)A'ield (typical) 

Modulus of elasticity (GPa) 
(typical) 

Steel (HY80) 

7.87 

482-689/276-517 

190-210 (200) 

Stainless steel (Type316) 

7.75-8.1 

620-560/415-290 

193 

Aluminum 6061-T6 

2.7 

310/276 

68.9 

Aluminum 7075-T6 

2.81 

572/503 

71.7 

Titanium Ti-6A1-4V 

4.43 

950/880 

113.8 

GFRP, E-glass 

2.44-2.48 

4800—4900/NA 

85-90 

GFRP, S-glass 

2.54-2.60 

3400—3500/NA 

65-75 

CFRP 

1.75-1.95 

2500—6000/NA 

200-800 

Acrylic 

0.7-1.25 (1.16) 

19.3-85.0 (64.3)/25.0—85.0 (60.2) 

0.950-3.79 (2.69) 

Polycarbonate 

0.95-1.54(1.2) 

46.1-124 (64.6)/37.0—191 (64.3) 

1.80-7.58 (2.37) 

Unreinforced polyoxymethylene 

1.14-2.23 (1.42) 

5.00-115 (55.6)/22.0—120 (26.8) 

0.586-12.1 (2.73) 

Acrylonitrile-butadiene-styrene 

0.35-3.5 (1.06) 

24.1-73.1 (38)/20.0—73.1 (43.3) 

0.778-6.10(2.3) 

Polyvinyl chloride 

1.4 

4.00-59.0/17.0-52.0 

0.00159-3.24 

Ceramic (96% alumina) 

3.7-3.97 

2070-2620/260-300 

393 

Metal matrix composite, Al/SiC 

2.78-3.01 

230-317 

125-270 


corrosion resistance and ease of machining and fabri¬ 
cation with relatively low cost. Additionally, POM has 
high stiffness, low friction, and excellent dimensional 
stability. PC and acrylic have high-impact strength. 
They are lightweight and flexible. Even though glass 
is brittle and, hence, subject to damage from impact, 
glass spheres are also appropriate for pressure ves¬ 
sels because of the immense strength-to-weight ratio, 
corrosion resistance, nonmagnetic and electrically non- 
conductive characteristics, and low cost [25.9]. Some 
properties are summarized in Table 25.1. 

The metal corrosion in seawater is the gradual de¬ 
terioration of a material based on the electrochemical 
reaction. There are galvanic corrosion and electrolytic 
corrosion which are known for fundamentally the same 
process and known for accelerating rates of the metal 
corrosion in underwater robot [25.3]. To protect against 
electrolytic corrosion, insulators, such as resin or rubber 
are attached to metal contact surfaces [25.6]. How¬ 
ever, surface treatment is an effective way to avoid 
any direct contact of seawater. Anodizing is a surface 
conversion process that changes the exterior physical 
properties of a metal to form a hard coating on the 
surface as results of a controlled electrochemical pro¬ 
cess. One thing we should remember is that anodizing 
works well only when the entire surface remains in¬ 
tact. In other words, a small chip in the surface can 
become the point of accelerated corrosion if there is 
contact with a less active material [25.10]. A topcoat 
will provide additional protection against corrosion. 
Sacrificial anodes, made of highly active metal like 
zinc and magnesium, are alternative for corrosion pro¬ 
tection. As sacrificial anodes corrode and deteriorate, 
they must be periodically inspected and replaced if 
needed. 


When designing an underwater robot, it is recom¬ 
mended to have small positive buoyancy, except the 
reserve buoyancy for payload. It will help an under¬ 
water robot returning to the surface in the case of any 
electronics and power failure. The amount of positive 
buoyancy would depend on the size of robots, typi¬ 
cally from a few hundred grams to several kilograms. 
There are several factors for consideration in choosing 
buoyancy materials, such as low density, high strength, 
nontoxic harmless, and weather resistance. The most 
important factors are low water absorption and low 
compressibility, which are all dependent on the tar¬ 
get operating depth [25.4]. For an underwater robot, 
expandable polystyrene, polyisocyanurate foam, and 
syntactic foam are the most common buoyancy mate¬ 
rials. The expandable polystyrene (EPS) is a super light 
and cheap material. However, it is only used for very 
shallow water because it is compressed by water pres¬ 
sure. Polyisocyanurate foams are generally low-density, 
insulation-grade foams, which have excellent insulating 
value and good compressive-strength properties. Since 
it is usually made in large blocks via a continuous extru¬ 
sion process, these blocks are reasonably inexpensive 
and effective for shallow water [25.4]. The syntactic 
foam consists of glass microballoons and epoxy resin. 
The amount of trapped air within the resin structure will 
determine the density as well as the durability of the 
foam at deeper depths. It is also noted here that ballast¬ 
ing is a procedure of adding, removing, or relocating 
weights or floatation on an underwater robot to correct 
its buoyancy, pitch, and roll. There are two types of bal¬ 
last systems: fixed (static) ballast and variable (active) 
ballast. In a fixed ballast system, an underwater robot 
is preset to the desired ballast and operates without any 
dynamic change of the ballast system. Fixed payload is 
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usually in the form of several lead blocks, which can be 
exchanged for equipment without adjusting the robots 


25.3 Power Systems 

Underwater robots obtain power from any of three cat¬ 
egories: on-board power, surface power by a tether, and 
the combination of the two. For an onboard power sys¬ 
tem, we can think of many types of power sources, such 
as primary/secondary batteries, internal/external com¬ 
bustion engines, and fuel cells. For surface powered 
underwater robots, the mother ship has to deliver power 
to the robot by a tether. Surface powered underwater 
robots have advantages of high power capacity and al¬ 
most unlimited power but also entail additional cost for 
a winch, a drum, a TMS, and a mother ship. This sec¬ 
tion reviews typical power sources including batteries, 
fuel cells, and surface power systems. 

25.3.1 On-Board Power Sources 

The on-board power source of an underwater robot is 
an air-independent power (AIP) system, and the most 
common AIP system is batteries. Performance spec¬ 
ifications of untethered vehicles like AUVs, such as 
working range, service time, and payloads are limited 
by the battery capacity. 

Battery technology has improved over the last 
decade. One would like to have better battery sources 
in terms of high energy density, high charge and dis¬ 
charge rates, desirable current characteristics, wide 
operating/storage temperature range, reliability, long 
lifecycle, low cost for purchasing and maintenance, 
and, most importantly, safety against fire and explo¬ 
sion. We should also consider details on safety such 
as nongassing, spill-proof electrolyte, status monitoring 
as well as special requirements depending on the shape 
and volume of an AUV and particular working environ¬ 
ment conditions such as depth, under sea ice near the 
North Pole [25.6,11,13]. 

Primary batteries usually have better endurance 
than secondary batteries. The lithium primary battery 
has a very high energy density but its operating cost 
is more expensive. The alkaline battery can be consid¬ 
ered the most common primary battery as it is relatively 
cheap, but it has safety issues because it outgases hydro¬ 
gen and it is sensitive to temperature [25.7]. 

There are a number of different kinds of sec¬ 
ondary batteries including lead-acid, silver zinc, nickel- 
cadmium, nickel-metal hydride, lithium-ion, lithium- 
polymer batteries. Lead-acid batteries are the oldest 
type of battery having a low energy density. How¬ 


buoyancy. Variable ballast is not common in most ROVs 
but the hybrid ROY, Nereus, has one [25.8]. 


ever, because of its relatively large power rate and low 
cost, they are the popular choice for many applications. 
Nickel-cadmium (Ni-Cd) batteries were widely used in 
commercial products because of their robust character¬ 
istics but had problems with the toxicity of the heavy 
metal cadmium. Nickel metal hydride batteries have 
been improved to eliminate drawbacks of Ni-Cd and 
replaced Ni-Cd in the market. During the last couple 
of decades, the silver-zinc battery was the most used 
power source in professional AUVs because of its bet¬ 
ter performance compared to that of old type secondary 
batteries such as lead-acid and Ni-Cd, even though it is 
very expensive. Today, lithium-ion and lithium-polymer 
batteries are popular in our daily life. We can find them 
in many IT devices. Li-ion batteries using lithium ions 
as an electrolyte have the advantages of higher energy 
density, longer cycle life, relatively low self-discharge, 
low maintenance, and light weight. However, the draw¬ 
backs include protection circuits and aging effects. In 
contrast, lithium-polymer batteries use a solid polymer 
composite. This reduces the chance of overcharging and 
electrolyte leakage and is more robust against physical 
damage [25.12], even though they have a lower energy 
density and a lower life cycle compared to Li-ion bat¬ 
teries. Some properties are summarized in Table 25.2. 

In the literature, as one of the alternatives, sea¬ 
water batteries with very long endurance capabilities 
are considered and have been studied to improve the 
performance. However, due to low power capacity, its 
applications are limited [25.14]. 

A fuel cell is an electrochemical device converting 
the chemical energy of fuels with oxygen directly into 
electricity to produce electric power, heat, and water. 
Since there is continuously replenished fuel, a fuel cell 
has a much larger capacity and longer endurance than 
batteries at the cost of system complexity. There are 
various types of fuel cells. The most common types are 
solid oxide (SOFC), direct methanol (DMFC), proton 
exchange membrane (PEMFC, or polymer electrolyte 
(PEFC)), phosphoric acid (PAFC), molten carbonate 
(MCFC), and alkaline (AFC), where each type operates 
at different temperature conditions using different ma¬ 
terials [25.15, 16]. 

A closed-cycle PEFC system is known to be the 
most suitable for underwater and it consists of a fuel 
cell generator, a high pressure oxygen tank, and a metal 
hydride tank. Its operating temperature is around 60 °C 
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Table 25.2 Summary of characteristics of commonly used batteries (after [25.7, 11, 12]) 


Type 

Advantages 

Disadvantages 

Energy/weight 

(Wh/kg) 

Energy/volume 

(Wh/1) 

Rating 

Life cycle 

Alkaline 

(primary) 

Cheap 

Leak gas at high tempera¬ 
ture 

140 

- 

- 

- 

Lithium 

(primary) 

High energy density, 
expensive 


375 

- 

- 

- 

Lead acid 
(secondary) 

Low cost, long life cycle 
(1000) 

Low energy density, leak 
hydrogen 

20-30 

60-80 

-40-50 

700 

Silver-zinc 

(Ag-Zn) 

(secondary) 

High energy density 

Limited life cycle (40—100), 
need careful maintenance, 
high cost 

100-120 

180-200 

-48-71 

100 

Nickel-cadmium 

(Ni-Cd) 

(secondary) 

Long life cycle, flat dis¬ 
charge curve, good low 
temperature performance 

Memory effects, envi¬ 
ronmentally unfriendly, 
exothermic charging process 

40-60 

50-150 

-40-60 

2000 

Nickel-metal 
hydride (Ni-MH) 
(secondary) 

High energy density, 
reduced memory effect 

Self-discharging, limited 
discharge current 

60-120 

140-300 

Ambient 

1000 

Lithium ion 
(secondary) 

High-energy density, long 
life cycle, no memory 
effect 

Aging effect 

100-265 

250-730 

-20-45 

400-1200 

Lithium polymer 
(secondary) 

Flexible form factor, light 
weight, Improved safety 

Lower energy density com¬ 
pared to Li-ion 

130-200 

170-300 

Ambient 

300-3000 


and its reactive product is only pure water. Urashima, 
the first prototype of a fuel cell-driven underwater ve¬ 
hicle developed by JAMSTEC, Japan, uses a solid 
polymer electrolyte fuel cell (PEFC) which allows 60 h 
endurance and 300 km cruising with max. 4kW power, 
and they achieved 60 h of nonstop electric power gen¬ 
eration and finished a sea trial [25.17]. In 2007, JAM¬ 
STEC started the research to develop a fuel cell for the 
second generation long-range cruising AUV (LCAUV) 
to cruise over 3000 km with over 600 h of continuous 
running [25.12]. 

The energy density of a fuel cell generator itself is 
high, but when we consider a whole fuel cell system 
including subsystems, such as hydrogen, oxygen, and 
reactant water tanks, auxiliary components, and control 
electronics, its energy density becomes low. Down¬ 
sizing of these subsystems is needed for underwater 
applications. So, a hybrid fuel/battery system would be 
an alternative to share full loads because it allows both 
components to be of smaller dimensions and operates 
with higher efficiency using direct hydrogen and pure 
oxygen [25.18]. 

There are two types of loads: propulsion load for 
propulsion and hotel load for on-board computers, 
lights, navigation sensors, and payloads. If an AUV 
is operated for professional long-term missions, we 
need very accurate simulations for adjusting trade-offs 
among various constraints. Conventional lead-acid and 
Ni-based batteries would not be the best choice for 
AUVs. Instead, lithium-based batteries are the attractive 
solution that does not entail time-consuming estima¬ 


tion techniques. Nonetheless, lead-acid batteries remain 
a practical and low-cost option for education, experi¬ 
mentation, and testing. 

There are two options for battery installation in¬ 
side an AUV. One is that they are housed in a pressure 
vessel, the other one is using pressure-tolerant and wa¬ 
terproof batteries without a pressure vessel. The latter 
eliminates burdens related to leakage in assembling 
the system, thus increases system reliability. Lithium- 
polymer batteries using the solid electrolyte were made 
without air in the packaging. Hence, they would be 
a candidate in designing pressure-tolerant on-board 
power source for AUVs [25.19]. 

A battery management system (BMS) is a very 
important subsystem for performance and safety. It 
monitors the internal status of voltages, current, tem¬ 
peratures and state of charge. Based on the above 
information, a BMS controls circuits for balancing 
and protects batteries from over-current, over/under¬ 
voltage, over/under-temperature, and any predescribed 
conditions. As a part of health monitoring, an on-board 
computer of an AUV should collect status from the 
BMS to reflect a power budget and health to given mis¬ 
sions. 

Just like adding volume of auxiliary systems and 
fuel storage units to calculate an energy density of 
a fuel cell, we need a measure for fair comparison 
of the energy density among different power sources. 
Since a power source is installed inside of an AUV, 
which should be neutrally buoyant to save the en¬ 
ergy in operation, energy density per neutrally buoyant 
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Table 25.3 Summary of pros and cons of DC and AC (alternating current) (data from [25.4, 20]) 


Form 

Pro 

Con 

Remarks 

DC 

Two conductors with less shielding in 
tether, low voltage drop 

Current protection, difficulties in voltage 
step-up and down 

For observation class ROV 

AC 

Longer transmission distances, flexibility in 
voltage up and down 

Inductance noise, reactive losses, need 
subsystem for freq. converting 

For deep-sea ROV 


kilogram should be a reasonable unit. This means 
that weight and volume for buoyancy materials and 
pressure vessels must be added to the power source 
itself, and this also means that the characteristics of 
buoyancy materials and pressure vessels (aluminum, 
titanium, composite material) under considering work¬ 
ing (or maximum) depth should be designed before or 
at the same time [25.21]. Between batteries and fuel 
cells, a semi-fuel cell can be another alternative. It 
is a generator, but its usability is like a battery be¬ 
cause one reactant is stored externally and brought 
to the reaction cell [25.11]. Aluminum/hydrogen per¬ 
oxide energy semi-fuel cells can practically gener¬ 
ate an energy density of about 400Wh/kg (theo¬ 
retically 3418Wh/kg) which is three times that of 
a lithium-ion system [25.12]. HUGIN 3000 powered 
by a semi-fuel cell developed by FFI (Norwegian De¬ 
fense Research Establishment) [25.7] operates at 4 km 
down to a sea depth of 3000 m for up to 60 h. This 
pressure-tolerant system is very attractive in deep-water 
systems. But it requires trained personnel for opera¬ 
tion and maintenance as well as special facilities and 
tools [25.14], 

25.3.2 Surface Power Systems 

In the case of ROVs, the whole power is delivered 
from a mother ship to the ROV through an umbili¬ 
cal/tether cable, offering very high power and theo¬ 
retically unlimited endurance. These electrical power 
transmission techniques may look simple, but actually 
involve a highly complicated procedure for optimiza¬ 
tion. A power system determines work capability of 
ROV as well as operating cost, subsystem size, and 
safety considerations. Typically, it consists of a surface 
system, a tether, and a subsea system. A surface system 


includes a shipboard power source (or power outlet), 
frequency converter, step-up transformers, and a ground 
fault monitoring device. A frequency converter is used 
only for high-frequency alternative current, but it is ad¬ 
vantageous in handling the power of a subsea system. 
In a subsea system, there are step-down transform¬ 
ers, rectifiers, power busses, power switching devices 
for distribution, and a ground fault monitoring device. 
Also, system configuration and setup reflects the power 
requirement of the devices installed on a ROV including 
thrusters, lights, sensors and any circuits. Once again, 
these design parameters of a power system are not in¬ 
dependent of each other [25.4, 20,22, 23]. 

A detail configuration depends on the form of cur¬ 
rent and needed capacity. First of all, we have to make 
a decision on the form of current. There are three types 
of current for ROV systems: 

1. 50/60 Hz 

2. High frequency alternating current (HFAC) at 

400 Hz 

3. Direct current (DC). 

The decision should be based on the design ob¬ 
jectives of maximizing power transfer, efficiency and 
lifetime, as well as minimizing the overall system 
cost using steady-state analysis and transient analy¬ 
sis [25.20]. Table 25.3 shows pros and cons of the form 
of current. 

Besides maximizing the power transfer to an ROV, 
we have to minimize the diameter of a cable, system 
cost, support systems on deck, and the weight of the 
ROV without losing a high level of electrical power 
quality. Therefore, most observation class ROVs use the 
DC power system mainly due to portability and cost, 
and work-class ROVs have a tendency to use a three 
phase AC power system. 


25.4 Underwater Actuators and Sensors 


It is inevitable that the overall performance of an un¬ 
derwater robot depends on the quality of its actuators 
and sensors. While one can think of different types of 
actuators for underwater robots, such as thrusters, wa¬ 
ter jets, and biomimetic actuators ( I -g&MllhhUiTl ), most 
underwater robots use thrusters that use propellers in¬ 


volving hydrodynamics of water flow when generating 
force to move through water. It is noted that the map¬ 
ping between thruster command and generated force is 
not linear, especially in the transient state of motion. 
The sensors for an underwater robot can be divided 
into three groups: navigation sensors, system sensors, 
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and mission sensors. They can also be categorized fur¬ 
ther by acoustic- or nonacoustic-based sensing. Sensing 
in underwater environments is very challenging, es¬ 
pecially position sensing. There is no electromagnetic 
wave propagation under water except for very short 
ranges and, therefore, the GPS (global positioning sys¬ 
tem) is not available. Due to the limited performance of 
a single sensor, the sensor fusion approach with multi¬ 
ple sensors is used. For example, the inertial navigation 
system (INS) includes an IMU (inertial measurement 
unit) and a DVL (Doppler velocity logger) with highly 
complicated algorithms. The performance of optical 
sensing depends on the turbidity level, while acous¬ 
tic sensing encounters significant challenges due to the 
dynamic properties and extreme nature of the ocean, 
such as multipath problems, variation of signal atten¬ 
uation, variation of acoustic channel, nonlinear effects, 
and many more [25.24,25). This section describes un¬ 
derwater actuators and sensors, and their design-related 
issues. 

25.4.1 Actuators 

There are two major types of thrusters - electrical 
thrusters and electro-hydraulic thrusters - along with 
other types such as bio-inspired smart actuators and 
jet propulsion. The efficiency of an electro-hydraulic 
thruster is about 15%, while the efficiency of a fully 
electrical thruster is about 43% [25.13], where the def¬ 
inition of efficiency is the ratio of shaft mechanical 
power output versus electrical power input. In spite of 
the low energy efficiency, electro-hydraulic thrusters 
are preferred for large size work-class ROVs because 
of their extensive thrust and practicality in the rugged 
conditions of the offshore work environment. Various 
factors contributing to the power conversion efficiency 
of the hydraulic propulsion system for ROVs are de¬ 
scribed in [25.26]. 

Many small ROVs and AUVs have fully electrical 
thrusters because of their compact size and high effi¬ 
ciency. Torpedo-type AUVs for cruising commonly use 
the screw-type propellers, which generate much more 
power, though only in the forward direction. Small 
ROVs and AUVs requiring slow but fine motion typ¬ 
ically use the propellers generating the same power in 
both forward/reverse directions. When two thrusters are 
operating on the same plane and the same direction of 
motion, an underwater robot must have counter-rotating 
thrusters to eliminate the counter torque effect of the 
thrusters rolling the robot opposite to the direction of 
propeller rotation. 

There are many propeller design parameters includ¬ 
ing propeller diameter, the number of blades, blade 
pitch, blade thickness, hub size, rake, etc. Propeller 


diameter and blade pitch are critical design parame¬ 
ters that impact a robot’s torque and power, and are 
related to the size and cost of an underwater robot. 
Therefore, we have to find an optimum combination 
of propeller diameter and blade pitch, along with care¬ 
ful matching to a motor to have an efficient electrical 
propulsion system [25.27], A typical underwater elec¬ 
trical thruster includes an electric motor, a motor am¬ 
plifier and controller, a thruster housing typically made 
of aluminum or titanium, and oil-filled, drive shafts 
designed for watertight using magnetic coupling or dy¬ 
namic seal, propellers, and ducts. The Kort nozzle has 
a relatively simple geometry and it performs efficiently 
at low speed. The Rice nozzle has a lower drag co¬ 
efficient and a better geometry to promote increased 
thrust [25.24]. 

25.4.2 Nonacoustic Sensors 

Various nonacoustic sensors can be considered for mea¬ 
suring an underwater robot’s position and pose, and 
detecting its linear and rotational motion. Magnetic sen¬ 
sors are the most common to get the north direction, 
but are susceptible to local magnetic disturbances gen¬ 
erated by the active elements of a robot system or any 
magnetized materials in the environment. 

Inertial sensors are sensors based on inertia, where 
linear acceleration is measured by accelerometers and 
the rate of rotation is measured by gyroscopes with¬ 
out an external reference. There are several types of 
accelerometers, such as pendulous accelerometers, pen¬ 
dulous integrating gyroscope accelerometers, force re¬ 
balance accelerometers, piezoelectric accelerometers, 
differential capacitor accelerometers, and vibrating 
beam accelerometers. They have each advantages and 
disadvantages. Some of them can be built using either 
conventional mechanical type or microelectromechani¬ 
cal system (MEMS) type. MEMS types like capacitive- 
based MEMS accelerometers and MEMS vibrating 
beam accelerometers, show generally higher sensitivity 
and better resolution [25.28, 29]. A gyroscope (gyro) is 
used to measure angular velocity of an object. There are 
three main types of gyro: spinning mass, vibratory, and 
optical. Spinning-mass type gyros operate on the prin¬ 
ciple of conservation of angular momentum, and this 
mechanical type gyro shows wide performance range 
(0.0001 to > 100°/h). However, it is very expensive. 
Vibratory (vibrating structure) type gyros use Coriolis 
forces that are small and of low cost because they use 
MEMS technology, but they have a limited performance 
range. Optical type gyros operate under the principle 
of the Sagnac effect, and there are two main types of 
optical gyros, RLG (ring-laser gyro) and interferomet¬ 
ric fiber-optic gyro (IFOG). A RLG has high sensitivity 
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and stability, a wide range of measurement (from 0.001 
to 1000°/h), excellent scale-factor stability and linear¬ 
ity, insensitivity to accelerations and fast turn-on (< 1 s). 
The RLG technology is a mature technology, and is well 
established in the medium- and high-performance mar¬ 
kets. However, RLGs are still complex and expensive. 
A fiber-optic gyro (FOG) can provide several important 
advantages of a pure solid-state device, such as reli¬ 
able and low maintenance due to no moving part, no 
sealed cavity and no acoustic noise, and high perfor¬ 
mance. The FOG technologies are expected to replace 
many of the current RLG-based systems. However, one 
particular area where the RLG is expected to retain its 
superiority over the FOG is in applications requiring ex¬ 
tremely high-scale-factor stability. 

An attitude and heading reference system (AHRS) 
is a sensor providing three-dimensional (3-D) orienta¬ 
tion information. Typically, it consists of (MEMS) gy¬ 
roscopes, accelerometers, and magnetometers on all 
three axes, and an onboard microprocessor running 
a nonlinear estimation algorithm, such as an extended 
Kalman filter to compute the orientation. An integra¬ 
tion drift is compensated by reference vectors, such as 
gravity and the earth magnetic field. 

The fusion of several navigation sensors can min¬ 
imize navigation errors. This sensor combination of 
accelerometers, gyros, navigation computer, and some 
electronics, called inertia navigation system (INS), pro¬ 
vides estimations of velocity, position, and orientation 
by integrations of the linear accelerations and angu¬ 
lar rates with an appropriate transformation between 
the body frame and the earth frame if given an ini¬ 
tial position and orientation [25.12,23]. Therefore, an 
INS is one of the most important devices for an un¬ 
derwater robot. Typically, the performance of an INS is 
usually rated in terms of its position error growth rate. 
USAF SNU84-1 categorizes INS units as shown in Ta¬ 
ble 25.4. 

To increase the accuracy of a navigation system, an 
INS has been combined with DVL (Doppler velocity 
log). As an example, an integrated Doppler-inertial sys¬ 
tem, Marpos, calculates the relative position from a start 
point by dead reckoning, which shows a positioning ac¬ 
curacy of around 0.03% of the total distance traveled, 
the equivalent of 1.7 m per hour at a vehicle speed of 
three knots by AUV Maridan M600 [25.30]. 


Table 25.4 Performance classification of INS (USAF 
SNU84-1) 


INS classification 

Position error 
growth rate 

Heading errors 

Low 

> 2 nm/h 

>0.2° 

Medium 

0.5—2 nm/h 

0.05 to 0.2° 

Precision 

< 0.5 nm/h 

< 0.05° 


For the vertical motion of an underwater robot, 
its depth is easily measured by pressure sensors with¬ 
out any cumulative error. Actually, a pressure sensor 
measures the external pressure experienced by an un¬ 
derwater robot, where the two most common pressure 
sensor technologies are (i) strain gauges and (ii) quartz 
crystals. Strain gauge pressure sensors can typically 
attain overall accuracies of up to about 0.1% of full 
scale and resolutions of up to about 0.01% of full 
scale. Quartz crystal pressure sensors can typically at¬ 
tain overall accuracies of about 0.01% of full scale and 
overall resolution of up to about 0.0001% of full scale, 
that is, a resolution of one part per million [25.31]. 
Common units for pressure are atmosphere (atm), bar, 
kilopascals (kPa), pound of force per square inch (psi), 
Torr (= mmHg), and one atmosphere (1 atm) of pres¬ 
sure is 101.325 kPa and one bar (1 bar) of pressure is 
lOOkPa. In sea water, since pressure increases by 1 atm 
for every 10.0 m of depth, depth is an approximate 
reading of a pressure sensor multiplied by lOm/atm. 
In fresh water, this simple formula underestimates the 
actual pressure by 2-3%, so you can get a more accu¬ 
rate depth by using 10.3 instead of lOm/atm [25.3]. 
Measured pressure by a sensor is relative pressure. For 
absolute pressure, we have to add 1 atm to the sensor 
measurement. 

Another nonacoustic sensor is an underwater com¬ 
puter vision system that can be considered as part of 
navigation sensors for underwater robots in addition 
to simply recording images of the underwater environ¬ 
ment. When the underwater environment has high tur¬ 
bidity and insufficient and/or nonuniform light, it would 
not be easy to obtain enough visual information for the 
navigation through identifying objects, detecting obsta¬ 
cles, measuring distance between objects, and so on. 
However, when approaching its target very closely, for 
example, at the final stage of approaching for docking, 
the robot could gather useful information using a vision 
system. 

On the other hand, underwater images are major 
products of underwater missions, especially for the 
inspection of submerged structures or observation of 
marine organisms, seafloor geology, etc. In this case, 
associating time stamp and localization data with the 
images increases the overall quality of the output. Be¬ 
cause of the limited light range, it is very difficult to 
cover a wide area with one single shot. Thus, an im¬ 
age mosaicking is a popular strategy to overcome this 
in which several images are merged together in order 
to create a larger image, just as is done on Google 
Earth. Real-time image mosaicking is used for naviga¬ 
tion stitches images with simple image manipulations, 
and high-quality image mosaicking is done by post¬ 
processing using various optimization techniques. 
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More recently, advanced, high-quality camera and 
video management systems have been developed such 
as HD (high definition) cameras, HD-SDI (serial dig¬ 
ital interface) transmission through a fiber-optic cable, 
high-quality digital video management systems, holo¬ 
graphic cameras, laser scanning systems, 3-D camera 
systems with 3-D monitors, a hybrid optical-acoustic 
imaging systems incorporating stereographic optical 
cameras, and high-resolution multibeam sonars [25.12, 
32, 33]. Such developments would greatly help increas¬ 
ing the effectiveness of the underwater robot operation. 

25.4.3 Acoustics Sensors 

There are many ways to calculate the speed of an un¬ 
derwater robot. We can simply estimate the speed by 
the integral of linear acceleration from an accelerom¬ 
eter even though there are large accumulative errors. 
We can also measure the speed directly using a pitot 
tube velocity log, a rotor type velocity log, an electro¬ 
magnetic velocity log, and an acoustic Doppler current 
profilers (ADCPs) [25.6], which are all water speed 
sensors, and are measuring speed with respect to the 
water, not the ground speed of a robot. ADCP provides 
measurements of the water’s current direction and mag¬ 
nitude by measuring the velocity of particles in water 
using principles of Doppler shift. However, Doppler 
velocity logs (DVLs) can be used to measure the ve¬ 
locity of robots moving in the water with respect to 
the ocean floor. Since ADCP and DVL use the same 
principles, typically DVL is operated in ADCP mode 
for a hydrographic instrument and bottom-lock mode 
for a navigation instrument. Since a DVL measures the 
Doppler shift of sonar signals reflected by the ground 
to obtain the velocity, there is a maximum range from 
seabed to the sensor, which is inversely proportional to 
the frequency of a signal, typically from 300 kHz to 
2 MHz. Table 25.5 shows the performances of DVLs 
currently available in the market. Recently phased ar¬ 
ray technology in DVL has been announced and has 


Table 25.5 Summary of DVL specifications available in 
market 


Specification 

Model A 

Model B 

Model C 

Frequency 

300 kHz 

600 kHz 

1200 kHz 

Max. range 

200-300 m 

90-110 m 

20-30 m 

Velocity 

range 

± lOm/s 



Velocity reso¬ 
lution 

0.1 cm/s 



Accuracy 

0 . 4 % ± 

0.2% ± 

0.2% ± 


2mm/s 

1 mm/s 

1 mm/s 

Average 

power 

3—10W 

2-6 w 

2-3 w 


brought significant advantages over piston array type 
DVL in terms of size and weight, which have extended 
the areas of applications for DVLs [25.34]. 

There are several types of sonar systems such as 
single-beam directional sonar, scanning sonar, side scan 
sonar, multibeam sonar, interferometric sonar and syn¬ 
thetic aperture sonar. Single-beam sonars send a pulse 
of sound at a particular frequency that is reflected from 
the seabed, and its echo is received by a transducer. 
It can be implemented in the form of a single chan¬ 
nel, for example, an altimeter measuring the distance 
between a robot and seabed, or multiple channels, for 
example, obstacle avoidance sonar measuring distances 
between a robot and any objects around it. A scan¬ 
ning sonar, that is, forward looking sonar, has a small 
transducer rotating on its axis and emits pulses at the 
particular direction, and then obtains a set of bins, 
where a bin represents the echo intensity returning from 
a specific place along the transducer axis. An image is 
synthesized by placing the set of bins in a Cartesian 
space with a set of colors assigning their intensities. On 
the other hand, a side scan sonar has a fixed transducer 
emitting fan-shaped acoustic pulses in the cross-track 
direction, and then the measured acoustic reflections are 
compiled to create and accurate image of the sea bot¬ 
tom. It is one of the most effective tools for quickly 
searching for any objects such as wreckage debris or 
mines on a large area of the seabed. However, it does not 
provide reliable information on their position [25.35]. 

An imaging sonar uses a fan beam typically with 
a 30° 3 dB beam width while retaining about 1.7° an¬ 
gular resolution, whereas a profiling sonar emits a spot 
or pencil beam with about a 1.7° 3 dB beam width from 
both horizontal and angular rotation, which means that 
it provides higher resolution measurements. A multi¬ 
beam sonar has an array of transducer emitting acoustic 
pulses simultaneously to cover a large area. This beam 
pattern is wide in the across-track field and narrow 
in the along-track field, which is appropriate for col¬ 
lecting highly accurate seafloor information at the cost 
of the high purchasing expense and high power con¬ 
sumption. An interferometric sonar is one possible tool 
providing wide-swath coverage and high-resolution 
bathymetry in shallow water using the technique of su¬ 
perimposing two (or more) waves, which significantly 
improves the efficiency and safety of hydrographic sur¬ 
vey operations. A synthetic aperture sonar (SAS) is 
a high-resolution acoustic imaging technique that sums 
multiple returns from multiple pings to produce out¬ 
put imagery with constant along track resolution which 
is independent of both range and frequency. This is 
a kind of artificial enlargement of a sonar array by ac¬ 
curate navigation data. Dual frequency technology has 
been announced for an imaging sonar and side scan 
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sonar, which offers more flexibility. A typical dual fre¬ 
quency imaging sonar uses frequency between 0.9 and 
2.5 MHz to obtain medium and ultra-high-quality res¬ 
olution from mid- to close-range. A side scan sonar’s 
typical range is from 100 to 500 kHz. Based on a high- 
quality imaging sonar, researchers have been trying to 
identify underwater objects. Figure 25.2 shows a re¬ 
sult of real-time tracking for artificial landmarks using 
DIDSON [25.36] (l<sa>MZEB!EIl), where the multiple 
landmarks are being tracked after being detected and 
recognized by a probabilistic based algorithm devel¬ 
oped by the Korea Research Institute of Ships and 
Ocean Engineering (KRISO) [25.37], 

Acoustic positioning systems are the popular ap¬ 
proach to locate the underwater robot by solving a tri¬ 
angulation problem based on distances between an 
underwater robot and the preinstalled transponders. The 
distances are calculated by the time of flight of the 
acoustic signal between them, that is, half the round- 
trip time multiplied by the speed of sound in water. It is 
theoretically simple with no cumulative error. However, 
to obtain accurate position information, we have to have 
an exact understanding of the system, consideration of 
the inherent physical error sources, and lots of practi¬ 
cal experience. The accuracy inherently depends on the 
configuration of the baseline, and the speed of sound 
underwater is affected by water temperature, pressure, 
and salinity. Additionally, in shallow water or in the 
highly reverberant underwater environment, multiple 
reflected copies of any transmitted signal reach the re¬ 
ceiver at delayed intervals. It is known as the multipath 
problem. Therefore, we have to know the characteristics 
of the underwater acoustic environment at the work site. 
Before explaining the configuration of types of acoustic 



Fig. 25.2 Result of real-time tracking for artificial land¬ 
marks using DIDSON 


positioning system, we need to review functions of its 
basic component. When an interrogate sends a signal, 
a transponder installed on the seabed or an underwater 
robot receives the interrogation signal, and then sends 
a reply signal to the interrogator. A responder is a trans¬ 
mitter installed on the seabed or an underwater robot 
just like a transponder, but it can be triggered by a hard¬ 
wired external control signal to transmit a signal. If 
a transmitter sends out a pulse on a particular frequency, 
it is called a beacon or pinger [25.4], 

There are three main types of acoustic positioning 
systems: long baseline (LBL), short baseline (SBL), 
and ultrashort baseline (USBL). An LBL system deter¬ 
mines the position by measuring the slant ranges from 
three or more transponders mounted widely spaced on 
the seabed and typically at the corners of the opera¬ 
tions site. This results in very high positioning accuracy 
and robustness regardless of water depth over wide op¬ 
erating area. However, this system requires multiple 
transponders which inevitably need a long deploy¬ 
ment/recovery time and a calibration time required at 
each location. An update rate is relatively long because 
of two-way ranging. In an SBL system, there are three 
or more transducers connected by wires to a central 
control unit; these transducers construct a baseline in¬ 
stalled on from a large barge to a small boat. It is just 
like an upside down version of an LBL system without 
requiring any seafloor premounted transponders. How¬ 
ever, accuracy depends on the transducer spacing and 
mounting method. When a wider spacing is available, 
the performance can be similar to LBL systems. Other¬ 
wise, the accuracy goes down. A USBL system consists 
of a small integrated transducer array and a transpon¬ 
der/responder installed on what we want to trace, where 
the position of the transponder/responder is calculated 
by the combination of a target distance by the time 
of flight of a received signal and a target direction by 
the phase shift of the received signal on each of the 
acoustic elements in the transceiver array, called phase- 
differencing. Since a small error of the target direction 
calculated by a USBL will become a larger position er¬ 
ror over a great distance, it should be mounted on the 
bottom end of a strong and rigid transducer pole, which 
is installed either on the side or in some cases on the 
bottom of a surface vessel. Additional sensors, includ¬ 
ing high-quality IMU and GPS, have to compensate for 
errors due to the any changing position and orientation 
of the integrated transducer array. 

USBL systems offer many advantages, such as low 
complexity, ease of use and not requiring sea floor 
transponders. But they have also some disadvantages. 
An installation of a rigid pole for mounting of a trans¬ 
ducer array is needed and a calibration process is 
required to determine offsets for the precise alignment 
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of a USBL system. Most importantly, their positioning 
accuracy and robustness are not as good as the LBL sys¬ 
tems. In the case of range, all types are quite accurate 
assuming that the line of sight is maintained. However, 
the bearing accuracy of LBL systems is less than 1°, 
whereas SBL and USBL systems typically have 1—3° 
bearing accuracy [25.4]. 

The GPS intelligent buoys (GIB) system is available 
as a promising acoustic positioning system [25.38]. The 
configuration of GIB is an inverted LBL, where several 
floating buoys equipped with submerged hydrophones 
and GPS receivers measure the time of arrivals (TOAs) 
of the acoustic signals sent by the emitter of an underwa¬ 
ter target and from the time measurements tracked po¬ 
sition is calculated in real time. This system allows fast 
and easy installation without a calibration process and 
a dedicated mother ship and provides real-time multiob¬ 
jects tracking with a fast update rate because of one-way 
acoustic signals from the emitter of an underwater tar¬ 
get to the floating buoys. The number of floating buoys 
is determined by the size of the area and the accuracy re¬ 
quirement of given operations. 

25.4.4 Other Sensor Issues 

Many sensors and communication devices of an un¬ 
derwater robot, such as altimeter, DVL, imaging sonar, 
ranging sonar, acoustic modem, and acoustic position¬ 
ing system are sonar-based devices. The integration and 


concurrent operation of multiple sonar sensors could 
cause cross-sensor acoustic interference problems. The 
contamination caused by such sensor crosstalk severely 
degrades the performance and sometimes, more se¬ 
riously, mangles the whole sensor system. Basically, 
we have to mediate a sensor configuration including 
working frequencies of all sensors and their mounting 
positions to avoid sensor crosstalk. For mission sen¬ 
sors, we can selectively turn on the sensors as needed, 
considering other sensors installed on the mother ship. 
Additionally, potential problems of bio fouling and 
corrosion of underwater sensors should be taken into 
consideration and the quality of measurements should 
be regularly maintained. 

Health-monitoring sensors are also very important. 
Even though they are small and relatively cheap com¬ 
pared to navigation sensors, it could save a robot. 
Health-monitoring sensors cover multipoint tempera¬ 
ture and humidity values inside pressure vessels, status 
of the power system, for example, status of a battery 
using a battery management system (BMS) or status 
of tether cable using a ground fault-monitoring system, 
and status of the control system. At the same time, an 
emergency plan should be prepared for cases in which 
the system is headed toward complete failure. Examples 
include a situational warning for high temperature or 
clear thruster commands when communication is lost. 
An emergency beacon and a flash have to be operated 
independently from any system of the robot. 


25.5 Computers, Communications, and Architecture 


Design of computer systems, software architecture, and 
communication systems is a major task in designing 
an underwater robot. Designing the overall hardware 
and software structures and selecting appropriate com¬ 
ponents are very challenging and crucial. Nowadays, 
tethered robots or ROVs use a fiber-optic cable for real¬ 
time data transfer between a simple on-board computer 
system and a high-performance computing system on 
a mother ship. Untethered robots or AUVs require high 
computing power resources on-board with an underwa¬ 
ter wireless communication system such as an acoustic 
modem. 

It is worthwhile to note the Woods Hole Oceano¬ 
graphic Institute (WHOI)’s hybrid ROV, also known 
as Nereus [25.8]. It operates in two different modes. 
For broad area survey, the vehicle can operate unteth¬ 
ered as an AUV capable of exploring and mapping 
the sea floor with sonars and cameras. For close-up 
imaging and sampling, Nereus can be converted at sea 
to operate as an ROV, using a lightweight, microthin, 
fiber-optic tether. Nereus uses its manipulator by tele¬ 


operation via the fiber-optic tether while the vehicle can 
be considered an AUV. In May 2009, Nereus explored 
the Mariana Trench, being the first vehicle to explore 
the Mariana Trench since 1998. 

25.5.1 On-Board Computers 

ROVs rely on computing systems on a mother ship 
and typical candidates are industrial PCs installed on 
a rack or on a shelf inside a control van as a sur¬ 
face control system. The overall system consisting of 
several PCs with a high-speed network handles data 
from the ROV and commands from its operator in 
order to control the ROV. Each computer is desig¬ 
nated for a certain mission such as control, naviga¬ 
tion, etc. However, the recent trend is to simplify the 
interface by an integration of multiple PCs and by 
specific software with expectation of cost saving ben¬ 
efits [25.39]. 

Unlike ROVs, AUVs must have compact, robust, re¬ 
liable, and high-performance on-board computer sys- 
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Fig. 25.3 Hardware configuration of Zynq-7020-based system, mController 


terns. One popular choice is a VMEbus (Versa Module 
Europa) system [25.40]. Many AUVs before the year 
2000 such as ODIN, Oceans Voyager II, and Aqua Ex¬ 
plorer 1000 used a VME system based on Motorola CPU 
with a real-time operating system, such as VxWorks and 
OS9 [25.2]. Some AUVs like Kambara and Oberon use 
Compact peripheral component interconnect (PCI) sys¬ 
tems [25.32,41]. The controller area network (CAN) 
bus was originally designed as a vehicle bus standard to 
allow many subsystems such as actuators and sensors to 
communicate with each other using only two wires by 
a maximum 1 MHz data transmission rate [25.12], The 
CAN bus could bring great advantages in designing the 
layout of wire harnesses for internal sensors of an AUV, 
but unfortunately, it has not been popular because there 
are few internal sensors that support it. 

PC-104+ is an embedded computer standard which 
has no backplane but the boards are stacked on top of 
each other via stackable ISA, PCI, and PCIe bus con¬ 
nectors [25.33]. PC-104+ based systems are widely 
used in various types of embedded applications be¬ 
cause of their advantages such as small size, stackable, 
rugged, compact, interoperable, PC compatible high- 


quality, low cost, and low power consumption [25.33, 
40]. Basically, it consists of a CPU board with an op¬ 
tional power supply module. Other required functions 
can be added by stacking multiple peripheral modules 
such as frame-grabber, analog digital conveter/digital 
analog converter (ADC/DAC) converter, digital IO, re¬ 
lay, serial communication, CAN network, and GPS 
receiver from different manufacturers. Each module of¬ 
fers high-performance specifications and environmental 
specifications, such as extended operating temperature 
from —40 to +85 °C. As an example, a decade ago, 
ODIN, originally developed with a VME-based on¬ 
board computer system, was refurbished to a PC-104+ 
based AUV, ODIN III [25.42], Today, many recently 
developed AUVs employ PC-104+ based systems. It 
is also noted that as a possibility of a heterogeneous 
bus system, there is a PC-104+ bus-based system (CPU 
module) along with standard VME-based I/O modules 
using a protocol interface card [25.40]. 

PC- 104+ is a good choice for a single CPU sys¬ 
tem. However, when we need more computing power 
for implementing complicated algorithms, additional 
PC-104s must be installed because multiple CPUs 
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Fig. 25.4 Typical diagram of fiber-optic communication system for ROV 


are not supported in a PC- 104+ stack. For example, 
yShark [25.43] has three PC104+S with a high-speed 
local area network (LAN) for robot control, image pro¬ 
cessing, and acoustic signal processing. This causes 
many difficulties, such as a complicated system struc¬ 
ture, wiring, space, and cooling in a pressure vessel. 
Alternative design of a distributed on-board system for 
an AUV is a combination of a PC- 104+ system and 
an ARM-based system. Besides consumer mobile de¬ 
vices, many mobile robots have used ARM processors 
32-bit RISC (reduced instruction set computer) micro¬ 
processors because of its high-performance, low power 
consumption, and low cost. As an example. Fig. 25.3 
shows a hardware configuration of mController (an¬ 
nounced by Redone technology, Korea) which is based 
on the Xilinx Zynq-7020 platform [25.44]. Its size is 
only 125 mm x 125 mm, and it consists of Dual core 
Cortex-A9 for typical motion control and vision pro¬ 
cessing, and an FPGA part for data parsing of various 
sensors. In this hardware hierarchy, PC- 104 could be 
designated to operate high-level tasks. 

An on-board-independent microcomputer-based 
emergency response system is very important to carry 
out critical functions, such as power disconnection of 
thrusters and weight drop when the on-board main sys¬ 
tem is dead. The emergency response system consists 
of a microprocessor that monitors the status of the on¬ 
board main system, an actuator, and a backup battery. 
Under normal conditions, the on-board main system 
would regularly check the status of the emergency 
response system. 

25.5.2 Communications 

For ROVs, an umbilical cable is used for communica¬ 
tion between an ROV and its mother ship. An umbilical 
cable commonly has a coaxial core or a fiber-optic core. 
A fiber-optic cable is much better than a coaxial cable 
in terms of performance and physical dimensions. Light 
and thin fiber-optic cables offer high-speed, secured, 
long-distance data transmission without electromag¬ 
netic interference, and less drag force on an ROV, even 
though it is an expensive solution mainly because of op¬ 


tic fiber distribution devices, fiber-optic tools, and the 
special knowledge required for a handling fiber core. 
An umbilical cable of a typical ROV has been designed 
to have 2—3 fiber-optic cores including a spare core. 
A fiber-optic modem connected to each end of a fiber¬ 
optic cable sends and receives data at the same time, 
working together with several sub-boards that convert 
data in various formats, for examples, NTSC Video, 
RS232, and LAN as shown in Fig. 25.4. 

Since the demand of high-quality video, such as 
multiple HD video, 3-D HD video, and UHD (ultrahigh 
definition) video is rapidly increasing, 100 Gbps opti¬ 
cal communication systems are released to handle mass 
data using a technology known as dense wave division 
multiplex (DWDM) [25.12]. Cable designers are also 
making an effort in developing new techniques to add 
more fiber-optic cores inside an umbilical cable, where 
each core could be designated to high-quality video de¬ 
vices. The Nereus team used a light fiber tether system 
to send a lightweight battery-powered vehicle to the full 
depth of the ocean of about 11 km in the ROV mode. 
The selected Sanmina/SCI buffered fiber is 0.25 mm 
in diameter, weighs only 0.173 kg for 11km cable in 
water, has a working strength of 8 N, and a breaking 
strength of 108 N [25.5], 

For untethered vehicles like AUVs, wireless com¬ 
munication is considered. Typical radio communication 
would be fine when an AUV is on the surface. In an un¬ 
derwater environment, acoustic communication is the 
practical solution at the moment even though a com¬ 
mercial acoustic modem has a limited range and a very 
limited data rate. Table 25.6 shows the specifications 
of commercial acoustic modems. Acoustic communi¬ 
cation is relatively slow compared to radio communi¬ 
cation, and it is fundamentally limited by the speed 
of sound in water, roughly 1500 m/s. Because acoustic 
attenuation increases with sound frequency, lower fre¬ 
quencies are desirable for long-range communications. 
But, to generate signals at very low frequencies, larger 
size transducers are needed. This results in lower data 
rates that are undesirable [25.2, 13]. The speed and di¬ 
rection of sound propagation vary depending on various 
factors such as surface waves, currents, sea state, tides. 
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Table 25.6 Specifications of commercial acoustic modems 


Specification 

A modem 

B modem 

C modem 

D modem 

Size (length x dia) 

585x 183 

200 x 165 

235 x 126(87) 

580x150 

Material 

Aluminum 

Aluminum 


Weight (air/water) 

11/6 

5/1.5 

4.2/2.3 

21/(n/a) 

Voltage 

18-50 V DC 

10-20 V DC 

12-24 V DC 

18—28V dc 

Power (transmit/receive/sleep) 

< 50 W/l/0.03 

20/0.6/0.005 

2/0.75/0.008 

40/0.9/0.009 

Working depth 

3000 

1000 

200 

7000 

Working temperature (°C) 

-5-40 

-5-40 

-5-45 

-5-45 

Beam width 

±25 

Hemispherical 

210 (Omni)/120 (wide)/70 

210 (Omni)/70 (Directional) 




(narrow) 


Working range 

7000 m 

Up to 20 km 

350 mm 

7000 m (Omni)/10 000 m(D) 

Frequency 

14-22 kHz 

7.5—12kHz 

26.77-44.62 kHz 

7.5-12.5 kHz 

Modulation 

QPSK 

MFSK and DPSK 

Hybrid 

Hybrid 

Raw data rate 

1.5—15 kbps 

2 kbps 

9.6 kbps 

2.5 kbps 

User data rate 

0.6—10 kbps 

0.3 kbps 

7 kbps 

2.0 kbps 

Bit error rate 

10“ 9 

10“ 6 

10“ 9 

10 -7 

Environment 

Near vertical, horizontal 

Near vertical, horizontal 

DPSK: differential phase shift keying 


temperature, conductivity, depth, and sea-bottom type. 
Hence, acoustic communication is inherently delayed, 
due to propagation delay. Several techniques in signal 
processing, data packaging, and coding schemes have 
been developed to overcome these difficulties due to 
signal absorption, geometric spreading losses, bound¬ 
ary effects, and multipath through. 

Acoustic modems consist of three main parts: an 
underwater transducer, an analog transceiver including 
a matching preamp and an amplifier, and a hardware 
platform for control and signal processing, such as mi¬ 
croprocessor, digital signal processor (DSP), or field 
programmable gate array (FPGA) [25.45]. When acous¬ 
tic communication technology was in its infancy, an 
analog-based hardware platform was developed. Then 
digital techniques brought important improvements for 
increasing the reliability of transmissions using explicit 
error-correction techniques and increasing some level 
of compensation for the channel reverberation both 
in time (multipath) and frequency (Doppler spread¬ 
ing) [25.46]. There are several methods in digital mod¬ 
ulation, where the most fundamental techniques are 
PSK (phase shift keying), FSK (frequency shift keying), 
ASK (amplitude shift keying), and quadrature ampli¬ 
tude modulation (QAM), and their improved versions 
are MFSK (multiple FSK), QPSK (quadrature phase 
shift keying), MPSK (M-ary phase shift keying), MSK 
(minimum shift keying), GMSK (Gaussian minimum 
shift keying), MQAM (M-ary quadrature amplitude 
modulation), and so on. 

Just like a typical network, a data packet is packed 
with a header and a tail for addressing, error detec¬ 
tion and correction not to be corrupted by transmis¬ 


sion errors. In particular, the data link layer protocol 
implemented in some modems detects errors in a re¬ 
ceived packet and then requests retransmissions, which 
degrades the effective transfer rate. To reduce the re¬ 
transmission rate, some modems have a function to 
adjust the speed of transmission depending on the chan¬ 
nel condition. Another important technique is about 
minimizing the effects of multipath, which is most 
prominent over long ranges and shallow water because 
the original signal can be bounced between the sur¬ 
face and bottom before arriving at the receiver. There 
are a few algorithms for robust data transmission in 
multipath environments, such as convolutional coding, 
multipath guard period, and data redundancy. Also, 
a time-reversal technique is an alternative to reducing 
the effects of multipath [25.12]. From a practical point 
of view, acoustic communication has some other lim¬ 
itations, such as horizontal communication in shallow 
water, the loss of line of sight between the modems, 
the in-band noise (typically caused by another sonar 
system), echo sounder from ships, a shallow zone, and 
clouds of air bubbles near a receiver. These limitations 
might degrade its performance or make it unreliable. 

Electromagnetic waves are strongly absorbed by 
sea water, but, for close-range communication, electro¬ 
magnetic communication would be useful because it 
would be less affected by multipass interferences. JAM- 
STEC (Japan Agency for Marine-Earth Science and 
Technology) has developed a new communication using 
electromagnetic waves. This was experimentally evalu¬ 
ated up to 50m distance [25.12]. 

Another approach is an optical underwater commu¬ 
nication that achieves much higher data transfer rates 
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than an acoustic communication system at significantly 
lower power consumption and smaller packaging. 
However, the operation requires a point-to-point 
communication setting, requiring the alignment of the 
receiver and the transmitter for the communication 
to work effectively. Their range and scope are also 
affected by the water clarity, water light absorption, 
and power loss due to propagation spherical spread¬ 
ing [25.47]. Recently, the Massachusetts Institute of 
Technology (MIT) proposed a real-time video delivery 
solution based on underwater optical communication, 
AquaOptical II that consists of a high-bandwidth 
wireless optical communication device and a two-layer 
digital encoding scheme designed for error-resistant 
communication of high-resolution images. Its maxi¬ 
mum transmission bandwidth of 4 Mbps and various 
video streaming tests showed similar performances 
at close ranges of up to 25 m in a pool trial [25.48], 
and up to 3—4 m on the Charles River near Boston. It 
requires accurate motion stabilization because of the 
narrow beams involved [25.49]. 

When an AUV is on the surface or an AUV exposes 
an antenna above the surface, radio communication is 
the best option. For a short range in which an AUV is 
operating near shore or its support craft, 900 MHz wire¬ 
less modems or 2.4 GHz 802.11 wireless networks are 
common considering antenna sizes [25.49]. For longer 
ranges, radio links make use of an earth orbiter satel¬ 
lite like Argos satellites and Iridium satellites. In some 
cases, we have to make our own pressure-tight antenna, 
including a motion stabilizer to track the satellite re¬ 
gardless of fluctuation of motion. A tracking system for 
the Urashima vehicle was developed using data on net¬ 
work design and the oscillating characteristics of the 
vehicle [25. 12]. Oscillation of the vehicle was measured 
with the high-accuracy inertial navigation system in¬ 
stalled in Urashima in a sea trial. An oscillation angle 
of 7° and period of 0.15 Hz were estimated. 

25.5.3 Software Architecture 

includes various software components as shown in 
Fig. 25.5. The lower part of the software architec¬ 
ture is middleware as a class of software technologies 
designed to help managing the complexity and hetero¬ 
geneity inherent in distributed systems [25.50]. It is 
defined as a layer of software above an operating system 
but below application programs that provide a common 
programming abstraction across a distributed system. 

In the literature, AUV architectures were classi¬ 
fied into four categories - the hierarchical architecture, 
the hierarchical architecture, the subsumption architec¬ 
ture, and the hybrid architecture [25.51] or into three 
categories - the deliberative architecture, the reactive 


architecture, and the hybrid architecture [25.52]. Since 
most of the recently developed architectures are hybrid, 
instead of describing each architecture, key features of 
recently proposed software architectures for AUVs are 
described below. 

Teleo-Reactive Executive (T-REX) was developed 
for a specific underwater robot by the Monterey Bay 
Aquarium Research Institute [25.53-55]. It is a goal- 
oriented hybrid executive with an embedded automated 
planning adaptive execution using agents. In order to 
make the embedded automated planning scalable, the 
T-REX enables the scope of deliberation to be parti¬ 
tioned functionally and temporally inside units called 
Teleo-reactors. A Teleo-reactor agent is considered the 
coordinator of a set of concurrent control loops, which 
is characterized by functional scope, temporal scope, 
and timing requirements. 

Huxley was developed by Bluefin Robotics origi¬ 
nally for its fleet of AUVs. Since it is for their wide 
range of commercial products, the most fundamental 
requirement was the flexibility in terms of robustness 
and reliability, extensibility for new hardware, main¬ 
tainability, and testability [25.56]. It consists of two 
layers - a reactive layer and an executive layer - and it 
allows expanding these core layers through a standard 
interface. Another key element is the Huxley messaging 
protocol, called the stream-oriented messaging archi¬ 
tecture (SOMA), where SOMA is a publish-subscribe 
messaging protocol. Huxley’s layered architecture with 
SOMA and SOMA interfaces can be extended along 
with new functionalities and/or modified configurations 
of future AUVs while localizing changes and increasing 
the reusability of components. 

The lightweight communications and marshalling 
(LCM) system was designed as an alternative for 
implementing inter-process communication (IPC) for 
real-time robotics in the marine environment. It was 
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Fig. 25.5 Typical configuration of software for robot 
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Fig. 25.6 Key characteristics of middleware 


developed specifically for low-latency, high-throughput 
communication accomplished by a set of software 
tools for message passing between modules using 
user datagram protocol (UDP) multicast, marshalling 
(encoding and decoding) of LCM messages, language- 
independent data structures for messages of any digital 
data, allowing the transparent distribution of the 
system. This architecture was verified by two marine 
robotics applications [25.57]. 

MOOS-IvP consists of two distinct open source 
software projects. The Mission Oriented Operating 
Suite (MOOS) is, as a core middleware, a suite of 
libraries and executables developed by the Mobile 
Robotics Group at the University of Oxford. It provides 
inter-process communication using a publish-subscribe 
model, as a star topology where all messages go through 
a central MOOS server. IvP (interval programming) 
refers to a multiobjective optimization method used by 
the IvP helm for arbitrating between competing behav¬ 
iors in its behavior-based architecture. MOOS provides 
modules for navigation, control and data logging, tools 
for mission replay from log files and communication 
debugging, and also provides driver modules for some 
sensors widely used with AUVs [25.57-59]. MOOS- 
IvP has been used for several AUVs including the 
Bluefin 21 in. UUV, the Hydroid REMUS-100 and 
REMUS-600 UUVs, the Ocean Server Iver2 UUV, the 


Ocean Explorer 21 in. UUV, autonomous kayaks from 
Robotic Marine Systems and SARA Inc., and two larger 
USVs from the NATO Underwater Research Center in 
La Spezia Italy [25.54,58]. 

Let us look at middleware that should be easy to use, 
robust against change, reliable for any faults, easy to 
maintain, efficiently flexible, and supportive for hetero¬ 
geneous and distributed hardware components. Well- 
structured proven middleware with some often-needed 
and value-added functions would minimize time and 
cost for development. In the literature [25.50-53, 56- 
58, 60-64], there are many demanded characteristics to 
realize above expectations, as shown in Fig. 25.6. Fun¬ 
damental characteristics of middleware are abstraction 
and modularity. Abstraction is expected to simplify and 
to lessen burdens of application development process, 
and modularity is a necessary condition of flexibil¬ 
ity and customizability. Abstraction allows developers 
to concentrate on building advanced algorithms them¬ 
selves by hiding details of low-level hardware resources 
of heterogeneous and distributed hardware environ¬ 
ments [25.51,62]. This concept includes more specified 
subissues: interoperability by using simplified inter¬ 
faces with communication and standard; customizabil¬ 
ity with real-time capacity; and flexibility and resource 
management. Additionally, resource management han¬ 
dles faults in hardware. From these characteristics, 


611 


Part B | 25.5 















































Part B | 25.5 


612 Part B 


Design 


a middleware is free from the specific hardware envi¬ 
ronments, and could obtain portability, self-configuring 
and self-optimizing. To achieve flexibility, we have 
to follow a modular-based development for increasing 
reusability of existing components based on modularity 
and interoperability. This development approach allows 
us to focus on building modules without considering 
specific relationships between existing modules. Based 
on this, expandability and maintainability can be real¬ 
ized, where expandability is the ability to incorporate 
new resources, for example, newly developed sensors 
without interfering with the existing parts, and main¬ 
tainability is about how easy to embrace modification 
of components without affecting the rest. To imple¬ 
ment a middleware, standards such as the common ob¬ 
ject request broker architecture (CORBA), the Internet 
communications engine (ICE), and the open dynamics 
engine (ODE) are applied [25.64], which bring many 
advantages. Basically, it requires less effort in imple¬ 
menting and evaluating a core engine from scratch, 
and some of these have additional functionalities such 
as standardized interface and security. Security as¬ 
pects such as authentication, authorization, and secure 
communication become essential for collaborations of 
multiple robots or classified applications [25.64]. So 
far, many research groups have proposed their own mid¬ 
dleware for their robots, such as MIRO (middleware 
for robot), ORCA (open robot control architecture), 
UPnP (universal plug and play), RT (robot-technology), 
the player and stage, OPRoS (the open platform for 
robotic services), ROS (robot operating system), ORO- 
COS (open robot control software), ERSP, MRDS (Mi¬ 
crosoft robotics developers studio), MOOS (the mission 
oriented operating suite), and Webots [25.50]. 

A development of middleware itself is not a trivial 
work. Understanding concepts is one thing and imple¬ 
menting them is another. The reasons for possible errors 
are mainly: 

1. Many inherently distributed and heterogeneous 
components which will vary to improve perfor¬ 
mance. 

2. The limited and restricted resources available on 
a robot platform. 

3. The high rate of system failures from incorporat¬ 
ing complicated algorithms and interacting with the 
environment, causing exceptional situations [25.50, 
62], 

Basically, an underwater robot’s middleware has 
almost the same requirements as a robot, but the weight¬ 
ing of requirements and the point of view of require¬ 
ments are little different. For an underwater robot’s 
middleware, the following points must be considered: 


1. Limitation of resources: Underwater robots use 
a low bandwidth of an underwater communication 
and thus, has to manage a mission using internal 
computing resources. 

2. Unstable sensor information: Underwater sensors 
based on sonar are not stable and accurate enough to 
directly use data for robot control systems. There¬ 
fore, assisting algorithms such as well-tuned fil¬ 
tering algorithms and sensor fusion algorithms are 
needed. 

3. Lack of accessibility: Underwater robots do not 
allow any access of an electrical contact used 
for debugging because of its water-proof pressure 
vessel. 

This is one of the most challenging aspects in build¬ 
ing an underwater robot. Thus, for the underwater robot, 
middleware should provide an independent channel for 
a supporting a debugging process. 

No middleware is perfect, but we believe that we 
can achieve significant improvements through trial and 
error coupled with extensive feedback from the com¬ 
munity [25.50]. When you develop system software 
and its architecture on your own, the implementation 
of abstraction and modularity can be understood with 
the white-box framework and the black-box framework 
concepts. The black-box framework supports extensi¬ 
bility by defining interfaces for components without 
detail information inside the components. White-box, 
as a base class in a class library can be designed for 
expanding with subclasses, based on object-oriented 
language features [25.63]. In the early stage of devel¬ 
opment, software architecture is mainly conceived as 
a white box framework. Hence, information inside the 
boxes, that is, classes, should be understood. As the ar¬ 
chitecture becomes more advanced and the components 
become more concrete, the black-box framework is the 
solution for most common expansions. At the moment, 
MOOS-IvP would be the best choice to start developing 
your vehicle software because it is open source soft¬ 
ware, and there is a relative wealth of information and 
case examples for AUVs online. 

As an alternative to formal software architectures 
discussed above, we propose a simple but effective so¬ 
lution in testing a robot as shown in Fig. 25.7. This 
architecture has one real-time thread with several non- 
real-time threads for controlling actuators and devices 
installed on the robot. 

In the Fig. 25.7: 

1. The packet conveys three types of commands from 
an operator - commands for teleoperation, com¬ 
mands for RT-thread (real-time thread) itself and 



Underwater Robots I 25.5 Computers, Communications, and Architecture 



e 


Fig. 25.7 Example diagram of simple software architecture 


functions in the selected tasks, and commands for 
devices of the robot. 

2. Device control commands run in a non-RT thread 
triggered by the RT-thread, typically every 1 s. 

3. All sensor data including teleoperation commands 
and results of any sensor fusion algorithms are inde¬ 
pendently saved in common memory, and thruster 
commands, that is, control input are also saved 
here. 

4. This architecture has a manual task coordinator and 
each task can be easily synthesized using functions 
from a function pool. 

5. A robot is basically in one of two modes, sleep and 
run, where the sleep mode is doing only health¬ 
monitoring with basic sensors, the run mode runs 
a selected task which consists of subtasks in func¬ 
tion pool. Tasks are designed for specific purposes, 
and one of them is selected by an operator before 


running. Of course, the default task should be the 
teleoperation task. 

6. The RT-thread carries three main functions: doing 
any critical device control commands first, DA con¬ 
verting for control input of previous sampling time 
saved in the common memory, and processing al¬ 
gorithms using a sensor input saved in the common 
memory. 

Since the function of this RT-thread is based on 
a basic sampling time, we could easily implement 
any multisampling time structures for various latency 
times of sensors and load-distributing structures to en¬ 
hance the usage of computing resources. This would 
be done by calling functions at multiples of the basic 
sampling time. This architecture has been applied to 
several test-bed AUVs and smart ROVs [25.42,43,65] 
dda >M'iMj<Kyi i 
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25.6 Underwater Manipulators 

Most commercial underwater robots with manipulators 
are ROVs. A typical work-class ROV for intervention 
missions has a 7 degrees-of-freedom (DOF) manipula¬ 
tor and a 5 DOF grabber. The manipulator is used for 
interactive manipulation by a human operator on the 
mother ship, while the grabber is used to maintain a rel¬ 
ative position of ROV to the target object by grabbing 
a target structure in the presence of an underwater cur¬ 
rent or wave. This section describes underwater manip¬ 
ulator systems based on materials presented in [25.66]. 
Table 25.7 lists samples of heavy duty underwater ma¬ 
nipulators currently available in the market. They are 
quite expensive but utilize proven technologies in the 
field. Flowever, most commercial underwater manip¬ 
ulators are hydraulic-driven and developed for ROVs 
and they are not suitable for AUVs that have limited 
on-board power and whose on-board sensors could be 
sensitive to noise. Therefore, electric-driven manipula¬ 
tors would be preferred for AUVs like SAUVIM. 

A typical work-class ROV for intervention missions 
requires two individual operators, one steering the vehi¬ 
cle and the other operating the manipulator. One of the 
main tasks of the vehicle operator is to keep the ROV 
as steady as possible (station-keeping) since the vehicle 
motion can disturb the manipulator and greatly affects 
the accuracy of its end-effector interacting with the en¬ 
vironment. The accuracy of the end-effector would be 
greatly improved if the manipulator controller could 
compensate for any disturbance due to the motion of 
the vehicle. However, most commercial ROV systems 
do not use vehicle motion compensation control for the 
manipulator. In the past, a very few AUV systems for 
intervention missions have been reported and they were 
used as a test-bed for developing autonomous manipu¬ 
lation or as a working model for research. 

25.6.1 Dynamics 

The analysis of the dynamics of underwater vehicle ma¬ 
nipulators is much more complex than that of regular 
manipulators on the factory floor. For example, it is dif¬ 
ficult to accurately introduce the effects of the added 
mass and the added moment of inertia in the model¬ 
ing process for multiple links of the manipulator. The 
frictional force and the drag force due to the veloc¬ 
ity of the manipulator itself, waves, and currents are 
also complex in modeling. The modeling and control 
of underwater manipulators were studied by several re¬ 
searchers, including [25.67-78]. In this section, it is not 
our intention to derive details of the full dynamic equa¬ 
tions of the underwater vehicle manipulators. Instead, 
we would like to point out several specific elements that 


differentiate underwater vehicle manipulators from reg¬ 
ular manipulators. The hydrodynamic forces acting on 
a rigid body moving in fluid can be found in [25.79, 
80]. Similar to the vehicle main body, each link of the 
manipulator will be influenced by hydrodynamic forces 
including added mass, buoyancy acting at the center of 
buoyancy of the link, the fluid acceleration force result¬ 
ing from the acceleration of the fluid itself, and the fluid 
drag forces exerted on the link. 

As the profile drag forces are dominant for slowly 
moving objects, the drag force exerting on an infinites¬ 
imal element of the j-th link and the total drag force on 
the link can be expressed as 

d D fi = ^pC D biVn\v ri \dx , (25.1) 

/ 

Dfi=^p J C D biV ri \v ri \dx , (25.2) 

0 

where Co is the drag coefficient, p is the fluid density, 
v r i is the relative velocity of the element to the fluid flow 
velocity, bj d.v is a projected area of the element, and / is 
the link length. 

Considering the effects of the hydrodynamic vari¬ 
ables above and assuming that the vehicle is stationary 
during the manipulation, the dynamic model of an un¬ 
derwater manipulator having a series of links and joints 
can be represented by the following equations, 

M m (q)q + C m (q,q)+D m (q,q) + F m (q) 

T" G m {q) T Td — r m , 

where q e R nxl is a joint angle vector, T m eR nXl 
a joint torque vector, M m (q) e R' !X " an inertia matrix 
including added mass terms, C m (q, q) e R" xl a nonlin¬ 
ear vector arising from centrifugal and Coriolis effects 
including added mass terms, D m (q, q) e R nXl a nonlin¬ 
ear vector due to hydrodynamic forces, such as drag 
forces, F m (q) e R" xl a nonlinear vector due to fric¬ 
tion at the manipulator joints, G m (q) e R" x 1 a nonlinear 
vector due to gravity and buoyancy forces, t, i e R nX 1 
a vector of unknown signals due to unmodeled dynam¬ 
ics or external disturbances such as current. 

During the manipulation, the end-effector or tool 
of the manipulator can be in contact with the envi¬ 
ronment, and forces and moments would occur at the 
end-effector. In this situation, (25.3) must be modified 
by adding the reaction torque to the end-effector force 
on the right-hand side of (25.3) 

U = Jm<We - 


(25.4) 
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Table 25.7 Samples of commercially available underwater manipulators 


Specification 

Eca robotics 

International 

submarine 
engineering (ISE) 

Kraft telerobotics 

FMC/Schilling 

robotics 

Western space and 
marine 

Model 

Arm 7H 

Magnum-7 

Predator 

Titan 4 

The ARM 

DOF 

6 plus gripper 

6 plus gripper 

6 plus gripper 

6 plus gripper 

6 plus gripper 

Power source 

Hydraulic 

Hydraulic: max 

1000 psi/191pm 
(5 gpm) 

Hydraulic: 

103 bar-207 bar 
(1500 psi-3000 psi), 
191pm (5 gpm) 

Hydraulic: 

103 bar-207 bar 
(1500 psi-3000 psi), 
191pm (5 gpm) 

Hydraulic: 
max 204 bar 
(3000 psi), 7.61pm 
(2 gpm) 

Material 

Titanium or alu¬ 
minum 

Aluminum with stain¬ 
less steel fittings 

Anodized aluminum 

and stainless steel 

Titanium 


Max grip force 

490N (llOlbf) 

2009 N (450 lbf) 

1334 N (300 lbf) 

4092 N (920 lbf) 

1467 N (330 lbf) 

Wrist torque 

108 Nm (80 ft-lbs) 

108 Nm (80 ft-lbs) 

135 Nm (100 ft —lbs) 

170 Nm (125 ft — lbs) 


Actuator 

Hydraulic cylinder 

Hydraulic cylinder 

Hydraulic cylinder 

Hydraulic cylinder 

Hydraulic cylinder 

Max reach 

1700 mm (66") 

1524mm (60") 

2019mm (79.50") 

1920 mm (75.7") 

1700 mm (66") 
from azimuth axis 
to fingers 

Max lift capacity 


454 kg (1000 lbs) 

227 kg (500 lbs) 

454 kg (1000 lbs) 

45.4 Kg (100 lbs) 

Max lift at full 

extent 

90 kg (198 lbs) 

295 kg (650 lbs) 

91 kg (200 lbs) 

122 kg (270 lbs) 

29.5 kg (65 lbs) 

Working depth 
(standard) 

7500 msw 
(24 600 fsw) 

6000 msw 
(19 700 fsw) 

3000 msw 
(9800 fsw) 

4000 msw (13 100 fsw) 

Unlimited 

Working depth 
(Extended) 

Unlimited 

11000 msw 
(36 000 fsw) 

6500 msw 
(21 000 fsw) 

7000 msw (22 967 fsw) 

Unlimited 

Homepage 

www. ecarobotic s. com 

www.ise.bc.ca 

WWW. 

krafttelerobotic s. com/ 

WWW. 

fmctechnologies. com/ 

S chillingrobotics. aspx 

www.wsminc.com 


where J m e R (,Xn is the manipulator Jacobian matrix, 
and f e e J? 6xl a vector of forces, and torques at the end- 
effector. 

The dynamics of the underwater vehicle manipula¬ 
tor is quite complicated, highly nonlinear, and involves 
coupled equations with unknown parameters and dis¬ 
turbances as shown in (25.3). It is almost impossible to 
accurately model the dynamic equations and to operate 
the manipulator at a nominal speed using a conventional 
controller. However, when the manipulator moves very 
slowly in a friendly environment with no or minimal 
current or wave, the dynamics of the manipulator can 
be represented by the simplified linear version of (25.3) 
since the effect of velocity-dependent terms become 
negligible. In fact, during the actual operation in the 
field, most commercial underwater manipulators move 
very slowly (much less than 1 rad/s) for safety and 
other reasons, and they use conventional joint con¬ 
trollers for the master-slave teleoperation. 

25.6.2 Teleoperation 

A typical setup for teleoperation involves the operator 
sitting in the control room located on the mother ship 
and holding a master arm that is a miniature of the 
actual underwater vehicle manipulator (slave arm) at¬ 


tached to the ROV. The operator controls the slave arm 
by moving the master arm and its motion is followed by 
the slave arm. During the operation, the operator relies 
on visual information of the work site in the form of 
a series of two-dimensional (2-D) video images that are 
captured by cameras on the vehicle main body and the 
slave arm. The images are then transmitted to the con¬ 
trol room monitors by an underwater cable. The view of 
the work site is not only limited, but the visual informa¬ 
tion especially for the deep sea operation is also often 
delayed. If the vehicle is not holding or sitting on an un¬ 
derwater structure, the ROV operator must try to control 
the vehicle for station-keeping during the manipulation. 
However, even if there is no current or wave during 
the station-keeping, the vehicle moves like free float¬ 
ing within the accuracy of the vehicle position sensors 
in the order of a meter or so. Therefore, it is difficult 
to achieve the accuracy at the end-effector since under¬ 
water vehicle manipulators are attached to the vehicles 
that are constantly moving, unlike industrial manipu¬ 
lators, whose bases are fixed on the factory floor. As 
mentioned in [25.81], many simple tasks on the ground 
such as plugging become very difficult to perform by 
the underwater vehicle manipulator. Underwater plug¬ 
ging tasks, done on a trial-error basis as the required 
precision cannot be achieved by the ROV manipulator 
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system, take hours to complete. Therefore, operator fa¬ 
tigue often becomes a critical issue in tele-operating the 
underwater vehicle manipulator. 

25.6.3 Autonomous Manipulation 

Autonomous manipulation on a moving base, such as 
terrestrial mobile robots, humanoids, and underwater 
robotic vehicles is a very challenging task in the area of 
robotics in general, especially in unstructured environ¬ 
ments, such as underwater. It is defined as the capability 
of a robot system that performs intervention tasks 
requiring physical contact with unstructured environ¬ 
ments without continuous human supervision. Unlike 
industrial manipulators which have fixed bases on the 
floor, autonomous manipulation requires a system ca¬ 
pable of assessing a situation, including self-calibration 
based on sensory information, and executing or revis¬ 
ing a course of manipulating action without continuous 
human intervention. Therefore, developing a system 
capable of fully autonomous manipulation would be 
a great achievement and make a substantial impact on 
a variety of application areas with significant economi¬ 
cal, societal, and scientific importance [25.82]. 

Let us look at one scenario of cutting an underwater 
cable. With given information about the cable’s location 
and shape, the vehicle has to navigate to the location. 



Fig. 25.8 The SAUVIM vehicle (Sand Island, Hawaii, 
2008) 


identify the cable, position itself, and cut the cable. It 
may sound like a very simple task. However, it would 
not be so simple if it has to be done in autonomous 
mode. Even in teleoperation mode, it would not be an 
easy task without the coordinated motion control if the 
vehicle is floating in water. In fact, the low bandwidth 
and significant time delay inherent in acoustic subsea 
communications represent a considerable obstacle to 
remotely operate a manipulation system, making it im¬ 
possible for remote controllers to react to problems in 
a timely manner. Nevertheless, robots for autonomous 
underwater intervention would pave the way for a dif¬ 
ferent range of new operations, such as deep-ocean 
and under-ice exploration, tasks in hazardous areas, 
tasks in natural or man-made disastrous regions, au¬ 
tomated searches, and surveillance missions, to name 
a few. 

In the past, many researchers have studied advanced 
control of AUV itself as reported in [25.83-91], but 
only a few AUVs with manipulators were introduced. 
OTTER is an AUV equipped with a single degree- 
of-freedom arm, which was designed to be used as 
a test-bed for developing autonomous technologies at 



Fig. 25.9 Girona 500 I-AUV 
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the Stanford Aerospace Robotics Lab in 1996. It is 
a hovering-capable underwater vehicle that operates 
in a test tank at the Monterey Bay Aquarium Re¬ 
search Institute (MBARI) [25.92]. Another intervention 
AUV, namely ALIVE, was developed by Cybemetix in 
2003. The aim of the EU-funded ALIVE project was 
to develop an intervention-AUV capable of docking to 
a subsea structure, which has not been specifically mod¬ 
ified for AUV use. A description of the ALIVE vehicle 
was given in [25.93]. 

The key technology in underwater intervention per¬ 
formed with autonomous vehicles is autonomous ma¬ 
nipulation. In the literature, no such system fully func¬ 
tional for autonomous manipulation was reported until 
recent developments with a semi-autonomous underwa¬ 
ter vehicle for intervention missions, SAUVIM [25.94— 
97], SAUVIM (Fig. 25.8) funded by the US Office 
of Naval Research was jointly developed by the Au¬ 
tonomous Systems Laboratory (ASL) of the University 
of Hawaii, Marine Autonomous Systems Engineering 
(MASE), Inc. in Hawaii, and Naval Undersea Warfare 
Center Division Newport (NUWC) in Rhode Island. 
The first fully autonomous manipulation in an un¬ 


structured ocean environment was demonstrated with 
SAUVIM at Snug Harbor, Honolulu, Hawaii on Jan¬ 
uary 20, 2010. Following its success, more underwater 
vehicles having the capability of autonomous manipula¬ 
tion are expected to develop in the near future. Among 
some exemplar, underwater intervention tasks are ob¬ 
ject recovery/rescue, and maintenance/repairing of un¬ 
derwater facilities. Therefore, one may consider a task 
like black-box recovery, grasping an object of inter¬ 
est and placing it at a desired location for benchmarks 
in future development. One recent development after 
SAUVIM is the TRIDENT project [25.98-101] in Eu¬ 
rope, which developed Girona 500 I-AUV (Fig. 25.9). 
They performed an autonomous underwater interven¬ 
tion task, with a black-box recovery in Port de Sobers 
harbor (Mallorca) in October 2012 [25.14]. 

As subsea business in oil and gas industry is about 
to boom in the very near future, AUVs for inter¬ 
vention missions will receive more attention. While 
autonomous underwater manipulation is still an active 
research topic, recent development with SAUVIM and 
Girona 500 I-AUV would accelerate advancement in 
the field. 


25.7 Conclusions and Further Reading 

In this chapter, we described major subsystems of un¬ 
derwater robots - mechanical systems, power systems, it 
actuators, sensors, computing systems, software archi- P 
tecture, communications, and manipulator systems, and ri 
also discussed critical design issues from the authors’ t( 
practical experiences. \ 

There are currently more than 1000 work-class n 
ROVs. They are used for various applications, such o 
as scientific research, military operations, and under- N 
water constructions. About 57% of ROVs are used b 
for offshore oil and gas industry. Most commercial h 
ROVs have used old but proven technologies in the tt 
past 20 years. However, the ROV operation is very si 
expensive as it requires a mother ship and, there- \ 
fore, developing advanced ROVs or smart ROVs for \ 
efficient operations in terms of operating hours and A 
costs has received much attention. There are more 
than 550 AUVs in use worldwide and some of them ti 
are commercially available. While the use of AUVs h 
is still limited, advantages of AUVs have been rec- d 
ognized by the recent AUV operations in the field: c 
post-incident monitoring of BP Macondo oil leakage u 
in 2010 and the successful recovery of black boxes 3 
of Air France flight 447 that crashed into the At- F 
lantic Ocean in 2009. Therefore, it is expected to see w 
more underwater robots for various applications in the C 
future. v 


Some web-based resources related to the top¬ 
ics of this chapter include the following sites: the 
Remotely Operated Vehicles Committee of the Ma¬ 
rine Technology Society (http://www.rov.org/), the Au¬ 
tonomous Undersea Vehicles Application Center (AU- 
VAC, http://www.auvac.org) and the technical com¬ 
mittee on Marine Robotics of the IEEE Society 
of Robotics and Automation (http://webuser.unicas.it/ 
MarineRoboticsTC). A list of AUVs and developers can 
be found at http://www.transitport.net/Lists/AUVs.Org. 
html. AUVSI (Association for Unmanned Vehicle Sys¬ 
tems International) and ONR (US Office of Naval Re¬ 
search) sponsor International Autonomous Underwater 
Vehicle Competition as well as Autonomous Surface 
Vehicle competition (http://www.auvsifoundation.org/ 
AUVSI/FOUNDATION/Competitions/). 

It is worthwhile to note that several important ini¬ 
tiatives [25.1,101] in underwater robotics in Europe 
have been made including AMADEUS for developing 
dexterous arms for underwater manipulation; GREX - 
coordination and control of cooperating heterogeneous 
unmanned systems in uncertain environments; CO- 
3AUVs - cognitive cooperative control for AUVs; 
FREESUBNET - a Marie Curie research training net¬ 
work on marine robotics; and TRIDENT that developed 
Girona 500 I-AUV for autonomous underwater inter¬ 
vention tasks. 
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Six-legged waking underwater robot, Crabster 

available from http://handbookofrobotics.org/view-chapter/25/videodetails/793 

Preliminary results of sonar-based SLAM using landmarks 

available from http://handbookofrobotics.org/view-chapter/25/videodetails/794 

First record of deep-sea diving of Flamire, depth was 5882m 

available from http://handbookofrobotics.org/view-chapter/25/videodetails/796 

Preliminary experimental result of an ROV, iTurtle 

available from http://handbookofrobotics.org/view-chapter/25/videodetails/797 
Preliminary experimental result of an AUV, ySharl<2 

available from http://handbookofrobotics.org/view-chapter/25/videodetails/799 
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Unmanned aircraft systems (UASs) have drawn 
increasing attention recently, owing to ad¬ 
vancements in related research, technology, and 
applications. While having been deployed suc¬ 
cessfully in military scenarios for decades, civil 
use cases have lately been tackled by the robotics 
research community. 

This chapter overviews the core elements of 
this highly interdisciplinary field; the reader is 
guided through the design process of aerial robots 
for various applications starting with a qualitative 
characterization of different types of UAS. Design 
and modeling are closely related, forming a typi¬ 
cally iterative process of drafting and analyzing the 
related properties. Therefore, we overview aerody¬ 
namics and dynamics, as well as their application 
to fixed-wing, rotary-wing, and flapping-wing 
UAS, including related analytical tools and practical 
guidelines. Respecting use-case-specific require¬ 
ments and core autonomous robot demands, we 
finally provide guidelines to related system inte¬ 
gration challenges. 
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26.1 Background and History 

The field of aerial robotics encompasses a very broad 
class of flying machines that nowadays often possess 
the perception capabilities and decisional autonomy to 
accomplish complex tasks without the need for any 
direct human interventioning. Historically and within 
the aerospace jargon, robotic flying machines are com¬ 
monly referred to as unmanned aerial vehicles (UAV s), 
while the entire infrastructures, systems and human- 
machine interfaces required for autonomous operation 
are often called unmanned aerial systems (UAS). Aerial 
robotic technologies are currently on the cutting edge 
of aerospace and robotic research. Breakthrough con¬ 
tributions take place in various fields such as design, 
estimation [26.1], perception [26.2], control [26.3], and 
planning [26.4], paving the way for a historical change 
on how flying systems are operated and what applica¬ 
tion challenges they fulfill. 

As a class of systems, aerial robots have their roots 
in the first guided missiles; however, nowadays they re¬ 
fer to a wide variety of advanced intelligent systems. 
According to the American Institute of Aeronautics 
and Astronautics (AIAA) [26.5], a UAV is defined 
as 

an aircraft which is designed or modified, not to 
carry a human pilot and is operated through elec¬ 
tronic input initiated by the flight controller or by 
an onboard autonomous flight management control 
system that does not require flight controller inter¬ 
vention. 

As is generally the case in robotics, aerial robots tend 
to become more and more complex systems as a result 
of the effort to achieve advanced decision making and 
planning capabilities based on its on-board perception 
of the environment and a set of relatively abstract mis¬ 
sion goals. 

Aerial robots posses the unique capability to gen¬ 
tly fly over terrain that other robots struggle to roll or 
crawl over. The price to be paid is related with the 
advanced challenges in terms of system design, propul¬ 
sion, perception, control, and navigation. Autonomous 
flight requires handling of all six degrees of freedom 
and advanced cognition capabilities within challenging 
environments. In that sense, perception and naviga¬ 
tion complexity drastically increase, while payload and 
available power consumption for processing tends to 
be limited, especially as scale decreases. Essentially, 
the design of aerial robots requires increased attention 
and thorough selection, or even combination, of one or 
more existing or new flying concepts, electronic com¬ 
ponents and algorithms. The design engineer has to 
assess specific optimization challenges and trade-offs 


as important desired goals like decreased weight and 
modularity typically contradict each other. 

26.1.1 A Glimpse of History 

Aerial robotics is a field of active research and promis¬ 
ing perspectives, yet it already accumulates more than 
a century of developments. Figure 26.1 depicts some 
historical as well as recent examples of UAVs in the 
military and civilian sector. Starting as conceptual de¬ 
signs in the context of the human efforts to develop 
flying machines, aerial robots soon proved their ex¬ 
tensive potential and have already created their own 
legacy. As was also the case for manned aviation, aerial 
robotic technologies accelerated within the framework 
of the 20th century world conflicts. Within World War I, 
Hewitt-Sperry developed an automatic plane that acted 
as a flying torpedo, carrying onboard intelligence to au¬ 
tonomously sustain flight over long periods of time. 
This page-turning success was achieved through the 
integration of (Sperry’s self-made) gyroscopes which 
were then mechanically connected to the control sur¬ 
faces and therefore established the necessary feedback 
control loop. During World War II, the German armed 
forces deployed one of the first successful cruise mis¬ 
siles, the V-l. Despite the fact that V-l had limited 
success rate it did incorporate most of the elementary 
components, estimation algorithms and control loops 
that can allow autonomous navigation and reference 
tracking. Military applications kept being, and still are, 
the main driving force of aerial robotics research and 
the newest developments in the area change and shape 
the modern warfare. With the introduction of global 
positioning systems (GPSs), aerial robots managed to 
achieve the first completely autonomous surveillance 
missions. As information and intelligence gathering be¬ 
came one of the most important aspects of the world’s 
open or silent conflicts, military research around the 
1970s led to systems equipped with cameras and other 
sensory systems, giving birth to the UAV prototype the 
way we know it today. However, civil applications are 
currently emerging at a very fast pace and the majority 
of market predictions converge to the conclusion that 
this area will take dominant characteristics, and most 
importantly, will become an equally important - if not 
more - innovation drive. 

Within this framework, the advancements in the 
field of microprocessors, miniaturized sensing, as well 
as actuator efficiency and downscaling greatly acceler¬ 
ated the field of aerial robots and paved the way for 
the great achievements we observe today. Aerial robots 
have advanced to a state in which sophisticated sensor 
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Fig. 26.1 A glimpse on the UAS history through some examples starting from the Hewitt-Sperry Automatic Airplane 
(1917), the V-l flying bomb (1944), and the Lockheed D-21 (1962) until the recent examples of military (Predator, 
Robocopter, nEuron) and civilian (AtlantikSolar, Firefly, Apid 60) aerial robots 

modules for onboard state estimation and environmen- communication interfaces, as well as high-end-mission- 
tal perception, powerful embedded processors running oriented payloads that enable the execution of challeng- 
sophisticated navigation algorithms, potentially several ing tasks, can be tightly integrated. 


26.2 Characteristics of Aerial Robotics 


This section aims to provide an overview of the key 
characteristic features and properties of different aerial 
robotic configurations as well as a classification based 
on the key advantages and limitations of some of the 
most common flying concepts found in unmanned avi¬ 
ation. 

26.2.1 Aerial Robots Classification 

Compared to the categorization of manned aviation, 
aerial robots classification is more complex, as the 
term currently refers to a very wide variety of sys¬ 
tems of different scale, mechanical configuration, and 
actuation principles. In their vast majority, aerial robots 
correspond, in one way or another, to miniaturized ver¬ 
sions of manned aircraft designs. Relatively classical 
fixed-wing unmanned aerial systems (FW-UAS) de¬ 


signs and rotary-wing unmanned aerial systems (RW- 
UAS) such as those shown in Fig. 26.2 are common 
vehicle configurations one may encounter in most ap¬ 
plications, including those of surveillance, monitoring, 
inspection, mapping, or payload transportation. How¬ 
ever, even within these relatively traditional concepts, 
several design aspects differ from those chosen for 
manned systems. This reflects the fact that for different 
scales, the variation of the physical properties behavior, 
along with the search for optimized designs, will natu¬ 
rally lead to modified and novel design considerations. 
This is further triggered by the fact that the absence 
of a pilot on-board unlocks a wide set of engineering 
choices, typically out of question or even forbidden in 
manned aviation. As expected for a multitude of en¬ 
gineering reasons, large UAS tend to follow design 
concepts closer to - while at smaller scale to classi- 
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Fig. 26.2 Classification of aerial robotics based on their endurance and maneuverability properties. Also note the signif¬ 
icant effect of scale which highlights that comparisons should be done on similar scales 


cal designs while as scale decreases innovation - at the 
level of the flying principle - becomes more and more 
intense. 

Apart from lighter-than-air systems (LtA-UAS), 
FW-UAS tend to be the most power efficient flying 
principle, while RW-UAS are tailored to increased ma¬ 
neuverability as well as the ability of stationary vertical 
flight (hovering). This general classification (also valid 
for manned aviation) is then further complicated with 
the relatively large class of convertible designs (such 
as tilt-rotors or cruise-flight-enabled ducted fans). This 
first attempt for aerial robots classification has then to 
be further augmented to account for the biologically in¬ 
spired concepts, and especially the emerging field of 
flapping-wing UAS (Fl-UAS). Figure 26.2 provides an 
abstract - yet incomplete - overview of the vehicle 
classes one may encounter in most of the application 
fields. As shown, a large diversity is observed as a re¬ 
sult of the engineering efforts to propose designs with 
optimized endurance, agility, controllability, or even 
simplicity in a very wide scale range. In the following 
subsections, a brief overview on how the main aerody¬ 
namic forces and effects depend on the design scale of 
an aerial vehicle are provided. 

26.2.2 The Effect of Scale 

The understanding of how aerial vehicles manage to 
remain airborne, provides a useful insight into the 
effect of scale, and how different dimensioning has 


a huge impact on the efficiency of every flying machine. 
Table 26.1 provides an overview of the formulas ex¬ 
pressing the lift force, as well as the drag forces that 
govern the flight of the most common UAS configu¬ 
rations. More detailed definitions on the aerodynamic 
forces can be found on the subsequent sections. 

Within these equations, p is the density of the air 
while the remaining parameters are specific to the ve¬ 
hicle configuration. For FW-UAS, Cl and cd represent 
the wing lift and drag coefficients, respectively, A is 
the wing area, and V t denotes the airspeed. For the 
case of RW-UAS, ex and cq denote the rotor thrust 
and drag coefficients, (jtR 2 ) is the rotor disk area, 
is the angular velocity of the rotor, and R is the rotor 
disk radius. Finally, for LtA-UAS, V LtA is the volume 
of the blimp, c^ tA is the drag coefficient depending 
on the blimp shape, V, is the blimp’s airspeed, A LtA 
is the blimp surface in the direction of motion and 
p gas is the filling gas density. Figure 26.3 illustrates 

Table 26.1 Formulas of the main aerodynamic forces and 
moments for common UAS configurations. FW stands for 
fixed-wing UAS, RW for rotary-wing, and LtA for lighter- 
than-air 


UAS 

Lift/Thrust 

Drag force/Moment 

FW 

L = \c^pAV 2 

D=\c dP AV 2 

RW 

T = c T p(yrR 2 )(RQ) 2 

Q = cqp(tzR 2 )(RQ) 2 R 

LtA 

U = -gV LtA (Pgas - P) 

D = \dfi, K pA UK V 2 
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Fig.26.3a-c Main aerodynamic forces applied on (a) fixed-wing, (b) rotary-wing, and (c) lighter-than-air systems, 
(a) AtlantikSolar is a solar-powered FW-UAS developed by the Autonomous Systems Lab at ETH Zurich, (b) Fire¬ 
fly is developed by Ascending Technologies GMBH, while (c) Skye is developed by students of ETH Zurich 


these forces on the body of the relevant aerial vehicle 
configurations. 

Derivation of scaling laws starts with the obser¬ 
vation of the lift and drag forces and how these are 
functions of scale-dependent parameters such as the 
area of the wing or the rotor radius. Proper dimen¬ 
sioning is essentially a very complex procedure where 
a multitude of factors has to be taken into account. 
Among others, one has to account for the issues of 
aerodynamics efficiency, availability of propulsion sys¬ 
tems at a given scale, the technologies they employ 
(e.g., electric motors, jet engines) as well as the sim¬ 
plicity and robustness of the corresponding mechanical 
configuration. In the following, scaling laws and rel¬ 
evant design guidelines for fixed-wing, rotary-wing, 
and lighter-than-air systems are provided. Only a brief 
overview is provided for the case of flapping-wing sys¬ 
tems, as the effect of scale on such UAS configurations 
is separately discussed within Sect. 26.6. 


FW-UAS 

Scaling laws express the dominant role of size and scale 
for a given vehicle configuration. In the case of fixed- 
wing systems, the wing loading, defined as the ratio of 
the weight (VP) versus the wing area A, is the key param¬ 
eter one has to focus to get some first insight on the role 
of scale. The Tennekes diagram shown in Fig. 26.4 pro¬ 
vides a visual interpretation of this fact [26.6]. Working 
around the point that the lift force exactly counteracts 
the weight, the indicated trend line was derived using 
the following formulas [26.7] 


W 

~A 

W 

~A 

A 


= IfWAl , 



= b„ 


( 26 . 1 ) 


where V, is the airspeed, W is the weight, A is the wing 
area, b w is the wing span, and c w is the wing chord. 
These equations express the role of the lifting properties 


of the airfoil and airspeed against the ratio of the weight 
of the flying body and its wing area. For this analysis, 
a fixed aspect ratio (A = /? w /c w ) is assumed for all sizes 
of aircraft. Although such a simple analysis does not 
account for the details of the fluid dynamics environ¬ 
ment between the different aircraft sizes, it is known 
that smaller aircrafts are typically built with lower as¬ 
pect ratios, and that the difference in aspect ratio over 
existing aircraft within the size range of interest is sig¬ 
nificant. 


RW-UAS 

For the case of rotorcraft configurations, similar scaling 
laws regarding the vehicle efficiency may be derived. It 
is important to highlight however that especially for ro- 
torcrafts, working with scaling laws demands that one 
has to simultaneously focus on both efficiency and dy¬ 
namic response in order to avoid undesired effects in the 
vehicle flight dynamics such as unstable oscillations. 
Regarding the power efficiency, let power loading (PL) 
be defined as T/P, where P corresponds to the ideal 
power. As the induced ideal power to hover is given by 
P = Tv h, the ideal power loading will be inversely pro¬ 
portional to the induced velocity at the rotor disk uj 


Vh = Vi => 


2p(nR 2 ) 


P 

T 


(PL)- 1 ■ 


( 26 . 2 ) 


Observing Fig. 26.5, it is shown that the ratio T/P 
decreases quickly with increasing disk loading. There¬ 
fore, configurations with proportionally smaller rotors 
against their mass will tend to be less efficient in hov¬ 
ering flight; that is, the rotor will require proportionally 
more power to generate the required amount of thrust. 
It is also to be noted, however, that calculation of the 
actual power loading and rotor efficiency requires the 
consideration of viscous losses. 

From the above brief analysis, we concluded that in 
general the tendency to increase the rotor dimension fa¬ 
vors efficiency. However, this is not the only scaling law 
one has to consider. Rotorcrafts are particularly com- 
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Fig. 26.4 Tennekes size trend relating 
wing loading and cruise speed to 
weight for insects, birds, and manned 
aircraft 


plex dynamic systems and scaling considerations also 
have to focus on dynamic aspects of their flight. A more 
concrete analysis may take place using Froude or Mach 
scaling models. Let N denote the length scale between 
two vehicles, R m the rotor radius of the model vehicle, 
and R p the rotor radius of the prototype vehicle: thus, 
a scale factor N denotes a helicopter 1 /N times the size 
of its prototype. Table 26.2 summarizes the Froude and 
Mach scaling laws that account for the role of scale in 
a set of significant parameters namely the length of the 
model and the prototype L m , L p , the dominant time con¬ 
stants t m , t p of the inner-loop characteristic response. 


the characteristic velocities V m , V p , the weight values 
W m , Wp, the expected moments of inertia 7 m , 7 p , and the 
response-dominant frequencies co m , <y p . 

These, slightly more advanced scaling laws, fur¬ 
ther provide the opportunity to assess the aspects of 
main rotor performance, and more specifically the ex¬ 
pected thrust margin. Traditional manned helicopters 
have small thrust margins in hover, typically 5—10% 
while miniaturized vehicles often present very high val¬ 
ues. Mach models predict in general faster rotor speeds 
as compared to Froude scaled models, which conse¬ 
quently leads to a lower expected thrust coefficient. The 




Flying Robots I 26.3 Basics of Aerodynamics and Flight Mechanics 629 


Hovering efficiency (power loading) (lb hp ') 


10 


6 

4 

2 

0 



1 


10 


100 


1000 


Effective disk loading (lb ft 2 ) 


Fig. 26.5 Hovering efficiency versus 
disk loading for a range of vertical lift 
aircraft 


Table 26.2 Scaling laws for conventional helicopters 


Dimension 

Froude 

Mach 

Length 

Lm = Lf/N 

L m — Lp/N 

Time constant 

tm = tp/V'W 

t m = tp/N 

Speed 

v m = v p /Vn 

& 

ii 

Weight 

Wm = Wp/JV 3 

W m = W p /N 3 

Inertia mat. 

Im=I f /N 5 

/m = I f /N 5 

Frequency 

COjji = (Dp -s/~N 

= (DpN 


thrust coefficient reflects the lift loading of the rotor. For 
a given, single rotor configuration, the maximum thrust 
is provided by the following expression 

Tmax = (c T /CT) max p(^ 2 )(^fl) 2 , (26.3) 

where a represents the blade solidity (Sect. 26.3.4). 
This relation gives a maximum thrust that scales as 
Tmax ot 1 /N 3 for a Froude model and as T mm oc 1 /N 2 
for a Mach model. Once divided by the vehicle weight 
which scales as W oc 1 /IV 3 , it is deduced that Froude 
models present a similar thrust-to-weight ratio. On the 
contrary, for a Mach model, there is an increasing 
expected maximum thrust-to-weight: (7’/VT) max oc N. 
Using these formulas, researchers in [26.8] calculated 


the scaling parameters for several conventional heli¬ 
copters that provide intuitive insight on how scaling 
laws work. 

LtA-UAS 

For the case of lighter-than-air vehicles simple scaling 
laws regarding the efficiency of the system hold. Con¬ 
sidering the example of a spherical blimp, it is directly 
deduced that the lift force scales with the cubic power 
of the radius. On the other hand, its mass, which de¬ 
pends on the surface, scales with the square power of 
the radius and also does the drag force. This essentially 
indicates that larger blimps will tend to have a higher 
maximum lift to weight and lift-against-drag force ra¬ 
tios. 

FI-UAS 

Analysis of the scaling laws for flapping wing systems 
requires a different treatment, as the flight modality 
changes while the robot operates in hover mode or nav¬ 
igates in forward flight. Furthermore, the lift and drag 
coefficients are dependent on the airfoil characteristics 
of the wing and also on the flapping frequency, a fact 
that further increases the complexity of the analysis on 
the effect of scale. Section 26.6 provides insight on how 
to deal with this challenging issue so that proper flap¬ 
ping wing systems design is achieved. 


26.3 Basics of Aerodynamics and Flight Mechanics 

Assembling an analytic representation of a UAS in- namics and appending the resulting effect to the vehicle 
volves the derivation of approximative expressions for body equations of motion. The goal of this section is 
the aerodynamic forces, accounting for the actuator dy- to provide the necessary insight and understanding of 
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Fig. 26.6 Variation of (a) air tempera¬ 
ture, (b) pressure, and (c) density with 
altitude in the lower part of the In¬ 
ternational Standard Atmosphere.The 
tropopause, above which the tem¬ 
perature is not further decreasing, 
corresponds to the red dashed line. It 
constitutes the upper limit of common 
weather phenomena 


the underlying mechanisms and physical phenomena 
along with the derivation of the formulae for the most 
dominant effects one has to account for with any UAS 
configuration. 

26.3.1 Properties of the Atmosphere 


In contrast to aerodynamics, where forces, first 
and foremost lift, is generated by motion of an object 
through the air, the aerostatic lift force is formed solely 
by static properties of an object. It forms the basis of 
operation for a balloon or blimp. 

According to Archimedes’ principle, the aerostatic 
lift L stat pointing upward, amounts to 


Assessing flow properties forms the basis for any fur¬ 
ther qualitative or quantitative aerodynamic analysis 
relevant for aircraft design, modeling and control. The 
international standard atmosphere (ISA) [26.9] pro¬ 
vides a reference for the average main air characteristics 
as a function of altitude. Figure 26.6 shows the evolu¬ 
tion of air temperature T^, pressure p, and density p. 
These parameters largely affect the Reynolds number 
Re, which can be interpreted as the influence of iner¬ 
tial forces as compared to viscous forces of a flow, as 
well as the Mach number Ma representing the ratio of 
airspeed versus speed of sound. 

It is noteworthy that the above parameters may be 
brought into relationship by the ideal gas law 


istat = pVg-mg , (26.5) 

where g stands for the Earth gravitational acceleration, 
V for the volume of the object, and m for its mass. To 
state an example, consider a helium balloon of spher¬ 
ical shape with a diameter amounting to 1 m in the 
lower atmosphere. Neglecting its hull weight (thus with 
m = PheiiumVj), it will generate an aerostatic lift force of 
5.4 N, representing an upper bound for any kind of total 
design mass including payload. Increasing the diameter 
of said sphere to 1.5 m has the huge effect of increasing 
lift to 18.1 N. 


p = pRT m , 


26.3.2 General Fluid Dynamics 
(26.4) and 2-D Flow around Airfoils 


with R = 286.97 m 2 /s 2 /°K denoting the ideal gas con¬ 
stant of air. 



Fig. 26.7 Characteristics of a 2-D flow around an airfoil 1 - Free 
stream velocity field, 2 - Streamline, 3 - Stagnation point, 4 - 
yLaminar boundary layer, 5 - Overpressure, 6 - ySuction, 7 - Tran¬ 
sition point, 8 - Turbulent boundary layer, 9 - Separation point, 
10 - Separated flow 


A general airflow around an aircraft is three- 
dimensional, unsteady, and may be turbulent, even 
interacting with a nonrigid structure. In this setting, 
computations are almost intractable. We will thus rely 
on certain simplifications in order to allow simpler cal¬ 
culations and notably enhancing the understanding of 
a flow field together with resulting forces and moments. 
In both airplane and rotorcraft aerodynamics, the as¬ 
sumption of a locally two-dimensional (2-D) flow can 
be very helpful to serve as a starting point for more ad¬ 
vanced computation. 

Before diving into a formal treatment, the example 
in Fig. 26.7 depicts some characteristic elements of 2-D 
flow around an airfoil. 

Notice that the pressure distribution on the airfoil 
contour is induced by the flow field, making the most 
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significant contribution to the aerodynamic force and 
moment. However, also viscous effects yield a typically 
unwanted share in the overall force (and moment) in 
the form of shear stress transmitted to the surface. The 
elements overviewed in Fig. 26.7 will now be explained 
in the following paragraphs. 

Finite Control Volume Analysis: 

Mass and Momentum Conservation 
Consider a finite control volume B bounded by the 
surface S with normal n, which may contain a body 
or airfoil, depending on the context. For convenience, 
parts of the boundaries are often chosen to be stream¬ 
lines (in 2-D) or stream surfaces (in 3-D). For this 
volume, the conservation of mass must be fulfilled 


// 


pv-ndS , 


s 


(26.6) 


where n denotes the fluid velocity vector. 

From classical Newtonian mechanics, we can fur¬ 
thermore postulate the applicability of conser\’ation of 
linear and angular momentum 


F tot = j^J pv(v-n)dS + JJ pndS 




(26.7) 


M fot — 


JJ p{v xr)(v -n)dS 


+ J t jjj ^ xr ) dS ’ 

B 


(26.8) 


where r denotes the position vector. 

Differential Volume Analysis: 

Euler and Bernoulli Equations 
When applied to a differential volume and assuming 
inviscid flow, (26.7) can be used to derive Euler’s equa¬ 
tion 


— + tr-vjtt + Vp = 0. 


(26.9) 


This equation forms the basis of many finite-element- 
based numerical tools neglecting viscous effects outside 
the boundary layer. Such methods employ potential 
flow theory along with some boundary layer analy¬ 
sis module; free example tools are JavaFoil [26.10] 


and xfoil [26.11] for 2-D flow computation as well as 
XFLR [26.12] allowing also 3-D flow extensions. 

When applying (26.9) along a streamline and under 
the assumption that the flow is incompressible (a fair as¬ 
sumption for low-speed aerodynamics up to Ma = 0.3), 
Bernoulli’s equation relating speed (V t ) and pressure 
can be formulated 

V 2 

p-^- + pgh +p = const, (26.10) 

where g denotes gravitational acceleration and h the el¬ 
evation - the aerostatic pressure component pgh can 
often be neglected due to small elevation changes along 
a streamline. 

Viscous Effects and the Boundary Layer 
While it is often a valid approximation to neglect vis¬ 
cous effects far enough from the body surfaces, they 
have to be considered within the boundary layer, where 
the fluid is slowed down to meet the speed of the sur¬ 
face. The friction shear stress r w transmitted to the 
surface is characterized by the gradient of the flow 
speed perpendicular to the surface 

dU 

7w — ’ (26.11) 

dn 

where /x denotes the dynamic viscosity of the fluid, U 
stands for the airspeed parallel to the surface, and n for 
the coordinate along the surface normal n. This tan¬ 
gential fluid velocity gradient in the boundary layer is 
visualized qualitatively in Fig. 26.7 . 

The boundary layer will be laminar around the nose, 
with the fluid moving parallel to the surface. At some 
point (at a critical local Reynolds number), however, 
influenced by disturbances such as surface roughness, 
a transition to a turbulent boundary layer will occur: it 
is characterized by stochastic fluctuations, significantly 
thicker and producing substantially more friction than 
before the transition. 



Fig. 26.8 Decomposition of the aerodynamic force by 
a 2-D flow around an airfoil: the section lift dZ. denotes the 
component perpendicular to the far-field inflow, and drag 
d D the one parallel to it 
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Fig.26.9a-c SA7036 low-speed airfoil lift (a), drag (b), and moment (c) polars for various Reynolds numbers calculated 
by Javafoil 


Section Lift, Drag, 
and Moment Representation 
with Dimensionless Coefficients 
Historically and for practical reasons, the aerodynamic 
force is split into a component perpendicular to the in¬ 
flow direction called lift, and a second one parallel to the 
inflow called drag. We write 2-D lift, drag, and moment 
as infinitesimal quantities dL, d D, and d M, respec¬ 
tively, as opposed to L, D, and M designating physical 
forces of a whole airplane. Figure 26.8 visualizes these 
quantities. Furthermore, we define the angle of attack a 
as the angle between inflow direction and the chord line 
of length c connecting airfoil leading edge and trailing 
edge. Note that force and moment are reduced to the 
point at 0.25c, i. e., one quarter of the chord behind the 
leading edge. 

Dimension analysis suggests the formulation of 
aerodynamic forces and moments in terms of section 
lift, drag, and moment coefficients ci, Cd, and c m 


1 , 

dL = -pV{ cicdy , 

(26.12) 

dL> = ~pV~ CiCdy , 

(26.13) 

AM = ^pVfc m c 2 dy , 

(26.14) 

where V t stands for the inflow speed and dy denotes an 
infinitesimal length element perpendicular to the 2-D 
flow (which can be interpreted as a length element into 


span-wise direction of an infinitely long wing). 

These coefficients largely depend on the angle of 
attack a; but furthermore, the Reynolds and Mach num¬ 


bers significantly influence them as well. The angle of 
attack dependences are typically given in the form of 
section lift, drag and moment polars, an example of 
which is provided in Fig. 26.9. Note that the drag com¬ 
ponent is originating both from viscous skin friction 
as well as form drag, caused by an asymmetric pres¬ 
sure distribution due to boundary layer development 
and separation. The lift curve shows its characteristic 
linear increase with increasing a for small angles of at¬ 
tack. The maximum and minimum lift values beyond 
which stall is entered are clearly visible in the lift po¬ 
lar. Note that the aerodynamic performance ci/cd of 
the airfoil generally decreases with smaller Reynolds 
numbers as expected. The choice of reference point at 
0.25c typically leads to a mostly constant moment coef¬ 
ficient when varying a, C\, respectively, as can be seen 
in Fig. 26.9 as well. 

Separation and Stall 

At the upper side of the airfoil, the fluid is moving 
from the under-pressure region toward a higher pres¬ 
sure at the trailing edge: the slower moving fluid in the 
boundary layer will at some point not be able to follow 
this adverse pressure gradient, leading to flow separa¬ 
tion. As the angle of attack is increased, the separation 
point suddenly moves far toward the leading edge: this 
condition is referred to as stall, with the catastrophic 
consequence of significant loss of lift and increase of 
drag. Figure 26.10 illustrates the changes in the flow 
and pressure distribution when varying the angle of at¬ 
tack. Note that the maximum lift and stall conditions 
are highly influenced by the choice of airfoil, Reynolds 
number, and Mach number. 
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26.3.3 Wing Aerodynamics 

So far the 2-D flow characteristics around airfoils were 
treated. This will form the basis for the understanding 
and computation of lift and thrust forces generated on 
any type of aircraft; the following treatment of a finite 
wing serves as an important example of how to include 
three-dimensional (3-D) flow effects. 

Recording lift and drag polars for a finite wing 
rather than just for its airfoil reveals less lift increase per 
angle of attack increase, less maximum lift, and higher 
drag at raised angles of attack. These observations are 
related to the concept of induced flow to be treated in 
the following. 

Vortex System of a Wing 

As a direct consequence of lift, we observe a downward 
flow deflection across an airfoil. This is intuitively ex¬ 
plained with conservation of linear momentum as stated 
in (26.7). Assuming an inviscid and incompressible 
fluid, the flow may be modeled with potential field the¬ 
ory [26.13], where the velocity vector field is defined as 
the gradient of a scalar function. This concept allows for 
the insertion of singularities into a free stream, such as 
sources, sinks, and vortices. Figure 26.11 shows a first 
approximation using a single vortex - conceptually il¬ 
lustrating the flow characteristics around a simplified 
wing. The vortex system consists of the bound vortex 
and tip vortices; note that the vortex will in theory have 
to be closed to a ring by a starting vortex. In prac¬ 
tice, i. e., in the presence of friction, the vortices will 
of course decay over time. Figure 26.12 illustrates the 
existence of tip vortices trailing an airplane wing. 

The vortices induce a downwash area behind the 
wing; nevertheless, the trailing vortices will also induce 
some downward flow at the wing. 

Induced Drag 

With the simplified concept of the vortices around 
a wing in mind, we conclude that the wing lift induces 
downward flow, thus reducing the effective angle of 
attack when looking at the 2-D-flow of a wing cross- 
section. Figure 26.13 illustrates this reduction of the 
angle of attack from off (free stream) to a e (effective) by 
the induced angle that is caused by the induced flow 
component W{. Note that this reduction of angle of at¬ 
tack is typically resulting in a smaller lift. Furthermore, 
when decomposing the lift into components parallel and 
perpendicular to the free stream velocity, it becomes ap¬ 
parent that a part dD t of the lift results parallel to the 
effective inflow, thus will contribute to the overall drag 
of the wing. The integral of these components is re¬ 
ferred to as induced drag. The actual amount of induced 
drag largely depends on the wing geometry; a variety of 


a) Small angle of attack 




c) Maximum lift, c Lmax 




Fig.26.10a-d Changes in the flow characteristics with increasing 
angle of attack a. For a small a (a) the example nonsymmetric air¬ 
foil will generate some lift. (b) depicts nominal operation. At some 
a, the maximum lift ci. ma x is reached (c). Beyond that angle of at¬ 
tack, stall occurs (d) 



Fig. 26.11 Simplified representation of the wing vortex 
system: as a consequence of lift, a bound vortex is formed 
along with trailing wingtip vortices inducing downwash 


approaches have been employed in order to minimize 
induced drag, the most popular of which are winglets. 
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Fig. 26.12 Wake vortex study by NASA at Wallops Island: 
the tip vortices are visualized using colored smoke rising 
from the ground 


than just one as introduced qualitatively earlier. Fig¬ 
ure 26.14 depicts the geometry and variables involved. 
Note that the method only provides reliable results, 
if the assumption holds that spanwise flow is negli¬ 
gible; in particular, spanwise variation of parameters 
such as chord length and twist is supposed to be rather 
small. The Kutta-Joukowsky theorem relates circula¬ 
tion and lift; applied to a discrete wing segment, we 
obtain 

A = 2 C <Tu(«eff)Vt , (26.16) 

with the segment (index k) circulation /j., the local 
chord length c k , and the local airfoil lift coefficient C\^. 
The lift coefficient depends on the effective angle of 
attack: a e = ctf — 04 . The induced downwash at posi¬ 
tion mk is obtained by adding the induced speeds of all 
the individual vortices according to Biot-Savart 



Fig. 26.13 Induced drag on a finite wing cross section: the induced 
downwash Wi causes a reduced effective angle of attack. As a con¬ 
sequence, the lift dL contains a component dA parallel to the free 
stream velocity vector 

For an approximately elliptical lift distribution, the in¬ 
duced drag coefficient can be roughly calculated as 


CD.i = 



neA 


(26.15) 


with the aspect ratio A and the Oswald efficiency e (de¬ 
viation from the truly elliptic distribution) amounting 
from 0.7 to 0.85 for typical configuration. 

Lifting Line Method 

In the following, we present one example of how 
to numerically approximate the lift and drag distri¬ 
bution of a wing including induced drag. The lifting 
line method is a 2.5-D (two-and-a-half-dimensional) 
approach in which the induced flow is viewed as gen¬ 
erated by several discrete horseshoe vortices rather 


Wi,k = 


n -\-1 

?r 4*11(0 


Ijew 

— mi) x ey 


x 



(Pj - m k ) - gy \ 

\\(Pj-mi)\\ J 


(26.17) 


where ey stands for the (unit) direction of flight. At m k , 
the induced angle of attack is calculated as 


Wi.k n,k 

ff, 1 = arctan —— ps- 

V t V t 


(26.18) 


Together with the respective 2-D polar data, the 
above relations allow calculating the lift, drag, and 
moment distribution (with respect to the free in¬ 
flow direction ey) from a known circulation distri¬ 
bution, and can be summed and reduced to, e.g., 
the center of mass of a whole airplane. Note that 
the section lift coefficient is often approximated lin¬ 
early as Ci(a) ps ci,o + c\ a a, which allows for a direct 
solution when applying (26.16) and (26.17). More 
accurate results, in particular in the domain near 
maximum lift, however, are obtained by using the 
nonlinear lift polar. In this case, a standard itera¬ 
tive numeric solver may be used. Furthermore, air¬ 
foil data with deflected control surfaces can be in¬ 
cluded, allowing to calculate control moments (and 
forces). 

Note that the above method is just one example of 
numerically solving for the wing characteristics know¬ 
ing the 2-D airfoil properties (polars) - well-suited 
for low-speed medium to high aspect ratio wings. For 
an overview on alternatives, the reader is referred to, 
e.g., [26.13]. 
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Fig. 26.14 k horse shoe vortices 
with circulation placed on the 
wing to model the induced flow. The 
lifting line is imagined through the 
quarter-chord (c*/4) locations. Vortex 
threads with strength Aare leaving 
the wing at the points pk along the 
inflow and induce downwash at the 
locations m j 


26.3.4 Performance of Rotors 
and Propellers 

The propulsion mechanism found on many robotic 
aerial vehicles is commonly a specific configuration of 
propellers or rotors. In the case of a robotic airplane, 
forward facing propellers produce thrust forces com¬ 
pensating drag in forward flight. In case of a tail sitter 
or multicopter UAV, the propellers may be facing up 
(or down) and produce the main lift component com¬ 
pensating the vehicle’s weight allowing it to hover in 
the air. Similarly, the more classic helicopter-type con¬ 
figurations (single rotor with tail rotor, coaxial rotor, 
tandem rotor, etc.), use rotors to generate the required 
thrust force to fly. 

In order to decide on a suitable rotor or propeller 
geometry and to define requirements for the UAV mo¬ 
tor drives, models must be available which allow for 
an assessment of the thrust and torque characteristics 
of a particular rotor or propeller. For this purpose, the 
blade element momentum theory (BEMT) has found 
widespread use, as it often provides a prediction accu¬ 
racy which is acceptable for the UAV design process 
(despite its simplicity). 


propulsion disc. Subsequently, the laws of mass, mo¬ 
mentum and power conservation are formulated across 
the boundaries of the defined control volume. The con¬ 
cept of this propulsion disk as well as the corresponding 
control volume are visualized in Fig. 26.15. 

From this simplistic model, two main conclusions 
may be drawn. First and foremost, it is possible to es¬ 
tablish a relation between the induced velocity i\ at the 
propulsion disk and the produced thrust force T. In nor¬ 
malized form, it can be expressed as 


Ct — 2A;(A; + Aoo) . 


(26.19) 


To simplify notation, the external airflow velocity ttoo 
as well as the induced velocity V\ have been normalized 
with the rotor or propeller tip speed 12 R 



_ ttoo 

~ ~QR ’ 


(26.20) 


and the nondimensional thrust coefficient ex is de¬ 
fined as 


T 

p(jrR 2 )(QR) 2 ' 


(26.21) 


Blade Element Momentum Theory 
One of the basic difficulties in aerodynamic rotor and 
propeller studies is the prediction of the induced inflow 
velocities discussed in Sect. 26.3.3. BEMT addresses 
this problem by combining two simple modeling ap¬ 
proaches, namely momentum theory (MT) and blade 
element theory (BET) which individually cannot di¬ 
rectly resolve this issue in an accurate manner [26.14]. 

The basic idea of momentum theory is to con¬ 
sider the revolving propeller or rotor as a propulsion 
disk which produces a thrust force by accelerating the 
surrounding (incompressible) air mass passing through 
it. A boundary volume is defined encapsulating the 



Fig. 26.15 (a) Side view on the slipstream control volume encom¬ 
passing the MT propulsion disc, (b) Top view on MT propulsion 
disk with incremental annular section and root cutout 
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The parameter p corresponds to the density of air, R 
to the rotor or propeller radius, and Q to the rotor or 
propeller angular speed. 

In case of a robotic airplane, the velocity Vco at 
the start of the control volume corresponds to the body 
forward flight velocity V, and in case of a rotorcraft con¬ 
figuration to the body climb respectively descent rate w. 

Similarly, an incremental expression for the thrust 
coefficient may be found from MT by evaluating the 
mass, moment, and power conservation laws over an 
annular ring of the defined control volume only. The 
corresponding expression can be found as 

dCf T = 4A i (A i + A 00 )7dr, (26.22) 

where r = £ and dr = jf are the normalized radial lo¬ 
cation and the radial increment of the propulsion disk 
annulus. 

Another relevant conclusion that may be drawn 
from MT is that ideally, the induced component v\ of 
the slipstream velocity at the propulsion disk will ac¬ 
celerate to two times its initial value before leaving the 
control volume. In consequence to this acceleration of 
the flow field, the radial slipstream boundary will (in 
the ideal case) contract to half the propulsion disk area 
at the end of the control volume. 

For the BET approach, the modeling process starts 
by investigating the aerodynamic lift and drag forces d L 
and d D on an individual rotor or propeller blade revolv¬ 
ing around its shaft. These lift and drag forces produced 
by each airfoil segment depicted in Fig. 26.16, con¬ 
tribute to the total thrust and torque increments d T and 



Fig. 26.16 (a) Rotor blade revolving around its shaft, 
(b) Blade element of a revolving rotor blade 


d Q of the respective rotor or propeller annular section. 
The corresponding relation can be established as 

dT = N h (dL-0dD)^N b dL, (26.23) 

dQ = N b r(dD+0dL) . (26.24) 

In this context, N b represents the number of rotor or 
propeller blades and 0 corresponds to the local inflow 
angle which is assumed to remain small. Under this as¬ 
sumption, the inflow angle 0 can be directly derived as 
the ratio between the local perpendicular inflow veloc¬ 
ity Up Vi + ifeo and the tangential velocity Uj ss Qr 
visualized in Fig. 26.16b. Additionally, the assumption 
introduced in (26.23) is justified by the fact that at low 
angles of attack a, the drag forces d D are at least one 
order of magnitude smaller than the corresponding lift 
forces dL. 

Based on (26.23) and the definition of the lift incre¬ 
ment (26.12), the local thrust coefficient at each radial 
blade station r may be derived as 

dC? E1 = -ac\r 2 dr, with 0 = —— . (26.25) 

T 2 nR 

The parameter c corresponds to the local blade chord 
and a is the so-called rotor or propeller solidity. The 
solidity is a rough metric representing how much of 
a propulsion disk is covered by rotor or propeller 
blades. The aerodynamic parameter ci = Ci(a, Re. Ma) 
corresponds to the airfoil lift coefficient in function of 
the local angle of attack a and the local Reynolds and 
Mach numbers Re and Ma. 

Accordingly, as established in (26.25), the thrust 
produced by a rotor or propeller strongly depends on 
the angle of attack a, which itself is a function of the 
local airfoil pitch angle 6 and the inflow angle 0 

oi = e-0^Q- — . (26.26) 

Uj 

In conclusion, MT as well as BET are capable of 
establishing a meaningful relation between the induced 
velocities Vi and the resulting thrust force T. How¬ 
ever, none of the two theories are capable of accurately 
predicting rotor or propeller performance as either the 
radial distribution of the induced velocity or the radial 
distribution of the thrust coefficient must be known to 
compute the other. 

The basic idea behind BEMT is to combine the 
thrust expression (26.22) resulting from MT with the 
thrust expression (26.25) from BET to compute the in¬ 
duced inflow velocity independently of the thrust force. 
Different BEMT implementations are possible depend¬ 
ing on how willing one may be to introduce further 
assumptions for the section lift coefficient c\. 














Flying Robots I 26.3 Basics of Aerodynamics and Flight Mechanics 637 


Reference [26.14] presents a straightforward ap¬ 
proach tailored toward helicopter rotors operating bel¬ 
low stall by introducing a linear model for cj in function 
of the angle of attack 

ci = cna + C 10 . (26.27) 


The parameters Cdo, Cdi, and c'd 2 can be computed from 
the drag polars of the modeled airfoil. 

Consequently, the thrust and torque increments may 
be integrated along the radial direction of the rotor or 
propeller disk to compute the total thrust and torque co¬ 
efficients 


The parameters cio and cn can be computed from the 
lift polars of a particular airfoil geometry for a given 
range of angles of attack, Reynolds, and Mach numbers. 
This linear approximation may have limited validity 
for very low Reynolds numbers and strongly cambered 
airfoils but is in general acceptable for many typical 
airfoil geometries found on UAV rotors and propellers. 
Assembling (26.22) and (26.25) under the assumption 
(26.27), an algebraic expression of the radial induced 
inflow distribution can be derived as 

Ai(r,Aco) = s/A 2 +B-A, (26.28) 


A = 


11 Aoo 
16 2 ~ 
— OC\\ Aqo 

A =l6 + ^- 

B=75Vf. 


Note that the lift-curve offset cjo has been absorbed in 
the virtual pitch angle 

e r = e+ — , (26.29) 

Cll 

to simplify the notation. 

Once the approximate radial distribution of inflow 
velocities has been found, the local rotor or propeller 
thrust increments (26.25) can be computed. Similarly, 
the rotor or propeller torque increments derived from 
BET as 

dc-Q ET = dcQi + dcQo , (26.30) 

dc Q i = -CTCi(Ai +Aoo)r 3 dr , (26.31) 

dcQo = -<7C d r 3 dr , (26.32) 

can be evaluated based on the inflow distribution given 
in (26.28). For clarity, the total torque coefficient in¬ 
crement has been separated into its induced component 
dcQi originating from the lift forces and its profile com¬ 
ponent dcQo due the drag forces. The aerodynamic drag 
coefficient Cd = Cd(a, Re, Ma) can be approximated us¬ 
ing a quadratic function in dependency of the angle of 
attack [26.14] 

c d = c d2 a 2 + Cdia + C d0 . (26.33) 


R 

/ j BET 

d c T , 

0 

R 

05 = / d<T. 

0 


(26.34) 

(26.35) 


These thrust and torque integrals are usually evaluated 
numerically, as the blade pitch 6 = 9(r) as well as the 
blade chord c = c(r) may be nonlinear functions of the 
radial direction r (blade twist and taper). 

Finally, note that the presented theory may be ex¬ 
tended to provide performance estimates under lateral 
inflow velocities such as in case of a rotorcraft in for¬ 
ward flight and may also be used to asses other types 
of rotor or propeller configurations such as, e.g., the 
coaxial rotor. Also note that the prediction accuracy of 
BEMT tools can be further improved by accounting for 
tip-loss effects and the nonlift producing rotor or pro¬ 
peller hub, e.g., using the Prandtl tip-loss function also 
presented in [26.14] and incorporating a root cutout ra¬ 
dius Rq. 

The resulting predictions are generally in good 
agreement with experimental data - nevertheless, an ex¬ 
perimental verification is strongly recommended. 


26.3.5 Drag 


The sources of drag on aircraft are manifold: histori¬ 
cally, the distinction between the lift-dependent induced 
drag and parasite drag is made. The latter is further 
subdivided into skin friction drag due to viscous shear 
stress at the surface, and form drag, generated by pres¬ 
sure loss along bodies (i. e., due to boundary layer 
development or even flow separation). Both these com¬ 
ponents contribute to the airfoil profile drag, i. e., the 
section drag coefficient Cd introduced before. 

On a whole aircraft, many more drag sources are 
distinguished. For an exhaustive overview, the inter¬ 
ested reader is referred to [26.13]. In the following, we 
will present an overview of drag generated by different 
typical shapes; note that when simply summing drag of 
different shapes associated with aircraft parts, the result 
may be a helpful initial estimate, but can be inaccurate, 
because of neglecting the interaction of flows resulting 
in interference drag. Depending on the stage of the de¬ 
sign process or the desired modeling accuracy, 2.5-D 


Part B | 26.3 




Part B | 26.3 


638 Part B I Design 


computations or even full 3-D computational fluid dy¬ 
namics (CFD) simulations might be necessary to satisfy 
the needs of aerodynamics calculations. 


Skin Friction 

The simple but important example of a flat plate of 
length l in parallel flow is well studied. As introduced in 
the airfoil theory Sect. 26.3.2, the boundary that devel¬ 
ops will be laminar near the leading edge and transitions 
into a turbulent one, generating more drag, at some 
point downstream. The friction coefficient is defined as 


2D{ 

pV?S w ’ 


(26.36) 


with the wetted surface S w and the friction drag force 
Df. According to [26.13], the coefficients can be ap¬ 
proximated by 

Laminar: Cf = 1.328Re~ 0 ' 5 , (26.37) 

Turbulent: Cf = O.455(log 10 Rei)^ 2 ' 58 . (26.38) 


Note that the point of transition is depending on the lo¬ 
cal Reynolds number Re v = pV t x/ /r, where x denotes 
the coordinate along the flow from the leading edge 
of the plate. Depending on the surface roughness and 
ambient turbulence, the critical (transition) Reynolds 
number varies; as an average guess for a flat plate, it 
will be in the order of Re VjCr j t = 3 xlO 5 . 

Drag Coefficients for Selected Bodies 
In the following, drag coefficients for a selection of 
2-D and 3-D bodies of rotation are given, obtained 


Table 26.3 Bodies the drag coefficients of which are 
largely Reynolds number independent 



from [26.13]. Table 26.3 overviews a category of bod¬ 
ies, the drag coefficients of which are largely indifferent 
to the Reynolds number, owing to their geometrically 
defined (sharp edge) flow separation point. 

Rounder bodies, most prominently the cylinder in 
cross-flow or the sphere, however, show a distinc¬ 
tively different behavior quantified in Table 26.4: below 
a critical Reynolds number Re cr it ss 4 xlO 5 , the drag co¬ 
efficient is significantly higher, where separation occurs 
before boundary layer transition. In contrast, above the 
critical Reynolds number, the turbulent, more energetic 
boundary layer separates only further downstream, re¬ 
ducing the wake and thus the amount of form drag. 

A third important object category is formed by 
streamlined and fuselage-like bodies: due to their com¬ 
parably high skin friction part, the fineness ratio largely 
influences the drag coefficient (along with the Reynolds 
number). The fineness is defined as body length divided 
by body diameter. We introduce a volumetric drag coef¬ 
ficient as CD m = 2D/(pV 2 V 2/3 ) with the body volume 
V m . Interestingly, this is minimal and approximately 
constant at fineness ratios between 4 and 10, which pro¬ 
vides a range for optimal sizing of such a body when 
a certain volume needs to be fitted. The values are given 
in Table 26.5. 

26.3.6 Aircraft Dynamics 

and Flight Performance Analysis 

Sections 26.3.1-26.3.5 shortly presented the basic the¬ 
ory of wings, rotors, and propellers as well as a few 
tools to assess their respective aerodynamic perfor¬ 
mance. To develop fully functional UAV platforms, 
merely evaluating the individual flight mechanisms is 
a good starting point but generally not sufficient. 

Designing high-performance aircraft systems re¬ 
quires a fundamental understanding of how the respec¬ 
tive design parameters affect the full flight dynamic 
response and application-specific capabilities. In order 
to comprehend how design changes affect an aerial 

Table 26.4 Sphere and cylinder drag coefficients 



Table 26.5 Volumetric drag coefficients for fineness 4—10 
(largely turbulent boundary layer) 


Fuselages and nacelles 

CDm ^ 0.027 

Streamlined bodies 

CDm ~ 0.024 
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Fig. 26.17 Coordinate frames with 
external forces and moments of 
rotorcraft and fixed-wing UAVs 


robotic system, representative models of its flight dy¬ 
namics are required. Such models must be capable of 
capturing the dominant system dynamics within the 
relevant part of the flight envelope. Furthermore, espe¬ 
cially in an interdisciplinary field such as robotics, these 
models must be accessible to the nonaerodynamic ex¬ 
pert (the roboticist) and thus need to be simple enough 
to provide the required insight for the aircraft design 
process. 

As many other types of robots, robotic flight plat¬ 
forms may be treated as a multibody system where 
a set of interlinked bodies exchanges kinetic and poten¬ 
tial energy under the influence of external forces and 
moments. For aircraft systems it is common to treat 
the entire aircraft as a single rigid body first, with re¬ 
lated body coordinate frame attached, as visualized in 
Fig. 26.17. Additional dynamics such as for example 
rotor flapping (as in case of a helicopter system) may 
be appended to these body dynamics in a subsequent 
step. 

The modeling process thus starts by treating the 
aircraft system as a rigid body affected by external 
forces F and external moments r. Using the Newton- 
Euler formalism to derive the aircraft body dynamics, 
one can directly write down the linear and angular mo¬ 
mentum balance for a single rigid body 


The most relevant contribution to the forces B F 
and the moments b t originates from the aerodynamic 
flight components such as wings, propellers and ro¬ 
tors. By integrating (26.39) over time one may compute 
a prediction of the aircraft’s dynamic response to these 
external forces and moments and thus the evolution of 
its absolute pose. This pose is commonly represented 
by the position of the vehicle’s center of gravity w r B = 
(. x , y, z) T as well as the vehicle’s orientation relative to 
an earth fixed world frame W which is considered in¬ 
ertial. The aircraft orientation is commonly represented 
using rotation matrices or quaternions. In the case of the 
rotation matrix representation, the aircraft orientation 
may be parameterized in three dimensional space by 
three consecutive rotations with the roll, pitch, and yaw 
angles ip e [—jr, 7r], 9 e [— n/2, rr/2] and 1 jr e [— n, it] 
as 

W Rb = RzW)R Y (6)R x (cp ) . (26.40) 

In this case, relations between the body frame veloc¬ 
ities B v and B u> and the world frame pose can be 
expressed as 

w r= w R B B v, 

W Rb = W RbI B <o] x , (26.41) 


m{ B i) + B a> x B v) = b F , 

B l B d)+ B co x( B l B co) = b t . (26.39) 

For simplicity’s sake, (26.39) is usually expressed with 
respect to a body fixed frame B located in the center 
of gravity of the aircraft. The velocity vectors B v = 
( u,v,w) T and B co = (p,q,r) r thus represent the air¬ 
craft linear and angular velocities with respect to B. The 
inertial properties of the above body dynamics are de¬ 
fined by the aircraft’s total mass m and its second mass 
moment of inertia B I also expressed with respect to B 
and its origin. 


where [ B <n] x corresponds to the skew-symmetric matrix 
of the vector B u>. 

In a minimal form, the orientation dynamics can be 
expressed in terms of roll, pitch, and yaw angles 


( \ 


( 4 >\ 


1 

sin ip tan 9 

cos <p tan 9 

6 

= 

0 

cosip 

— sinip 



0 

sin ip /cos 9 

cos cp/ cos 9 



(26.42) 
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Note that the Jacobian J, becomes singular at the 
boundaries of 9 = ±n/2. 

As a singularity-free, but still compact represen¬ 
tation of orientation, quaternions may be used. Using 
the representation w q B = (q w , q x , q y , q z ) T with real part 
q w , the rotational kinematics become 

w q B = Ut( B co) w q B , (26.43) 

with the matrix ft defined as 

,, / 0 V\ 

B< ">=(-■» [■.]>)■ 126441 

In summary (using quaternions), we thus have the 
equations of motion 

w r= w R ls B v, 

w q B = X -Sl( B u) w q B , 

B i, = - B F- B (ox B v , 
m 

"ii = *r 1 fi-*wx(®I B a)]. (26.45) 

Note that the external forces and moments are re¬ 
lated to the system’s actuator inputs u, the vehicle’s 
orientation with respect to W and they typically are also 
functions of the linear and angular body motion as well 
as additional dynamics terms, represented here by the 
vector e, 

b F = B F( w r, w q B , B v, B co, e r , u) , (26.46) 

b t = B r( w r, w q B , B v, B (o,e r ,u ) . (26.47) 

The additional dynamics e r may account for structural 
dynamics such as rotor flapping in case of a robotic he¬ 
licopter or relevant actuator dynamics. 

The resulting nonlinear system dynamics can usu¬ 
ally be cast into state-space form and represented as 

x =f(x,u) , 

X = (x h ,x r ) J , 

u = (mi, ..., un) t , (26.48) 

where the nonlinear functions / define the rate of 
change of the aircraft body states Xf, as well as the addi¬ 
tional states x, affected by the set of N actuator inputs 
u\ to i4y 

In order to gain a deeper understanding of how 
(26.48) is affected by changes of the flight system’s 
geometric, structural, inertial, and aerodynamic pa¬ 
rameters, three main problems are commonly of rele¬ 
vance [26.15]: 


• The trim problem deals with the computation of the 
set of actuator inputs u = «o under which the non¬ 
linear dynamic system presented in (26.48) remains 
in a desired trim point x = xq and thus f(x 0 , «o) = 
0. The most simple example of such a trim point X(, 
is the hover condition for a rotorcraft system where 
one may want to find the required rotor speed I2o 
to hover or the steady forward flight condition for 
a fixed-wing UAV at a forward velocity V,. 

• The topic of stability deals with the question of 
how easily the system (26.48) will deteriorate from 
a specific trim condition (xq, «o) under the influence 
of small disturbances Ajc and Ah. This investiga¬ 
tion commonly involves the linearization of (26.48) 
according to 



Ax=AAx + BAu, (26.49) 

where the eigenvalues and vectors of A will pro¬ 
vide deeper insight into the motion characteristics 
and stability properties of an aircraft. 

• Analyzing the System Response 

t 

x{t) = x(0) + J xdt (26.50) 

0 

to characteristic inputs such as steps, pulses or 
specific input frequencies will provide additional 
information about the flight characteristics of a spe¬ 
cific aircraft configuration. 

These modeling and analysis concepts commonly 
find wide applicability for various types of robotic flight 
configurations and will be discussed in more detail for 
the specific UAV types presented hereafter. 

Actuator Dynamics 

Deriving the aerodynamic forces is combined with the 
dynamic equations of motion in order to assemble 
a complete model of the flying vehicle. However, as the 
employed actuators are of naturally limited bandwidth, 
accurate modeling furthermore requires the integration 
of the relevant motor or servo dynamics. 

Nowadays, motors utilized in small size unmanned 
systems often belong to the category of brushless direct- 
current (DC) electric motors (BLDC). BLDCs are syn¬ 
chronous motors powered by a DC electric source via 
an integrated switching power supply. The equations of 
motion for a such a system are essentially nonlinear and 
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rather complex. However, working with small UAS, we 
may solely focus on the input-output dynamics which 
can be described with the following transfer function 

Ato(t) __ -K m (l + r a s) _ 

A Q m (s) (1 + r m s)(l + r a s) + K m K a (Ki e0 ) 2 ' 

(26.51) 

where Aa>(s), A<2 m (s) correspond to the Laplace ex¬ 
pressions of the linearized angular velocity and input 
torque, K m is the mechanical gain, r m represents the 
mechanical time constant, K a is the rotor gain, r a is 
the rotor time constant, K depends on electromagnetic 
properties of the motor and 4o denotes the stator current 
linearization point [26.16]. Often, a satisfactory speed 
controller and BLDC dynamics description is obtained 
as the relation between a reference angular velocity 


and the actual output taking the even simpler first-order 
form 


Atu r (5) 1 + x mc s ’ 

with the time constant r mc of the controlled motor. 

Accounting for motor dynamics is essential for 
high-bandwidth control of agile vehicles that highly de¬ 
pend on such actuators (i. e., multirotors). However, in 
several other UAS configurations such as fixed-wing 
vehicles or conventional helicopters may, if needed, 
rather account for servo dynamics acting on control sur¬ 
faces or a swashplate. Again, the relevant servo angle 
dynamics can be captured by an identified first-order 
transfer function of the form 1/(1 + t s s), with the servo 
time constant r s . 


26.4 Airplane Modeling and Design 

Ever since the beginning of aviation, a broad spectrum 
of airplanes has been built and operated successfully: 
size, speed, and maneuverability vary widely and as 
a function of application. Since design and modeling 
are strongly related, we want to first give an overview 
of the physical principles common to all such config¬ 
urations, and provide analysis tools for characterizing 
static and dynamic properties of an airplane. The design 
problem somewhat constitutes the inverse problem: 
for specified target characteristics, the engineer needs 
to find a suitable configuration; we therefore provide 
a summary of design guidelines aimed at fast con¬ 
vergence to a suitable design. Finally, a simple and 
classical autopilot scheme is presented underlining the 
need for models also at that stage. 

26.4.1 Forces and Moments 

Consider Fig. 26.18 for the introduction of airplane 
geometry definitions and main forces. Forces and mo¬ 
ments are reduced to the airplane center of gravity 
(COG). Note that the angle of attack (AOA) a is de¬ 
fined as the angle between the x-axis and the true 
airspeed vector v t projected into the body x-z-plane, fi 
denotes the sideslip angle, causing a typically unwanted 
sideslip force Y, L, and D denote lift and drag, W stands 
for the weight and T for thrust, which may act into 
a direction different from x (at a thrust angle ex)- We 
furthermore write the aerodynamic moment vector as 
b Ta = [Fa, Ma, Na] t ■ Also note the introduction of the 
main control surfaces that are designed to mainly influ¬ 
ence the aerodynamic moment: with ailerons, elevator. 


and rudder, the roll (La), pitch ( Ma ), and yaw (Na) mo¬ 
ments are controlled. The indicated flaps, if available, 
are used for increasing lift for take-off and landing, in 
order to achieve a slower minimum speed. 

Aerodynamic Forces and Moments 
The aerodynamic forces and moments can be modeled 
to various accuracy using full 3-D CFD or with 2.5-D 
tools: Sect. 26.3.3 overviews such an approach which 
can be used to model the aerodynamic surfaces in incom¬ 
pressible flow. For enhanced accuracy, fuselages may be 



Fig. 26.18 Geometric definitions and main forces acting on the air¬ 
plane (general case, not in equilibrium) 
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considered using again a combination of potential flow 
(placing singularities) and boundary layer theory. Re¬ 
spective ready-to-use software such as AVL [26.17] and 
XFLR [26.12] is available for free. 

The forces and moments may again be written with 
dimensionless coefficients as 


forces and moments to be opposing a disturbance. We 
assume stationary conditions in the sense of constant 
linear and angular speeds: the respective force and mo¬ 
ment balance is typically straightforward to apply in 
order to determine the starting point of the stability 
analysis. 


L= '-pV?c L A, 

(26.53) 

D = \pV?c d A , 

(26.54) 

1 9 

M a = -pV t c M cA, 

(26.55) 


with the wing area A, the mean chord length c, and the 
true airspeed V t = ||u t ||. The moments L A and N A are 
made dimension-less with the wingspan b rather than 
the chord length. 

Static Performance Considerations 
Having characterized lift and drag of an airplane, three 
operating points are of particular interest. 

First, stall is occurring at CL.max- This condition can 
be directly translated into constant-speed level-flight 
stall speed by applying the lift balance L = mg. 

Second, the maximum cl/cd ratio, or the glide ra¬ 
tio characterizes the maximum aerodynamic efficiency, 
i. e., the operating point for maximum range (assuming 
constant propulsive efficiency). 

Finally, the maximum c^/c^ ratio, or the climb fac¬ 
tor describes the condition at which power consumption 
is minimized, thus maximizing flight time (again as¬ 
suming constant propulsion unit efficiency). 

The latter two conditions have direct interpretation 
in gliding (or propulsion shut-off), in terms of maxi¬ 
mum distance reached per altitude lost and minimum 
sink rate, respectively. Again, corresponding velocities 
can be found using the lift balance. 

Thrust 

For detailed insight into the variety of propulsion sys¬ 
tems and respective models, the interested reader is 
referred to [26.13,18]. As an approximation for the 
important case of a propeller, the BEMT method as 
described in Sect. 26.3.4 is suggested. For many ap¬ 
plications, choosing the propeller speed as the system 
input and neglecting motor dynamics is sufficient. 

26.4.2 Static Stability 

Various forms of stability constitute central character¬ 
istics of an airplane related to whether or not it can 
be flown by a human pilot or flight controller. Simple 
stability criteria can be derived by requiring reaction 


Longitudinal Static Stability 
We will take a close look at the example of longitudinal 
static stability playing a central role in airplane analy¬ 
sis and design. Leaving aside possible influence of the 
propulsion unit, the respective directional stability cri¬ 
terion is stated as 


dc M 
da 


< 0 , 


(26.56) 


at the equilibrium condition cm = 0. Figure 26.19 il¬ 
lustrates an exemplary moment coefficient as a func¬ 
tion of AOA. Note that elevator actuation will move 
this curve up and down, and with it the equilibrium 
point (2) toward higher or lower angles of attack (i. e., 
lower or higher trimmed speeds). Figure 26.20 illus¬ 
trates the forces and moments in the stable equilibrium 



Fig. 26.19 Moment coefficient cm as a function of AOA a 
the equilibrium point (2) is stable for dcyi/da < 0 (brown 
curve ) 



Fig. 26.20 Forces and moments at the main wing and tail 
for zero lift (1), the stable equilibrium AOA (2), and for 
high lift (3). The forces and moments are drawn into the 
individual surfaces’ aerodynamic centers. Note the tilted 
inflow at the tail due to downwash. The green filled circle 
denotes the overall airplane aerodynamic center (AC) 
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with a simplified airplane side-view as compared to the 
points (1) and (3), i. e., zero-lift (a = 0) and high-lift, 
respectively. The stability criterion can be equivalently 
stated as: the airplane COG needs to be in front of the 
airplane aerodynamic center. Note the main parameters 
that influence the stability are tail lever arm, tail area, 
the longitudinal dihedral Ae (Fig. 26.20 for its geomet¬ 
ric definition), and the COG location along the jc-axis. 

26.4.3 Dynamic Model 


While some core characteristics such as static stability 
and performance measures may already be established 
using aerodynamics coefficients only, we now turn to 
analyze the dynamics, since they provides a much richer 
insight into airplane characteristics. 

For application of the 6 -D (six-dimensional) rigid 
body dynamics (26.45), the forces and moments from 
the various sources need to be assembled and repre¬ 
sented in the body frame 


b F = 


f L sin a — D cos a + T cos ex ^ 
Y 

^—L cos a — D sin a + T sin 6xy 

f La + Lj ^ 

T A/t , 

\Na+ Nj / 


+ B W, 


(26.57) 

(26.58) 


with the weight in body coordinates = 
0, — mg] T , and where the T-subscript indi¬ 
cates (possible) moment components from thrust. Note 
that the system inputs u are hidden inside these forces 
and moments. Also be aware of a and ft containing 
parts of the state vector 


a = arctan 2 («it, « t ), ( 26 . 59 ) 

/l = arcsin(u t /V , t) , (26.60) 


where the true airspeed components are used 


u t 


v, : = 


= B v — b R w w w 


(26.61) 


Parametric Force and Moment Models 
Let us consider the example of a simple airplane con¬ 
figuration with ailerons, a rudder and an elevator plus 
a propeller, driven by an electric motor at rotation speed 
COp. 

We define the system input vector u = [<5 a , S e , <5 r , <5x] 
as normalized aileron, rudder, and elevator action, 
<5 a , <5 e , 8 r G [— 1, 1] as well as (normalized) thrust (5x g 
[ 0 , 1 ], 

The fully parametric nonlinear model provided be¬ 
low largely follows [26.19]. We approximate the lift, 
drag as well as sideslip coefficients with polynomials in 
ot and ft 

cl ~ cl.o + cl,q.o: + cl.ckiW + cl.^Q ! 3 , 

CD ~ Cd,0 + C£) :Ce a + Cu,a20t~ + cd,P2^ 2 > 

Cy^Cy.pP. (26.63) 

For many applications except slow flying airplanes, the 
second-order and third-order term of cl can be omitted. 

As far as the torques are concerned, we introduce 
also dependencies on normalized angular rates 


— (/ 2 n, 4 n• C n ) 


pb qc rb \ T 
2Vt’ 2Vt’ 2 VJ 


(26.64) 


A suitable approximation of the moment coefficients is 
now made as 


ci « ci,o + ci'8 a 8 a + ci,ygj8 -I- ci,pjj n + ci, rn r n , 

Cm ^ Cm.O + Cm ,& e 8e + Cm.ck® + CM.^n^n , 

Cn » Cn.O + CN,S,5 r + Cn,/ 3^ + CN,r n C n • 

(26.65) 

Finally, the propeller thrust force needs to be mod¬ 
eled. Using the advance ratio J = 7 - 7 , with propeller 
diameter d, we can approximate the thrust coefficient 

ex ~ cx,o + Cx ,jJ + Cx,/ 2 -f 2 • (26.66) 

The thrust is then obtained as 


w t 


with the wind vector w w. 

Furthermore, due to the airplane symmetry plane, 
the inertia matrix becomes 


(L x 0 I X7 \ 

0 1 yy 0 

\hz 0 i zz J 


(26.62) 


7 ’=p(^-)“r/ 4 c T . (26.67) 

Linearized Dynamics 

As common throughout literature, the linearized air¬ 
plane dynamics are written using Euler angles, which 
is why we will follow the same approach. But concep¬ 
tually, they could be written in a singularity-free form 
using a minimal quaternion perturbation. 
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Typically, a separation into longitudinal and lateral 
dynamics is made, in order to assess related character¬ 
istics separately. Furthermore, the state is transformed 
to contain a, p, and V t rather than B v. 

The linear dynamics around a reference state xo and 
input uq vector takes the form 

A-t-lon — A |on AX] on T ^lon AW] on 


and 


Axi a t — A ]at Ax| a [ T Ah | a | . 


The following formulation follows [26. 19] to a large ex¬ 
tent. 

Figure 26.21 describes the separation in terms of in¬ 
puts Ah and states Ax for the linearized system. 

The longitudinal nonlinear equations are given as 

k = Wa + M T - (/« - I zz )pr + I xz (p 1 - r 2 )] , 

lyy 

V t = — [— D cos P + Y sin P 
m 

+ T cos(a — ex) cos p + mg 1 ] , 

a = —- 1 — (—L — T sin(a — e T ) + mg 3 ) + q A 
cos p [mV t 

9 = q cos tp — r sin tp , 

(26.68) 



Fig.26.21a,b Linearized (a) longitudinal, and (b) lateral 
plants for inputs Ah (around Ho) of the local states Ax 
(around xo) 


a) Longitudinal poles b) Lateral poles 



Fig. 26.22 (a) Longitudinal, and (b) lateral poles of an example aer¬ 
obatic RC airplane 


and the lateral nonlinear equations amount to 

4(L a +L t - 7p) I xz (N a +N t - T r ) 


P = 


I I —I 2 

1 xx 1 zz *xz 


I I —I 2 

1 XX 1 ZZ 1 XZ 


. I xz (L a + Lj —T p ) I u (N a + N t - T r ) 
r = -—-^-h 


I I -I 2 

1 xx 1 zz 1 xz 

1 


/7_/2 

1 xx 1 zz L xz 


P = —r A - [Y cos p + D sin P 

mV t 

— T cos(ci! — 6 t ) sin P + /wgj] , 

(p = p + q sin tp tan 6 + r cos tp tan 6 . 


(26.69) 


where the following terms were used 


gi = g(— cos a cos P sin 6 + sin P sin tp cos 6 
+ sin a cos P cos tp cos 9) , 

82 = g(cos a sin P sin 6 + cos P sin tp cos 6 
— sin a sin P cos tp cos 6) , 
g 3 = g(sin a sin 6 + cos a cos tp cos 9) , 
q A = qcos P — p sin P cos a — r sin a sin P , 
r A = r cos a — p sin a , 

Tp = ( l zz — Iyy)qr + I xz pq ■ 

Tr — {lyy Ixx)qp L .q * • 

(26.70) 

The linearizations of (26.68) and (26.69) are 
straightforward to obtain and not provided here due to 
space constraints. For a specific operating point, typ¬ 
ically stationary (tpo = 0 , 9q = ffo, an d B u> = 0 ), the 
standard tools of linear systems analysis can be em¬ 
ployed. Most importantly, the pole locations in the 
imaginary plane will tell the characteristic modes and 
their dynamic stability. Figure 26.22 shows a pole loca¬ 
tion plot for an example RC airplane and introduces the 
related mode names. 

In the case of a real pole jr,-, it has the time constant 
r i = — 1/Re(ff,). In the case of a complex conjugate 
pole pair, it is associated with a damping ratio = 
—Re( 7 r;)/^/lm( 7 r ;) 2 + Re( 7 r ,) 2 and with an eigenfre- 
quency &), = rJ\m{jti) 2 +Re(^;) 2 . 

Figure 26.23 illustrates and characterizes the main 
modes. 

26.4.4 Design Guidelines 

Airplane design typically consists of the three stages 
conceptual design, preliminary design, and detail de¬ 
sign. Here, we focus on the first two phases; due to 
space constraints, details of airplane structural design 
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and analysis are not covered here. The reader is re¬ 
ferred to respective literature, e.g., [26.20], or [26.21], 
the latter covering RC-type aircraft. In the following, 
we provide a quick overview of practical guidelines for 
the typically iterative design process related to achiev¬ 
ing characteristics as described above. The guidelines 
follow largely [26.22] and [26.18], with focus on slow- 
flying small-scale UAS. 

Sizing and Geometry of Main Components 
In the following, we provide some rules of thumb as 
initial guess for the design process in terms of sizing 
the wing, tail, control surfaces, and the propulsion unit. 
As a first and very general advice, the engineer is en¬ 
couraged to minimize wetted area and cross section, as 
well as any kind of nacelles for increased aerodynamic 
efficiency. 

Wing. First, an existing airfoil shall be chosen with 
characteristics meeting the requirements in the target 
flow regime (Re and Ma). When it comes to deter¬ 
mining the overall wing size, a first estimate of design 
weight including structure, avionics, payload, propul¬ 
sion unit, and energy storage is of crucial importance 
(26.2). With the target speed V r and design lift coeffi¬ 
cient ci, a rough guess can be made for the wing area 

A = 2mg/(pV?c,) . 


lifting surfaces), and, at low Ma, no sweep-back - as 
long as still implementable with a structural concept 
and staying at reasonably high Re numbers. For high 
efficiency, it is advisable to achieve an elliptic lift distri¬ 
bution to some extent by geometry. For benign stalling 
characteristics, it is furthermore highly advisable to 
twist the wing leading edge downward with increased 
spanwise distance (which also influences the lift distri¬ 
bution). 

Finally, some dihedral should be considered for roll 
stability. 

Tail. Various types of tails and even exotic configura¬ 
tions like the canard have been suggested; here, we want 
to simply point out the importance of the so-called tail 
volume coefficient, cy t and cht concerning vertical and 
horizontal tail, respectively, 

c'vt = hr Ayr/ (M), (26.71) 

cht = /htAht/(cA) , (26.72) 

with the wing to vertical tail lever arm /vt and the 
wing to horizontal tail /ht (these are referenced to the 
individual mean 1 /4-chord points). Typical values for 
small-size slow airplanes are cvt=0.02-0.04 and cht = 
0.5-0.7. Furthermore, care should be taken that control 
surfaces are not completely blanketed in the case of stall 
(for stall/spin recovery). 


Concerning wing shape, clearly highest efficiency 
is reached with high aspect ratios (plus no multiple 


a) Phugoid: exchange of potential and kinetic energy 



b) Short period: AoA oscillation 



c) Dutch roll: combined 
yaw/roll oscillation 



d) Spiral: unstable 
(divergent) example 




Fig.26.23a-d Characteristic trajectories of excited longi¬ 
tudinal modes (a,b) and lateral modes (c,d) 


Control Surfaces. Ailerons typically extend from 
around 50% in span direction to 90%; in this setting, 
20—30% of wing chord is suggested as aileron depth. 
Tail control surface depth is typically chosen around 
40% of the respective chord. 

Propulsion. Finally, some advice is given concern¬ 
ing the propulsion unit. Some UAVs are required to 
be handlaunched: note that this imposes limits on the 
overall maximum take-off mass and minimum/stall air¬ 
speed. Experience shows that reasonable limits are 
<9 m/s minimum/stall speed and 7 kg airplane mass. 
For such small UAVs, a static thrust to weight ratio of 
at least 50% is highly recommended. In general, the 
propulsion unit must be sized to meet the specifications 
in terms of climb rates, maximum level flight speed and 
service ceiling. For the highest efficiency, the propul¬ 
sion unit should be designed such as to provide highest 
efficiency at the design operating point (subscript r) 
T r /W = CD,r/cL,r- For a hobbyist brushless DC outrun¬ 
ner type motor, the maximum power per motor mass 
ratio of 3.4kW/kg can be used for estimation of the 
propulsion unit weight [26.23] (gearbox and propeller 
mass not included). 
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Handling Qualities 

Manned aviation introduced the notion of handling 
qualities, assessing how well an aircraft can be flown 
by a human pilot as a basis for certification of both 
civil and military airplanes. Since UAS typically rely 
on autopilot systems enabling a certain degree of au¬ 
tonomy, these concepts may not be directly applied, 
but are still extremely relevant. Most importantly, the 
handling qualities concerning static and dynamic sta¬ 
bility, as well as controllability determine the success 
of a UAS design. 

While an autopilot can handle more and faster insta¬ 
bilities than a pilot, it can certainly not compensate for 
missing actuation authority. As detailed in Sect. 26.7, 
it is advisable to implement de facto manual operation 
mode as a testing, backup, or even standard operation 
mode, in which the airplane is either steered manually 
or through some stability augmentation system (SAS). 
Therefore, it is highly advisable that the resulting sys¬ 
tem complies with the following core requirements 
(simplified from [26.18]): 

• Static longitudinal stability: most aft COG at least 
5% of c in front of aerodynamic center (static mar¬ 
gin, SM). 

• Phugoid damping £ p h > 0.2. 

• Short-period oscillation <y s h > 2, £ s > 0.5. 

• Spiral mode may be unstable, if r sp > 20 s. 

• Roll acceleration at maximum aileron deflection 
|p(^ a ,max)| > 5 rad/s 2 , roll subsidence time constant 
As > Is. 

• Dutch roll damping £<jr >0.1. 

• Spins shall not be entered abruptly and must always 
be recoverable. 



Fig. 26.24 Simple cascaded airplane guidance and control system 


Fig. 26.25 Illustration of L\ lateral guidance 


26.4.5 A Simple Autopilot 

As shown earlier, airplane dynamics are nonlinear 
multiple-input-multiple-output (MIMO) systems with 
a significant amount of cross-coupling, thus they are 
inherently challenging to control. While a plethora 
of control strategies have been suggested as autopi¬ 
lots, we will provide a simple yet functional approach 
here that employs the popular concept of cascaded 
control loops as well as simple linear single-input- 
single-output (SISO) PID controllers acting on sub¬ 
parts of the dynamics. This approach is still widely 
deployed and well-understood, despite the fact that 
more advanced controllers, such as model-based lin¬ 
ear quadratic regulators (LQR) with gain scheduling or 
nonlinear dynamic inversion (NDI) may achieve sig¬ 
nificantly better performance. Reference [26.24] con¬ 
stitutes an eccelent reference for in-depth treatment of 
small UAS guidance and control with cascaded control 
loops. 

Cascaded Control Architecture 
Figure 26.24 introduces the overall controller archi¬ 
tecture. It presents the (typical) separation of per-axis 
rate controllers at the innermost loop, followed by an 
attitude controller and a combined altitude and speed 
controller TECS (total energy control system), as well 
as by a lateral L\ guidance. In general, care must be 
taken to separate successively closed loops by around 
a decade in terms of bandwidth. 

Assuming little cross-axis sensitivity, the rate con¬ 
trollers may be implemented with simple individual 
P-controllers, optionally with 1/V 2 gain scaling. Also 
the attitude controllers can be as simple as P and PI- 
controllers for roll and pitch, respectively. Note that 
the desired roll and pitch angle derivatives need to be 
transformed in the static block T v into angular reference 
rates: this can be achieved by applying the inverse Ja¬ 
cobian J r from (26.42) - where the missing yaw angle 
time derivative can be computed from the coordinated 
turn constraint = 0, ft = 0 in (26.69) 

• g sirup 6 a sin (p 

\/f = -1-- . (26.73) 

Vt cos a cos (p cos </> cos 6 

The combined altitude and speed controller (TECS) 
inspired from [26.25] uses the difference to the refer¬ 
ence altitude Ah = Iia — h to compute a desired climb 
rate of the form Iia = h a a j + Kp^Ah, with the given 
trajectory rate of climb A tra j and a P-gain K P ^ t . Know¬ 
ing the speed and corresponding angle of attack, hi can 
be simply converted into a desired pitch angle 6a - 
to be saturated according to maximum thrust (climb) 
and drag (sink). Since climb rate must be provided via 
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additional thrust, the respective power component is 
computed as AT^in* = rngh^/V Concerning the speed 
control, the second thrust component is computed with 
the P-gain K PyVe 1 as A7 ilcc = mK PtVe iAV t . 

As a last autopilot component, the £1 lateral 
guidance [26.26] proceeds as follows (illustrated in 
Fig. 26.25): a reference circular path of radius R is cal¬ 
culated that intersects the reference path given by the 
waypoint sequence at look-ahead distance L\. In or¬ 


der to track this reference, a centripetal acceleration of 
Vf/R is needed, which can now be directly translated 
into a desired roll angle <p = arctan(V t 2 /(Pg)) corre¬ 
sponding to a coordinated (level) turn - saturated with 
maximum bank angles. 

Note that the suggested scheme should be enhanced 
by stall prevention and recovery (AOA monitoring and 
control), as well as by preventing sideslip for safer (and 
more efficient) operation. 


26.5 Rotorcraft Modeling and Design 

Various types of rotorcraft UAS configurations have 
been developed in the past (some examples are shown in 
Fig. 26.26), from helicopter-type UAVs such as [26.27, 
28], over a vast selection of multicopter configura¬ 
tions such as [26.29,30] and tail-sitter vehicles such 
as [26.31,32] up to completely new types of flight 
mechanisms [26.33,34]. The design, modeling, and 
system analysis process for all these RW-UAS types 
is essentially very similar and is largely based on 
the methodologies originally developed within the 
aerospace community for full-scale rotorcraft design 
and evaluation [26.15,35], In this context, it is im¬ 
portant to realize that the rotorcraft design process 
goes beyond mere efficiency and payload considera¬ 
tions focused on the propulsion components (e.g., using 
BEMT). Designing an effective RW-UAS should in 
principle also include flight dynamics assessments of 
the entire robotic flight platform. 

Flight performance assessment is commonly based 
on one of two types of modeling approaches referred 
to as quasi-steady and hybrid [26.36]. The quasi-steady 
modeling method employs a single rigid body represen¬ 
tation of the aircraft affected by the steady-state forces 
and moments originating from the propulsion subsys¬ 
tem. Hybrid models treat the rotorcraft as a multibody 
system where the dynamics of the aircraft body are cou¬ 
pled with additional dynamics of the rotor or propeller 
blades (e.g., blade flapping dynamics). For propeller- 
based RW-UAS like multicopter and tail-sitter vehicles 
using the quasi-steady approach to, e.g., model attitude 
dynamics is most widespread. This may be related to 
the fact that for these vehicle configurations, properly 
accounting for motor dynamics may be more rele¬ 
vant than accounting for high-order effects related to 
structural deformations of the propeller blades. For 
helicopter-type UAVs the hybrid approach is more com¬ 
mon as some dynamic modes of the rotor system are 
likely to couple with the attitude dynamics of the main 
rotorcraft body. Note that a proper application of the hy¬ 
brid approach is considerably more involved than using 


quasi-steady models and should only be resorted to if 
justified. 

A detailed treatment of the specific modeling and 
design procedures for every robotic rotorcraft config¬ 
uration is beyond the scope of this chapter and thus 
the presented considerations focus on helicopter-type 
and multicopter UAVs. Based on the extensive theoreti¬ 
cal aerospace-related background available in [26.14, 
15,35] amongst others, models for the most relevant 
rotor respectively propeller forces and moments are 
presented and subsequently appended to the rotorcraft 
body dynamics discussed in Sect. 26.3.6. A simplified 



Fig. 26.26 (a) Conventional helicopter configuration. 
Swiss UAV Neo S-300. (b) Quadro-copter configuration. 
Microdrones MD4-1000 
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hybrid modeling approach is introduced and reductions 
to quasi-steady models are discussed where applicable. 
Finally, a few metrics meaningful for rotorcraft design 
and control purposes are discussed and summarized 
shortly. 

26.5.1 Mechanical Design of Rotors 
and Propellers 

The main control mechanism for any RW-UAS is its 
rotors or propellers. Accordingly, to understand the 
working principles and the dynamics of the rotorcraft 
type of aircraft, it is worth investigating a few of the 
main design characteristics found in these flying mech¬ 
anisms. 

The following discussion focuses on the operation 
principles of helicopter rotors and will be expanded 
to propellers subsequently. Figure 26.27 visualizes the 
typical rotor degrees of freedom realized via the flap, 
lead-lag and feathering (also referred to as pitch) 
hinges. The flap hinge allows the rotor blade to flap due 
to aerodynamic and inertial loads affecting the blade 
body during flight. The lead-lag hinge responds to lat¬ 
eral rotor blade moments due to Coriolis forces related 
to flapping. Where the flap and the lead-lag hinges 
are usually passive, possibly augmented with spring or 
damper elements, the pitch hinge is active in order to 
adjust the blade angle of attack and thus the generated 
aerodynamic forces. 

Three types of rotor hubs are typically found in 
modern helicopters referred to as teetering, articulated, 
and hingeless depending on the mechanical realization 
of the flap hinge (Fig. 26.28). In the case of teeter¬ 
ing rotor, a single hub flap hinge is located directly 
on the rotorshaft axis, rigidly connecting a set of two 
rotor blades. For the articulated rotor, the blades and 
the rotor hub are connected via mechanical hinges at 
a specific offset e from the rotor shaft axis, thus al¬ 
lowing each blade to flap individually. The hingeless 
rotor flaps through the deformation of elastic elements 



Fig. 26.27 Typical hinge configuration of an articulated 
rotor blade 


connecting the hub with each individual rotor blade or 
directly through structural deformation of the blades 
themselves. In this case, a virtual hinge offset can be 
defined at the intersection of the rotor hub plane and the 
tangent to the deflecting blade body at 75% of the rotor 
radius [26.14]. The specific characteristics of the flap 
hinge (offset from rotor shaft, stiffness, and damping) 
are fundamental for rotor blade flapping and in con¬ 
sequence for the rotorcraft pitch and roll dynamics as 
discussed later. 

In the case of most propeller-based rotary-wing 
(RW) UAVs (e.g., multicopter systems) the feathering 
and lead-lag degrees of freedom do not exist. Nev¬ 
ertheless, propeller-based RW-UAVs may exert blade 
flapping by deformation of the propeller blades. 

26.5.2 Rotorcraft Dynamics 

As discussed in Sect. 26.3.6, the main rotorcraft body 
dynamics can be directly described by the simplified 
differential (26.45) which for most rotorcraft systems 
maintain the presented mathematical structure. For 
helicopter-type and multicopter UAV configurations the 
dominant set of external forces and moments affecting 
these dynamics can be summarized as 


N t N, 

F = F G + ^F' + X> h +f d , 

1=1 1=1 

Nr N r Nr N T 

r = r o +t t+ t h+ y t b 

i= 1 i=l i= 1 i=l 


(26.74) 


(26.75) 


The vector Fq represents the weight force and F\ is 
the thrust force of the /-th rotor or propeller out of the 
total set of N r rotors or propellers (Fig. 26.29). These 
forces are related to the body heave dynamics but due 



Teetering rotor 



Articulated rotor 


Hingeless rotor 


Fig. 26.28 Rotor hub design concepts of a teetering rotor, 
an articulated rotor with hinge offset e and a hingeless rotor 
with a virtual hinge offset e 
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to the underactuation of most rotorcraft are also respon¬ 
sible for the lateral rotorcraft acceleration. 

The additional in-plane hub forces F' n represent 
drag-related effects that may be neglected near hover 
but become more dominant for higher lateral flight ve¬ 
locities [26.15]. The vector F D represents the drag force 
associated with the rotorcraft main body. For simplic¬ 
ity’s sake, it is assumed that the center of pressure of 
the rotorcraft body is collocated with its center of grav¬ 
ity which is not necessarily the case. 

The relevant set of external moments is defined by 
the torques Tq affecting the vehicle yaw dynamics, the 
thrust-induced moments Tj, the moments introduced 
by the hub forces t' h and the flapping moments related 
to the rotor hub stiffness r jj. 

More explicit expressions for the respective force 
and moment terms may be found in Tables 26.6 
and 26.7, where 7] is the averaged thrust force mag¬ 
nitude, H’ x and H' y are the hub force components along 
the rotorcraft body frame x- and y-axes, and Q, is the 
torque generated by the i-th propeller or rotor. The aero¬ 
dynamic drag of the main rotorcraft body, represented 
by the components D x , D y , and D z , has been discussed 
in some detail in Sect. 26.3.5. 

The vector B r t corresponds to the displacement of 
the i-th rotor or propeller hub from the rotorcraft body 
frame origin and B n, is the tip-path plane normal of the 
i-th propeller or rotor disk as explained in [26.37]. 

The coefficients and Pi s represent the longi¬ 
tudinal and lateral flapping coefficients [26.15] de¬ 
scribing the tilting of the rotor or propeller disk as 
elaborated in more detail later in this section. Fi¬ 
nally, the parameter k‘ b corresponds to the flapping 
spring stiffness of the i-th rotor or propeller hub. 
In the case of articulated or teetering hubs this tor¬ 
sional spring stiffness represents potential flap hinge 
springs and in case of hingeless hubs approximates 


Table 26.6 Typical rotorcraft forces 


Type of force 

Expression 

Gravity 

B F G = b r w 

( : ) 


Thrust 

Br?i _ fi 

— 

itjTi 

Hub forces 

B F‘h = 

H ‘y 

W 


Rotorcraft body drag 

b Ed = 

A, 

vv 




Fig. 26.29 A conventional helicopter and a quadro-copter plat¬ 
form. Flapping angles have been exaggerated and only the rotor 
and propeller forces relevant near hover have been visualized 

the structural bending stiffness of a particular rotor or 
propeller. 

In the case of a helicopter tail rotor, flapping is usu¬ 
ally neglected and the thrust direction is modeled as, 
e.g., 


/ 0 \ 

1 

w 


(26.76) 


To finalize the models of these external forces and 
moments, the respective aerodynamic effects as well as 
the role and characteristics of blade flapping must be 
discussed next. 


26.5.3 Simplified Aerodynamics 


In order to accurately predict the forces and moments 
generated by a rotorcraft system, accounting for var¬ 
ious ranges of operation conditions, detailed design 
specifications, aerodynamic interactions between the 
different rotors or propellers and possibly the rotorcraft 
body itself, highly sophisticated aerodynamics simula- 


Table 26.7 Typical rotorcraft moments 


Type of moment 

Expression 

Torque 

%= 0 

\Qo 

Thrust moment 

B Tj = V X ’% 

Hub moment 

B T ‘ H = B r i x B f i n 

Flap moment 

(-pi,) 

% = 4 pi 

V 0 > 
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tion tools are required. Such tools are usually not easily 
accessible to or operated by the roboticist and may hide 
some of the fundamental flight dynamics characteristics 
of a particular platform due to the involved complexity. 
For a repeated evaluation of the general flight properties 
of a rotorcraft it may thus be preferable to derive ap¬ 
proximate, analytical models of the aerodynamic forces 
and moments. This may provide profound insight into 
the core working principles of rotorcraft operation. 

The general approach to derive such models is 
the employment of BET as partially presented in 
Sect. 26.3.4. BET, as opposed to BEMT or MT, main¬ 
tains the notion of individual rotor or propeller blades. 
Flence, it may account for changes in the angle of attack 
a, the aerodynamic inflow velocities Uj and Up and in 
consequence of the local lift and drag increments dL 
and d D, not only in function of the radial position r 
of the observed airfoil segment but also in dependency 
of the blade azimuth £. To further the understanding of 
this dependency in f, one may examine Fig. 26.16 as 
well as Fig. 26.30 where the hub coordinate frame H is 
introduced. 

The perpendicular and tangential inflow velocities 
Up and Uj (and thus also a) are strongly related to 
the rotor angular speed 12 and the induced inflow ve¬ 
locity Vi (as discussed for BEMT). In flight operation 
beyond hover, these inflow velocities are also affected 
by the rotorcraft linear and angular velocities B v and 
B o), as well as the potential flapping motion of the rotor 
or propeller blades. From the perspective of a revolving 
rotor or propeller blade, these additional inflow veloci¬ 
ties vary periodically in function of £. 

Inspecting Fig. 26.30, one can establish the follow¬ 
ing relations for the airfoil inflow velocities [26.35] 

U-p = £2(e + r) — Vy + Pco x r , (26.77) 

Up = Vi — w + co y (e + r) + fir— fiv x . (26.78) 



Fig. 26.30 (a) Top view of revolving rotor/propeller blade and the 
relevant velocities, (b) Side view of revolving rotor/propeller blade 
and the relevant velocities 


The linear and angular velocities v x , Vy, a> x , and C 0 y, cor¬ 
respond to the projections of the vehicle body velocities 
u, v, p, and q into the hub frame H 


u cos £ + v sin f , 

(26.79) 

v cos £ — m sin£ , 

(26.80) 

p cos £ + q sin £ , 

(26.81) 

qcos% — psin£ . 

(26.82) 


For helicopter-type UAVs the azimuth dependency of 
a may also be related to periodic changes in the blade 
pitch angle 9. For helicopter configurations the feath¬ 
ering angle 6 can usually be controlled collectively for 
the entire rotor disk or cyclically in dependency of the 
blade position £ using a swashplate mechanism [26.38] 

9 = 9q + 0ic cos £ + @i s sin £ . (26.83) 

The swashplate mechanism essentially provides the 
means to adjust the collective pitch angle 9q as well as 
the cyclic pitch angles 9 i c and $i s individually. Note 
that for twisted propeller or rotor blades 6q merely 
represents the pitch angle at the blade root and an ad¬ 
ditional term accounting for the radial variation in pitch 
must be introduced [26.14]. 

By altering 9q , the swashplate provides some con¬ 
trol over the average angle of attack of the entire rotor 
disk and thus the generated average thrust and torque. 
Similarly, varying the cyclic components creates a lift 
imbalance between opposing sides of the rotor disc. 
This lift imbalance induces a periodic flapping motion 
of the individual rotor blades (also depending on f) 
which affects the pitch and roll moments rg and tj. 
For helicopter-type UAVs this is the main control mech¬ 
anism for the pitch and roll attitude dynamics. 

For the majority of multicopter vehicles 9q is usu¬ 
ally fixed and the cyclic angles do not exist ($i c = 9i s = 
0). In this case, attitude control is realized through tt 
only by changing the propeller speed 12, of the individ¬ 
ual propellers. 

To describe the aforementioned periodic changes in 
blade flapping the following representation of the flap¬ 
ping angle ft is customary when modeling rotorcraft 
dynamics [26.39] 

P = Po(f) + Pic(t) cos£(f) + Pu(t) sin £ (t) . 

(26.84) 

The coefficient fio(t) corresponds to the coning angle 
of the entire rotor or propeller disc, whereas Pi c (t) 
and Pi s (t) are the longitudinal and lateral disk tilting 
angles. 
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Unlike the blade pitch coefficients in (26.83), the 
flap coefficients in (26.84) are assumed to be functions 
of time t which is later exploited when modeling flap¬ 
ping dynamics. 

Although it is crucial to capture all these peri¬ 
odic dependencies in f, it can be obstructive for the 
purpose of creating an efficient rotorcraft simulation. 
Directly simulating the body dynamics (26.45) un¬ 
der the influence of the external forces and moments 
(26.74) and (26.75) while accounting for all depen¬ 
dencies in would require the blade motion to be 
simulated step by step while moving around the rotor 
or propeller shaft. This can lead to comparatively stiff 
differential equations which may be hard to simulate 
efficiently. 

An alternative approach is to derive simplified force 
and torque models by integrating the lift and drag in¬ 
crements along the rotor or propeller radius, summing 
over the number of blades N\, and averaging around the 
rotor azimuth as 

2 n R 

T =^ff dL ' 126851 

0 0 

2tt R 

e =st// r ( do+ tH- 12689 

0 0 

2tt R 

H * = ^// sin K dD+ S dL ) ’ (26 - 87) 

0 0 

2n R 

//. I /cos ? (dD +r -dL) . (26.88) 

0 0 

Combining (26.77) to (26.88) with (26.12), (26.13), 
(26.27), and (26.33) and assuming simple rotor or pro¬ 
peller blade geometries (e.g., linear blade twist) the 
above expressions can be evaluated algebraically, e.g., 
using a symbolic computation program. As opposed to 
BEMT these averaged aerodynamic formulations are 
capable of accounting for the effects of the periodic 
changes of inflow velocities such as, e.g., the dissymme¬ 
try of lift phenomenon [26. 15] as well as for the varying 
blade feathering and flapping angles. 

To gain some appreciation of how these averaged 
models may be useful in providing insight into the core 
characteristics of rotorcraft operation, a strongly simpli¬ 
fied thrust model directly resulting from (26.85) under 
the assumption of hover and negligible flapping angles 
is given as an example 

T= (ki\9o-\-ki2~^ +&T3^ • (26.89) 


The coefficients k-/-\ , k T 2 , and £ 7-3 essentially de¬ 
pend on aerodynamic and geometric rotor or propeller 
properties. As one may deduce directly from (26.89), 
thrust is a linear function of the collective pitch an¬ 
gle Oq and the average induced rotor or propeller in¬ 
flow velocity vq which is further defined in the next 
section. 

26.5.4 Nonuniform Inflow 

In order to account for the varying induced flow field 
over the rotor or propeller disk in hover and forward 
flight, the following approximate induced inflow distri¬ 
bution may be assumed as discussed in [26.14] 

r 

Vi = Vo+ -Oi c cos£ + ?; ls sin £) . (26.90) 

K 

The velocity component vq represents the average in¬ 
duced inflow at the center of the modeled rotor respec¬ 
tively propeller disc. It is directly related to the thrust 
level the rotor or propeller is operating at and may be 
computed, e.g., based on the iterative method discussed 
in [26.15]. The inflow coefficients V\ c and v\ s account 
for changes in the induced flow field due to lateral rotor¬ 
craft flight velocities. Various steady-state models for 
Vi c and v: s have been proposed in the past with moder¬ 
ate prediction quality only. 

Reference [26.40] presents these models in de¬ 
tail and discusses an alternative method which also 
accounts for transient effects. Accounting for inflow 
dynamics, the transient response of vq, Vi c , and vi s 
to, e.g., sudden blade pitch changes can be captured 
and the aforementioned iterative computations may be 
avoided. 

26.5.5 Flapping Dynamics 

The dynamics of rotor or propeller flapping corresponds 
to a second-order differential equation of the flapping 
angle /3 which essentially represents an aerodynami- 
cally damped oscillator [26.35] 

ft = £ 2, u, V, w,p, q,p, q , ft, 6) . (26.91) 

Reference [26.41] and references therein elaborate the 
modeling process required to derive the above differen¬ 
tial equation from first principles. 

As for the simplified aerodynamic forces and mo¬ 
ments, the dependency of (26.91) on rotor azimuth 
is important but problematic for efficient blade flap 
simulations. Here, the averaging operation discussed 
in [26.39] has been employed. This essentially leads to 
the following differential equation of the flapping coef- 


Part B | 26.5 



Part B | 26.5 


652 Part B I Design 


ficients /So, /Sic and flu 

Ap (i + Ap ft + Ap fi = Aq 9 + Ac h) + A v v , 



(Po\ 


fvo-w'' 

p = 

Pic 

, V = 

Vic 




V Uls / 


(o — 


(P\ 

q 

p 

\q) 


( 8q\ 

1 C 

W 
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implement a flight simulator based on this set of non¬ 
linear differential equations and analyze the simulated 
flight responses. The linearization process presented in 
Sect. 26.3.6 may provide additional conclusions and 
is often followed by a separation into subsystems an¬ 
alyzed individually. 

One of the most crucial rotorcraft subsystems corre¬ 
sponds to the pitch and roll attitude dynamics as it is this 
subsystem which ultimately defines how agile, stable, 
or accurate a rotorcraft may be able to fly in the horizon¬ 
tal plane of motion. Assuming near hover operation and 
neglecting higher order dynamics, the linearized open- 
loop roll dynamics for example, can usually be reduced 
to a first-order transfer function 


The involved matrices from A ^ to A v depend on an 
abundance of aerodynamic, geometric, inertial, and 
structural parameters of the modeled rotor or propeller 
system as well as on the lateral flight velocity of the ro¬ 
torcraft. The structure of these matrices is thus strongly 
related to the type of rotor or propeller hub that is be¬ 
ing modeled. From the resulting mathematical relations 
it is for example possible to conclude that for a teeter¬ 
ing rotor without hinge springs, the maximal flapping 
response /3 to a cyclic pitch input in 6 will follow with 
a 90° phase shift in rotor azimuth [26.15]. 

Considering the body time constants of most ro¬ 
torcraft configurations in comparison with the time 
constants of the above dynamics, one may in general 
assume that the fast poles in (26.92) are negligible and 
they mostly represent rotor or propeller vibrations. Ac¬ 
cordingly, (26.92) is often reduced to a first-order dif¬ 
ferential equation by introducing the assumption /3 = 0. 
For very rigid rotors or propellers one may even assume 
/8 = 0 and compute the steady-state flapping response 
only (quasi-steady model). Flowever, this is not the case 
for all RW-UAS configurations. To what degree higher 
order effects such as flapping or inflow dynamics have 
to be accounted for, depends on the level of frequency 
separation between the body poles versus the poles of 
the rotor dynamics. 

26.5.6 Flight Dynamics Assessment 

Assembling the dynamics of the rotorcraft body with 
the forces and moments defined in the previous sec¬ 
tion, as well as possibly introducing higher order effects 
such as blade flapping and inflow dynamics, results 
in a set of nonlinear differential equations represent¬ 
ing the flight characteristics of the modeled platform. 
In order to inspect these flight characteristics one may 


G(J) 


P(s) K p 

U(s) s — D p 


(26.93) 


The frequency function P(s) represents the Laplace 
transform of the rotorcraft roll rate p and U(s) the 
Laplace transform of a control input affecting the roll 
subsystem. For a helicopter, this control input com¬ 
monly corresponds to a cyclic swashplate command and 
for a quadro-copter to the differential speed between 
two propellers on opposing sides of its airframe. The 
abstracted system parameter K p corresponds to the so- 
called roll control derivative and D p to the so-called 
roll damping [26.37], The control sensitivity K p defines 
how strong the initial rotorcraft roll acceleration p will 
respond to a control input represented by U(s). The 
parameter D p defines how well the system’s dynamic 
response will be damped in the following. For hybrid 
rotorcraft models the corresponding transfer functions 
are generally of higher order but are equally useful 
when analyzed with the well-known tools of linear sys¬ 
tem theory [26.15, 35], 

In general, these transfer functions and the result¬ 
ing metrics such as, e.g., K p and D p can be derived in 
dependency of a specific subset of physical rotorcraft 
parameters. Using these simple metrics and including 
the knowledge that may be gathered from the nonlin¬ 
ear flight simulator, flight performance trends can be 
assessed depending on the parameters of the UAV con¬ 
figuration under investigation. Such parameters may for 
example include the body pitch and roll inertia, the lo¬ 
cation of the body center of gravity or the location of 
the rotors or propellers. The fundamental understand¬ 
ing gained in this evaluation process is crucial for the 
development of effective robotic flight systems and the 
required control laws. 
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26.6 Flapping Wing Modeling and Design 


A variety of animals, from insects to birds, are capa¬ 
ble of flight maneuvers which are presently impossible 
in micro aerial vehicles, such as flying in turbulence or 
cluttered airspace. Additionally, animals are more ma¬ 
neuverable and can fly longer distances. People have 
made many attempts at building flapping robots or 
ornithopters. While several are successful, many ei¬ 
ther never take off or fly only for a short duration 
due to their higher complexity or poor design. Until 
recently, ornithopters represented a niche of flying ve¬ 
hicles. The development of lithium polymer batteries 
produced a light-weight high-power energy resource to 
power ornithopters. Among the first successful elec¬ 
tric ornithopters were the Caltech and Aerovironment 
microbats in 1998 [26.42,43]. Many designs still fail 
to fly despite the rapidly increasing population build¬ 
ing electric ornithopters. A major problem in most 
designs is an inability to generate enough lift to take 
off in the first place. This precludes additional flight 
research, such as maneuverability, flight distance or 
time. Engineers have believed that flapping wings are 
essential to further development of micro aerial ve¬ 
hicles since the first electric ornithopters took off 
and biologists started to understand the aerodynam¬ 
ics of flapping insect wings. The main reason behind 
this focus is the idea that they are aerodynamically 
more efficient at the small Reynolds number of insects 
(10—10000) when viscosity effects start to dominate 
airflow. 

26.6.1 Aerodynamic Mechanisms 

Our understanding of insect aerodynamics provides us 
with the most detailed model of the aerodynamic func¬ 
tion of a flapping wing [26.45]. There is some evidence 
that wing flexibility can improve aerodynamic perfor¬ 
mance of a flapping wing by roughly 10% [26.46] if 
the angle of attack is not optimized for a stiff wing. 
However, a parametric study using a robot model of an 
insect wing suggests that wing flexibility does not im¬ 
prove performance if we can optimize angle of attack 
independently of wing stiffness [26.47]. Ignoring aeroe- 
lastic effects that change angle of attack distribution, 
the key known aerodynamic mechanisms of a flapping 
wing are [26.45]: 

1. A stable leading edge vortex (LEV) that enables 
the wing to operate at high angles of attack with¬ 
out stall during the quasi-steady mid-stroke phase 
(Fig. 26.31). During stroke reversal the aerodynam¬ 
ics is not quasi steady. In this phase, five additional 
affects are thought to be important: 


2. Added mass effects due to fluid acceleration in re¬ 
sponse to the reversal. 

3. The Wagner effect explaining that changes in vor¬ 
tex strength need time to build-up over a few chord 
lengths of travel. 

4. Rotational lift due to the timing of changes in an¬ 
gle of attack during stroke reversal and its effect on 
vortex lift through the Kramer effect. 

5. Wake capture when the wing reverses direction 
and interacts with the momentum jet of its shed 
wake. 

6 . Clap and fling when the wings become close enough 
to (nearly) touch and air is forced out of the cavity 
formed by the two wings and sucked back in, which 
can increase lift [26.48]. 


a) 


b) 




(1) Stable leading 
edge vortex 


(2) Added mass 

(3) Wagner effect 

(4) Rotational lift 

(5) Wake capture 

(6) Clap & fling 
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Fig.26.31a,b Flapping insect wing aerodynamics can be under¬ 
stood through the interaction of a myriad of complex aerodynamic 
mechanisms, (a) The key high-lift mechanism insects employ, is 
a stable leading edge vortex (LEV) generated during the up and 
downstroke. (b) A flapping cycle consists of a quasi-steady part 
during which the wing accelerates little. During this phase, the sta¬ 
ble LEV is the key high-lift mechanism (1). During stroke reversal 
there is evidence that up to five effects (2)-(6) could be important 
(after [26.44]) 
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c) Lift coefficient d) Power factor e) Lift coefficient 



Drag coefficient Glide number Drag coefficient 


Fig.26.32a-e The aerodynamics of a flapping (insect) wing scale from insect to bird scale, (a) A stable LEV enables 
flapping wings to operate at high angles of attack without stall, (b) The key parameter explaining LEV stability is the 
wing’s swing, its spinning motion, as demonstrated by this spinning model of a fly wing which generates a stable LEV 
and similarly elevated forces as in flapping wings, (c) At insect scale fixed (translating) wings underperform, whereas 
flapping and spinning wings generate similarly high lift. Spinning wings generate less drag which makes them more 
efficient. (d) The power factor of a spinning wing is higher than for a flapping wing, higher indicating that less power 
is needed to support body weight, (e) The dimensionless lift and drag averaged over a full flapping cycle is independent 
of scale to within good approximation (Reynolds number 110: fruit fly: 1400; house fly: 14000; hummingbird). This 
makes flapping wing aerodynamics scalable enabling the use of dimensional analysis [26.49] 


There exist, however no quantitative experimen¬ 
tal studies or theories that fully dissect these effects 
and quantify their relative importance for aerodynamic 
lift and power. While flapping wing aerodynamics is 
complex and not fully understood, it is simple from 
a robot design perspective, because it is scalable from 
insect to bird size (Fig. 26.32). This enables prototyp¬ 
ing at larger, more cost effective, scales and enables 
scaling the design down as technology advances, and 
smaller components and fabrication methods become 


available [26.49]. Flapping wings generate more lift 
than translating wings because they generate a stable 
LEV. To generate a stable vortex over the whole wing, 
the aspect ratio with respect to the center of rotation 
needs to be equal to or smaller than about 4 [26.53]. 
Flapping wings with an aspect ratio larger than 4 can 
stall outboard [26.53]; whereas more stubby flapping 
wings cannot. This can explain why the majority of in¬ 
sect, bird, and bat wings have an aspect ratio of around 
2—4 with respect to the shoulder joint [26.53]. The 


Wingspan (cm) 
Mass (g) 
m/m 0 

Flight time 
Frequency (Ftz) 
Mechanism 



Delfly II 
28 
16 
1.26 
15 min. 

14 

Gearbox and 4-bar 


Scale (mm) 10 2 -10° 

Power 1.4 W 

Current 380 mA 



Nano-Hummingbird 
16 
19 
1.37 
11 min. 

30 

Gearbox and string 
rollers 
10 2 - 10 ° 

3.27 W 
880 mA 



RoboBee 

3 

0.06 

N/A (tethered) 
N/A (tethered) 
110 

Piezo-electric 
Elastic 4-bar like 
10 2 10" 1 
N/A (tethered) 
N/A (tethered) 


Fig.26.33a-c Examples of three 
different types of successful flap¬ 
pers. (a) Jaap Oldenkamp, Delfly 
II (after [26.49,50]), (b) Nano 
Hummingbird (after [26.51]), (c) 
RoboBee (after [26.52]) 
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main advantage of stubby wings is that they do not stall 
at high angles of attack enabling animals to take-off 
and land vertically by increasing angle of attack in¬ 
stead of flapping frequency [26.53] using LEVs [26.54]. 
Insects [26.55], bats, hummingbirds [26.56], and other 
birds [26.57], but also auto rotating seeds generate sta¬ 
ble LEVs. This shows that stable LEVs are a convergent 
evolutionary solution for high lift at high angle of attack 
in nature [26.53]. 

Comparison of flapping versus spinning (propeller¬ 
like) insect wings shows spinning insect wings generate 
similar elevated lift forces by generating a LEV at lower 
drag. Helicopters with stubby rotors are, therefore, 
aerodynamically more efficient than stubby flapping 
wings, because they need less power to fly, as quali¬ 
tatively presented in Fig. 26.32d [26.49]. This is con¬ 
firmed experimentally for the most advanced hovering 
ornithopter at present, the Nano Hummingbird [26.51]. 
Comparing its flapping wing with a spinning wing 
showed for various forward speeds that flapping wings 
require more power for the same lift, in part due to 
aerodynamics [26.49, 53], and in part due to inertia 
losses [26.49, 51]. The key advantage of flapping wings 
seems to be the potential for extreme maneuverability 
and robustness. For instance, flapping wings may fare 
better in turbulence, close to the ground, near verti¬ 
cal surfaces and through clutter, when helicopters can 
become unstable due to stall and complex rotor-wake 
interactions [26.58]. 

26.6.2 Sizing New Flappers 

An improved understanding of the detailed aerodynam¬ 
ics is scientifically invaluable, but perhaps not critical 
for designing successful ornithopters at a time when 
most struggle to take-off. Instead, sizing an ornithopter 
in terms of gross design parameters such as wing span, 
weight, and flapping frequency is more critical for take¬ 
off. The design methodology introduced here explains 
how one can transform successful designs to meet other 
mission perspectives. These designs can then enable 
flight studies that can advance our understanding of 
ornithopters versus Ro-UAS and FW-UAS to better ap¬ 
preciate their unique advantages. 

Amongst successful flappers, there are three main 
archetypes as shown in Fig. 26.33. Historically, most 
flappers have relied on variants of a four-bar mecha¬ 
nism to generate the flapping motion which generates 
lift. One example of this is the Delfly family of or¬ 
nithopters, which are capable of both fast forward 
flying and hover using this approach (l‘S3>JEiiil£IJTl). 
A recent design which demonstrates both prolonged 
hovering flight and maneuverability, although lacks 
the ability to fly fast forward, is the Aerovironment 


Nano-Hummingbird [26.51]. The Nano-Hummingbird 
uses a flapping mechanism composed of rollers and 
strings, while still using a geared down motor to pro¬ 
vide power at the right frequency. Additionally, the 
wings provide control, rather than traditional tail con¬ 
trol surfaces. Another more modern development is 
centimeter scale ornithopters which use piezoelectric 
actuators to generate flapping motion and control such 
as the Harvard Robobee [26.52] and the Berkeley 
Micromechanical Flying Insect. These are capable of 
tethered flight only, because no batteries exist that can 
supply high enough power in an enough lightweight 
package. 

Despite the differences in design, these flappers 
share common trends in parameters, as shown in 
Fig. 26.34. To design a functional ornithopter, we start 
with a desired mission such as surveillance, search 
and rescue, or military applications. The mission de¬ 
termines an appropriate wingspan, and also determines 
a minimum time for task completion. Figure 26.34 
shows that empty weight (mass without battery) fol¬ 
lows an exponential pattern with wingspan, especially 
over the mid-range of wingspans. The main observation 
is that the power defining scale is not 3, but approx¬ 
imately 1.5. This may be because significant portions 
of the mass of smaller ornithopters comes from elec¬ 
tronics, gearboxes and actuators, whose masses are not 
dependent on wingspan. Additionally, required flapping 
frequency decreases with wingspan, enabling an ap¬ 
proximation of required flapping frequency based on 
wingspan that works well for all sizes of ornithopters, 
as expected using scaling relations. 

Using initial design parameters from a success¬ 
ful ornithopter, we can design another ornithopter that 
is also capable of flapping flight using scaling rela¬ 
tionships of geometry, fluid mechanics, and battery 
physics [26.27]. We need to decide on design param¬ 
eters for the new flapper, including the wingspan b, 
weight W, aspect ratio A, and battery weight Wbatt- 
Here, the aspect ratio is wingspan divided by chord 
length, as these are both easily measured design pa¬ 
rameters. Example of initial parameters for the Delfly II 
are: b\ = 28cm, m\ = 16g (W = mg), Ai = 3.5, f\ = 
14Hz, Pi = 1.4W, Rbatt.i = 2.7 g, t\ = 15 min. Initial 
design parameters are denoted with subscript 1; while 
new design parameters are denoted with subscript 2. 
Using the curve fitted through successful ornithopters 
as shown in Fig. 26.34, one can make an initial approx¬ 
imation of empty weight. First, we can calculate the 
wing area, An, of the new flapper and the old flapper 
using the same equation for each 

b 2 


(26.94) 
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a) Empty mass (g) 



Wingspan (cm) 

b) Flapping frequency (Hz) 



Fig.26.34a,b Current ornithopter trends of empty mass 
and flapping frequency with changes in wingspan, (a) 
The empty mass of successful ornithopters does not 
scale with wingspan cubed, but with wingspan to the 
power 1.5 ( R 2 = 0.79). The power law predicts the ap¬ 
proximate masses effectively in the 10—50 cm wingspan 
range, while it overestimates the mass for those with 
wingspans below 10 cm. The curve to the third power 
consistently underestimates the unloaded masses of cur¬ 
rent ornithopters. (b) To support the weight of the or¬ 
nithopter, flapping frequency needs to increase inverse 
to wingspan for smaller wingspans. Ornithopters in (a) 
fly freely and have a flight time of at least one minute. 
The Micromechanical Flying Insect and Harvard Robobee 
follow the same trend line for flapping frequency as 
larger ornithopters; even though they fly tethered (they 
would need to flap faster with batteries onboard). The 
relationship here fits a power curve with the exponent 
equal to —1.01 with R 2 = 0.96. Abbreviations are as fol¬ 
lows: MFI - Berkeley Micromechanical Flying Insect; 
HMF - Harvard Robobee; KU1,2,3,4 - Konkuk Univer¬ 
sity ornithopters; DFI,II,M-Delfly I,II and Micro; Nano - 
Aerovironment Nano-Hummingbird; UMD SB. JB. BB - 
University of Maryland Small Bird, Big Bird, Jumbo Bird; 
AM - Brian’s Ornithopter; uB3 - NiCad powered Caltech 
Microbat 


In hovering or steady forward flight, it is reasonable to 
assume that weight is proportional to lift 


Wcx -c L pU t An . 


(26.95) 


We assume that cl (lift coefficient), p (density,) and g 
(gravitational acceleration) are constant [26.49], which 
is reasonable for flights on earth at low altitudes. Then, 
rearranging produces the following relationship be¬ 
tween forward velocities, V t 




Wi A fl , 2 


(26.96) 


We can then assume that the advance ratio J is constant 
for both vehicles, which is a reasonable approximation 
for ornithopters with similar wing kinematics, shape, 
and deformation. The advance ratio J is the ratio of 
maximum forward speed to wingtip speed 


/ = 


4 f&R ' 


(26.97) 


Since wingspan is twice the radius, and we can use the 
assumption that J is constant to obtain the following 
relationship for flapping frequencies 


Vt,2 b\ 0\ 

y u z?2 02 


(26.98) 


Then, assuming that flapping amplitude, 0 is constant 
between the two designs (reasonable for designs that 
follow the same parameters and keep the same gear¬ 
boxes) we can simplify the relationship for flapping 
frequencies 


h 


Vu2bi_ 

Vu b 2 J 


(26.99) 


The required power to fly is proportional to the weight 
and flight speed 


P oc mgV t = WV t . 


(26.100) 


Thus, we can calculate the power required of the new 
flapper relative to that of the old flapper 

y t2 w 2 

Pl = Pl V^W- (26 ’ 101) 

y t ,i Wi 

Using the power calculated above, the flight time can be 
estimated as 
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(26.102) 
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in which Uuvo = 3.7 V for a LiPo battery, and where, 
as in Fig. 26.35, the capacity can be approximated as 

6*LiPo — 77?batt^batt • (26.103) 

Front the scaling equations (particularly (26.101) 
and (26.102)), we can produce a set of graphs as in 
Fig. 26.36, allowing us to use the wingspan and flight 
time to design a scaled ornithopter. Beginning with 
the approximate wingspan and flight time desired, we 
use Fig. 26.36a to choose the appropriate battery mass. 
An increase in wingspan creates the option for heav¬ 
ier batteries and an increase in flight time as does an 
increase in battery mass. The wingspan and battery 
mass specify the required flapping frequency. This al¬ 
lows us to choose a motor and gear ratio. If this turns 
out to be impractical with available components, we 
can adjust parameters and iterate between the equations 
shown in Fig. 26.36. In general, for an ornithopter with 
equal mass, increasing the wingspan decreases the nec¬ 
essary flapping frequency. Alternatively, increasing the 
battery mass to improve flight time also requires in¬ 
creasing flapping frequency, electric power, and current 
to carry the extra payload. This explains why increas¬ 
ing battery mass beyond empty weight causes little 
increase in flight time, because the airframe needs to 
become much stronger at the cost of weight. A penalty 
in the flight time scaling equation needs to be imple¬ 
mented to correct for the increase in structural weight. 
The required flapping frequency and battery mass ratio 

Battery capacity (mAh) 



Fig. 26.35 Battery capacity as a function of mass for 
many small lithium polymer (LiPo) batteries in the size 
range (< 10 g) which would be used for ornithopters with 
10—50 cm wingspan. The graph shows the technology 
is linearly scalable. The approximate capacity density of 
small LiPo cells (3.7 V) is 37 mAh/g 


specify the required power. Power increases signifi¬ 
cantly with wingspan. Additionally, power increases 
with added battery mass due to the increase in flapping 
frequency required to lift the larger mass. Finally, we 
can determine the current the battery needs to supply, 
which is proportional to the power assuming we use 
the same kind of battery and efficiency of motor. Iter¬ 
ating between these steps enables finding solutions that 
best meet the mission specifications. We note that many 
ornithopters could fly significantly longer by doubling 
their current battery mass (Fig. 26.36a) at the expense 
of control response (inertia) and airframe loading. 

If flight time needs to increase for a wingspan- 
constrained ornithopter design, and battery mass and 
chemistry is already optimized, we should reduce air- 



Fig.26.36a-d These four figures show the effects of 
changing wingspan and adding battery mass to an or¬ 
nithopter on the flight time, power consumption, current re¬ 
quirement, and flapping frequency requirement. The value 
of the empty mass, m e , is determined using the fitted curve 
in Fig. 26.34a for each wingspan. The figures are then 
scaled from the initial reference (Delfly II) whose position 
is at (1,1) in each figure, (a) Increasing the battery mass 
ratio increases the flight time up until the ratio becomes 
equal to 3. This ignores additional airframe mass needed 
to carry these batteries, (b) However, increasing the bat¬ 
tery mass also increases the required flapping frequency. 
(c,d) Increasing the frequency also increases the necessary 
power ( P ) and current (I). Using these parameters, we can 
iterate back and forth between the plots until a feasible de¬ 
sign is found 
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a)///o b) Design parameter c) f/f 0 





Fig.26.37a-c Changing additional parameters can modify performance of a scaled vehicle, (a) Adjusting the flapping 
amplitude allows the user to change the required flapping frequency to use available motor/gearbox combinations. Gen¬ 
erally, larger flapping angles result in increased lift coefficient and decreased drag (after [26.59]). Thus, increasing the 
amplitude to match it with the motor and gear train can decrease the required power to fly. (b) As the aspect ratio 
increases at a constant wingspan, the wing area decreases, and therefore the flight time decreases while the required flap¬ 
ping frequency (and hence the power and current) increases, (c) Flight time decreases with additional payload (weight) 


frame mass (Fig. 26.37) and increase wing area [26.49]. 
Mass can be further decreased by airframe optimization 
using underutilized aerospace optimization strategies, 
and by critically reevaluating the payload. Wing area 
can be increased by decreasing aspect ratio and select¬ 
ing a biplane instead of a monoplane configuration. 
Whereas such wing design changes reduce aerody¬ 
namic efficiency of the wing, they increase the overall 
vehicle energy efficiency, and therefore increase flight 
time. Ornithopters that fly long enough to complete 
missions are often controlled by low-weight underpow¬ 
ered actuators that sacrifice maneuverability. 

To control the ornithopter’s flight and to utilize its 
maneuverability, we need to generate enough control 
torques with lightweight actuators. Designs optimized 
for flight time, such as the Delfly, use control surfaces 
added to the tail in the style of a traditional rudder 
or elevator. More maneuverable designs use the flap¬ 
ping wings as control surfaces, by changing their angle 
of attack (Nano-Hummingbird [26.51]) or left versus 
right wing relative flapping motions (Robobee [26.52]). 
The two dominant off the shelf actuators are stan¬ 
dard servos and magnetic actuators. Standard servos 
have small electric motors and potentiometers and 
move to specified positions; while magnetic actuators 
have a small magnet inside a small coil of wire and 
apply specified amounts of torque. Magnetic actua¬ 
tors are available at lower masses than servos, which 
proves critical in optimizing performance of smaller 
ornithopters. This shows that selecting appropriate ac¬ 
tuators involves a tradeoff between flight duration and 
maneuverability. Ornithopters that are more maneuver- 
able require more powerful and precise servo actuators. 
The required servo torque of a scaled ornithopter can 
be estimated assuming isometric scaling: Torque should 
be proportional to total weight times wingspan, because 
aerodynamic force is proportional to weight, and arm 


length to wingspan. Knowing the required torque, we 
need to find a servo that can provide it. To reduce trial 
and error we have plotted current servo data to deter¬ 
mine how torque correlates with mass to budget for its 
weight. The data in Fig. 26.38 shows that torque is pro¬ 
portional to mass squared for current servo technology. 


Servo torque (kg cm) 



Servo mass (g) 


Fig. 26.38 Servo (dots) and actuator ( crosses ) torques in¬ 
creases with mass. The intensity of dots represents the 
servo speed, with darker dots representing faster servos 
(the magnetic actuators do not have speeds shown, as they 
apply a force rather than specify a position). The servo 
speed does not correlate strongly with mass, as it is de¬ 
pendent on the motors, gears, and other internal hardware 
of the servo, as well as the supply voltage. There are mag¬ 
netic actuators available in the range of 0.8—1.8 g, they are 
not included here due to lack of data available from manu¬ 
facturers 
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while empty ornithopter mass scales with wingspan to 
the power of 1.5 (Fig. 26.34), so as wingspan increases 
the actuator mass can become proportionally smaller. 

We have demonstrated current design strategies 
based off scaling successful designs that ensure or- 
nithopters fly. These upgraded rules of thumb are pow¬ 
erful because current aerospace design analysis and 


optimization techniques for ornithopters lack predictive 
power and are therefore less informative than estimates 
based on scaled flying designs. If current designers base 
their first iteration of new ornithopters on current state- 
of-the-art ornithopters, the field can progress at a faster 
pace through successful flight testing of new concepts 
that meet novel mission criteria. 


26.7 System Integration and Realization 


Enabling autonomous flights with UAS incorporates 
solving many challenges. This requires an interdis¬ 
ciplinary approach, bringing together expertise from 
many different fields. As shown in Fig. 26.39, knowl¬ 
edge in the field of aircraft design, as detailed in this 
chapter, is required, as well as in many fields of engi¬ 
neering and robotics. 

26.7.1 Challenges for Autonomous UAS 

Given the agility of UAS and their strict limitations 
on weight and power consumption, the choice of sen¬ 
sors, processors, and algorithms impose great technical 
and scientific challenges. Also, major differences exist 
between ground vehicles and UAS - sensors and algo¬ 
rithms that work well on ground vehicles cannot simply 
be applied on UAS due to inherent challenges. 

Limited Payload and Power Supply 
Weight and size restrictions require that lightweight 
sensors have to be used, usually at the cost of having 
noisier and less accurate data. These limitations also re¬ 
strict the choice of onboard computers being used to 



Fig. 26.39 UAS design and research is interdisciplinary as 
challenges from both aircraft design and mobile robotics 
are combined 


process information from the sensors. Thus, algorithms 
need to cope with such data to achieve robust estimates, 
while having limited computational power. As a refer¬ 
ence, the take-off weight of a commonly used small 
multirotor UAS (e.g., shown in Fig. 26.40) is 1.5 kg, 
including 0.5 kg of payload, while the flight autonomy 
is approximately 15 min. For this class of UAS, approx¬ 
imately lOOmW are required per 1 g additional take-off 
weight for hovering - a fact that has to be considered 
even for mounting small additional payload. Detailed 
studies of take-off weight, payload, and flight time of 
such systems can be found in [26.60]. 

Degrees of Freedom 

Compared to typical ground vehicles, there are two ad¬ 
ditional degrees of freedom (DOF) for the vehicle's 
attitude (pitch and roll angle) as well as one additional 
degree of freedom for the altitude, that need to be es¬ 
timated and controlled. This requires state estimation, 
control, and planning to be performed in full 6-D space, 
and without simplifying assumptions. 

Under-Actuated Systems 

The types of UAS studied here are usually under¬ 
actuated, as there are less control inputs available than 
DOF. As a result, the attitude has to be changed for 
many maneuvers. This in return changes the field of 
view of onboard sensors interacting with the UAS’ en¬ 
vironment like cameras, or distance sensors. 

Constant Motion and Inherent Instability 
UAS cannot simply stop to acquire sensor readings, 
when state estimation is delayed or contains high uncer¬ 
tainty. While waiting for measurements or re-evaluation 
of uncertain state estimates, the vehicle continues mov¬ 
ing and further falsifies these estimates. 

26.7.2 Levels of Autonomy 

We classify levels of autonomy and interaction with the 
pilot or operator into three categories: Manual flight, 
semiautonomous and autonomous. Industry and the re- 
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Fig. 26.40 Multi-rotor UAS and its 
main components (after [26.60]) 


search community have made great progress toward 
semiautonomous flights, while there are still many open 
questions and research topics for truly autonomous op¬ 
eration. 

Manual Flight 

We refer to this mode when operation of the UAS re¬ 
quires pilot skills. A remote-pilot has to handle attitude 
dynamics and throttle/thrust in a way such that the 
UAS remains in a stable state. That is, the pilot cannot 
leave the hands off the remote control. This also means 
that line-of-sight has to be maintained up to a distance 
where the pilot can observe the state of the UAS prop¬ 
erly. Piloting may be aided by a stability augmentation 
system (SAS), like rate stabilization commonly used on 
fixed wing aircraft, or attitude control commonly used 
on multirotor UAS. 

Semiautonomous (Automatic) 

This flight mode does not require piloting skills any¬ 
more, which is why we refer to a pilot rather than 
an operator. Onboard sensors and algorithms are in 
charge of stabilizing the UAS, such that the opera¬ 
tor can leave the hands off the remote control and the 
UAS remains in a stable state, waiting for input such as 
waypoints or desired velocities. However, it is still the 
full responsibility of the operator to find feasible paths, 
safely navigate around obstacles and to interact with 
other air traffic. This requires line of sight to be main¬ 
tained, however, first-person view could be thought of, 



Fig. 26.41 Components for an autonomous UAS 


as long as legal requirements are fulfilled. Examples for 
such modes are off-the-shelf multirotor UAS [26.61] 
equipped with GPS sensors or small fixed-wing air¬ 
craft [26.62] used for surveying. 

Autonomous 

Full autonomous mode relaxes the constraints from 
semiautonomous or automatic mode. Only mission 
goals are set, or high-level task allocation is handled 
by the operator, while the UAS navigates through the 
environment safely by itself. This includes global path 
planning, collision avoidance with both static and dy¬ 
namic objects and replanning where necessary. The 
main idea is that the UAS can be left alone while 
performing the task at hand, and, in case, warns the op¬ 
erator ahead of time if intervention becomes necessary. 

26.7.3 UAS Components 

Autonomous UAS require numerous components from 
various fields, which need to be carefully designed in 
order to address the challenges detailed above. We high¬ 
light important design considerations of building blocks 
first, and show implementation aspects for real systems. 
Figure 26.40 shows the components for a commonly 
used multirotor UAS. 

Components for Autonomous Flights 
The levels of autonomy defined in Sect. 26.7.2 require 
several components to be designed and to work with 
each other. These components are shown in Fig. 26.41 
and are described in the following: 

Perception and State Estimation. Perception for au¬ 
tonomous UAS mainly involves both localization of 
the UAS and sensing of its environment. For localiza¬ 
tion, commonly used satellite-based localization sys¬ 
tems such as GPS (GLONASS or GALILEO in the 
future) may not be accurate enough, especially dur¬ 
ing tasks that involve operation in close proximity to 
(man-made) structure, and certainly not indoors. Di- 
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rect view to satellites is obstructed in these cases, 
compromising localization accuracy. Also, it may be 
considered whether sole reliance on these services is 
acceptable, as authorities may deteriorate the accuracy 
on purpose (selected availability). Thus, additional sen¬ 
sors such as cameras or laser-rangefinders should be 
used onboard UAS in combination with simultaneous 
localization and mapping (SLAM), or visual or laser 
odometry algorithms, yielding additional localization 
information [26.63-67]. 

Another important requirement to enable au¬ 
tonomous flights on the levels defined above, is sensing 
or reconstruction of the environment in order to navi¬ 
gate around obstacles, and for sense-and-avoid maneu¬ 
vers with other air traffic. Obstacle sensing is especially 
important when navigating close to terrain at low al¬ 
titude. Satellite-based navigation in combination with 
terrain maps may not be accurate enough, outdated, 
and cannot handle dynamic obstacles. Again, onboard 
sensors such as cameras, laser-rangefinders, or even 
radar, in combination with appropriate algorithms need 
to be used to overcome the limitations mentioned be¬ 
fore [26.68,69]. 

Having information from a multitude of sensors, 
from IMU over cameras to satellite-based, intelligent 
sensor fusion methods are required, taking all informa¬ 
tion into account and yielding a best estimate of the 
state of the UAS and its environment. As such, percep¬ 
tion and state estimation are building blocks, providing 
essential information to the components described in 
the following. 

Control. As UAS typically present highly dynamic un¬ 
certain flight behavior, appropriate and robust control 
techniques are required across all levels of autonomy. 
This ranges from SAS used for supporting the pilot 
during manual operation, to higher level control such 
as waypoint following, and more advanced techniques 
such as trajectory tracking, and may include handling 
of failure situations. Cascaded control structures are 
commonly used for this task [26.70]: inner- or low- 
level control loops stabilize attitude dynamics (these 
may be cascaded already), while outer- or high-level 
control loops stabilize translational dynamics (veloc¬ 
ity or position). Employed control approaches range 
from simple P(I)D structures to more advanced tech¬ 
niques like model predictive control (MPC) [26.3]. 
During waypoint following, specified waypoints have 
to be reached and via-points have to be passed, while 
the velocity and attitude profile is left to the con¬ 
troller to optimize. For trajectory tracking in contrast, 
a tracking controller [26.71] has to follow specific pro¬ 
files for position and attitude dynamics. These profiles 
are usually planned with approaches presented below. 


and yield more smooth paths than simple waypoint 
following. 

Planning. Planning in the context of UAS depicts the 
process of planning appropriate waypoints or dynamic 
trajectories for a UAS depending on the mission at 
hand. This involves the latter two levels of autonomy 
described in the previous section. Requirements range 
from (dynamic) obstacle avoidance, taking obstacles 
and the dynamics of the UAS into account, to more 
sophisticated complete mission plans [26.72]. While 
taking into account vehicle dynamics is a straight¬ 
forward choice, additional constraints such as quality 
of state estimation along a path, or battery endurance 
may have to be considered as well. Furthermore, there 
may be additional objectives like area covered or, for in¬ 
stance, energy optimizations for solar airplanes in order 
to stay airborne 24 h or longer. 

While analytic approaches [26.73] can be used for 
simple planning tasks, random sampling-based plan¬ 
ners [26.74-76] dominate the literature in the area of 
path planning for UAS, due to their ability to cope with 
nonlinear vehicle dynamics and high-dimensional state 
spaces. Based on these planners, a number of success¬ 
ful approaches exist for UAS, even taking into account 
localization uncertainty [26.4,77-79]. 

Communication. Communication can be thought of 
the glue between the components mentioned above. 
While components have to communicate onboard 
across multiple computation devices, there is also the 
need of communication with a ground station, or with 
the operator remote control device, via radio-links. 
Here, the requirements in terms of range, delay, and 
transmission rate (commonly referred to as bandwidth) 
can vary greatly and should be tailored to the appli¬ 
cation: computationally heavy tasks, being offloaded 
to a ground station require high-bandwidth connec¬ 
tions, where either WiFi (IEEE 802.1 In or IEEE draft 
802.1 lac) or ultra wide band (UWB) techniques can be 
applied. However, a reasonable maximum range is in 
the order of 100 m. More autonomy means also less de¬ 
pendence on radio-links and realtime constraints. Mis¬ 
sions involving larger distances have to rely on longer 
range radio-links, even up to satellite links, but at the 
expense of bandwidth and delay or significant weight 
and power requirements. That radio-links should never 
be used inside any (real-time) critical control loops, 
goes without saying. 

Integration Onboard UAS 

The components identified above have different re¬ 
quirements in terms of real-time constraints and com¬ 
putational complexity. Instead of real time, it is more 
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Fig. 26.42 Example implementation of flight-relevant components 
onboard a multirotor UAS. All time-critical tasks such as con¬ 
trol and the prediction part of an extended Kalman filter (EKF) 
are performed on real-time microcontroller hardware, while com¬ 
putationally more demanding, but less time-critical, tasks (visual 
localization, EKF update) are processed on the onboard computer 
using a standard operating system 

appropriate to consider that a task has to be completed 
in time. As an example, vehicle control tasks have to be 
computed within a few milliseconds, but these task are 


computationally less complex. Path planning in con¬ 
trast is fairly complex, but it is usually perfectly fine 
if a path is computed within seconds, since planning 
horizons span much longer time. Perception and parts 
of state estimation lie in-between and need computa¬ 
tion times in the order of tens of milliseconds. This 
defines how, and on which computation devices these 
components should be implemented. An example for 
a multirotor UAS is shown in Fig. 26.42. The microcon¬ 
trollers on the flight-control-unit (FCU) are closest to 
inertial sensors and actuators, but least powerful. Con¬ 
trol loops and the prediction parts of the state estimator 
are implemented here, running at guaranteed rates on 
a real-time operating system, or even without. This re¬ 
laxes real-time constraints on the onboard computer, 
which can compute demanding tasks such as visual lo¬ 
calization and update steps for state estimation. Local 
planning tasks are computed here as well, while less 
critical parts can be offloaded to a ground station. 

Application Related Payload 
While there is a set of sensors required to enable 
autonomous navigation, which are referred to as naviga¬ 
tion sensors, additional sensors, or payload in general, 
need to be considered in the design phase. Not only in 
terms of payload, i. e., weight, but also such that the 
UAS is kept balanced. Furthermore, it has to be guar¬ 
anteed that the additional payload does not interfere 
(electromagnetically) with components in the critical 
stabilization loop. Applications and their required sen¬ 
sors are detailed in the next section. 


26.8 Applications of Aerial Robots 

From an application perspective, one may distinguish 
between UAS that mostly operate as remote controlled 
or semiautonomous systems (typically referred to as 
drones) and intelligent systems, robots that present 
advanced levels of autonomy. Drones are essentially 
tele-operated aircraft or systems capable of tracking 
predefined trajectories while they further integrate on¬ 
board sensors to provide situational awareness. Most 
often such situational awareness is visual (using optics) 
but can also include meteorological and environmen¬ 
tal tasks like hurricane-monitoring and chemical plume 
detection. As such, most drones fly high with a prede¬ 
termined and structured flight plan and mission profile. 
Drones will continue to be valuable assets and as fea¬ 
tured in daily media headlines world-wide, positively 
impact both civilian and military missions. 

Nowadays, a constant trend is to develop aerial 
robots of advanced intelligence. Machine cognition, 


perception and vehicle control algorithms work in con¬ 
cert to perform applications that go beyond just situ¬ 
ational awareness. Reaching new levels of autonomy, 
these robots are designed to handle unforeseen events, 
interact with their environment, and adapt to a broad 
range of scenarios. In essence, drones were in their 
vast majority passive , providing eyes on scene whereas 
modern aerial robots tend to become active, allow¬ 
ing their users to engage the scene, act autonomously 
and possibly interact with surroundings. Figure 26.43 
presents indicative examples of current and emerging 
applications. 

26.8.1 Demonstrated Applications of UAS 

The Flandbook’s first edition listed eight categories of 
possible applications. Since then, all these applications 
have been realized, albeit with various levels of ma- 
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Fig. 26.43 Indicative demonstrated and emerging applications of aerial robotics 


turity. UAS (both drones and aerial robots) have been 
deployed in: 

1. Remote sensing 

2. Disaster response 

3. Surveillance 

4. Search and rescue 

5. Image acquisition 

6. Communications 

7. Transportation 

8. And payload delivery. 

The first five categories broadly fall into the area 
of reconnaissance, surveillance, and target acquisition 
(RSTA). Drones have continued to successfully per¬ 
form RSTA-based tasks like volcanic sampling, damage 
assessment, border patrol, and cinematography. In con¬ 
trast, the latter three categories are being realized with 
advanced aerial robots. Equipped with computational 
intelligence aerial robots can flock. Aerial robots can 
also airlift, maneuver and interact in near-Earth envi¬ 
ronments like in-and-around buildings, forests, caves 
and tunnels. As such current trends in aerial robotics 
push the application envelope beyond RSTA. The eight 
categories are now revisited with context on the ex¬ 


pected breakthroughs to come while the topic of visual 
inspection for industrial purposes is separately dis¬ 
cussed: 

• Remote sensing: Drones are already used for 
pipeline spotting, power line monitoring, volcanic 
sampling, mapping, meteorology, geology, agri¬ 
culture and unexploded mine detection. Advanced 
aerial robots will be able to conduct pipeline risk as¬ 
sessment and repair, power line maintenance, real¬ 
time mapping, crop care and mine defusing. 

• Disaster response: Drones are also used for chem¬ 
ical sensing, flood monitoring, and wildfire man¬ 
agement. Advanced aerial robots will be able to 
conduct infrastructure repair, flood mitigation, and 
wildfire fighting. 

• Surveillance: Drones are employed for law en¬ 
forcement, traffic monitoring, coastal and maritime 
patrol, and border patrols. Advanced aerial robots 
will be able to accomplish tasks like crowd control, 
traffic redirection, and inspection of maritime and 
trucking containers. 

• Search and rescue: The vision of using drones for 
search and rescue operations within low-density or 
hard-to-reach areas has already become a reality. 
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Advanced aerial robots will be able to go beyond 
casualty extraction by assessing care and delivering 
first-aid support. 

• Visual inspection: For quite some time, drones are 
utilized to provide direct visual feeds to ground 
operators inspection industrial and civil structures. 
Future aerial robots will be able to act as a new high 
fidelity autonomous inspection tools that could also 
be capable of conducting maintenance work tasks. 

• Transportation: Drones are already used for small- 
and large-cargo transport, and possibly passenger 
transport. Advanced aerial robots will be able to 
conduct in-flight refueling of other aircraft, pick-up 
and drop-off of cargo as well as loading and extrac¬ 
tion of casualties. 

• Communications: The use of drones as permanent 
or ad hoc communication relays for voice and data 
transmission, as well as broadcast units for tele¬ 
vision or radio is currently a reality. Advanced 
aerial robots will be able to conduct perch-and- 
stare to serve as bug on the wall listening devices, 
perch-and-stare to harvest energy from power lines, 
flocking to establish networks especially in areas 
with degraded communications. 

• Payload delivery: Drones are already employed for 
firefighting or crop dusting. Advanced aerial robots 
will be able to accomplish dexterous manipulation 
of payloads like tools for repairing structures, deliv¬ 
ering, and insertion of logistics as well as handling 
crops. 

• Image acquisition: Cinematography or real-time 
entertainment is another relatively established use- 
case of drones. Advanced aerial robots will be able 
to conduct acting as pixels and operate in formation 
flight to serve as physical displays in the air. 

26.8.2 Current Applications and Missions 

Since the Handbook’s first edition, media headlines 
continue to document UAS and their impact in all the 
aforementioned application categories. No longer are 
such applications notional but are rather considered as 
routine. The following provides recent state-of-the-art 
examples in each category to underscore this message. 

Drone usage for RSTA-based military operations 
is routine. Tracking targets and, in growing instances, 
destroying them, is commonly performed. But also non¬ 
military missions in remote sensing, disaster response, 
and surveillance are becoming routine. Beyond image 
acquisition, drones are routinely used to gather me¬ 
teorological data. NASA and NOAA regularly deploy 
drones for real-time monitoring of hurricanes. On¬ 
board sulfur dioxide sensors gather airborne samples 
from volcanic plumes. Drones are frequently deployed 


after disasters: in Fukushima (2011) and Hurricane 
Sandy (2012) aerial images were gathered to assess 
building damage. Monitoring of maritime piracy and 
cartel drug-trafficking also underscore the usefulness of 
drones to persistently survey using high-definition cam¬ 
eras and night vision systems. 

Currently, drones are deployed to open and/or re¬ 
stricted areas where airborne collision risk is reduced. 
The need for situational awareness will however con¬ 
tinue to be fulfilled by UAS. The demand for more 
data will push the development of newer drones. As 
such, well-known and cutting-edge aerodynamic design 
principles will be applied so that UAS fly longer and 
farther and incorporate different flight modalities like 
vertical take-off and landing (VTOL). Sensor suites will 
also grow in sophistication to collect higher-resolution 
and/or multispectral data. Propulsion technologies will 
also be engineered to enable UAS to carry more sen¬ 
sors. The net effect is that as missions become more 
routine and frequent, the variations of drones and their 
performance will grow to keep up with expanding 
RSTA-based needs. 

26.8.3 Aerial Robots: Emerging Categories 

The road ahead is full of challenging emerging appli¬ 
cations that will benefit from the utilization of aerial 
robotic technologies. This is especially the case when 
one considers near-Earth environments like in-and- 
around buildings, through forests and down tunnels and 
caves. Beyond RSTA, such environments provide op¬ 
portunities for aerial robots to dexterously interact with 
objects. Today’s drones release retardants over wildfires 
but tomorrow’s aerial robots would attach hoses and 
breach walls for firefighting. Interaction demands ad¬ 
vances in areas like aircraft design (flight modalities, 
payload capability), algorithms (perception, control, 
motion planning, and grasping) and manipulators (arms 
and end-effectors). The latter two areas overlap with 
the greater domain of robotics research. As deeper un¬ 
derstanding and realizations develop, the application 
space, demand and impact of aerial robots will grow 
even greater. In the following, a brief overview of excit¬ 
ing emerging applications will be listed. 

Assembly Work Task Execution 
One notional concept employs aerial robots to assembly 
physical structures. Such assembly pushes the bound¬ 
aries of the categories of transportation and payload 
delivery. Labs like those of University of Pennsylva¬ 
nia [26.80], Switzerland (ETH Zurich [26.81]) have 
used gripper-mounted quadrotors to illustrate proof- 
of-concept. Demonstrations include the pickup and 
drop-off of workpieces to defined locations. These 
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workpieces self-connect using magnets or premachined 
joints but illustrate capabilities to coordinate multi¬ 
ple aircraft, mitigate disturbances like ground effect, 
sense and map, and execute agile maneuvers. Groups 
at Drexel, Twente, Seville, and Yale are equipping 
rotorcraft robots with manipulators to illustrate more 
dexterous tasks like inserting, screwing, and remov¬ 
ing workpieces. Assembly and in general physical 
work-task execution will find applications in categories 
like disaster response (repairing structures) and remote 
sensing (inserting sensors). 

Inspection Through Contact 
Inspecting containers at docks, ships, and border cross¬ 
ings is a daunting task. Sensor technology is advanc¬ 
ing to look inside such containers. However, dexterous 
aerial robots that can open hatches and trunks could also 
yield many applications. Inspection as an emerging cate¬ 
gory can employ aerial robots to physically interact with 
structures. Aerial robots can physically probe bridges, 
pipelines, and power lines to repair and replace parts. 
Interactive inspection could emerge to applications like 
crop handling and mine defusing. Labs like those of ETH 
Zurich [26.82] and the University of Bologna [26.83] 
have presented relevant results in the topic. 

Innovative Media 

Both the number of vendors and affordability of rotor- 
craft robots have increased since the Handbook’s first 
edition. In mid-2000, it was not surprising to see a com¬ 
mercial quadrotor costing 40 000 USD. Today, there 
are dozens of companies providing robotic rotorcraft 
ranging in price (hundreds to under 10 000 USD), size 
(from pucks to bike wheels) and configurations (mono, 
co-axial, and multirotor). Such range will likely yield 
innovative and entrepreneurial applications. 

The Firefly project at the MIT SENSEable Lab per¬ 
haps best illustrates the notion of coordinating multiple 
aerial robots to display images. Each aerial robot acts 
as flying pixel and maneuver into position to form 2 
or 3-D images. Beyond visual and dynamic art, such 
capabilities could yield airborne displays for tasks like 
crowd control, traffic redirection and SOS signals. Air¬ 
borne displays could also expand creative expression to 
enhance concerts, advertise at stadiums, and inform au¬ 
diences. 

Autonomous Structural Inspection and 3-D 
Reconstruction 

From the perspective of the role of aerial robotics 
in accelerating growth, mapping of areas, structural 
inspection of buildings and infrastructure as well as 
recognition of objects or areas of interest is among the 
most important application fields. Aerial robots have 


achieved great milestones in such scenarios. Among 
others, recently researchers managed to extract a high- 
fidelity 3-D model of the Swiss mountain top Mat¬ 
terhorn utilizing a small FW-UAS and sophisticated 
robotic vision techniques [26.84], other efforts led to 
the inspection of a real power plant boiler using a mul¬ 
tirotor vehicle and tightly integrated visual-inertial al¬ 
gorithms [26.85], while in terms of industrial adoption 
aerial robots have provenly minimized the inspection 
times of power infrastructure [26.86] and are often 
used in combination with geographic information sys¬ 
tem (GIS) data [26.84], It is worth noting that some 
of the largest asset owners and service providers in the 
field of civil infrastructure and civil engineering works 
have shown interest and participate in large consortia 
that aim to make such aerial robotic technologies an 
integral and game-changing factor of how things are 
done (see e.g., Petrobot Project [26.87], ARCAS: Aerial 
Robotics Cooperative Assembly System [26.88], Eu- 
RoC: European Robotics Challenges [26.89], ARGOS 
Challenge [26.90].). 

Precision Agriculture 

Precision agriculture is considered to be among the 
fields where the use of aerial robots may become a key 
factor boosting growth and improving the production 
quality. The Japanese farming sector has historically 
been in a position to lead the relevant research efforts 
with successful designs like the Yamaha RMAX being 
extensively used for monitoring as well as spraying op¬ 
erations [26.5]. Already in December 2002, 1687 aerial 
robots were used in Japan to conduct precision agri¬ 
culture operations. However, current research contri¬ 
butions and pioneering technology early adopters have 
opened an even more promising channel for the widest 
possible utilization of aerial robotic technologies for the 
benefit of agriculture production. Mainly through the 
use of miniaturized aerial robotics at farms of smaller 
scale and via the integration of advanced multispec- 
tral perception sensors (i. e., NDVI) [26.91], accurate 
mapping and analysis regarding the quality of the field, 
the level of plant growth and existence of diseases 
becomes possible. Such technologies are expected to 
become a critical tool to optimize and enhance agri¬ 
cultural services. Furthermore, manipulators-equipped 
aerial robots will be able to autonomously physically 
act based on their perception of the field and minimize 
the time and effort required for several agriculture ser¬ 
vices and tasks. 

26.8.4 Open Issues 

Emerging categories underscore both the distinction 
between more traditional drones and emerging aerial 
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robotic technologies as well as how much further the 
state-of-the-art has to be pushed. Clearly, today’s UAS 
have limited payloads but adding manipulators goes be¬ 
yond lift issues. When interacting with objects, aerial 
robots must handle reaction forces and torques. Under¬ 
standing such reactions remains an open issue. How¬ 
ever, both space and underwater roboticists have looked 
at dexterous manipulators for tasks like satellite repair 
and turning valves on ocean-based oil rigs. As research 
clarifies this issue and as lift capacities increase, dexter¬ 
ous aerial robots will be realized. 

Airspace access is an often cited issue that may limit 
aerial robot development and vast utilization. A US 
congressional bill was passed in 2012 that sets require¬ 
ments for UAS to fly in the national airspace by 2015. 
This has also sparked competition among a dozen or 


so states to establish UAS flight testing sites. In Eu¬ 
rope, similar events have happened and there are UAS 
test sites in places like Finland (Kemijarvi), Sweden 
(NEAT), and Wales (Parc Aberporth). 

Lastly, somewhat ironic is that today’s unmanned 
drones require a crew of highly skilled operators. In 
the case of some Predator missions, crew sizes can be 
up to a dozen people. Also ironic is that human er¬ 
ror is the most cited cause for drone accidents. As the 
number of UAS in the national airspace increases, the 
need for even more operators will also grow. This has 
the potential to raise the risk of UAS-related accidents. 
The issues of effective UAV pilot training, certifying 
operators, handling emergency landings, and sharing 
airports with manned aircraft will also emerge as press¬ 
ing ones. 


26.9 Conclusions and Further Reading 


Design of aerial robots requires background knowl¬ 
edge in a multitude of subjects, from aerodynam¬ 
ics to dynamics, control, and system integration: 
we have overviewed the relevant basics along with 
analytical tools and guidelines to go through the 
stages of designing, modeling, and setting up oper¬ 
ation of various types of unmanned aerial systems 
(UAS). An emphasis was given on custom tailor¬ 
ing a system to a specific application, in order to 


optimally meet related requirements in terms of en¬ 
durance, range, agility, size, complexity, as well as 
from a system integration point of view. The com¬ 
pilation at hand shall serve as a starting point, fur¬ 
ther motivating the reader to study the various fields 
with their related literature, ranging from aircraft and 
system design to the classical autonomous robotics 
challenges involving perception, cognition and motion 
control. 
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Delfly II in hover 

available from http://handbookofrobotics.org/view-chapter/26/videodetails/493 
AtlantikSolar field-trials 

available from http://handbookofrobotics.org/view-chapter/26/videodetails/602 
senseSoar UAV Avionics testing 

available from http://handbookofrobotics.org/view-chapter/26/videodetails/603 
Structural inspection path planning via iterative viewpoint resampling 
with application to aerial robotics 

available from http://handbookofrobotics.org/view-chapter/26/videodetails/604 
sFly: Visual-inertial SLAM for a small helicopter in large outdoor environments 
available from http://handbookofrobotics.org/view-chapter/26/videodetails/688 
UAV stabilization, mapping & obstacle avoidance using Vl-sensor 
available from http://handbookofrobotics.org/view-chapter/26/videodetails/689 
Project Skye - autonomous blimp 

available from http://handbookofrobotics.org/view-chapter/26/videodetails/690 
Flight stability in aerial redundant manipulators 

available from http://handbookofrobotics.org/view-chapter/26/videodetails/693 
The astounding athletic power of quadcopters 

available from http://handbookofrobotics.org/view-chapter/26/videodetails/694 
Robots that fly ... and cooperate 

available from http://handbookofrobotics.org/view-chapter/26/videodetails/695 
A robot that flies like a bird 

available from http://handbookofrobotics.org/view-chapter/26/videodetails/696 
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IdaMESiESi Robotic insects make first controlled flight 

available from http://handbookofrobotics.org/view-chapter/26/videodetails/697 

Towards valve turning using a dual-arm aerial manipulator 

available from http://handbookofrobotics.org/view-chapter/26/videodetails/719 
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27. Micro-/Nanorobots 


Bradley J. Nelson, Lixin Dong, Fumihito Arai 


The field of microrobotics covers the robotic ma¬ 
nipulation of objects with dimensions in the 
millimeter to micron range as well as the design 
and fabrication of autonomous robotic agents that 
fall within this size range. Nanorobotics is defined 
in the same way only for dimensions smaller than 
a micron. With the ability to position and orient 
objects with micron- and nanometer-scale di¬ 
mensions, manipulation at each of these scales 
is a promising way to enable the assembly of 
micro- and nanosystems, including micro- and 
nanorobots. 

This chapter overviews the state of the art of 
both micro- and nanorobotics, outlines scaling 
effects, actuation, and sensing and fabrica¬ 
tion at these scales, and focuses on micro- and 
nanorobotic manipulation systems and their ap¬ 
plication in microassembly, biotechnology, and 
the construction and characterization of micro 
and nanoelectromechanical systems (MEMS/NEMS). 
Material science, biotechnology, and micro- and 
nanoelectronics will also benefit from advances in 
these areas of robotics. 
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27.1 Overview of Micro- and Nanorobotics 

Progress in robotics over recent years has dramatically and manipulate the world on a variety of scales extend- 
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of the sea, down to individual atoms (Fig. 27.1). At 
the lower end of this scale, technology has been mov¬ 
ing toward greater control of the structure of matter, 
suggesting the feasibility of achieving thorough control 
of the molecular structure of matter atom by atom, as 
Feynman first proposed in 1959 in his prophetic article 
on miniaturization [27.1]: 

What I want to talk about is the problem of manip¬ 
ulating and controlling things on a small scale... I 
am not afraid to consider the final question as to 
whether, ultimately — in the great future — we can 
arrange the atoms the way we want: the very atoms, 
all the way down! 

He asserted that 

At the atomic level, we have new kinds of forces 
and new kinds of possibilities, new kinds of effects. 
The problems of manufacture and reproduction of 
materials will be quite different. The principles of 
physics, as far as I can see, do not speak against the 
possibility of maneuvering things atom by atom. 

This technology is now labeled nanotechnology. 

The great future of Feynman began to be realized 
in the 1980s. Some of the capabilities he dreamed 
of have been demonstrated, while others are being 
actively pursued. Feynman foresaw the possibility of 
employing a microrobotic manipulator ( a master-slave 
system) for bottom-up manufacturing (manipulation, 
assembly, etc.) of minute machines; one such device 
he described as swallowing the surgeon, which he at¬ 
tributed to his friend Albert R. Hibbs. He also imagined 
we could build a billion tiny factories, models of each 
other, which are manufacturing simultaneously, drilling 
holes, stamping parts, and so on [27.1]. Micro- and 
nanorobotics research has progressed from these see m - 
ingly far-out concepts of the 1960s and 1970s to reality 
when microelectromechanical systems (MEMSs) began 
to emerge in the late 1980s. These building blocks took 
the form of surface-micromachined micromotors and 
microgrippers made of polysilicon fabricated on a sili¬ 
con chip [27.2], In the late 1980s and early 1990s, more 
concrete suggestions on how one could realize MEMS- 
based microrobotic devices using such micromotors as 
well as potential applications were published [27.3,4], 
Today, a variety of microrobotic devices are enabling 
new applications in various fields. 

In industry, interesting areas for microrobotics in¬ 
clude assembly [27.5,6], characterization, inspection 
and maintenance [27.7, 8], microoptics (positioning of 
microoptical chips, microlenses and prisms) [27.9], and 
microfactories [27.10]. Many of these applications re¬ 
quire automated handling and assembly of small parts 
with accuracy in the submicron range. 


Other important fields include biology (manipula¬ 
tion, capturing, sorting and combining cells [27.11]) 
and medical technology [27.12,13]. In surgery, the 
use of steerable catheters and endoscopes is very at¬ 
tractive and the development of increasingly small 
microrobotic devices is rapidly progressing. Wireless 
untethered microrobots that will explore and repair 
our bodies ( swallowing the surgeon ) appear to be 
simply a matter of time. In fact, endoscopy using 
wireless capsules (camera pills) are already on the 
market and allow for endoscopic imaging of the en¬ 
tire gastrointestinal tract [27.14], something currently 
not possible using standard scopes. Magnetic steer¬ 
ing or crawling-type motions serve as promising ways 
for such devices to locomote in a controlled fash¬ 
ion [27.15]. Doctors could steer pill-mounted cameras 
and other actuators to areas of interest for visual in¬ 
vestigation and biopsies beyond the range of current 
endoscopes. 

Nanorobotics represents the next stage in mini¬ 
aturization for maneuvering nanoscale objects. 
Nanorobotics is the study of robotics at the nanometer 
scale, and includes robots that are nanoscale in size, 
i. e., nanorobots, and large robots capable of manip¬ 
ulating objects that have nanometer dimensions with 
nanometer resolution, i. e., nanorobotic manipulators. 
The field of nanorobotics brings together several dis¬ 
ciplines, including nanofabrication processes used for 
producing nanoscale robots, nanoactuators, nanosen¬ 
sors, and physical modeling at nanoscales. Nanorobotic 
manipulation technologies, including the assembly of 
nanometer-sized parts, the manipulation of biological 
cells or molecules, and the types of robots used to 
perform these types of tasks also form a component of 
nanorobotics. 

As the 21st century unfolds, the impact of nanotech¬ 
nology on the health, wealth, and security of humankind 
is expected to be at least as significant as the combined 
influences in the 20th century of antibiotics, the inte¬ 
grated circuit, and human-made polymers. For example, 
N. Lane stated in 1998, [27.16] 

If I were asked for an area of science and engineer¬ 
ing that will most likely produce the breakthroughs 

of tomorrow, I would point to nanoscale science and 

engineering. 

The great scientific and technological opportunities 
nanotechnology presents have stimulated extensive ex¬ 
ploration of the nanoworld and initiated an exciting 
worldwide competition, which has been accelerated 
by the publication of the National Nanotechnology 
Initiative by the US government in 2000 [27.17]. 
Nanorobotics will play a significant role as an enabling 
nanotechnology and could ultimately be a core part of 
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Fig. 27.1 Robotic exploration at micro- and nanometer scales 

nanotechnology if Drexler ’s machine-phase nanosys¬ 
tems based on self-replicative molecular assemblers via 
mechanosynthesis can be realized [27.18]. 

By the early 1980s, scanning tunneling microscopes 
(STMs) [27.19] radically changed the way in which 
we interacted with and even regarded single atoms and 
molecules. The very nature of proximal probe methods 
encourages the exploration of the nanoworld beyond 
conventional microscopic imaging. Scanning probes 
now allow us to perform engineering operations on 
single molecules, atoms, and bonds, thereby providing 
a tool that operates at the ultimate limits of fabrication. 
They have also enabled exploration of molecular prop¬ 
erties on an individual nonstatistical basis. 

STMs and other nanomanipulators are nonmolec- 
ular machines that use bottom-up strategies. Although 
performing only one molecular reaction at a time is 
obviously impractical for making large amounts of 
a product, it is a promising way to provide the next gen¬ 
eration of nanomanipulators. Most important, it is pos¬ 
sible to realize the directed assembly of molecules or 
supermolecules to build larger nanostructures through 
nanomanipulation. The products produced by nanoma¬ 
nipulation could be the first step of a bottom-up strategy 


in which these assembled products are used to self- 
assemble into nanomachines. 

One of the most important applications of 
nanorobotic manipulation will be nanorobotic assem¬ 
bly. However, it appears that, until assemblers capable 
of replication can be built, the combination of chemical 
synthesis and self-assembly are necessary when start¬ 
ing from atoms; groups of molecules can self-assemble 
quickly due to their thermal motion, enabling them to 
explore their environments and find (and bind to) com¬ 
plementary molecules. Given their key role in natural 
molecular machines, proteins are obvious candidates 
for early work in self-assembling artificial molecular 
systems. Degrado [27.20] demonstrated the feasibil¬ 
ity of designing protein chains that predictably fold 
into solid molecular objects. Progress is also being 
made in artificial enzymes and other relatively small 
molecules that perform functions like those of natu¬ 
ral proteins. Several bottom-up strategies using self- 
assembly appear feasible [27.21]. Chemical synthesis, 
self-assembly, and supramolecular chemistry make it 
possible to provide building blocks at relatively large 
sizes beginning from the nanometer scale. Nanorobotic 
manipulation serves as the base for a hybrid approach to 
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construct nanodevices by structuring these materials to 
obtain building blocks and assembling them into more 
complex systems. 

Despite the claims of many futurists, the form 
nanorobots of the future will take and what tasks 
they will actually perform remain unclear. However, 
it is clear that nanotechnology is progressing towards 
the construction of intelligent sensors, actuators, and 
systems that are smaller than 100 nm. These nanoelec¬ 
tromechanical systems (NEMSs) will serve as both the 
tools to be used for fabricating future nanorobots as 
well as the components from which these nanorobots 
may be developed. Shrinking device size to these di¬ 
mensions presents many fascinating opportunities such 
as manipulating nano-objects with nanotools, mea¬ 
suring mass in femtogram ranges, sensing forces at 
piconewton scales, and inducing gigahertz motion, 

27.2 Scaling 

27.2.1 The Size of Things 

Things we can potentially observable range from 
10“ 35 m (the Planck length) to 10 26 m (the radius of the 
observable universe). A nanometer, 10“ 9 m, is about 
ten times the size of the smallest atoms, such as hy¬ 
drogen and carbon, while a micron is barely larger than 
the average wavelength of visible light, thus invisible to 
the human eye. A millimeter, the size of a pinhead, is 
roughly the smallest part typically fabricated using tra¬ 
ditional machining techniques. The range of scales from 
millimeters to nanometers is one million (Fig. 27.1), 
which is also about the range of scales in present-day 
mechanical technology from the largest skyscrapers to 
the smallest conventional mechanical machine parts. 
The vast opportunity to make new machines spanning 
almost six orders of magnitude from 1 mm to 1 nm, is 
one take on Feynman 's famous statement, there is plenty 
of room at the bottom [27.1]. If L is taken as a typical 
length, 0.1 nm for an atom, perhaps 2 m for a human, 
this scale range in L would be 2 x 10 10 . If the same scale 
range were to apply to an area, 0.1 nm x 0.1 nm versus 
2 m x 2 m, the scale range for area L 2 is 4 x 10 2 °. Since 
a volume L 3 is enclosed by sides L, we can see that the 
number of atoms of size 0.1 nm in a (2 m 3 ) volume is 
about 8 x 10 3 °, recalling that Avogadro’s number Na = 
6.022 x 10 23 is the number of atoms in a gram-mole, 
supposing that the atoms were 12 C, molar responding 
to a density 1.99 x 10 4 kg/m 3 . A primary working tool 
of the nanotechnologist is facility in scaling the mag¬ 
nitudes of various properties of interest, as the length 
scale L shrinks, e.g., from 1 mm to 1 nm. 


among other new possibilities waiting to be discov¬ 
ered. These capabilities will, of course, drive the 
tasks that future nanorobots constructed by and with 
NEMS will perform. NEMS and the components of 
nanorobots will be the products of nanorobotic manip¬ 
ulation. Large nanorobotic manipulators will be able 
to shrink in size due to this development, thus en¬ 
abling nanosized robotic manipulators and other forms 
of nanorobots. All of these form the scope of the area 
of nanorobotics. 

This chapter focuses on micro- and nanorobotics in¬ 
cluding actuation, manipulation, and assembly at the 
micro- and nanoscale. The main goal of these fields of 
robotics is to provide an effective technology for the 
experimental exploration of the micro- and nanoworld, 
and to push the boundaries of this exploration from 
a robotics research perspective. 


Clearly, the number of atoms in a device scales 
as L 3 . If a transistor on the micron scale contains 10 12 
atoms, then on the nanometer scale, If /L = 10~ 3 it will 
contain 1000 atoms, likely too few to preserve its func¬ 
tion. 

Normally, we will think of scaling as an isotropic 
scale reduction in three dimensions. However, scaling 
can be thought of usefully when applied only to one 
or two dimensions, scaling a cube to a two-dimensional 
(2-D) sheet of thickness a or to a one-dimensional ( 1 -D) 
tube or nanowire of cross-sectional area a 2 . The term 
zero-dimensional (0-D) is used to describe an object 
small in all three dimensions, having volume a 3 . In elec¬ 
tronics, a zero-dimensional object (a nanometer-sized 
cube a 3 of semiconductor) is called a quantum dot (QD) 
or artificial atom because its electronic states are few 
and sharply separated in energy, and thus resemble the 
electronic states of an atom. 

27.2.2 Predominate Physics 

at the Micro- and Nanoscales 

The predominate physics at the micro- and nanoscale 
can be dramatically different from at the macroscale. 
Surface and intermolecular forces, such as adhesion 
forces originating from surface tension forces, van 
der Waals forces, and electrostatic forces, become 
more significant than volumetric forces such as grav¬ 
itational forces for objects with sizes well below 
1000 pm [27.22], Although the laws of classical New¬ 
tonian physics may well suffice to describe changes in 
behavior down to 10 nm (100 A), the range of scaling 
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is tremendous. Therefore, the changes in magnitudes of 
many important physical properties, such as resonant 
frequencies, are so great, that completely different ap¬ 
plications may appear. 

The more challenging question for the nanotechnol¬ 
ogist is to understand and hopefully to exploit those 
changes in physical behavior that occur at the end 
of the classical scaling range. The end of the scal¬ 
ing is the size scale of atoms and molecules, where 
nanophysics [27.23] is the proven conceptual replace¬ 
ment of the laws of classical physics. Modern physics, 
which includes quantum mechanics as a description of 
matter on a nanometer scale, is a very well-developed 


and proven subject whose application to real situations 
is limited only by modeling and computational compe¬ 
tence. 

In the modern era, simulations and approximate 
solutions increasingly facilitate the application of 
nanophysics to almost any problem of interest. Many 
central problems are already (adequately, or more 
than adequately) solved in the extensive literatures 
of theoretical chemistry, biophysics, condensed matter 
physics, and semiconductor device physics. The practi¬ 
cal problem is to find the relevant work, and, frequently, 
to convert the notation and units systems to apply the 
results to the problem at hand. 


27.3 Actuation at the Micro- and Nanoscales 


The positioning of nanorobots and nanorobotic ma¬ 
nipulators depends largely on nanoactuators. While 
nanosized actuators for nanorobots are still under 
exploration and relatively far from implementation, 
MEMS-based efforts are focused on shrinking their 
sizes [27.24]. Nanometer resolution motion has been 
extensively investigated and can be generated using 
various actuation principles. Electrostatics, electromag¬ 
netics, and piezoelectrics are the most common ways 
to realize actuation at nanoscales. For nanorobotic ma¬ 
nipulation, besides nanoresolution and compact sizes, 
actuators generating large strokes and high forces are 
best suited for such applications. The speed criteria are 
of less importance as long as the actuation speed is in 
the range of a couple of hertz and above. Table 27.1 
provides a small selection of early works on actua¬ 
tors [27.25-30] suitable for micro- and nanorobotic 
applications (partially adapted from [27.24]). 

Several extensive reviews on various actuation prin¬ 
ciples have been published [27.4,31-34], During the 
design of an actuator, the trade-offs among range of 
motion, force, speed (actuation frequency), power con¬ 
sumption, control accuracy, system reliability, robust¬ 
ness, load capacity, etc. must be taken into considera¬ 

Table 27.1 Actuation with MEMS 


tion. This section reviews basic actuation technologies 
and potential applications at nanometer scales. 

27.3.1 Electrostatics 

Electrostatic charge arises from a build up or deficit of 
free electrons in a material, which can exert an attractive 
force on oppositely charged objects, or a repulsive force 
on similarly charged objects. Since electrostatic fields 
arise and disappear rapidly, such devices will likewise 
demonstrate very fast operation speeds and be little af¬ 
fected by ambient temperatures. 

Previous investigations have produced many ex¬ 
amples of miniature devices using electrostatic force 
for actuation including silicon micromotors [27.35, 36], 
microvalves [27.37], and microtweezers [27.38]. This 
type of actuation is important for achieving nanoscale 
actuation. 

Electrostatic fields can exert great forces, but gener¬ 
ally across very short distances. When the electric field 
must act over larger distances, a higher voltage will be 
required to maintain a given force. The extremely low 
current consumption associated with electrostatic de¬ 
vices makes for highly efficient actuation. 


Actuation 

Type of 

Volume 

Speed 

Force 

Stroke 

Resolution 

Power density 

Reference 

principle 

motion 

(mm 3 ) 

(s" 1 ) 

(N) 

(m) 

(m) 

(W/m 3 ) 


Electrostatic 

Linear 

400 

5000 

1 x 10 -7 

6 x 10“ 6 

NA 

200 

[27.25] 

Magnetic 

Linear 

0.4 x 0.4 x 0.5 

1000 

2.6 x 10 -6 

1 x 10 -4 

NA 

3000 

[27.26] 

Piezoelectric 

Linear 

25.4x12.7x1.6 

4000 

350 

1 x 10 -3 

7x 10 -8 

NA 

[27.27] 

Actuation 

Type of 

Volume 

Speed 

Torque 

Stroke 

Resolution 

Power density 

Reference 

principle 

motion 

(mm 3 ) 

(rad/s) 

(Nm) 

(rad) 

(rad) 

(W/m 3 ) 


Electrostatic 

Rotational 

tr/4 x 0.5 2 x 3 

40 

2 x 10“ 7 

2 71 

NA 

900 

[27.28] 

Magnetic 

Rotational 

2 x 3.7 x 0.5 

150 

1 x 10 -6 

2 7t 

5/36jt 

3000 

[27.29] 

Piezoelectric 

Rotational 

ir/4x 1.5 2 x 0.5 

30 

2 x 10“ 11 

0.7 

NA 

NA 

[27.30] 
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Table 27.2 Comparison of nano actuators 


Method 

Efficiency 

Speed 

Power 

density 

Electrostatic 

Very high 

Fast 

Low 

Electromagnetic 

High 

Fast 

High 

Piezoelectric 

Very high 

Fast 

High 

Thermomechanical 

Very high 

Medium 

Medium 

Phase change 

Very high 

Medium 

High 

Shape memory 

Low 

Medium 

Very high 

Magnetostrictive 

Medium 

Fast 

Very high 

Electrorheological 

Medium 

Medium 

Medium 

Electrohydrodynamic 

Medium 

Medium 

Low 

Diamagnetism 

High 

Fast 

High 


27.3.2 Electromagnetics 


Electromagnetism arises from electric current moving 
through a conducting material. Attractive or repulsive 
forces are generated adjacent to the conductor and pro¬ 
portional to the current flow. Structures can be built 
which gather and focus electromagnetic forces, and har¬ 
ness these forces to create motion. 

Electromagnetic fields arise and disappear rapidly, 
thus permitting devices with very fast operation speeds. 
Since electromagnetic fields can exist over a wide range 
of temperatures, performance is primarily limited by the 
properties of the materials used in constructing the ac¬ 
tuator. 

One example of a microfabricated electromagnetic 
actuator is a microvalve which uses a small electro¬ 
magnetic coil wrapped around a silicon micromachined 
valve structure [27.39]. The downward scalability of 
electromagnetic actuators into the micro- and nano¬ 
realm may be limited by the difficulty of fabricating 
small electromagnetic coils. Furthermore, most electro¬ 
magnetic devices require perpendicularity between the 
current conductor and the moving element, presenting 
a difficulty for planar fabrication techniques commonly 
used to make silicon devices. 

An important advantage of electromagnetic devices 
is their high efficiency in converting electrical energy 
into mechanical work. This translates into less current 
consumption from the power source. 


27.3.3 Piezoelectrics 

Piezoelectric motion arises from the dimensional 
changes generated in certain crystalline materials when 
subjected to an electric field or to an electric charge. 
Structures can be built which gather and focus the 
force of the dimensional changes, and harness them to 
create motion. Typical piezoelectric materials include 
quartz (SiCT), lead zirconate titanate (PZT), lithium 
niobate, and polymers such as polyvinylidene fluoride 
(PVDF). 

Piezoelectric materials respond very quickly to 
changes in voltages and with great repeatability. They 
can be used to generate precise motions with repeatable 
oscillations, as in quartz timing crystals used in many 
electronic devices. Piezoelectric materials can also act 
as sensors, converting tension or compression strains to 
voltages. 

On the microscale, piezoelectric materials have 
been used in linear inchworm drive devices and mi¬ 
cropumps [27.40]. STMs and most nanomanipulators 
use piezoelectric actuators. 

Piezoelectric materials operate with high force and 
speed, and return to a neutral position when unpow¬ 
ered. They exhibit very small strokes (under 1 %). 
Alternating electric currents produce oscillations in the 
piezoelectric material, and operation at the sample’s 
fundamental resonant frequency produces the largest 
elongation and highest power efficiency [27.41]. Piezo¬ 
electric actuators working in the stick-slip mode can 
provide millimeter to centimeter strokes. Most com¬ 
mercially available nanomanipulators adopt this type 
of actuators, such as Picomotors from New Focus and 
Nanomotors from Klock. 

27.3.4 Other Techniques 

Other techniques include thermomechanical, phase 
change, shape memory, magnetostrictive, electrorhe- 
ological, electrohydrodynamic, diamagnetism, magne¬ 
tohydrodynamic, shape changing, polymers, and bi¬ 
ological methods (living tissues, muscle cells, etc.). 
Table 27.2 compares these techniques. 


27.4 Imaging at the Micro- and Nanoscales 


A brief overview is given of common imaging tools 
used in the research area of micro- and nanorobotics 
including optical, electron, and scanning probe mi¬ 
croscopy. The application and integration of these tools 
for micro- and nanorobotics are discussed in separate 
sections. 


In selecting a proper tool for imaging/sensing at 
these scales, the following factors should be first con¬ 
sidered: 

1. Specimen: size, conductivity, and environment 
compatibility are among the most important aspects 









Micro-/Nanorobots I 27.4 Imaging at the Micro-and Nanoscales 677 


Table 27.3 Comparison of optical microscopes (OM) and electron microscopes 


Feature 

OM 

TEM 

SEM 

General use 

Surface morphology 
and sections (1—40 |im) 

Section (40— 150nm) or small particles 
on thin membranes 

Surface morphology 

Source of illumination 

Visible light 

High-speed electrons 

High-speed electrons 

Best resolution 

ca. 200 nm 

ca. 0.2 nm 

ca. 3—6 nm 

Magnification range 

10-1000X 

500—500.000X 

20—150.000X 

Depth of field 

0.002-0.05 nm (N.A. 1.5) 

0.004-0.006mm (N.A. 10~ 3 ) 

0.003—1 mm 

Lens type 

Glass 

Electromagnetic 

Electromagnetic 

Image ray-formation spot 

On eye by lenses 

On phosphorescent plate by lenses 

On cathode tube 
by scanning device 


to consider. For example, in vivo bioapplications 
generally require air or liquid, so lower-resolution, 
light microscopy should be the first choice. If higher 
resolution is needed, atomic force microscopy or 
scanning near-field optical microscopy can be used. 

2. Resolution: the ability to see fine details of a speci¬ 
men. Once you can resolve fine details then you can 
magnify them. Every microscope has a finite resolu¬ 
tion; if you magnify objects beyond the resolution the 
result will be empty magnification. Roughly speak¬ 
ing, a light microscope cannot provide a resolution 
better than 200 nm. The best commercially available 
scanning electron microscopes (SEM) have approx¬ 
imately 1 nm resolution, while transmission elec¬ 
tron microscopes (TEM) can achieve approximately 
0.2 nm, and a scanning tunneling microscope (STM) 
working in ultrahigh vacuum under very low temper¬ 
ature can resolve atomic-level structures. A compar¬ 
ison of optical microscopy and electron microscopy 
is given in Table 27.3. 

3. Depth of field: the range of depth that a specimen is 
in acceptable focus. A microscope that has a small 
depth of field will have to be continuously focused 
up and down to view a thick specimen. 

4. Contrast: the ratio between dark and light. Typ¬ 
ically, most microscopes use absorption contrast, 
i. e., the specimen is subjected to stains in order 
to be seen. This is called bright-field microscopy. 
There are other types of microscope that use more 
exotic means to generate contrast, such as phase 
contrast, dark field, and differential interference 
contrast. 

5. Brightness: the amount of light. The higher a mi¬ 
croscope magnifies the more light will be required. 
The illumination source should also be at a wave¬ 
length (color) that will facilitate interaction with 
the specimen. All microscopes fall into either of 
two categories based on how the specimen is illu¬ 
minated. In the typical compound microscope the 
light passes through the specimen and is collected 
by the image forming optics. This is called dias- 
copic illumination. Dissecting (stereo) microscopes 


generally use episcopic illumination for use with 
opaque specimen. The light is reflected onto the 
specimen and then into the objective lens. 

27.4.1 Optical Microscopy 

Since their invention in the late 1500s, light micro¬ 
scopes have enhanced our knowledge in basic biology, 
biomedical research, medical diagnostics, and mate¬ 
rials science. Light microscopes can magnify objects 
up to 1000 times, revealing microscopic details. Light- 
microscopy technology has evolved far beyond the 
first microscopes of Robert Hooke and Antoni van 
Leeuwenhoek. Special techniques and optics have been 
developed to reveal the structures and biochemistry of 
living cells. Most optical microscopes in current use are 
known as compound microscopes, where a magnified 
image of an object is produced by the objective lens, 
and this image is magnified by a second lens system 
(the ocular or eyepiece) for viewing. Microscopes have 
even entered the digital age, using charge-coupled de¬ 
vices (CCDs) and digital cameras to capture images. 

The development of modern microscopy has en¬ 
abled a large family of optical microscopes. For spe¬ 
cial purposes, other types of optical microscopes can 
be selected. These include phase-contrast microscopy, 
fluorescence microscopy, confocal scanning optical mi¬ 
croscopy, and deconvolution microscopy image recon¬ 
struction. 

27.4.2 Electron Microscopy 

Scanning Electron Microscope (SEM) 

Since the commercial availability of the SEM in 1966, 
it has been a valuable resource for viewing samples at 
a much higher resolution and depth of field than the 
typical optical microscope. Conventional SEMs can re¬ 
solve down to the nanometer scale (« 1 nm) whereas an 
optical microscope can only resolve down to approx¬ 
imately 200nm [27.42]. Unlike conventional optical 
microscopes, SEMs have a high depth of field which 
gives imaged samples a three-dimensional appearance. 
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Early SEMs were limited to viewing conductive sam¬ 
ples. However, many of today’s SEMs can image non- 
conductive samples in addition to conductive samples 
using variable-pressure chambers. 

Transmission Electron Microscope (TEM) 

The TEM can resolve to an atomic scale down to about 
1 d. h. (i. e., 0.1 nm). The TEM mode of operation is 
similar to that of the SEM in that both microscopes con¬ 
tain an electron gun source of illumination. However, 
the TEM detects the electrons that pass through a given 
sample. As a result, the electron gun of the TEM oper¬ 
ates at higher energy levels between 50—1000 kV, while 
the SEM’s electron gun operates at around 1—30kV. 
In order for proper imaging to take place, the sam¬ 
ple must be very thin so that electrons from the beam 
can pass through the specimen. Electrons that do not 
pass through the sample cannot be detected. Unlike the 
SEM, the TEM produces images that are two dimen¬ 
sional in appearance. 

27.4.3 Scanning Probe Microscopy 

Scanning Tunneling Microscope (STM) 

Similar to the TEM, the STM [27.19] can also re¬ 
solve specimens down to the atomic scale. The scanning 
probe of the STM is comprised of a noble metal sharp¬ 
ened to an atomic-sized tip, which is mounted on 
a piezoelectrically driven ( x , y, z) linear stage. The STM 
makes use of the quantum-mechanical effect known as 
tunneling. Electron tunneling occurs when electrons, 
driven by a small potential difference, flow across the 
gap between the probe tip and sample. This event takes 
place at Angstrom-scale distances between the probe tip 
and the sample [27.43]. The tunneling current, which 
is typically on the order of a few nanoamperes, is di¬ 
rectly related to the tip-sample separation distance. 
Thus, the tunneling current can be measured and is kept 
at a constant value by controlling the tip-sample gap 
distance (z) with a feedback control system. The probe 
tip is then scanned ( x , y ) along the entire surface of the 
sample. Since the control system maintains a constant 
tunneling current, and thus maintains a constant tip- 
sample distance (z), the result of a scan yields a z(x, y) 
terrain map of the sample with enough resolution to de¬ 


27.5 Fabrication 

The design of micro- and nanorobotic devices is in¬ 
extricably linked to available fabrication techniques. 
However, though the development of microfabrication 
processes has become somewhat stable over the past 


tect atomic-scale features. The STM can achieve faster 
imaging by operating in so-called constant-height mode 
in which the probe tip is scanned in a plane parallel to 
the average surface portion. The tip-sample distance (z) 
can then be inferred directly from the measured tunnel¬ 
ing current [27.43]. 

Atomic Force Microscope (AFM) 

The AFM [27.44] is considered to be a spin-off of the 
STM. One shortcoming of the STM is that it requires 
conductive probe tips and samples to work properly. 
The AFM was developed in order to view nonconduc- 
tive samples, giving it a wider applicability than the 
STM. In addition to imaging nonconductive samples, 
the AFM can also image samples immersed in liquid, 
which is useful for biological applications [27.45], Al¬ 
though the STM and AFM are similar in that they both 
scan the surface of a sample with an atomically sharp 
probe, they operate under slightly different principles. 
The AFM is based on interatomic forces as opposed to 
the electron tunneling used in the STM. The AFM probe 
tip is mounted on the end of a microscale cantilever 
beam. At very short separations, the forces between 
atoms in the probe tip and atoms in the sample cause the 
cantilever to deflect. This deflection is usually measured 
by striking the back of the cantilever with a laser. The 
reflection of the laser beam hits a photodetector, which 
can be used to recover the deflection of the cantilever. 
The force can then be calculated by using Hooke’s law, 
which simply relates the applied force to the stiffness 
and deflection of a material. The forces that are mea¬ 
sured can be on the scale of piconewtons [27.43]. The 
AFM has three main modes of operation known as con¬ 
tact mode, noncontact mode, and tapping mode. 

Unlike the SEM and TEM, both the STM and AFM 
do not require a vacuum environment in order to func¬ 
tion. However, a high vacuum is advantageous in order 
to keep the samples from becoming contaminated from 
the surrounding environment as well as controlling hu¬ 
midity. In addition, atomic resolution in air is hardly 
possible with an AFM due to humidity. As a result of 
humidity, a water film is formed and creates problems 
because of capillary forces. This can be resolved by op¬ 
erating in vacuum environment or completely in a liquid 
solution [27.43]. 


decade, nanofabrication processes are still being ac¬ 
tively pursued, and the design constraints generated by 
these processes are relatively unexplored. This section 
will briefly highlight the processes used in conventional 
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microfabrication including lithography, thin-film de¬ 
position, chemical etching, and electrodeposition, and 
describe some emerging techniques for nanofabrica¬ 
tion. 

Most micro- and nanofabrication techniques have 
their roots in the standard fabrication methods de¬ 
veloped for the semiconductor industry [27.46-48]. 
Therefore, a clear understanding of these techniques is 
necessary for anyone embarking on a research and de¬ 
velopment path in the micro/nano area. 

27.5.1 Microfabrication 

In this section, we will discuss the major microfabrica¬ 
tion techniques used most frequently in the manufactur¬ 
ing of microstructures. 

Photolithography 

Lithography is the technique used to transfer 
a computer-generated pattern onto a substrate (sil¬ 
icon, glass, GaAs, etc.). This pattern is subsequently 
used to etch an underlying thin film (oxide, nitride, etc.) 
for various purposes (doping, etching, etc.). Although 
photolithography, i. e., lithography using an ultraviolet 
(UV) light source, is by far the most widely used 
lithography technique in microelectronic fabrication, 
electron-beam (e-beam) and X-ray lithography are two 
alternatives that have attracted considerable attention 
in the MEMS and nanofabrication areas. We will 
discuss photolithography in this section and postpone 
the discussion of e-beam and X-ray techniques to the 
subsequent sections dealing with nanofabrication. 

The starting point following the creation of the 
computer layout for a specific fabrication sequence is 
the generation of a photomask. This involves a se¬ 
quence of photographic processes (using optical or 
e-beam pattern generators) that results in a glass plate 
having the desired pattern in the form of a thin (f« 
lOOnm) chromium layer. Following the generation of 
photomask, the lithography and etching process can 
proceed as shown in Fig. 27.2. After depositing the 
desired material on the substrate, the photolithogra¬ 
phy process starts with spin-coating the substrate with 
a photoresist. This is a polymeric photosensitive ma¬ 
terial that can be spun onto the wafer in liquid form; 
usually an adhesion promoter such as hexamethyld- 
isilazane (HMDS) is used prior to the application of 
the resist. The spinning speed and photoresist viscos¬ 
ity will determine the final resist thickness, which is 
typically 0.5-2.5 |im. Two different kinds of photore¬ 
sist are available: positive and negative. With a positive 
resist, the UV-exposed areas will be dissolved in the 
subsequent development stage, whereas with a nega¬ 
tive photoresist, the exposed areas will remain intact 


after the development. After spinning the photoresist 
on the wafer, the substrate is soft-baked (5—30 min at 
60-100 °C in order to remove the solvents from the re¬ 
sist and improve the adhesion. Subsequently, the mask 
is aligned to the wafer and the photoresist is exposed to 
a UV source. 

After exposure, the photoresist is developed in 
a process similar to the development of photographic 
film. The resist is subsequently hard-baked (20—30 min 
at 120-180°C in order to further improve the adhesion. 
The hard-bake step concludes the photolithography se¬ 
quence by creating the desired pattern on the wafer. 
Next, the underlying thin film is etched, and the pho¬ 
toresist is stripped in acetone or other organic solvent. 
Figure 27.3 shows a schematic of the photolithography 
steps with a positive photoresist. 

Thin-Film Deposition and Doping 
Thin-film deposition and doping are used extensively 
in micro- and nanofabrication technologies. Most of the 
fabricated structures contain materials other than that of 
the substrate, which are obtained by various deposition 
techniques, or by modification of the substrate. These 
techniques include oxidation, doping, chemical vapor 
deposition (CVD), physical vapor deposition (PVD), 
and electroplating. 

Etching and Substrate Removal 
For micro- and nanofabrication, in addition to thin film 
etching, often the substrate (silicon, glass, GaAs, etc.) 
needs to be removed in order to create various me¬ 
chanical structures (beams, plates, etc.). Two important 
figures of merit for any etching process are selectivity 
and directionality. 

Selectivity is the degree to which the etchant can 
differentiate between the masking layer and the layer to 


Fig. 27.2 Typical microfabrication process flow 
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be etched. Directionality has to do with the etch profile 
under the mask. In an isotropic etch, the etchant attacks 
the material in all directions at the same rate, creating 
a semicircular profile under the mask (Fig. 27.4a). In an 
anisotropic etch, the dissolution rate depends on spe¬ 
cific directions, and one can obtain straight sidewalls 
or other noncircular profiles (Fig. 27.4b). One can also 
divide the various etching techniques into wet and dry 
categories. Due to the lateral undercut, the minimum 
feature size achievable with wet etchants is limited 
to > 3 |im. Photoresist and silicon nitride are the two 
most common masking materials for the wet oxide etch. 
Anisotropic and isotropic wet etching of crystalline (sil¬ 
icon and gallium arsenide) and noncrystalline (glass) 
substrates are important topics in micro- and nanofabri¬ 
cation [27.49-53]. 


a) Oxidize the substrate b) Spin the photoresist 

and soft bake 

Photoresist 


Si0 2 


Si0 2 

Si 

Si 


c) Expose the photoresist 
Light 
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Photomask 
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and hard bake 
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e) Etch the oxide f) Stripe the photoresist 



Fig.27.3a-f Schematic drawing of the photolithographic 
steps with a positive photoresist (PR) (a) Oxidize the sub- 
strat (b) spin the photoresist and soft bake (c) expose the 
photoresist (d) develop the photoresist and hard bake (e) 
etch the oxide (f) stripe the photoresist 


a) b) 


Mask 


Mask 

Film 

Film 

Substrate 



Fig.27.4a,b Profile for isotropic (a) and anisotropic (b) etch 
through a photoresist mask 


The anisotropic behavior of these etchants with re¬ 
spect to the (111) planes have been used extensively to 
create beams, membranes, and other mechanical and 
structural components. Figure 27.5 shows the typical 
cross sections of (100) silicon wafers etched with an 
anisotropic wet etchant. As can be seen, the (111) slow 
planes are exposed and creating 54.7° sloped sidewalls. 
Depending on the dimensions of the mask opening, 
a V-groove or a trapezoidal trench is formed in the (100) 
wafer. A large enough opening will allow the silicon to 
be etched all the way through the wafer, thus creating 
a thin dielectric membrane of the other side. It should 
be mentioned that exposed convex corners have a higher 
etch rate than the concave ones, resulting in an under¬ 
cut that can be used to create dielectric (e.g., nitride) 
cantilever beams. 

Dry etching techniques are largely plasma based. 
They have several advantages compared with wet etch¬ 
ing. These include smaller undercut (allowing smaller 
lines to be patterned) and higher anisotropy (allowing 
high-aspect-ratio vertical structures). However, the se¬ 
lectivity of dry etching techniques is lower than the wet 
etchants, and one must take into account the finite etch 
rate of the masking materials. The three basic dry etch¬ 
ing techniques, namely high-pressure plasma etching, 
reactive-ion etching (RIE), and ion milling, utilize dif¬ 
ferent mechanisms to obtain directionality. 

27.5.2 Nanofabrication 

The design and fabrication of NEMS is an emerg¬ 
ing area being pursued by an increasing number of 
researchers. Two approaches to nanofabrication, top- 
down and bottom-up, have been identified by the 
nanotechnology research community and are being in¬ 
dependently investigated by various researchers. Top- 
down approaches are based on microfabrication and 
include technologies such as nanolithography, nanoim¬ 
printing, and chemical etching. Presently, these are 
2-D fabrication processes with relatively low resolu¬ 
tion. Bottom-up strategies are assembly-based tech¬ 
niques. Currently these strategies include techniques 
such as self-assembly, dip-pen lithography, and directed 
self-assembly. These techniques can generate regular 
nanopatterns at large scales. 
























































Micro-/Nanorobots I 27.6 Microassembly 681 


In this section, we will discuss three major nanofab¬ 
rication techniques. These include: 

1. e-beam and nanoimprint fabrication 

2. Epitaxy and strain engineering , and 

3. Dip-pen nanolithography. 

E-beam Lithography 
and Nanoimprint Fabrication 
In previous sections, we discussed several important 
lithography techniques commonly used in MEMS and 
microfabrication. These include various forms of UV 
(regular, deep, and extreme) and X-ray lithography. 
However, due to the lack of resolution (in the case 
of the UV), or the difficulty in manufacturing mask 
and radiation sources (X-ray), these techniques are 
not suitable for nanometer-scale fabrication. E-beam 
lithography is an attractive alternative technique for 
fabricating nanostructures [27.53]. It uses an elec¬ 
tron beam to expose an electron-sensitive resist such 
as polymethyl methacrylate (PMMA) dissolved in 
trichlorobenzene (positive) or polychloromethylstyrene 
(negative). 

The e-beam gun is usually part of an SEM, although 
a TEM can also be used. Although electron wavelengths 
on the order of 1 A can easily be achieved, electron scat¬ 
tering in the resist limits the attainable resolutions to 
> 10 nm. The beam control and pattern generation are 
achieved through a computer interface. 

E-beam lithography is serial and hence has a low 
throughput. Although this is not a major concern in fab¬ 
ricating devices used in studying fundamental micro¬ 
physics, it severely limits large-scale nanofabrication. 
E-beam lithography, in conjunction with such processes 


27.6 Microassembly 

Assembly is often required in macroscale product man¬ 
ufacturing in order to reduce the complexity and cost 
of the manufacturing process. Assembly makes it pos¬ 
sible to build complex products from relatively simple 
parts and to integrate incompatible manufacturing pro¬ 
cesses. It also makes maintenance and replacement 
possible. The extension of assembly techniques into the 
microscale is driven by the development of modem de¬ 
sign and manufacturing technologies in the pursuit of 
miniaturization and function integration, especially by 
the development of integrated circuit (IC) [27.59] and 
MEMS fabrication techniques. With the extension of 
manufacturing technology into the microscale and even 
nanoscale domain, the term microassembly has been 
created to refer specifically to assembly operations per¬ 


as lift-off, etching, and electrodeposition, can be used to 
fabricate various nanostructures. 

An interesting new technique that circumvents the 
serial and low-throughput limitations of the e-beam 
lithography for fabricating nanostmctures is nanoim¬ 
print technology [27.54]. This technique uses an 
e-beam-fabricated hard material master (or mold) to 
stamp and deform a polymeric resist. This is usually 
followed by a reactive-ion etching step to transfer the 
stamped pattern to the substrate. This technique is eco¬ 
nomically superior, since a single stamp can be used 
repeatedly to fabricate a large number of nanostmc¬ 
tures. 

Epitaxy and Strain Engineering 
Atomic-precision deposition techniques such as 
molecular-beam epitaxy (MBE) and metallo-organic 
chemical vapor deposition (MOCVD) have proven to 
be effective tools in fabricating a variety of quantum 
confinement structures and devices (quantum well 
lasers, photodetectors, resonant tunneling diodes, 
etc.) [27.55-57], 

Dip-Pen Nanolithography 
In dip-pen nanolithography (DPN), the tip of an AFM 
operated in air is inked with a chemical of inter¬ 
est and brought into contact with a surface. The ink 
molecules flow from the tip onto the surface as with 
a fountain pen. Line widths down to 12 nm with spa¬ 
tial resolution of 5 nm have been demonstrated with 
this technique [27.58]. Species patterned with DPN 
include conducting polymers, gold, dendrimers, de¬ 
oxyribonucleic acid (DNA), organic dyes, antibodies, 
and alkanethiols. 


formed at the micro/mesoscale [27.60]. To give a formal 
definition, microassembly is the assembly of objects 
with microscale and/or mesoscale features under mi¬ 
croscale tolerances [27.61]. 

27.6.1 Automated Microassembly Systems 

Microassembly plays the role of an enabling tech¬ 
nology in various processes of MEMS fabrication, 
including device fabrication, packaging, and intercon¬ 
nection. MEMS device fabrication is fundamentally 
different from the highly modular IC fabrication in 
that it often requires the machining of complex-shaped 
three-dimensional (3-D) mechanical structures [27.51]. 
However, almost all current MEMS fabrication tech- 
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niques are subject to constraints in limited allowable 
materials, limited capabilities in true 3-D fabrication, 
and the requirement of fabrication process compatibil¬ 
ity. Microassembly provides a possible solution to these 
constraints. For example, incompatible fabrication pro¬ 
cesses can be integrated through assembly. This makes 
it possible to use nontraditional fabrication techniques 
that are not necessarily based on semiconductor materi¬ 
als, such as laser cutting, microwire electrical discharge 
machining (EDM), and micromilling [27.62]. Complex 
3-D structures can also be developed using parts with 
relatively simple geometry [27.63,64]. Microassembly 
is also crucial to MEMS packaging and interconnec¬ 
tion [27.65,66]. 

From the perspective of robotic systems and au¬ 
tomation, MEMS device fabrication and packaging 
share many common assembly requirements. A fun¬ 
damental commonality of both processes is the re¬ 
quirement of the ability to manipulate micro/mesoscale 
objects so that precise (i. e., microscale tolerance) spa¬ 
tial relations can be established (e.g., die alignment, 
part insertion) and certain physical/chemical processes 
(e.g., die bonding, surface coating) can be performed. 
Another common requirement is to control the interac¬ 
tion force involved. MEMS devices often have fragile 
structures such as thin beams or membranes. This re¬ 
quires controlling the interactive force in manipulation 
operations. Typically, the force magnitude resides in the 
range from millinewtons to micronewtons. 

MEMS devices are often three-dimensional. MEMS 
packaging requires both electrical interconnection for 
signal transmission and mechanical interconnection for 
the interaction of the packaged device with its exter¬ 
nal environment [27.51,67]. Many such mechanical 
interconnections require three-dimensional manipula¬ 
tion and three-dimensional force control. The actual 
operations are highly application specific, which pose 
great challenges to the development of automated mi¬ 
croassembly systems. Automated IC packaging sys¬ 
tems can be used for the packaging of certain MEMS 
devices such as accelerometers and gyros. However, 
packaging of microfluidic devices, optical MEMS de¬ 
vices, and hybrid microsystems often require the devel¬ 
opment of new automated microassembly techniques 
and systems. 

The selection of assembly mode is among the 
first decisions to be made in developing automated 
microassembly systems. Sequential microassembly re¬ 
quires the use of micromanipulators and sensory feed¬ 
back. At each moment, only one or a few parts are being 
assembled. Depending on the physical effects used, 
parallel microassembly can be either deterministic or 
stochastic [27.68]. Die bonding is an example of deter¬ 
ministic parallel microassembly. In stochastic parallel 


microassembly, large numbers of parts are assembled 
simultaneously using distributed physical effects such 
as electrostatic force, capillary force, centrifugal force, 
or vibration [27.60,69-71]. In fact, the basic philoso¬ 
phy of stochastic parallel microassembly is to minimize 
the use of sensory feedback. 

Each of these assembly modes has both advantages 
and disadvantages. Each has its own suitable appli¬ 
cations. Deterministic parallel assembly shares several 
commonalities with sequential microassembly. For ex¬ 
ample, sensory feedback is often used in deterministic 
parallel microassembly. However, deterministic parallel 
microassembly requires high relative positioning ac¬ 
curacy between parts. In addition, only simple planar 
structure features can be assembled in order to make 
parallel operation possible. 

Due to the requirement of MEMS packaging for 
three-dimensional manipulation and microassembly, it 
can be expected that automated sequential microassem¬ 
bly will be the most widely adopted solution. In partic¬ 
ular, those MEMS packaging applications that require 
the control of multiple-degree-of-freedom (DOF) inter¬ 
action force should use sequential microassembly. The 
major possible disadvantage of sequential microassem¬ 
bly is its low throughput. This constraint can often be 
overcome by appropriate system design. 

Here we introduce a three-dimensional mi¬ 
croassembly example that originates from an industrial 
application [27.61]. It is significantly different from 
wire bonding and die bonding applications in that 
high-precision 3-D part insertion is required. Together 
with other examples, it will be used throughout this 
section to illustrate the major concepts and techniques 
and the logic connections between each functional unit 
of an automated microassembly system. 

The assembly task is to pick up micromachined 
thin metal parts that are transferred to the assembly 
workcell on a vacuum-release tray (Fig. 27.6a-c), and 
insert them into vertically deep reactive ion etching 
(DRIE) etched holes in a silicon wafer (Fig. 27. 6d). 
The wafers can have diameters up to 8 inches. The 
holes on each wafer form regular arrays consisting 
of approximately 50 holes each. However, these ar¬ 
rays may not be regularly distributed on the wafer. 
Typically, hundreds of parts are to be assembled on 
each wafer. In general, each assembly operation is 
a typical rectangular-peg-into-a-rectangular-hole prob¬ 
lem. Each metal part is approximately half a millimeter 
in width and less than 100 (im in thickness at its 
rectangular tip. The total assembly tolerance is typi¬ 
cally smaller than 10 |im in the vertical direction, and 
smaller than 20 (im in the horizontal direction. This 
task is a typical application of 3-D microassembly 
techniques. 
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27.6.2 Microassembly System Design 

This section discusses the design of automated mi¬ 
croassembly systems from the perspective of robotic 
systems and automation. Performance objects to be 
achieved include high reliability, high throughput, high 
flexibility, and low cost. 

General Guidelines 

Taking a System Perspective. An automated mi¬ 
croassembly system consists of many functional units 
and must integrate techniques from a diverse range of 
areas such as robotics, computer vision, microscope op¬ 
tics, physics, and chemistry. It is therefore important to 
consider the interaction between these units. 

Emphasizing the Coupling with Packaging Pro¬ 
cesses. Although in this section microassembly tech¬ 
niques are addressed primarily from the perspective 
of robotics for the clarity of presentation, robotic sys¬ 
tem designers must recognize the strong dependence 
of architecture design of the robotic systems on the 
packaging process being implemented. This connection 
should be emphasized from the beginning of system de¬ 
velopment. 

Design for Reconfigurability. Automated packaging 
machines must be designed so that they can be cus¬ 
tomized for a wide variety of applications. Reconfigura¬ 
bility is therefore a basic design requirement. Typically, 
modular design based on function decomposition is de¬ 
sirable. Support for tool replacement is essential. 

The design of automated microassembly systems is 
strongly dependent on the assembly tolerance required. 
First of all, the repeatability of the motion control sys¬ 
tem and the micromanipulator used is determined by 
the required assembly tolerance. In addition, assem¬ 
bly tolerance often determines the minimum resolu¬ 
tion of the microscope optics. General microassembly 
tasks may only require microscopic vision feedback 
and manipulators with microscale repeatability, while 
complex microassembly tasks may also require the in¬ 


tegration of microforce and vision feedback [27.72, 
73], 

Automated microassembly systems must be able to 
support a wide variety of material-handling tools, which 
include part transfer tools, bulk feeders, wafer-handling 
tools, magazine loaders and unloaders, etc. 

The role of the micromanipulator is to provide 
multiple-DOF fine motion control. The role of the mi¬ 
crogripper is to grasp objects in pick-and-place and 
other assembly operations. Their efficiency and robust¬ 
ness will to a great extent decide the performance of 
the entire system. Microgripper design is also closely 
related to the design of fixtures for microassembly op¬ 
erations. 

Major environment factors include clean-room re¬ 
quirements, temperature, humidity, airflow, etc. Certain 
assembly operations must be performed in a clean 
room. This requires that the design of automated mi¬ 
croassembly systems comply with relevant standards. 
Some packaging processes such as eutectic bonding 
must be performed under high temperature. Conse¬ 
quently, the potential influence of high temperature 
on motion control system and microscope optics must 
be considered. For the manipulation of microscale 
objects, environment conditions such as temperature 
and humidity can have a major influence on adhesion 
forces [27.74,75]. Therefore, it is often important to 
consider environment control in system design. 

Assembly Process Flow: An Example 
The micromachined metal parts (Fig. 27.6c) are hor¬ 
izontally transferred to the workcell on the vacuum- 
release tray (Fig. 27.6b). The wafer is placed on 
a wafer mount perpendicular to the horizontal plane 
(Fig. 27.6b). This configuration does not require the 
flipping of the thin metal parts and is advantageous in 
terms of reliability and efficiency. There are two major 
operations in each assembly cycle: pickup and inser¬ 
tion. Under the configuration shown in Fig. 27.7a, all 
the operations are performed by the same workcell. 
Complex packaging operations must often be decom¬ 
posed and performed by multiple workcells [27.76]. 
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Fig.27.6a-e A 3-D microassembly example (a) Parts on release tray (b) parts versus a US dime (c) part shape (d) DRIE holes 
(e) assembly results 
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General System Architecture 
An automated microassembly system typically consists 
of the following functional units. 

Large Workspace Positioning Unit. A large work¬ 
space and long-range positioning motion are required in 
most microassembly operations. The large workspace 
is necessary to accommodate different functional units, 
part feeders, and various tools. 

In general, off-the-shelf motion control systems de¬ 
veloped for automated IC packaging equipment can be 
adopted directly. For the task described in Sect. 27.6.1, 
the DRIE-etched holes are distributed on wafers up to 
8 inches in diameter. This requires the assembly sys¬ 
tem to have a commensurate working space and high 
positioning speed. The coarse positioning unit has four 
DOFs (Fig. 27.7a). Planar motion in the horizontal di¬ 
rection is provided by an open-frame high-precision XY 
table with a travel of 32 cm (12 inch) and a repeatability 
of 1 |un in both directions. Position feedback with a res¬ 
olution of 0.1 |im is provided by two linear encoders. 
A dual-loop PID (proportional-integral-derivative) plus 
feedforward control scheme is used for each axis. The 
internal speed loop is closed on the rotary encoder on the 
motor. The external position loop is closed on the lin¬ 
ear encoder. Each wafer is placed on the vertical wafer 
mount that provides both linear and rotational control 
(Fig. 27.7b). Vertical motions of the wafer mount are 
provided by a linear slide with a travel of 20 cm (8 inch) 
and a repeatability of 5 |xm. It is also controlled using 
a PID plus feedforward algorithm. Both the XT table and 
the vertical linear slides are actuated using alternating- 
current (AC) servo motors. The rotation of the wafer 
mount is actuated by an Oriental PK545AUA microstep 
motor with a maximum resolution of 0.0028° step. All 
low-level controllers are commanded and coordinated 
by a host computer [27.61]. 

For applications requiring repeatability of one mi¬ 
cron or greater, it is convenient to use conventional posi¬ 


tioning tables to implement coarse range motions. These 
tables typically use leadscrew or ballscrew drives and 
ball or roller bearings. For applications requiring sub¬ 
micron or nanometer repeatability, a few solutions are 
also commercially available. For example, piezoactua¬ 
tors are often used for nanometer repeatability motion. 
The disadvantage of piezoactuators is that their travel 
range is small, typically on the order of 100 microns. 
As another example, a series of positioning stages with 
submicron repeatability based on direct-drive linear ac¬ 
tuators and air bearings is available from Aerotech. It is 
also possible to use parallel structure mechanisms such 
as a Stewart platform [27.77]. In general, the develop¬ 
ment of IC manufacturing towards the deep submicron 
level provides a major driving force behind the develop¬ 
ment of these motion control techniques. 

Micromanipulator Unit. Fine pose (position and ori¬ 
entation) control is required in operations such as 3-D 
precision alignment and assembly. For example, the six 
DOFs required by the task introduced in Sect. 27.6.1 are 
implemented on separate structures. The three Carte¬ 
sian DOF are provided by an adapted Sutter MP285 
micromanipulator, which also provides yaw motions 
with its rotational DOF (Fig. 27.7c). Roll motions are 
implemented on the wafer mount (Fig. 27.7b). The 
pitch movement of the metal part after pickup is not mo¬ 
torized and is implemented through manual adjustment 
and calibration before assembly. More discussion of mi¬ 
cromanipulator configuration can be found in [27.61]. 

Two principles need to be considered in implement¬ 
ing motion control for automated 3-D microassembly. 
The first is the partition of large-workspace coarse posi¬ 
tioning unit and the fine positioning micromanipulator 
unit. The second is the decomposition and distributed 
implementation of multiple DOFs. In practice, the ac¬ 
tual implementations of these principles are highly 
application specific. For certain applications, if a large- 
range positioning unit is sufficient to satisfy assembly 



Fig.27.7a-c An experimental microassembly workcell (a) System overview (b) assembly scene (c) micromanipulator 
unit 
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tolerance requirements, the separate implementation of 
a micromanipulator may even be unnecessary. 

In general, the separation of a micromanipula¬ 
tor unit will facilitate the implementation of high- 
bandwidth motion control. However, this separation 
normally brings redundancy to the entire motion control 
system. The functioning of the high-precision micro¬ 
manipulator must also rely on closed-loop feedback 
control, especially microscopic vision feedback. 

Automated Microgripper Unit. The function of a mi¬ 
crogripper is to provide geometrical and physical con¬ 
straints (grasping) in pick-and-place and assembly op¬ 
erations. The reliability and efficiency of the microgrip¬ 
per is critical to the performance of the entire automated 
microassembly system. Several factors must be consid¬ 
ered in micromanipulator design. First of all, as the 
micromanipulator end-effector, the microgripper must 
be constantly monitored under a microscope. There¬ 
fore, it must be small in size and suitable in shape 
to remain in the microscope’s field of view and to 
minimize occlusion. Secondly, it is important to con¬ 
sider the different governing physics and to explore 
the use of various gripping forces [27.77]. Thirdly, mi¬ 
crogrippers are used in both part pick-and-place and 
assembly. Since assembly operations typically require 
more constraints, microgrippers designed for pick-and- 
place operations may not necessarily be suitable for 
assembly operations. In fact, the microassembly task 
described in Sect. 27.6.2 is performed using a combined 
microgripper [27.61]. Microgripper development is of¬ 
ten closely related to the development of fixtures that 
can also have microscale sizes. 

Microscope Optics and Imaging Unit. The function 
of the microscope optics and imaging unit is to provide 
noncontact measurement of the geometry, motion, and 
spatial relations of assembly objects. Typical configura¬ 
tions of commercial device bonding systems use one or 
two vertical microscopes. An inverted microscope con¬ 
figuration is commonly used for backside alignment. 
On the other hand, 3-D microassembly may require two 
camera views in a stereo configuration. In the system 
shown in Fig. 27.7a, a total of four different views can 
be provided to its human operator: a global view of the 
entire assembly scene, a vertical microscopic view for 
part pickup, and two lateral microscopic views for the 
fine position and orientation adjustments during the fi¬ 
nal microassembly operations. Each view uses a CCD 
camera with a matching optical system. All images are 
captured using a Matrox Corona peripheral component 
interconnect (PCI) frame grabber. 

Microscopic visual feedback is crucial for precise 
3-D alignment. Provided that resolution requirements 


of the assembly task can be satisfied, microscope optics 
with larger working distances is desirable. For the as¬ 
sembly task addressed in Sect. 27.6.1, an Edmund Sci¬ 
entific VZM 450i zoom microscope with a 1 x objective 
is used to provide the right view. Its working distance 
is approximately 90mm, with a resolution of 7.5 |xm. 
The vertical view is also provided by a VZM 450i 
microscope with a 0.5 x objective to guide pickup oper¬ 
ations. Its field of view can range from 2.8 x 2.8 mm to 
17.6 x 17.6 mm. Its working distance is approximately 
147 mm. 

If visual servoing is required in automate assem¬ 
bly operations, a stereo configuration formed by adding 
another lateral view may be necessary. Higher resolu¬ 
tion may be necessary in some microassembly tasks. 
In such cases, the same configuration can be used 
with higher-resolution microscope optics, for example, 
two Navitar TenX zoom microscopes with Mitutoyo 
ultralong-working-distance M Plan Apo lOx objectives 
have been used in this configuration by the authors. The 
resolution of each microscope is 1 p,m, with a work¬ 
ing distance of 33.5mm. Consequently, the usable 
workspace of the micromanipulator is reduced [27.61]. 

The global view is implemented using a miniature 
Marshall V-1260 board camera to monitor the status of 
the entire assembly scene. It plays an important role 
in helping the operator to understand gross spatial re¬ 
lations and preventing operation errors. 

From the perspective of robotic systems, the de¬ 
velopment of automation microassembly systems for 
complex 3-D microassembly operation will rely on the 
development of: 

1. Compact, robust and high-speed micromanipulators 
with 5-6 DOFs. 

2. Highly reliable and efficient microgrippers that are 
suitable for working under microscopes. Such grip¬ 
pers should have active force control or passive 
compliance to avoid damage to MEMS devices. 

3. Three-dimensional microscopic computer vision 
techniques, three-dimensional microforce measure¬ 
ment and control techniques, and their integration. 

27.6.3 Basic Microassembly Techniques 

This section introduces a few supporting techniques 
important to automated microassembly systems, in¬ 
cluding machine vision techniques, microforce control 
techniques, and simulation verification of assembly 
strategy. 

Machine Vision Techniques 
Machine vision techniques are widely used in the semi¬ 
conductor industry. The major difference between ma- 
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chine vision and general computer vision [27.78] is that, 
unlike natural objects and scenes, industrial objects and 
scenes can often be artificially designed and configured. 
This advantage often makes it possible to significantly 
reduce the complexity and enhance the robustness of 
vision techniques. The applications of machine vision 
techniques can generally be categorized into the follow¬ 
ing two classes based on the requirement of real-time 
processing. 

Non-Time-Critical Vision Applications. These ap¬ 
plications do not require visual feedback for high- 
bandwidth real-time control. Examples include object 
recognition and packaging quality inspection [27.79]. 

Time-Critical Vision Applications. These applica¬ 
tions require real-time visual feedback. Examples in¬ 
clude vision-guided pick-and-place, alignment, inser¬ 
tion, etc. [27.80]. An introduction to 3-D computer 
vision techniques can be found in [27.81]. A standard 
introduction to visual servoing techniques can be found 
in [27.82], 

Several commercial software packages are available 
from suppliers such as Cognex, Coreco Imaging, and 
National Instruments. 

Microforce Control Techniques 
The theory of force control has been studied by the 
robotics community for more than 50 years [27.83, 84], 
Several theoretical frameworks and many control algo¬ 
rithms have been proposed and experimentally verified. 
A variety of macroscale multiple-DOF force sensors 
have been developed. 

Force control is also crucial to microassembly. For 
example, contact forces in device bonding must often 
be programmed and precisely controlled. In general, 
many of the macroscale force control techniques can 
be applied at micro/mesoscales. Force control for de¬ 
vice bonding is essentially 1-D and involves force on 
the order of several newtons. On the other hand, in the 
manipulation of micro/mesoscale parts, the magnitude 
of interactive force typically ranges from millinew- 
tons (10~ 3 N) to micronewtons (10 6 N). Forces of this 
magnitude are often referred to as microforces. A major 
technical challenge in implementing microforce control 
is the lack of multiple-DOF microforce sensors. A ba¬ 
sic requirement for multiple-DOF microforce sensors is 
that they must be miniature in size. The manufacturing 
of these sensors normally requires the use of micro¬ 
machining, including MEMS techniques. There are two 
major microforce sensing configurations. 

Stand-Alone Force Sensor. The advantage of this 
configuration is that the sensor is general purpose 


and can be used with different microgrippers. Most 
macroscale multiple-DOF force sensors are of this type. 
However, this also requires the sensor to have sufficient 
structural stiffness to support the static load of micro¬ 
gripper, which is often significantly larger than the force 
resolution to be reached. 

Embedded Force Sensor. Micro strain gages can be 
attached to microgrippers [27.85]. Force-sensitive ma¬ 
terials can also be deposited on microgrippers. This 
configuration avoids the issue of static load. However, 
such force sensing capabilities are dependent on the 
microgripper design, which often is not necessarily 
optimal for the measurement of multiple-DOF micro¬ 
force/torque. 

Complex and high-precision microassembly tasks 
also require the integration of microscopic machine 
vision with microforce control. Simple integration tech¬ 
niques use a gating/switching scheme [27.72,86]. For 
more integrated approaches, visual impedance can be 
used [27.73], 

Simulation Verification of Assembly Strategies 
In many microassembly tasks, the distances between 
adjacent features are often on the meso/microscale. 
Therefore, selecting the correct assembly sequence is 
important for collision avoidance. In addition, due 
to the limited working distance of microscopes, mi¬ 
croassembly operations must often be performed in 
a limited space using micromanipulators. Collision 
avoidance is critical to avoiding equipment or device 
damage. Potential collisions can be found and avoided 
by using offline simulation software. Many commer¬ 
cially available offline robot programming tools can 
provide this function. 

Microassembly Tool 

The end-effector of a micromanipulator in an auto¬ 
mated microassembly system is often in the form of 
a microgripper, whose reliability and efficiency greatly 
influence the reliability and efficiency of the entire 
system. The microgripper must often be as small as pos¬ 
sible. Its design must also minimize potential damage to 
fragile MEMS parts. This often requires passive com¬ 
pliance in structural design. 

Microgrippers with integrated MEMS actuators can 
be fabricated monolithically and thus can be more com¬ 
pact in size. Several physical effects are commonly 
used in MEMS actuators, including electrostatic force 
and piezoelectric force [27.87], shape memory alloy 
(SMA) [27.88, 89], and thermal deformation [27.63, 
90]. Currently, the major limitation is that it is diffi¬ 
cult for MEMS actuators to generate sufficient travel, 
force, and power output. Alternatively, another solution 



Micro-/Nanorobots I 27.7 Microrobotics 687 


is to provide actuation externally [27.61,91] The ad¬ 
vantage is that sufficient travel and force and power 
output can be more easily obtained. The major dis¬ 


27.7 Microrobotics 

Today, more and more microrobotic devices are en¬ 
abling new applications in various fields. Besides mi¬ 
croassembly, microrobotics can play important roles 
in other industry fields for manipulation, characteriza¬ 
tion, inspection, and maintenance, and in biotechnology 
for, e.g., manipulating cells, a field referred to as bio¬ 
microrobotics. 

27.7.1 Introduction 

Microrobotics is a field that combines the established 
theory and techniques of robotics with the exciting 
new tools provided by MEMS technology in order 
to create intelligent machines that operate at micron 
scales. As stated by authors reviewing the micro¬ 
robotics field [27.31,92], many micro terms such as 
micromechatronics, micromechanism, micromachines, 
and microrobots are used synonymously to indicate 
a wide range of devices whose function is related to 
a small scale; however, small scale is a relative term 
so a clearer definition is needed. 

The obvious difference between a macrorobot and 
a microrobot is the size of the robot. Thus, one defi¬ 
nition of a microrobot is a device having dimensions 
smaller than classical watch-making parts (i. e., [im to 
mm) and having the ability to move, apply forces and 
manipulate objects in a workspace with dimensions in 
the micrometer or submicrometer range [27.93]. How¬ 
ever, in many cases it is important that the robot can 
move over much larger distances. This task-specific 
definition is quite wide and includes several types of 
very small robots as well as stationary micromanipula¬ 
tion systems, which are a few decimeters in size but can 
carry out very precise manipulation (in the micron or 
even nanometer range) [27.92]. 

Besides classification by task or size, microrobots 
can also be classified by their mobility and functional¬ 
ity [27.31,94]. Many robots usually consist of sensors 
and actuators, a control unit, and an energy source. 

Depending on the arrangement of these compo¬ 
nents, one can classify microrobots according to the fol¬ 
lowing criteria: locomotive and positioning possibility 
(yes or no), manipulation possibility (yes or no), control 
type (wireless or tethered), and autonomy. Figure 27.8 
illustrates 15 different possible microrobot configura¬ 
tions by combining the four criteria [27.31,94]. 


advantage is that the microgripper is less compact 
in size. This could become a major obstacle to its 
applications. 


As depicted in Fig. 27.8 (taken from [27.24]), the 
classification is dependent on the following microrobot 
components: the control unit (CU), the power source 
(PS), the actuators necessary for moving the robot 
platform (i. e., the robot drive for locomotion and posi¬ 
tioning; AP), and the actuators necessary for operation 
(i. e., manipulation using robot arms and hands; AM). 
Besides the different actuation functions, sensory func¬ 
tions are also needed, for example, tactile sensors for 
microgrippers or charge-coupled device (CCD) cam¬ 
eras for endoscopic applications (compare Fig. 27. 8d 
and a). 

The ultimate goal is to create a fully autonomous, 
wireless mobile microrobot equipped with suitable mi¬ 
crotools according to Fig. 27. 80 . Because this is a very 
difficult task, a good start is to investigate the possibility 
of making silicon microrobot platforms that are steered 
and powered through wires, like the one in Fig. 27.8c, 
and to study their locomotion capability. 

The majority of MEMS-based microrobotic de¬ 
vices developed so far could be categorized as move- 
able links - microcatheters [27.95,96], according 
to Fig. 27.8a, or microgrippers [27.25], such as those 
in Fig. 27. 8d, or the microgrippers [27.97, 98] shown 
in Fig. 27. 8e. Among the research publications covering 
locomotive microrobots, most publications have ad¬ 
dressed microconveyance systems (Fig. 27.8b) [27.99— 
102]. Robots using external sources for locomotion 
could be used (compare Fig. 27.8b,f,j,n). According 
to Fatikow and Rembold [27.92], several researchers 
are working on methods to navigate micromechanisms 
through human blood vessels; however, these micro¬ 
robots are difficult to control. Examples of partially 
autonomous systems (compare Fig. 27. 8j) are the con¬ 
cept for so-called smart pills. Centimeter-sized pills for 
sensing temperature and/or pH inside the body have 
been presented [27.103,104] as well as pills equipped 
with video cameras [27.105]. The pill is swallowed and 
transported to the part of the body where one wants 
to measure or record a video sequence. The informa¬ 
tion of the measured parameter or the signals from the 
camera is then transmitted (telemetrically) out of the 
body. More sophisticated approaches involving actua¬ 
tors for drug delivery of various kinds have also been 
proposed [27.92,104]. The position of the pill inside 
the body is located by an X-ray monitor or ultrasound. 
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Fig.27.8a-o Classification of microrobots by functionality (modification of earlier presented classification 
schemes [27.12,31,92,93]). (CU indicates the control unit; PS, the power source or power supply; AP, the actuators 
for positioning; AM, the actuators for manipulation): (a-c) Microrobots without manipulation functions (wire con¬ 
trolled), (d-g) microrobots for manipulation operations (wire controlled), (h — k) wireless microrobots for manipulation 
operations, (l-o) autonomous microrobots for manipulationg operations, (d,h,l) microrobots physically mounted to po¬ 
sitioning system fixed to external environment 


As soon as the pill reaches an infected area, a drug en¬ 
capsulated in the pill can be released by the actuators 
onboard. External communication could be realized 
through radio signals. 

Several important results have been presented re¬ 
garding walking microrobots (Fig. 27.8c,g,k) fabri¬ 
cated by MEMS technologies and batch manufactur¬ 
ing. Different approaches for surface-micromachined 
robots [27.106, 107] and for a piezoelectric dry- 
reactive-ion-etched microrobot should be mentioned. 
A suitable low-power application-specific integrated 
circuit (ASIC) for robot control has been successfully 
tested and is planned to be integrated on a walk¬ 
ing microrobot [27.108]. The large European Esprit 
project MINIMAN (1997) has the goal of developing 
moveable microrobotic platforms with integrated tools 
with six degrees of freedom for applications such as 
microassembly within an SEM and involves differ¬ 
ent MEMS research groups from several universities 
and companies across Europe. Further, miniature robot 
systems with MEMS/MST (microsystem technology) 
components have been developed [27.109]. 

Several research publications on gnat minirobots 
and actuator technologies for MEMS micro¬ 
robots [27.3,110] were reported by US researchers 
in the early 1990s, and several groups in Japan are 


also currently developing miniaturized robots based on 
MEMS devices [27.8]. In Japan, an extensive ten-year 
program on micromachine technology, supported 
by the Ministry of International Trade and Industry 
(MITI), started in 1991. One of the goals of this 
project is to create microsized and miniature robots for 
microfactory, medical technology, and maintenance 
applications. Several microrobotic devices, including 
locomotive robots and microconveyers, have been 
produced within this program. Miniature robot devices 
or vehicles [27.111] for locomotive tasks, containing 
several MEMS components, have been presented. 
Even though great efforts have been made on robot 
miniaturization using MEMS technologies, no experi¬ 
mental results on MEMS batch-fabricated microrobots 
suitable for autonomous walking (i. e., robust enough 
to be able to carry its own power source or to be 
powered by telemetric means) have been presented yet. 
The first batch-fabricated MEMS-based microrobot 
platform able to walk was presented in 1999 [27.112]. 
However, this robot was powered through wires and 
was not equipped with manipulation actuators. Besides 
walking microrobotic devices, several reports on fly¬ 
ing [27.113-115] and swimming [27.116] robots have 
been published. Micromotors and gear boxes made 
using LIGA technology (a high-precision, lithograph- 
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ically defined plating technology) are used to build 
small flying microhelicopters, which are commercially 
available from the Institute of Microtechnology in 
Mainz, Germany, as rather expensive demonstration 
objects [27.114]. Besides the pure mechanical micro¬ 
robots, hybrid systems consisting of electromechanical 
components and living organisms such as cockroaches 
have also been reported [27.1 17]. 

27.7.2 Bio-Microrobotics 

Biomanipulation entails such operations as position¬ 
ing, grasping, and injecting material into various lo¬ 
cations in cells. Research topics in bio-microrobotics 
include the autonomous manipulation of single cells 
or molecules, the characterization of biomembrane me¬ 
chanical properties using microrobotic systems with 
integrated vision and force sensing modules, and more. 
The objective is to obtain a fundamental understanding 
of single-cell biological systems and provide charac¬ 
terized mechanical models of biomembranes for de¬ 
formable cell tracking during biomanipulation and cell 
injury studies. 

Existing biomanipulation techniques can be clas¬ 
sified into noncontact manipulation including laser 
trapping [27.118-121] and electrorotation [27.122- 
124], and contact manipulation, referred to as me¬ 
chanical micromanipulation [27.125]. When laser trap¬ 
ping [27.1 18-121] is used for noncontact biomanipula¬ 
tion, a laser beam is focused through a large-numerical- 
aperture objective lens, converging to form an optical 
trap in which the lateral trapping force moves a cell in 
suspension toward the center of the beam. The longitu¬ 
dinal trapping force moves the cell in the direction of 
the focal point. The optical trap levitates the cell and 
holds it in position. Laser traps can work in a well- 
controlled manner. However, two features make laser 
trapping techniques undesirable for automated cell in¬ 
jection. The high dissipation of visible light in aqueous 
solutions requires the use of high-energy light close 
to the UV spectrum, raising the possibility of damage 
to the cell. Even though some researchers claim that 
such concerns could be overcome using wavelengths 
in the near-infrared (IR) spectrum [27.120], the ques¬ 
tion as to whether the incident laser beam might induce 
abnormalities in the cells’ genetic material still exists. 
One alternative to using laser beams is the electrorota¬ 
tion technique. Electric-field-induced rotation of cells 
was demonstrated by Mischel et al. [27.126], Arnold 
and Zimmermann [27.127], and Washizue t al. [27.124]. 
This noncontact cell manipulation technique is based 
on controlling the phase shift and magnitude of elec¬ 
tric fields. These fields, appropriately applied, produce 
a torque on the cell. Different system configurations 


have been established for cell manipulation based on 
this principle [27.122,123], which can achieve high ac¬ 
curacy in cell positioning. However, it lacks a means to 
hold the cell in place for further manipulation, such as 
injection, since the magnitude of the electric fields has 
to be kept low to ensure the viability of cells. The limits 
of noncontact biomanipulation in the laser trapping and 
electrorotation techniques make mechanical microma¬ 
nipulation desirable. The damage caused by laser beams 
in the laser trapping technique and the lack of a hold¬ 
ing mechanism in the electrorotation technique can be 
overcome by mechanical micromanipulation. 

To improve the low success rate of manual oper¬ 
ation, and to eliminate contamination, an autonomous 
robotic system (shown in Fig. 27.9) has been developed 
to deposit DNA into one of the two nuclei of a mouse 
embryo without inducing cell lysis [27.11,128]. The 
laboratory’s experimental results show that the success 
rate for the autonomous embryo pronuclei DNA injec¬ 
tion is dramatically improved over conventional manual 

Cell holding unit Inverted microscope 3DOF microrobot 



Readout circuit board with a wire bonded force sensor 


Fig. 27.9 Robotic biomanipulation system with vision and force 
feedback 



Fig. 27.10 Cell injection process, (a) Before and (b) during the in¬ 


jection of a mouse oocyte zona pellucida (ZP) 


Part B | 27.7 











Part B | 27.7 


690 Part B 


Design 


injection methods. The autonomous robotic system fea¬ 
tures a hybrid controller that combines visual servoing 
and precision position control, pattern recognition for 
detecting nuclei, and a precise autofocusing scheme. 
Figure 27.10 illustrates the injection process. 

To realize large-scale injection operations, a MEMS 
cell holder was fabricated using anodic wafer-bonding 
techniques. Arrays of holes are aligned on the cell 
holder, which are used to contain and fix individual 
cells for injection. When well calibrated, the system 
with the cell holder makes it possible to inject large 
numbers of cells using position control. The cell injec¬ 
tion operation can be conducted in a move-inject-move 
manner. 

A successful injection is determined greatly by in¬ 
jection speed and trajectory, and the forces applied to 
cells. To further improve the robotic system’s perfor¬ 
mance, a multi-axial MEMS-based capacitive cellular 
force sensor is being designed and fabricated to pro¬ 
vide realtime force feedback to the robotic system. 
The MEMS cellular force sensor also aids research in 
biomembrane mechanical property characterization. 

MEMS-Based Multi-Axis Capacitive Cellular 
Force Sensor 

The MEMS-based two-axis cellular force sen¬ 
sor [27.129] shown in Fig. 27.11 is capable of resolving 
normal forces applied to a cell as well as tangential 
forces generated by improperly aligned cell probes. 
A high-yield microfabrication process was developed 
to form the 3-D high-aspect-ratio structure using deep 
reactive ion etching (DRIE) on silicon-on-insulator 
(SOI) wafers. The constrained outer frame and the 
inner movable structure are connected by four curved 
springs. A load applied to the probe causes the inner 
structure to move, changing the gap between each pair 
of interdigitated comb capacitors. Consequently, the 
total capacitance change resolves the applied force. The 
interdigitated capacitors are orthogonally configured to 
make the force sensor capable of resolving forces in 
both the x- and y-directions. The cellular force sensors 
used in the experiments are capable of resolving forces 
up to 25 p. N with a resolution of 0.01 p N. 


Tip geometry affects the quantitative force mea¬ 
surement results. A standard injection pipette (Cook 
K-MPIP-1000-5) tip section with a tip diameter of 
5 |x m is attached to the probe of the cellular force sen¬ 
sors. 

The robotic system and high-sensitivity cellular 
force sensor are also applied to biomembrane mechan¬ 
ical property studies [27.130]. The goal is to obtain 
a general parameterized model describing cell mem¬ 
brane deformation behavior when an external load is 
applied. This parameterized model serves two chief 
purposes. First, in robotic biomanipulation, it allows 
online parameter recognition so that cell membrane 
deformation behavior can be predicted. Second, for 
a thermodynamic model of membrane damage in cell 
injury and recovery studies, it is important to appreciate 
the mechanical behavior of the membranes. This allows 
the interpretation of such reported phenomena as me¬ 
chanical resistance to cellular volume reduction during 
dehydration, and its relationship to injury. The estab¬ 
lishment of such a biomembrane model will greatly 
facilitate cell injury studies. 

Experiments demonstrate that robotics and MEMS 
technology can play important roles in biological stud¬ 
ies such as automating biomanipulation tasks. Aided 
by robotics, the integration of vision and force sensing 
modules, and MEMS design and fabrication techniques, 
investigations are being conducted in biomembrane me¬ 
chanical property modeling, deformable cell tracking, 
and single-cell and biomolecule manipulation. 

27.7.3 Bio-Mimetic/Untethered Microrobots 

Similar to the larger scale, bio-mimicking is a signifi¬ 
cant approach for the design of micro- and nanorobots. 
However, at the micro-/nanoscale, the objectives we are 
learning from are not animals, fishes, birds, or insects 
anymore, rather we learn from such categories as bac¬ 
teria, molecular motors, DNA molecules, and so on. 

For instance, it is well known that a variety of mi¬ 
croorganisms swim in liquid using flagella, a swimming 
strategy that is particularly well suited to their low- 
Reynolds-number regime [27.131]. Eukaryotic flagella 



Fig. 27.11 A cellular force sensor with 
orthogonal comb drives detailed 
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Fig.27.12a-e Artificial bacteria flagella, (a) SEM micrograph of an as-fabricated ABF with a diameter of 2.7 pm. (b) 
Schematic drawing of driving an ABF swims forward and backward as a rotary-to-linear motion converter. For forward 
and backward motion, the helical tail turns clockwise (CW) and counterclockwise (CCW), respectively (as seen by an 
observer in front of the ABF), (c) A series of frames taken from a video camera showing an ABF swim forward in 
water. The ABF has an 81 pm long InGaAs/GaAs/Cr helical tail, and a 4.5 pm x 4.5 pm square head. B,f, 9, tp represent 
four parameters, i. e., magnetic field, field frequency, yaw and pitch, which can be controlled by the coil pairs, (d) The 
dependence of ABF velocity to frequency of external magnetic field generated by the coil pairs, (e) Experimental setup. 
All examples are from the authors' work 


are active organelles that deform to create paddling 
motions, such as propagating waves or circular translat¬ 
ing movements. Previous work experimentally demon¬ 
strated that this kind of motion can be approximated by 
a microscopic artificial swimmer consisting of a chain 
of DNA-bound magnetic beads connected to a red 
blood cell [27.132]. In contrast, bacterial (prokaryotic) 
flagella work differently by using a molecular motor to 
turn the base of a flagellum or bundle of flagella, which 
deform passively into a helical structure [27.133]. 

One example inspired by this is artificial bacte¬ 
rial flagella (ABFs) [27.134], which are proposed as 
microrobots capable of locomotion in liquid. An ar¬ 
tificial bacterial flagellum consists of a helical tail 
resembling a natural flagellum, fabricated by the self¬ 
scrolling of helical nanobelts [27.135,136], and a thin 
soft-magnetic head, consisting of a Cr/Ni/Au multi¬ 
layer. Experimental investigation shows that an ABF 
can be propelled forward, backward and steered by 
a controlled external rotating magnetic field. ABFs have 
comparable swimming speed to bacteria propelled by 
flagella but under precise control. 

An artificial bacterial flagellum (ABF) consists of 
two parts: a helical tail and a soft-magnetic metal 


head as shown in Fig. 27.12. Previous work has shown 
that the geometrical shape of the helical tail, i. e., chi¬ 
rality, helicity angle and diameter, can be precisely 
controlled. To control the locomotion of the ABF re¬ 
motely, three orthogonal electromagnetic coil pairs 
were employed to generate a uniform rotating magnetic 
field [27.137], 

In principle, for forward and backward motion, the 
ABF acts as a corkscrew to convert rotary motion to 
linear motion as shown in Fig. 27.12b. Figure 27.12c 
shows a left-handed ABF swim controlled by coil pairs. 
It can be seen in Fig. 27. 12c that the ABF swim forward 
until the 10th second by rotating the magnet field clock¬ 
wise (CW). In contrast to a left-handed ABF, when the 
same field is applied to a right-handed ABF, though 
it rotates in the same direction as left-handed one, the 
swimming direction will be opposite. The magnetic 
torque for rotation is generated by the thin head at¬ 
tempting to align with the applied field, and torque for 
steering is generated by the diagonal axis of the square 
head attempting to align with the applied field, since 
the diagonal axis is the easy magnetization axis of thin 
square plates in weak applied fields. Thus, to steer the 
ABFs in water, two more parameters were applied, i. e., 
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yaw (0) and pitch (tp), for the turning and tilting of the 
ABF. 

The velocity of ABF is investigated as a function of 
magnetic field strength and rotation frequency in water 
(Fig. 27.12d. A 45 |xm long ABF is used for the tests 
with three different field strengths: 1.0 mT, 1.5 mT and 
2.0 mT. The field’s rotational frequency is increased 
from 5 FIz to 35 FIz. The results show that at low fre¬ 
quency the swimmer’s rotation is synchronized with the 
applied field, and ABF velocity increases linearly with 
frequency, as expected in the low-Reynolds-number 
regime [27.131]. After reaching a maximum value, the 
velocity reduces and becomes less deterministic with 
increasing field frequency, since the available magnetic 
torque is no longer sufficient to keep the swimmer 
synchronized with the applied field. The fluctuations 
in the curves are attributed to unmodeled boundary 
conditions, such as wall effects and intermolecular in¬ 
teractions of the ABF with the substrate. The maximum 
velocity we have achieved is 18 p/s, which is com¬ 
parable to bacteria, such as Escherichia coli [27.138], 

27.8 Nanorobotics 

Nanorobotics represents the next stage in miniaturiza¬ 
tion for maneuvering nanoscale objects. Nanorobotics 
is the study of robotics at the nanometer scale, 
and includes robots that are nanoscale in size, i. e., 
nanorobots, and large robots capable of manipulating 
objects that have dimensions in the nanoscale range 
with nanometer resolution, i. e., nanorobotic manip¬ 
ulators. Robotic manipulation at nanometer scales is 
a promising technology for structuring, characteriz¬ 
ing, and assembling nanoscale building blocks into 
NEMS. Combined with recently developed nanofab¬ 
rication processes, a hybrid approach is realized to 
build NEMS and other nanorobotic devices from indi¬ 
vidual carbon nanotubes and SiGe/Si nanocoils. Mate¬ 
rial science, biotechnology, electronics, and mechanical 
sensing and actuation will benefit from advances in 
nanorobotics. 

27.8.1 Introduction 

Nanomanipulation, or positional and/or force control 
at the nanometer scale, is a key enabling technol¬ 
ogy for nanotechnology by filling the gap between 
top-down and bottom-up strategies, and may lead to 
the appearance of replication-based molecular assem¬ 
blers [27.18]. These types of assemblers have been 
proposed as general-purpose manufacturing devices for 
building a wide range of useful products as well as 
copies of themselves (self-replication). 


which swim by rotating their flagella with a frequency 
of about 100 FIz at room temperature. The results also 
indicate that by exerting a stronger magnetic torque on 
the ABF, higher driving frequencies can be achieved 
resulting in higher linear velocity. By extrapolating ex¬ 
perimental results, if a frequency of 100 FIz can be 
achieved, the resulting swimming speed of ABFs may 
be beyond bacteria swimming speeds. 

Self-propelled devices such as these are of inter¬ 
est in fundamental research and biomedical applications 
for mimicking and understanding natural organisms, for 
signaling inter- or intra-cellular information, for manip¬ 
ulating cellular or sub-cellular objects, and for targeted 
drug delivery. In contrast to bacteria that are propelled 
by nature flagella and move somewhat randomly in 
liquid, ABF motion can be precisely controlled by 
magnetic fields. ABFs can be used as a test bed for 
understanding the swimming behavior of bacteria and 
other swimming microorganisms, and can potentially 
be used for targeted drug delivery and as wireless ma¬ 
nipulators for medical and biological applications. 


Presently, nanomanipulation can be applied to the 
scientific exploration of mesoscopic physical phenom¬ 
ena, biology, and the construction of prototype nan¬ 
odevices. It is a fundamental technology for prop¬ 
erty characterization of nanomaterials, nanostructures, 
and nanomechanisms, for the preparation of nanoscale 
building blocks, and for the assembly of nanodevices 
such as NEMS. 

Nanomanipulation was enabled by the inventions 
of the STM [27.19], AFMs [27.44], and other types 
of scanning probe microscopes (SPMs). Besides these, 
optical tweezers (laser trapping) [27.139] and mag¬ 
netic tweezers [27.140] are also potential nanomanip¬ 
ulators. Nanorobotic manipulators (NRMs) [27.141, 
142] are characterized by the capability of 3-D po¬ 
sitioning, orientation control, independently actuated 
multiple end-effectors, and independent real-time ob¬ 
servation systems, and can be integrated with scanning 
probe microscopes. NRMs largely extend the complex¬ 
ity of nanomanipulation. 

A concise comparison of STM, AFM, and NRM 
technology is shown in Fig. 27.13. With its incom¬ 
parable imaging resolution, an STM can be applied 
to particles as small as atoms with atomic resolution. 
Flowever, limited by its 2-D positioning and available 
strategies for manipulations, standard STMs are ill- 
suited for complex manipulation and cannot be used 
in 3-D space. An AFM is another important type 
of nanomanipulator. Manipulation with an AFM can 
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be done in either contact or dynamic mode. Gen¬ 
erally, manipulation with an AFM involves moving 
an object by touching it with a tip. A typical ma¬ 
nipulation starts by imaging a particle in noncontact 
mode, then removing the tip oscillation voltage and 
sweeping the tip across the particle in contact with 
the surface and with the feedback disabled. Mechan¬ 
ical pushing can exert larger forces on objects and, 
hence, can be applied for the manipulation of relatively 
larger objects. 1- to 3-D objects can be manipulated 
on a 2-D substrate. However, the manipulation of in¬ 
dividual atoms with an AFM remains a challenge. 
By separating the imaging and manipulation func¬ 
tions, nanorobotic manipulators can have many more 
degrees of freedom including rotation for orientation 
control, and hence can be used for the manipulation 
of 0-D (symmetric spheres) to 3-D objects in 3-D 
free space. Limited by the lower resolution of elec¬ 
tron microscopes, NRMs are difficult to use for the 
manipulation of atoms. However, their general robotic 
capabilities, including 3-D positioning, orientation con¬ 
trol, independently actuated multiple endeffectors, sep¬ 
arate real-time observation system, and integration with 
SPMs inside, make NRMs quite promising for complex 
nanomanipulation. 

The first nanomanipulation experiment was per¬ 
formed by Eigler and Schweizer in 1990 [27.143], They 
used an STM and materials at low temperatures (4 K) 
to position individual xenon atoms on a single-crystal 
nickel surface with atomic precision. The manipula¬ 
tion enabled them to fabricate rudimentary structures 
of their own design, atom by atom. The result is the fa¬ 
mous set of images showing how 35 atoms were moved 
to form the three-letter logo IBM , demonstrating that 
matter could indeed be maneuvered atom by atom as 
Feynman suggested [27.1]. 

A nanomanipulation system generally includes 
nanomanipulators as the positioning device, micro¬ 
scopes as eyes , various end-effectors including probes 
and tweezers among others as its fingers, and various 
types of sensors (force, displacement, tactile, strain, 
etc.) to facilitate the manipulation and/or to determine 
the properties of the objects. Key technologies for 
nanomanipulation include observation, actuation, mea¬ 
surement, system design and fabrication, calibration 
and control, communication, and the human-machine 
interface. 

Strategies for nanomanipulation are basically deter¬ 
mined by the environment - air, liquid or vacuum - 
which is further decided by the properties and size 
of the objects and observation methods. Figure 27.14 
depicts the microscopes, environments, and strategies 
of nanomanipulation. In order to observe manipulated 
objects, STMs can provide sub-Angstrom imaging res¬ 



Scale of objects (nm) 

Fig. 27.13 Comparison of nanomanipulators 
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Fig. 27.14 Microscopes, environments, and strategies of 
nanomanipulation 

olution, whereas AFMs can provide atomic resolutions. 
Both can obtain 3-D surface topology. Because AFMs 
can be used in an ambient environment, they provide 
a powerful tool for biomanipulation that may require 
a liquid environment. The resolution of SEM is limited 
to about 1 nm, whereas field-emission SEM (FESEM) 
can achieve higher resolutions. SEM/FESEM can be 
used for real-time 2-D observation for both the objects 
and end-effectors of manipulators, and large ultrahigh- 
vacuum (UHV) sample chambers provide enough space 
to contain an NRM with many degrees of freedom 
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(DOFs) for 3-D nanomanipulation. However, the 2-D 
nature of the observation makes positioning along the 
electron-beam direction difficult. High-resolution trans¬ 
mission electron microscopes (HRTEM) can provide 
atomic resolution. However, the narrow UHV specimen 
chamber makes it difficult to incorporate large manipu¬ 
lators. In principle, optical microscopes (OMs) cannot 
be used for nanoscale (smaller than the wavelength of 
visible lights) observation because of diffraction limits. 
Scanning near-field OMs (SNOMs) break this limi¬ 
tation and are promising as a real-time observation 
device for nanomanipulation, especially for ambient en¬ 
vironments. SNOMs can be combined with AFMs, and 
potentially with NRMs for nanoscale biomanipulation. 
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Fig.27.15a-c Basic strategies of nanomanipulation. In the figure, 
A, B, C,.. .represent the positions of end-effector (e.g., a tip); A', 
B', C'.. the positions of objects; 1, 2, 3,...the motions of end- 
effector; and 1', 2', 3V . .the motions of objects. Tweezers can be 
used in pick-and-place to facilitate the picking-up, but are gen¬ 
erally not necessarily helpful for placing, (a) Lateral noncontact 
nanomanipulation (sliding), (b) Lateral contact nanomanipulation 
(pushing/pulling), (c) Vertical nanomanipulation (picking and plac¬ 
ing) 


Nanomanipulation processes can be broadly classi¬ 
fied into three types: 

1. Lateral noncontact 

2. Lateral contact 

3. Vertical manipulation. 

Generally, lateral noncontact nanomanipulation is 
mainly applied for atoms and molecules in UHV with 
an STM or bio-object in liquid using optical or mag¬ 
netic tweezers. Contact nanomanipulation can be used 
in almost any environment, generally with an AFM, but 
is difficult for atomic manipulation. Vertical manipula¬ 
tion can be performed by NRMs. Figure 27.15 shows 
the processes of the three basic strategies. 

Motion of the lateral noncontact manipulation 
processes is shown in Fig. 27.15a. Applicable ef¬ 
fects [27.144] able to cause the motion include long- 
range van der Waals (vdW) forces (attractive) generated 
by the proximity of the tip to the sample [27.145], 
electric-field-induced fields by the voltage bias be¬ 
tween the tip and the sample [27.146,147], tunnel¬ 
ing current local heating or inelastic tunneling vibra¬ 
tion [27.148, 149]. With these methods, some nanode¬ 
vices and molecules have been assembled [27.150, 
151]. Laser trapping (optical tweezers) and magnetic 
tweezers are possible for noncontact manipulation of 
nanoscale biosamples, e.g., DNA [27.152,153]. 

Noncontact manipulation combined with STMs has 
revealed many possible strategies for manipulating 
atoms and molecules. However, for the manipulation 
of carbon nanotubes (CNTs) no examples have been 
demonstrated. 

Pushing or pulling nanometer objects on a sur¬ 
face with an AFM is a typical manipulation using this 
method as shown in Fig. 27.15b. Early work showed 
the effectiveness of this method for the manipulation of 
nanoparticles [27.154—158]. This method has also been 
shown in nanoconstruction [27.159] and biomanipula¬ 
tion [27.160]. A virtual-reality interface facilitates such 
manipulation [27.161-163] and may create an opportu¬ 
nity for other types of manipulation. This technique has 
been used in the manipulation of nanotubes on a sur¬ 
face, and some examples will be introduced later in this 
chapter. 

The pick-and-place task as shown in Fig. 27.15c is 
especially significant for 3-D nanomanipulation since 
its main purpose is to assemble prefabricated building 
blocks into devices. The main difficulty is in achiev¬ 
ing sufficient control of the interaction between the tool 
and object and between the object and the substrate. 
Two strategies have been presented for micromanip¬ 
ulation [27.164] and have also proven to be effective 
for nanomanipulation [27.142]. One strategy is to apply 
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a dielectrophoretic force between a tool and an object 
as a controllable additional external force by applying 
a bias between the tool and the substrate on which the 
object is placed. Another strategy is to modify the van 
der Waals and other intermolecular and surface forces 
between the object and the substrate. For the former, 
an AFM cantilever is ideal as one electrode to generate 
a nonuniform electrical field between the cantilever and 
the substrate. 

27.8.2 Nanorobotic Manipulation Systems 

Nanorobotic manipulators are the core components of 
nanorobotic manipulation systems. The basic require¬ 
ments for a nanorobotic manipulation system for 3-D 
manipulation include nanoscale positioning resolution, 
a relative large working space, enough DOFs includ- 


a) MM3A b) Installation 




Fig. 27.16 (a) Nanomanipulator MM3A from Klein- 
diek (b) inside an SEM (c) kinematic model 


Table 27.4 Specifications of MM3A 


Item 

Specification 

Operating range q\ and q 2 

240° 

Operating range Z 

12 mm 

Resolution A (horiz.) 

10 —7 rad (5 nm) 

Resolution B (vert.) 

I0 —7 rad (3.5 nm) 

Resolution C (linear) 

0.25 nm 

Fine (scan) range A 

20 )im 

Fine (scan) range B 

15 pm 

Fine (scan) range C 

i pm 

Speed A, B 

10 mm/s 

Speed C 

2 mm/s 


ing rotational ones for 3-D positioning and orientation 
control of the end-effectors, and usually multiple end 
effectors for complex operations. 

A commercially available nanomanipulator 
(MM3A from Kleindiek) installed inside an SEM (Carl 
Zeiss DSM962) is shown in Fig. 27.16. The manip¬ 
ulator has three degrees of freedom, and nanometer- 
to subnanometer-scale resolution (Table 27.4). Calcu¬ 
lations show that when moving/scanning in the A/B 
direction at joint qi/q 2 , the additional linear motion 
in C is very small. For example, when the arm length 
is 50 mm, the additional motion in the C direction is 
only 0.25 nm to lnm when moving 5-10 |xm; in the 
A direction; these errors can be ignored or compensated 
with an additional motion of the prismatic joint p$, 
which has a 0.25 nm resolution. 

Figure 27.17a shows a nanorobotic manipulation 
system that has 16 DOFs in total and can be equipped 
with three or four AFM cantilevers as end-effectors for 
both manipulation and measurement. The positioning 
resolution is sub-nanometer and strokes are on the order 



Nano fabrication system 
Nano instrumentation system 
Nanorobotic manipulation system 


Fig.27.17a,b Nanorobotic system, (a) Nanorobotic manip¬ 
ulators. (b) System setup 
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of centimeters. The manipulation system is not only for 
nanomanipulation, but also for nanoassembly, nanoin¬ 
strumentation, and nanofabrication. Four-probe semi¬ 
conductor measurements are perhaps the most complex 
manipulation that this system can perform, because it is 
necessary to actuate four probes independently by using 
four manipulators. With the advancement of nanotech¬ 
nology, one could shrink the size of nanomanipulators 
and insert more DOFs inside the limited vacuum cham¬ 
ber of a microscope, and perhaps the molecular version 
of manipulators such as that dreamed of by Drexler 
could be realized [27.18]. 

For the construction of multi-walled carbon nan¬ 
otube (MWNT)-based nanostructures, manipulators 
position and orient nanotubes for the fabrication 
of nanotube probes and emitters, for performing 
nanosoldering with electron-beam-induced deposition 
(EBID) [27.165], for the property characterization of 
single nanotubes for selection purposes and for char¬ 
acterizing junctions to test connection strength. 

A nanolaboratory is shown in Fig. 27.17b. The 
nanolaboratory integrates a nanorobotic manipulation 
system with a nanoanalytical system and a nanofab¬ 
rication system, and can be applied for manipulating 
nanomaterials, fabricating nanoscale building blocks, 
assembling nanodevices, and for in situ analysis of 
the properties of such materials, building blocks, and 
devices. Nanorobotic manipulation within the nanolab¬ 
oratory has opened a new path for constructing 
nanosystems in 3-D space, and will create opportuni¬ 
ties for new nanoinstrumentation and nanofabrication 
processes. 

27.8.3 Nanorobotic Manipulation 
and Assembly 

Nanomanipulation is a promising strategy for nano¬ 
assembly. Key techniques for nanoassembly include the 
structuring and characterization of nanoscale building 
blocks, the positioning and orientation control of the 
building blocks with nanoscale resolution, and effec¬ 
tive connection techniques. Nanorobotic manipulation, 
which is characterized by multiple DOFs with both po¬ 
sition and orientation controls, independently actuated 
multiprobes, and a real-time observation system, have 
been shown effective for assembling nanotube-based 
devices in 3-D space. 

The well-defined geometries, exceptional mechan¬ 
ical properties, and extraordinary electric characteris¬ 
tics, among other outstanding physical properties of 
CNTs [27.166] qualify them for many potential appli¬ 
cations, especially in nanoelectronics [27.167], NEMS, 
and other nanodevices [27.168]. For NEMS, some 
of the most important characteristics of nanotubes 


include their nanometer diameter, large aspect ratio 
(10—1000), TPa-scale Young’s modulus [27.169-171], 
excellent elasticity [27.141], ultralow interlayer fric¬ 
tion, excellent capability for field emission, various 
electric conductivities [27.172], high thermal conduc¬ 
tivity [27.173], high current-carrying capability with 
essentially no heating [27.174], sensitivity of conduc¬ 
tance to various physical or chemical changes, and 
charge-induced bond-length change. 

Helical 3-D nanostructures, or nanocoils, have 
been synthesized from various materials, including 
helical carbon nanotubes [27.175] and zinc oxide 
nanobelts [27.176]. A new method of creating struc¬ 
tures with nanoscale dimensions has recently been pre¬ 
sented [27.177] and can be fabricated in a controllable 
way [27.178,179]. The structures are created through 
a top-down fabrication process in which a strained 
nanometer-thick heteroepitaxial bilayer curls up to form 
3-D structures with nanoscale features. Helical ge¬ 
ometries and tubes with diameters between 10 nm and 
10 |im have been achieved. Because of their inter¬ 
esting morphology, mechanical, electrical, and elec¬ 
tromagnetic properties, potential applications of these 
nanostructures in NEMS include nanosprings [27.180], 
electromechanical sensors [27.181], magnetic field de¬ 
tectors, chemical or biological sensors, generators 
of magnetic beams, inductors, actuators, and high- 
performance electromagnetic wave absorbers. 

NEMS based on individual carbon nanotubes 
and nanocoils are of increasing interest, indicating 
that capabilities for incorporating these individual 
building blocks at specific locations on a device 
must be developed. Random spreading [27.182], di¬ 
rect growth [27.183], self-assembly [27.184], dielec- 
trophoretic assembly [27.185,186], and nanomanipula¬ 
tion [27.187] have been demonstrated for positioning 
as-grown nanotubes on electrodes for the construction 
of these devices. However, for nanotube-based struc¬ 
tures, nanorobotic assembly is still the only technique 
capable of in situ structuring, characterization, and as¬ 
sembly. Because the as-fabricated nanocoils are not 
free-standing from their substrate, nanorobotic assem¬ 
bly is virtually the only way to incorporate them into 
devices at present. 

Nanorobotic Assembly of Carbon Nanotubes 
Nanotube manipulation in two dimensions on a sur¬ 
face was first performed with an AFM by contact 
pushing on a substrate. Figure 27.18 shows the typi¬ 
cal methods for 2-D pushing. Although similar to that 
shown in Fig. 27.15b, the same manipulation caused 
various results because nanotubes cannot be regarded 
as a 0-D point. The first demonstration was given by 
Wong et al. for measuring the mechanical properties of 
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a nanotube [27.188]. They adopt the method shown 
in Fig. 27.18b, i. e., to bend a nanotube by pushing 
one end of it and fixing the other end. The same strat¬ 
egy was used for the investigation of the behavior of 
nanotubes under large strain [27.189]. Postma et al. 
applied the strategies shown in Fig. 27.18c,d to ob¬ 
tain a kinked junction and crossed nanotubes [27.190]. 
Hertel et al. combined this technique with an inverse 
process, namely straightening by pushing along a bent 
tube, and realized the translation of the tube to an¬ 
other location [27.191] and between two electrodes 
to measure the conductivity [27.192]. This technique 
was also used to place a tube on another tube to 
form a single-electron transistor (SET) with a cross¬ 
junction of nanotubes [27 .193]. Pushing-induced break¬ 
ing (Fig. 27.1 8d) has also been demonstrated for a nan¬ 
otube [27.191]. The simple assembly of two bent tubes 
and a straight one formed a Greek letter 9. To investi¬ 
gate the dynamics of rolling at the atomic level, rolling 
and sliding of a nanotube (as shown in Fig. 27.1 8e,f) are 
performed on graphite surfaces using an AFM [27. 194]. 
Besides pushing and pulling, another important process 
is indentation. By indenting a surface, mechanical prop¬ 
erty characterization [27.195] and data storage [27.196] 
can be realized. 

Manipulation of CNTs in 3-D space is important 
for assembling CNTs into structures and devices. Basic 
techniques for the nanorobotic manipulation of car¬ 
bon nanotubes are shown in Fig. 27.19 [27.197]. These 
serve as the basis for handling, structuring, characteriz¬ 
ing, and assembling NEMS. 

The basic procedure is to pick up a single tube 
from nanotube soot (Fig. 27.19a). This has been 
shown first by using dielectrophoresis [27.142] through 
nanorobotic manipulation (Fig. 27.19b). By applying 
a bias between a sharp tip and a plane substrate, 
a nonuniform electric field can be generated between 
the tip and the substrate with the strongest field near the 
tip. This field can cause a tube to orient along the field 
or further jump to the tip by electrophoresis or dielec¬ 
trophoresis (determined by the conductivity of objective 
tubes). Removing the bias, the tube can be placed at 
other locations at will. This method can be used for 
free-standing tubes on nanotube soot or on a rough 
surface on which surface van der Waals forces are gen¬ 
erally weak. A tube strongly rooted in CNT soot or 
lying on a flat surface cannot be picked up in this way. 
The interaction between a tube and the atomic flat sur¬ 
face of AFM cantilever tip has been shown to be strong 
enough for picking up a tube onto the tip [27.198] 
(Fig. 27.19c). By using EBID, it is possible to pick up 
and fix a nanotube onto a probe [27.199] (Fig. 27.19d). 
For handling a tube, weak connection between the tube 
and the probe is desired. 



Fig.27.18a-f Two-dimensional manipulation of CNTs. Starting 
from the original state shown in (a), pushing the tube at different 
site with different force may cause the tube to deform as in (b) and 
(c), to break as in (d), or to move as in (e) and (f). (a) Original state, 
(b) Bending, (c) Kinking, (d) Breaking, (e) Rolling, (f) Sliding 

Bending and buckling a CNT as shown in 
Fig. 27.19e,f are important for in situ property char¬ 
acterization of a nanotube [27.200], which is a simple 
way to obtain the Young’s modulus of a nanotube 
without damaging the tube (if performed within its 
elastic range) and hence can be used for the selec¬ 
tion of a tube with desired properties. By buckling an 
MWNT over its elastic limit, a kinked structure can 
be obtained [27.201]. To obtain any desired angle for 
a kinked junction it is possible to fix the shape of 
a buckled nanotube within its elastic limit by using 
EBID [27.202]. For a CNT, the maximum angular dis¬ 
placement will appear at the fixed left end under pure 
bending or at the middle point under pure buckling. 

A combination of these two kinds of loads will achieve 
a controllable position of the kinked point and a desired 
kink angle. If the deformation is within the elastic limit 
of the nanotube, it will recover as the load is released. 

To avoid this, EBID can be applied at the kinked point 
to fix the shape. 

Stretching a nanotube between two probes or 
a probe and a substrate has generated several interest¬ 
ing results (Fig. 27.19g). The first demonstration of 3-D 
nanomanipulation of nanotubes took this as an exam¬ 
ple to show the breaking mechanism and to measure 
the tensile strength of CNTs [27.141], By breaking an 
MWNT in a controlled manner, interesting nanodevices 
have been fabricated. This technique - destructive fab¬ 
rication - has been presented to get sharpened and lay- 
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Fig.27.19a-h Nanorobotic manipulation of CNTs. The basic technique is to pick up an individual tube from CNT soot 
(as in (a)) or from an oriented array; (b) shows a free-standing nanotube picked up by dielectrophoresis generated by 
a nonuniform electric field between the probe and substrate, (c) (after [27.196]) and (d) show the same manipulation by 
contacting a tube with the probe surface or fixing (e.g., with EBID) a tube to the tip (inset shows the EBID deposit). 
Vertical manipulation of nanotubes includes bending (e), buckling (f), stretching/breaking (g), and connecting/bonding 
(h). All examples with the exception of (i) are from the authors' work 


ered structures of nanotubes, and to improve the length 
control of a nanotube [27.201]. Typically, a layered and 
a sharpened structure can be obtained from this process, 
similar to that achieved from electric pulses [27.202]. 
Bearing motion has also been observed in an incom¬ 
pletely broken MWNT [27.201]. The interlayer friction 
has been shown to be very small [27.203,204]. 

The reverse process, namely the connection of bro¬ 
ken tubes (Fig. 27.19h), has been demonstrated re¬ 
cently, and the mechanism is revealed as rebonding 
of unclosed dangling bonds at the ends of broken 
tubes [27.205]. Based on this interesting phenomenon, 


mechanochemical nanorobotic assembly has been per¬ 
formed [27.206]. 

Assembly of nanotubes is a fundamental technol¬ 
ogy for enabling nanodevices. The most important 
tasks include the connection of nanotubes and plac¬ 
ing of nanotubes onto electrodes. Pure nanotube cir¬ 
cuits [27.206] created by interconnecting nanotubes of 
different diameters and chirality could lead to further 
size reductions in devices. Nanotube intermolecular 
and intramolecular junctions are basic elements for 
such systems [27.207]. Room-temperature (RT) single¬ 
electron transistors (SETs) [27.208] have been shown 
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with a short 20 nm) nanotube section that is cre¬ 
ated by inducing local barriers into the tube with an 
AFM, and Coulomb charging has been observed. With 
a cross-junction of two single-walled carbon nanotubes 
(SWNTs) (semiconducting/metallic), three- and four- 
terminal electronic devices have been made [27.209]. 
A suspended cross-junction can function as an elec¬ 
tromechanical nonvolatile memory [27.210]. 

Although some kinds of junctions have been synthe¬ 
sized with chemical methods, there is no evidence yet 
showing that a self-assembly-based approach can pro¬ 
vide more complex structures. SPMs were also used to 
fabricate junctions, but they are limited to a 2-D plane. 
We have presented 3-D nanorobotic manipulation- 
based nanoassembly, which is a promising strategy both 
for the fabrication of nanotube junctions and for the 
construction of more complex nanodevices with such 
junctions as well. 

Nanotube junctions can be classified into differ¬ 
ent types by the kinds of components (SWNTs or 
MWNTs), geometric configurations (V kink, I, X cross, 
T, Y branch, and 3-D junctions) conductivity (metallic 
or semiconducting), and connection methods (inter- 
molecular (connected with van der Waals force, EBID, 
etc.) or intramolecular (connected with chemical bonds) 
junctions). Here we show the fabrication of several 
kinds of MWNT junctions by emphasizing the con¬ 
nection methods. These methods will also be effective 
for SWNT junctions. Figure 27.20 shows CNT junc¬ 
tions constructed by connecting with van der Waals 
forces (a), joining by electron-beam-induced deposi¬ 
tion (b), and bonding through mechanochemistry (c). 

Figure 27.20a shows a T-junction connected with 
van der Waals forces, which is fabricated by positioning 
the tip of an MWNT onto another MWNT until they 
form a bond. The contact is checked by measuring the 
shear connection force. 

EBID provides a soldering method to obtain 
stronger nanotube junctions than those connected 
through van der Waals forces. Hence, if the strength 
of nanostructures is emphasized, EBID can be applied. 
Figure 27.20b shows an MWNT junction connected 
through EBID, in which the upper MWNT is a sin¬ 
gle one with 20 nm in diameter and the lower one is 
a bundle of MWNTs with an extruded single CNT 
with 0 30 nm. The development of conventional EBID 
has been limited by the expensive electron filament 
used and low productivity. We have presented a paral¬ 
lel EBID system by using CNTs as emitters because of 
their excellent field-emission properties [27.211]. The 
feasibility of parallel EBID is presented. It is a promis¬ 
ing strategy for large-scale fabrications of nanotube 
junctions. As in its macroscale counterpart, welding, 
EBID works by adding materials to obtain stronger 
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Fig.27.20a-c MWNT junctions, (a) MWNTs connected with van 
der Waals forces, (b) MWNTs joined with EBID. (c) MWNTs 
bonded with a mechanochemical reaction 


connections, but in some cases, the added material 
might influence normal functions for nanosystems. So, 
EBID is mainly applied to nanostructures rather than 
nanomechanisms. 

To construct stronger junctions without adding ad¬ 
ditional materials, mechanochemical nanorobotic as¬ 
sembly is an important strategy. Mechanochemical 
nanorobotic assembly is based on solid-phase chemi¬ 
cal reactions, or mechanosynthesis, which is defined as 
chemical synthesis controlled by mechanical systems 
operating with atomic-scale precision, enabling direct 
positional selection ofreaction sites [27.18]. By picking 
up atoms with dangling bonds rather than natural atoms 
only, it is easier to form primary bonds, which provides 
a simple but strong connection. Destructive fabrication 
provides a way to form dangling bonds at the ends of 
broken tubes. Some of the dangling bonds may close 
with neighboring atoms, but generally a few bonds will 
remain dangling. A nanotube with dangling bonds at its 
end will bind more easily to another to form intramolec¬ 
ular junctions. Figure 27.20c shows such a junction. 

Three-dimensional nanorobotic manipulation has 
opened a new route for the structuring and assembly 
of nanotubes into nanodevices. However, at present 
nanomanipulation is still performed in a serial manner 
with master-slave control, which is not a large-scale 
production-oriented technique. Nevertheless, with ad¬ 
vances in the exploration of mesoscopic physics, better 
control on the synthesis of nanotubes, more accurate ac¬ 
tuators, and effective tools for manipulation, high-speed 
and automatic nanoassembly will be possible. Another 
approach might be parallel assembly by positioning 
building blocks with an array of probes [27.212] and 
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joining them together simultaneously, e.g., with the par¬ 
allel EBID [27.199] we presented. Further steps might 
progress towards exponential assembly [27.213], and in 
the far future to self-replicating assembly [27.18]. 

Nanorobotic Assembly of Nanocoils 
The construction of nanocoil-based NEMS involves the 
assembly of as-grown or as-fabricated nanocoils, which 
is a significant challenge from a fabrication stand¬ 
point. Focusing on the unique aspects of manipulating 
nanocoils due to their helical geometry, high elastic¬ 
ity, single-end fixation, and strong adhesion of the coils 
to the substrate from wet etching, a series of new pro¬ 
cesses has been presented using a manipulator (MM3 A, 
Kleindiek) installed in an SEM (Zeiss DSM962). As- 
fabricated SiGe/Si bilayer nanocoils (thickness 20 nm 
without Cr layer or 41 nm with Cr layer; diameter D = 
3.4 pm) are manipulated. Special tools have been fab¬ 
ricated including a nanohook prepared by controlled 
tip-crashing of a commercially available tungsten sharp 
probe (Picoprobe T-4-10-1 mm and T-4-10) onto a sub¬ 
strate, and a sticky probe prepared by tip dipping into 
a double-sided SEM silver conductive tape (Ted Pella, 
Inc.). As shown in Fig. 27.21, experiments demon¬ 


strate that nanocoils can be released from a chip by 
lateral pushing, picked up with a nanohook or a sticky 
probe, and placed between the probe/hook and another 
probe or an AFM cantilever (Nano-probe, NP-S). Ax¬ 
ial pulling/pushing, radial compressing/releasing, and 
bending/buckling have also been demonstrated. These 
processes have shown the effectiveness of manipulation 
for the characterization of coil-shaped nanostructures 
and their assembly for NEMS, which have been oth¬ 
erwise unavailable. 

Configurations of nanodevices based on individ¬ 
ual nanocoils are shown in Fig. 27.22. Cantilevered 
nanocoils as shown in Fig. 27.22a can serve as 
nanosprings. Nanoelectromagnets, chemical sensors, 
and nanoinductors involve nanocoils bridged between 
two electrodes as shown in Fig. 27.22b. Electromechan¬ 
ical sensors can use a similar configuration but with 
one end connected to a moveable electrode as shown 
in Fig. 27.22c. Mechanical stiffness and electric con¬ 
ductivity are fundamental properties for these devices 
that must be further investigated. 

As shown in Fig. 27.21h, axial pulling is used to 
measure the stiffness of a nanocoil. A series of SEM 
images are analyzed to extract the AFM tip displace- 



Fig.27.21a-h Nanorobotic manipulation of nanocoils (a) original state, (b) compressing/releasing, (c) hooking, (d) lateral 
pushing/breaking, (e) picking, (f) placing/inserting, (g) bending, and (h) pushing and pulling 
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ment and the nanospring deformation, i. e., the relative 
displacement of the probe from the AFM tip. From this 
displacement data and the known stiffness of the AFM 
cantilever, the tensile force acting on the nanospring 
versus the nanospring deformation was plotted. The de¬ 
formation of the nanospring was measured relative to 
the first measurement point. This was necessary be¬ 
cause the proper attachment of the nanospring to the 
AFM cantilever must be verified. Afterwards, it was not 
possible to return to the point of zero deformation. In¬ 
stead, the experimental data as presented in Fig. 27.22d 
has been shifted such that with the calculated linear 
elastic spring stiffness the line begins at zero force and 
zero deformation. From Fig. 27.22d, the stiffness of the 
spring was estimated to be 0.0233 N/m. The linear elas¬ 
tic region of the nanospring extends to a deformation 
of 4.5 pm. An exponential approximation was fitted to 
the nonlinear region. When the applied force reached 
0.176pN, the attachment between the nanospring and 
the AFM cantilever broke. Finite-element simulation 
(ANSYS 9.0) was used to validate the experimental 
data [27.181]. Since the exact region of attachment can¬ 
not be identified from the SEM images, simulations 
were conducted for 4, 4.5, and 5 turns to estimate the 
possible range according to the apparent number of 
turns of the nanospring. The nanosprings in the sim¬ 


ulations were fixed at one end and had an axial load 
of 0.106 pN applied at the other end. The simulation 
results for the spring with four turns yield a stiffness 
of 0.0302 N/m; for the nanospring with 5 turns it is 
0.0191 N/m. The measured stiffness falls within this 
range with 22.0% above the minimum value and 22.8% 
below the maximum value, and very close to the stiff¬ 
ness of a 4.5-turn nanospring, which has a stiffness of 
0.0230 N/m according to the simulation. 

Figure 27.22e shows the results from electrical char¬ 
acterization experiments on a nanospring with 11 turns 
using the configuration shown in Fig. 27.21g. The 7— 
V curve is nonlinear, which may be caused by the 
resistance change of the semiconductive bilayer due 
to ohmic heating. Another possible reason is the de¬ 
crease in contact resistance caused by thermal stress. 
The maximum current was found to be 0.159 mA un¬ 
der an 8.8 V bias. Higher voltage causes the nanospring 
to blow off. From the fast scanning screen of the SEM, 
an extension of the nanospring on probes was observed 
around the peak current so that the current does not drop 
abruptly. At 9.4 V, the extended nanospring is broken 
down, causing an abrupt drop in the I-V curve. 

From fabrication and characterization results, the 
helical nanostructures appear to be suitable to function 
as inductors. They would allow further miniaturization 
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Fig.27.22a-e Nanocoil-based devices. Cantilevered nanocoils (a) can serve as nanosprings. Nanoelectromagnets, chem¬ 
ical sensors, and nanoinductors involve nanocoils bridged between two electrodes (b). Electromechanical sensors can 
use a similar configuration but with one end connected to a moveable electrode (c). Mechanical stiffness (d) and electric 
conductivity (e) are basic properties of interest for these devices 
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compared to state-of-the-art microinductors. For this 
purpose, higher doping of the bilayer and an additional 
metal layer would result in the required conductance. 
Conductance, inductance, and quality factor can be 
further improved if, after curling up, additional metal 
is electroplated onto the helical structures. Moreover, 
a semiconductive helical structure, when functional¬ 
ized with binding molecules, can be used for chemi¬ 
cal sensing under the same principle as demonstrated 
with other types of nanostructures. With bilayers in 
the range of a few monolayers, the resulting struc¬ 
tures would exhibit very high surface-to-volume ra¬ 
tio with the whole surface exposed to an incoming 
analyst. 

27.8.4 Nanorobotic Systems 

Nanorobotic devices involve tools, sensors, and actua¬ 
tors at the nanometer scale. Shrinking device size makes 
it possible to manipulate nanosized objects with nano¬ 
sized tools, measure mass in femtogram ranges, sense 


force at piconewton scales, and induce GHz motion, 
among other amazing advancements. 

Top-down and bottom-up strategies for manufactur¬ 
ing such nanodevices have been independently investi¬ 
gated by a variety of researchers. Top-down strategies 
are based on nanofabrication and include technologies 
such as nanolithography, nanoimprinting, and chemical 
etching. Presently, these are 2-D fabrication processes 
with relatively low resolution. Bottom-up strategies are 
assembly-based techniques. At present, these strate¬ 
gies include such techniques as self-assembly, dip-pen 
lithography, and directed self-assembly. These tech¬ 
niques can generate regular nanopattems at large scales. 
With the ability to position and orient nanoscale objects, 
nanorobotic manipulation is an enabling technology for 
structuring, characterizing and assembling many types 
of nanosystems [27.197]. By combining bottom-up and 
top-down processes, a hybrid nanorobotic approach 
based on nanorobotic manipulation provides a third way 
to fabricate NEMS by structuring as-grown nanoma¬ 
terials or nanostructures. This new nanomanufacturing 
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Fig.27.23a-h Configurations of individual nanotube-based NEMS. Scale bars : (a) 1 |im ( inset'. 100 nm), (b) 200 nm, (c) 
1 |im, (d) 100 nm, (e) and (f) 1 |xm, (g) 20 |im, and (h) 300 nm. All examples are from the authors’ work 
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Fig. 27.24 A roadmap for nanorobotic 
systems 


technique can be used to create complex 3-D nanode¬ 
vices with such building blocks. Nanomaterial science, 
bio-nanotechnology, and nanoelectronics will also ben¬ 
efit from advances in nanorobotic assembly. 

The configurations of nanotools, sensors, and ac¬ 
tuators based on individual nanotubes that have been 
experimentally demonstrated are summarized as shown 
in Fig. 27.23. 

For detecting deep and narrow features on a surface, 
cantilevered nanotubes (Fig. 27.23a, [27.199]) have 
been demonstrated as probe tips for an AFM [27.206], 
a STM, and other types of SPM. Nanotubes provide 
ultrasmall diameters, ultralarge aspect ratios, and excel¬ 
lent mechanical properties. Manual assembly [27.214], 
and direct growth [27.215] have proven effective for 
their construction. Cantilevered nanotubes have also 
been demonstrated as probes for the measurement 
of ultrasmall physical quantities, such as femtogram 
masses [27.170], piconewton-order force sensors, and 
mass flow sensors [27.197] on the basis of their 
static deflections or change of resonant frequencies 
detected within an electron microscope. Deflections 
cannot be measured from micrographs in real time, 
which limits the application of this kind of sensor. 
Interelectrode distance changes cause emission cur¬ 
rent variation of a nanotube emitter and may serve as 


a candidate to replace microscope images. Bridged in¬ 
dividual nanotubes (Fig. 27.23b, [27.185]) have been 
the basis for electric characterization. Opened nan¬ 
otubes (Fig. 27.23c, [27.216]) can serve as atomic or 
molecular containers, thermometers [27.217], or spot 
welders [27.218]. The electrostatic deflection of a nan¬ 
otube has been used to construct a relay [27.219]. 
A new family of nanotube actuators can be constructed 
by taking advantage of the ultralow interlayer friction of 
a multiwalled nanotubes. Linear bearings based on tele¬ 
scoping nanotubes and a microactuator with a nanotube 
as a rotation bearing have been demonstrated [27.203, 
220], and batch fabrication has been realized based on 
dielectrophoretically assembled arrays [27.221]. A pre¬ 
liminary experiment on a promising nanotube linear 
motor with field-emission current serving as posi¬ 
tion feedback has been shown with nanorobotic ma¬ 
nipulation (Fig. 27.23d, [27.216]). Cantilevered dual 
nanotubes have been demonstrated as nanotweez¬ 
ers [27.222] and nanoscissors (Fig. 27.23e) [27.179] by 
manual and nanorobotic assembly, respectively. Based 
on electric resistance change under different tempera¬ 
tures, nanotube thermal probes (Fig. 27.23f) have been 
demonstrated for measuring the temperature at precise 
locations. These thermal probes are more advantageous 
than nanotube-based thermometers because the latter 
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require TEM imaging. The integration of the aforemen¬ 
tioned devices can be realized using the configurations 
shown in Fig. 27.23g,h [27.183]. The arrays of individ¬ 
ual nanotubes can also be used to fabricate nanosensors, 
such as position encoders [27.223]. 

Nanorobotic systems remains a rich research field 
with a large number of open problems. New materials 
such as nanowires, nanobelts, graphene, and polymer 
at the nanoscale will enable a new family of sensors 
and actuators for the detection and actuation of ultra¬ 
small quantities or objects with ultrahigh precision and 
frequencies. Through random spreading, direct growth, 
optical tweezers, and nanorobotic manipulation, proto¬ 
types have been demonstrated. However, for integration 
into nanosystems, self-assembly processes will become 
increasingly important. Among them, we believe that 
dielectrophoretic nanoassembly will play a significant 
role for large-scale production of regular 2-D structures. 


27.9 Conclusions 

Despite the claims of many futurists, such as Issac 
Asimov ’s legendary submarine Proteus inside the hu¬ 
man body [27.225] and Robert A. Freitas’s nanomed¬ 
ical robots [27.226], the form that micro/nanorobots 
of the future will take and the tasks they will ac¬ 
tually perform remain unclear. However, it is clear 
that technology is progressing towards the construc¬ 
tion of intelligent sensors, actuators, and systems on 
small scales. These will serve as both the tools to be 
used for fabricating future micro/nanorobots as well 


A roadmap for nanorobotic systems [27.224] 
is shown in Fig. 27.24. Knowledge from meso¬ 
scopic physics, mesoscopic/supramolecular chemistry, 
and molecular biology at the nanometer scale con¬ 
verges to form the field. Various disciplines con¬ 
tribute to nanorobotics, including nanomaterial syn¬ 
thesis, nanobiotechnology, and microscopy for imag¬ 
ing and characterization. Such topics as self-assembly, 
nanorobotic assembly, and hybrid nanomanufacturing 
approaches for assembling nano building blocks into 
structures, tools, sensors, and actuators are consid¬ 
ered areas of nanorobotic study. A current focus of 
nanorobotics is on the fabrication of NEMS and other 
nanosystems, which may serve as components for fu¬ 
ture nanorobots. The main goals of nanorobotics are to 
provide effective tools for the experimental exploration 
of the nanoworld, and to push the boundaries of this ex¬ 
ploration from a robotics research perspective. 


as the components from which these robots may be 
developed. Shrinking device size to these dimensions 
presents many fascinating opportunities such as manip¬ 
ulating nano-objects with nanotools, measuring mass 
in femtogram ranges, sensing forces at piconewton 
scales, and inducing GHz motion, among other new 
possibilities waiting to be discovered. These capabil¬ 
ities will, of course, drive the tasks that future mi¬ 
cro/nanorobots constructed by and with MEMS/NEMS 
will perform. 


Video-References 
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Artificial bacterial flagella 

available from http://handbookofrobotics.org/view-chapter/ 27 /videodetails /11 
The electromagnetic control of an untethered microrobot 
available from http://handbookofrobotics.org/view-chapter/ 27 /videodetails /12 
A transversely magnetized rod-shaped microrobot 

available from http://handbookofrobotics.org/view-chapter/ 27 /videodetails /13 
Attogram mass delivery from a carbon nanotube 

available from http://handbookofrobotics.org/view-chapter/ 27 /videodetails /489 

Multi-beam bilateral teleoperation of holographic optical tweezers 

available from http://handbookofrobotics.org/view-chapter/ 27 /videodetails /490 

High-speed magnetic microrobot actuation in a microfluidic chip by a fine V-groove surface 

available from http://handbookofrobotics.org/view-chapter/ 27 /videodetails /491 

Linear-to-rotary motion converters for three-dimensional microscopy 

available from http://handbookofrobotics.org/view-chapter/ 27 /videodetails /492 
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Part C covers material related to sensing and percep¬ 
tion. The section covers all the aspects of sensing for 
basic measurement of physical parameters in the world 
to making sense of such data for the purpose of en¬ 
abling a robot to perform its tasks. Right now robotics 
is seeing a revolution in use of sensors. Traditionally 
robots have been designed to have maximum stiffness 
and applications have been designed to be predictable 
in their operation. As robots emerge from the fences 
areas and we deploy robots for a wider range of ap¬ 
plications from collaborative robotics to autonomously 
driving cars it is essential to have perception capabil¬ 
ities that allow estimation of the state of the robot but 
also the state the surrounding environment. Due to these 
new requirements the importance of sensing and per¬ 
ception has increased significantly over the last decade 
and will without doubt to continue to grow in the future. 

The topics addressed in this section include all 
aspects from detection and processing physical con¬ 
tact with the world through force and tackle sensing 
over augmented environments for detection of posi¬ 
tion and motion in the world to image based methods 
for mapping, detection and control in structured and 
unstructured environments. In many settings a single 
sensory modality/sensor is inadequate to give robust es¬ 
timate of the state of the environment. Consequently 
a chapter on multi-sensory fusion is also included in this 
part. Part C considers primarily the sensing and percep¬ 
tion aspects of robotics with a limited view to many of 
the other aspects. Part A is providing the basic, Part B 
provides the kinematic structures that we need to design 
control methods for interaction, while Part D will cover 
the grasping and manipulation of objects. The remain¬ 
ing parts of the handbook covers application verticals 
where sensors and perception plays a key role. 

With this brief overview of Part C, we provide 
a brief synopsis of each chapter. 

Chapter 28 covers force and tactile sensing, which 
is essential for physical interaction with the world. 
Tactile sensing addresses the issue of detecting and han¬ 
dling contact with objects and structures in the world. 
Tactile sensing is essential both for safety and manipu¬ 
lation applications as the control of a robot changes due 
to contact which changes the kinematic configurations. 
Force sensing addresses estimation of force and torque 
as part of dynamic motion but also for interaction with 
physical objects such as inserting an object or turning 
a valve. The model chosen for force estimation also im¬ 
pacts the control of the robot. 

Chapter 29 covers use of odometry, inertial and 
GPS for estimation of motion and position in the world. 
Odometry is the ego-estimation of your position/motion 


based on robot mounted sensors. With the introduction 
of sensors for estimation of accelerator and rotational 
velocity it is possible improve the estimation of position 
and motion. With the introduction of inexpensive iner¬ 
tial measurement units (IMU), driven by the game and 
phone industry, the use of IMU has increased signifi¬ 
cantly. For outdoor operation it is frequently possible 
to utilize Global Navigation Systems such as the global 
positioning systems (GPS). The integration of odome¬ 
try, IMU and GPS information allow design of systems 
for precision agriculture, autonomous driving, etc. The 
chapter provides a description of the core techniques to 
make this possible. 

One of the first sensors to be used for large scale 
estimation of distance to external objects was sonars. 
Chapter 30 covers the basic methods for sound based 
ranging and localization. Sonars are widely used for 
underwater mapping and localization, but is also used 
as an inexpensive modality for localization and ground 
vehicles and landing of Unmanned Aerial Vehicles. 
Typically a single sensor has limited precision but 
through used of phased array techniques and multiple 
sensors it is possible to achieve high-fidelity ranging. 
The chapter covers both the basic physics, sensing, and 
estimation methods. 

Over the last two decades we have seen tremen¬ 
dous progress on laser based ranging as a complement 
to stereo/multi-ocular methods for distance estimation. 
Light detection and ranging (LIDAR) is today used 
as an effective modality for mapping, localization and 
tracking of external objects in the environment. In 
Chapter 31 the basic techniques for range estimation 
using light based ranging is described and fundamen¬ 
tal methods for three-dimensional (3-D) modeling of 
the environment are described. Recently there has been 
a renaissance in range based sensing due to the intro¬ 
duction of RGB-D sensors, that utilize structured light 
and cameras to generate a dense range maps at very- 
low prices. 

Chapter 32 covers the general topics of 3-D vision 
which covers estimation of distance from the paral¬ 
lax between two or more cameras and/or the motion 
of a camera over time. The parallax between two or 
more images allows estimation of the distance to exter¬ 
nal objects, which can be leveraged for localization and 
navigation, but also for grasping and interacting with 
objects in the environment. The chapter covers both the 
basic aspects of 3-D estimation and some of the com¬ 
mon applications of 3-D vision for robot control. 

Chapter 33 covers the topic of object recognition. 
Computer based object recognition has been studied 
for more than 50 years. Over the last decade we have 
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seen tremendous progress on object recognition due 
to improved cameras and availability of much better 
computers and increased memory. We are seeing both 
image based recognition techniques and methods for 
recognition based on the 3-D structure of objects. More 
recently we have seen a renewed interesting use of neu¬ 
ral nets for detection of objects due to new Bayesian 
methods and the problem of object categorization - 
recognizing categories of objects such as car, motorcy¬ 
cle, traffic sign, people, etc. has become an important 
problem. The chapter covers both view and 3-D based 
methods for recognition and discusses also categoriza¬ 
tion of objects. 

Chapter 34 covers the problem of image servoing. 
When trying to interact with an object we can use im¬ 
age data to drive the end-effector to a goal location. Two 
common configurations are eye-in-hand and hand-to- 
eye. In addition the control can be performed directly 
in image coordinates or through recovery of two-and- 
a-half-dimensional (2.5-D) or 3-D pose for an object. 


To enable all of this an impact aspect is to derived the 
relation between changes in robot motion and changes 
in the image/pose, which is derivation of the Jacobian 
for the system. This chapter discusses both eye-in-hand 
and hand-to-eye visual servoing and image/pose based 
control of the process and provides examples of use of 
visual servoing in real scenarios. 

Finally, the part is concluded by Chapter 35 which 
discusses multi-sensory data fusion. As mentioned ear¬ 
lier most applications require use of multiple sensors to 
generate robust/complete methods for control. Data fu¬ 
sion involves a number of different aspects from time 
synchronization to transformation into a common ref¬ 
erence frame to integration of data over time/space to 
generate more robust/accurate estimate of the world 
state. Recently a number of new methods for Bayesian 
data fusion have emerged. In this chapter the funda¬ 
mental of data fusion are reviewed and a number of the 
most common techniques for multi-sensory fusion are 
introduced. 
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28. Force and Tactile Sensing 



Mark R. Cutkosky, William Provancher 


This chapter provides an overview of force and 
tactile sensing, with the primary emphasis placed 
on tactile sensing. We begin by presenting some 
basic considerations in choosing a tactile sen¬ 
sor and then review a wide variety of sensor 
types, including proximity, kinematic, force, dy¬ 
namic, contact, skin deflection, thermal, and 
pressure sensors. We also review various transduc¬ 
tion methods, appropriate for each general sensor 
type. We consider the information that these var¬ 
ious types of sensors provide in terms of whether 
they are most useful for manipulation, surface 
exploration or being responsive to contacts from 
external agents. 

Concerning the interpretation of tactile infor¬ 
mation, we describe the general problems and 
present two short illustrative examples. The first 
involves intrinsic tactile sensing, i.e., estimating 
contact locations and forces from force sensors. 
The second involves contact pressure sensing, i. e., 
estimating surface normal and shear stress distri¬ 
butions from an array of sensors in an elastic skin. 
We conclude with a brief discussion of the chal¬ 
lenges that remain to be solved in packaging and 
manufacturing damage-tolerant tactile sensors. 
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28.1 Overview 

Tactile sensing has been a component of robotics for 
roughly as long as vision. However, in comparison to 
vision, for which great strides have been made in terms 
of hardware and software and which is now widely 
used in industrial and mobile robot applications, tac¬ 
tile sensing always seems to be a few years away from 
widespread utility. So before reviewing the technolo¬ 
gies and approaches available, it is worthwhile to ask 
some basic questions: 


• How important is tactile sensing? 

• What is it useful for? 

• Why does it remain comparatively undeveloped? 

In nature, tactile sensing is an essential survival tool. 
Even the simplest creatures are endowed with large 
numbers of mechanoreceptors for exploring and re¬ 
sponding to various stimuli. In humans, tactile sensing 
is indispensable for three distinct kinds of activity: ma- 
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nipulation, exploration and response. The importance 
of tactile sensing for manipulation is most evident in 
fine motor tasks. When we are chilled, tasks like but¬ 
toning a shirt can become an exercise in frustration. 
The problem is primarily a lack of sensing; our mus¬ 
cles, snug in our coat sleeves, are only slightly affected 
but our cutaneous mechanoreceptors are anesthetized 
and we become clumsy. For exploration, we continu¬ 
ally assimilate information about materials and surface 
properties (e.g., hardness, thermal conductivity, fric¬ 
tion, roughness) to help us identify objects. We may 
have difficulty distinguishing real leather from synthetic 
leather by sight, but not by touch. Finally, the impor¬ 
tance of tactile response, whether to a gentle touch or 
an impact, is seen in the damage that patients with pe¬ 
ripheral neuropathy (e.g., as a complication of diabetes) 
accidentally do to themselves. 

As Fig. 28.1 indicates, the same functional cat¬ 
egories apply to robots. However, in comparison to 
animals, with hundreds or thousands of mechanore¬ 
ceptors per square centimeter of skin, even the most 
sophisticated robots are impoverished. One reason for 
the slow development of tactile sensing technology as 
compared to vision is that there is no tactile analog 
to the charge-coupled device (CCD) or complemen¬ 
tary metal-oxide semiconductor (CMOS) optical array. 
Instead, tactile sensors elicit information through phys¬ 
ical interaction. They must be incorporated into skin 
surfaces with compliance, for conforming locally to 
surfaces, and with adequate friction for handling ob¬ 
jects securely. The sensors and skin must also be robust 
enough to survive repeated impacts and abrasions. And 
unlike the image plane in a camera, tactile sensors must 

28.2 Sensor Types 

This section outlines five main types of sensors: propri¬ 
oceptive, kinematic, force, dynamic tactile, and array 
tactile sensors, as well as sensors that provide thermal 
or material composition data. However, the emphasis 
is on sensors that provide mechanoreception, as sum¬ 
marized in Table 28.1. The most important quantities 
measured with tactile sensors are shape and force. Each 
of these may be measured as an average quantity for 
some part of the robot or as a spatially resolved, dis¬ 
tributed quantity across a contact area. In this chapter 
we follow the convention of studies of the human sense 
of touch and use the term touch sensing to refer to the 
combination of these two modes. Devices that measure 
an average or resultant quantity are sometimes referred 
to as internal or intrinsic sensors. The basis for these 
sensors is force sensing, which precedes the discussion 
of tactile array sensors. 


Manipulation : Grasp force 
control; contact locations and 
kinematics; stability assessment. 

Exploration: Surface texture, 
friction and hardness; thermal 
properties; local features. 


Response: Detection and reaction 
to contacts from external agents. 

Fig. 28.1 Uses of tactile sensing in robotics 

be distributed over the robot appendages, with partic¬ 
ularly high concentrations in areas like the fingertips. 
The wiring of tactile sensors is consequently another 
formidable challenge. 

Nonetheless, considerable progress in tactile sen¬ 
sor design and deployment has been made over the 
last couple of decades. In the following sections we 
review the main functional classes of tactile sensors 
and discuss their relative strengths and limitations. 
Looking ahead, new fabrication techniques offer the 
possibility of artificial skin materials with integrated 
sensors and local processing for interpreting sensor sig¬ 
nals and communicating over a common bus to reduce 
wiring. 

There is an extensive literature describing touch 
sensing research. Recent general reviews include [28.1- 
4] and these cite a number of useful older reviews 
including [28.5-7]. 


28.2.1 Proprioceptive 

and Proximity Sensing 

Proprioceptive sensing refers to sensors that provide 
information about the net force or motion of an ap¬ 
pendage, analogous to receptors that provide informa¬ 
tion in humans about tendon tensions or joint move¬ 
ments. Generally speaking, the primary source for 
spatial proproceptive information on a robot is pro¬ 
vided by joint angle and force-torque sensors. Since 
joint angle sensors such as potentiometers, encoders, 
and resolvers are well established technologies, they 
do not warrant discussion here. Instead, a brief re¬ 
view of proximity sensing via whiskers and antennae 
as well as noncontact proximity sensing is provided. 
Force-torque sensors are discussed in greater detail in 
Sect. 28.2.4. 




Normal pressure Piezoresistive array Array of piezoresistive junctions Suitable for mass production Low force thresholds/high 

[28.8-12J Embedded in a elastomeric skin Simple design sensitivity 

Cast or screen printed Simple signal conditioning Frail 

Signal drift and hysteresis 

Capacitive array [28.13-17] Array of capacitive junctions Good sensitivity Complex circuitry 
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Whisker and Antenna Sensors 
For many animals whiskers or antennae provide an ex¬ 
tremely accurate combination of contact sensing and 
proprioceptive information. For example, cockroaches 
can steer themselves along curved walls using only po¬ 
sition and rate information from their antennae [28.33]. 
Other insects and arthropods use numerous small hair 
sensors on the exoskeleton to localize contacts. Rats 
perform highly accurate whisking to explore the shapes 
and textures of objects in their vicinity, with sen¬ 
sor processing performed by a specialized barrel cor¬ 
tex [28.34]. 

In robotics, this potentially very useful hybrid of 
proprioceptive and tactile sensing has received compar¬ 
atively little attention, although examples date at least 
to the early 1990s [28.35,36]. In more recent work, 
Clements and Rahn [28.37] demonstrated an active 
whisker with a sweeping motion and Lee et al. [28.33] 
used a passive, flexible antenna to steer a running 
cockroach-inspired robot. Prescott et al. [28.34,38] 
have conducted extensive work on active robotic whisk¬ 
ing for object exploration and identification, inspired by 
mammalian models. 

Proximity 

While proximity sensing does not strictly fall under 
the category of tactile sensing, a number of researchers 
have employed proximity sensors for collision detec¬ 
tion between a robot arm and the environment; hence, 
we briefly review these technologies here. Three pri¬ 
mary sensor technologies which include capaciflec- 
tive, infrared (IR) optical, and ultrasonic sensors have 
been used in this application. Vranish et al. devel¬ 
oped an early capaciflective sensor for collision avoid¬ 
ance between the environment and a grounded robot 
arm [28.39]. Early examples of IR emitter/detector 
pairs in an artificial skin include [28.40, 41]. A more re¬ 
cent design using optical fibers is reported in [28.42], 
and an adaptation for a prosthetic hand is reported 
in [28.43]. Other researchers have developed robot skin 
that includes ultrasonic and IR optical sensors for colli¬ 
sion avoidance [28.44]. Wegerif and Rosinski provide 
a comparison of the performance of all 3 of these 
proximity sensing technologies [28.45]. For a further 
review of some of these sensors, see Chap. 31 on range 
sensing. 

28.2.2 Other Contact Sensors 

There are a variety of other contact-based sensors 
that are capable of discerning object properties such 
as electromagnetic characteristics, density (via ultra¬ 
sound), or chemical composition (cf. animals’ senses 
of taste and smell). While this is beyond the scope of 


the current chapter. Chap. 75, on biologically inspired 
robots, briefly discusses biologically inspired chemical 
sensors related to smell and taste. For completeness, 
thermal sensors and material composition sensors are 
also briefly discussed below. 

Thermal Sensors 

Thermal sensing is an important element of human 
tactile sensing, useful for determining the material com¬ 
position of an object as well as to measure surface 
temperatures. Since most objects in the environment are 
at about the same (room) temperature, a temperature 
sensor that contains a heat source can detect the rate at 
which heat is absorbed by an object. This provides in¬ 
formation about the heat capacity of the object and the 
thermal conductivity of the material from which it is 
made, making it easy, for example, to distinguish met¬ 
als from plastics. 

Buttazzo et al. [28.46] note that the piezoelectric 
polymer used in their tactile sensing system is also 
strongly pyroelectric, and use a superficial layer as 
a thermal sensor. Other sensors use thermistors as trans¬ 
ducers [28.47—49], Some systems purposely provide 
an internal temperature reference and use the tem¬ 
perature differential from the environment to detect 
contacts [28.50,51]; however, objects with a tempera¬ 
ture the same as the reference will not be detected. Most 
of these sensors have a relatively thick outer skin cover¬ 
ing the heat sensitive elements, thus protecting delicate 
components and providing a conformal surface at the 
expense of slower response time. 

As a more recent example of thermal sensing, En¬ 
gel et al. [28.52] present a flexible tactile sensor design 
that includes integrated gold film heaters and RTDs (re¬ 
sistance temperature devices) on a polymer microma- 
chined substrate. Lin et al. [28.24] include a thermistor 
as part of the sensing suite in their artificial fingertip. 
While there is a high level of integration in these sen¬ 
sors, tradeoffs concerning construction, performance, 
and protection of sensing elements remain an ongoing 
challenge. 

Material Composition Sensors 
There has been some work on sensors for material com¬ 
position. In analogy with the human senses of taste 
and smell, liquid and vapor phase chemical sensors 
could potentially determine the chemical composition 
of a surface [28.53,54], However, the large majority 
of robotic chemical sensing has involved non-contact 
sensing of airborne plumes [28.55,56]. Another sens¬ 
ing modality which provides information about material 
properties is electromagnetic field sensing, using de¬ 
vices such as eddy current or Hall effect probes to 
measure ferromagnetism or conductivity [28.57,58]. 
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28.2.3 Kinematic Sensors 

Although they are not generally regarded as tactile sen¬ 
sors, sensors that detect the position of a limb can 
provide the robot with geometric information for ma¬ 
nipulation and exploration, particularly when the limb 
also includes sensors that register contact events. Ex¬ 
amples of combining joint angle sensing and contact 
sensing with compliant fingers to learn about the con¬ 
figuration of a grasp include [28.59, 60]. 

28.2.4 Force and Load Sensing 

Actuator Effort Sensors 

For some actuators such as electric servo motors, a mea¬ 
sure of the actuator effort can be obtained directly by 
measuring the motor current (typically using a sens¬ 
ing resistor in series with the motor). However, because 
motors are typically connected to robot limbs via gear¬ 
boxes with output/input efficiencies of 60% or less, it is 
usually much more accurate to measure the torque at the 
output of the gearbox. Solutions to this problem include 
torque load cells and mechanical structures at the robot 
joints whose deflections can be measured using electro¬ 
magnetic or optical sensors. For cable or tendon-driven 
arms and hands it is useful to measure the cable ten¬ 
sion - both for purposes of compensating for friction in 
the drive-train and as a way of measuring the loads upon 
the appendage [28.61, 62]. When fingers or arms make 
contact with objects in the environment, cable tension 
sensing becomes an alternative to endpoint load sens¬ 
ing for measuring components of the contact forces. Of 
course, only those components that produce significant 
torques can be measured with accuracy. Chapter 19 on 
multifingered hands contains more details concerning 
tendon tension measurement. 

Force Sensors 

When actuator effort sensors are not sufficient to mea¬ 
sure the forces exerted by or on a robot appendage, dis¬ 
crete force sensors are typically utilized. These sensors 
are found most often at the base joint or wrist of a robot 
but could be distributed throughout the links of a robot. 

In principle, any type of multi-axis load cell could 
be used for manipulator force-torque sensing. How¬ 
ever, the need for small, lightweight units with good 
static response eliminates many commercial sensors. 
The design of force sensors for mounting above the 
gripper at the wrist has received the most atten¬ 
tion [28.63, 64], but fingertip sensors for dextrous hands 
have also been devised. Often these sensors are based 
on strain gauges mounted on a metal flexure [28.65- 
67] which can be fairly stiff and robust. Sinden and 
Boie [28.68] propose a planar six axis force-torque 


sensor based on capacitive measurements with an elas¬ 
tomer dielectric. Design considerations for force sen¬ 
sors include stiffness, hysteresis, calibration, amplifi¬ 
cation, robustness, and mounting. Dario et al. present 
an integrated fingertip for robotic hands integrated 
FSR (force sensing resistor) pressure array, piezoce¬ 
ramic bimorph dynamic sensor, and force-torque sen¬ 
sor [28.29]. More recently Eclin et al. [28.69] have de¬ 
veloped a miniature a multi-axis fingertip force sensor 
(Fig. 28.2). For applications where immunity to electro¬ 
magnetic noise is desirable. Park et al. [28.70] present 
a robot fingertip with embedded fiber optic Bragg grat¬ 
ings, used as optical strain gages. Bicchi [28.71] and 
Uchiyama et al. [28.72] consider the optimal design of 
multi-axis force sensors in general. 

Information from the force sensors can also be com¬ 
bined with knowledge of fingertip geometry to estimate 
contact location, as implied in Fig. 28.3. This method of 
contact sensing is referred to as intrinsic tactile sensing 
and was first presented by Bicchi et al. [28.73]. A com¬ 
parison between intrinsic and extrinsic contact sensing 



Fig. 28.2 Miniature fingertip force-torque sensor for 
a prosthetic hand (after [28.69]) 



Fig. 28.3 Robot hand with fingertip force and tactile sens¬ 
ing. Information from the force sensors can be combined 
with knowledge of fingertip geometry to estimate contact 
location, referred to as intrinsic tactile sensing 
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(i. e., using distributed contact sensors) is presented by 
Son et al. [28.74], This topic is discussed further in 
Sect. 28.3.1. 

28.2.5 Dynamic Tactile Sensors 

Taking a cue from the role of fast-acting or dynamic tac¬ 
tile sensors in human manipulation [28.75], researchers 
have developed dynamic sensors for slip detection and 
for sensing textures and fine features. 

Early slip sensors based on displacement detected 
the motion of a moving element such as a roller or 
needle in the gripper surface [28.76,77]. Subsequent 
work has typically used accelerometers or other sensors 
that are inherently sensitive to small, transient forces or 
motions. Early examples include [28.31,78-80]. Many 
subsequent contributions are reviewed in [28.81]. 

For the case of hard objects held in metal grippers, 
acoustic emissions may reveal slip [28.82]. Because 
these signals are at very high frequency (over 100 kHz) 
they can be useful for distinguishing among different 
kinds of contact events in noisy environments [28.83]. 
Another approach for improving the signal/noise ro¬ 
bustness of dynamic sensing is to use an actively stim¬ 
ulated sensor and measure the change in response as 
contact conditions change [28.84, 85]. 

For grasp force control, it is especially useful to de¬ 
tect incipient slips, which are accompanied by small, 
localized vibrations, or micro-slips, at the periphery of 
a finger/object contact region before gross sliding oc¬ 
curs. The challenge is to distinguish these from other 
events that produce vibrations [28.86-89]. 

For detecting features and fine surface features 
it can be effective to use small fibers, a skin with 
fingerprint-like ridges, or a stylus that drags over the 
surface like a fingernail [28.46, 90, 91]. 

Because many dynamic tactile sensors only pro¬ 
duce a transient response, they are often combined 
with pressure sensing arrays or force sensors to provide 
a combination of low frequency and high frequency 
tactile sensing [28.24,29,32,46,89,92,93]. An alter¬ 
native, which can be simpler from the standpoint of 
sensor integration, is to use conventional pressure sens¬ 
ing arrays for slip detection. In this case, the array 
resolution and scanning rate must be sufficient to de¬ 
tect motion or incipient motion quickly, to prevent 
grasp failures. Fortunately, this is becoming increas¬ 
ingly feasible [28.19,81,94,95]. loJBSESilEB shows 
an example of dynamic tactile sensing. 

28.2.6 Array Sensors 

Hundreds of designs for tactile array sensors have ap¬ 
peared in the literature in the last 25 years, and many 


of them are designed for use with dextrous hands. In 
terms of transducers, the fundamental requirement is 
to unambiguously recover either the shape or pressure 
distribution across the contact. Shape sensing requires 
a compliant skin, which can also have advantages for 
grasp stability (see Chaps. 37 and 38 on contact mod¬ 
eling and grasping). Examples of shape-sensing tactile 
arrays include [28.23, 24]. However, the far more com¬ 
mon approach is to measure subsurface strains, which 
can be correlated with surface pressure distributions, as 
presented by Fearing and Hollerbach [28.96] and dis¬ 
cussed further in Sect. 28.3.2. 

Contact Location Sensors 

The simplest and perhaps most robust tactile arrays pro¬ 
vide measurements solely of contact location. Some 
such sensors utilize a membrane switch design like 
that found in keyboards [28.97], As another example, 
a robust two-dimensional (2-D) switch array can be 
embedded in a prosthetic hand [28.69]. Some optical 
tactile sensors have also been used primarily as contact 
location sensors. Maekawa et al. [28.98] used a single 
optical position sensing device (PSD) or a CCD cam¬ 
era array to detect the position of scattered light off of 
a hemispherical optical wave guide fingertip with a sil¬ 
icone rubber cover. Light is scattered at the locations 
of contacts. With a textured skin, the magnitude of the 
force can also be estimated, as the contact area grows 
in proportion to the pressure. However, an issue with 
fingertips that use a compliant skin covering a hard sub¬ 
strate is that adhesion between the two materials results 
in hysteresis. In addition, when the fingertip is dragged 
over a surface, the friction can produce a shift in the 
estimated contact position. 

Pressure Sensing Arrays 

Capacitive Pressure Sensing Arrays. Capacitive 
pressure sensing arrays are one of the oldest and most 
common tactile sensor types. Some of the earliest anal¬ 
ysis of such sensors is presented by Fearing [28.99] 
for capacitive arrays embedded into a soft robot finger¬ 
tip. The arrays consist of overlapping row and column 
electrodes separated by an elastomeric dielectric, form¬ 
ing an array of capacitors. A change in capacitance 
results from compressing the dielectric between row- 
column plates at a particular intersection. The equation 
for capacitance, based on the physical parameters, is 
expressed as C eA/d, where e is the permittivity 
of the dielectric between the plates of the capacitor, 
A is the area of the plates, and d is the spacing be¬ 
tween them. Pressing on the skin reduces the plate 
spacing d, thus providing a linear response with dis¬ 
placement. Through appropriate switching circuitry, 
a region of a sensor array can be isolated at a particular 
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row/column intersection. Examples of similar capaci¬ 
tive tactile arrays can be found in [28.100,101] and 
commercially [28. 14]. Large capacitive arrays have also 
been developed to cover the arms of a robot [28. 15-17]. 
A woven fabric with capacitive sensing junctions is re¬ 
ported in [28.102], 

With a suitable dielectric and plate design, capaci¬ 
tive arrays can be robust, have a large dynamic range 
(ratio between minimum and maximum detectable 
pressure) and low hysteresis, which is desirable for fast 
response. However, a common problem is the need to 
shield the array from stray capacitance effects (e.g., 
from an approaching metallic surface) and to mini¬ 
mize parasitic capacitance associated with wiring to 
and from the active elements. For these reasons, the re¬ 
cent trend is to use local microprocessors specialized 
for capacitance measurement as part of an integrated 
sensing system (Sect. 28.4). An example is shown in 
Fig. 28.4. actually shows results taken 

with the sensor depicted in Fig. 28.4. 

Piezoresistive Pressure Sensing Arrays. Many re¬ 
searchers have produced tactile sensor arrays that are 
piezoresistive. In most cases, these sensors utilize 
a conductive rubber that is bulk molded or a piezoresis¬ 
tive ink that is patterned via screen printing or stamping. 
Each of these approaches employs a conductive ad¬ 
ditive (typically carbon black or silver) to create its 
conductive/piezoresistive behavior. As a more flexible 
and durable alternative, some researchers have devel¬ 
oped fabric-based piezoresistive sensors, discussed sep¬ 
arately below. 

Russell [28.27] presented one of the first molded 
conductive rubber tactile sensor arrays composed of 



Fig. 28.4 Capacitive touch and joint angle sensors on 
a flexible circuit for incorporation in a robotic hand, shar¬ 
ing the same microprocessors for signal processing and 
communication (after [28.103]) 


conductive rubber column and row electrodes with 
piezoresistive junctions. However, this sensor exhib¬ 
ited significant drift and hysteresis, which fueled re¬ 
search to minimize these effects through proper se¬ 
lection of molding material [28.8]. These issues were 
never completely solved due to the hysteretic nature 
of elastomers, but this sensing approach remains at¬ 
tractive due to its ease of manufacturing. Hence, it has 
continued to find applications, for example, in the ap¬ 
pendages of humanoid robots where extreme accuracy 
is not required [28.104]. Significant improvements in 
dynamic range and robustness are also possible using 
a treated elastomer that exploits a quantum tunneling 
effect [28.105], and commercial versions are now avail¬ 
able [28.106]. 

A number of researchers and companies have devel¬ 
oped tactile sensors that utilize conductive (piezoresis¬ 
tive) ink, generally referred to as FSRs (force sensitive 
resistors). This is by far the most common and economi¬ 
cal means to incorporate tactile sensing via off-the-shelf 
sensors. However, to make highly integrated, dense sen¬ 
sor arrays, custom fabrication is necessary. Examples 
of such sensors include [28.9,29]. To take this basic 
concept a step further, Someya [28.107] has produced 
robotic skin that employs patterned organic semicon¬ 
ductors for local amplification of the piezoresistive 
sensor array, printed on flexible polyimide film. Al¬ 
though fabricated on a flexible substrate, these sensor 
arrays are somewhat susceptible to bending fatigue. 

Piezoresistive fabrics have been developed to ad¬ 
dress fatigue and fragility issues found in tactile arrays. 
Examples of these sensors are presented in [28.10-12, 
108] and are utilized in applications such as the arms or 
legs of humanoid robots. Because this technology has 
the potential to replace ordinary cloth, it is a promising 
technology for applications in wearable computing or 
even smart clothing. A different solution to the problem 
of making a sensor that will work on joints and loca¬ 
tions with extensive flexing is to employ fine channels 
of liquid metal in a stretchy elastomer [28.109]. 

One final design that does not fall under the 
above fabrication categories is a sensor designed by 
Kageyama et al. [28.110] that utilizes a piezoresistive 
conductive gel pressure array along with a multi-level 
contact switch array via variable contact resistance 
within sensor layers that they developed for use in hu¬ 
manoid robots. 

MEMS arrays. MEMS technology is attractive for pro¬ 
ducing dense sensor arrays with integrated packag¬ 
ing, signal processing and communications electronics. 
Early devices were produced in silicon through stan¬ 
dard silicon micro-machining techniques, allowing high 
spatial resolution and dedicated hardware for multi- 
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plexing, etc. [28.111, 112]. However, such silicon-only 
devices can be brittle and difficult to integrate into 
a robust, compliant skin. More recent efforts have 
addressed these problems by using combinations of 
semiconductors and other materials, including organic 
semiconductors [28.18, 19,52,113,114]. 

Returning to the idea of using silicon load cells 
for tactile sensing, Valdastri et al. [28.115] developed 
a miniature MEMS silicon-based three-axis load cell 
that resembles a joystick and is appropriate for embed¬ 
ding within an elastomeric skin for detecting shear and 
normal stresses (Fig. 28.5) [28.93]. Other recent efforts 
include flexible arrays with high spatial resolution using 
capacitive or piezoresistive technology [28.20, 116]. 

Skin Deflection Sensing 

Brocket [28. 117] was one of the first to propose the idea 
of using deformable membranes as robot fingertips. As 
noted by Shimoga and Goldenberg [28.118], there are 
several advantages to using deformable fingertips over 
more rigid robot fingertips, which include: 1) improved 
grasp stability, 2) reduced shock, and 3) reduced fa¬ 
tigue for embedded sensor elements. Other early work 
includes a deformable fingertip with skin covering 
polyurethane foam and instrumented with elastomeric 
strain gages [28.27]. 

Nowlin [28.25] used Bayesian algorithms to im¬ 
prove data interpretation of a deformable tactile sen¬ 
sor that used magnetic field sensing. A 4 x 4 array 
of magnets were supported above paired hall-effect 



Fig. 28.5 (a) Scanning electron microscope (SEM) micro¬ 
graphs of a MEMS 3-axis tactile force sensor, (b) MEMS 
force sensor wirebonded to a Hex-circuit and embedded 
within a silicone rubber skin (after [28.115]) 


sensors off a rigid base by individual fluid filled 
balloons. Hall-effect sensors measure the strength of 
the local magnetic field, which should increase with 
proximity to the magnets; however, this is compli¬ 
cated by neighboring magnets in the array. Hence, 
noisy data were combined using Bayesian algorithms 
to predict membrane deformation for a deformable 
fingertip. 

Later, Russell and Parkinson [28.26] developed an 
impedance tomographic tactile sensor, capable of mea¬ 
sure skin deformation over an 8 x 5 array. This sensor 
was constructed with of neoprene rubber and filled with 
distilled water, similar to the more recent BioTac sen¬ 
sor [28.24]. Row and column electrodes were made 
from copper and conductive rubber for the rigid sub¬ 
strate and neoprene skin, respectively. Like the capaci¬ 
tive tactile sensors described above, this sensor utilized 
multiplexing electronics to reduce the number of elec¬ 
trical interconnects. Square waveform driver electronics 
are used to estimate the resistance of a column of wa¬ 
ter formed between row and column elements providing 
a signal that is proportional to the current skin height. 

Ferrier and Brocket [28.23] implemented a tactile 
sensor which used optical tracking in combination with 
models of the skin to predict sensor fingertip defor¬ 
mations. The fingertip sensor consists of a tiny CCD 
camera focused on a 7 x 7 array of dots marked on the 
inside of a gel-filled silicone membrane. An algorithm 
is used to construct a 13 x 13 grid over the array of dots. 
This algorithm uses a combination of the position that is 
sensed by the CCD camera, which provides the location 
along a line radially outward from the focal point, in 
combination with a mechanical model used to solve for 
the radial distance from the camera focal point, based 
on energy minimization. 

A more recent example of a shape sensing array is 
the commercial Syntouch BioTac that grew out of re¬ 
search on a multimodal fingertip tactile sensor [28.24]. 
In this design, measuring the changes in resistance 
between terminals in a water-filled skin provides an es¬ 
timate of the skin deformation. 

Other Array Tactile Array Sensors 
While less common than capacitive or piezoresistive 
sensors, optical tactile sensors have consistently been 
attractive for their immunity to electromagnetic inter¬ 
ference. The basic approaches include designs that use 
a small camera to measure skin deformations and ar¬ 
rays of optical emitters and detectors. Optical fibers can 
also be used to separate the transducer from the contact 
site [28.42,119]. 

An interesting tactile sensor uses vision to track an 
array of spherical markers embedded in a transparent 
elastomer to infer the stress state of the skin material 
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due to applied forces [28.21]. This sensor is currently 
being commercialized under the tradename GelForce. 

Miniaturization of optical components has also 
made it possible to construct a tactile array of surface- 
mounted optical devices. In a physically robust design 
that can be adapted for shear and normal sensing, emit¬ 
ters and detector pairs are covered by a thin translucent 
layer of silicone rubber followed by an opaque outer 
layer. As the silicone skin is depressed, there is a varia¬ 
tion in the amount of light reflected from the emitter to 
the detector [28.22, 120], 

The concept of reflection off the underside of a com¬ 
pliant skin has also been used with acoustic or ultra¬ 
sonic sensors. Shinoda et al. [28.121] present a sensor 
that looks at the change in reflected accoustic en¬ 
ergy from a resonator chamber near the surface of the 


28.3 Tactile Information Processing 

In discussing the processing of tactile information, we 
return first to the three main uses depicted in Fig. 28.1. 
For manipulation, we require, foremost, information 
about contact locations and forces so that we can grasp 
objects securely and impart desired forces and motions 
to them. For exploration, we are concerned with ob¬ 
taining and integrating information about the object, 
including the local geometry, hardness, friction, texture, 
thermal conductivity, etc. For response, we are con¬ 
cerned especially with the detection of events, such as 
contacts produced by an external agent, and in assessing 
their types and magnitudes. The uses of information are 
often coupled. For example, we manipulate objects in 
order to explore them, and we use the information ob¬ 
tained through object exploration to improve our ability 
to control forces and motions in manipulation. Recog¬ 
nizing contact events is also important for manipulation 
and exploration, as it is for response. 

28.3.1 Tactile Information Flow: 

Means and Ends of Tactile Sensing 

Figure 28.6 summarizes the general flow of information 
from each type of sensor reviewed in the previous sec¬ 
tion through primary sensed quantities to information 
provided for manipulation, exploration and response. 
A useful thought exercise is to consider exactly what in¬ 
formation we use to perform a task such as turning a pen 
end-over-end between the fingers. We can easily per¬ 
form this task with our eyes closed. What information 
are we using? We need to track the position and orienta¬ 
tion of the pen and to monitor the forces that we impose 
on it to maintain stable manipulation. In other words. 


skin, with application to friction measurement [28.122]. 
Ando et al. [28.123] present a more sophisticated ultra¬ 
sound sensor that achieves 6-DOF (degree of freedom) 
displacement sensing via paired plate elements that uti¬ 
lize 4 ultrasound transducers per plate. 

Multimodal Arrays 

Human sensing is inherently multi-modal, with com¬ 
binations of fast- and slow-acting mechanoreceptors 
as well as specialized thermal and pain receptors. In 
an analogous approach, a number of researchers have 
developed multi-modal tactile sensing suites that com¬ 
bine mechanical and thermal sensing [28.24,49,52, 
124, 125]. As noted earlier, other work has combined 
static and dynamic sensing [28.24,29,32,46,92,93, 
126]. 


we need to know the configuration of our grasp, the lo¬ 
cations and movements of contacts over the surfaces of 
our fingers, the magnitudes of grasp forces and the con¬ 
tact conditions with respect to friction limits, etc. The 
same requirements apply for robots and are provided by 
the information flow in Fig. 28.6. 

At the upper left comer of the figure, joint angles, 
combined with the forward kinematic model of the ma¬ 
nipulator and knowledge of external link geometries, 
establish the positions and orientations of coordinate 
frames embedded in the fingertips. This information 
is needed to integrate local information about object 
shape, surface normal orientation, etc. so that the over¬ 
all geometry and pose of the object can be determined. 

Actuator effort sensors provide information about 
the resultant forces, using the Jacobian transpose: 
J T / = r, where / is an n x 1 vector of external forces 
and moments taken with respect to a coordinate frame 
embedded in the appendage. J T is the Jacobian trans¬ 
pose, mapping external forces and moments to joint 
torques, and r is an m x 1 vector of joint torques for 
a serial kinematic chain with m degrees of freedom. We 
require that the k-th column of J T have elements that are 
relatively large compared to the overall condition num¬ 
ber of J to provide an accurate measurement of the k -th 
element of /. Eberman and Salisbury [28.127] show 
that it is possible to measure contact force and location 
using only joint torque measurements if the manipula¬ 
tor has clean dynamics. 

Alternatively, we can use a multi-axis force-torque 
sensor in the fingers, as shown in Fig. 28.3, or robot 
wrist to obtain contact forces. This approach has the 
advantage of providing dynamic force signals with 
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Fig. 28.6 Force and tactile sensor information flow and signal processing 


a higher signal/noise ratio because they are not masked 
by the inertias of the robot arm or fingers and their 
transmissions. If the geometry of the fingertip is known, 
one can use intrinsic tactile sensing [28.65,128] to 
compute the contact location as well as the contact force 
by examining ratios of resultant forces and torques at 
the sensor. 

When the contacts are small compared to the finger¬ 
tips (so that a point-contact approximation applies) and 
the fingertips are convex shapes, the contact location 
is easily computed. Fig. 28.7 shows a contact force /, 
contacting the fingertip surface at a location r. A force- 
torque sensor such as that in Fig. 28.2 measures the 
moment, r = r x/, with respect to the origin. If we con¬ 
sider the lever arm h, perpendicular to the line of action 
off, then h/h =f/fy. r/r, where h = x // is the mag¬ 
nitude of li. We can then write that r = h — af, where a 
is a constant obtained by solving for the intersection of 
the line of action and the fingertip surface. For a convex 
fingertip, there will be two such points, of which only 
one corresponds to a positive (inward) contact force. 

From the contact location one can deduce the lo¬ 
cal contact normal and contact kinematic type from 
a small number of force measurements. Bicchi presents 
algorithms for extending these methods to soft fin¬ 


gers [28.128]. Brock and Chiu [28.66] describe the use 
of force sensors for the perception of object shape using 
this approach and for measuring the mass and center of 
mass of a grasped object. 

For precision tasks involving small objects or small 
forces and motions, cutaneous sensors provide the most 
sensitive measurements. In general, as task require - 



Fig. 28.7 Intrinsic tactile sensing: a contact produces 
a unique line of action and moment about the origin of 
a coordinate system in the fingertip. The contact location 
can be obtained by solving for the intersection of the line 
of action and the fingertip surface 
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ments get smaller, the sensor must be located closer 
to the contact so that the compliance and inertia of 
the intervening parts of the manipulator do not inter¬ 
fere with the measurement. Dario [28.129] suggests 
that finger tip force sensors are useful for forces of 
0.1—10.0 N while array sensors can measure distributed 
forces of 0.01—1.0N. Son et al. [28.74] find that intrin¬ 
sic tactile sensing and array sensors can both provide 
accurate (within 1 mm) estimates of contact location; 
however, the intrinsic tactile sensing method is in¬ 
herently sensitive to the accuracy of the force-torque 
sensor calibration and can produce transient errors due 
to unmodeled dynamic forces. 

Proceeding down the left side of Fig. 28.6 we come 
to the large category of cutaneous array sensors. The in¬ 
terpretation of information from array sensors depends 
initially on the transducer type. For arrays of binary 
contact or proximity sensors, interpretation amounts 
mainly to establishing the location and general shape of 
the contact area. Techniques common to binary vision 
can be used to obtain sub-pixel resolution and to iden¬ 
tify contact features. This information, in combination 
with measurements of the grasp forces from actuator 
effort or force-torque sensors, is sufficient for basic ma¬ 
nipulation tasks [28.69]. 

28.3.2 Solid Mechanics and Deconvolution 

A basic problem associated with tactile array sensors 
is to reconstruct what is happening at the surface of the 
skin from a finite set of measurements obtained beneath 
the surface. Typically we are interested in determining 
pressure and perhaps shear stress distributions associ¬ 
ated with contacts on the skin. In other cases, as when 
the fingertips consist of a gel or soft foam covered by 
a thin membrane so that the pressure is nearly constant, 
the local geometry of the contact is of interest. 

In the following example we consider the case of 
an array of elements located at a depth d below the 
surface of an elastomeric skin. A contact has resulted 
in a pressure distribution over the region of interest. 
We establish a coordinate system with z pointing in 
the inward normal direction and, for simplicity, we ex¬ 
amine a one-dimensional loading case, p(y), in which 
the pressure distribution is unchanging in the x direc¬ 
tion. We further assume that the extent of the skin in 
the x direction is large compared to the skin thickness 
so that strains in the x direction are inhibited, leading 
to a plane strain elasticity problem. We assume that 
the skin is a homogenous, isotropic material and the 
strains are small enough that linear elasticity theory 
can be applied. Of course, none of these assumptions 
is entirely valid in practical cases; however, the results 
do agree qualitatively with the measurements obtained 


with actual robot fingers and tactile arrays. A thorough 
discussion of the general approach and of the accuracy 
of the linear elastic models can be found in [28.91, 96, 
99,130,131]. 

Figure 28.8 illustrates the case of two line loads, or 
knife edges, pressed against the surface of the skin (akin 
to a planar version of the two-point discrimination test 
for human tactile acuity). The solution for a single line 
load, or impulse response, was derived by Boussinesq in 
1885. For the case of plane strain the principal stresses 
in the (y, z) plane from a normal unit impulse can be 
expressed in cartesian coordinates as [28.132] 


^(y.z)-f 2 ) 1 

(28.1) 

\* Z J [1 + (v/t:) 2 ]- 


=n w => , 

Wz/[i + ev/z) 2 ] 2 

(28.2) 

a x (y, z) = v(oy + a z ), 

(28.3) 


where v is Poisson’s ratio for the material (typically 0.5 
for elastomeric materials). 
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Fig. 28.8 Plane strain stress response for two (unit magni¬ 
tude) line loads. Note the blurring that occurs with greater 
depth 
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For two such line loads located at distances 5] 
and 82 from the origin, the solution can be obtained by 
superposition 



For more general pressure distributions the stresses 
can be found by convolution of the pressure distribu¬ 
tion, p(y), and the impulse response G, (v, z ) 


07 = 


z=y 

J [p(r)dr]Gi(y 

r=—00 


T,Z) . 


(28.6) 


Also plotted in Fig. 28.8 are curves corresponding 
to the vertical stress components, < 7 -, at two different 
depths d\ = 2A and c /2 = 3A, where A is the sensor 
spacing. As we go deeper beneath the skin, the stresses 
become smoothed or blurred, and the ability to dis¬ 
tinguish between closely spaced impulses diminishes. 
Flowever, the blurring of concentrated pressure distri¬ 
butions can also provide an advantage when we have 
a limited number of sensors because the stresses and 
strains spread over a larger area and are more likely to 
affect at least one sensor. The elastic skin also provides 
a kind of automatic edge enhancement because stresses 
are high at the transitions between loaded and unloaded 
regions of the skin. 

In most cases, for example in the case of capac¬ 
itive or magnetic sensors, the sensing elements will 
measure strains or local deformations of the skin ma¬ 
terial in the vertical direction. In a few cases, such as 
pieces of piezoelectric film embedded in an elastomeric 
skin [28.91], the sensors are sufficiently stiff compared 
to the surrounding material that they can be considered 
to measure stresses directly. 


For the case of elastic plane strain, the strains are 
related to the stresses by [28.133] 

e? = ^ [o y - v (<j A + O'-)] , (28.7) 

= - \a z - v(a x + ff,)] , (28.8) 

Jb 

where E is the Young’s modulus and v is the Poisson’s 
ratio, which we assume is 0.5 for an elastomeric skin. 

Figure 28.9 shows the typical measurements that 
may be obtained from a row of sensing elements from 
the two line loads applied in Fig. 28.8. Each bar cor¬ 
responds to the strain, e z j measured by a corresponding 
element and computed using (28.8), with stresses ob¬ 
tained from (28.3)-(28.5). 

The problem at this point is to produce a best es¬ 
timate of the surface pressure distribution, p(y), from 
this finite set of subsurface strain measurements. The 
problem is a classic example of estimating a signal 
from a sparse set of remote measurements. One ap¬ 
proach to this process is based on deconvolution tech¬ 
niques [28.91,96,99]. The measured signal from the 
sensors e z is convolved with the inverse of the im¬ 
pulse strain response H(y) to find an estimate of the 
surface pressure that produced the signals. The inver¬ 
sion tends to amplify high frequency noise, and the 
inverse filter bandwidth must be limited according to 
the spatial density of the sensors and their depth below 
surface. 

Another approach [28.21, 130] is to assume that the 
surface pressure distribution can be approximated by 
a finite set of impulses p = \pi,pi.. -PnY■ The sen¬ 
sor readings form a vector, e = [ei, 62 ... where 
m> n for the bandwidth limitations discussed above. 
The strain response can then be written as a matrix 


Relative strain e z (y,z), z = 2 A 
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Fig. 28.9 Measured strain with assumed 10% noise 
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equation 

e=Hp. (28.9) 

Each element of H is computed using (28.8) with < 7 - 
and o y computed using equations similar to (28.4) and 
(28.5) and with a x from (28.3). The estimated discrete 
pressure distribution is then found by taking the pseu¬ 
doinverse of H 

p = H + f. (28.10) 

Using the strain measurements from Fig. 28.9, at 
a depth cl = 2A, the estimated pressure distribution us¬ 
ing the pseudoinverse method is seen in Fig. 28.10. 
In this example, because the assumed set of seven im¬ 
pulses matches fortuitously with the actual loading, the 
reconstruction is fairly accurate despite the assumed 
noise. 

An alternative approach to constructing soft robot 
finger tips is to enclose a compliant medium such as 
foam rubber or fluid in a thin elastic membrane [28.25, 
27,117, 134-137]. Some of the tactile array sensors de¬ 
veloped for these fingers are able to measure directly 
the shape of the membrane, so that a physical model 
is not needed for interpretation of the signal [28.25]. 
Another sensing scheme uses an array of magnetic sen¬ 
sors at the center of the finger to measure the changes 
in the magnetic field generated by deformations of the 
magnet-loaded membrane [28.137]. A statistical algo¬ 
rithm has been developed that can robustly determine 
the membrane shape from the sensor signals [28.25]. 
However, a mechanical model is still required to find the 
pressure distribution across the contact from the shape 
information provided by all of these sensors. 

Estimated p (y, r), - = 2 X 



Fig. 28.10 Estimated surface pressure distribution using 
pseudoinverse method for 11 sensors and 7 assumed im¬ 
pulses 


28.3.3 Curvature and Shape 
Information 

Another alternative to measuring subsurface strains or 
deflections is to measure directly the local curvature 
at each element of an array of sensors. The curvature 
information can be applied directly toward identifying 
contact type and centroid location or integrated to ob¬ 
tain the local shape of the contact [28.28, 138]. 

Returning to Fig. 28.6, once the local contact 
shape or geometry has been established, the next steps 
typically include feature identification (e.g., identify¬ 
ing corners or ridges on an object) and determin¬ 
ing the overall shape and pose of the object in the 
hand. 

Often the object shape is at least partially known 
a priori in which case a variety of surface or data fitting 
methods can be used. For example. Fearing [28.139] 
developed a method for calculation of the radius of 
curvature and orientation of a generalized cylinders 
from tactile array data and [28.140] developed a neu¬ 
ral network that performs a similar calculation. Other 
schemes use contact locations, surface normals and 
contact forces to determine information about object 
shape and orientation with respect to the hand [28.141- 
144], 

Allen [28.145] uses several different primitive rep¬ 
resentations for object shape attributes based on the 
particular exploratory procedure used to sense the ob¬ 
ject. Object volume and approximate shape are per¬ 
ceived with enclosure grasping, and the resulting shape 
is modeled using superquadric surfaces. Similarly, mea¬ 
surement of the lateral extent of object faces leads 
to a face-edge-vertex model and contour following to 
a generalized cylinder representation. 

The question of what constitutes an appropri¬ 
ate set of features is not well understood, although 
it clearly depends on the intended application. El¬ 
lis [28.146] considers appropriate feature sets and 
methodologies for acquiring the needed data. Lederman 
and Browse [28.147] suggest that surface roughness, 
surface curvature, and oriented edges are used in human 
haptic perception. 

28.3.4 Object and Surface 
Identification 

The most common application of touch information has 
been in object recognition and classification. In object 
recognition the goal is to identify one object from a set 
of known objects using information derived from touch. 
In classification the goal is to categorize objects ac¬ 
cording to preselected sensed properties. These systems 
are usually based on geometric information derived 
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from tactile array or force sensors, although some ap¬ 
proaches use additional information (e.g., compliance, 
texture, thermal properties and the kinematic pose of 
the hand) [28.148-154]. 

28.3.5 Active Sensing Strategies 

Because touch provides only local information, move¬ 
ment is an integral part of touch sensing for recognition 
and exploration. A review of tactile sensing focusing on 
exploration and manipulation is found in [28.2]. Several 
researchers have developed strategies for scheduling 
sensor movements so that each additional observation 
decreases the number of objects which are consistent 
with prior observations. This is sometimes described 
as a hypothesize and test approach. Early examples in¬ 
clude [28.141,155-157], 

Klatzky et al. [28.149] have suggested that robotics 
systems can employ the same exploratory procedures 
used by humans in haptic exploration. These proce¬ 
dures prescribe the finger motions needed for tasks such 
as tracing object contours, measuring compliance, and 
determining the lateral extent of object surfaces. Early 
work on exploratory tactile sensing includes [28.145, 
158, 159]. More recent examples include [28.32, 160, 
161]. 

Edge tracking and surface following strategies 
have received particular attention. Early examples in¬ 
clude [28.162-166]; a more recent example is found 
in [28.167], 


28.4 Integration Challenges 

A critical problem that we have not yet addressed is 
the difficulty of connecting to a large and diverse ar¬ 
ray of tactile sensors. In 1987 Jacobsen et al. [28.172] 
cited the routing of wires as perhaps the most difficult 
problem in dexterous hand design and, to a large extent, 
this remains true today. However, some solutions to this 
problem have been presented in recent years using wire¬ 
less sensor arrays or digital busses for power and signal 
connections. 

In early work, Shinoda and Oasa [28.173] embed¬ 
ded tiny wireless sensing elements in an elastic skin 
that uses an inductive base coil to provide power and 
signal transmission. Hakozaki and Shinoda [28.174] 
embed tactile sensor chips between 2 layers of conduc¬ 
tive rubber provide power and serial communication. In 
other work, [28.18,20] produce large scale printed ar¬ 
rays with built-in multiplexing and communications. 


28.3.6 Dynamic Sensing 
and Event Detection 

For dynamic tactile sensors used to detect such events 
as gentle contacts or slippage between the fingertips 
and an object, the main challenge is to reliably de¬ 
tect the event in question without false positives. The 
dynamic tactile sensors that produce large signals in re¬ 
sponse to contact events are also prone to producing 
large signals in response to vibrations from the robot 
drive train and to rapid accelerations of the robot hand. 
Solutions for more robustly detecting contact events 
include comparing the signals from dynamic tactile sen¬ 
sors at and away from the contact regions and statistical 
pattern recognition methods to identify the signature 
of true contact events. Early examples include [28.86, 
127,168]. More recent examples have taken advantage 
of the much greater realtime processing capability that 
is now available for signal processing and event classi¬ 
fication [28.32, 161,169,170], 

28.3.7 Integration of Thermal 
and Other Sensors 

Sensors such as thermal contact sensors are rarely used 
in isolation; their signals are generally integrated with 
those from tactile arrays and other sensors to produce 
additional information for identifying objects. Exam¬ 
ples of integrated thermal and mechanical sensors for 
surface characterization include [28.24, 52, 171]. 


In a different approach, Yamada et al. [28.168] use 
wireless sensor chips and light transmitted through 
a transparent elastomer both for power and to commu¬ 
nicate to a power-receiver chip. Ascari et al. [28.175] 
propose a communications scheme with all information 
transmitted along optical fibers. 

In recent years, the proliferation of microprocessors 
that can perform local signal conditioning, multiplex¬ 
ing and digital communications has been particularly 
helpful.These devices, adapted from touch-screens, 
are small enough to be located at the fingertips of 
a hand. One solution is to modify the barometric pres¬ 
sure sensing chips common in cell phones, converting 
them into pressure sensing elements [28.95, 176]. An¬ 
other is to use processors adapted for capacitive touch 
screens [28.94, 103]. Still another involves arrays of op¬ 
tical tactile devices [28.22]. 
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28.5 Conclusions and Future Developments 


In comparison to computer vision, tactile sensing al¬ 
ways seems to be a few years away from widespread 
adoption. As explained in the introduction to this chap¬ 
ter, the reasons include physical problems (placement 
and robustness of sensors, wiring challenges) and the 
diversity of sensor types for detecting forces, pressures, 
local geometries, vibrations, etc. As we have seen, the 
transduction and interpretation methods are typically 
different for each of these tactile quantities. However, 
there are some basic issues that apply to tactile sensing 
in general. For example, sensors are generally located 
within or beneath a compliant skin, which affects the 
quantities that they sense in comparison to pressures, 
stresses, thermal gradients or displacements applied to 
the skin surface. 

When choosing tactile sensors for a robot arm or 
hand, it is effective to begin with a consideration of 
which tactile quantities are most desired and for what 
purpose. For example, if the main concern is to ob¬ 
tain accurate measurements of loads or contact forces 


at sufficient data rates for force servoing, then intrinsic 
tactile sensing may make the most sense. If manipu¬ 
lating objects with soft contacts and with sliding or 
rolling, curved array sensors for measuring pressure 
distributions, or perhaps local skin deflections, may be 
desirable. If exploring objects to learn about their tex¬ 
ture and material composition, dynamic tactile sensors 
and thermal sensors may be effective. 

In an ideal world, one would incorporate all these 
tactile sensors in a robotic end-effector without regard 
to cost, signal processing or wiring complexity. For¬ 
tunately, the cost and size of transducers suitable for 
tactile sensing are steadily dropping and the ability to 
perform localized processing is improving with surface- 
mounted devices on flexible circuits. In the near future 
it will be increasingly possible to fabricate dense arrays 
of transducers in-situ on contoured surfaces, using ma¬ 
terial deposition and laser machining techniques. In this 
way, robots may finally start to approach the tactile sen¬ 
sitivity and responsiveness of animals. 


Video-References 


i®XKia 




VIDEO 15 


The effect of twice dropping, and then gently placing, a two gram weight on a small capacitive tactile 
array 

available from http://handbookofrobotics.org/view-chapter/28/videodetails/14 
Capacitive tactile sensing 

available from http://handbookofrobotics.org/view-chapter/28/videodetails/15 
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29. Inertial Sensing, GPS and Odometry 


Gregory Dudek, Michael Jenkin 



This chapter examines how certain properties of 
the world can be exploited in order for a robot or 
other device to develop a model of its own motion 
or pose (position and orientation) relative to an ex¬ 
ternal frame of reference. Although this is a critical 
problem for many autonomous robotic systems, 
the problem of establishing and maintaining an 
orientation or position estimate of a mobile agent 
has a long history in terrestrial navigation. 
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29.1 Odometry 

The word odometry is a contraction of the Greek words 
hodos meaning travel or journey, and metron mean¬ 
ing measure. Given its importance to a wide variety 
of applications from civil engineering to military con¬ 
quest, the basic concepts that underly odometry have 
been studied for over 2000 years. Perhaps the earliest 
reference to odometry appears in his Ten Books on Ar¬ 
chitecture where Vitruvius describes [29.1] 

a useful invention of the greatest ingenuity, trans¬ 
mitted by our predecessors, which enables us, while 
sitting in a carriage on the road or sailing by sea, to 


know how many miles of a journey we have accom¬ 
plished. 

In the context of autonomous vehicles, odometry usu¬ 
ally refers to the use of data from the actuators (wheels, 
treads, etc.) to estimate the overall motion of the vehi¬ 
cle. The basic concept [29.2] is to develop a mathemat¬ 
ical model of how commanded motions of the vehicles 
wheels/joints/etc. induce motion of the vehicle itself, 
and then to integrate these commanded motions over 
time in order to develop a model of the pose of the 
vehicle as a function of time. The use of odometry infor- 
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mation to estimate the pose of the vehicle as a function 
of time is known as Dead Reckoning or Deductive 
Reckoning and finds wide application in navigation at 
sea [29.3]. 

The details of odometry estimation varies with ve¬ 
hicle design. In the context of mobile robots perhaps the 
simplest vehicle for odometry estimation is the differen¬ 
tial drive vehicle (Fig. 29.1). A differential drive vehicle 
has two driveable wheels which are independently con¬ 
trollable and which are mounted along a common axis. 
Assuming that the location of the wheels are fixed 
on the vehicle, then for the wheels to remain in con¬ 
stant contact with the ground, the two wheels must 
describe arcs on the plane such that the vehicle rotates 
around a point (known as the ICC - instantaneous cen¬ 
ter of curvature) that lies on the wheels’ common axis 
(Fig. 29.1). If the ground contact speeds of the left and 
right wheels are and v r respectively, and the wheels 
are separated by a distance 2 d, then 

a>(R + d) = v r , 
a>(R — d) = v\. 

We can rearrange these two equations to solve for u> the 
rate of rotation about the ICC and R the distance from 
the center of the robot to the ICC 

a) = (v r — v\)/2d 
R = d{v r + v\)/(v r - . 


The instantaneous velocity of the point midway be¬ 
tween the robot’s wheels is given by V = cvR. 

Now as U| and v r are functions of time we can gen¬ 
erate the equation of motion for the differential drive 
robot. Using the point midway between the wheels as 
the origin of the robot, and writing 0 as the orientation 
of the robot with respect to the jr-axis of a global Carte¬ 
sian coordinate system obtains 

x(t) = j V{t) cos [0(f)] df, 

y(t) = j V(t) sin [0(f)] df, 

0 (f) = J <w(f)df. 

This is the solution for the odometry of a differen¬ 
tial drive vehicle on the plane. Given control inputs (ui 
and v T ) and some initial state estimate, we can estimate 
where an idealized robot using this motion model will 
be at any time f. 

Given such a model and complete knowledge of 
the control inputs, we should, in principle, be able 
to estimate a robot’s pose at any time. In a perfect 
world this would be all that is necessary to estimate 
accurately the robot’s pose at any time in the fu¬ 
ture. Unfortunately errors in the modelling (incorrect 
estimations of wheel size, vehicle size), uncertainty 
about the control inputs, realities of the motor con¬ 
troller (errors between commanded wheel rotation and 
true rotation), errors in the physical modelling of the 
robot (wheel compaction, ground compaction, wheel 
slippage, non-zero tire width), etc., introduces an error 
between the dead reckoning estimate of the vehicle mo¬ 
tion and its true motion. The problem of correcting for 
this error is the problem of pose maintenance for the 
vehicle, and requires the integration of the dead reckon¬ 
ing estimate with estimates obtained from other sensor 
systems. 

Other chapters in this handbook examine sensors 
that rely on external events, visual and otherwise, that 
can provide information as to the robot’s pose or 
changes in it’s pose. Here we consider sensors that 
transduce physical properties of matter under the influ¬ 
ence of external forces and properties of matter and the 
use of global positioning systems (GPS). 
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29.2 Gyroscopic Systems 

The goal of gyroscopic systems is to measure changes 
in vehicle orientation by taking advantage of physi¬ 
cal laws that produce predictable effects under rotation. 
A rotating frame is not an inertial frame, and thus many 
physical systems will appear to behave in an apparently 
non-Newtonian manner if the frame is rotating. By mea¬ 
suring these deviations from what would be expected in 
a Newtonian frame the underlying self-rotation can be 
determined. 

29.2.1 Mechanical Systems 

Mechanical gyroscopes and gyrocompasses have a long 
history in navigation, Bohnenberger is generally cred¬ 
ited with the first recorded construction of a gyro¬ 
scope [29.4]. The gyrocompass was patented in 1885 
by Martinus Gerardus ven den Bos. In 1903 Herman 
Anschiitz-Kaempfe constructed a working gyrocom¬ 
pass and obtained a patent on the design. In 1908 Elmer 
Sperry patented a gyrocompass in the US and then at¬ 
tempted to sell this device to the German Navy. A patent 
battle followed and Albert Einstein testified in the case. 
([29.5-8] for more details on the history of the gyro¬ 
compass and its inventors.) 

Gyroscopes and gyrocompasses rely on the princi¬ 
ple of the conservation of angular momentum [29.9]. 
Angular momentum is the tendency of a rotating ob¬ 
ject to keep rotating at the same angular speed about 
the same axis of rotation in the absence of an external 
torque. The angular momentum L of an object with mo¬ 
ment of inertia / rotating at angular speed a> is given by 

L = I x co. 

Consider a rapidly spinning wheel mounted on 
a shaft so that it is free to change its axis of rotation 
(Fig. 29.2a). Assuming no friction on the bearings or 
air resistance, the rotor axis will remain constant regard¬ 
less of the motion of the external cage. This constancy 
of orientation can be exploited to maintain a bearing in¬ 
dependently of the motion of the vehicle, although it is 
not usually desirable to utilize the principle of conserva¬ 
tion of angular momentum via a gyroscope directly. To 
see this, suppose that a gyroscope is set on the equator, 
with its spinning axis aligned pointing along the equa¬ 
tor (Fig. 29.2b). As the earth spins, the gyroscope will 
maintain a constant axis of orientation and thus to an 
earth-fixed observer will appear to rotate returning to 
its original orientation every 24 hours. Similarly, if the 
gyroscope were to be positioned on the equator such 
that its spinning axis was parallel to the axis of rotation 
of the earth, the gyroscope’s axis of rotation would re¬ 


main stationary and would appear to remain stationary 
to an earth-fixed observer as the planet rotates. 

Although this global motion limits the mechanical 
gyroscope’s ability to sense absolute orientation di¬ 
rectly, gyroscopes can be used to measure local changes 
in orientation, and thus are well suited to vehicular 
robotic applications. Rate gyros (RGs) measure a ve¬ 
hicle’s rotation rate (its angular rate of rotation). This 
is the fundamental measurement that is the basis of all 
gyroscopic systems. Rate-integrating gyros (RIGs) use 
embedded processing to internally integrate the angu¬ 
lar rotation rate to produce an estimate of the absolute 
angular displacement of the vehicle. 

In order to exploit a gyroscope for navigation with 
respect to an earth-stable frame, it is desirable for the 
rotational axis of the shaft to remain fixed within the 
earth frame, rather than remaining fixed with respect to 
an external frame. A gyrocompass obtains this by rely¬ 
ing on precession. When a torque is applied to change 
the axis of rotation of a spinning object, conservation 
of angular momentum will cause a change in the spin 
direction that is perpendicular to the angular momen¬ 
tum and the direction in which the torque is applied. 
This is the effect that causes gyroscopes suspended at 
one end to spin around the end from which they have 
been suspended. Consider the pendulus gyro sketched 
in Fig. 29.3a. This is a standard gyroscope with a weight 
suspended below the rotational axis. As before, imagine 
this pendulus gyrocompass set spinning on the equator 
with the axis of rotation aligned with the axis of rota¬ 
tion of the planet, and with the weight hanging directly 
down. As the planet spins, the gyroscope’s axis of ro¬ 
tation remains stationary and would appear to remain 



Fig.29.2a,b Mechanical gyroscope, (a) Traditionally gimballed 
gyroscope. The gimball provides the gyroscope the freedom to 
rotate about its axis as the base of the gyroscope is rotated (b) gyro¬ 
scope as it is rotated around the planet. The wheel of the gyroscope 
(grey) remains in the same orientation as it revolves with the planet. 
To an observer on the planet the gyroscope will appear to rotate 
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a) 

Vertical axis 


b) 



Fig.29.3a,b Simple gyrocompass: (a) Pendulus gyro; (b) Precessional motion 


stationary as the planet rotates. Now suppose that in¬ 
stead of being aligned with the spin axis of the planet, 
the spin axis is aligned with the equator. As the planet 
spins, the spin axis is drawn out of the plane as it re¬ 
mains aligned with the original spin axis. As it becomes 
drawn out of the plane, the mass hanging below the gy¬ 
rocompass is raised up and generates a torque down due 
to gravity. The direction perpendicular to the spin axis 
and the torque rotates the spin axis away from the equa¬ 
tor (aka towards the true pole). This process is sketched 
in Fig. 29.3b. 

Unfortunately the pendulus gyro is not an ideal de¬ 
vice for navigation. Although its rotation axis will align 
with the rotation axis of the planet, it does not converge 
to this value directly but oscillates about it. The solu¬ 
tion to this damping problem is to use oil reservoirs, 
rather than a solid weight, as the counterbalance, and to 
restrict the motion of the oil in the reservoir [29.10]. 

a ) Start b) 

Start End 


Fig.29.4a,b Circular light path, (a) Stationary path; (b) Moving 
path 


The gyrocompass finds true north by controlling the 
precession of a gyroscope. In practice, the performance 
of a mechanical gyrocompass is impacted by external 
forces acting on the compass which also contribute to 
the precession of the gyroscope. This includes forces 
generated by motion of the device containing the gyro¬ 
compass, as well as any external forces acting on the 
vehicle. Another issue for mechanical gyrocompasses 
is that in latitudes away from the equator, the stable 
position of the gyrocompass is not horizontal, and ac¬ 
curate estimates of true north in such latitudes requires 
corrections to be applied to raw gyrocompass values. 
Finally, mechanical gyrocompasses require an external 
force to be applied to maintain the spin of the gyro¬ 
scope. This process introduces unwanted forces into the 
system which can further corrupt the measurement pro¬ 
cess. 

Given the complexity, cost, size, the delicate nature 
of gyrocompasses, and the availability of less expensive 
and more reliable technologies, mechanical gyrocom¬ 
passes have given way to optical- and MEMS-based 
systems (MEMS or microelectromechanical systems 
are also known as micromachines or micro systems 
technology and refer to mechanisms at the scale of inte¬ 
grated circuits). 

29.2.2 Optical Systems 

Optical gyroscopes rely on the Sagnac effect rather than 
rotational inertia in order to measure (relative) head¬ 
ing. The mechanism is based on the behaviour of an 
optical standing wave in a rotating frame. Historically 
this was first produced using lasers and an arrangement 
of mirrors, but it is now typically obtained using fibre- 
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optic technology. The Sagnac Effect is named after its 
discoverer Georges Sagnac [29.11, 12]. The underlying 
concept can be traced back even earlier to the work of 
Harress [29.13], and perhaps finds its most famous ap¬ 
plication in terms of the measurement of the rotation of 
the earth [29.14], 

Ignore relativistic effects and consider the circu¬ 
lar light path shown in Fig. 29.4a. If two light pulses 
are sent in opposite directions around a stationary path 
of perimeter D = 2 icR they will travel the same dis¬ 
tance at the same speed. They will arrive at the starting 
point simultaneously, taking time t = D/c where c is 
the speed of light in the medium. Now let us sup¬ 
pose that instead of being stationary, this circular light 
path rotates clockwise about its center at rotational 
speed a) (Fig. 29.4b). The light travelling clockwise 
around the path must go farther to reach the starting 
point, while light travelling counterclockwise around 
the path goes a shorter distance. The clockwise path 
has distance 

D c = 2jtR + coRt c , 

where 4 is the time taken in the clockwise direction, 
while the counterclockwise path has distance 

D a = 2 icR — a>Rt a , 

where 4 is the time taken in the counterclockwise di¬ 
rection. But D c = ct c and D a = c 4 , so 


and 

4 = 2nR(c + u>R) . 

The time difference At = 4 — 4 is given by 

At=2TTR\ —----) . 

V c — co R c + coR ) 

By measuring At, the rotational speed can be com¬ 
puted. Note that although the above derivation assumes 
classical mechanics and ignores relativistic effects, the 
derivation also applies when relativistic speeds are 
taken into account [29.15]. See [29.16] for an in depth 
review of the Sagnac effect and ring-lasers. 

In optical gyroscopes lasers are typically used as the 
light source. Optical gyroscopes either utilize straight 
line light paths with mirror surfaces or prisms at the 
edges to direct the light beam (a ring laser gyroscope - 
RFG), or a polarization maintaining glass-fiber loop 


(fiber optic gyro - FOG). The glass-fiber may actu¬ 
ally loop multiple times, thus extending the effective 
length of the light path. The time delay between the 
clockwise and counter-clockwise is detected by exam¬ 
ining the phase interference between the clockwise and 
counter-clockwise light signals. Multiple optical gyro¬ 
scopes with non-parallel axes can be ganged together in 
order to measure 3-D rotations. 

Various techniques can be used to measure the 
time difference including examining the Doppler (fre¬ 
quency) shift of the laser light due to the motion of 
the gyro and an examination of the beat frequency 
of the interference pattern between the clockwise and 
counter-clockwise paths [29.17]. Ring interferometers 
typically consist of many windings of fiber optic lines 
that conduct light of a fixed frequency in opposite direc¬ 
tions around the loop and measure the phase difference. 
A ring laser consists of a laser cavity in the shape of 
a ring. Light circulates in both directions around this 
cavity producing two standing waves with the same 
number of nodes in both directions. Since the optical 
path lengths are different in the two directions, the res¬ 
onate frequencies differ. The difference between these 
two frequencies is measured. An unfortunate side effect 
of the ring laser approach is that the two signals will 
lock to each other for small rotations and it is typically 
necessary to physically rotate the device in a controlled 
manner in order to ensure that this lock-in effect can be 
avoided. 

29.2.3 MEMS 

Almost all MEMS gyroscopes are based on vibrating 
mechanical elements to sense rotation. Vibratory gyro¬ 
scopes rely on the transfer of energy between vibratory 
modes based on Coriolis acceleration. Coriolis acceler¬ 
ation is the apparent acceleration that arises in a rotating 
frame of references. Suppose an object moves along 
a straight line in a rotating frame of reference. To an 
outside observer in an inertial frame the object’s path is 
curved - thus there must be some force acting on the 
object to maintain the straight line motion as viewed 
by the rotating observer. An object moving in a straight 
line with local velocity v in a frame rotating at rate S2 
relative to an inertial frame will experience a Coriolis 
acceleration given by 

a = 2v x i2 . 

Transducing acceleration in a MEMS gyroscope 
amounts to inducing some local linear velocity and 
measuring the resultant Coriolis forces. 

Early MEMS gyroscopes utilized vibrating quartz 
crystals to generate the necessary linear motion. More 
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Fig. 29.5 MEMS gyroscope principle of operation 

recent designs have replaced the vibrating quartz crys¬ 
tals with silicon based vibrators. Various MEMS struc¬ 
tures have been developed including 

Tuning Fork Gyroscopes 

Tuning fork gyroscopes use a tuning fork-like structure 
(Fig. 29.5) as the underlying mechanism. As the tun¬ 
ing forks vibrate within a rotating frame, Coriolis forces 
cause the tines of the fork to vibrate out of the plane of 
the fork which is measured. This is the effect used by 
the InertiaCube sensor [29.18]. 

Vibrating Wheel Gyroscopes 
Vibrating wheel gyroscopes use a wheel that oscillates 
about its rotational axis. External rotation of the frame 
causes the wheel to tilt which can be sensed. 

29.3 Accelerometers 

Just as gyroscopes can be used to measure changes in 
orientation of a robot, other inertial sensors - known 
as accelerometers - can be used to measure external 
forces acting on the vehicle. One important factor con¬ 
cerning accelerometers is that they are sensitive to all 
external forces acting upon them - including gravity. 
Accelerometers use one of a number of different mech¬ 
anisms to transduce external forces into a computer 
readable signal. 

29.3.1 Mechanical Accelerometer 

A mechanical accelerometer (Fig. 29.6a) is essentially 
a spring-mass-damper system with some mechanism 


Wine Glass Resonator Gyroscopes 
Wine glass resonator gyroscopes use the effect of Cori¬ 
olis forces on the position of nodal points on a resonat¬ 
ing structure to estimate the external rotation. Sadly, no 
actual wine glasses are involved. 

As MEMS gyroscopes have no rotating parts, 
have low power consumption requirements, and are 
very small, MEMS gyros are quickly replacing me¬ 
chanical and optical gyroscope sensors in robotic 
applications. 

With the exception of gyrocompasses, gyroscopes 
measure relative rotational motion around a single axis. 
They accomplish this measurement by exploiting phys¬ 
ical properties of rotating frames of reference. Earlier 
technologies based on mechanical gyroscopes have 
given way to optical- and MEMS-based devices but the 
underlying principle remains unchanged - that rotating 
frames of reference show specific physical properties 
can be measured to estimate the relative rotation. 

A problem common to all gyroscopes is that of drift. 
Each of the relative motion measurements is corrupted 
by an error process, and these errors accumulate over 
time. This, coupled with specific measurement errors 
associated with the individual gyroscope technologies 
means that unless the error is corrected through ref¬ 
erence to some alternate (external) measurement, the 
drift will eventually exceed the required accuracy of the 
measurement. 

As individual gyros only measure rotation about 
a single axis, it is common to gang multiple gyros 
together with orthogonal axes of sensitivity in or¬ 
der to measure 3-D rotations. These collections of 
gyros are often integrated with other sensors (com¬ 
passes, accelerometers, etc.) in order to construct iner¬ 
tial measurement units (or IMUs). This is considered in 
Sect. 29.4. 


for external monitoring. When some force is applied 
(e.g., gravity), the force acts on the mass and dis¬ 
places the spring. Assuming an ideal spring with a force 
proportional to its displacement, the external forces bal¬ 
ance the internal ones 

('applied — ('intcrtial T ('damping T ('spring * 

= mx + CX + kx , 

where c is the damping coefficient. This equation can be 
solved to show that depending on the size of the damp¬ 
ing coefficient relative to the expected external force 
and the mass, the system can be made to reach a stable 
final value in a reasonably short period of time when- 
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Fig. 29.6 Accelerometers, (a) Mechanical accelerometer; 
(b) piezoelectric accelerometer ◄ 

ever a static force is presented. This need to pre-esti¬ 
mate the expected force and the resulting (potentially 
long) time for the system to converge on a final mea¬ 
surement coupled with non-ideal performance of the 
spring limits the applicability of mechanical accelerom¬ 
eters. Another issue with mechanical accelerometers is 
that they are particularly sensitive to vibration. 

29.3.2 Piezoelectric Accelerometer 

Rather than relying on a direct mechanical measure¬ 
ment of external forces, piezoelectric accelerometers 
are based on a property exhibited by certain crystals 
which generate a voltage across them when stressed. 
A small mass can be positioned so that it is only sup¬ 
ported by the crystal, and as forces cause the mass to 
act upon the crystal this induces a voltage that can be 
measured (Fig. 29.6b). 


29.4 IMU Packages 

An inertial measurement unit (IMU) is a device that 
utilizes measurement systems such as gyroscopes and 
accelerometers to estimate the relative position, ve¬ 
locity and acceleration of a vehicle in motion. First 
demonstrated in 1949 by C. S. Draper, IMU’s have 
become a common navigational component of aircraft 
and ships. Historically an IMU is self-contained and 
provides this estimate without reference to external ref¬ 
erences, however the definition has become less precise 
in recent years and now it is common to have the term 
IMU also refer to systems which do include such exter¬ 
nal references. 

IMU’s come in two basic flavours, gimbaled sys¬ 
tems and strap-down systems. As their name suggests, 
gimbaled IMU’s are mounted within complex gimball 
structures in order to provide a stable platform from 
which measurements can be made. Gyroscopes are used 
to ensure that the gimball remains aligned with the ini¬ 
tial reference frame at power up. The orientation of the 
gimbaled platform relative to the vehicle is used to map 
measurements taken within the IMU to the reference 
frame of the vehicle. Strap down IMU’s, on the other 
hand, have the IMU rigidly connected to the vehicle 
(strapped down), so no such transformation is required. 
In either case estimating the motion relative to the initial 
frame requires integrating information from the sen¬ 
sors within the IMU (accelerometers, gyroscopes, etc.) 
in real time. This was a significant computational ef¬ 


fort in the early days of IMU’s, and thus historically 
(prior to the 1970s) the gimbaled IMU was more com¬ 
mon. Given the low expense of such computation today, 
and the costs associated with manufacturing and operat¬ 
ing gimbaled IMU’s, strap down IMU’s are much more 
common today [29.19]. 

The sensing technology of an IMU is typically inte¬ 
grated with an on-board computational unit to produce 
an attitude and heading reference unit (AHRS) which 
is a single device that maintains a 6-DOF estimate of 
the pose of the vehicle (position ( x , y, z) and orien¬ 
tation (roll, pitch, yaw)). In practice, the distinction 
between an AHRS and an IMU is often unclear since 
the combinations of software and device outputs is quite 
varied, and most MEMS devices contain a substantial 
computational substrate and are able to directly offer 
up a position, velocity, and attitude (PVA) solution. In 
general, an AHRS includes gyroscopes, accelerometers 
and a magnetometer. These are often combined with 
a GPS as described in the next section. In addition to 
maintaining a 6-DOF pose of the vehicle, commercial 
IMU’s also typically maintain estimates of velocity and 
acceleration and can usually provide a diverse set of 
lower-level measurements as well. 

The basic computational task of an IMU is shown in 
Fig. 29.7. This IMU uses three orthogonal accelerome¬ 
ters and three orthogonal gyroscopes. The Gyroscope 
data u> is integrated to maintain an ongoing estimate 
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Fig. 29.7 IMU block diagram 


of vehicle orientation 6. At the same time, three ac¬ 
celerometers are used to estimate the instantaneous 
vehicle acceleration a. This data is then transformed via 
the current estimate of the vehicle orientation relative to 
gravity, so that the gravity vector can be estimated and 
extracted from the measurement. The resulting acceler¬ 
ation is then integrated to obtain vehicle velocity v and 
then integrated again to obtain position r. 

IMU’s are extremely sensitive to measurement er¬ 
rors in the underlying gyroscopes and accelerometers. 
Drift in the gyroscopes leads to mis-estimates of the 
vehicle orientation relative to gravity, resulting in an 
incorrect cancellation of the gravity vector. As the 
accelerometer data is integrated twice, any residual 
gravity vector will result in a quadratic error in po¬ 
sition [29.18]. As it is never possible to completely 
eliminate the gravity vector and this and any other er¬ 
ror is integrated over time drift is a fundamental issue 
for any IMU. Given a sufficiently long period of opera¬ 
tion all IMU’s eventually drift and reference to some 
external measurement is required to correct this. For 
many field robots GPS has become an effective source 
for these external corrections. 

29.4.1 Performance 

Gyroscopes and Inertial measurements units today fre¬ 
quently include the required computational support to 
directly integrate the inertial signals and provide po¬ 
sition, velocity, and attitude (PVA) estimates as direct 
output. The quality of these estimates is critically de¬ 


pendent not only on the sensing technology, but on 
the algorithms (almost universally including an ex¬ 
tended Kalman filter) used in the estimation process and 
their calibration. While the extended Kalman filer is, 
of course, a well understood algorithm, the subtlety of 
its implementation remains a significant distinguishing 
factor among different brands of IMU. 

Intertial measurement units can be evaluated with 
respect to various factors that determine performance. 
Several of these are enumerated below: 

1. Bias repeatability. This is maximum deviation of the 
gyroscope under fixed inertial operation conditions 
with constant temperature, i. e., the drift of the read¬ 
ing under ideal conditions. This is measured over 
different time scales and leads to short term and 
long term bias repeatability. 

2. Angle random walk. This measures the noise in the 
angular rate data coming from the gyro. 

3. Scale factor ratio. This parameter is not specific to 
IMU’s or gyros and is a general measurement of sig¬ 
nal amplitude. It measures the ratio of the output 
analogue voltage to the sensor parameter of inter¬ 
est. For a gyroscope this is typically measured in 
mV/(deg/s) whereas for an accelerometer it is typ¬ 
ically mV/(m/s 2 ). 

4. Position accuracy. The accuracy in practice of po¬ 
sition estimation depends on the sensors and the 
integration algorithm, both of which are intrinsic to 
the IMU, as well as the nature of the trajectory ex¬ 
perienced by the IMU. 


29.5 Satellite-Based Positioning (GPS and GLS) 

The global navigation satellite system (GNSS) and mechanism for location estimation. It provides a three 
it’s most commonplace instance, the global position- dimensional position estimate in absolute coordinates 
ing system (GPS), is the single most commonly used as well as current time and date and is available any- 
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where on the earth’s surface. Standard GPS provides 
a position estimate in the horizontal plane to within 
about 20 meters. It was original developed for mil¬ 
itary applications but has become widely adopted in 
civilian applications including automobile navigation 
systems, recreational orienteering, and inventory track¬ 
ing for transportation companies. It is rapidly being 
superseded (via augmentation) by systems that add re¬ 
lated technologies to basic GPS reception. 

29.5.1 Overview 

GPS is based on received radio signals transmitted by 
an ensemble of satellites orbiting the earth. By compar¬ 
ing the time delays from the different satellite signals, 
a position fix can be computed. The most widely ac¬ 
cepted GPS system is based on the NAVSTAR satellite 
system deployed and maintained by the United States, 
specifically by the Air Force Space Command. As a US 
military service, the US government reserves the right 
to terminate or modify it’s availability at their discre¬ 
tion. A similar system named GLONASS (globalnaya 
navigatsionnaya sputnikovaya sistema) is operated by 
the Russian government. Many modern GPS receivers 
and cell phones can access both the US GPS and Rus¬ 
sian GLONASS systems. Another alternative system is 
being deployed by the European Union, named Galileo, 
with the explicit expectation that it not be under mili¬ 
tary control. It is expected to offer two different classes 
of service, an open service and an encrypted higher- 
quality commercial service. Other GPS systems such 
as the Chinese Beidou, the Japanese quasi-zenith satel¬ 
lite system (QZSS), the Indian regional navigational 
satellite system (IRNSS) system are intended to pro¬ 
vide similar functionality. GPS receivers that are able 
to receive position data from multiple different satellite 
systems have become readily available and are referred 
to as global navigation satellite system (GNSS) sys¬ 
tems. 

Technically, the term GPS always refers to the 
NAVSTAR system although in common parlance it is 
sometime used to refer to any system that provides 
satellite-based positioning. Historically NAVSTAR pro¬ 
vided two different services: the precise positioning 
system (PPS), reserved primarily for military users, and 
the standard position system (SPS) with lower accuracy. 
The difference between these was referred to as selec¬ 
tive availability (SA). This difference in accuracy was 
artificially induced via pseudo-random noise in the SPS 
signal for strategic reasons, and was eliminated on May 
2, 2000. Although this accuracy distinction could, in 
principle, be reinstated this seems unlikely, and recent 
GPS satellites (GPS Block III series, not launched at 
this time) are not equipped with the SA feature. US mil¬ 


itary users have access to a precise positioning service 
(PPS). This second frequency enables military users 
to correct for radio signal degradation caused by the 
Earth’s atmosphere. 

The GPS satellite network is based on a base con¬ 
stellation of 24 orbiting satellites along with up to six 
supplementary additional satellites that are also oper¬ 
ational. These satellites are in almost-circular medium 
earth orbit. As opposed to being geostationary the or¬ 
bits are semi-synchronous, meaning that their position 
relative to a ground observer is constantly changing, 
and that their orbital period is exactly half a sidereal 
day. The orbits are selected so that from almost any 
point of the earth’s surface there will always be four or 
more satellites directly visible - a criterion for obtain¬ 
ing a GPS position estimate. The satellites are organized 
into six orbital planes with four satellites in each. The 
system is designed such that averaged over the entire 
earth’s surface and over a 24 hour internal, the satellites 
should provide 99.9% coverage at the worst-covered lo¬ 
cation in any 24 hour interval, and the signal should 
be at least 83.92% available at the worst place on earth 
on the worst day over a 30 day measurement interval. 
Note that this criterion for availability takes into ac¬ 
count transmitted operational factors above and beyond 
simple coverage. Of course, this criterion ignores the 
reality of topographic features like mountains, as well 
as other objects such as buildings that can obstruct the 
like-of-sight. 

Each satellite repeatedly broadcasts a data packet 
known as the coarse-acquisition (C/A) code, which is 
received by the GPS receiver on the LI channel at 
1575.42MHz. The simple principle is that if the re¬ 
ceiver knows the absolute positions of the observed 
satellites, the receiver position can be directly deter¬ 
mined. If the signal propagation time for the radio 
signals were known, the receiver position could be 
computed directly via trilateration (Fig. 29.8). This im¬ 
plies that an absolute timing reference is present on 
the receiver, which would be prohibitively costly. In¬ 
stead, only the satellites have highly accurate atomic 
clocks. The receiver computes the difference in sig¬ 
nal propagation times between the different satellites, 
and uses this to compute a range estimate referred to 
as a pseudo-range (to explicitly indicate that it is cor¬ 
rupted by several sources of measurement noise). The 
specific geometric problem is referred to as multilat- 
eration or hyperbolic positioning and the solution is 
computed using a sophistical Kalman filter within the 
GPS receiver. To avoid retaining an ephemeris (pose) 
table for the satellites and a very accurate clock in the 
receiver, each satellite broadcasts its own position and 
an accurate time signal as part of the data packet that it 
transmits. 
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GPS satellites broadcast at several different fre¬ 
quencies known as LI through L5 although until re¬ 
cently only LI (1575.42MHz) and L2 (1227.6MHz) 
were being using for civilian GPS receivers. Dual fre¬ 
quency L1/L5 GPS systems have become more broadly 
largely due to the emergence other GNSS systems in 
addition to the original NAVSTAR GPS. 

The standard service offered by NAVSTAR and the 
performance criteria for it are determined by the LI 
signal, which contains two unencrypted components: 
the acquisition message (coarse acquisition message 
C/A) and a navigation data message. It is also possible 
to use the encrypted L2 signal as well, even with¬ 
out the secret decryption keys, to provide augmented 
error correction (by observing the relative effects of 
ionospheric distortion as a function of frequency). The 
restricted access signal broadcast on both the LI and 
L2 channels is the P-code (as well as a fairly recent 
M-code) which, is known as the Y-code or P(Y) or 



Fig. 29.8 GPS trilateration on the plane. Suppose that one 
receives signals from three transmitters (A, B, and C) 
with known locations. Knowledge of the signal delay from 
one emitter (say A) localizes the receiver to lie on a cir¬ 
cle of known diameter (^a) whose center is the emitter. 
The constraints from two emitters intersect at two points 
(maximum). A third emitter is required to disambiguate 
these two solutions. In 3-D, the signal propagation con¬ 
straint from a single emitter constrains the receiver to lie 
on a sphere. The intersection of the constraints from two 
emitters constrains the receiver to lie on a circle. The in¬ 
tersection of the constraints from three emitters constrains 
the receiver to one of two points 


P/Y code once it is encrypted. Both the C/A and P(Y) 
code include the navigation message stream that spec¬ 
ifies clock bias data, orbital information, ionospherical 
propagation corrections factors, ephemeris data, status 
information on all the satellites, universal time code 
and other information. The satellite performance is co¬ 
ordinated by the master control station located at the 
Schriever Air Force Base near Colorado Springs, (Col¬ 
orado, USA) and is connected to a global network of 
five additional monitoring stations (Cape Canaveral, 
Ascension Island, Kwajalein Atoll (Marshall Islands), 
Diego Garcia Atoll, Hawaii) which are used to pro¬ 
duce the measurements that are uplinked to generate the 
navigation message stream. Finally, it should be noted 
an additional signal is now available on the L2 fre¬ 
quency band. This L2C signal on satellites designated 
block IIR-M promises to provide much improved re¬ 
ceiver sensitivity so that position fixes can be obtained 
in environments, such as in forests, where they are cur¬ 
rently not readily available. 

29.5.2 Performance Factors 

GPS performance depends on several factors: satel¬ 
lite transmission accuracy, environmental conditions, 
interactions with ground-based obstacles, and receiver 
properties. 

In the context of robotics, factors that effect the 
performance of the satellites themselves and the atmo¬ 
spheric conditions are essentially uncontrollable. Nev¬ 
ertheless, it should be noted that these can be sources of 
error and that the GPS signal itself may not always be 
reliable. A service failure is defined as a set of circum¬ 
stances where the positioning service exhibits atypical 
error behavior (i. e., incorrect signals). Such failures are 
classified into minor and major failures. Minor failures 
are those that have limited impact on a receiver and lead 
to ranging error of under 150 m. Major failures are those 
that lead either to larger errors or data processing over¬ 
loads in the receiver. If a single satellite experiences an 
error that leads to a major failure, then within a 6 hour 
period, approximately 63% of the earth’s surface will 
have the satellite in view a some point. 

The controllable factors in using GPS for accu¬ 
rate localization are (i) it requires an unobstructed 
line of sight to the satellites, (ii) it depends on atmo¬ 
spheric conditions, and (iii) it depends the ability to 
receive (weak) radio-frequency communications. There 
is a potential for wildly incorrect estimates. Generally 
satellites that are directly overhead provide better sig¬ 
nal than those near the horizon. In addition, since the 
basis of GPS position is differential signal analysis, it 
is best if the satellites used in the GPS computation are 
widely spaced in the sky. 
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GPS signals are in the microwave band and, as such, 
they can pass through plastic and glass, but are ab¬ 
sorbed by water (wood, heavy foliage) and are reflected 
by many materials. As a consequence, GPS is unreli¬ 
able in heavy forest, deep canyons, inside automobiles 
and boats, heavy snowfall or between tall buildings. 
In some cases, partial obstruction of the sky may not 
prevent a position estimate from being computed. As¬ 
suming the minimum number of satellites operating 
at any time is 24, then on average across the earth’s 
surface 8 satellites are in view so that even partial 
occlusion of the sky can often be tolerated. On the 
other hand, partial occlusion can lead to reduced accu¬ 
racy since the selection of available satellites used for 
computing position becomes limited and optimal accu¬ 
racy is obtained by using as many satellites as possible 
(weighting them appropriately in the internal Kalman 
filter). 

Secondary factors that differentiate different GPS 
receivers are the rate at which the signals are collected, 
the receiver sensitivity, the number of satellites used 
in the final computation, the number of factors taken 
into account in the estimator and the exploitation of 
supplementary positioning schemes such as WAAS (de¬ 
scribed later). A major factor in determining the rate 
at which estimates can be produced is the number of 
independent receiver elements in the GPS system. Se¬ 
quential single-channel receivers are simpler and thus 
more economical (and potentially smaller), but they 
must sequentially lock onto each satellite being used. 
Parallel multi-channel receivers can lock onto to more 
than one satellite at once, and are generally faster and 
more costly; some degree of parallelism is the norm in 
good-quality consumer devices. 

GPS computations are based on an estimation of the 
dilution of precision (DOP) and specifically for the di¬ 
lution of precision of the positional parts of the system 
(i. e., positional dilution of precision PDOP). These cor¬ 
respond to partial derivatives of the error with respect to 
position and allow the most accurate ensemble of visi¬ 
ble satellites to be determined at any time. The standard 
implementation for GPS systems specifies that PDOP 
values be recomputed every 5 minutes. 

The minimum performance parameters for GPS 
receivers are based upon transforming instantaneous 
range residuals to a user position estimate using a lin¬ 
earized position solution from a stationary, surveyed 
location. Most GPS receivers use additional techniques 
such as range residual smoothing, velocity aiding, 
Kalman filtering, or multiple satellite ( all-in-view satel¬ 
lite) solutions. That said, formal performance for the 
system is measured with respect to the minimum. The 
GPS position estimation algorithm is summarized as 
follows: 


1. Select the best four satellites based upon minimum 
error measured in terms of PDOP. 

2. Update every five minutes, or whenever a satellite 
being used in the solution sets. 

3. Measure the pseudo range to each satellite. Each 
of the four measurements must have a reception 
timetag within 0.5 s of the solution time. The re¬ 
ception timetag is based upon measurement system 
time, and the transmission timetag is based upon 
satellite time. 

4. Determine the ephemeris for each of the satel¬ 
lites being used, and compute the Earth-centred, 
Earth-fixed (ECEF) coordinates for each. Correct 
for the earth’s rotation and thus compute an estimate 
pseudo-range measurement that should be obtained 
for each satellite. 

5. Compute the range residuals as the differences be¬ 
tween the actual and observed measurements. 

6. Estimate the matrix G that determines the overall 
system solution, know as the position solution ge¬ 
ometry matrix. The matrix can be described in terms 
of a collection of row vectors one for each of the 
satellites being used, each row being made up of 
the x, y, z and time coordinate direction cosines for 
the vector between the user and the satellite (the 
is with respect to a fixed reference frame for the 
planet called the World Geodetic System, WGS84). 
WGS84 represents position on the Earth in terms 
of latitude, longitude, and height above the sur¬ 
face, which is modeled as an ellipsoid. The WGS84 
coorinate system represents longitude as a value 
from —180 to 180 degrees, with 0 degrees at the 
Earth’s prime meridian, and longitude as a value 
from 90 degrees (at the North pole) to —90. 

7. Compute the user position. 

The standard implementation of GPS is based upon 
a position fix rate of once per second, although faster 
and slower rates are possible. Under typical operating 
conditions and without specialized enhancements GPS 
accuracy is roughly 20—25 m in the horizontal plane 
and 43 m in the vertical direction. The restricted PPS 
signal provides an accuracy of at least 22 m (typical 
values are 7—10m) in the horizontal plane and 27.7 m 
in the vertical direction as well as UTC time accuracy 
within 200 ns based on a reference signal from the US 
Naval Observatory. 

GPS signals can be affected by multipath issues, 
where the radio signals reflect off surrounding terrain - 
buildings, canyon walls, hard ground, etc. This delay 
in reaching the receiver causes inaccuracy. A variety 
of receiver techniques, most notably narrow correlator 
spacing [29.20], have been developed to mitigate multi- 
path errors. For long delay multipath, the receiver itself 
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can recognize the wayward signal and discard it. To 
address shorter delay multipath due to the signal reflect¬ 
ing off the ground, specialized antennas may be used. 
This form of multipath is harder to filter out as it is 
only slightly delayed as compared to the direct signal, 
causing effects almost indistinguishable from routine 
fluctuations in atmospheric delay. 

29.5.3 Enhanced GPS 

A wide range of augmentation systems have been de¬ 
veloped both by government agencies and private com¬ 
panies to enhance the basic GPS signal. 

Wide Area Augmentation System (WAAS) 

The wide area augmentation system (WAAS) is an aug¬ 
mentation system operated by the US Federal Aviation 
Administration (FAA) to provide better GPS and timing 
performance for aircraft operating over North America. 
The WAAS system is based around a supplementary 
signal that can be received by GPS receivers to im¬ 
prove their accuracy. WAAS increases the accuracy of 
horizontal position estimates from 10—12 m with GPS 
alone, to between 1 and 2 m. The WAAS signal contains 
corrections for the GPS signal that reduce the effects 
of errors due to timing errors, satellite position cor¬ 
rections, and local perturbations due to variations in 
the ionosphere. These correction terms are estimated 
by ground-based stations at fixed and accurately-known 
positions and uplinked to satellites which broadcast 
them to suitably enabled GPS receivers. The WAAS 
signal is only computed and available for North Amer¬ 
ica, but similar correction signals are becoming avail¬ 
able elsewhere as part of the standardization of satellite- 
based augmentation systems (SBASs). This includes 
Europe (where is it called EGNOS, the European Geo¬ 
stationary Navigation Overlay Service) and Japan and 
parts of Asia (where it is called MSAS, the multi¬ 
functional satellite augmentation system). Further en¬ 
hancements to GPS and WAAS, in the form of global 
navigation satellite system landing system (GLS), are 
slated for completion in 2013. 

Differential GPS 

Differential GPS (DGPS) is a technique for correcting 
GPS signals by using a nearby GPS receiver located at 
a known accurately-surveyed position. In fact, several 
variations on this basic idea exist and are also known 
under the general rubric of ground based augmentation 
systems (GBASs). DGPS uses the same principles as 
WAAS but on a local scale without resorting to the use 
of satellite uplinks. The receiver at the known position 
computes the error in the GPS signal and transmits it to 
the nearby receiver at the unknown location. Since the 


error varies as a function of position on the earth, the 
effectiveness of the correction degrades with distance, 
typically with a maximum effective range of a couple 
of hundred miles. The method was especially desir¬ 
able before the suspension of selective availability and 
the development of WAAS (which can be viewed as 
a form of DGPS). In the USA and Canada, a network of 
ground based DGPS transmitters are in place, send sig¬ 
nals using radio frequencies between 285 and 325 kHz. 
Commercial DGPS solutions also exist akin to WAAS. 

29.5.4 Nationwide Differential 
GPS System (NDGPS) 

The (US) nationwide differential GPS system (NDGPS) 
is an augmentation system that integrates costal and 
ground transportation systems to provide 10—15 cm ac¬ 
curacy throughout most of the US. The system relies on 
a network of radio signals broadcast from base stations. 
Similar systems exist in a number of other countries in¬ 
cluding Canada. 

Receiver Autonomous Integrity Monitor (RAIM) 
Receiver autonomous integrity monitor (RAIM) is 
a technique by which multiple pseudo-range measure¬ 
ments are obtained (i. e., pose estimates) using different 
combinations of satellites. If inconsistent measure¬ 
ments are obtained, it indicates that a failure of some 
sort has taken place in the system. A position fix using 
at least satellites is needed to detect such an error which 
at least six satellite are needed to exclude the data from 
a single bad satellite and still obtain a reliable estimate. 

Real-Time Kinematic Positioning (RTK) 

One GPS enhancement technology with rapidly grow¬ 
ing acceptance in robotics applications is real-time 
kinematic positioning or RTK [29.21], RTK-GPS in 
a GNSS configuration also exploits multiple satellite 
constellations and is often regarded as the most high- 
performance GPS solution that is readily available. 
Because of it’s high accuracy, it is often referred to 
as sur\>ey grade. In practice, commonplace RTK-GPS 
units can achieve horizontal positioning accuracy on 
the order of 1 cm. RTK-GPS augments standard dif¬ 
ferential GPS technology by enhancing the effective 
resolution of the pseudo-range distance estimates us¬ 
ing corrections obtained by a base station. Specifically, 
RTK estimates the number of carrier cycles in the com¬ 
munication path between a GPS satellite and each of the 
base station and the receiver, and thus is essentially de¬ 
pendent on the phase of the signal. Such units typically 
ignore the actual data payload of the signal and exploit 
the phase of the GPS signal (typically the LI band); 
this is a fundamentally different mode of operation from 
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the commonplace code phase computation that low cost 
receivers use. The base station that is used to provide 
a reference position for the estimation process can be 
part of a custom system being deployed, but various 
networks reference base stations accessible to the pub¬ 
lic now exist. When a network of base stations are used 
to provide correction signals over a wide region, the 
system is referred to as network real-time kinematic 
(NRTK) GPS and a class of networks as continuously 
operating reference station (CORS) networks. NRTK 
networks are evolving rapidly and a range computing 
protocols have been developed or employed including 
the Radio Technical Commission for Maritime Services 
Special Committee 104 (RTCMS Cl 04) protocol for 
transmitting GNSS data. 

If properly estimated, RTK-GPS can obtain position 
estimates that are several orders of magnitude more ac¬ 
curate than the multiple meter accuracy associated with 
conventional DGPS. Critical to RTK is estimating the 
number of whole signal cycles that are present in the 
path from the satellite to the station. GPS carrier waves 
have a wavelength of about 19 cm, and this coupled with 
an estimate of the phase of the carrier signal can result in 
pseudo-range estimates with errors in the 1 mm range. 

29.5.5 GPS Receivers and Communications 

GPS receivers are classified according to their perfor¬ 
mance and cost. The best receivers are referred to as 
geodetic grade (or survey grade) with economical mod¬ 
els referred to as resource grade or recreational. In 
general the costs of these different models varies by sev¬ 
eral orders of magnitude, but the gap in performance is 
gradually narrowing. 

Receivers come in two types: code phase and carrier 
phase. Code phase receivers use the satellite naviga¬ 
tion message part of the data stream to provide the 
ephemeris data and produce real-time output. There is 
a delay for them to lock into the satellites, but then they 
produce output continuously without an initial posi¬ 


29.6 GPS-IMU Integration 

It has become commonplace for GPS and IMU tech¬ 
nologies to be integrated into a single GPS-aided in¬ 
ertial navigation system (GPS/INS) to leverage their 
complementary advantages. Although GPS offers the 
promise of high resolution positioning information on 
or about the surface of the planet, it does not solve 
all of the problems associated with robot pose esti¬ 
mation. First, it does not directly obtain information 
about vehicle orientation - to determine the orien¬ 


tion estimate. The C/A signal is a 1023 pseudo-random 
noise (PRN) bit string with a known key. The actual 
pseudo-range data is determined by finding the offset 
of this bit string, and computing the correct shift of this 
bit string is the essential computation that determined 
the delay before a lock is achieved. 

Carrier phase receivers, on the other hand, use the 
phase of the raw GPS signal rather than the embedded 
(digital) C/A signal. The LI and L2 signals have wave¬ 
lengths of 19 and 24 cm respectively, and good quality 
phase measurements allow horizontal positioning accu¬ 
racies on the order of millimeters in principle. These 
measurements, however, only provide relative position 
information within a neighborhood of some tens of kilo¬ 
meters. 

Serial Protocols 

Consumer GPS devices almost universally support 
some variant of the National Marine Electronics Asso¬ 
ciation (NMEA) protocol as well as various proprietary 
protocols specific to each manufacturer. NMEA is a se¬ 
rial protocol sometimes encapsulated within other pro¬ 
tocols. Several variants of the protocol exist, but NMEA 
0183 is the most commonly supported while NMEA 
2000 supports higher data rates and uses a very different 
communication protocol. While the protocol is propri¬ 
etary and the official specification can only be pur¬ 
chased from the NMEA, there are several open-source 
descriptions of the protocol that have been reversed en¬ 
gineered and its basic message characteristics are very 
widely used and distributed. 

The NMEA 0183 protocol supports an ASCII mode 
of communications based on a talker (the GPS re¬ 
ceiver) and one or most listeners (computers) which 
receiver simple protocol strings called sentences. Anec¬ 
dotal evidence suggests that there are ambiguities in the 
protocol that can lead to difficulties is assuring compat¬ 
ibility between devices (this suggestion is necessarily 
made without inspecting the proprietary documenta¬ 
tion, which might preclude its description here). 


tation of the vehicle (yaw, and for many vehicles 
pitch and roll) must be estimated by either differ¬ 
entiating the GPS signal or through integration with 
other sensors including compasses, gyrocompasses, and 
IMU’s. Second, GPS receivers are generally unable 
to provide continuous independent estimates of po¬ 
sition. Estimates are only available at distinct time 
instances with (for inexpensive receivers at least) con¬ 
siderable delays between measurements. A continuous 
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estimate of pose requires estimation of pose between 
GPS readings. Finally, it is not always possible to 
obtain a GPS fix. Local geography (e.g., mountains, 
buildings, trees) or an overhead cover that is opaque 
to radio signals (e.g., indoors, underwater) can block 
the signal entirely. Integration of a GPS receiver with 
other sensor technology (often an IMU) can be used 
to deal with these issues, at least for short periods of 
time. 

The process of integrating GPS and IMU data is typ¬ 
ically expressed as an extended Kalman filter estimation 
process (Sect. 35.2.3). Essentially the IMU data is used 
to bridge between solid GPS measurements and is com- 

29.7 Further Reading 

29.7.1 Odometry 

Many general robotics books including [29.2] 
and [29.23] provide considerable information on 
vehicle odometry and derive odometry equations for 
standard vehicle designs. 

29.7.2 Gyroscopic Systems 

Everett 's book [29.24] provides a review of various 
sensor technologies including gyroscopic systems and 
accelerometers. Interesting historical documentation on 
the gyrocompass and its inventor can be found in 
Hughes’ book [29.5]. 


29.8 Currently Available Hardware 

Although the specific models listed below are likely to 
have a short shelf life, the list of contacts is may be 
a good starting point for the identification of specific 
inertial sensing devices. 

29.8.1 Gyroscopic Systems 

• KVN DSP-3000 Tactical Grade 
Fiber Optic Gyro (FOG), 

KVH Industries Inc., 50 Enterprise Center, 
Meddletown RI, 02842-5279, 

USA, 1-401-847-3327 

• Fiber Optic Gyroscope HOFG-1 (A), 

Corporate Headquarters Hitachi Cable Ltd. 

4-14-1 Sotokanda, Chiyoda-ku, 

Tokyo 101-8971, Japan. 


bined in a least squares optimal sense with the GPS data 
when both are available. Given the complementary na¬ 
ture and true independence of the two sensors, a wide 
range of commercial packages have been developed to 
integrate GPS and IMU data [29.22]. 

When GPS measurements are integrated in a robotic 
vehicle, and in particular when they are combined with 
data from an IMU the orientation of the antenna be¬ 
comes a consideration. The GPS position fix is based 
on the position of the GPS antenna and if it is located 
far from the IMU or the center of the vehicle coordinate 
system then this must be accounted for or instabilities 
in the position or orientation can occur. 


29.7.3 Accelerometers 

Everett’s book [29.24] provides a review of various 
sensor technologies including gyroscopic systems and 
accelerometers. 

29.7.4 GPS 

Considerable details on the theory and implemen¬ 
tation of GPS systems can be found in Leick’s 
book [29.25]. See also [29.26]. Details of various 
approaches to GPS/INS integration can be found 
in [29.27] and [29.28] 


• Rate Gyroscope CRS03, 

Silicon Sensing Systems Japan, Ltd. 

1-10 Fusocho (Sumitomo Precision Complex), 
Amagaxaki, Hyogo 660-0891, Japan. 

29.8.2 Accelerometers 

• Accelerometer GS A 101, 

A-KAST Measurements and Control Ltd., 
1054-2 Centre St. Suite #299, 

Thornhill, ON, L4J 8E5, Canada. 

• ENDEVCO MODEL 22, 

Briiel and Kjaer, 

DK-2850 Naerum, Denmark. 
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29.8.3 IMU Packages 

• yilMU, 

MEMSense, 2693D Commerce Rd., 

Rapid City, SD 57702, USA. 

• IMU400 MEMS Inertial Measurement Unit, 
Crossbow Technology Inc., 

4145 N. First St., San lose, CA 95134, USA. 

• IntertiaCube3, (3DOF IMU), 

Intersense, 

36 Crosby Dr, #15, 

Bedford, MA 01730, USA. 


29.8.4 GPS Components 

• GarminGPS18, 

Garmin International Inc., 

1200 East 151st St., 

Olathe, KS 66062-3426, USA. 

• Magellan Meridian Color, 

Thales Navigation 

471 El Camino Real, 

Santa Clara, CA 95050-4300, USA. 

• TomTom Bluetooth GPS Receiver, 
Rembrandtplein 35, 

1017 CT Amsterdam, 

The Netherlands. 
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30. Sonar Sensing 


Lindsay Kleeman, Roman Kuc 


Sonar or ultrasonic sensing uses the propaga¬ 
tion of acoustic energy at higher frequencies than 
normal hearing to extract information from the 
environment. This chapter presents the funda¬ 
mentals and physics of sonar sensing for object 
localization, landmark measurement and classifi¬ 
cation in robotics applications. The source of sonar 
artifacts is explained and how they can be dealt 
with. Different ultrasonic transducer technologies 
are outlined with their main characteristics high¬ 
lighted. 

Sonar systems are described that range in 
sophistication from low-cost threshold-based 
ranging modules to multitransducer multipulse 
configurations with associated signal process¬ 
ing requirements capable of accurate range and 
bearing measurement, interference rejection, 
motion compensation, and target classification. 
Continuous-transmission frequency-modulated 
(CTFM) systems are introduced and their ability 
to improve target sensitivity in the presence of 
noise is discussed. Various sonar ring designs that 
provide rapid surrounding environmental cover¬ 
age are described in conjunction with mapping 
results. Finally the chapter ends with a discussion 
of biomimetic sonar, which draws inspiration from 
animals such as bats and dolphins. 
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30.1 Sonar Principles 

Sonar is a popular sensor in robotics that employs 
acoustic pulses and their echoes to measure the range 
to an object. Since the sound speed is usually known, 
the object range is proportional to the echo travel time. 
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Fig.30.1a-d Sonar ranging principles: (a) sonar configuration, 
(b) echo waveform, (c) range dot placement, (d) sonar map 
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Fig.30.2a-c False range reading: (a) sonar configuration, (b) prob¬ 
ing pulse 2 transmitted before echo from pulse 1 arrives, (c) false 
range (FR) is measured from transmission time 2 


At ultrasonic frequencies the sonar energy is concen¬ 
trated in a beam, providing directional information in 
addition to range. Its popularity is due to its low cost, 
light weight, low power consumption, and low com¬ 
putational effort, compared to other ranging sensors. 
In some applications, such as in underwater and low- 
visibility environments, sonar is often the only viable 
sensing modality. 

Sonars in robotics have three different, but related, 
purposes: 

1. Obstacle avoidance: the first detected echo is as¬ 
sumed to measure the range to the closest object. 
Robots use this information to plan paths around 
obstacles and to prevent collisions. 

2. Sonar mapping: a collection of echoes acquired by 
performing a rotational scan or from a sonar array 
is used to construct a map of the environment. Sim¬ 
ilar to a radar display, a range dot is placed at the 
detected range along the probing pulse direction. 

3. Object recognition: a sequence of echoes or sonar 
maps is processed to classify echo-producing struc¬ 
tures composed of one or more physical objects. 
When successful, this information is useful for 
robot registration or landmark navigation. 

Figure 30.1 shows a simplified sonar system, from 
its configuration to the resulting sonar map. A sonar 
transducer, T/R, acts as both the transmitter (T) of 
a probing acoustic pulse (P) and the receiver of echoes 
(£). An object O lying within the sonar beam, indicated 
by the shaded region, reflects the probing pulse. A part 
of the reflected signal impinges on the transducer as is 
detected as an echo. The echo travel time ? 0 , commonly 
called the time-of-flight (TOF) is measured from the 
probing pulse transmission time. In this case the echo 
waveform is a replica of the probing pulse, which usu¬ 
ally consists of as many as 16 cycles at the resonant 
frequency of the transducer. The object range r 0 is com¬ 
puted from to using 



where c is the sound speed (343 m/s at standard tem¬ 
perature and pressure). The factor of 2 converts the 
round-trip (P + E ) travel distance to a range measure¬ 
ment. Beam-spreading losses and acoustic absorption 
limit sonar range. 

In forming a sonar map, a range dot is placed along 
the direction corresponding to the transducer’s physi¬ 
cal orientation. A sonar map is usually built by rotating 
the sensor about the vertical axis, indicated by the ori¬ 
entation angle 9, through a series of discrete angles 
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separated by A 8 and placing sonar dots the correspond¬ 
ing ranges. Since the range from the object O to the 
center of T/R is almost constant as T/R rotates, the 
range dots typically fall on a circle as long as O lies 
within the beam. Hence, sonar maps are made up of 
arcs. 

The major limitations of sonar include: 

1. The wide sonar beam causes poor directional res¬ 
olution. Objects are located at the middle of iso¬ 
lated arcs, but shorter-range objects shorten the 
arcs of those at farther ranges, and the arcs pro¬ 
duced by a collection of objects are often difficult 
to interpret. A consequence of this effect is that 
wide beams occlude small openings, limiting robot 
navigation. 

2. The slow sound speed, relative to an optical sensor, 
reduces the sonar sensing rate. A new probing pulse 
should be transmitted after all detectable echoes 
from the previous pulse have expired, otherwise the 
false reading shown in Fig. 30.2 can occur. The echo 
from probing pulse 1 occurs after probing pulse 2 is 
emitted. Sonar measures the TOF from the most re¬ 
cent probing pulse. Many sonars transmit probing 
pulses every 50 ms, but encounter false readings in 
reverberant environments. 

3. Smooth surfaces at oblique incidence do not pro¬ 
duce detectable echoes. Figure 30.3 shows a pla¬ 
nar surface (a wall) that acts as a mirror to the 
sonar beam. The important point is that the nearby 
wall does not itself produce a detectable echo, and 
a robot using sonar for obstacle avoidance may col¬ 
lide with the wall. 

4. Artifacts caused by beam side-lobes and multiple 
reflections produce range readings in the environ¬ 
ment where no objects exist. Figure 30.3 also shows 
the redirected beam enclosing object O. The echo is 
also redirected by the wall back to the transducer. 
From the transducer’s reference, the object is at the 
virtual object location VO, and it would generate the 
same sonar map as shown in Fig. 30.1. Since there 
is no physical object corresponding to the sonar dot 
location, it is an artifact. Also, note that the acous¬ 
tic energy indicated by the dot-dashed line reflected 
back to the transducer is not detected because it 
does not lie within the beam cone. Beam side-lobes 
often detect these echoes and produce shorter-range 
readings but placed along the sonar orientation. 

5. Travel time and amplitude variations in the echoes 
caused by inhomogeneities in the sound speed. Both 
effects cause random fluctuations in the detected 
echo travel time, even in static environments. Fig- 



Fig. 30.3 The smooth surface redirects the beam, causing a sonar 
artifact at virtual object (VO) location. The dot-dashed echo path 
falls outside the sonar beam and does not produce a detectable echo 

a) 



Time 

Fig.30.4a f b Random echo jitter, (a) Sonar configuration; thermal 
inhomogeneities in the acoustic transmission medium cause refrac¬ 
tion effects, (b) Examples of variations in echo travel times and 
amplitudes in a static environment 

ure 30.4 illustrates thermal fluctuations that cause 
speed up, retardation, and travel redirection by re¬ 
fraction of echoes. These cause temporal and am¬ 
plitude variations in the echoes and jitter in the 
range readings. While these typically introduce mi¬ 
nor changes in sonar maps, they often cause havoc 
with approaches using finer analysis. 

This chapter describes the physical and mathemat¬ 
ical details that extend this simplified sonar model to 
practical sonar systems. 
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30.2 Sonar Beam Pattern 


To derive a qualitative description of the sonar trans¬ 
ducer, we apply elementary acoustics theory to a sim¬ 
plified model to achieve a simple analytic form [30.1]. 
A sonar emitter is commonly modeled as a circular pis¬ 
ton surface of radius a vibrating with frequency / in an 
infinite planar baffle. The wavelength A is 



(30.2) 


where c the sound speed in air, 343 m/s at 25 °C [30.2], 
When a > A the emitted pressure field forms a beam 
consisting of a main lobe surrounded by side-lobes. In 
the far field, or range greater than a 2 /A, the beam is 
described by its directivity pattern, which is the two- 
dimensional Fourier transform of the aperture function, 
in this case the circular aperture produces a Bessel func¬ 
tion. The emitted pressure amplitude at range r and 
angle 9 relative to the piston axis can be written as 

„ aa 2 f ( 2J\ (ka sin 9) \ 

P E (r, 9) = -— I , , (30.3) 

r \ ka sm 9 ) 


where a is a proportionality constant that includes the 
density of air and the source strength, k = 2 jt/X, and J\ 
is the Bessel function of the first kind. The term in the 
brackets evaluates to 1 along the sonar axis, 9=0. The 
a 2 term indicates that the emitted pressure increases 
with the piston area. The frequency / appears in the 
numerator because the faster-moving piston generates 
higher pressures. The range r appears in the denomi¬ 
nator because the conservation of energy requires the 
pressure to decrease as the beam widens with range. 

The main lobe is defined by its first off-axis null, 
which occurs at an angle of 


9o = arc sin 



(30.4) 


For example, the popular electrostatic instrument 
grade transducer, formerly produced by Polaroid [30.3], 
had a radius of a = 1.8 cm and is conventionally driven 
at/ = 49.4kHz, making A = 0.7 cm and 9q = 14.7°. 

An object that is small compared to A and located 
in the emitted pressure field produces an echo with 
a spherical wavefront whose amplitude decays with 
the inverse of the distance propagated. In the common 
pulse-echo single transducer ( monostatic ) ranging sen¬ 
sor, only part of the echo wavefront impinges on the 
receiving aperture. The sensitivity pattern of the circu¬ 
lar aperture, now acting as the receiver, has the same 
beam-like Bessel function form given in (30.3) by the 


reciprocity theorem [30.1]. If the reflecting object is lo¬ 
cated at ( r , 9) relative to the transducer, the detected 
echo pressure amplitude, referenced to the receiver out¬ 
put, is given by 


Z fa 4 / 2J\ (ka sin 9) \ 
r 2 \ ka sin 9 ) 


(30.5) 


where / is a proportionality constant that includes pa¬ 
rameters that cannot by controlled in a design, such as 
the density of air. The additional a 2 in the numerator 
occurs because larger apertures detect more of the echo 
wavefront. 

Figure 30.5 shows the echo amplitude from a small 
(point-like) object located in the far field as a function 
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Fig.30.5a,b Normalized amplitude of echo from small ob¬ 
ject predicted by the piston model as a function of angle, 
(a) Linear scale. (b) Decibel scale 
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of angle detected by the electrostatic instrument grade 
transducer. The curve has been normalized by the on- 
axis echo amplitude. 

This model is qualitative in that it provides the fol¬ 
lowing practically useful insights: 

• For a small reflector size relative to the wavelength, 
the echo amplitude decreases inversely with the 
square of the range because there is a 1 /r dispersion 
loss from the transmitter to the object, followed by 
an additional 1 /r dispersion loss in the echo back 
to the receiver. However, larger reflectors can be 
treated using a Huygens principle approach [30.4] 
by dividing them into smaller reflectors and coher¬ 
ently adding their echo contributions. When this 
is done in two dimensions over a normally inci¬ 
dent extended plane reflector, the echo amplitude 
decreases as 1/r rather than 1/r 2 . A cylindrical 
reflector extends over one dimension and results 
in an amplitude variation with range between 1/r 
and l/r 2 . A more-extreme situation can occur with 
a concave reflector that acts as an acoustic magni¬ 
fier, resulting in an amplitude that decreases with 
a negative power of range of less than one. 

• The transducer excited by an approximation to a si¬ 
nusoid exhibits side-lobes due to null caused by 
phase cancellation. For example, the 16-cycle exci¬ 
tation employed in the conventional sonar exhibits 
side-lobes. The peak of the first side-lobe is —35 dB 
relative to the echo amplitude when a small reflec¬ 
tor lies on the transducer axis. The specification 
sheet for the 600 series instrument-grade transducer 
shows the first off-axis null at 15° and a first side- 
lobe peak magnitude of — 26 dB. We presume that 
these measurements were made using a plane as 
a reflector. 

• This model can be used to compute approximate 
beam parameter values for other common transduc¬ 
ers. For example, the SensComp 7000 Series [30.5] 
with a =1.25 cm yields 0 = 20°, equal to the 
specified value. However, the specified first side- 
lobe peak magnitude equals approximately — 16 dB, 
which is substantially different from the expected 
-35 dB. 

The limitations of the qualitative model include: 

• Actual transducers only approximate pistons vibrat¬ 
ing in an infinite planar baffle. The infinite baffle 
directs all the radiated sound pressure into the half¬ 
space in front of the transducer. Actual transducers 
radiate in all directions, but most of the acoustic en¬ 
ergy is concentrated within the main lobe. 


• All pulse-echo ranging sonars operate with finite- 
duration pulses rather than infinite-duration sinu¬ 
soids. Several systems described below use pulses 
that are quite different from a sinusoidal excita¬ 
tion, either in duration or in form. These are com¬ 
monly analyzed by computing the spectrum of the 
pulse and decomposing it into several sinusoidal 
frequencies, each having its own beam pattern. 
For example, the echo amplitude predictions above 
are reasonably accurate, including beam width and 
side-lobes, for the 16-cycle pulses. However, when 
impulse or swept-frequency excitations are used, 
the net beam profile becomes the superposition (of 
linear amplitudes) of the beam patterns produced by 
each frequency component in the excitation. Such 
broadband excitations do not exhibit nulls because 
the nulls formed by one frequency are filled in by 
main and side-lobes of beams produced by other 
frequencies. 

• Most sonar transducers are encased in protective 
housings. The electrostatic instrument-grade trans¬ 
ducer cover forms a mechanical filter that enhances 
the acoustic output at 49.4 kHz. The cases of other 
transducers may distort the transmitted field, but 
most form some type of directional beam. 

• The model does not include frequency-dependent 
acoustic absorption of the transmission medium. 
These reduce the echo amplitudes predicted by the 
model. 

The analytic model above is limited to simple con¬ 
figurations. With current computational power, trans¬ 
ducers can be extended to those with arbitrary, even 
multiple, apertures and with various excitations. Wave¬ 
forms of echoes from objects having arbitrary shapes 
can be simulated by using Huygens principle [30.4]. 
The transmitter, receiver, and object surfaces are broken 
up into two-dimensional (2-D) surface arrays of emit¬ 
ting, reflecting, and detecting elements, using squares 
of dimension < A/5 (the smaller the better, but taking 
longer). The impulse response of a given configuration 
is computed by assuming an impulsive emission and 
superimposing the travel times along all possible paths 
from all transmitter elements to all object elements and 
then to all receiver elements. The temporal resolution 
should be < (20/ max ) , where/ max is the maximum fre¬ 

quency in the excitation. A 1 —s resolution is adequate 
for a 16-cycle 49.4 kHz excitation. A much finer resolu¬ 
tion (< 0.1 —s) is required for an impulsive excitation. 
The echo waveform is then computed as the convolu¬ 
tion of this impulse response with the actual transmitted 
pulse waveform [30.4], 
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30.3 Speed of Sound 


The speed of sound c varies significantly with atmo¬ 
spheric temperature, pressure, and humidity and can be 
critical in determining the accuracy of a sonar system. 
This section outlines the relationship between c and 
these variables and is based on [30.6,7], 

The speed of sound in dry air at sea-level air density 
and one atmosphere pressure is given by 


the following expression can be used 
7c + 273.16 

cp = 20.05 ---ms , (30.8) 

, 1 —3.79 xl0~ 3 

where the saturation pressure of air, p sat is dependent on 
temperature as follows 


c T = 20.05 Vr c + 273.16 ms -1 , (30.6) 

where Tq is the temperature in degrees Celsius. Under 
most conditions (30.6) is accurate to within 1%. How¬ 
ever, should the relative humidity be known, a better 
estimate can be made as 

ch = c T + /?, [ 1.0059 x 10 -3 

+ 1.7776 xl0 _7 (7’c + 17.78) 3 ] ms -1 . 

(30.7) 



= 10.796 
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m 


+ 1.5047xlO -4 ( 1- 10 82927 [Ui) 1 


+ 0.42873 xlO -3 (-1 + lO 4 ' 76961 
-2.2196 


(l-(M 


(30.9) 


Equation (30.7) is accurate to within 0.1% for tempera¬ 
tures in the range —30 to 43 °C for most pressures at sea 
level. Should atmospheric pressure, p s be known then 


p s o is the reference atmospheric pressure of 
101.325 kPa and 7oi is the triple-point isotherm 
temperature with the exact value of 273.16 K. 


30.4 Waveforms 

Sonars employ a variety of waveforms, the most com¬ 
mon types of which are shown in Fig. 30.6. Each 
waveform can be considered the echo from a nor¬ 
mally incident plane. Waveforms are classified as being 
narrow- or wide-band depending on their spectral band¬ 
width. Narrow-band pulses provide superior detection 
performance in the presence of additive noise, while 
wide-band pulses provide better range resolution and 
do not have side-lobes. 

Figure 30.6a shows the waveform produced by the 
Murata 40 kHz piezoelectric transducer excited by an 
eight-cycle 40 kHz square wave 40V rms . The Murata 
sensor is small, lightweight, and efficient, but has an 
approximately 90° beam width. These transducers are 
used in monostatic, bistatic, and multiple transducer ar¬ 
rays [30.8,9]. 

The next three waveforms were produced by the Po¬ 
laroid 600 electrostatic transducer. Similar waveforms 
are generated by the smaller Polaroid 7000 transducer. 
Figure 30.6b shows the waveform produced by the 
6500 ranging module. This ranging module with its 
10 m range, low cost, and simple digital interface is 


a popular choice for implementing sonar arrays and 
rings. While the electrostatic transducer is inherently 
wide-band, with a usable frequency range from 10 to 
120 kHz [30.10], narrow-band pulses are produced by 
exciting the transducer with 16 cycles at 49.4 kHz. 
Figure 30.6c illustrates a means to exploit the wide 
bandwidth of the Polaroid electrostatic transducer by 
exciting it with a decreasing-frequency square wave. 
Such frequency-sweep pulses are processed by a band 
of band-pass filters to extract the frequency dependence 
of reflecting objects. A correlation detector, also know 
as a matched filter, compresses swept-frequency pulses 
to improve range resolution. Longer-duration (100 ms) 
pulses are used in CTFM systems. Figure 30. 6d shows 
a wide-band pulse when the excitation is a 10|xs- 
duration 300 V pulse. The metal protective mesh, which 
also acts as a mechanical filter resonant at 50 kHz, was 
removed by machining to achieve a usable bandwidth 
from 10 to 120 kHz, with the peak occurring at 60 kHz. 
Such wide-band pulses are useful for object classifica¬ 
tion [30.10, 11]. These pulses have small amplitudes, 
limiting their range to 1 m or less. 
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a) b) 

320 |XS 



Fig.30.6a-d Common sonar pulse waveforms, (a) Murata 40kHz transducer (narrow band), (b) Polaroid 600 electro¬ 
static transducer excited with 16-cycle sinusoid at 49.4kHz (6500 ranging module - narrow band), (c) Polaroid 600 
electrostatic transducer excited with decreasing-frequency excitation signal (wide band), (d) Polaroid 600 electrostatic 
transducer excited with 10|is 300 V pulse (wide band) 


30.5 Transducer Technologies 

Electrostatic and piezoelectric transducers are the two 
major types available that operate in air and can in prin¬ 
ciple operate both as a transmitter and receiver - some 
samples are shown in Fig. 30.7. In general electrostatic 
devices have a higher sensitivity and bandwidth but typ¬ 
ically require a bias voltage above 100 V. Piezoelectric 
devices operate at lower voltages, making their elec¬ 
tronic interfacing simpler, but have a high-Q resonant 
ceramic crystal and this results in a narrow frequency 
response compared to electrostatic transducers. 

30.5.1 Electrostatic 

An example of an electrostatic transducer is the Po¬ 
laroid instrument-grade transducer (now available from 
SensComp.com) constructed from a gold-coated plastic 
foil membrane stretched across a round grooved alu¬ 
minium back plate. The conductive foil is charged via 
a bias voltage of 150 V with respect to the back plate. 
Incoming sound waves vibrate the foil and change the 
average distance between the foil and back plate and 
thereby changing the capacitance of the foil. Assum¬ 


ing that the charge q is constant, the voltage v(t ) is 
generated proportional to this varying capacitance C(t) 
as v(t ) = qC(t). As a transmitter, the transducer mem¬ 
brane is vibrated by applying 0—300 V pulses across 
this capacitor, typically using a pulse transformer. The 



Fig. 30.7 Left to right. Series 9000, instrument-grade, and 
Series 7000 transducers; front and back views are shown 
(courtesy Acroname, Inc., Boulder) 
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charge induced by the 300 V on the capacitor causes 
an electrostatic attraction force between the membrane 
and the back plate. The grooves on the back plate allow 
stretching of the membrane and by creating random¬ 
ness in the back plate roughness a broad resonance can 
be achieved in the frequency response. For example 
the bandwidth of the 7000 series Polaroid transducer is 
20 kHz. A front grille is mounted on the transducer and 
removing this grille reduces losses and reverberation 
between the grill and the membrane. Another electro¬ 
static transducer was designed by Kay and details of its 
design can be found in [30.12]. 

30.5.2 Piezoelectric 

Piezoelectric ceramic transducers can be used as both 
transmitters and receivers, however some manufactur¬ 
ers sell transmitters and receivers separately in order to 
optimize the transmitted power and receiver sensitivity, 
respectively. A piezoelectric resonant crystal mechan¬ 
ically vibrates when a voltage is applied across the 
crystal, and in reverse generates a voltage when me¬ 
chanically vibrated. Often a conical concave horn is 
mounted on the crystal to acoustically match the crys¬ 
tal acoustic impedance to that of air. An example is the 
Murata MA40A5R/S receiver and sender transducers, 
which operate at 40 kHz. This device has a diameter of 
16 mm and a 60° beam angle for transmitter combined 
with receiver for — 20 dB loss compared to the maxi¬ 
mum sensitivity. The effective bandwidth of transmitter 
and receiver is only a few kHz due to the resonant na¬ 


30.6 Reflecting Object Models 

Modeling the reflection processes helps in interpreting 
echo information. In this section we consider three sim¬ 
ple reflector models: planes, corners, and edges, shown 
in Fig. 30.8. These models apply to both single trans¬ 
ducers and arrays. 

A plane is a smooth surface that acts as an acous¬ 
tic mirror. Smooth walls and door surfaces act as planar 
reflectors. The plane must be sufficiently wide to pro¬ 
duce the two reflections whose path is shown in dotted 
line. The plane reflector is then slightly larger than the 
intersection area of the beam with a plane of infinite ex¬ 
tent. Smaller planes produce weaker echoes because of 
a smaller reflecting surface and negative interference by 
echoes diffracted from the edges of the plane. An acous¬ 
tic mirror allows the analysis using a virtual transducer, 
indicated by primes in the figure. 

A corner is the concave right-angle intersection of 
two surfaces. Corners formed by intersecting walls, the 


ture of the crystals. This limits the envelope rise time 
of pulses to around 0.5 ms. An advantage is the abil¬ 
ity to drive piezoelectric devices with low voltages, 
for example by connecting each terminal to comple¬ 
mentary CMOS logic outputs. There is a wide range 
of resonant frequencies for piezoelectric transducers 
from 20 kHz to megahertz. Also available is piezoelec¬ 
tric film called polarized fluoropolymer, polyvinylidene 
fluoride (PVDF) from [30.13]. This flexible film can 
be cut to shape and custom ultrasonic transmitters and 
receivers can be formed. The sensitivities of the trans¬ 
mitters and receivers made from PVDF is generally 
lower than that of ceramic crystal transducers and most 
applications are short range where the broadband na¬ 
ture of PVDF allows short pulses to be formed, allowing 
pulse-echo ranging to as little as 30 mm. 

30.5.3 MEMS 

Microelectromechanical system (MEMS) ultrasonic 
transducers can be fabricated on a silicon chip and 
integrated with electronics. The sensors offer a low- 
cost mass-produced alternative to standard transducers. 
MEMS ultrasonic transducers operate as electrostatic 
capacitive transducers where the membrane can be 
made from thin nitride. Devices operate at frequencies 
up to several megahertz and offer advantages in signal- 
to-noise ratio over piezoelectric devices due to their 
better matching to air acoustic impedance [30.14]. Two- 
dimensional arrays of devices can be deployed on a chip 
that are well matched and steerable. 


sides of file cabinets, and door jambs are commonly 
observed corner reflectors in indoor environments. The 
novel feature of the corner, and its three-dimensional 
(3-D) counterpart the corner cube, is that waves reflect 
back in the same direction from which they originate. 
This is caused by planar reflections at each of the two 
surfaces defining the corner. The virtual transducer is 
then obtained by reflecting the transducer about one 
plane of the corner and then the other plane. This gives 
rise to a reflection through the intersection point of the 
corner as shown in Fig. 30.8b. The virtual transducer 
analysis indicates that, for a monostatic sonar, echoes 
from a plane and corner are identical and that planes and 
corners can generate identical sonar maps [30.4]. The 
difference in the virtual transducer orientation between 
planes and corners has been exploited using trans¬ 
ducer arrays to differentiate these reflectors [30.11, 

15 ]. 
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b) 


c) 


Fig.30.8a-c Reflector models: (a) plane, (b) comer, and 
(c) edge 




Fig. 30.9 Random reflector model. Echoes reflect back 
from normally incident surface sections within the beam 


The edge shown in Fig. 30.8c models physical 
objects such as convex corners and high-curvature sur¬ 
faces (posts), where the point of reflection is approx¬ 
imately independent of transducer position. Edges are 
encountered in hallways. While planes and comers gen¬ 
erate strong echoes, edges generate weak echoes that 
are detected only a short range [30.4], making them 
difficult objects to detect. Early robot sonar researchers 
placed bubble wrap material on edge surfaces to make 
them reliably detectable. 

Many environmental objects can be configured as 
a collection of planes, corners, and edges. Models for 
echo production [30.16, 17] indicate that normally in¬ 
cident surface patches and locations at which sharp 
changes in the surface function and its derivatives 
generate echoes. Objects with rough surfaces or a col¬ 
lection of many objects generate echoes from a variety 
of ranges and bearings, as illustrated in Fig. 30.9. If p(t) 
represents a single echo waveform, often a replica of 
the probing waveform, the total echo waveform pj(t) 
is the sum of individual echoes pft) from N normally 
incident patches at range r ; and bearing scaled by 
amplitude a t , or 


N 

Pr(t) = 

z'= 1 



(30.10) 


where 0 ,( 0 ,) is an amplitude factor related to the sur¬ 
face patch size and its bearing in the beam. Wide- 
bandwidth echoes are more complicated because their 
waveform changes in a deterministic fashion due to 
diffraction [30.1 1]. 

Sonars that analyze pj(t) employ analog-to-digital 
converters to obtain waveform samples [30.1 1, 18]. Re¬ 
flecting patches separated in range produce isolated 
patches [30.11], but more often the incremental travel 
time is less than the pulse duration, causing pulse 
overlap. Rough surfaces and volume scatterers, such 
as indoor foliage, have large N, allowing pr(t) to be 
treated as a random process [30.19,20]. Conventional 
TOF sonars output the first time that pj(t) exceeds 
a threshold [30.11], 


30.7 Artifacts 

Sonars usually work well in simple environments, while 
complex environments often produce mysterious read¬ 
ings, artifacts, that foil attempts to build reliable sonar 
maps. Artifacts have given sonar a bad reputation as 
being a noisy, or low-quality, sensing modality. Sonar 
stalwarts believe sonar would open up many new appli¬ 


cations, if only we understood echoes at a level that ap¬ 
proximates that employed by bats and dolphins [30.21]. 
Sonar stalwarts divide into two categories in terms of 
how they treat artifacts. The first attempts to build in¬ 
telligent sensors that identify and suppresses artifacts 
before transmitting data to a higher-level reasoning pro- 
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gram. Previous approaches [30.22,23] required custom 
electronics, which other researchers have been reluctant 
to adopt because of expense or lack of experience. An 
alternate approach is to control conventional sonar in 
a novel fashion to produce a series of spikes and re¬ 
quires only software changes [30.24]. Sonar arrays have 
been used to find consistent data [30.25-27]. Echoes 
from specular reflectors, such as planes, corners, or 
posts, exhibit detectable features, which can be ob¬ 
scured by artifacts. 

The second category of sonar users attempts to 
eliminate artifacts produced by conventional sensors 
by using higher-level postprocessing. These include 
proponents of occupancy (or certainty) grids [30.28, 
29], including those that apply simplified physical 
models, such as sonar arcs [30.30,31]. In simple en¬ 
vironments postprocessing usually eliminates artifacts 
that are inconsistent with a feature [30.32] or with 
a physical map [30.31]. More-sophisticated methods 
handle artifacts by treating them as noise and apply¬ 
ing hidden Markov models (HMM) [30.33]. However, 
multiple passes are needed to successfully teach the 
system about relatively simple environments, mostly 
because artifacts are not amenable to being treated as 
independent additive noise. Eliminating troublesome 
artifacts would replace HMM with simpler Markov 
chains [30.34,35], and sufficient sonar data can be ob¬ 
tained in a single pass. What frustrates this second 
category, and mildly amuses the first, is that this post¬ 
processing works well in simple environments, but fails 
in real-world environments. This second category even¬ 
tually abandons sonar and joins the camera and laser 
ranging crowd. 

There are two important classes of artifacts: axial 
multiple reflection (MR) artifacts and dynamic artifacts. 
These artifacts are important in sonar mapping when 
they indicate the presence of a static object at a lo¬ 
cation where none exists. Troublesome MR artifacts 
are caused by delayed echoes produced by a previ¬ 
ous probing pulse exceeding the detection threshold 
after the current probing pulse has been transmitted. 
Such artifacts then appear as close-range objects and 


30.8 T0F Ranging 

Most conventional sonars employ Polaroid 6500 rang¬ 
ing modules [30.38] connected to the Polaroid 600 
series electrostatic ultrasound transducer. The module is 
controlled with digital signals on two input lines (INIT 
for initialization and probing pulse transmission and 
BLNk for clearing the indication and resetting the de¬ 
tector) and the TOF reading occurs on its output line 


obscure actual farther-range objects in conventional 
sonars. Most sonars employ probing pulse emission pe¬ 
riods longer than 50 ms to avoid MR artifacts, although 
some reverberant environments can still produce arti¬ 
facts [30.36]. 

Dynamic artifacts are produced by moving objects, 
such as individuals passing through the sonar beam. 
Even though these are actual objects and echoes indi¬ 
cate their true range, their presence should not be part of 
a sonar map that describes the static environment. Such 
dynamic artifacts make quantitative matchings between 
stored and generated sonar maps error-prone. 

Another common artifact is a nonaxial MR arti¬ 
fact [30.4] caused by an obliquely incident smooth 
surface that redirects the sonar beam to some other 
echo-producing object. The TOF produces a range read¬ 
ing that is positioned along the sonar axis. While the 
object is not at the location indicated on a sonar map, 
its location in the sonar map is a stable element and can 
be useful for navigation. 

One may argue that, if the locations of all objects are 
known, the echoes can be determined and should not be 
treated as random processes. However, the presence of 
speed fluctuations in the medium due to thermal gra¬ 
dients and ever-present electronic noise cause random 
fluctuations in the times that thresholds are exceeded. 
Even a stationary sonar in a static environment exhibits 
random fluctuations [30.37], similar to the visual expe¬ 
rience of fading when viewing objects beyond a heated 
surface. 

Sonar can identify artifacts by applying three phys¬ 
ical criteria that are met by echoes from static environ¬ 
mental objects. Artifact features include [30.36]: 

1. Echo amplitude - echoes with amplitudes less than 
a specified threshold 

2. Coherence - echoes forming constant-range az¬ 
imuthal intervals less than a specified threshold, and 

3. Coincidence - echoes detected with a sonar array 
at different times (lacking temporal coherence) or 
corresponding to different locations (lacking spatial 
coherence). 


(ECHO). A logic transition on INIT causes the trans¬ 
ducer to emit a pulse lasting for 16 cycles at 49.4 kHz. 
The same transducer detects echoes after a short delay 
to allow transmission transients to decay. Another in¬ 
terrogation pulse is typically emitted only after all the 
echoes produced by the previous pulse have decayed 
below a detection threshold. 
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The module processes echoes by performing recti¬ 
fication and lossy integration. Figure 30.10 illustrates 
a simulation of the processed waveform applied to the 
threshold detector. While the echo arrives at time to 
after the emission, ECHO exhibits a transition at the 
measured TOF time f m , the first time the processed echo 
signal exceeds a detection threshold r. By convention, 
the range r of the reflecting object is calculated by 



where c is the speed of sound in air, usually taken as 
343 m/s. 

Figure 30.10b shows details around the threshold 
detection point including the residual high-frequency 
ripple after full-wave rectification and integration. Two 
effects can be noticed. First, t m will always occur af¬ 
ter to, making threshold detection a biased estimate 
of the true echo arrival time. Moreover, this bias is 
related to the echo amplitude: stronger echoes will pro¬ 
duce an integrator output having a greater slope, which 
exceeds x sooner than t,„. Second, as the echo ampli¬ 
tude decreases, for example, when the object moves 
away from the transducer axis, the threshold level oc¬ 
curs later in the integrator output and t m will experience 
small jumps in time approximately equal to half the pe¬ 
riod [30.39]. 

The first step in developing a model of the detection 
process is to develop a model for the echo amplitude 
as a function of bearing. The Polaroid transducer is 
often modeled as a vibrating piston to yield the trans¬ 
mitter/receiver beam pattern shown in Fig. 30.11. To 
simplify the analysis, the peak of the beam profile is 




Fig.30.10a,b Simulation of Polaroid ranging module op¬ 
eration. (a) Processed echo waveform, (b) Expanded time 
and amplitude scale around threshold crossing point 


approximated with a Gaussian function, a parabola in 
logarithmic units in Fig. 30.11, to determine the echo 
amplitude as a function of object bearing 8, or 


Ag = Aq exp 



(30.12) 


where Ao is the on-axis amplitude and a is a measure of 
the beam width. The value <7 = 5.25° provides a good 
fit around the peak of the beam pattern. The Gaussian 
model is reasonable only over the central section of the 
main lobe that produces detectable echoes. 

We assume that the echo arrival time to does not 
change significantly with transducer orientation (object 
bearing); this effect was investigated and found to be 
minor [30.4]. In contrast, measured TOFs, denoted by 
t m and t' m in Fig. 30.12, are amplitude dependent and 
a function of object bearing, which affects echo ampli¬ 
tude as shown in Fig. 30.11. 

The module processes the detected echo wave¬ 
form by rectification and lossy integration, as discussed 
above. To derive a useful analytic model, assume that 
the integration is lossless and the rectified echo is a unit 
step function with amplitude A. This approximates the 
processed waveform shown in Fig. 30.10b by a lin¬ 
ear function around time t m , shown in Fig. 30.12. The 
model ignores the residual ripple and the decreasing 
slope of the waveform as the lossy rectification ap¬ 
proaches a constant value, shown in Fig. 30.10. The 
linear function with a slope proportional to the echo am¬ 
plitude is given by Ag(t— to), for t > to- This function 


Echo magnitude (dB) 



Bearing (deg) 


Fig. 30.11 Piston model transmitter/receiver pattern for the 
Polaroid 600 series transducer. A Gaussian approxima¬ 
tion with SD = 5.25° is shown in dashed line. Equivalent 
threshold levels for plane, pole, and rod objects at 1.5 m 
range are shown as dot-dashed lines 
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Amplitude 



Fig. 30.12 TOF values t m and t' m for idealized processed 
echo waveforms having two amplitudes. The solid line in¬ 
dicates a larger-amplitude echo 


Fig.30.13a-c TOF data from an object at 1.5 m range; 
mean of 100 measurements with bars indicating ±1SD. 
The dashed lines are model predictions, (a) 1 m wide plane 
(r/A = 0.15 |xs). (b) 8.9cm diam pole (r/A = 0.67 |is). 
(c) 8 mm diam rod (r/A = 2.68 |xs) ► 

exceeds the threshold r at 



For fixed r, the incremental delay in t m is a func¬ 
tion of the bearing 6 and inversely proportional to the 
echo amplitude. When a constant echo amplitude A (in 
volts) is applied to the integrator, the slope of the lin¬ 
ear output is A V/s, with typical values on the order of 
Aq = 10 5 V/s. If r = 0.10 V, x/Aq = 10~ 6 s = lps. 

Experiments were conducted with a Polaroid 600 
series transducer connected to a model 6500 ranging 
module [30.39]. The Polaroid module was operated 
conventionally to generate t m values as a rotational 
scan was performed. Objects include a 1 m-wide plane, 
a 8.9 cm-diameter pole, and an 8mm-diameter rod, all 
located at 1.5 m range. A rotational scan was performed 
from —40° to +40° in 0.3° steps. At each angle, 100 t m 
values were recorded. The mean deviations from the t m 
when the object is on the sonar axis (9 = 0) were de¬ 
termined and the standard deviation (SD) values were 
computed. There were no other objects in proximity to 
the object being scanned. Echoes from objects beyond 
2 m were eliminated by a range gate. 

Figure 30.13a shows the data for the plane, 
Fig. 30.13b for the pole, and Fig. 30.13c for the rod. 
The values are shown relative to the t m value ob¬ 
served at 0° bearing. Dashed lines indicate the values 
predicted by the model. The t m values showed a varia¬ 
tion with SD = 5(is (0.9mm) at zero bearing, which is 
about nine times greater than that predicted by sampling 


a) Time (ps) 




Bearing (deg) 


c) Time (ps) 



Bearing (deg) 


jitter alone. This random time jitter is caused by dy¬ 
namic thermal inhomogeneities in the air transmission 
medium, which change the local sound speed and cause 
refraction [30.37,40]. The SD increases with deviation 
from zero bearing because smaller echoes exceed the 
threshold later in the processed waveform. The smaller 
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slope of the latter part of the processed echo waveform 
shown in Fig. 30.10 causes greater t m differences for 
a given variation in echo amplitude, thus increasing the 
SD. 

One feature in the data not described by the model 
is due to residual ripple in the integrator output, which 
causes jumps in TOF readings equal to half peri¬ 
ods (lOps) added to the value predicted by (30.13). 
These jumps are clearly evident in the mean values of 
Fig. 30.13. 

The angular extent over which echoes were detected 
was 45° for the plane, 22.8° for the pole, and 18.6° for 
the rod. Side-lobes produced by the plane are visible 


and have small echo amplitudes, which cause their t m 
values to be retarded in time. These angular extents can 
be related to the echo amplitudes that would have pro¬ 
duced the respective arcs according to the piston model; 
these are indicated in Fig. 30.11 . For the plane, the 
threshold level relative to the maximum echo amplitude 
is —38 dB, for the pole —25 dB, and for the rod —13 dB. 
Since the ranging module threshold at 1.5 m range is 
the same for each object, the difference in levels indi¬ 
cates the relative echo strength from each object, i. e., 
the plane echo is 13dB (a factor of 4.5) greater than 
the pole echo, and the pole echo is 12 dB (a factor of 4) 
greater than the rod echo. 


30.9 Echo Waveform Coding 

Systems that display echo information beyond the first 
echo have been investigated [30.11, 18,25,41-43], but 
typically employ custom electronics. One motivation 
for examining the entire echo waveform is the suc¬ 
cess of diagnostic medical ultrasound imaging systems, 
which adopt this method [30.44, 45]. 

As a less expensive alternative to analog-to-digital 
conversion, the Polaroid ranging module can detect 
echoes beyond the initial echo by repeatedly resetting 
the detection circuit. The 6500 module specification 
suggests a delay before resetting to prevent the current 
echo from retriggering the detection circuit [30.3]. Let 
us ignore this suggestion and control the Polaroid mod¬ 
ule in nonstandard way to provide information about 
the entire echo waveform. Since the echo amplitude 
is estimated from the digital output produced by the 
Polaroid module, this operation has been called pseudo¬ 
amplitude scan (PAS) sonar [30.24]. 

The conventional ranging module processes de¬ 
tected echoes by performing rectification and forming 
a lossy integration, as illustrated in Fig. 30.14a. 

The BLNK input is typically kept at zero logic 
level, which enables the ECHO output. ECHO exhibits 
a transition at the time when the processed echo sig¬ 
nal exceeds a threshold, as shown in Fig. 30.14b. By 
convention, the time interval between the INIT and 
ECHO transitions indicates the time of flight (TOF), 
from which the range r of the reflecting object is cal¬ 
culated by 


r = 


c x TOF 
2 


(30.14) 


to account for all 16 returning cycles in the echo and to 
allow it to decay below the threshold for the largest ob¬ 
servable echo. The largest echoes typically saturate the 
detection circuit, providing a predetermined maximum 
value. This duration corresponds to the time interval 
over which the processed signal is above the threshold, 
as shown in Fig. 30.14a. 

When an ECHO event is observed, the PAS sys¬ 
tem issues a short 3 ps (corresponding to a software 
query period) pulse on the BLNK input line, which 
clears the ECHO signal as shown in Fig. 30.14c. Upon 
being cleared, the Polaroid module exhibits a delay in¬ 
versely related to the echo amplitude, lasting at least 
140 ps for large-amplitude echoes, and then produces 



Echoes occurring after the initial echo can be de¬ 
tected by resetting ECHO by pulsing the BLNK input. 
The specification suggests that the BLNK pulse should 
be delayed after the ECHO indication by at least 440 ps 


Fig.30.14a-c Polaroid ranging module operation modes, 
(a) Processed echo waveform, (b) ECHO output produced 
in conventional time-of-flight mode, (c) ECHO output pro¬ 
duced in PAS mode 
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another ECHO event if the processed echo signal still 
exceeds the threshold. The PAS system repeatedly is¬ 
sues a BLNK pulse whenever an ECHO event is ob¬ 
served. Hence, a strong echo is represented by three 
pulses on the ECHO line, the first corresponding to the 
conventional TOF, followed by two more pulses. Be¬ 
cause lower amplitude echoes spend less time above the 
threshold, a weaker echo produces two pulses spaced 
farther apart, and a very weak echo may produce only 
one pulse. Shorter INIT pulses discharge the integra¬ 
tor by a lesser amount and thus increase the number 
of pulses produced by the standard Polaroid emis¬ 
sion [30.46,47], 

A PAS sonar map is generated by placing a range 
dot along the transducer axis as a rotational scan is ex¬ 
ecuted. With multiple readings per interrogation pulse, 
a PAS sonar map contains multiple dots at each inter¬ 
rogation angle. Rotational scans then form arcs, with 
isolated arcs indicating weak echoes, arc pairs moder¬ 
ate echoes, and arc triplets large echoes. To illustrate, 
Fig. 30.15 shows arcs formed by a large plane (2.3 m 
width by 0.6 m height) and five cylinders with different 
diameters, all placed at 1 m range. Examining objects at 
the same range eliminates effects caused by the range- 
dependent gain of the module. 

A conventional TOF sonar map by comparison 
would display only the nearest arc in the PAS map for 
each object. Qualitatively, the Eire length increases and 
the number of Eircs increase with the echo amplitude, 
which is bearing dependent. The strongest reflectors 
produce concave arcs [30.4,48]. This occurs because, 
with echo amplitudes much greater than the thresh¬ 
old, the threshold is exceeded near the beginning of 
the echo, yielding a nearly constant range reading over 
a significant extent over bearing. In contrast, the weak¬ 
est reflectors produce convex Eircs, caused by echoes 
whose amplitudes are comparable to the threshold. As 
the echo amplitude decreases the threshold is exceeded 
at later points along the processed waveform, produc¬ 
ing greater range readings. This effect also appears at 
the edges of the arcs produced by strong reflectors. 


Object 

Plane 

8.9 cm dia post 
2.85 cm dia post 
8 mm dia pole 
1.5 mm dia wire 
0.6 mm dia wire 



Computing the beam pattern of the vibrating pis¬ 
ton model [30.1], which is a reasonable approximation 
to the Polaroid transducer, yields the curve shown in 
Fig. 30.16. 

This figure describes the detected echo magni¬ 
tude normalized to have a maximum of OdB, which 
occurs along the beam axis. The larger echoes are 
much greater than the threshold, such as those pro¬ 
duced by the plane, whose maximum amplitudes can 
be 44 dB relative to the threshold. The —44 dB thresh¬ 
old agrees with the PAS map for a plane: strong echoes 
(three stripes) occur within 10° of normal incidence, 
range readings increase due to echo amplitude re¬ 
duction at ±15.6°, approximating the predicted nulls 
at ±14.7°, and smaller-amplitude echoes from the side- 
lobes are present. Weaker reflectors correspond to larger 
thresholds when their on-axis echoes normalize to 0 dB. 
The beam pattern model explains how arc length varies 
with object reflecting strength. The indicated thresholds 
were found by matching the angular beam width to the 
arc extent. 

It is apparent that PAS maps provide information 
useful for solving the inverse problem, that of de¬ 
termining the identity of the object from the echoes. 
Figure 30.15 shows that PAS maps contain informa¬ 
tion about the echo amplitude. While it is true that the 
conventional TOF sonar maps, represented by the clos¬ 
est arc, can determine the object location from the arc 
center and can infer the echo amplitude from the arc 
extent, it is also true that this information is presented 
in a more robust way in the PAS maps. For this sim¬ 
ple case of isolated objects, the posts can be clearly 
differentiated from the pole, while the corresponding 
conventional TOF arcs are comparable. A tenfold in- 


Echo magnitude (dB) 



Fig. 30.15 PAS sonar maps of six objects located at 1 m 
range. The sonar is located below the objects in the figure 


Fig. 30.16 Transmitter-receiver beam pattern. Dashed 
lines indicate the equivalent threshold level for each object 
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Range (m) 

2 

1.8 
1.6 
1.4 
1.2 
1 

0.8 
0.6 
0.4 
0.2 
0 

Fig. 30.17 PAS sonar map of 2.85 cm-diameter post (p) 
and 8.9 cm-diameter post (P). The transducer is located at 
(0,0). A: Artifact caused by echo originating at transmitter 
T, reflected by p, bouncing off P back to p, and directed 
toward receiver T—»-p—*P—»-p—^R. B: T—^P—^p—^- 
P—^R. C:T—>p—>-P—and T —y P —*• p —>• R 

crease in post diameter yields only a modest increase 
in the conventional TOF arc length, while increasing 
the number of arcs from two to three in the PAS maps. 
Spike data can be processed to produce sonar images 
of the environment that are similar to diagnostic ul¬ 
trasound B-scans [30.49]. l<gtM'iiiR>lHrtl shows a sonar 
B-scan of a tree. 

When examining the entire echo waveform, one 
must account for artifacts that are produced when ob¬ 


B 



T/R T/R 


-10123 

Cross range (m) 


jects interact acoustically. Some artifacts occur after the 
first detected echo, so these are not a problem in TOF 
sonar maps [30.4], but must be addressed in interpreting 
PAS maps. Consider a simple environment consisting 
of two posts: a 2.85 cm-diameter post (p), located at 
r = lm and bearing 12°, and an 8.9cm-diameter post 
(P), at r = 1.3 m and bearing —10°. The corresponding 
PAS map shown in Fig. 30.17 displays the echoes from 
the two objects plus additional echoes that illustrate two 
types of multiple reflection artifacts. 

The first type, indicated by A and B, results when 
only one object is within the transducer beam. An in¬ 
terrogation pulse that is redirected by a reflected object 
must be directed back to the receiver within its beam 
pattern in order to be detected. The paths that do this 
are shown in the figure. The single-arc convex shape 
of A indicates that the echo has a small amplitude. This 
is reasonable since both reflectors are nonplanar, and 
hence weak. 

The second type of artifact (C) shown in Fig. 30.17 
occurs when both objects are within the beam pattern. 
This allows two distinct paths for the echoes to return to 
the receiver, occurring in opposite directions and dou¬ 
bling the artifact amplitude. With both objects lying 
near the beam edges, the echo amplitude is small. Since 
the distance traveled by these echoes is slightly greater 
than the range to the farther object, this artifact shows 
a range slightly beyond the more distant object. The su¬ 
perposition of these two components makes the echo 
from the farther object appear spread out in time. This 
pulse stretching explains why four arcs are observed, 
and at one angle five arcs. If the bearing angle between 
p and P was increased to exceed the beam width, this 
artifact would disappear. 


30.10 Echo Waveform Processing 


In this section pulse-echo sonar that processes sam¬ 
pled digitized receiver waveforms is described. These 
systems offer superior performance over the simple Po¬ 
laroid ranging module systems described above, which 
report the TOF based on a threshold. Echo waveform 
processing does however incur the overhead of more- 
complex electronics and signal processing and is not 
readily available commercially. 

30.10.1 Ranging 

and Wide-Bandwidth Pulses 

It is shown in [30.11,50] that the maximum-likelihood 
estimator (MLE) for the TOF is obtained by maximiz¬ 
ing the correlation cor(r) between the received pulse 
p{t) (containing Gaussian white noise) and the known 


pulse shape shifted by r, rec(t— r) 

, , /'Xt)rec(t—r)df 
cor(r) = , 

y/ fl V(0df/Vc 2 (f)dt 


(30.15) 


where the pulse extends from time a to b. The known 
pulse shape at the receiver depends on the angle of 
transmission and reception with respect to the normals 
of the respective transducers. The pulse shape can be 
obtained by collecting a good signal to noise pulse at 
1 m range at normal incidence to the receiver and trans¬ 
mitter and using elliptical impulse response models to 
obtain template pulses at angles different to normal in¬ 
cidence. Pulse shape also changes with range due to 
the dispersive properties of absorption due losses in air 
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transmission. These can be modeled using an estimate 
of the impulse response due to one meter path through 
air as is done in [30.1 1]. 

The correlation, cor(r) is normalized in (30.15) to 
be between —1 and +1. The correlation at the maxi¬ 
mum thus gives a good indication of the match between 



Fig. 30.18 Bearing (6) calculation for a plane using a transceiver 
T/Rl and a receiver R2. T r is the virtual image of T 



Fig. 30.19 Bearing (d) calculation for a comer using a transceiver 
T/Rl and a receiver R2. T' is the virtual image of T 



Fig. 30.20 Bearing (0) calculation for an edge using a transceiver 
T/Rl and a receiver R2. No virtual image is present since the edge 
radiates from a point source at the edge 


the expected and actual pulse shapes and can be used 
to assess the quality of the TOF estimate. In practice 
(30.15) is used in discrete time form, where the inte¬ 
grals are replaced by sums of products and digital signal 
processors are an ideal implementation since they are 
highly optimized to perform this calculation [30.51, 
52]. Finding the maximum correlation of (30.15) is 
known as template matching. To achieve an arrival time 
estimator with resolution smaller than the discrete time 
sample rate, parabolic interpolation can be used on the 
maximum three correlations [30.11], Of interest is the 
jitter standard deviation ctr in the TOF estimator due to 
receiver noise. From [30. 11, 50] 


Or 




( 30 . 16 ) 


where the summation index k is over the entire receiver 
pulse sampled every T s seconds (1 —s in [30. 11,5 1]), B 
is the bandwidth of the receiver pulse, and < 7 ,, is the stan¬ 
dard deviation of the receiver noise. Equation (30.16) 
shows that broadband high-energy pulses achieve low 
errors in the TOF estimator. In [30.11] this is achieved 
by using a 300 V pulse to excite the transmitter and 
achieve close to the impulse response from the device 
with a pulse shape similar to that shown in Fig. 30. 6d. 


30.10.2 Bearing Estimation 


There are many proposed methods for bearing esti¬ 
mation. A single transducer [30.53] can be used by 
exploiting the dependency of the received pulse shape 
on the angle of reception. This approach works for 
angles within one half of the beam width since the 
pulse shape is symmetric with respect to the transducer 
normal angle. Differences in zero-crossing times ei¬ 
ther side of the maximum amplitude of the pulse are 
used to obtain an accuracy of the order of 1°. Other 
single-receiver techniques rely on repeated measure¬ 
ments from a scan across the scene [30.54, 55] and 
achieve a similar level of accuracy but at much slower 
sensing speed since multiple readings are necessary. 

Other single measurement approaches rely on two 
or more receivers [30.11,12,25]. This gives rise to 
a correspondence problem where data must be associ¬ 
ated between the receivers. The closer the spacing be¬ 
tween receivers, the simpler and more reliable the corre¬ 
spondence procedure. The misconception that bearing 
accuracy improves with larger receiver spacing ignores 
the correlation between measurement errors that can 
arise due to the measurements sharing an overlapping 
space of air in the propagation of the ultrasound. Due 
to the high accuracy of TOF estimation in [30.11] the 
receivers could be spaced as close as physically feasi¬ 
ble (35 mm) and still bearing accuracies lower than any 
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other systems are reported. Standard deviations of bear¬ 
ing errors are reported to be below 0.2° for a plane at 
a range of 4 m within a —10° to +10° beam width. 

There are two common approaches to bearing esti¬ 
mation - interaural amplitude difference (IAD) [30.56] 
and interaural time difference (ITD) [30.11,25,51-53, 
56]. IAD uses two receivers pointing away from each 
other so that an echo has a different amplitude response 
in each receiver’s beam width. In ITD both receivers 
usually point in the same direction and the TOF is mea¬ 
sured on each receiver and triangulation is applied to 
determine the angle of arrival. The bearing calculation is 
dependent on the target type, such as a plane, corner or 
edge; these geometries are analyzed in [30. 1 1]. A simple 
arrangement with a transceiver and receiver is shown in 
Fig. 30.18, where T/Rl is the transceiver and R2 is the 
second receiver, spaced a distance d from each other. 

The virtual image of the transmitter is shown as T\ 
The two TOFs measured on the two receivers are t\ and 
t 2 , and these are used to estimate the bearing angle, 9, to 
the plane which is the angle to the plane normal. Apply¬ 
ing the cosine rule to the triangle R2 R1 T' in Fig. 30.18 
gives 

d 2 + c 2 t\ — c z tx 

cos(90 — 9) = sin# = ----. (30.17) 


When d (30.17) can be approximated by 


sin# 


c(t\ -t 2 ) 
d 


(30.18) 


Note that any common (i. e., correlated) noise in t\ and 
t 2 is removed by the difference in (30. 1 8) and hence the 
correlation in noise components of the TOF cannot be 
overlooked in bearing estimation as described above. 

The situation for a corner is shown in Fig. 30.19 and 
the same result applies as in (30.17). 

For an edge the situation is shown in Fig. 30.20, 
where R1 has a TOF from T to the edge and back to 
Rl, whilst R2 has a TOF from T to the edge and back 
to R2. 

From the geometry, we use the same approach as in 
(30.17) to give 


sin 


2 dc'j 

d 2 + c 2 t 2 (t l -t 2 ) 
dct\ 


(30.19) 


Note that (30.19) can be approximated by (30.18) when 
d <SC ct\. 


30.11 CTFM Sonar 

The continuous-transmission frequency-modulated 
(CTFM) sonar differs from the more common pulse- 
echo sonar discussed in previous sections in the 
transmission coding and the processing required to 
extract information from the receiver signal. 

30.11.1 CTFM Transmission Coding 

The CTFM transmitter continuously emits a varying- 
frequency signal, usually based on a sawtooth pattern 
as shown in Fig. 30.21, where the frequency is often 
swept through an octave every sweep cycle T. 

The transmitted signal with a linearly changing fre¬ 
quency can be expressed as 

S(t) = cos [2jr (fiit — (if 2 )] (30.20) 

for 0 < t < T. The sweep cycle is repeated every T sec¬ 
onds as shown in Fig. 30.21. Frequency is 1/2;r times 
the time derivative of the phase in (30.20). Note that 
the highest frequency is fu and the lowest transmitted 
frequency is /h — 2bT, where b is a constant that de¬ 
termines the sweep rate. We can then define the swept 


frequency AF as 

AF = 2bT . (30.21) 

30.11.2 CTFM TOF Estimation 

Echoes are generated when the transmitted wavefront 
encounters reflectors and are an attenuated, delayed ver¬ 
sion of the transmitted signal 

( 2F\ 

E(t)=ASlt- — j , (30.22) 

where R is the range to the reflector, c is the speed of 
sound and A is the amplitude that may in the case of 
curved objects depend on the frequency of the sound at 
reflection. 

The TOF is estimated by the two step process of 
demodulation and spectral analysis. Demodulation is 
achieved by multiplying the received signal by a copy 
of the transmitted signal and low pass filtering. This can 
best be understood in the simple case of one echo. The 
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signal D(t ) is obtained using (30.20) and (30.22) 


D(t) = E(t)S(t ) 


for fe 


fu 


-[cos (2nf e t — <p) 

+ cos (2nf u t — 2br — 0)] 
4 Rb 


c 



2 R 4 bR 2 

4> — fu -1- 

c & 


(30.23) 


where the following trigonometric identity has been 
used in (30.23) 


cos(x) cos (y) 


y[cos(jc — y) + cos(jf + y)] . (30.24) 


A low-pass filter removes frequency components above 
fu and this results in the base-band signal 


D h (t) 


A 

2 


cos 




(30.25) 


which has a frequency proportional to the range R. The 
ranges of echoes can be extracted by examining the 
spectrum of D\, using, for example, a discrete Fourier 
transform (DFT) or the fast Fourier transform (FFT). 
From (30.25) for a frequency peak of/ r FIz the corre¬ 
sponding range R is given by 


current sweep as assumed in the analysis above. The 
sweep time T needs to be much larger than this blind 
time for the sonar to operate effectively. The blind 
time can be eliminated at the expense of introducing 
complexity in the demodulation process as described 
in [30.57], where an interlaced double demodulation 
scheme is described. 

30.11.3 CTFM Range Discrimination 
and Resolution 


We define range discrimination as the separation in 
range of two targets that can be simultaneously detected 
as distinct. The range resolution is defined as the small¬ 
est increment in range that can be measured by the 
sonar. 

Suppose that, in order to extract ranges of tar¬ 
gets, T>b(0 from (30.25) is sampled at AT intervals 
and k samples are collected before a DFT (or FFT) 
is performed. The frequency samples of the DFT will 
be A f= \/(kAT) apart. From (30.26), this represents 
a range resolution A R of 


cAf _ c 
4b ~ 4bkAT ' 


(30.27) 


We can relate this to the swept frequency A F from 
(30.21) as 


A R = 


c T 
2A F X kAT ’ 


(30.28) 


R =./ r — . (30.26) 

Note that the above analysis relies on excluding the 
receiver waveform at the start of each sweep for a blind 
time (Fig. 30.21) of R m /2c, where R m is the maximum 
target range. During this blind time the receiver signal 
is dependent on the previous sweep rather than the 


Frequency 



Fig. 30.21 18 Plot of frequency versus time for CTFM. 
The blind time applies if the shown echo corresponds to 
a maximum range target at R m 


where the second term is the ratio of the sweep time to 
the spectral sample time. In order to discriminate two 
peaks in the DFT, they must be at least two samples 
apart and hence 

c T 

range discrimination = -x-. (30.29) 

S A F kAT 

Note that (30.28),(30.29)) show that, subject to 
signal-to-noise constraints, CTFM can lengthen the 
data integration time kAT in order to improve the 
range discrimination and resolution of the sonar. Also 
it is possible, subject to signal noise, to use interpo¬ 
lation techniques (e.g., parabolic interpolation) on the 
DFT peaks to resolve to a smaller than A f frequency 
and hence improve range resolution (but not range 
discrimination). 

30.11.4 Comparison of CTFM 
and Pulse-Echo Sonar 

• The range resolution of pulse-echo sonar and 
CTFM sonar is theoretically the same given the 
same signal-to-noise ratios and bandwidths [30.57]. 
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The range discrimination in pulse-echo sonar is lim¬ 
ited by the pulse length, where shorter pulse lengths 
require higher bandwidth. However in CTFM, 
range discrimination can be improved by increas¬ 
ing the data integration time, allowing more design 
flexibility. 

• CTFM also allows for the energy of the transmit¬ 
ted signal to be spread evenly over time, resulting 
in lower peak acoustic power emission compared to 
pulse-echo systems with the same receiver signal- 
to-noise ratio. CTFM can provide a greater aver¬ 
age power in a practical context and consequently 
a greater sensitivity to weak reflectors is possible. 

• CTFM requires more-complex transmitter circuitry 
and the requirement for FFT processing on the re¬ 
ceiver side. 

• Separate transmitter and receiver transducers are 
necessary with CTFM, whilst pulse-echo systems 
can use a single transducer for both transmission 
and reception, resulting in restriction on the mini¬ 
mum range of pulse-echo sonar due to the blanking 
of the receiver during transmission. CTFM has no 
inherent restriction on minimum range. 

• CTFM sonar can continuously derive range infor¬ 
mation from targets every kAT seconds at a delay of 
R/c + kAT compared to every 2R m /c with a delay 
of 2 R/c in pulse-echo sonar (ignoring processing 
delays in both), which may be important in real¬ 
time tracking applications. 

• Other benefits of CTFM are that the number of 
range measurements per cycle is limited only by the 
range discrimination constraint of (30.28) and the 
signal-to-noise ratio. 

• In terms of bearing estimation and classification of 
targets from a moving platform, short pulse-echo 
sonar systems like [30.27,51] do not suffer from 
the CTFM data integration time required to esti¬ 
mate accurately the frequencies corresponding to 
ranges (and hence bearing). During the data inte¬ 
gration time, the target can move with respect to the 
sensor and blur the measurements, making bearing 
estimation and classification less accurate. In short 
pulse-echo systems, the target is effectively sampled 
with a pulse of less than 100 |xs, resulting in a con¬ 
sistent snapshot of the target. 

30.11.5 Applications of CTFM 

Kay [30.58, 59] developed a mobility aid for blind peo¬ 
ple using a CTFM sonar system based on a sweep 
of /h = 100 down to 50 kHz with a sweep period of 
T = 102.4 ms. After demodulation, ranges are heard 
as audible tones with frequencies up to 5 kHz cor¬ 
responding to ranges of up to 1.75 m. The system 


uses one transmitter and three receivers as shown in 
Fig. 30.22. 

Users of the system can listen to the demodulated 
signal in stereo headphones corresponding to the left 
and right receivers, each mixed with the large central 
oval receiver. Higher frequencies correspond to more- 
distant ranges. To illustrate the sensitivity, a 1.5 mm- 
diameter wire is easily detectable at 1 m range - the echo 
produced is 35 dB above the noise floor in the system. 

CTFM sonar has been used to recognize isolated 
plants [30.41,60]. The advantage gained from CTFM 
is that extensive range and echo amplitude information 
is obtained from the whole plant given the spectrum 
of the demodulated received signal, and these echoes 
are obtained from an excitation across an octave of 
frequencies from 100 down to 50 kHz with a high 
signal-to-noise ratio that allows weak reflections from 
leaves to be sensed. This information is called the 
acoustic density profile and 19 different features are 
found to be useful in classifying the plants, such as the 
number of range cells above a threshold in amplitude, 
the sum of all range cells, the variation about the cen¬ 
troid, the distance from the first to the highest amplitude 
cell, and the range over which reflections are detected. 
With a population of 100 plants, an average of 90.6% 
correct pairwise classification was obtained using a sta¬ 
tistical classifier. 



Fig. 30.22 Aid for blind people - the small oval trans¬ 
ducer is the transmitter and the other three components 
are receivers. The large oval receiver provides high reso¬ 
lution, enabling fixation by users’ fine neck control (after 
Kay [30.58]) 
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Scanning CTFM with a singe transmitter and sin¬ 
gle receiver has been successfully applied to mapping 
of indoor environments that include smooth and rough 
surfaces [30.55] with bearing errors on the order of 
0.5° for smooth surfaces and higher for edges. The 
classification uses amplitude information that is nor¬ 
malized with range using a fixed attenuation constant of 
sound. In practice this attenuation constant varies with 
temperature and humidity and needs to be calibrated 
before each experiment for consistent results. Greater 
robustness, speed, and accuracy has been demonstrated 
with TOF methods of classification that require at least 
two transmitter positions and two receivers as described 
in [30. 11,51]. CTFM could be applied to array systems 
to achieve higher sensitivity to weak targets than the ex¬ 
isting pulse-echo systems. 


30.12 Multipulse Sonar 

This section examines sonar systems that employ more 
than one pulse in the transmitter(s). The main motiva¬ 
tions are interference rejection and on-the-fly classifi¬ 
cation. Multipulse sonar has also been used to generate 
a better signal-to-noise ratio by creating longer trans¬ 
mitted pulse sequence using Barker codes [30.61]. The 
autocorrelation of a Barker code gives a narrow peak 
with low autocorrelation away from the central lobe. 
The matched filter then gives rise to pulse compression 
that averages noise over a longer time period. 

30.12.1 Interference Rejection 

External acoustic noise, such as compressed air, is 
a source of sonar interference. Sonar systems attempt 
to reduce the effects of external interference by filter¬ 
ing the signal and the optimal filter is the matched filter 
where the impulse response is the time reversal of the 
pulse shape that is expected. Since a time-reversed con¬ 
volution is a correlation, the matched filter then acts as 
a correlation with the expected pulse shape as discussed 
in Sect. 30.10. Approximations to matched filtering can 
be designed based on a bandpass filter with a frequency 
response that is similar to the spectrum of the expected 
receiver pulse. CTFM systems allow robust suppression 
of external interference by employing a matched filter 
across a broad range of frequencies contained in the 
continuous chirp transmission. 

When more than one sonar system operates in 
the same environment, the transmitted signal from 
one sonar system can be received by another, causing 
crosstalk errors. This is particularly evident in classical 


CTFM has been employed in three binaural sys¬ 
tems [30.12] where a rigorous theoretical and experi¬ 
mental comparison of these ultrasonic sensing systems 
based on different range and bearing estimators is made. 
Stanley [30.12] also contains detailed engineering de¬ 
sign information of CTFM sonar systems. The conclu¬ 
sion is that CTFM can insonify large areas due to its 
higher average power transmissions and consequently 
good signal to noise performance. The use of autore¬ 
gressive estimators for spectral lines in the demodu¬ 
lated signal were found to provide better resolution 
than the DFT. The interaural distance and power differ¬ 
ence CTFM approaches provided state-of-the-art per¬ 
formance except that the pulse-echo approach in [30. 1 1] 
using a high-energy short pulse was found to be a factor 
of six to eight times superior in bearing precision. 


sonar rings constructed from Polaroid ranging modules. 
Error-eliminating rapid ultrasonic firing strategies have 
been developed [30.62] and are claimed to remove most 
of this interference and allow faster operation of these 
sonar rings. 

More-sophisticate coding of transmitted pulse(s) 
has been employed [30.23, 63-66] to allow rejection 
of external interference and crosstalk. One difficulty 
with multiple transmitted pulses over a greater time 
period than a single pulse is that target clutter can pro¬ 
duce many overlapping pulses at the receivers that are 
difficult to unravel and interpret, and the sonar range 
discrimination can be compromised. 

30.12.2 0n-the-Fly Target Classification 

Target classification into planes, cylinders, and edges 
has been achieved by deploying a single transmitter 
and three receivers [30.25] using a single measurement 
cycle. At least two transmitters are required to differen¬ 
tiate planes from concave right-angled corners [30.11] 
where a two transmitter arrangement is used to classify 
targets into planes, corners, and edges in two successive 
measurement cycles. The method of classification can 
be understood with virtual images and mirrors, since 
specular sonar reflections occur. Looking into a plane 
mirror gives an image that is left-right reversed com¬ 
pared to looking into a right-angled mirror. An edge is 
analogous to observing a high-curvature specular sur¬ 
face, such as a polished chair leg, where the whole 
image is compressed to a point. Sonar classification ex¬ 
ploits the difference in bearing angles to a target from 
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two transmitters to classify as follows: a positive dif¬ 
ference S indicates a plane, a negative S value indicates 
a corner, and zero difference an edge, where the angle S 
depends on the sensor geometry and target range. More 
sophistication can be added by using range measure¬ 
ments in addition to bearing, with maximum-likelihood 
estimation. 

This arrangement [30.11] was refined to work with 
just one measurement cycle of around 35 ms to 5 m 
range: hence the term on-the-fl y in [30.51]. This on- 
the-fly approach uses pulses fired at a precise time 
difference A T and 40 mm apart from two transmitters 


30.13 Sonar Rings and Arrays 

Since sonar only detects objects lying within its beam, 
a common means to scan the entire environment outside 
the robot is to use an array of sonars, or a ring [30.68]. 

30.13.1 Simple Ranging Module Rings 

The most common is the Denning ring that contains 24 
sonars equally spaced around the robot periphery. This 
15° spacing allows some overlap in the sonar beams so 
at least one of the sonars will detect a strong reflecting 
object. The sonars in the ring are typically employed 
sequentially one at a time. Using a 50 ms probing pulse 
period to reduce false readings a complete environmen¬ 
tal scan is accomplished every 1.2 s. This sample time is 
adequate for a translate-and-stop operation in research 
settings, but may be too slow for a continually moving 
robot. A robot moving at 1 m/s may not detect an object 
with sufficient warning to prevent a collision. Some re¬ 
searchers propose simultaneously employing sonars on 
opposite ends of the ring to speed up acquisition times, 
while others also reduce the probing pulse period and 
attempt to identify artifacts. 

30.13.2 Advanced Rings 

Yata et al. [30.53] developed a 32 cm-diameter sonar 
ring with 30 transmitters and 30 receivers placed al¬ 
ternately. Murata piezoelectric MA40S4R wide-angle 
transducers are used to enable overlapping reception 
of echoes produced by firing all the transmitters si¬ 
multaneously. An axial symmetrical exponential horn 
structure is used to narrow the beam shape of the trans¬ 
mitters vertically to avoid reflections from the floor. 
Received signals are compared with a decaying thresh¬ 
old to produce a 1 bit digitized sampled signal without 
rectification. Bearing is estimated from the leading edge 


with two further receivers completing a square. AT 
is usually around 200 (is but can vary randomly from 
cycle to cycle to achieve interference rejection (both 
crosstalk and environmental) with identical sonar sys¬ 
tems. Classification is performed simultaneously in one 
measurement cycle. The sensor achieves high accu¬ 
racy in range and bearing with robust classification 
by exploiting the tight correlation between TOF jit¬ 
ter in the different transmitter to receiver paths due 
to the close temporal and spatial arrangement. The 
sensor has been deployed for large-scale mapping 
in [30.67], 


of echoes and an error standard deviation of 0.4° is re¬ 
ported for ranges up to 1.5 m. 

A sonar ring with seven digital signal processors 
(DSPs) [30.52, 69, 70] that uses 24 pairs of 7000 se¬ 
ries Polaroid transducers consisting of a transceiver 
and receiver has been developed (Fig. 30.23). Each 
pair can derive range and accurate bearing informa¬ 
tion using template-matching digital signal processing 
(Sect. 30.10) on each of the two receiver channels, 
which are sampled at 250 kHz with 12 bit analog to 
digital converters. In total eight receiver channels are 
processed per DSP. All transceivers are fired simul¬ 
taneous to enable full surrounding sensing of the en¬ 
vironment approximately 11 times a second to a 6m 
range with experimentally validated range and bear¬ 
ing accuracies to smooth targets of 0.6 mm and 0.2°, 
respectively. To suppress interference between neigh- 



Fig. 30.23 DSP sonar ring hardware 
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boring pairs, two different transmitted pulse shapes 
are employed in an interleaved fashion around the 
perimeter of the ring. The pulse shapes are derived 
from two and three cycles of 65 kHz excitation. The 



Fig.30.24a-c DSP sonar ring mapping an indoor environ¬ 
ment (a), raw data (b) and SLAM feature map with feature 
number and number of associations shown as numbers (c). 
The sonar ring is moving at lOcm/s with a 11.5 Hz sam¬ 
pling rate 


DSP sonar ring allows rapid and accurate wall follow¬ 
ing, map building, and obstacle avoidance due to the 
high repetition and accurate range and bearing sens¬ 
ing. The beam width of the transducer pairs allows 
full 360° coverage with respect to smooth specular 
targets to a range of 3 m. An example of the DSP 
sonar ring producing a feature for a simultaneous lo¬ 
calization and mapping (SLAM) map is shown in 
Fig. 30.24. 

30.13.3 Sonar Rings 

with FPGA Hardware Processing 

One of the limitations of DSP based implementation of 
the signal processing arrangments such as [30.52, 69, 
70] is that insufficient processing is available to per¬ 
form matched filtering on the complete receiver echo 
signals in a multi-channel sonar ring. That is selected 
segments of the echo are processed where the signal 
exceeds a threshold in order to limit the computational 
load on the DSP processors. Once the full echo has 
arrived, matched filtering is then applied to these seg¬ 
ments. More recent work [30.71,72] describes a sonar 
ring that processes the complete echo signals from 48 
receivers as the signal arrives with matched filtering. 
Complete processing of the entire receiver signal al¬ 
lows weak echoes to be detected that would be below 
the threshold of segmentation in [30.52, 69, 70]. This is 
achieved using a single field programmable gate array 
(FPGA) configured with a custom data path hardware 
design performing 4.9 Giga-arithmetic operations per 



Fig. 30.25 FPGA based sonar ring 
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second. shows DSP sonar tracking an ob¬ 

ject. Range and bearing measurements are obtained at 
a rate of 30 Hz to 4 m range with full surrounding 360° 
angular coverage. 

A photo of the FPGA sonar ring is shown in 
Fig. 30.25. Note that two layers of receivers are em¬ 
ployed to reduce the radius of the 48 receiver ring. The 
other notable feature is the use of a single transmitter 
at the center of the ring. The transmitter is a hemi¬ 
spheric tweeter ESTD01 from Murata and a conical 
parabolic reflector to distribute the transmitter pulse 
uniformly in all directions horizontally [30.73]. Two cy¬ 
cles of a 48 kHz sine wave form the transmitter pulse. 
The transmitter can be modelled as point source and 
the pulse shape is approximately invariant with respect 
to received angle for each pair of receivers. This sim¬ 
plifies the matched filtering since the template for each 
receiver pair does not depend on the receiver angle. 
However there is some variation of pulse shape be¬ 
tween different pairs in practice and as a function of 
range. The template match correlation processing hard¬ 
ware has been designed to allow dynamic switching of 
the template at predetermined ranges to account for the 
range variation of the pulse shape. 

30.13.4 Double Refresh Rate Sonar Ring 

Sonar rings have been limited in their firing rate by 
waiting for maximum range echoes to return before 
a consecutive firing of the transmitter. For example, 
the FPGA sonar ring presented in [30.71] is limited to 
30 Hz refresh rate by this constraint. A double refresh 
rate FPGA sonar ring is presented in [30.74] that op¬ 
erates at 60 Hz for a 5.7 m range. To achieve a double 
refresh rate, the next sonar transmission is scheduled 
by selecting a transmit time from a random set so as 
to minimize interference with predicted echo arrival 


30.14 Motion Effects 

When a sensor moves with respect to its targets, sonar 
measurements are effected. For example, a sonar sen¬ 
sor moving at a speed of 1% of the speed of sound 
(around 3.4m/s) will experience errors of the order 
of 0.6° for some bearing measurements. The effects 
of linear velocity on the TOF and reception angle are 
dependent on the target type and hence for motion com¬ 
pensation to be meaningful a target classification sensor 
is needed. We consider the classical plane, edge, and 
comer target types in this section. Rotational motion 
effects are discussed in [30.27] where it is shown that 
very high speeds of rotation are necessary to give rise to 


times. Two matched filters are applied to the receiver 
signal based on the two possible transmit times. Echoes 
are associated with a transmit epoch with the following 
criteria: 

• The arrival time must be consistent with previous 
echoes since this is bounded by the sensor to obsta¬ 
cle relative speed 

• The pulse energies should be within 50% of a pre¬ 
vious echo from the same obstacle, and finally 

• The template match correlation must be acceptable 
for the assumed range. 

30.13.5 Sparse 3-D Arrays 

The work by Steckel et al. [30.75] presents a sonar sys¬ 
tem that borrows beamforming theory from the design 
of antenna arrays. The sonar array is based on a sparse 
random array of 32 condensor omnidirectional ultra¬ 
sonic microphones and a single Polaroid 7000 acting 
as a transmitter. The microphone signals are sampled at 
a maximum rate of 500 kHz and processed by an FPGA 
to produce a measurement repetition rate of 12 Hz. The 
transmitter waveform is a carefully controlled hyper¬ 
bolic chirp of 100 down to 20 kHz in 3 ms. Matched 
filtering and beamforming processing are applied to the 
received signals to generate an energy scape - the distri¬ 
bution of energy from reflectors from discrete directions 
of around 1° resolution. Experiments show the sys¬ 
tem can discriminate 20 mm diameter poles at 800 mm 
with 5° separation. The minimum angular separation is 
related to the width of the mainlobe of the spatial ar¬ 
ray filters. Azimuth and elevation standard deviations 
of errors are reported as 1.1 and 0.6° respectively for 
a 80 mm sphere at 1.5 m. The advantage of the system 
is its ability to discriminate overlapping echoes in clut¬ 
tered environments. 


a small bearing error (e.g., 0.1° error for approximately 
1700 deg/s). Narrowing of the effective beam width is 
another effect of high rotation speeds of a sonar sensor. 

The sensor is assumed to transmit from a point la¬ 
beled T and receiver measurements are referenced to 
this position on the sensor. However, due to the motion 
of the sensor, the ground referenced position R at the 
time of reception of the echo moves from T over the 
course of the TOF. For a linear velocity, the distance 
between T and R is TOFxi;, where v is the magni¬ 
tude of the sensor velocity vector relative to the ground, 
with components v x and Vy parallel to their respective 
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Moving 

observer 





Fig.30.27a-c Observing a target from a moving sensor. T is the 
position of the transmitter and R is where the echo is received at the 
end of the TOF. The target is a plane in (a), a corner in (b). and an 
edge in (c) 


Fig. 30.26 Observation of arriving wave from a moving 
observer < 


coordinate axes. The expressions derived for linear mo¬ 
tion apply to any sonar sensor, since only the physics of 
sound propagation and reflection are used. All targets 
are assumed to be stationary. 

This section is based on [30.27], where further ex¬ 
perimental work not included here can be found. 

30.14.1 Moving Observation of a Plane 


A plane target reflects the transmission from position T 
to R as shown in Fig. 30.26a. The TOF is broken up into 
two parts: fi is the time of propagation to the plane and 
ti from the plane to the receiver R. Here we derive the 
effect of linear motion on the TOF = 0 + ^2 and the an¬ 
gle of reception 6 all taken from the view of a stationary 
observer. A moving observer is discussed below. 

From the right-angle triangle on the left of 
Fig. 30.27 a, we have 


sin 9 = — and cos 6 = 
c 



and also 


di_ 

he 


di 

c cos 6 


(30.30) 


(30.31) 


From the right-angled triangle on the right of 
Fig. 30.27 a, we have 


h — 


{h + t2)Vy + d\ 

he 

{t\ + h)Vy + di 
c cos 0 


(30.32) 


The TOF is obtained by adding (30.31) and (30.32) and 
then substituting (30.31) giving 



(30.33) 


The first factor in (30.33) represents the stationary TOF. 
The second factor approaches unity as the velocity ap¬ 
proaches zero. 


30.14.2 Moving Observation of a Corner 


Figure 30.27 b shows the situation for a comer with the 
virtual image of T is shown as T'. From the right-angled 
triangle T'XR 

c 2 TOF 2 = (2c?! + tyTOF) 2 + t^TOF 2 , (30.34) 
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which gives 


2d\ 

TOF = — 
c 


Vi-(t) 2 


-<^) 2 +? x 


v 


i-(f) 


(30.35) 


/ 


where v 2 = v 2 + v 2 . The left-hand term of (30.35) is 
the stationary TOF and the right-hand term approaches 
unity for small velocities. The angle </> in Fig. 30.27 b is 
the angle deviation due to motion as reference by a sta¬ 
tionary observer. From the triangles T'XR and CXR 


tan# = 


u-TOF 


tan(# + <p) = 


2d 1 + lyTOF 
vyTOF 


and 


d\ + tyTOF 
From (30.36), we have 


+ t^TOF 


and solving for tan <p yields 


tan (p = tan f 


1 - sin 2 


u v TOF , , , .2 

-b 1 + sin” I 


tof ^ 


1 - sin 2 


i; v TOF . ! . • 2 , 

^ + 1 + stn“ ( 


d 1 


,~ v x 

4 >« — • 
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30.14.4 The Effect of a Moving 

Observation on the Angle 
of Reception 

The expressions for the reception angle in the previous 
sections are based on an observer that is stationary with 
respect to the propagating medium air. In practice the 
observer is the sensor, and is moving with a velocity v. 
Suppose that the sonar wave arrives at an angle a rela¬ 
tive to the air, as shown in Fig. 30.26. 

The velocity components of the wavefront relative 
to the observer, w x and Wy, are as follows 


= c sin a — v x and Wy = c cos a — Vy . 


(30.41) 


(30 36) F rom (30.41) the observed angle of arrival, /3 is 


tan/i = 


c sm a — v x 


-Vy 


sm a —- 


cos a —- 


(30.42) 


(30.37) 


(30.38) 


For v x , v y « c, sin (9 <<C 1 and 2d\ /TOF rs c we can ap¬ 
proximate (30.38) as 


30.14.5 Plane, Corner, and Edge 

Moving Observation Arrival Angles 

In this section the arrival angles (in radians) for each tar¬ 
get type are summarized and approximated for speeds 
expected of a mobile robot. The speed is assumed 
to be less than a few percent of the speed of sound 
(typically 340 m/s at room temperature). These effects 
have been observed experimentally at speeds of up to 
1 m/s [30.27], 

Equations (30.41) and (30.30) cancel exactly, and 
for a plane the arrival angle relative to the sensor is ex¬ 
actly zero 


/"plane — 0 . 


(30.43) 


(30.39) 


30.14.3 Moving Observation of a Edge 

Since an edge reradiates the incoming ultrasound from 
an effective point source, the reception angle with re¬ 
spect to a stationary observer is unaffected by motion 
as shown in Fig. 30.27 c. The TOF is affected due to 
the motion moving the receiving position. From the 
right-angled triangle XER, d\ = ( d\ + Vy ) 2 + v 2 TOF 2 
and d\ + d 2 = cTOF leads to 


This can be explained by noting that the forward wave 
velocity component is always the same as the sensor’s 
as reflection preserves this component. 

For a corner the angle </> results in a wavefront that 
appears to be displaced in the same direction as the sen¬ 
sor motion from the real comer direction, as can be seen 
in Fig. 30.27 b. The effect of the moving observer dou¬ 
bles this effect as seen by (30.41) and (30.39) 


21U 


(30.44) 


TOF = — [ ——jL 
C \l-^ 


2dx 

c 


(l-F-^), (30.40) 


where the approximate holds in (30.40) for d«c. 


For an edge the result is due to the observer only 

0-Fr 


: tan 


cos a —- 


Vx 

c 


(30.45) 
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30.15 Biomimetic Sonars 

The success of biosonars, bats and dolphins [30.76], 
has led researchers to implement sonars based on 
biosonar morphology, strategy, nonlinear processing 
and cleverness [30.77]. The capabilities exhibited by 
biosonars have caused researchers to examine biomim¬ 
icking (biomimetic) systems. 

Biosonar morphology typically has a single 
transmitter and a pair of receivers. Bats transmit 
sound pulses through the mouth or nose, while 
dolphins transmit through a melon. The two re¬ 
ceivers correspond to ears that permit binaural pro¬ 
cessing. Mimicking binaural hearing has led to 
small arrays that localize objects [30.8] and scan¬ 
ning strategies [30.78]. IdajBIHSjEBi, 

|<s>M3EI13BIl show biomemetic sonars in action. Mov¬ 
able pinnae observed in bats have motivated research 



Fig. 30.28 Biomimetic configuration sonar with center 
transmitter flanked by receivers that rotate 



Fig. 30.29 Biomimetic sonar mounted on the end of 
a robot arm 


in receivers that rotate [30.79, 80] and deform [30.81]. 
I<gt»iwmai shows pinna deformation. Figure 30.28 
shows one such example. 

Rotating the receivers so their axes fall onto the 
reflecting object not only increases the detected echo 
amplitude, but also its bandwidth, both effects improv¬ 
ing the ability to classify an object. 

Biosonar strategy provides clues for successful ob¬ 
ject localization. It is well known that the object lo¬ 
cation within the transducer beam affects the echo 
waveform and complicates the inverse problem of ob¬ 
ject classification [30.10,82]. Dolphin movies show 
that they maneuver to position an object at a re¬ 
peatable location and range, guided by binaural echo 
processing. This has motivated a dolphin-mimicking 
movable sonar positioned at the end of a robot 
arm for object classification [30.10,82], as shown in 
Fig. 30.29. 



b) 



bearing (degrees) 

Fig.30.30a,b Binaural vergence sonar (a) and beam pat¬ 
terns (b) along the sonar axis with vergence angle a = 8°: 
solid-left transducer, dashed-right transducer 
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This system was able to differentiate reliably the 
head and tail side of a coin, but only after introducing 
a scan in elevation to accommodate for the lack of such 
positioning afforded by binaural hearing. The idea for 
a scan over elevation was motivated by the nodding mo¬ 
tion that dolphins exhibit when searching for prey lying 
under the sand. 

Another useful strategy, suggested by probing 
pulses emitted by bats, is processing echo sequences. 
As an extension to the conventional stop-and-scan oper¬ 
ation of most sonars, sonar data were acquired while the 
sonar was moving along piecewise linear paths to re¬ 
veal hyperbolic trends, similar to acoustic flow [30.83], 
Matching data to hyperbolic trends permits the es¬ 
timation of the passing range, which is useful for 
collision avoidance and passing through narrow open¬ 
ings [30.83]. 

Most sonar systems use classical estimation pro¬ 
cedures involving correlation detection and spectrum 
analysis. The cochlear model has led to multiple 
band-pass filters to process wide-band pulses for envi¬ 
ronmental landmark classification [30.43]. The action 
potential spikes observed in the biological nervous 
system also suggest neuromorphic processing based 
on coincidence detection. The sparse information pro¬ 
vided by conventional TOF measurements motivated 
sonar detectors that provided complete echo wave¬ 
form information from multiple detections that result 
in spike-like data [30.24,84]. Applying temporal and 
spacial coincidence to spike data has led to rever¬ 
beration artifact recognition [30.36] and passing-range 
estimation [30.85]. l<Bf’j| , H | ll!ai shows a simulation of 
side-looking sonar passing two targets. I^MSESEEi 
shows camera view of mobile robot traveling down cor¬ 
ridor. |45f'j|>jynm shows sonar spike map of corridor. 

One shortcoming of sonar is its relatively wide 
beam that limits the accuracy of the reflector bear¬ 
ing estimate. Biosonars have two ears that assist in 
echolocation, with a strategy that is surprising for engi¬ 
neers. Figure 30.30 shows a biomimetic binaural sonar 
described in [30.86] that employs two Polaroid 6500 


30.16 Conclusions 

Sonar is a useful, inexpensive, low-power, light-weight 
and simple ranging sensor for robotics applications 
that can provide accurate object localisation. For sonar 
to be effectively employed, understanding its physical 
principles and implementation is important and these 
topics are covered in the early sections of this chap¬ 
ter. Various approaches to sonar sensing are highlighted 
from simple single-transducer ranging to more sophis¬ 


ranging modules connected to series-7000 ultrasound 
transducers (dia d = 2.5 cm), designated as right and 
left. Both transducers transmit emissions simultane¬ 
ously, but each makes a TOF reading. The transducers 
are separated by D cm, with D minimized so that the 
incident echo wavefront is as similar as possible across 
both transducers. If both transducers are oriented par¬ 
allel to the sonar axis, the TOF difference for an echo 
from an object at bearing 9 is 

D 

ATOF = — sin#;, (30.46) 

c 

which is small for small D. 

While most sensor engineers would maximize sig¬ 
nal strength by having both transducers point at the 
object, biological sonars trade off signal strength for 
improved localization by directing the ears away from 
the object with vergence angle a. This vergence an¬ 
gle spreads the sensitivity patterns away from the 
sonar axis, as shown in Fig. 30.30. As was shown 
in Fig. 30.12, this enhances ATOF by increasing the 
echo amplitude difference. lojjiil'Jf'HIB shows ver¬ 
gence sonar in operation. 

Such a vergence sonar has several applications for 
mobile robots. By noting when both TOFs are nearly 
coincident, the binaural vergence sonar can detect when 
a reflecting object is within ±0.1° of the sonar orienta¬ 
tion. This sonar is useful for mobile robot navigation 
because, in addition to a range measurement, it in¬ 
dicates whether an obstacle is to the left or right of 
the intended path. losJEEHEO shows mobile robot 
equipped with multiple vergence sonars. Also, the oc¬ 
clusion problem associated with sonar sensing is solved 
by placing vergence sonars on the outside limits of a 
robot to determine if an opening lies to the outside of 
the robot dimensions, thus allowing the robot to pass 
through narrow openings. 

Such biomimetic techniques provide insights into 
the information content present in echoes and the type 
of sensing tasks for which sonar is best suited. 


ticated multi-transducer and multi-pulse configurations 
with associated signal processing requirements. Sophis¬ 
ticated sonars are capable of measuring target range and 
angle accurately as well as classifying targets, reject¬ 
ing interference, and compensating for motion. Sonar 
rings provide surrounding environmental coverage, and 
CTFM systems improve the sensitivity for detecting 
small reflectors. Research is ongoing in areas such as 
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signal and data processing, sonar map building, sonar sonar that draw inspiration from biological sonar sys- 
configurations, transducer technology and biomimetic terns, such as used by bats and dolphins. 


Video-References 


tol'll'H'Hia 

i^MHiEEa 

i43»*zm2Ea 


Sonar guided chair at Yale 

available from http://handbookofrobotics.org/view-chapter/30/videodetails/295 
Vergence sonar 

available from http://handbookofrobotics.org/view-chapter/30/videodetails/301 
Side-looking TOF sonar simulation 

available from http://handbookofrobotics.org/view-chapter/30/videodetails/302 
Side-looking multi-pulse sonar moving down cider-block hallway 
available from http://handbookofrobotics.org/view-chapter/30/videodetails/303 
Antwerp biomimetic sonar tracking complex object 

available from http://handbookofrobotics.org/view-chapter/30/videodetails/311 
Biological bat ear deformation in sonar detection 

available from http://handbookofrobotics.org/view-chapter/30/videodetails/312 
Monash DSP sonar tracking a moving plane 

available from http://handbookofrobotics.org/view-chapter/30/videodetails/313 

Side-looking sonar system traveling down hallway (camera view) 

available from http://handbookofrobotics.org/view-chapter/30/videodetails/314 

B-scan image of indoor potted tree using multi-pulse sonar 

available from http://handbookofrobotics.org/view-chapter/30/videodetails/315 

Antwerp biomimetic sonar tracking single ball 

available from http://handbookofrobotics.org/view-chapter/30/videodetails/316 
Antwerp biomimetic sonar system tracking two balls 

available from http://handbookofrobotics.org/view-chapter/30/videodetails/317 
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31. Range Sensing 


Kurt Konolige, Andreas Nuchter 


Range sensors are devices that capture the three- 
dimensional (3-D) structure of the world from the 
viewpoint of the sensor, usually measuring the 
depth to the nearest surfaces. These measurements 
could be ata single point, across a scanning plane, 
or a full image with depth measurements at ev¬ 
ery point. The benefits of this range data is that 
a robot can be relatively certain where the real 
world is, relative to the sensor, thus allowing the 
robot to more reliably find navigable routes, avoid 
obstacles, grasp objects, act on industrial parts, 
etc. 

This chapter introduces the main represen¬ 
tations for range data (point sets, triangulated 
surfaces, voxels), the main methods for extracting 
usable features from the range data (planes, lines, 
triangulated surfaces), the main sensors for ac¬ 
quiring it (Sect. 31.1 - stereo and lasertriangulation 
and ranging systems), how multiple observations 
of the scene, for example, as if from a moving 
robot, can be registered (Sect. 31.3) and several in¬ 
door and outdoor robot applications where range 
data greatly simplifies the task (Sect. 31.4). 
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31.1 Range Sensing Basics 

Here we present the basic representations used for range 
image data and discuss some issues relating to repre¬ 
sentations and robotics applications. While there are 
ranging devices that use sound or other waves to de¬ 
termine distance, this chapter concentrates on sensors 
that use light. 

31.1.1 Range Images and Point Sets 

Range data is a two-and-a-half-dimensional (2.5-D) 
or 3-D representation of the scene around the robot. 


The 3-D aspect arises because we are measuring the 
(X, Y. Z) coordinates of one or more points in the 
scene. Often only a single range image is used at each 
time instance. This means that we only observe the 
front sides of objects - the portion of the scene vis¬ 
ible from the robot. In other words, we do not have 
a full 3-D observation of all sides of a scene. This is 
the origin of the term 2.5-D. Figure 31.1 shows a sam¬ 
ple range image and a registered reflectance image, 
where each pixel records the level of reflected infrared 
light. 
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Fig.31.1a,b 3-D laser scan with a field of view of 360° x 100° acquired in an urban environment (Bremen downtown), 
(a) Registered infrared reflectance image, (b) Range image where closer is darker 



Fig. 31.2 (a) Mesh showing the 
rasterization of a 3-D range scanner 
using rotating mirrors and axis. 

(b) Sphere with superimposed 
reflectance values 


There are two standard formats for representing 
range data. The first is an image d(i,j), which records 
the distance d to the corresponding scene point (x, y, z) 
for each image pixel (ij). There are several common 
mappings from ( i,j,d(i,j )) to (X,Y,Z), usually aris¬ 
ing from the geometry of the range sensor or from the 
application needs. The most common image mappings 
are illustrated in Fig. 31.3. Range scanners with ro¬ 
tating mirrors sample in spherical coordinates, that is, 

Fig. 31.3 (a) Equirectangular projection of a spherical 
range scan. Latidute and longitude mesh is shown, (b) Rec¬ 
tilinear projection with three images compined to unwrap 
a scanning sphere ► 
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Fig. 31.4 3-D range scan represented with three rectilinear projections (Fig. 31.1) 


(9,tp,d). Figure 31.2 shows a sphere as longitude and 
latitude mesh and superimposed with the reflectance 
values of Fig. 31.1. The most natural range image rep¬ 
resentation is to put ( 6 , 1 p) on the (7,/j-axis of the range 
image, that is, 

i=6 , 
j=<P, 

where the longitude 9 and the latitude < p are the spher¬ 
ical coordinates. This is a projection of the samples 
sphere to a two-dimensional (2-D) image, which in¬ 
cludes a distortion. The conversion between (i,j, d) = 
(9, tp, d) and ( x,y , z ) proceeds as 

d= six 1 + y 2 +z 2 , 



tp = arctan2 (y, x) . 

Conversely, the Cartesian coordinates may be retrieved 
from the spherical coordinates by 

x = d sin 9 cos <p , 
y = d sin 9 sin sp , 

Z = dcos9 . 

The further common projection is the perspective one, 
usually called rectilinear projection. It is also called 
gnomonic or tangent-plane projection. This include 
range images produced by kinetic-like devices. Here the 


range values are assumed to be projected in a pinhole 
camera like fashion. The primary advantage of the rec¬ 
tilinear projection is that it maps straight lines in the 
3-D space and in the 2-D image. Its disadvantage is the 
smaller field of view (FOV). The projection is stretched 
toward the corners and the distortion grow with larger 
fields of view. Furthermore, FOVs are often not realiz¬ 
able by the hardware. The projection proceeds as 

co&tp sin (9 — 9q) 

i = --— , 

sin ip 1 sin <p + cos tpi cos <p cos (9 — do) 

cos <pi sin tp — sin <p\ cos tp cos (9 — 9q) 

^ sin tpi sin tp + cos <pi cos tp cos (9 — 9q ) 

where 9q and tp\ are the central longitude and central 
latitude, respectively. Figure 31.4 shows a 360° scan 
composed of three rectilinear range images. 

Further common projections are the isogonic Mer¬ 
cator projection, the cylindrical projection, Pannini 
projection, and the stereographic [31.1]. All these pro¬ 
jections are used to unwrap the scanning sphere to a 2-D 
array representation. In some applications, orthographic 
projections are processed from 3-D point cloud data. 
Some range sensors only record distances in a slice, so 
the scene (jf, z) is represented by the linear image d(i) 
for each pixel i. 

The second format is as a list {(x,, z,)} of 3-D 

data points, but this format can be used with all of 
the mappings listed above. Given the conversions from 
image data d(i,j ) to (x, y, z) the range data is only 
supplied as a list, which is usually called the point 
cloud. 


31.2 Sensor Technologies 

There are two main technologies for range sensing: 
triangulation and time-of-flight. There are many vari¬ 
ations of each of these, with different strengths and 
weaknesses. Here we briefly describe the basic con¬ 


cepts, and summarize the characteristics of the main 
types of sensors. More detailed descriptions of stereo 
and structured light sensors are given in subsequent 
sections. 
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31.2.1 Triangulation 


Triangulation sensors measure depth by determining 
the angle formed by the rays from a world point to 
two sensors. The sensors are separated by a baseline 
of length b, which forms the third segment of a triangle 
between the two sensors and the point. For simplicity, 
let one of the rays form a right angle with the baseline. 
Then the angle 9 of the other sensor ray is related to the 
depth Z perpendicular to the baseline by 


tan 9 = 


Z 

b 


(31.1) 


An image sensor measures the angle 9 by an offset on 
an image plane from the primary ray; this offset x is 
called the disparity. If we assume that the image plane 
is parallel to the baseline, then tan 9 =f/x, and we get 
the basic equation of triangulation depth sensors 


Z = — . (31.2) 

x 

Depth Precision 

An important concept is the depth resolution of a sen¬ 
sor: how precisely can the sensor measure the depth? 
Differentiating with respect to x and substituting for x 
gives 


dZ _ -Z 2 
dx fb 


(31.3) 


Triangulation precision falls off with the square of the 
distance to an object: if it is 1 mm at 1 m, then it is 
4 mm at 2 m. Increasing the baseline or reducing the 
field of views FOV is inversely related to precision, 
that is, if the baseline is doubled or FOV is halved, the 
precision is halved (e.g., from 1 mm at 0.1 m baseline 
to 0.5 mm at 0.2 m baseline). This makes triangulation 
sensors more difficult to use at a distance; changing 
the FOV or baseline can help compensate, but brings 
in other tradeoffs (Sect. 31.2.4). 


including painting a texture with a projector (not to be 
confused with structured light), using different kinds of 
support neighborhoods and regularization methods. 

Structured light sensors project a known pattern 
from one end of a baseline, and view the pattern with 
a camera from the other. By decoding the pattern in 
the image, the placement of the pattern in the projec¬ 
tor is determined, and hence the disparity. The simplest 
type of projectors output a point or a line, usually with 
a laser; the decoding problem is easy, but the amount of 
information is restricted to the point or line, and must 
be scanned to produce a full 3-D image. 

A full-image structured light system projects a pat¬ 
tern covering the whole image, that can be decoded to 
get information at each pixel or small group of pix¬ 
els. The PrimeSense technology, used in the Kinect, 
projects a known pattern of dots and decodes a small 
neighborhood to find the projected coordinates; thus 
this sensor also blurs out the spatial resolution. Other 
sensors project a time series of images with a vertical 
structure, so that each pixel on a horizontal line accu¬ 
mulates a unique code, which is then decoded against 
the projected images. Popular binary codes are Gray 
codes, which require N images if the image width is 2 W ; 
and sinusoidal phase-shifted patterns, which typically 
use three images and are decoded by determining the 
phase at each point. These time-series structured light 
systems can produce very good spatial and depth reso¬ 
lution. 

In general, structured light systems are not used out¬ 
doors because of interference from natural light, and 
for distant objects indoors. They can also have diffi¬ 
culty with high-contrast and specular objects, and the 
phase-based systems can suffer from 2 jt phase ambigu¬ 
ities. Relative motion between the sensor and the scene 
can also distort readings for time series and scanning 
sensors, although very fast projectors and cameras can 
alleviate this problem for the former. 

31.2.2 Time of Flight 


Types of Triangulation Sensors 
The two main types are stereo cameras, which have 
two cameras separated by a baseline, and structured 
light sensors, which substitute a projector for one of the 
cameras. 

Stereo cameras take images of a scene from two 
slightly different viewpoints, and match texture in the 
images to determine corresponding points and dispari¬ 
ties. Some issues with stereo cameras are that there is 
often too little texture, especially indoors, to make re¬ 
liable matches; and the matching uses small patches, 
which blurs out the spatial resolution. Many ingenious 
methods have been developed to deal with these issues, 


The principle behind time-of-flight (TOF) sensors is 
like that of radar: measure the time it takes for light to 
be projected out to an object and return. Because light 
travels about 0.3 m per nanosecond, very precise timers 
are required for direct TOF measurement; an alterna¬ 
tive is indirect methods, typically phase difference with 
a modulated reference beam. 

Because they measure time of flight, these sen¬ 
sors can theoretically have constant precision in depth 
measurement, no matter how far an object - unlike 
triangulation sensors, which fall off as the square of 
distance. But TOF sensors cannot duplicate the very 
fine precision of triangulation sensors for close objects. 
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and hence are not used in close-range metrology ap¬ 
plications, such as small object reconstruction or parts 
quality measurements. 

Direct Time of Flight 

In direct TOF sensors, the travel time is measured 
by a high-speed chronometer. Laser-based direct TOF 
range sensors are also called LIDAR (light detection 
and ranging) or LADAR (laser radar) sensors. The 
travel time multiplied by the speed of light (in the given 
medium - space, air, or water and adjusted for the den¬ 
sity and temperature of the medium) gives the distance 

2 d = ct, (31.4) 

where cl is the distance to the object, c is the speed of 
light, and t is the measured travel time. Error in measur¬ 
ing the time t gives rise to a proportional distance error. 
In practice, one tries to measure the peak of the output 
pulse, which has a finite extent; weak reflections from 
distant objects make it harder to measure this peak and 
hence the error tends to increase with distance. Aver¬ 
aging multiple readings can reduce the random error in 
these readings. 

The simplest TOF sensors transmit only a single 
beam, thus range measurements are only obtained from 
a single surface point. Robotics applications usually 
need more information, so the range data is usually sup¬ 
plied as a vector of range to surfaces lying in a plane 
(Fig. 31.5) or as an image (Fig. 31.1). To obtain these 
denser representations, the laser beam is swept across 
the scene. Normally, the beam is swept by a set of mir¬ 
rors rather than moving the laser and detector them¬ 
selves (mirrors are lighter and less prone to motion dam¬ 
age). The most common technologies for this are using 
a stepper motor (for program-based range sensing) or 
rotating or oscillating mirrors for automatic scanning. 

Scanning or averaging requires multiple TOF pulses 
at perhaps high repetition rates, which can give rise to 


Range 



Fig. 31.5 Plot of ideal one-dimensional (1-D) range image 
of sample distance versus angle of measurement 


ambiguity about which pulse is actually being received. 
If 8 1 is the time between pulses, then the ambiguity in¬ 
terval for the device is 1 /2c8t, for example, for a pulse 
repetition rate of 100 kHz, the ambiguity interval is 
1500 m. If an object is further than this, it will be seen as 
within the interval, with distance zmodl/2c8t. An un¬ 
winding algorithm can recover the true depth, if range 
values for the scanning system change slowly, and start 
within the ambiguity interval. 

Typical ground-based time of flight sensors suitable 
for robotics applications have a range of 10—100 m, 
and an accuracy of 5—10 mm. The amount of the scene 
scanned will depend on the sweep rate of the mirrors 
and the pulse rate, but l-25k points per second are 
typical. Manufacturers of these sensors include Acuity, 
Hokuyo, Sick, Mensi, DeltaSphere, and Cyrax. 

Multiple-beam scanning LIDARs can increase the 
amount of information available. Velodyne [31.2] 
makes three devices, with 16, 32, and 64 beams ar¬ 
ranged vertically, that acquire point data at rates of up 
to 15 scans/s (1.3 MPixel/s), with a full 360° horizon¬ 
tal scan and a 27° vertical FOV from the laser array. 
The laser pulses are 5 ns long, and depth precision is 
approximately 2 cm. These devices are typically used 
in autonomous driving, for environment reconstruction 
and obstacle avoidance. 

Flash LIDAR 

In contrast to scanning devices, a flash LIDAR has a 2-D 
detector array (also called a. focal plane array similar 
to a camera imager, but where each pixel incorporates 
timing circuitry to measure TOF of a laser pulse). In¬ 
stead of a single or multiple laser beams, a light source 
pulse (LED (light-emitting diode) or laser) is shaped to 
cover a large area. All pixels have their timers started 
when the pulse is initiated, and measure the time it 
takes to receive the backscattered light. Typically, some 
tens of samples are captured and averaged, to reduce 
noise in the measurements - the amount of energy re¬ 
ceived is quite small, since the laser is not focused into 
a beam. As might be expected, the detector array pixels 
are quite large because of the timing electronics; a typ¬ 
ical device from ASC [31.3] has 128 x 128 pixels, and 
can capture data at rates up to 60 Hz (f« 1 MPixel/s, 
similar to the Velodyne devices). These devices are 
expensive, and hence not used in consumer applica¬ 
tions. Alternatively, indirect ways of measuring TOF 
have simpler per-pixel electronics - see the next subsec¬ 
tion. Table 31.1 lists the characteristics of one of ASC’s 
devices. 

Table 31.1 Characteristics of TigerEye device 

Manuf./Model Resolution Max range Repeatability 

ASC Inc. TigerEye 128x 128 60—1100m 0.04m @ 60m 
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Indirect Time of Flight Sensors 
Indirect TOF sensors measure the distance by inferring 
transit times from particular properties of a propagating 
beam. The two main methods are based on modulation 
and phase differences and gated intensity. 

Modulation-Based TOF 

Modulation-based range sensors are commonly of two 
types, where a continuous laser signal is either ampli¬ 
tude or frequency modulated. By observing the phase 
shift between the outgoing and return signals, the signal 
transit time is estimated and from this the target distance. 

Amplitude modulation works by varying the inten¬ 
sity of a light signal s(t) = sin(2^/r) with frequency/. 
The signal reflected from an object has a phase shift ip, 
and the returned signal is 


r(t) = R sin(2 nft— tp) = R sin 




(31.5) 


with c is the speed of light, d the distance to the object, 
and R the reflected amplitude. Measuring the phase shift 
yields the distance; note because the phase shift wraps 
around at 2n, the ambiguity interval is c/2/. For a mod¬ 
ulation of 10 MHz, the interval is 15 m. 

To measure the phase difference, the returned sig¬ 
nal is mixed with the original reference signal and a 90° 
shifted version, then low-pass filtered. Figure 31.6 com¬ 
pares the two mixed signals that give both the phase 
difference and the intensity. 

There are commercial devices using single-beam 
and planar array versions of amplitude modulation. The 
planar arrays take advantage of CMOS integration to 
implement the full signal comparison at each pixel, in 
a compact sensor [31.4, 5]. Because the signal-to-noise 



Fig. 31.6 Comparing the two mixed signals gives both the phase 
difference and the intensity 


ratio (SNR) depends on the amplitude of the returned 
signal, these devices typically have a depth resolution 
that falls off as d 2 , comparable to triangulation devices. 
Typical repeatability is about 3 cm at 0.5 m, signifi¬ 
cantly less than triangulation devices; the arrays are 
up to 200 x 200 pixels. The attraction of the planar ar¬ 
ray is that it is a single-chip ranging device, simple 
and potentially cheap to produce. They also have good 
ambient light rejection, and potentially can be used 
outdoors. 

Frequency modulation, also called frequency mod¬ 
ulation continuous wave (FMCW), modulates a laser 
frequency with a sawtooth ramp of period t m , where 
the maximum frequency spread is A/. The outgoing 
and incoming signals are mixed to give a signal dif¬ 
ference f, which is measured by frequency counting. 
The distance is calculated as d =/cf m /2A/. Since the 
frequency difference can be measured very accurately, 
this technique can yield excellent depth precision, on 
the order of 5 mm at 2 m [31.6]. The electronics re¬ 
quired for mixing and frequency counting, as well as 
the difficulty of linear laser frequency modulation, have 
restricted this technique to high-end single-beam de¬ 
vices [31.7]. 

Range-Gated Intensity 

Another indirect method for measuring TOF of a pulse 
is to measure the amount of the pulse that gets re¬ 
turned within a precise time period. If a pulse of width 
w is emitted, a detector will start to receive the pulse 
reflected from an object of distance d at to = 2 d/c, 
and finish receiving it at t w = (w + 2d)/c. If the de¬ 
tector is open for a precise amount of time t, then 
the amount of the reflected signal it receives, relative 
to the full pulse reflection, is a linear measure of the 
distance d. Thus, by comparing two returns, one with 
a large open time to measure the full pulse, and one with 
a smaller open time, d can be determined. A planar ar¬ 
ray CMOS device can be fabricated to measure the two 
pulse returns sequentially, as long as each pixel can be 
triggered to be photosensitive for a precise amount of 
time. Commercial devices using this principle have pre¬ 
cision on the order of a few cm at 1 m, with arrays up to 
128x96 [31.8,9], 

31.2.3 Comparison of Methods 

A useful overall summary of different types of range¬ 
sensing technology is in reference [31.6]. The two main 
parameters are spatial precision and depth precisions. 
In Fig. 31.7, available sensors are classified along these 
two dimensions. The direct TOF laser scanners (LMS, 
UT devices) cluster around 0.5—lcm in both spatial 
and range precision. The Photon80 device uses FMCW 
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modulation with outstanding spatial and range preci¬ 
sion; it is also a scanning device. 

Triangulation devices (stereo, Kinect, and struc¬ 
tured light in general) have less favorable characteris¬ 
tics. However, the target is at 2 m, and with triangulation 
precision falling with d 2 , the comparison is biased. All 
these devices would be better than the TOF devices 
at 0.5 m in terms of range error. The stereo sensors 
can also exhibit much better spatial density in different 
configurations, that is, higher resolution sensors, larger 
baseline, smaller correlation window. In general, we 
would expect the triangulation sensors to outperform 
TOF sensors at shorter distances. 

Finally, a single-chip flash LIDAR device, the IFM 
03D200, uses amplitude modulation, and gives charac¬ 
teristically poor results relative to the other sensors. 

31.2.4 Stereo Vision 

This section discusses stereo analysis in more detail. 
Stereo analysis uses two or more input images to es¬ 
timate the distance to points in a scene. The basic 
concept is triangulation : a scene point and the two cam¬ 
era points form a triangle, and knowing the baseline 
between the two cameras, and the angle formed by the 
camera rays, the distance to the object can be deter¬ 
mined. 

In practice, there are many difficulties in making 
a stereo imaging system that is useful for robotics 
applications. Most of these difficulties arise in find¬ 
ing reliable matches for pixels in the two images that 
correspond to the same point in the scene. A further 
consideration is that stereo analysis for robotics has 
a real-time constraint, and the processing power needed 
for some algorithms can be very high. But, in recent 
years much progress has been made, and the advantage 
of stereo imaging is that it can provide full 3-D range 
images, registered with visual information, potentially 
out to an infinite distance, at high frame rates - some¬ 
thing which no other range sensor can match. 

Fig. 31.8 Ideal stereo geometry. The global coordinate 
system is centered on the focal point (camera center) of 
the left camera. It is a right-handed system, with positive Z 
in front of the camera, and positive X to the right. The cam¬ 
era principal ray pierces the image plane at C x , C y , which 
is the same in both cameras (a variation for verged cameras 
allows C x to differ between the images). The focal length is 
also the same. The images are lined up, with y = y/ for the 
coordinates of any scene point projected into the images. 
The difference between the .r coordinates is called the dis¬ 
parity. The vector between the focal points is aligned with 
the X-axis ► 


Interpolation distance (m) - Lower is better 
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Fig. 31.7 Range device characteristics (after [31.6]). Spatial preci¬ 
sion is on the T-axis, depth precision on the X-axis. Measurements 
are taken on a target at 2 m 



In this subsection, we will review the basic algo¬ 
rithms of stereo analysis, and highlight the problems 
and potential of the method. For simplicity, we use 
binocular stereo. 
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Stereo Image Geometry 

This subsection gives some more detail of the fun¬ 
damental geometry of stereo, and in particular the 
relationship of the images to the 3-D world via pro¬ 
jection and reprojection. A more in-depth discussion of 
the geometry and the rectification process can be found 
in [31.10], 

The input images are rectified, which means that the 
original images are modified to correspond to ideal pin¬ 
hole cameras with a particular geometry, illustrated in 
Fig. 31.8. Any 3-D point S projects to a point in the 
images along a ray through the focal point. If the prin¬ 
cipal rays of the cameras are parallel, and the images are 
embedded in a common plane and have collinear scan 
lines, then the search geometry takes a simple form. 
The epipolar line of a point s in the left image, de¬ 
fined as the possible positions of s' in the right image, 
is always a scan line with the same y coordinate as s. 
Thus, search for a stereo match is linear. The process 
of finding a rectification of the original images that puts 
them into standard form is called calibration, and is dis¬ 
cussed in [31.10]. 

The difference in the x coordinates of s and s' is 
the disparity of the 3-D point, which is related to its 
distance from the focal point, and the baseline T x that 
separates the focal points. 

A 3-D point can be projected into either the left or 
right image by a matrix multiplication in homogenous 
coordinates, using the projection matrix. The 3-D co¬ 
ordinates are in the frame of the left camera (Fig 31.8) 

( F * 0 c * - F M 

P= 0 Fy Cy 0 . (31.6) 

\ 0 0 1 0 / 

This is the projection matrix for a single camera. F x 

and F y are the focal lengths of the rectified images, and 

C x and C y are the optical center; T x is the translation of 
the camera relative to the left (reference) camera. For 
the left camera, it is 0; for the right camera, it is the 
baseline times the x focal length. 

A point in 3-D is represented by homogeneous co¬ 
ordinates and the projection is performed using a matrix 
multiply 


jection matrix 


/I 

0 

0 

—C x 

0 

1 

0 

-Cy 

0 

0 

0 

F x 

1 ° 

0 

-1 /T x 

(c*— cy) 
T x 


(31.8) 


The primed parameters are from the left projection ma¬ 
trix, the unprimed from the right. The last term is zero 
except for verged cameras. If x,y and x',y are the two 
matched image points, with d = x — x', then 


(X\ 

Y 

Z 

= Q 

'V' 

y 

d 

\w) 


VV 


where (X/W,Y/W,Z/W) are the coordinates of the 
scene feature, and d = x — x' is the disparity. Assum¬ 
ing C x = C' x , the Z distance assumes the familiar inverse 
form of triangulation 

F T' 

Z = -FJt- . (Bi.io) 

d 

Reprojection is valid only for rectified images - for 
the general case, the projected lines do not intersect. 
The disparity d is an inverse depth measure, and the 
vector (jc, y, d) is a perspective representation range im¬ 
age (Sect. 31.1.1), sometimes called the disparity space 
representation. The disparity space is often used in ap¬ 
plications instead of 3-D space, as a more efficient 
representation for determining obstacles or other fea¬ 
tures (Sect. 31.4.3). 

Equation (31.9) is a homography between dispar¬ 
ity space and 3-D Euclidean space. Disparity space 
is also useful in translating between 3-D frames. Let 
Pq = [xo,yo, do, I ] in frame 0, with frame 1 related 
by the rigid motion R, t. From the reprojection (3 1 .9) 
the 3-D position is Qpo- Under the rigid motion, this 
becomes 



t 

1 


Qpo, 



(31.7) 


where ( x/w,y/w ) are the idealized image coordinates. 

If points in the left and right images correspond to 
the same scene feature, the depth of the feature can be 
calculated from the image coordinates using the repro¬ 


and finally applying Q~ 1 yields the disparity represen¬ 
tation in frame 1. The concatenation of these operations 
is the homography 

H(R, t) = Q _l (q jW (3l.ll) 

Using the homography allows the points in the refer¬ 
ence frame to be directly projected onto another frame, 
without translating to 3-D points. 
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Stereo Matching Methods 
The fundamental problem in stereo analysis is matching 
image elements that represent the same object or object 
part in the scene. Once the match is made, the range to 
the object can be computed using the image geometry. 

Matching methods can be characterized as local 
or global. Local methods attempt to match small re¬ 
gions of one image to another based on intrinsic fea¬ 
tures of the region. Global methods supplement local 
methods by considering physical constraints such as 
surface continuity or base of support. Local methods 
can be further classified by whether they match dis¬ 
crete features among images, or correlate a small area 
patch [31.11]. Features are usually chosen to be light¬ 
ing and viewpoint independent, for example, corners 
are a natural feature to use because they remain cor¬ 
ners in almost all projections. Feature-based algorithms 
compensate for viewpoint changes and camera differ¬ 
ences, and can produce rapid, robust matching. But 
they have the disadvantage of requiring perhaps expen¬ 
sive feature extraction, and yielding only sparse range 
results. 

In the next section, we present local area correlation 
in more detail, since it is one of the most efficient and 
practical algorithms for real-time stereo. A survey and 
results of recent stereo matching methods are in [31. 1 2], 
and the authors maintain a web page listing up-to-date 
information in [31.13]. 

Area Correlation Stereo 

Area correlation compares small patches among im¬ 
ages using correlation. The area size is a compromise, 
since small areas are more likely to be similar in images 
with different viewpoints, while larger areas increase 
the signal-to-noise ratio. In contrast to the feature-based 
method, area-based correlation produces dense results. 
Because area methods need not compute features, and 
have an extremely regular algorithmic structure, they 
can have optimized implementations. 

The typical area correlation method has five steps 
(Fig. 31.9): 

1. Geometry correction. In this step, distortions in the 
input images are corrected by warping into a stan¬ 
dard form. 

2. Image transform. A local operator transforms each 
pixel in the grayscale image into a more appropriate 
form, for example, normalizes it based on average 
local intensity. 

3. Area correlation. This is the correlation step, where 
each small area is compared with other areas in its 
search window. 

4. Extrema extraction. The extreme value of the corre¬ 
lation at each pixel is determined, yielding a dispar- 



Greyscale image Transform Area correlation Filtered disparity 

A pixels images disparities D image 


ity image: each pixel value is the disparity between 
the left and right image patches at the best match. 

5. Post-filtering. One or more filters clean up noise in 
the disparity image result. 

Correlation of image areas is disturbed by illu¬ 
mination, perspective, and imaging differences among 
images. Area correlation methods usually attempt to 
compensate by correlating not the raw intensity images, 
but some transform of the intensities. Let u,v be the 
center pixel of the correlation, d the disparity, and I x%y , 
I' x the intensities of the left and right images: 


E J .vfc. y -/x. y ] VLd.y-t’rdA 

VJA.v - Ky? Z x ,yK,y - I'x-d.y] 1 2 3 4 

2. High-pass filter such as Laplacian of Gaussian 
(LOG). The Laplacian measures directed edge in¬ 
tensities over some area smoothed by the Gaussian. 
Typically, the standard deviation of the Gaussian is 
1-2 pixels 

T. '(LOG,. y - LOG A _,/. y ), 

*.7 

where s(x) is x 2 or | |jc| |. 

3. Nonparametric. These transforms are an attempt to 
deal with the problem of outliers, which tend to 
overwhelm the correlation measure, especially us¬ 
ing a square difference. The census method [31.14] 
computes a bit vector describing the local environ¬ 
ment of a pixel, and the correlation measure is the 
Hamming distance between two vectors 

y > Iu,v) 0 (j x —d,y > ^u, it) ■ 

x >y 

Results on the different transforms and their error 
rates for some standard images are compiled in [31.13]. 

Another technique for increasing the signal-to-noise 
ratio of matches is to use more than two images [31.15]. 


1. Normalized cross-correlation 


Fig. 31.9 Basic stereo processing (see text for details) 
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Table 31.2 Post-filtering techniques for eliminating false 
matches in area correlation 


Correlation surface [31.17] 

Peak width: Wide peak indi¬ 
cates poor feature localization 
Peak height: Small peak 
indicates poor match 

Number of peaks: Multiple 
peaks indicate ambiguity 

Mode filter 

Lack of supporting disparities 
violates smoothness 

Left/right check [31.18,19] 

Nonsymmetric match indi¬ 
cates occlusion 

Texture [31.20] 

Low texture energy yields 
poor matches 


This technique can also overcome the problem of view¬ 
point occlusion, where the matching part of an object 
does not appear in the other image. The simple tech¬ 
nique of adding the correlations between images at the 
same disparity seems to work well [31.16]. Obviously, 
the computational expenditure for multiple images is 
greater than that for two. 

Dense range images usually contain false matches 
that must be filtered, although this is less of a problem 
with multiple-image methods. Table 31.2 lists some of 
the post-filters that have been discussed in the literature. 

Disparity images can be processed to give subpixel 
accuracy, by trying to locate the correlation peak be¬ 
tween pixels. This increases the available range resolu¬ 
tion without much additional work. Typical accuracies 
are 1/10 pixel. 

Stereo Range Quality 

Various artifacts and problems affect stereo range im¬ 
ages. 

Smearing. Area correlation introduces expansion in 
foreground objects, for example, the woman’s head in 
Fig. 31.10. The cause is the dominance of strong edges 
on the object. Nonparametric measures are less subject 
to this phenomenon. Other approaches include multiple 
correlation windows and shaped windows. 


Dropouts. These are areas where no good matches can 
be found because of low texture energy. Dropouts are 
a problem for indoor and outdoor man-made surfaces. 
Projecting a random texture can help [31.21]. 

Range Resolution. Unlike LADAR devices, stereo 
range accuracy is a quadratic function of distance, 
found by differentiating (31.10) with respect to 
disparity 

F T' 

8 Z=--^i. (31.12) 

The degradation of the stereo range with distance 
can be clearly seen in the 3-D reconstruction of 
Fig. 31.10. 

Processing. Area correlation is processor-intensive, 
requiring Awd operations, where A is the image 
area, w is the correlation window size, and d is 
the number of disparities. Clever optimizations take 
advantage of redundant calculations to reduce this 
to Ad (independent of window size), at the ex¬ 
pense of some storage. Real-time implementations ex¬ 
ist for standard personal computers (PC)s [31.22,23], 
graphics accelerators [31.24,25], digital signal pro¬ 
cessors (DSPs) [31.26], field programmable gate ar¬ 
rays (FPGAs) [31.22,27], and specialized application- 
specific integrated circuits (ASICs) [31.28]. 

Other Visual Sources of Range Information 
Here we briefly list the most popular, but less reliable 
sources of range information. These sources can poten¬ 
tially supplement other sensors: 

• Focus/defocus: Knowledge of the camera parame¬ 
ters and the amount of blur of image features allows 
the estimation of how far the corresponding scene 
features are from the perfect focus distance [31.29]. 
Sensors may be passive (using a precaptured image) 
or active (capturing several images with different 
focus settings). 



Fig.31.10a-c Sample stereo results from an outdoor garden scene; baseline is 9 cm. (a) original left image, (b) computed 
3-D points from a different angle, (c) disparity in pseudo-color 
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• Structure and motion: Structure and motion algo¬ 
rithms compute 3-D scene structure and the sensor 
positions simultaneously [31.30]. This is essentially 
a binocular stereo process (see discussion before), 
except that only a single moving camera is used. 
Thus, the images needed by the stereo process 
are acquired by the same camera in several differ¬ 
ent positions. Video camcorders are also used, but 
they have lower resolution. One important advan¬ 
tage of this approach over the normal algorithm is 
that features can be tracked easily if the time be¬ 
tween frames or the motion is small enough. This 
simplifies the correspondence problem ; however, 
it can lead to another problem. If the pair of im¬ 
ages used for the stereo calculation are taken close 
together in time, then the separation between the 
cameras’ images will not be much - this is a short 
baseline. Triangulation calculations are then more 
inaccurate, as small errors in estimating the po¬ 
sition of image features results in large errors in 
the estimated 3-D position (particularly the depth 
estimate). This problem can be partly avoided by 
tracking for longer periods. A second problem that 
can arise is that not all motions are suitable for es¬ 
timate of the full 3-D scene structure. For example, 
if the video recorder only rotates about its optical 
axis or about its focus point, then no 3-D informa¬ 
tion can be recovered. A little care can avoid this 
problem. 

• Shading: The pattern of shading on a surface is re¬ 
lated to the orientation of the surface relative to the 
observer and light sources. This relationship can 
be used to estimate the surface orientation across 
the surface. The surface normals can then be in¬ 
tegrated to give an estimate of the relative surface 
depth. 

• Photometric stereo: Photometric stereo [31.31] is 
a combination of shading and stereo processes. The 
key concept is that the shading of an object varies 
with the position of the light sources. Hence, if you 
had several aligned pictures of an object or scene 
with the light source in different positions (e.g., 
the sun moved), then you can calculate the scene’s 
surface normals. From these, the relative surface 
depth can be estimated. The restriction of a station¬ 
ary observer and changing light sources makes this 
approach less likely to be useful to most robotics 
applications. 

• Texture: The way uniform or statistical textures vary 
on a surface is related to the orientation of the sur¬ 
face relative to the observer. As with shading, the 
texture gradients can be used to estimate the surface 
orientation across the surface [31.32]. The surface 


normals can then be integrated to give an estimate 

of the relative surface depth. 

31.2.5 Structured Light Stereo 

Recently, the Kinect range-sensing device became 
the fastest-selling consumer electronics device in his¬ 
tory [31.33]. The Kinect uses a structured-light stereo 
technique from PrimeSense (bought out by Apple, Inc.) 
to give a reasonable balance of spatial and depth pre¬ 
cision, in a highly-integrated device, at a very low 
price. 

The PrimeSense technology projects a struc¬ 
tured pattern generated from an IR (infrared) laser 
and patented combination of diffractive elements 
(Fig. 31.11). The pattern consists of bright dots on 
a grid, with unique combinations of dots over a horizon¬ 
tal area. A camera with resolution 1280 x 960 captures 
the IR image at a horizontal offset of 7.5 cm from 
the projector, and correlates a 19 x 19 block against 
the known dot pattern for a given horizontal line. If 
a match is found, the position of the pattern in the 
known projected pattern gives the angle subtended by 
the reflecting object to the projector and camera, and 
hence the distance. A depth image is returned at a reso¬ 
lution of 640 x 480, with 11 bits of depth precision, over 
an FOV of approximately 50°; the depth is interpolated 
to a precision of 1/8 pixel. Error characteristics for the 
Kinect have been studied in [31.34]. The depth decision 
is limited by the resolution, similarly to (31.3). 

There are additional sources of error. Dropouts oc¬ 
cur when the surfaces are minimally reflective, or spec¬ 
ular. False positives, where there are gross mismatches 
in the match, are very rare with PrimeSense technology. 
Spatial precision is much lower than the returned 640 x 
480 depth image would indicate, because of the smear- 



fig. 31.11 The laser grid of the Kinect for calculating depth 
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ing effect of block correlation. The projected pattern 
expands and contracts based on temperature, leading to 
offset in disparities along the vertical axis. While the 
projector is temperature controlled via a Peltier unit, its 


temperature can vary. A temperature sensor measures 
this variation, and applies disparity steps across the im¬ 
age, which are apparent as vertical steps in the depth 
image. 


31.3 Registration 


This section introduces techniques for 3-D localiza¬ 
tion of parts for robot manipulation, self-localization of 
robot vehicles, and scene understanding for robot nav¬ 
igation. All of these are based on the ability to register 
3-D shapes, for example, range images to range images, 
triangulated surfaces or geometric models. Registration 
puts two or more independently acquired 3-D shapes 
into one frame of reference. It can be solved as an 
optimization problem that uses a cost function for the 
quality of the alignment. The 3-D shapes are registered 
by determining the rigid transformation (rotation and 
translation) which minimizes the cost function. Feature 
based registration extracts distinguishing features of the 
range images and uses corresponding features for calcu¬ 
lating the alignment. 

31.3.1 The ICP Algorithm 


All the corresponding points can be represented in a tu¬ 
ple (mj,dj) where m,- e M C M and d, e D C D. Two 
things have to be calculated. First, the corresponding 
points, and second, the transformation (R, t) that mini¬ 
mizes E(K.t) on the basis of the corresponding points. 
The ICP algorithm uses closest points as correspond¬ 
ing points. A sufficiently good starting guess enables 
the ICP algorithm to converge to the correct minimum. 
Figure 31.12 shows two 3-D point clouds, their ini¬ 
tial alignment and the final registration after a few ICP 
iterations. 

Implementations of ICP (Algorithm 31.1) use 
a maximal distance for closest points to handle par¬ 
tially overlapping point sets. In this case, the proof 
about ICP’s monotonic convergence in [31.35] no 
longer holds, since the number of points as well as 
the value of £(R, t) might increase after applying 
a transformation. 


The following method is used for the registration of 
point sets. The complete algorithm was invented at the 
same time in 1991 by Besl and McKay [31 .35], by Chen 
and Medioni [31 .36], and by Zhang [31 .37]. The method 
is called iterative closest points (ICP) algorithm. It is the 
de-facto standard for registration of point sets, but is also 
applicable to 3-D shapes, if one samples them. 

Given two independently acquired sets of 3-D 
points, M (model set) and D (data set) which correspond 
to a single shape, we want to find the transformation 
(R, t) consisting of a rotation matrix R and a translation 
vector t which minimizes the following cost function, 

1 N 

E(R,t) = -^2\\m i -(Rd i + t)\\ 2 - (31-13) 


Algorithm 31.1 The ICP algorithm 

1: for ; = 0 to maxlterations do 

2: for all dj e D do 

3: find the closest point within a range <7 max in the 

set M for point dj 

4: end for 

5: Calculate transformation (R, t) that minimizes 

the error function (31.13) 

6: Apply the transformation found in step 5 to the 

data set D. 

7: Compute the difference of the quadratic error, 

that is, compute the difference of the value 
||£,_i (R, t) — £)(R, i) || before and after the ap- 



Fig.31.12a,b Initial alignment of 
two 3-D point clouds (a) and after 
optimization with ICP (b) 
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plication of the transformation. If this difference 
falls below a threshold e, terminate. 

8: end for 


Current research in the context of ICP algo¬ 
rithms mainly focuses on fast variants of ICP al¬ 
gorithms [31.38]. If the input are 3-D meshes then 
a point-to-plane metric can be used instead of (31.13). 
Minimizing using a point-to-plane metric outperforms 
the standard point-to-point one, but requires the com¬ 
putation of normals and meshes in a preprocessing 
step. 

The computation of closest points is the crucial 
step of the ICP algorithm. A naive implementation 
examines all points in M and the resulting computa¬ 
tion time for ICP is 0(\D\ \M\), that is, 0(n 2 ). Note 
that N can be very large; advanced high-precise 3-D 
laser scanners such as the Zoller + Frohlich yield a data 
rate up to 1 000 000 3-D points per second. An effi¬ 
cient tree data structure, a k-d tree, is widely used to 
speed up the closest point computation. Every node 
of the k-d tree represents a partition of the point set 
into two distinct sets, the successor nodes. The root 
of the tree represents the whole point set. The leaves 
of the tree are called buckets and are a partition of 
the set into small, disjunctive point sets. Furthermore, 
every node of the tree consists of the center and the 
dimension of the point set. In the original k-d tree 
paper, these so-called split dimensions have been cho¬ 
sen depending on the depth of node in a robin-round 
fashion [31.39]. The split dimension and value define 
an axis-aligned hyperplane in the ^-dimensional space. 
The data are partitioned according to their position 
to the hyperplane into the successor nodes. The k-d 
tree is constructed until the number of points in the 
nodes falls below a threshold b (bucket size). Only the 
leaves of the tree contain the data points. Searching 
in k-d trees is done recursively. A given 3-D point p q 
needs to be compared to the separating plane (split¬ 
ting dimension and splitting value) in order to decide 
on which side the search must continue. This proce¬ 
dure is executed until the leaves are reached. There, 
the algorithm has to evaluate all bucket points. How¬ 
ever, the closest point may be in a different bucket, 
iff the distance d of the query point p q to the lim¬ 
its is smaller than the one to the closest point in 
the bucket p\,. In this case backtracking has to be 
performed. The test is known as Ball-Within-Bounds 
test [31.39-41]. The optimized k-d tree chooses the split 
dimension and split value [31.40], such that the ex¬ 
pected amount of backtracking is minimized. Since one 
typically has no information about the query points k-d 
tree algorithms take only the distribution of the given 
points into account. For all possible queries, this works 


sufficiently, but it will not be optimal for a specific 
query [31.40]. This enables the recursive construction 
and avoids the overall optimization that is known to be 
NP-complete [31.42], 

Improvements to k-d tree search, especially for 
small dimensions, have been shown in the last decade. 
They include approximate k-d tree search [31.41], reg¬ 
istration using d 1 -trees [31.43] and cached k-d tree 
search [31.44]. In addition, the spatial data structure oc¬ 
tree might, which is discussed later in this section, can 
be used to search the point clouds with similar perfor¬ 
mance. 

In each ICP iteration, the transformation can be cal¬ 
culated in 0(N) by any of these four methods: 

1. A singular value decomposition (SVD)-based 
method by A run et al. [31 .45] 

2. A quaternion-based method by Horn et al. [31.46] 

3. An algorithm using orthonormal matrices by 
Horn [31.47] 

4. A calculation based on dual quaternions by Walker 
et al. [31.48]. Besides these closed-form solutions, 
there are several linearized, approximative ver¬ 
sion [31.49]. 

The challenge is to ensure thar R is orthonormal. 
Most often, the first method is implemented, due to it’s 
simplicity and the availability of numerical SVDs in 
various libraries. The rotation R is represented as an 
orthonormal 3x3 matrix. The optimal rotation is cal¬ 
culated by R = VU T . Here the matrices V and U are 
derived by the SVD H = UA V T of a cross-correlation 
matrix H. This 3x3 matrix H is given by 

' V / Sxx Sxy S , \ 

H = = s yx s yy s yz , (31.14) 

' — 1 \ 5V.X S Z y S Z7 J 

where 

N N 

Sxx = y] m x i d' x j, Sxy = J2 m'x,idy,v • • • • 
i= 1 i=l 

31.3.2 Marker and Feature-Based 
Registration 

To avoid issues with starting guess in the ICP frame¬ 
work, marker-based registration uses defined artificial 
or natural landmarks as corresponding points. This 
manual data association ensures that by minimizing 
(31.13) the scans are registered at the correct location. 
Iterations are no longer required, but possible as ver¬ 
ification with the RANSAC algorithm. The RANSAC 
algorithm is a general, randomized procedure that iter- 
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atively finds an accurate model for observed data that can be interpreted as fitting a Gaussian distribution and 
may contain a large number of outliers [31 .50]. computing the main axes. For each query point 


3-D Feature Representations and Extraction 
There are many representations available for encoding 
3-D scene structure and model representations, but the 
following representations are the ones most commonly 
encountered in robotics applications. Some scene mod¬ 
els or descriptions may use more than one of these 
simultaneously to describe different aspects of the scene 
or object models. 

Normals. Normals describe the surface orientation in 
a points. There are a number of approaches that have 
been developed for computing normals in point clouds 
and range images. Most of these approaches involves 
some form of eigenvalue decomposition resembling 
total linear least squares. The normals are usually com¬ 
puted when taking into account the nearby neighbor 
points which are computed using a variety of methods 
like nearest neighbor search. These methods include 
k nearest neighbors (fc-NN) and radius search among 
others. The method of total least squares is robust to 
noise as it inherently includes low-pass filtering, but 
it is sensitive to the distribution and density of the 
point samples as well as the curvature of the underlying 
manifold. Improvements to the above method by using 
higher order surfaces are discussed in [31.51] and it is 
noted that such methods can fail even with arbitrarily 
dense samples. 

The total least-squares problem finds the plane pa¬ 
rameters that optimally fit a small surface area in a point 
set given by the the nearest neighbors. A plane is de¬ 
fined by 

n x x + n y y + n z z — d = 0 , 



k 

E*. 

1=1 

k 

- ^ J(pi - /*•) • 

i=i 


The vector corresponding to the smallest eigenvalue of 
the above matrix S is the normal direction if the neigh¬ 
boring points belong to a plane and this is also the 
closed form solution of the total least-squares problem. 

For equirectangular range images, that is, spheri¬ 
cal coordinates, a fast algorithm has been developed 
in [31.52] that avoids computing eigenvalues. Dividing 
(31.15) by d 2 gives a simplified function and further di¬ 
vision by the squared range p 2 yields 

e= ’ 
i= 1 

( cos dj sin <pi 
sin 6j sin <pi 
cos <pi 


Therefore, a solution for n is given as 
n = M” 1 * , 


where 


k 


k 


m=x>*m = E? 


i— 1 




where p = (x,y, j) T lies on the plane and ( n x , n y , n z , d) 
are the parameters to compute. Given a subset of k 3-D 
points Pi,i= 1,2of the surface, least squares 
finds the optimal normal vector n = (n x , n y , «-) T and 
scalar d that minimizes the following error equation 

k 

e = Yfpn - d) . (31.15) 

i=i 

The basic method for normal estimation from the neigh¬ 
boring points using a fitting plane is the principal 
component analysis (PCA). PCA is the eigenvalue de¬ 
composition of the data covariance (or correlation) 
matrix or singular value decomposition of a data ma¬ 
trix, usually after mean centering (and normalizing) the 
data matrix for each attribute (3-D query point). PCA 


This way the computation of eigenvalues is avoided and 
the matrix M can be precomputed for the desired im¬ 
age coordinates as it does not depend on the range. The 
tangential surface and therefore its normal vector are 
obtained by simply taking the derivative of the surface 
function in the point of interest 


n = Vp = Vp(0, <p) 


3~D Point Features. This is a set \p, = (x, , y,, z ,)} of 
3-D points that describe some salient and identifiable 
points in the scene. They might be the centers of spheres 
(often used as markers), corners where three planes 
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intersect, or the extrema of some protrusion or inden¬ 
tation on a surface. They may be a subset of an initially 
acquired 3-D full scene point set, or they might be ex¬ 
tracted from a range image, or they might be computed 
theoretical points based on extracted data features. 

An early example of 3-D point features and de¬ 
scriptors are spin images, which are used for surface 
matching between point clouds and meshes. Scanned 
3-D points become the vertices of the mesh and con¬ 
nectivity is established by the 3-D scanning geometry. 
A fundamental component of the surface matching rep¬ 
resentation is an oriented point, a three-dimensional 
point with an associated direction. Huber et al. define 
an oriented point O at a surface mesh vertex using the 
3-D position of the vertex p and the surface normal at 
the vertex n [31.53]. Two coordinates can be calculated 
given an oriented point: a the radial distance to the sur¬ 
face normal line L and /3 the axial distance above the 
tangent plane P (Fig 31.13). 

0) (a,/3) 

= (' /\\x-p\\ 2 -[n-(x-p )] 2 .11 ■ (x -p)J 

The term spin map originates from the cylindrical 
symmetry of the oriented point basis; the basis can spin 
about its axis with no effect on the coordinates of points 
with respect to the basis [31.53]. A consequence of the 
cylindrical symmetry is that points that lie on a circle 
that is parallel to P and centered on L will have the same 
coordinates (a, fi) with respect to the basis. 

Seminal work on 3-D point features are the point 
feature histogram (PFH) descriptors. They encode 
a point’s ^-neighborhood geometrical properties by 
generalizing the mean curvature around the point using 
a multidimensional histogram of values [31.54]. This 
highly dimensional hyperspace aims at providing an in¬ 
formative signature for the feature representation, being 
invariant to the six-dimensional (6-D) pose of the un¬ 
derlying surface, and coping very well with different 
sampling densities or noise levels present in the neigh¬ 
borhood [31.55]. 



To formulate the new feature space, the concept of 
a dual-ring neighborhood is first introduced. Following 
the notation and the text of [31 .54], let P be a set of 3-D 
points with x,-, > 7 , z,- being the geometric coordinates. 
A point pi of P is said to have a dual-ring neighborhood 
if 


/'i, 7*2 e Rr 1 < 7*2 , such that 


. pk 1 
. pk 2 


with 0 < k\ < k. 2 . The two radii r\ and r 2 are used to 
determine two distinct layers of feature representations 
for p,. The first layer represents the surface normal at 
the query point from the neighborhood patch P kl . The 
second layer comprises the PFH as a set of angular fea¬ 
tures (Fig. 31.14) 

a = v -n { , 

(Pi -Ps) 

v=u .—^, 

9 = arctan(tu • ;i t , u ■ n t ) . 

In addition to 3-D structural features, 3-D features 
derived from texture, for example, coregistered color 
images or scan reflectivities, are widely used for reg¬ 
istration [31.56, 57]. Figure 31.15 shows scale invariant 
feature transform (SIFT) features extracted from a 3-D 
scanner with calibrated reflectivity values. 



Fig.31.14a,b Angular features in PFHs (a) and the defini¬ 
tion of the two regions (b) 
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Fig. 31.15 Sift features extracted from the reflectivity val¬ 
ues of a 3-D laser scan 


Planes. A planar surface may only be described by 
the infinite surface as given by the equation, but it 
may also include a description of the boundary of the 
surface patch. Convenient representations for robotics 
applications are lists of the 3-D points { (Xj, z/) } that 
form the patch boundary, or polylines, which repre¬ 
sent the boundary by a set of connected line segments. 
A polyline is represented by the sequence of 3-D points 
{(x,,yi,Zi)} that form the vertices that join the line 
segments. 

Plane extraction, or plane fitting, is the problem of 
modeling a given 3-D point cloud as a set of planes 
that ideally explain every data point. The RANSAC al¬ 
gorithm is one possible algorithm. When adapted for 
plane finding, this algorithm selects three 3-D points 
at random (although exploiting some locality to the 
point selection algorithm can improve the efficiency of 
the algorithm). These three points determine a plane 
with a parameter vector a. Test all points \p\ in the 
set for belonging to the plane (\pi - a |< r). If enough 
points are close to the plane, then potentially a plane 
has been found. These points should also be processed 
to find a connected set, from which a more accurate 
set of plane parameters can be estimated using the 
least-squares algorithm given above. If a planar patch 
is successfully found, the points that lie in that plane 
are removed from the dataset. The random selection of 
three points then continues until no more planes are 
found (a bound on how many tries to make can be 
estimated). For example, Schnabel et al. [31.58] have 
adapted RANSAC for plane extraction and found that 
the algorithm performs precise and fast plane extrac¬ 
tion, but only if the parameters have been fine-tuned 
properly. For their optimization, they use information 
, which is not readily available in point cloud data, 
such as normals, neighboring relations, and outlier 
ratios. 


A further standard method for plane detection is 
region growing, based on a seed patch. When the 
scene largely consists of planes, a particularly sim¬ 
ple approach is based on selecting a previously un¬ 
used point and the set of points {pi = (x,-, y,-, z,)} in its 
neighborhood. A plane is fit to these points using the 
least-squares method (cf. normal computation). This 
hypothesized plane then needs to be tested for reason¬ 
ableness by 1) examining the smallest eigenvalue - it 
should be small and of the order of the square of the ex¬ 
pected noise level and 2) ensuring that most of the 3-D 
points in the fitted set lie on the plane (| Pj a | < r). 

Larger planar regions are grown by locating new 
adjacent points pi that lie on the plane (| p,-a |< r). 
When enough of these are found, the parameters a of 
the plane are re-estimated. Points on the detected plane 
are removed and the process is repeated with a new seed 
patch. This process continues until no more points can 
be added. Complete descriptions of planar feature ex¬ 
traction with region growing are given in [31.59]. 

Bauer and Polthier use the radon transform to de¬ 
tect planes in volume data [31.60]. The idea and the 
speed of the algorithm are similar to that of the stan¬ 
dard Hough transform. The Hough transform [31.61] 
is a method for detecting parametrized objects. For the 
Hough transform, planes are represented in the Hesse 
normal form, using normal vectors. A plane is thereby 
given by a point p on the plane, the normal vector n that 
is perpendicular to the plane and the distance p to the 
origin 

p = p n = p x n x + p y n y + p z n z = p . 

Considering the angles between the normal vector and 
the coordinate system, the coordinates of n are factor¬ 
ized to 

p x ■ cos 9 ■ sin (p + p y ■ sin <p ■ sin 9 + p z ■ cos <p = p . 

(31.16) 

with 9 is the angle of the normal vector on the xy- 
plane and (p the angle between the xy-plane and the 
normal vector in the z direction, i p, 9, and p define the 3- 
dimensional Hough space ( 9 , (p , p) such that each point 
in the Hough space corresponds to one plane in M 3 . 
To find planes in a point set, one calculates the Hough 
transform for each point. Given a point p in Cartesian 
coordinates, one finds all planes the point lies on, that 
is, find all the 9, (p, and p that satisfy (31.16). Mark¬ 
ing these points in the Hough space, that is, leads to 
a 3-D sinusoid curve as shown in Fig. 31.16. The inter¬ 
sections of two curves in the Hough space denote the 
planes that are rotated around the line built by the two 
points. Consequently, the intersection of three curves 
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Fig. 31.16 Transformation of three points from R to the 
Hough space (9,(p,p). The intersection of the curves 
{marked in black) depicts the plane spanned by the three 
points 


in the Hough space corresponds to the polar coordi¬ 
nates defining the plane spanned by the three points. 
In Fig. 31.16, the intersection is marked in black. Given 
a set P of points in Cartesian coordinates, one trans¬ 
forms all points p, e P into the Hough space. The more 
curves intersect in hj e (0, <p, p), the more points lie on 
the plane represented by hj and the higher is the proba¬ 
bility that hj is actually extracted from P. 

The standard Hough transform is far too slow for 
plane detection in real-world data. A variant called 
randomized Hough transform is the method of choice 
when dealing with 3-D data due to its exceptional 
performance as far as runtime and quality are con¬ 
cerned [31.62]. 

Other plane extraction algorithms are highly spe¬ 
cialized for a specific application and are not in 
widespread use for miscellaneous reasons. Lakaemper 
and Latecki [31 .63] used an expectation maximization 
(EM) algorithm to fit planes that are initially randomly 
generated, Wulf et al. [31 .64] detected planes relying 
on the specific properties of a sweeping laser scanner 
and Yu et al. [31.65] developed a clustering approach to 
solve the problem. 

Triangulated Surfaces. Most commonly, surfaces are 
approximated by polygonal meshes, particularly tri¬ 


angle meshes, a standard data structure in computer 
graphics to represent 3-D objects. This representation 
describes an object or scene by a set of triangular 
patches. More general polygonal surface patches or 
even various smooth surface representations are also 
used, but triangles are most commonly used because 
they are simpler and there are inexpensive PC graphics 
cards that display triangles at high speed. 

The triangles can be large (e.g., when represent¬ 
ing planar surfaces) or small (e.g., when representing 
curved surfaces). The size chosen for the triangles re¬ 
flects the accuracy desired for representing the object or 
scene surfaces. The triangulated surface might be com¬ 
plete in the sense that all observable scene or objects 
surfaces are represented by triangles, or there might 
be disconnected surface patches with or without inter¬ 
nal holes. For grasping or navigation, you do not want 
any unrepresented scene surface to lie in front of the 
represented portion of the surface where a gripper or 
vehicle might collide with it. Hence, we assume that 
the triangulation algorithms produce patch sets that, if 
completely connected at the edges, implicitly bound 
all real scene surfaces. Figure 31.17 shows an exam¬ 
ple of a triangulated surface and the original point 
cloud. 

The de-facto standard is the marching cubes method 
introduced by Lorensen nad Cline [31.66]. This algo¬ 
rithm subdivides the scanned volume into cubic cells 
or voxels. For each cell, the intersections between the 
cell edges and the surface are calculated. Precalculated 
surface patterns are then used to generate a local trian¬ 
gle mesh approximation. An example of such patterns 
are given in Fig. 31.18. To interpolate the intersections, 
implicit continuous surface representations like planes 
or splines are fitted to the local data using least-squares 
fits [31.67,68]. A feature of the marching cubes algo¬ 
rithm is that it produces more triangles than are needed 
to represent an object. Hence, several mesh simplifi¬ 
cation algorithms have been introduced over the past 
years. Most of them define error metrics that indicate 
the error that a certain operation causes to the model, 
that is, the removal of an edge [31.69, 70]. To optimize 
the model, the edges causing the minimal error to the 



Fig. 31.17 (a) Image acquired with 
a kinect-like sensor, (b) Reconstruc¬ 
tion with kinect fusion 
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Fig. 31.18 Pattern for meshing 3-D 
voxels. 256 combinations are possible, 
but it is sufficient to model 16, iff 
symmetries are considered 


topology are removed iteratively. Since after each edge 
removal new vertices have to be inserted into the mesh, 
the initial topology can be altered. 

The kinect fusion approach modifies Hoppe’s dis¬ 
tance function [31.71]. It exploits the properties of 
the depth image (rectangular) to calculate normals and 
the associated planes. The method is massively paral¬ 
lelized by a GPU (graphics processing unit) implemen¬ 
tation and can reconstruct and register meshes in real 
time. 

3~D Lines. The 3-D lines where planar surfaces meet 
are features that can be easily detected by both stereo 
and range sensors. These features occur commonly in 
built environments (e.g., where walls, floors, ceilings, 
and doorways meet, around the edges of wall structures 
like notice boards, at the edges of office and warehouse 
furniture, etc.). They are also common on manmade ob¬ 
jects. In the case of stereo, changes of surface shape or 
coloring are detected as edges, which can be matched in 
the stereo process to directly produce the 3-D edge. In 
the case of a range sensor, planar surfaces can be easily 
extracted from the range data (see the next section) and 
adjacent planar surfaces can be intersected to give the 
edges. 

The most straightforward representation for 3-D 
lines is the set of points x =p + Xv for all A, where v 
is a unit vector. This has 5 degrees of freedom; more 


complex representations for example, with 4 degrees of 
freedom exist [31.10]. 

Voxels. The voxel (volume pixel) approach represents 
the 3-D world by 3-D boxes/cells that indicate where 
there is a scene structure and where there is free space. 
The simplest representation is a 3-D binary array, en¬ 
coded as 1 for having a structure and 0 for free space. 
This can be quite memory intensive, and also requires 
a lot of computation to check many voxels for content. 
A more complex but more compact representation is 
the hierarchical representation called the octree [31.72]. 
This divides the entire (bounded) rectangular space into 
eight rectangular subspaces called octants (Fig. 31.19). 
A tree data structure encodes the content of each octant 
as empty, full or mixed. Mixed octants are then sub¬ 
divided into eight smaller rectangular octants, encoded 
as subtrees of the larger tree. Subdivision continues un¬ 
til some minimum octant size is reached. Determining 
whether a voxel is empty, full, or mixed depends on the 
sensor used, however, if no 3-D data points are located 
in the volume of a voxel, then it is likely to be empty. 
Similarly, if many 3-D points are present, then the voxel 
is likely to be full. Currently, many implementations 
using octrees for range data are available [31.73,74], 
Furthermore, these voxel representations are the basis 
of surface/mesh reconstruction algorithms as described 
earlier. 
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For the purpose of robot navigation, localization or 
grasping, only the surface and free-space voxels need to 
be marked accurately. The interior of objects and scene 
structure are largely irrelevant. 

Straight Lines. While straight lines are common in 
man-made scenes, direct extraction from 3-D datasets 
is not easy. The main source of the difficulty is that 
the 3-D sensors often do not acquire good responses 
at edges of surfaces. For this reason, most 3-D line de¬ 
tection algorithms are indirect, whereby planes are first 
detected, for example, using the method of the previous 
section, and then adjacent planes are intersected. Adja¬ 
cency can be tested by finding paths of connected pixels 
that lead from one plane to the other. If planes 1 and 2 
contains points p 1 and pi and have surface normals 11 1 
and tii, respectively, then the resulting intersection line 
has equation x = a + Xd where a is a point on the line 
and d = "t x " 2 j s the line direction. 

||«iXm 2 ]| 

There are an infinite number of possible points a, 
which can be found by solving the equations a'ti\ = 
p[n\ and a'ni = p' n iii. A reasonable third constraint 
that obtains a point near pi is the equation a'd = pl,d. 
This gives us an infinite line. Most practical applica¬ 
tions require a finite segment. The endpoints can be 
estimated by (1) finding the points on the line that 
lie close to observed points in both planes and then 
(2) finding the two extremes of those points. On the 
other hand, finding straight 3-D lines can be easier 
with a stereo sensor, as these result from matching two 
straight 2-D image lines. 

31.3.3 Multiple-View Registration 

A globally consistent representation of a robot’s en¬ 
vironment is crucial for many robotic applications. 
Equipped with a 3-D depth-perceiving sensor, many 
mobile systems gather spatial information about their 



Fig. 31.19 (a) Spatial subdivisions of an octree up to 
level 3. Occupied leaf nodes are shaded grey, (b) The cor¬ 
responding tree structure of the sparse data structure 


local 3-D environments. Any iterative application of 
matching algorithms leads to inconsistencies due to 
sensing errors and due to the inaccuracies in the 
matching procedures itself. To avoid these problems, 
global matching algorithms are needed, taking global 
correspondences between range sensor data into ac¬ 
count. Simultaneous localization and mapping (SLAM) 
algorithms as discussed in Chap. 46 solve a very 
similar problem. In addition to registering multiple 
views, they estimate a map. Multiple-view registra¬ 
tion also relates to bundle adjustment in the pho- 
togrammetry community and structure from motion 
(SFM). 

If /7-views have to be registered, any sequential ap¬ 
plication of a two point-set registration method will 
accumulate errors, and therefore the registration algo¬ 
rithm (Sect. 31.3. 1) has to be extended. The global error 
function becomes 

£ = || (R irn u + ti) - ( R k d kJ + 4 ) II 2 , 

t-Hfc i 

(31.17) 

where all views have their unique pose (R. t). After the 
point pairs for all overlapping views (/, k ) have been 
found, (31.17) is minimized. Unfortunately, a closed- 
form solution for minimizing (31.17) is not know, but 
a small angle approximation or the helix transform 
yield a system of linear equations that can be solved by 
Choleskey decomposition [31.49]. In an ICP-like fash¬ 
ion after every transformation new point pairs have to 
be found (Algorithm 31.2). 


Algorithm 31.2 The globally consistent ICP algo¬ 
rithm 

1 : for i = 0 to maxlterations do 

2: find the closest point within a range d mm of every 

pair of overlapping 3-D point clouds (/, k). 

3: Calculate n transformations (R, t) simultane¬ 

ously that minimize the error function (31.17) 

4: Apply the n transformations found in step 4 to all 

data sets. 

5: Compute the difference of the quadratic error, 

that is, compute the difference of the value 
||£,-_i — Ef\\ before and after the application of 
the transformation. If this difference falls below 
a threshold e, terminate. 

6: end for 


A probabilistic SLAM-like notation of (31.17) was 
formulated in [31.75] for 2-D range scans. For each 
pose X, the term X denotes a pose estimate, and AX 
is the pose error. The positional error of two poses Xj 
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and is described by 

m 

Ej,k = ^ ||-X/ ® di — Xk ® ni/1 2 . 

/=l 

Here, ® is the compounding operation that transforms 
a point into the global coordinate system. For small 
pose differences, Ej^ can be linearized by using a Tay¬ 
lor expansion. With the linearized error metric E' k and 
the Gaussian distribution (£/,*, C/,*.) a Mahalanobis dis¬ 
tance that describes the global error of all the poses is 
constructed 

W = J2 (Ej.k-tf.kf (K k ~ E j,k) 

j^k 

= E[^-( X J~X' k )]C- k l [E' uk -(X'-X' k )\ , 

j *k 

(31.18) 

which can be solved efficiently using iterative least 
squares techniques, for example, Levenberg-Marquadt 
or conjugate gradient methods [31.76,77]. The covari¬ 
ances are computed from point pairs. In the presence 
of correct covariances, (31.18) is minimized once. 
In case of scan matching, new pose estimates yield 
new closest point pairs and in turn new covariances. 
Iterating the process of calculating point pairs and 
minimization yields a stable algorithm that converges 
rapidly. The probabilisty notation and the global ICP 
notation are very similar [31.49]. The solution for 

2- D range scans has been extended to to 6-DOF 
in [31.78]. 

31.3.4 Model Matching 

Model matching is the process of matching some stored 
representation to some observed data. In the case dis¬ 
cussed here, we assume that both are 3-D representa¬ 
tions. Furthermore, we assume that the representations 
being matched are both of the same type, for example, 

3- D model and scene lines. (While different types of 
data can also be matched, we ignore these more spe¬ 
cialized algorithms here.) 

A special case of matching is when the two struc¬ 
tures being matched are both scene or model surfaces. 
The algorithm used for matching depends on the com¬ 
plexity the structures being matched. If the structures 
being matched are extended geometric entities such as 
planes or 3-D lines, then a discrete matching algorithm 
like the Interpretation Tree algorithm [31.79] can be 
used. It is suitable for matching small numbers (e.g., 
less than about 20—30) discrete objects, such as ver¬ 
tical edges seen in 2-D or 3-D. If there are M model 
and D data objects, then potentially there are M° dif¬ 


ferent matches. The key to efficient matching is to 
identify pairwise constraints that eliminate unsuitable 
matches. Constraints between pairs of model features 
and pairs of data features also greatly reduce the match¬ 
ing space. If the constraints eliminate enough features, 
a polynomial time algorithm results. The core of the 
algorithm is defined as follows. Let {m,} and {dj} be 
the sets po model and data features to be matched, 
u(mj,dj) is true if m, and dj are compatible features, 
b(m,, irij , 4.4) is true if the four model and data fea¬ 
tures are compatible and T is the minimum number of 
matched features before a successful match is declared. 
Pairs is the set of successfully matched features. The 
function truesizeof counts the number of actual 
matches in the set, disregarding matches with the wild¬ 
card * which matches anything. 


Algorithm 31.3 

pairs=it(0,{}) 

if truesizeof(pairs) >= T, then success 

function pairs=it(level,inpairs) 

if level >= T, then return inpairs 
if M-level+truesizeof(inpairs) < T 
then return {} % can never succeed 
for each d_i % loopD start 
if not u(m_level,d_i), then 

continue loopD 

for each (m_k,d_l) in inpairs 
if not b(m_level,m_k,d_i,d_l) 
then continue loopD 
endfor 

% have found a~successful new pair 

% to add 

pairs = it(level+l, 

union(inpairs, (m_level, d_i))) 
if truesizeof(pairs) >= T, then return 
endfor % loopD end 

% no success, so try wildcard 
it(level+1,union(inpairs,(m_level,*))) 


31.3.5 Relative Pose Estimation 

Central to many tasks is the estimation of the coordinate 
system relative position or pose transformation between 
two coordinate systems. For example, this might be the 
pose of a scanner mounted on a mobile vehicle relative 
to scene landmarks. Or, it might be the relative pose 
of some scene features as observed in two views taken 
from different positions. 

We present here three algorithms that cover most 
instances of the pose estimation process, which differ 
slightly based on the type of feature being matched. 
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Point Set Relative Pose Estimation 
The ICP algorithm can be used relative pose estimation 
as well (Sect. 31.3.1). 

Straight Line Relative Pose Estimation 
If 3-D lines are the features that are extracted, then the 
relative pose transformation can be estimated as fol¬ 
lows. Assume N paired lines. The first set of lines is 
described by direction vectors {e,} and a point on each 
line {a,}. The second set of lines is described by di¬ 
rection vectors {/,-} and a point on each line {ft,}. In 
this algorithm, we assume that the direction vectors 
on the matched segments always point the same direc¬ 
tion (i. e., are not inverted). This can be achieved by 
exploiting some scene constraints, or trying all com¬ 
binations and eliminating inconsistent solutions. The 
points a, and ft, need not correspond to the same point 
after alignment. The desired rotation matrix R mini¬ 
mizes JT 111 1 2 . Construct the 3 x N matrices 
E that consists of the vectors {e,} stacked up. Con¬ 
struct the 3 x N matrices F in a similar way from the 
vectors {/,-}. Compute the singular value decomposi¬ 
tion svd(FE / ) = U / DV / . Compute the rotation matrix 
R = VU / . The translation estimate t minimizes the sum 
of the square of the distances A,- between the rotated 
points a, and corresponding line (/,.ft,). Define ma¬ 
trix L = £T(I— ////(I— /;). Define the vector n = 
I];(I-//D / ( I -/i/T)(R fl ;-^)- Then the translation is 
t = — L ~ 1 n. 

Plane Relative Pose Estimation 
Finally, if planes are the 3-D features extracted for 
matching, then the relative pose transformation can be 
estimated as follows. Assume N paired planes. The first 
set of planes is described by surface normals {e,} and 
a point on each plane {a,-}. The second set of planes 
is described by surface normals {/,-} and a point on 
each plane {ft,}. Here we assume that the surface nor¬ 
mals always point outward from the surface. The points 
a, and ft,- need not correspond to the same point af¬ 
ter alignment. The desired rotation matrix R minimizes 
JT || Re,-—/,- || 2 . Construct the 3 x N matrices E that 
consists of the vectors [e,| stacked up. Construct the 
3 x A matrices F in a similar way from the vectors {/, }. 
Compute the singular value decomposition svd(FE / ) = 
U / DV / [31.45]. Compute the rotation matrix R = VU'. 
The translation estimate t minimizes the sum of the 
square of the distances A, between the rotated point 
a, and the corresponding plane (/,, ft,). Define matrix 
L = £;//;• Define the vector n = E,///Ra, -ft,). 
Then the translation is t = —L ~ l n. 

In all of the calculations described above, we as¬ 
sumed normally distributed errors. For techniques to 
robustify these sorts of calculations, see Zhang [31.80]. 


31.3.6 3-D Applications 

This section links the techniques presented above to 
the robotics applications of 3-D localization of parts 
for robot manipulation, self-localization of robot vehi¬ 
cles and scene understanding for robot navigation. The 
robotics tasks mentioned here are discussed in more de¬ 
tail in other chapters in the series. While this chapter 
focusses on robotics applications, there are many other 
3-D sensing applications. An area of much current re¬ 
search is that of acquiring 3-D models, particularly for 
reverse engineering of mechanical parts [31.81], histor¬ 
ical artifacts [31.82], buildings [31.83] and people for 
computer games and movies (Cyberware Whole Body 
X 3-D Scanner). 

The key tasks in robot manipulation are: 

1. Identification of grasping points (Chaps. 37 and 38), 

2. Identification of a collision free grasp (Chaps. 37 
and 38), 

3. Recognition of parts to be manipulated (Chap. 32) 
and 

4. Position estimation of parts for manipulation 
(Chaps. 32 and 42). 

The key tasks in robot navigation and self¬ 
localization are: 

4. Identification of a navigable groundplane (Sect. 
31.4), 

5. Identification of a collision free path (Chap. 47), 

6. Identification of landmarks (Chap. 45) and 

7. Estimation of vehicle location (Chap. 53). 

The mobile and assembly robotics tasks link to¬ 
gether rather naturally. Tasks 1 and 5 have a connection, 
when we consider these tasks in the context of un¬ 
known parts or paths. Part grasping requires finding 
regions on a part that are graspable, which usually 
means locally planar patches that are large enough that 
a gripper can make good contact with them. Similarly, 
navigation usually requires smooth ground regions that 
are large enough for the vehicle - again locally pla¬ 
nar patches. Both tasks are commonly based on tri¬ 
angulated scene methods to represent the data, from 
which connected regions of nearly coplanar patches can 
be extracted. The main difference between these two 
tasks is the groundplane detection task is looking for 
a larger patch, that must be on the ground and upward 
facing. 

Tasks 2 and 6 require a method of representing 
empty space along the proposed trajectory of the grip¬ 
per contacts or the vehicle. The voxel representation is 
good for this task. 
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Tasks 3 and 7 are model matching tasks and can use 
the methods of Sect. 31.3.3 to match observed scene 
features to prestored models of known parts or scene 
locations. Commonly used features are large planar sur¬ 
faces, 3-D edges and 3-D feature points. 


Tasks 4 and 8 are pose estimation tasks and can use 
the methods of Sect. 31.3.5 to estimate the pose of the 
object relative to the sensor or vehicle (i. e., sensor) rel¬ 
ative to the scene. Again, commonly used features are 
large planar surfaces, 3-D edges and 3-D feature points. 


31.4 Navigation and Terrain Classification and Mapping 


One of the more compelling uses for range data is for 
navigation of mobile robot vehicles. Range data pro¬ 
vides information about obstacles and free space for 
the vehicle, in a direct geometric form. Because of the 
real-time constraints of navigation, it is often impracti¬ 
cal to reconstruct a full 3-D model of the terrain using 
the techniques presented in this chapter. Instead, most 
systems use an elevation model. An elevation model 
is a tesselated 2-D representation of space, where at 
each cell there is information about the distribution of 
3-D points in the cell. In its simplest incarnation, the 
elevation map just contains the mean height of range 
points above the nominal ground plane (Fig. 31.20). 
This representation is sufficient for some indoor and 
urban environments; more sophisticated versions that 
determine a local plane, scatter of points in the cell, etc., 
are useful for more complicated off-road driving. Ele¬ 
vation maps marked with obstacles have obvious utility 
for planning a collision-free path for the vehicle. 

31.4.1 Mobile Mapping 

Laser range scanning provides an efficient way to ac¬ 
tively acquire accurate and dense 3-D point clouds of 
object surfaces or environments. Mobile scanning is 
currently used for modeling in architecture and agri¬ 
culture as well as urban and regional planning. Modem 
systems like the Riegl VMX-450 and the Lynx mobile 
mapper as produced by Optech work along the same 
basic principal. They combine a highly accurate global 



Fig. 31.20 Elevation map in urban terrain. Each cell holds the 
height of the terrain at that point. More extensive features can also 
be incorporated: slope, point variance, etc. 


positioning system (GPS), a high precision inertial mea¬ 
surement unit (IMU) and the odometry of the vehicle to 
compute the fully timestamped trajectory. Using a pro¬ 
cess called motion compensation this trajectory is then 
used to unwind the laser range measurements that were 
acquired by the 2-D laser scanner also mounted on the 
vehicle. The quality of the resulting point cloud de¬ 
pends on several factors: 

• The calibration of the entire system, that is, the ac¬ 
curacy to which the position and orientation of each 
individual sensor in relation to the vehicle has been 
determined. 

• The accuracy of the external positioning sensors, 
that is, the GPS, IMU and odometry. 

• The availability of the GPS, as it may suffer tem¬ 
porary blackouts under bridges, in tunnels and be¬ 
tween high-rises. 

• The accuracy of the laser scanner itself. 

The movement of the mobile laser scanner between 
time to and t n creates a trajectory T = {Vo. •.., V,,}, 
where V; = {t Xti , t y j, t z j, 9 x j, 0 y j, is the 6-DOF 
(degree of freedom) pose of the vehicle at time with 
to < ti < t n . Using the trajectory of the vehicle a 3-D 
representation of the environment can be obtained by 
unwinding the laser measurements M to create the final 
map P. However, sensor errors in odometry, IMU and 
GPS as well as systematic calibration errors and the 
accumulation of pose errors during temporary GPS 
outages degrade the accuracy of the trajectory and 
therefore the point cloud quality. Furthermore, please 
note that modern systems easily create thouthands of 
3-D scan slices. 

These mobile mapping systems extend the early 
work on 3-D reconstruction with two 2-D lasers un¬ 
der exploitation of the robot motion [31.84]. One laser 
scanner is scanning horizontal, one is mounted verti¬ 
cally. The resultant point cloud is typically registered 
using the pose of the robot as corrected by the 2-D 
SLAM algorithm, rather than any of the 3-D registration 
techniques covered in this chapter. The current state of 
the art developed by Bosse et al. [31.85] for improving 
overall map quality of mobile mappers in the robotics 
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Fig. 31.21 (a) Mobile mapping car and (b) Zedebee hand¬ 
held mapper, (c) 3-D point cloud acquired with the Riegl 
VMX-250 system, (d) 3-D point cloud from Zedebee 


community is to coarsely discretize the time. This re¬ 
sults in a partition of the trajectory into subscans that are 
treated rigidly. Then rigid registration algorithms like 
the ICP and other solutions to the SLAM problem are 
employed. 

Figure 31.21 shows a car equipped with the Riegl 
VMX-450 mobile mapping system of the company 
Riegl laser measurement systems and the Zebedee 
handhald mapper from CSIRO. Figure 31.21 shows typ¬ 
ical resulting 3-D point clouds as well. 

31.4.2 Urban Navigation 

In urban navigation, the environment is structured, with 
roads, buildings, sidewalks, and also moving objects - 
people and other vehicles. There are two main chal¬ 
lenges: how to register laser scans from a fast-moving 
vehicle for consistent mapping, and how to detect mov¬ 
ing objects using range scans (of course, other methods 



Fig. 31.22 Elevation map of an urban scene, using 10 x 
10 cm cells. Obstacles in red, ground plane in green (af¬ 
ter [31.88]) 


are also used for detecting moving objects, for example, 
appearance-based vision). 

Outdoor vehicles can use precision GPS, inertial 
measurement units, and wheel odometry to keep track 
of their position and orientation, typically with an ex¬ 
tended Kalman filter. This method is good enough 
to obviate the need for precise registration matching 
among scans, as long as the motion model of the vehi¬ 
cle, and timing from the range scanner, is used to place 
each scan reading in its proper position in the world 
model. This method also works in relatively easy off¬ 
road terrain such as in the DARPA (Defense Advanced 
Research Projects Agency) Grand Challenge [31 .86]. In 
all cases, the reduction of pose estimation error is criti¬ 
cal for good performance [31.87]. 

Once scan readings are registered using the vehi¬ 
cle pose estimation, they can be put into an elevation 
map, and obstacles detected using the slope and verti¬ 
cal extent of the range readings in the cells of the map. 
A complication is that there may be multiple levels of 
elevation in an urban setting, for example, an overpass 
would not be an obstacle if it were high enough. One 
proposal is to use multiple elevation clusters within 
each cell; this technique is called a multilevel surface 
map (MLS, [31.88]). Each cell in the map stores a set 
of surfaces represented by a mean height and variance. 
Figure 31.22 shows an MLS with a cell size of 10 cm 2 , 
with ground plane and obstacles marked. 

For dynamic objects, real-time stereo at 15—30FIz 
can capture the motion of the objects. When the stereo 
rig is fixed, range background subtraction isolates just 
the moving objects [31 .89]. When the rig is on a moving 
vehicle, the problem is more difficult, since the whole 
scene is moving with respect to the rig. It can be solved 
by estimating the motion of the rig with respect to the 
dominant rigid background of the scene. Let R, t be the 
motion of the rig between two frames, estimated by 
extracting features and matching them across the two 
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Fig.31.23a-c Independent motion detection from a moving platform. Reference image on panel (a) is forward-projected 
using the motion homography to the image on panel (b); image on panel (c) is the difference with the actual image 


temporal frames, using the techniques of Chap. 46. The 
homography H(R,t ) of (31.11) provides a direct pro¬ 
jection of the disparity vectors po = [xo, yo- do, 1] of the 
first frame to their correspondences H(R, t )po under R, t 
in the second frame. Using the homography allows the 
points in the reference frame to be directly projected 
onto the next frame, without translating to 3-D points. 
Figure 31.23 shows the projected pixels under rigid mo¬ 
tion from a reference scene. The difference between 
the projected and actual pixels gives the independently 
moving objects (from [31.90]). 

31.4.3 Rough Terrain 

Rough outdoor terrain presents two challenges: 

• There may be no extensive ground plane to charac¬ 
terize driveability and obstacles. 

• Vegetation that is pliable and driveable may appear 
as an obstacle in range images. 


Figure 31.24 shows a typical outdoor scene, with 
a small (1 m) robot driving through vegetation and 
rough ground [31.91]. Range data from stereo vision 



Fig. 31.24 Rough terrain, no ground plane, driveable veg- 
atation 


on the robot will see the top of the vegetation and some 
ground points below. The elevation model can be ex¬ 
tended to look at point statistics within each cell, to 
capture the notion of a local ground plane and pene¬ 
trability related to vegetation. In [31.92], for example, 
the set of proposed features includes: 

• Major plane slope using a robust fit (Sect. 31.3.2, 
3-D Feature Representations and Extraction) 

• Height difference of max and min heights 

• Points above the major plane 

• Density: ratio of points in the cell to rays that pass 
through the cell. 

The density feature is interesting (and expensive to 
compute), and attempts to characterize vegetation such 
a grass or bushes, by looking at whether range readings 
penetrate an elevation cell. The idea of using vegeta¬ 
tion permeability to range readings has been discussed 
in several other projects on off-road driving [31 .94—96]. 

Elevation map cells can be characterized as ob¬ 
stacles or driveable through learning or hand-built 
classifiers. Among the learning techniques are neu¬ 
ral nets [31.92] and Gaussian mixture models with 



Fig. 31.25 Classification using point statistics. Red is pla¬ 
nar surface, blue is thin linear surface, green is scattered 
penetrable surface (after [31.93]) 
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expectation-maximization learning [31.93]. The latter 
work also includes a lower level of interpretation, clas¬ 
sifying surfaces into planar patches (ground plane, solid 
obstacles), linear features (telephone wires), and scat¬ 
tered features (vegetation). Figure 31.25 shows some 
results from a laser-scanned outdoor scene. Linear fea¬ 
tures such as telephone wires and the telephone pole are 
accurately determined, as well as vegetation with high 
penetrability. 

Some additional problems occur in rough-terrain 
navigation. For planar laser rangefinders that are swept 
over the terrain by vehicle motion, the precision of vehi¬ 
cle pose estimation is important for accurate reconstruc¬ 
tion. Attitude errors of less than 0.5° can cause false 


positives in obstacle detection, especially for sweeps far 
ahead of the vehicle. In [31.87], this problem is solved 
by looking at the time of each laser reading, and noting 
a correlation between height errors and time difference 
in the readings. 

Negative obstacles (ditches and cliffs) are difficult 
to detect with range information, because the sensor 
may not see the bottom of the obstacle. This is espe¬ 
cially true for vehicle-mounted sensors that are not very 
high off the ground, and that are looking far ahead. Neg¬ 
ative obstacles can be inferred when there is a gap in the 
ground plane, and a plane slanted upward at the back 
edge of the gap. Such artifacts can be efficiently found 
using column search on the disparity image [31.97]. 


31.5 Conclusions and Further Reading 

Range sensing is an active and expanding field of re¬ 
search in robotics. The presence of new types of de- n 
vices - flash ladars, multibeam ladars, on-camera stereo ii 
processing - and the continuing development of robust 1< 
algorithms for object reconstruction, localization and a 
mapping has helped to bring applications out of the lab- fi 
oratory and into the real world. Indoor navigation with fi 
ladars is already being exploited in commercial products 1 
(for example, [31.98]). As the basic capabilities become s 
more robust, researchers are looking to perform useful t; 
tasks, such as fetching items or doing dishes [31 .99]. o 


Another set of challenges are found in less be¬ 
nign environments, such as urban and off-road driv¬ 
ing (DARPA Grand Challenge and Urban Chal¬ 
lenge [31.86]). Stereo vision and laser rangefinding 
also will play a role in helping to provide autonomy 
for a new generation of more-capable robotic plat¬ 
forms that rely on walking for locomotion [31.100]. 
The challenges are dealing with motion that is less 
smooth than wheeled platforms, environments that con¬ 
tain dynamic obstacles, and task-oriented recognition of 
objects. 
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32. 3-D Vision for Navigation 

and Grasping 


Danica Kragic, Kostas Daniilidis 


In this chapter, we describe algorithms for 
three-dimensional (3-D) vision that help robots 
accomplish navigation and grasping. To model 
cameras, we start with the basics of perspective 
projection and distortion due to lenses. This pro¬ 
jection from a 3-D world to a two-dimensional 
(2-D) image can be inverted only by using infor¬ 
mation from the world or multiple 2-D views. If we 
know the 3-D model of an object or the location 
of 3-D landmarks, we can solve the pose estima¬ 
tion problem from one view. When two views are 
available, we can compute the 3-D motion and 
triangulate to reconstruct the world up to a scale 
factor. When multiple views are given either as 
sparse viewpoints ora continuous incoming video, 
then the robot path can be computer and point 
tracks can yield a sparse 3-D representation of the 
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world. In order to grasp objects, we can estimate 
3-D pose of the end effector or 3-D coordinates of 
the graspable points on the object. 


With the rapid progress and cost reduction in digital 
imaging, cameras became the standard and probably the 
cheapest sensor on a robot. Unlike positioning (global 
positioning system - GPS), inertial (IMU), and dis¬ 
tance sensors (sonar, laser, infrared) cameras produce 
the highest bandwidth of data. Exploiting information 
useful for a robot from such a bit stream is less explicit 
than in case of GPS or a laser scanner but semanti¬ 
cally richer. In the years since the first edition of the 
handbook, we had significant advances in hardware 
and algorithms. RGB-D sensors like the Primesense 
Kinect enabled a new generation of full model re¬ 
construction systems [32.1] with an arbitrary camera 
motion. Google’s project Tango [32.2] established the 
state of the art in visual odometry using the latest fu¬ 
sion methods between visual and inertial data ([32.3] 
and 3-D modeling became a commod¬ 

ity software (see, for example, 123D Catch App from 


Autodesk) and the widely used open source Bundler 
([32.4] loiMliiliiliM) has been possible by advances in 
wide baseline matching and bundle adjustment. Meth¬ 
ods for wide baseline matching have been proposed 
for several variations of pose and structure from mo¬ 
tion [32.5]. Last, the problem of local minima for non- 
minimal overconstrained solvers has been addressed by 
a group of method using Branch and Bound global 
optimization of a sum of fractions subject to convex 
constraints [32.6] or an Loo-norm of the error func¬ 
tion [32.7]. 

Let us consider the two main robot perception domains: 
navigation and grasping. Assume for example the sce¬ 
nario that a robot vehicle is given the task of going 
from place A to place B given as instruction only inter¬ 
mediate visual landmarks and/or GPS waypoints. The 
robot starts at A and has to decide where is a driv- 
able path. Such a decision can be accomplished through 
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the detection of obstacles from at least two images by 
estimating a depth or occupancy map with a stereo al¬ 
gorithm. While driving, the robot wants to estimate its 
trajectory which can be accomplished with a matching 
and structure from motion algorithm. The result of the 
trajectory can be used to build a lay out of the environ¬ 
ment through dens matching and triangulation which 
in turn can be used as a reference for a subsequent 
pose estimation. At each time instance the robot has to 
parse the surrounding environment for risks like pedes¬ 
trians, or for objects it is searching for like a trash-can. 
It has to become aware of loop closing or a reentry if 
the robot has been kidnaped or blind for a while. This 
can be accomplished through object and scene recogni¬ 


32.1 Geometric Vision 

Let us start by introducing the projection of the world 
to an image plane. Assume that a point in the world 
(X, 7, Z) has coordinates (X ci , Y ci ,Z ci ) with respect to 
the coordinate system of a camera c, related to each 
other by the following transformation 

X ci \ fx\ 

Y ci =R, 7 +T,, (32.1) 

ZciJ \z) 

where R, is a rotation matrix whose columns are the 
world axes with respect to the camera. The translation 
vector Tj is starting from the origin of the camera and 
ending at the origin of the world coordinate system. 
The rotation matrix is orthogonal R T R = 1 with deter¬ 
minant one. We assume that the center of projection is 
the origin of the coordinate system and that the optical 
axis is the Z ci axis of the camera. If we assume that the 
image plane is the plane Z ci = 1 then the image coordi¬ 
nates (xi, yj) read 

X c j Y ci 

Xi = ^, yi = y~- < 32 - 2 > 

In practice, what we measure are the pixel coordinates 
( Uj , Vi) in the image which are related to image coordi¬ 
nates ( xi,yi ) with the affine transformation 

Uj = faxi + + c u , Vi =fy t + c v , (32.3) 

where / is the distance of the image plane to the 
projection center measured in pixels. It is also called fo¬ 
cal length, because they are considered approximately 
equal. The aspect ratio cr is a scaling induced by 
nonsquare sensor cells or different sampling rates hor¬ 
izontally and vertically. The skew factor f} accounts 


tion yielding the what and where of objects around the 
robot. In an extreme scenario, a vehicle can be left to 
explore a city and build a semantic 3-D map as well as 
a trajectory of all places it visited, the ultimate visual 
simultaneous localization and semantic mapping prob¬ 
lem. In the case of grasping, the robot detects an object 
given a learnt representation, and subsequently, it has 
to estimate the pose of the object and in some cases its 
shape by triangulation. When a camera is not mounted 
on an end-effector, the absolute orientation between the 
hand the object has to be found. 

In the next section we will present the geometric 
foundations for 3-D vision and in the last section we 
describe approaches for grasping. 


for a shearing induced by a nonperfectly frontal image 
plane. The image center c u , c v is the point of intersec¬ 
tion of the image plane with the optical axis called the 
image center. These five parameters are called intrinsic 
parameters and the process of recovering them is called 
intrinsic calibration. Upon recovering them we can talk 
about a calibrated system and we can work with the im¬ 
age coordinates (x,-. >’/) instead of the pixel coordinates 
(ui, Vi). In many vision systems in particular on mobile 
robots, wide-angle lenses introduce a radial distortion 
around the image center which can be modelled poly- 
nomially 

xf‘ sl = x,( 1 + k\r + k 2 r + k 2 r 3 + ...) 

yf lst = y,(l + hr + k 2 r 2 + k 2 r 3 + ...) 
where r 2 = xf + y 2 , 

where we temporarily assumed that the image cen¬ 
ter is at (0,0). The image coordinates y,) in 
((32.3)) have to be replaced with the distorted coordi¬ 
nates (x 11181 ,/ 1181 ). 

32.1.1 Calibration 

Recovering the intrinsic parameters when we can make 
multiple views of a reference pattern like a checker 
board without variation of the intrinsic parameters 
has become a standard procedure using tools like the 
MATLAB calibration toolbox or Zhang’ s OpenCV cal¬ 
ibration function [32.8]. When intrinsics like the focal 
length vary during operation and viewing reference pat¬ 
terns is not practically feasible, we rely on the state 
of the art method by Pollefeys et al. [32.9, 10]. When 
all intrinsic are unknown on the Kruppa equations and 
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several stratified self-calibration approaches [32.11,12] 
which require at least three views. Apart radial dis¬ 
tortion, the projection relations shown above can be 
summarized in matrix form. By denoting 

Ui = ( Uj , Vi, 1 ) 

and 

X= (X , Y, Z, 1) 
we obtain 

A,Hi = K,- (R, Ti)X=PX , (32.4) 

where A,- = Z ci is the depth of point X in camera coordi¬ 
nates and P is the 3x4 projection matrix. The depth A, 
which can be eliminated to obtain two equations relat¬ 
ing the world to the pixel coordinates. 

32.1.2 Pose Estimation or PnP 

When we have landmarks in the world with known 
positions X and we can measure their projections, the 
problem of recovering the unknown rotation and trans¬ 
lation in the calibrated case is called pose estimation 
or the Prespective-n-Point problem (PnP). Of course, 
it presumes the identification of the world points in 
the image. In robotics, the pose estimation is a vari¬ 
ant of the localization problem in a known environment. 
When grasping objects of known shape PnP yields the 
target pose for an end-effector module the grasping 
point positions. We assume that a camera is cali¬ 
brated and that measurements of N points are given in 
world coordinates Xj= i.jv and calibrated image coor¬ 
dinates Xj=i,,N- Let us assume two scene points and 
denote the known angle between their projections jci 
and *2 as 8 12 (Fig. 32.1). Let us denote the squared dis¬ 
tance ||X,' — Ajjl 2 with and the lengths of Xj with d 2 . 
Then cosine law reads 

d 2 + d 2 — 2dfdj cos 8y = d 2 - . (32.5) 

If we can recover d, and dj the rest will be an absolute 
orientation problem 

djXj = RAQ + T (32.6) 

to recover translation and rotation between camera and 
world coordinate system. 

Minimal Solution 

The cosine law has two unknowns d\ and d 2 so with 
three points we should be able to solve for the pose es¬ 
timation problem. Indeed, three points yield a system of 


three quadratic equations in three unknowns, so it will 
have a maximum of eight solutions. 

We follow here the analysis of the classic solution 
in [32.13] and set do = ud\ and d^ = vd\ and solve all 
three equations for d\ 


1 u 2 + v 2 — 2uv cos 823 

.2 _ __ 

1 1 + v 2 — 2i/cosi5i3 


1 u 2 T 1 — 2ucos8\2 

which is equivalent to two quadratic equations in u 
and v 

d 2 2 (1 + v 2 — 2?;cos<5i3) 

= dl 3 (u 2 -I- 1 — 2mcos<5i2) , (32.7) 

d 2 3 (u 2 + v 2 — 2uv cos 8 23 ) 

= dj 3 (l + v 2 —2v cos<5 i 3 ) . (32.8) 

Solving (32.8) for u 2 and substituting in (32.7) allows 
solving El for 11 because u appears linearly. Substitut¬ 
ing u back in (32.8) yields a quartic in v which can 
have as many as four real roots. For each v we obtain 
two roots for u through any of the quadratic equations 
yielding a maximum of eight solutions [32.13, 14]. Pop¬ 
ular pose estimation algorithms are based either on an 
iterative method [32.15, 16] or linear versions using 
auxiliary unknowns of higher dimension [32.17, 18]. 

A more recent method [32.19] for n world points ex¬ 
presses 3-D points as the barycentric coordinates with 



Fig. 32.1 Pose estimation problem: A camera seeing 3 
points at unknown distances di, cfj. an d d 3 with known 
angles between the rays and known point distances 
d12.d13.d23 
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respect to four virtual control points 


If the singular value decomposition (SVD) of 


4 

•V, !>/</• 


i= 1 


4 

where ay = 1 . 
i= 1 


(h[ h' 2 h\ x h' 2 ) = USV T , 

then the solution is [32.20] 


A rigid transformation to the camera coordinate system 
leaves the barycentric coordinates invariant and a per¬ 
spective projection yields 

4 

A/X/ ^ ' (%ij (Xch he,■ z ci ) 

1=1 

Eliminating A, yields two linear equations for each 
point 

4 

^ a ijCjc c j = ol ijXiC Zc j 

1=1 

4 

) , a ijCyci = a yf’,C Zcj . 

J =1 

with the coordinate triples of the control points in the 
camera frame being the 12 unknowns. This is a lin¬ 
ear homogeneous system with the solution being the 
nullspace of a 2« x 12 matrix. The unknown control 
points are found up to a scale factor which is easily fixed 
because we know the inter point distances. The pose is 
found from absolute orientation between control points 
in the camera and the world frame. This yields a very 
efficient solution for n > 6 points but leaves you with 
the initial choice of the control points as a factor affect¬ 
ing the solution. 

In case that n > 4 points lie on a plane we can 
compute the homography H between the world and the 
camera plane [32.8]. Assuming Z = 0 is the world plane 
the homography reads 



where r\ 2 are the first two columns of the rotation ma¬ 
trix and denotes the projective equivalence, namely, 
for any two points p and p' in the projective plane p mp' 
iff p = A p' for real A / 0. Hence the first two columns 
of K~ 1 H 

K~ 1 H=(h' l h' 2 h' 3 ) 

have to be orthogonal. We seek thus an orthogonal ma¬ 
trix R that is the closest to ( h[ h 2 h\ x h 2 ) 


(10 0 \ 

R = {/ 0 1 0 V T . (32.9) 

\0 0 det (UV T )J 


The diagonal matrix is a projection from the orthogonal 
group 0(3) to the special orthogonal group SO(3). 

Last, we present a method [32.21] for n points that 
computes all local minima of the over constrained PnP 
problem. This involves solving the first derivatives ex¬ 
plicitly with respect to the pose unknowns. To achieve 
this, following observation allows the elimination of the 
depths A and the translation. Rigid transformation Xx = 
RX + T can be written for n points as a linear system 
for Ay=i..„ and the translation T 


(x\ 


-A 


/Ai\ 


/RXA 


V 



\R Xj 


We can solve for the unknown depths-translation vector 
and back substitute it into a least squares minimization 
problem with respect to rotation parameters. It turns out 
that if we use the three Rodriguez parameters as ro¬ 
tation parametrization the necessary conditions for an 
extremum (vanishing derivatives) turn out to be three 
cubic equations [32.21]. Last we would like to point out 
to the reader that a nonlinear function of the rotation 
matrix can also be solved as an optimization problem 
on the Lie-group SO(3) [32.22-24] for the case of line 
correspondences. 

32.1.3 Triangulation 


When we know both the intrinsics and extrinsics or 
their summarization in matrix P and we measure a point 
we cannot recover its depth from just one camera posi¬ 
tion. Assuming that we have the projection of the same 
point X in two cameras 



(32.10) 


arg min ||R — ( h[ h 2 h[ x h' 2 ) ||^. 

fiSSO(3) 


with known projection matrices Pj and Pi we can re¬ 
cover the position X in space, a process well known 
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as triangulation. Observe that we can achieve triangula¬ 
tion without decomposing the projection matrices into 
intrinsic and extrinsic parameters, we neeed though to 
remove the distortion in order to write them as above. 

Having correspondences of the same point in two 
cameras with known projection matrices Pi and P, we 
can solve the two projection equations for the world 
point X. It is worth noting that each point provides two 
independent equations so that triangulation becomes 
an overconstrained problem for two views. This is not 
a contradiction since two rays do not intersect in gen¬ 
eral in space unless they satisfy the epipolar constraint 
as presented in the next paragraph. The following ma¬ 
trix in the left hand side has in general rank 4 unless 
the epipolar constraint is satisfied in which case it has 
rank 3. 


/jtPi(3,:)-Pi(1,:)\ 

(x\ 

yP 1 (3,:)-P 1 (2,:) 

Y 

xP r (3,:)-P r (l,:) 

Z 

vyPr(3,:) — P r (2,:)/ 

W 


then by solving equations (32.14) for (Rj, 2"i), a prob¬ 
lem called absolute orientation. Alternatively one can 
avoid the second triangulation and solve the pose es¬ 
timation problem between triangulated points in 3-D 
and points in the left image only. The most popular vi¬ 
sual odometry system today is libviso [32.26] and is 
based on a moving stereo rig (l<E»JKilill!ll±l). 

Absolute Orientation 

The treatment for moving stereo will be short and the 
reader is referred to a similar treatment in the chapter 
about range sensing. We assume that correspondences 
between two time instances have been established based 
on tracking in the images so that we can formulate equa¬ 
tions of the form 

X 2 = RVi + T. 

The standard way [32.20, 27] to solve this problem is 
to eliminate the translation by subtracting the centroids 
yielding 


where P («\ :) means the z'-th row of matrix P. 

Obviously, the homogeneous system above can be 
transformed into an inhomogeneous linear system with 
unknowns (X, Y, Z). Otherwise it can be solved by find¬ 
ing the vector closest to the null-space of the 4x4 
matrix above using SVD. A thorough treatment of tri¬ 
angulation is the classic [32.25]. 

32.1.4 Moving Stereo 

Imagine now that a rigid stereo system consisting of 
cameras ci (left) and c r (right) 

hi, ~ Pi*,, (32.12) 

HrfWPA, (32.13) 

is attached to a moving robot and observe this system at 
two time instances 


X 2 -X 2 = R(X l -X l ). 

We need at least three points in total to obtain at least 
two noncollinear mean-free X — X vectors. If we con¬ 
catenate the mean free vectors for n points into an n x 3 
matrix A \ :2 we can formulate the following minimiza¬ 
tion of the Frobenius norm 

min \\A 2 — RAjIL, 
rsso(3) 

which is known as the Procrustes problem. It can be 
shown [32.20] that the solution is obtained through 
SVD as in (32.9) where U, V are obtained from the sin¬ 
gular value decomposition 

A 2 Aj = USV T . 


Xo - R 1 X 1 + Ti , (32.14) 

where Ao are point coordinates with respect to the world 
coordinate system, usually assumed aligned with one of 
the camera instances, and X\ are the coordinates of the 
same point with respect to the camera rig, after a mo¬ 
tion (Ri.T’i). To estimate the motion of the rig, we 
have to solve two correspondence problems, first, be¬ 
tween left and right image, and second, between left 
(or right) at the first time instance and left (or right, re¬ 
spectively) at the second time instance. The left to right 
correspondence enable the solution of the triangulation 
problem at each time instance. Motion can be obtained 


Solutions are usually obtained with RANSAC by sam¬ 
pling triples of points and verification with the Pro¬ 
crustes method. 

32.1.5 Structure from Motion 

Relax now the assumption that projection matrices are 
known and remain with measuring and matching cor¬ 
responding points 11 1 and u 2 . This is the well known 
structure from motion problem or more precisely struc¬ 
ture and 3-D-motion from 2-D motion. In photogram- 
metry, it is well known as relative orientation problem. 
Even after eliminating the A’s from equations (32.12) 
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or by writing them in projective equivalence form 



(32.15) 


we realize that we if (X. Pi. Pi) is a solution than 
(ILK, PiH~^PjH^ 1 ) is a solution, too, where H an in¬ 
vertible 4x4 real matrix or in other words a collineation 
in P 3 . Even if we align the world coordinate system 
with the coordinate system of the first camera, which 
practice is common 

M| « (/ 0)X, 

u 2 ^P 2 X, (32.16) 


intersection of the two rays Rx] and x 2 is that the two 
rays are coplanar with the baseline T 

xj(Tx'Rx l ) = 0, (32.19) 

which is the epipolar constraint (Fig. 32.2). To avoid the 
scale ambiguity we assume that T is a unit vector. We 
proceed by summarizing the unknowns into one matrix 

E = TR (32.20) 

where T is the 3x3 skew-symmetric matrix to the vec¬ 

tor T. The E matrix is called the essential matrix. The 
epipolar constraint reads then 

xTExi=0, (32.21) 


we remain with the same ambiguity where H is of the 
form 


(I 

0 

0 

0 \ 

0 

1 

0 

0 

0 

0 

1 

0 

v^41 

/J 42 

/i43 

h w) 


with h 44 ^ 0. This ambiguity is possible when the 
projection matrices are arbitrary rank 3 real matrices 
without any constraint on their elements. If we assume 
that we have calibrated our cameras then the projection 
matrices depend only on displacements 

u i « (I 0) X , 

m 2 «(R T)X, (32.18) 

and the only remaining ambiguity is the scale ambi¬ 
guity where H looks like an identity matrix except 
/Z 44 = s 1 being the scale factor. In other words if 
(R, T, X) is a solution then (R, sT , 1 /s.Xj is a solu¬ 
tion, too. These remarks generalize in multiple views. 
Because, in robotics the (R, T) matrices correspond 
to location and X to mapping of the environment, the 
problem has the more proper term SLAM: Simultane¬ 
ous localization and mapping. However, because the 
term SLAM has been used with a variety of sensors 
like sonar and laser range scanners, the term monoc¬ 
ular SLAM is better suited to describe structure from 
motion from multiple views [32.28]. 

Epipolar Geometry 

This is probably one of the most studied problems in 
computer vision. We constrain ourselves to the cali¬ 
brated case which is most relevant to robotics appli¬ 
cations. The necessary and sufficient condition for the 


which is the equation of a line in the x 2 plane with co¬ 
efficients Ex i or a coefficient of a line in the jci plane 
with coefficients E T x 2 . These lines are called epipo¬ 
lar and form pencils whose centers are the epipoles e\ 
and e 2 , in the first and second image plane respectively. 
The epipoles are the intersections of the baseline with 
the two image planes, hence e 2 ^T and e i « — R 1 T. 
Looking at the equations of the epipolar lines we can 
immediately infer that E T e i = 0 and Ee 2 = 0. 

The set of all essential matrices 

T = ]EgE 3x3 | E = JR, 
where T e § 2 and R e SO(3)} 

has been characterized as a manifold of dimension 
5 [32.29]. It has been proven [32.30] that 


Proposition 32.1 

A matrix E e R 3x3 is essential if and only if it has two 
singular values equal to each other and third singular 
value equal zero. 



Fig. 32.2 A point is perspectively projected to calibrated 
image vectors Rri and x 2 which are coplanar with base¬ 
line T 
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We present here Nister’s method [32.31] for recovering 
an essential matrix from five point correspondences and 
which gained in popularity because of its suitability for 
RANSAC methods. 

Minimal Case 

We expand the epipolar constraint in terms of homoge¬ 
neous coordinates jci = (jci,yi,zi) and *2 = (x 2 ,y 2 ,z 2 ) 
(when the points are not at infinity n = 1 ) and obtain 

(xfxj y\xl zixJ)Z? s = 0, (32.22) 

where E s is the raw by raw stacked version of ma¬ 
trix E. When we use only five point correspondences 
the resulting linear homogeneous system will have as 
a solution any vector in the four dimensional kernel of 
the data matrix 

E s = A \U{ -T X 2 u 2 1 A3W3 -}- A4W4 . (32.23) 

At this point we want the matrix E resulting from E s to 
be an essential matrix satisfying Proposition 32.1. It has 
been proven [32.30] that 


Proposition 32.2 

A matrix E e M 3x3 is essential if and only if 

EE t E = ytrace(EE T )E . (32.24) 


Though the det(E) = 0 constraint can be inferred from 
(32.24) we are still going to use it together with (32.24) 
to obtain ten cubic equations in the elements of E. 
As described in [32.31], one can obtain a tenth de¬ 
gree polynomial in A 4 . The number of real roots of 
this polynomial are computed with a Sturm sequence. 
There is no proof beyond physical plausibility of the 
existence of at least one solution that a real root will ex¬ 
ist at all. Several alternative 5-point solvers have been 
proposed since Nister’s paper [32.32-35] and an exten¬ 
sive list including code has been established by Pajdla’s 
group [32.36]. 

Assuming that we have recovered an essential 
matrix from point correspondences, the next task is 
to recover an orthogonal matrix R and a unit vec¬ 
tor translation T from the essential matrix. If E = 
C/diag((T, a. 0 )V T , there are four solutions for the pair 
(T. R) 

( T U R t) = (UR-' +7T/2 ZU t , URl +7l/2 V T ) . 
(f 2 .R 2 ) = (URz,-7r/2ZU T , URl_ n/2 V T ) , 

(Ti,R 2 ) = ( UR z , +7r/2 ZU T , URl^V 7 ), 
(T 2 ,Ri) = mz,-n/iZU T , URl +n/2 V T ), 


where R. denotes rotation around the z-axis. The four 
solutions can be split into two two-fold ambiguities: 

• Mirror ambiguity. If T is a solution, then —7’ is a so¬ 
lution, too. There is no way to disambiguate from 
the epipolar constraint: x\{(—T) x Rx \) = 0. 

• Twisted pair ambiguity. If R is a solution, then 
also Rt.ttR is a solution. The first image is twisted 
around the baseline 180 degrees. 

These ambiguities are resolved by checking if 
depths of triangulated points are positive. 

Critical Ambiguities 

The approach with five point correspondences has a fi¬ 
nite number of feasible (feasible means that they may 
produce multiple interpretations of structures in front 
of the camera) solutions when the points in the scene lie 
on a plane (a two fold ambiguity) [32.37] or when the 
points on the scene and the camera centers lie on a dou¬ 
ble sheet hyperboloid with the additional constraint that 
the camera centers lie symmetrically to the main gen¬ 
erator of the hyperboloid [32.38]. These are inherent 
ambiguities which hold for any number of point cor¬ 
respondences when one seeks a solution for an exact 
essential matrix. 

When someone is solving the linear least squares 
system for the essential matrix, a planar scene as well 
as the case of all points and the camera centers lying 
on a quadric causes a rank deficiency of the system and 
thus infinite solutions for E. 

Beyond the ambiguous situations, there is a con¬ 
siderable amount of literature regarding instabilities 
in the two view problem. In particularly, it has been 
shown [32.37, 39, 40] that a small field of view and in¬ 
sufficient depth variation can cause an indeterminacy 
in the estimation of the angle between translation and 
optical axis. An additional small rotation can cause 
a confounding between translation and rotation [32.41]. 
Moreover, it has been shown, that there exist local min¬ 
ima close to the global minimum that can fool any 
iterative scheme [32.42,43]. 

3-Point SfM 

Minimal solutions based on 5 points are still too slow to 
be used on mobile platforms where additional informa¬ 
tion like a reference gravity vector might be obtained 
from an IMU. We present here a recent solution using 
a reference direction and only 3 points [32.44]. 

We are given three image correspondences from 
calibrated cameras, and a single directional correspon¬ 
dence like the gravity vector or a vanishing point. This 
problem is equivalent to finding the translation vec¬ 
tor t and a rotation angle 9 around an arbitrary rotation 
axis. 
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Let us choose the arbitrary rotation axis to be Cn = 
[0,1,0] T . After taking the directional constraint into ac¬ 
count, from the initial five parameters in the essential 
matrix, we now only have to estimate three. We can use 
the axis-angle parameterization of a rotation matrix to 
rewrite the essential matrix constraint as follows 

P2i T £'pi= 0, (32.25) 

where 

E = t(I + sin 0e 2 + (1 — cos 0 J? 2 ) > 


frame and that its depth with respect to the 1st frame 
is A] 


A,x ; = R,(A,X|) + T, . (32.28) 

Taking the cross product with x, and writing it for n 
frames yields a homogeneous system 


f£ 2 R 2 *i 

V.r,. R.v i 


x 2 T 2 ^ 

XlTnJ 



(32.29) 


and t = (jc,y, 1). 

Each image point correspondence gives us one such 
equation, for a total of three equations in three un¬ 
knowns (elements of t and 9). To create a polynomial 
system, we set s = sin 6 and c = cos 9, and add the 
trigonometric constraint s 2 + c 2 — 1 = 0, for a total of 
four equations in four unknowns. In order to reduce the 
number of unknowns, we choose the direction of the 
epipole by assuming that the translation vector t has the 
form [x, y, 1] T . This means that for each t that we re¬ 
cover, —t will also need to be considered as a possible 
solution. 

Once we substitute for E in (32.25), the result¬ 
ing system of polynomial equations has the following 
form 


anxs + aaxc + days + a^yc 
+ di5X— ans + anc + cifc = 0 (32.26) 

for i = I.... 3. and the equation 

s 2 + c 2 - 1 = 0. (32.27) 

This polynomial system can be solved in closed form 
and has up to four solutions. The total number of pos¬ 
sible pose matrices arising from our formulation is 
therefore at most 8, when we take into account the fact 
that we have to consider the sign ambiguity in transla¬ 
tion. 

32.1.6 Multiple Views SfM 

When we talk about simultaneous localization and map¬ 
ping we obviously mean over a longer period of time. 
The question is how do we integrate additional frames 
in our 3-D motion estimation (localization) process. 

To exploit multiple frames we introduce rank con¬ 
straints [32.45]. We assume that the world coordinate 
system coincides with the coordinate system of the first 
frame and that a scene point is projected to jc, in the i-th 


that has the depth of a point in the first frame as an 
unknown. The 3 n x 2 multiple view matrix has to have 
rank one [32.46], a constraint that infers both the epipo- 
lar and the trifocal equations. The least squares solution 
for the depth can easily be derived as 


E"=tfo x TjfiXj x R,x i) 
||jc, X R/JT! II 2 


(32.30) 


Given a depth for each point we can solve for motion 
by rearranging the multiple views constraint (32.29) as 


^A[jc} r (g) xj 
K X'lx'{ r ®x'; 



(32.31) 


where jc' 1 is the n-th image point in the i-th frame 
and R,, 7j is the motion from 1st to the i-th frame 
and R f acked is the 12x1 vector of stacked elements of 
the rotation matrix R, . Suppose that k is the 12x1 ker¬ 
nel (or closest kernel in a least squares sense) of the 
3« x 12 matrix in the left hand side obtained through 
singular value decomposition and let us call A the 3x3 
matrix obtained from the first 9 elements of k and a the 
vector of elements 10—12. To obtain a rotation matrix 
we follow the SVD steps in the solution of absolute ori¬ 
entation (32.14) to find the closest orthogonal matrix to 
an arbitrary invertible matrix. 


Bundle Adjustment 

On top of such an approach, a bundle adjust¬ 
ment [32.47] minimizes the sum of all deviations be¬ 
tween image coordinates and the backprojections of the 
points to be reconstructed. 

argmine T C~*e , 

Rl ,Tf ,Xp 


minimized with respect to all 6 (F — 1) motions and 
3 N — 1 structure unknowns, where e is the vector con- 
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taining all errors 

€ f = ( x f _ ^11 x p + ^2 y p + Rf i 2 Z v + T * 

p V " 4l X P + *32 Y P + + T z 

y R 2\^p + *22 + *23^ + T y \ 

' P * 3 ,^, + R{ 2 Y p + ^334 + Tj 

and C is the error covariance matrix. We will continue 
with the assumption that C = I. 

Call the objective function 0(u) = e(u) T e(u) with u 
the vector of unknowns. Given a starting value for the 
vector of unknowns u we iterate with steps Am by lo¬ 
cally fitting a quadratic function to 0(u) 

0(u+ Au) = 0(u) + Au J V0(u)+ - Am t H(m)Am , 

where V4> is the gradient and H is the Hessian of 0. 
The minimum of this local quadratic is at Au satisfying 

H Su = -V0(u) . 

If 0(u) = e(u) T e(u) then 

V0 = 2 ei(u)Ve,(u) T = J(u) T e , 


where the Jacobian J consists of elements 



and the Hessian reads 


and we will explain this case better if we assume two 
motion unknowns a\ and 02 corresponding to 2 frames, 
and 3 unknown points Z?i, £» 2 -^ 3 - 

For keeping symmetry in writing we do not deal 
here with the global reference and the global scale am¬ 
biguity. 

The Jacobian for 2 frames and 3 points has 6 pairs 
of rows (one pair for each image projection) and 15 
columns/unknowns 

/ \ 



Aj 

0 B| 

0 

0 


0 


0 

0 

de 

A\ 

0 0 


0 

3 (a, b ) 

0 

A\ 0 

B i 

0 


A 3 

0 0 

0 

Bi 


0 

A 2 0 

0 

Bj 


motion structure 


with A matrices being 2x6 and B matrices being 2x3 

f 

being Jacobians of the error ej- of the projection of the i- 
th point in the /-th frame. We observe now a pattern 
emerging in J T J 


/u 1 

0 wj 

W2 

wh 


0 

U 2 Wj 

w 2 


TJ 

Ql 


• • Vi 

0 


, 3 


0 

v 2 

0 

0 

V- 

0 

0 

V, J 

u» 

rsj 


with the block diagonals for motion and structure sepa¬ 
rated. Let us rewrite the basic iteration (J T J) Am = J T e 


H = 2 E (vg(m)V 6 ,(m) t + €,-(«) 0 ) 

= 2 ( J(w) t J(m) + J2 « 2 J(m) t J(m) 

by omitting quadratic terms inside the Hessian. This 
yields the Gauss-Newton iteration 

(J T J) Am = J T e , 

involving the inversion of a (6F+ 31V—7) x (6F + 3N — 
7) matrix. Bundle adjustment is about the art of invert¬ 
ing efficiently (J T J). 

Let us split the unknown vector u into u = (a, b ) fol¬ 
lowing [32.48] obtaining 

• 6F — 6 motion unknowns a, 

• 3P — 1 structure unknonws b. 



and premultiply with 

(i wr’\ / u w\ /A«\ 

VO I J [xv r YJ\Ab) 

/ WV - 1 
0 I 

We find out that motion parameters can be updated sep¬ 
arately by inverting a 6F x 6F matrix 

(J]-WV _ 1 W T )Afl = e'- WV _ 1 e b . 




Each 3-D point can be updated separately by inverting 
a 3 x 3 matrix V 


Y Ab = e b — W t Am 
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It is worth mentioning that bundle adjustment though 
extremely slow captures the correlation between motion 
estimates and structure (3-D points) estimates which is 
artificially hidden in the iterative scheme in (32.29). 

The largest scale motion estimation and registration 
of views has been performed by Teller et al. [32.49] 
with a decoupled computation first of relative rotations 


32.2 3-D Vision for Grasping 

In this section we will move from the basic geometry 
required for grasping to the main 3-D vision challenges 
associated with the limited knowledge we might have 
about the shape of the object as well as the actual selec¬ 
tion of 3-D grasping poses. 

Naturally, object grasping and manipulation is 
closely related to general scene understanding and 
problems such as object detection, recognition, catego¬ 
rization and pose estimation. Taking all the above, there 
are very few approaches that address all the problems 
in a single system. One example, reported in [32.50], 
addresses the problem of enabling transfer of grasp 
knowledge between object categories, defined using 
both their physical properties and functionality. This is 
a challenging problem given that a number of objects 
with similar physical properties afford different tasks. 
An example can be a screwdriver and a carrot that are 
structurally alike, but only the former can be used as 
a tool, or a ball and an orange where only the latter af¬ 
fords eating (Fig. 32.3). 

In relation to object grasping in particular, there are 
methods that assume that full 3-D model of the object 
is available and concentrate on grasp synthesis solely. 
In addition, many of the approaches conduct experi¬ 
ments in a simulated environment without working with 
real sensory data. However, the knowledge generated in 
simulation can also be applied later onto sensory data. 
Another group of approaches considers grasp synthe¬ 
sis on real sensory data directly, dealing with problems 
such as noise, occlusions and missing data. 

If the object to be grasped is known, there are ap¬ 
proaches that store a database of grasp hypotheses, 
generated either in simulation or through experiments 
in a real setting. Most of the approaches assume that 
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and finally of relative translations. The above multiple 
view SfM techniques can also be applied in a sliding 
window mode in time. Davison et al. [32.28] showed 
the first real-time recursive approach by decoupling the 
direction of the viewing rays from the depth unknowns. 
For other recursive approaches the reader is referred to 
the corresponding SLAM chapter. 


a 3-D mesh of the object is available and the challenge 
is then to automatically generate a set of feasible grasp 
hypotheses. This involves sampling the infinite space of 
possible hand configurations and ranking the resulting 
grasps according to some quality metric. 

To simplify the process, a common approach is 
to approximate object’s shape with a constellation of 
primitives such as spheres, cones, cylinders, boxes or 
superquadrics [32.51-55]. The purpose of using shape 
primitives is to reduce the number of candidate grasps 
and thus prune the search space for finding the optimal 
set of grasp hypotheses. 

One example, shown in Fig. 32.4 and reported 
in [32.52], decomposes a point cloud from a stereo cam¬ 
era into a constellation of boxes. Grasp planning is per¬ 
formed directly on the boxes which reduces the number 
of potential grasps. El-Khoury and Sahbani [32.56] 
distinguish between graspable and nongraspable parts 
of an object where each part is represented by fit¬ 
ting a superquadric to the point cloud data. Pelossof 
et al. [32.57] approximate an object with a single su¬ 
perquadric and use a Support Vector Machines based 
approach to search for the grasp that maximizes the 
grasp quality. Boularias et al. [32.58] model an object 
as a Markov random field (MRF) in which the nodes 
are points from the point cloud and edges are spanned 
between the six nearest neighbors of a point. A node 
in the MRF carries either one of the two labels: a good 
or a bad grasp location. Detry et al. [32.59] model the 
object as a constellation of local multimodal contour 
descriptors. The set of associated grasp hypotheses is 
modeled as a nonparametric density function in the 
space of six-dimensional (6-D) gripper poses, referred 
to as a bootstrap density. Papazov et al. [32.60] demon- 
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similar objects that afford different 
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Fig. 32.4 Generation of grasp candidates through ob¬ 
ject shape approximation and decomposition from (af¬ 
ter [32.52]) 
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Fig. 32.5 Ranking of approach vectors on different objects 
given a specific task. The brighter an area the higher the 
rank. The darker an area, the lower the rank (after [32.62]) 


strates 3-D object recognition and pose estimation in 
a grasping scenario considering cluttered scenes. Weisz 
and Allen [32.61] proposes ametric suitable for predict¬ 
ing grasp stability under pose uncertainty. 

There are several approaches that deal specifically 
with incomplete point clouds. Marton et al. [32.63] 
exploit symmetry by fitting a curve to a cross sec¬ 
tion of the point cloud. Rao et al. [32.64] concen¬ 
trates of depth segmentation and sample grasp points 
from the surface of a segmented object using sur¬ 
face normals. Bohg et al. [32.65] presents a related 
approach that reconstructs full object shape assuming 


planar symmetry and generates grasps based on the 
global shape of the object. Bone et al. [32.66] makes 
no prior assumption about the shape of the object 
and apply shape carving for generating a parallel-jaw 
gripper grasps. Hsiao et al. [32.67] employs heuris¬ 
tics for generating grasp hypotheses dependent on the 
shape of the point cloud. Recent work in [32.68] 
identifies regions that afford force closure grasps by 
evaluating local curvature of the objects to create an 
initial opposing grasp with two or three fingers, de¬ 
pendent on the relative size of the object with re¬ 
spect to the hand. Richtsfeld and Vincze [32.69] uses 
a stereo-camera setup to generate a 3-D representa¬ 
tion of a scene with several objects and then gen¬ 
erates various top grasps on object candidates. Mal¬ 
donado et al. [32.70] use time-of-flight range data, 
model objects using 3-D Gaussians and rely on fin¬ 
ger torque information during grasping to monitor the 
grasp execution. Stuckler et al. [32.71] generate grasp 
hypotheses based on eigenvectors of the object’s foot¬ 
prints that are generated by projecting the 3-D object 
point cloud onto the supporting surface. The work 
of [32.72] presents a system for general scene under¬ 
standing used for grasp planning and execution. The 
system uses a bottom-up grouping approach where con¬ 
tour and surface structures are used as the basis for 
grasp planning. The work builds upon previous work 
presented in [32.73], 

Most of the recent work concentrates on grasp 
generalization either by observing human grasping or 
through off- and on-line learning directly on the robot. 
Kroemer et al. [32.74] demonstrates generalization ca¬ 
pabilities using a pouring task scenario. The goal of 
the approach is to find a part of the object that is most 
likely to afford the demonstrated action. The learning 
method is based on the kernel logistic regression. Her¬ 
zog et al. [32.75] stores a set of local templates of object 
that a human is interacting with. If a local part of an 
object segmented online is similar to a template in the 
database, the associated grasp hypothesis is executed. 
Song et al. [32.62] approach the problem of inferring 
a full grasp configuration in relation to a specific task 
the object is intended for. As in [32.76], the joint dis¬ 
tribution over various grasping variables is modeled as 
a Bayesian network. Additional variables like task, ob¬ 
ject category and task constraints are introduced. The 
structure of this model is learned given a large number 
of grasp examples generated in a simulator and anno¬ 
tated with grasp quality metrics as well as suitability for 
a specific task. The learned quality of grasps on specific 
objects given a task is visualized in Fig. 32.5. 
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32.3 Conclusion and Further Reading 

As main additional sources of reading, we recommend 
the textbooks by Hartley and Zisserman [32.12], Ma 
et al. [32.46], Faugeras [32.77], and Faugeras and Lu- 
ong [32.11]. The reader is referred to Chap. 5 for 
fundamentals of estimation, to Chap. 35 for sensor fu¬ 
sion, to Chap. 34 for visual servoing, to Chap. 31 for 
range sensing, to Chap. 45 for 3-D models of the world, 
and to Chap. 46 for SLAM. 

3-D vision is a rapidly advancing field and 
in this chapter we have covered only geomet¬ 


ric approaches based on RGB cameras. Although 
depth sensors will become ubiquitous indoors and 
might be outdoors as well, RGB cameras re¬ 
main formidable because of the higher number and 
larger diversity of features that can be matched 
and used for pose estimation and 3-D-modelling. 
Long range sensing can still be covered from mo¬ 
tion with large translation while active sensors are 
constrained in terms of energy reflected from the 
environment. 
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Google's project Tango 

available from http://handbookofrobotics.org/view-chapter/32/videodetails/120 
Finding paths through the world's photos 

available from http://handbookofrobotics.org/view-chapter/32/videodetails/121 
LIBVISO: Visual odometry for intelligent vehicles 

available from http://handbookofrobotics.org/view-chapter/32/videodetails/122 
Parallel tracking and mapping for small AR workspaces (PTAM) 
available from http://handbookofrobotics.org/view-chapter/32/videodetails/123 
DTAM: Dense tracking and mapping in real-time 

available from http://handbookofrobotics.org/view-chapter/32/videodetails/124 
3-D models from 2-D video - automatically 

available from http://handbookofrobotics.org/view-chapter/32/videodetails/125 
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33. Visual Object Class Recognition 


Michael Stark, Bernt Schiele, Ales Leonardis 


Object class recognition is among the most funda¬ 
mental problems in computer vision and thus has 
been researched intensively over the years. This 
chapter is mostly concerned with the recognition 
and detection of basic level object classes such as 
cars, persons, chairs, or dogs. We will review the 
state of the art and in particular discuss the most 
promising methods available today. 
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33.1 Object Classes 

Generic object class recognition has been one of the 
goals of computer vision since its beginnings [33.1, 
2], In this context, it is important to emphasize that 
the notion and the abstraction level of object classes is 
far from being uniquely and clearly defined. Notably, 
the question of how humans organize knowledge at dif¬ 
ferent levels has received much attention in cognitive 
psychology [33.3, 4], Taking an example from Brown’s 
work, a dog can not only be thought of as a dog , but also 
as a boxer, a quadruped, or in general an animate be¬ 
ing [33.3]. Yet, dog is the term that comes to mind most 
easily, which is by no means accidental. Experiments 
show that there is a basic level in human categorization 
at which most knowledge is organized [33.5]. Accord¬ 
ing to Rosch et al. and Lakoff [33.5,6], this basic level 
is also: 

• The highest level at which category members have 
similar perceived shape 


The highest level at which a single mental image 
can reflect the entire category 
The highest level at which a person uses similar mo¬ 
tor actions for interacting with category members 
The level at which human subjects are fastest at 
identifying category members 
the first level named and understood by children. 



Given that the basic level of categorization is eas¬ 
iest for humans, it should be a good starting point 
for machine vision, too [33.7]. The next lower level, 
subordinate categories, corresponds to the individual 
level used in object identification, e.g., the next higher 
level, super-ordinate categories, requires a higher de¬ 
gree of abstraction and world knowledge. The fol¬ 
lowing will thus discuss state-of-the-art approaches 
first for basic-level categorization but then also briefly 
touch upon subordinate categorization for fine-grained 
classification. 
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33.2 Review of the State of the Art 

One can think of all current methods in object class 
recognition as being part-based methods, in which ob¬ 
ject class instances are described by a set of parts (such 
as human body parts, for instance), and the topology 
between the parts (such as the kinematic tree defined 
by the joints that connect the human body parts). The 
following sections give a systematic treatment of differ¬ 
ent instantiations of state of the art, part-based methods, 
ranging from the bag-of-words model (Sect. 33.2.1) and 
spatial pyramid matching (Sect. 33.2.2), over flexible 
part models (Sect. 33.2.3) and rigid histogram of ori¬ 
ented gradients templates (Sect. 33.2.4) to deformable 
part models (Sect. 33.2.5) and convolutional neural net¬ 
works (Sect. 33.2.6). Table 33.1 summarizes the various 
part-based methods presented in this section. Later sec¬ 
tions highlight recent trends in fine-grained categoriza¬ 
tion (Sect. 33.2.7) and datasets for visual recognition 
(Sect. 33.2.8). 


Table 33.1 Classification of some prominent approaches 
as part-based models 


Model 

Parts 

Topology 

Bag-of-words 

Feature 

clusters 

No spatial information 
coded 

Spatial pyramid 

Feature 

clusters 

Hierarchical but rigid 
spatial grid 

Constellation 

Feature 

Full covariance matrix 

model 

clusters 

between clusters 

Implicit shape 
model 

Feature 

clusters 

Star-shaped model 

HOG 

HOG-cells 

Rigid spatial grid 

DPM 

HOG-parts 

Star-model 

Pictorial 

structures model 

Semantic parts 

Tree structure 

Convolutional 

Convolution 

Spatial spooling and 

neural networks 

filters 

hierarchy 


33.2.1 Bag-of-Words Models 

Apart from allowing to establish pointwise correspon¬ 
dences between multiple images of the same scene, 
local image patches also provide the foundation for 
a variety of object class representations for visual 
recognition. Inspired by the natural language process¬ 
ing literature, the bag-of-words (BOw) or bag of fea¬ 
tures model [33.8] represents an image as an unordered 
collection (a bag) of local patches. This representation 
assumes that most of the important information about 
the image content is captured purely by which patches 
occur (and how often), irrespective of where they oc¬ 
cur. While this is clearly an approximation, it seems 
reasonable from the text document analogy: even if the 
words in a text document are shuffled randomly, the 
mere occurrence of certain key terms provides a strong 
cue about the document’s topic. Likewise, a collection 
of local image patches, such as two eye patches, mouth, 
and nose, provide strong cues for the presence of a face 
in an image. 

The bag-of-words model is typically based on 
a codebook of visual words: a fixed, discrete set of vi¬ 
sual features relative to which all others are described 
(Fig. 33.1c). An input image (Fig. 33.1a) can then be 
described relative to that codebook, by counting how 
often each codeword appears (Fig 33.1b). In the fol¬ 
lowing paragraphs, we describe in detail the extraction 
of visual features, the generation of codebooks of vi¬ 
sual words, and their use in image classification using 
probabilistic generative and discriminative models. 

Feature Extraction 

Locations and scales for feature extraction are typ¬ 
ically either picked sparsely, based on an interest 
operator, such as the difference of Gaussian (DOG) 



Fig.33.1a-c The bag-of-words 
representation, (a) Image; (b) BOw 
histogram; (c) codebook of words 
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interest point detector for scale invariant feature trans¬ 
form (SIFT) [33.9], Harris comers, scale- and affine- 
invariant Hessian-Laplace interest points [33.10], max¬ 
imally stable extremal regions (MSER) [33.11], or 
based on dense feature sampling using a fixed spatial 
grid [33.12]. While interest operators generally pro¬ 
vide well-localized, repeatable features with sufficient 
recall, the increasing availability of compute power to¬ 
day has made very dense feature sampling feasible 
(e.g., with a stride of only one pixel, resulting in many 
thousands of features in an image) - in the limit, dense 
sampling typically outperforms sparse interest opera¬ 
tors in image classification as interest point detectors 
are designed to sample particular types of image struc¬ 
ture whereas dense sampling allows to sample any type 
of image structure (note that this might be different for 
other applications, such as structure from motion, in 
which it is more important to find smaller numbers of 
features with high precision). 

Codebook Generation 

Codebooks of visual features are typically constructed 
by applying an unsupervised clustering algorithm, 
such as k-means or hierarchical agglomerative cluster¬ 
ing [33.13], to a training set of image feature vectors. 
The result is a vector-quantized representation of the 
visual feature space that groups similar feature descrip¬ 
tors into clusters. Each cluster thus effectively general¬ 
izes over its members, such that small visual differences 
are suppressed - this is often seen as a crucial trait for 
categorization even though larger codebook sizes typi¬ 
cally result in better performance. Each cluster is iden¬ 
tified with a representative member, i. e., its centroid 
(for k-means) or medoid (for agglomerative clustering). 
In order to represent a new image with respect to the 
generated codebook, image features are extracted, and 
matched to the representative of each cluster (e.g., by 
computing pairwise distances and defining a matching 
threshold). The result of this matching is then recorded 
(e.g., as a histogram over the number of times each clus¬ 
ter has been matched) to yield the image or object rep¬ 
resentation. Note that in the context of BOw models the 
terms object and image representation are synonymous. 

For codebook generation, the number of clusters is 
an important free parameter: it has been found that large 
codebook sizes typically improve recognition perfor¬ 
mance, since they represent the original feature space 
more faithfully, with lower reconstruction error. On the 
other hand, large codebooks generalize less, and come 
at the cost of increased computational cost when build¬ 
ing bag-of-words representations and classifying them 
with nonlinear classifiers. Typically, codebooks with 
several and even 10000s (multiples of ten thousand) of 
clusters are chosen for optimal performance [33.14], 


Generative Models 

Inspired by their success in the text domain, genera¬ 
tive probabilistic models have also been adopted for 
image classification in connection with BOw represen¬ 
tations. The arguably simplest one is the naive Bayes 
model [33.8, 15], in which the posterior probability of 
an image having a certain class factors into independent 
contributions from each histogram bin (i. e., for a given 
class, knowing about the presence of one feature does 
not make the presence of another feature any more or 
less likely). While this model is agnostic of correlations 
that often exist between features (for instance, having 
already observed two wheel features in a sideview of 
a car makes it unlikely to observe one more, assuming 
that only a single car is present in the image), it often 
works well in practice and is appealing due to its sim¬ 
plicity. 

In line with the text domain, more powerful models 
have been proposed for image classification that ex¬ 
plicitly capture the co-occurrence of features through 
a generative process. Here, the features contained in 
an image are assumed to be generated in a sequence 
of steps, where each step determines the choices avail¬ 
able for the next step. More precisely, each step is 
implemented by sampling from a probability distri¬ 
bution, conditioned on the outcome of the previous 
step. 

A popular class of generative models is the topic 
model [33.16,17]: an image is considered to be an 
orderless document of visual words, composed of mul¬ 
tiple topics (in an image depicting a beach scene, there 
could be a water topic, a sand topic, and a sky topic, 
for instance). In order to generate a particular feature 
in the image, a topic is first sampled from a dis¬ 
crete distribution over topics (a beach scene could be 
a roughly uniform distribution over water, sand, and 
sky). Then, a visual feature is sampled from a dis¬ 
crete distribution over features, conditioned on the 
previously sampled topic (e.g., for a water topic, the 
sampled feature is likely to be a blue image patch 
with a ripple pattern). In practice, the assignment of 
visual features to topics is represented through la¬ 
tent variables, which means that it has to be inferred 
from other quantities that can be observed (such that 
the likelihood of the features is maximized). This is 
typically implemented by means of the expectation 
maximization (EM) [33.18] algorithm, which alternates 
(re-)estimation of topic distributions (E-step) and im¬ 
putation of latent variables (M-step) in an iterative 
procedure. 

In the literature, there exists a large variety of 
topic models, ranging from the basic probabilistic la¬ 
tent semantic analysis (PLSA) [33.16] over its Bayesian 
extension, latent Dirichlet allocation (LDA) [33.17], to 
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Fig. 33.2 Spatial pyramid matching 
as a multilevel BOw representation 
(after [33.20]) 


infinite hierarchical models of images and their fea¬ 
tures [33.19]. 

Discriminative Models 

Once an image is represented as a histogram of matched 
features with respect to a codebook, it can be classified 
by means of any discriminative classifier that is appli¬ 
cable to histograms. Popular choices include support 
vector machines [33.21] with histogram intersection 
kernels or their approximate counterpart [33.22], boost¬ 
ing [33.23], or random forest classifiers [33.24], 

Extensions 

While the bag-of-words representation inherently ig¬ 
nores spatial information, it can be re-introduced by 
restricting the support of a bag-of-words representation 
to a local image region, and processing multiple re¬ 
gions independently. Specifically, instances of an object 
class can be localized by sliding a rectangular window 
over the image under consideration, extracting a bag- 
of-words representation at each location, and feeding 
the result into a classifier that distinguishes the object 
class of interest from a background. Objects of differ¬ 
ent sizes can be localized by applying this scheme to 
multiple re-scaled versions of the input image [33.25] 
(called an image pyramid). 

Rather than applying the sliding window in a brute 
force fashion, it has been realized that the classifier 
score that an image window can attain can be bounded 
from above by the scores of the subwindows that it 
contains. This observation gives rise to an efficient 
subwindow search [33.26] that greatly reduces the num¬ 
ber of classifier evaluations needed in connection with 
sliding window bag of words representations, making 
powerful but slow nonlinear kernel SVMs applicable. 

33.2.2 Spatial Pyramid Matching 

Instead of completely ignoring the spatial layout of 
local features in an image, it seems reasonable to in¬ 
clude at least a rough notion of locality in order to 
increase the discriminative power of an object class 
representation. Spatial pyramid matching implements 
this notion as an extension of the bag-of-words model 
(Sect. 33.2.1), by extracting multiple bag-of-words rep¬ 
resentations for a single input image at different spatial 


locations and scales, and aggregating these local rep¬ 
resentations into a single global representation for the 
entire image. Specifically, spatial pyramid matching 
(SPM) [33.27] extracts bag-of-words histograms for 
each cell in a rectangular grid overlaid onto the input 
image (typically, coarse grids of 2 1 for le {0,1,2}). 
This process is repeated multiple times with different 
numbers of cells (e.g., four by four plus two by two 
plus a single cell), forming an overcomplete pyramid 
structure of decreasing spatial resolution (Fig. 33.2). 
The resulting bag-of-words histograms for each cell are 
concatenated (and re-normalized). The appeal of SPM 
lies in representing the spatial layout at multiple levels 
of resolution without committing to any one in partic¬ 
ular - instead, the representations of all resolutions are 
kept, and have the chance to contribute to image classi¬ 
fication in case they prove useful. 

The basic SPM methodology appears in numerous 
variations in the literature, but often follows the same 
pipeline of four steps (feature extraction, feature cod¬ 
ing, spatial pooling, and classification), each of which 
we review in the following sections. 

Feature Extraction 

Like the basic bag-of-words model, the SPM model 
has been used in connection with various feature types, 
ranging from SIFT [33.9] appearance features to lo¬ 
cal edge configurations [33.28] and self-similarity fea¬ 
tures [33.29]. 

Feature Coding 

Representing visual features relative to a pretrained 
codebook obtained by vector quantization is only one 
special case of feature coding (performing a change of 
representation), and several improvements have been 
proposed in the literature [33.30-32]. Most importantly, 
it has proven beneficial to explicitly formulate the code¬ 
book training and feature coding as a joint optimization 
problem, such that the original features can be recon¬ 
structed with small error from their respective codes. 
In addition, this formulation allows us to relax the as¬ 
sumption that each feature is matched to exactly one 
codebook entry by replacing the cardinality constraint 
with an LI regularization. The resulting sparse coding 
(SC) formulation [33.31] has been shown to outperform 
the original SPM [33.27] by significant margins, even 
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when combined with a linear SVM (while SPM works 
best in connection with nonlinear histogram intersec¬ 
tion or chi-square kernels, which are computationally 
expensive). An improved version of SC is given by 
locality constrained linear coding (LLC) [33.32], It re¬ 
places sparsity by locality constraints that encourage 
features to be reconstructed by nearby features in fea¬ 
ture space, such that similar features yield similar codes 
(which is obviously desirable in order to facilitate clas¬ 
sifier learning). 

Instead of relying solely on counting the number 
of occurrences of visual features in a spatial region 
as promoted by SPM, SC, and LLC, the Fisher vec¬ 
tor [33.30] method characterizes that region by how well 
a pretrained generative probabilistic model (a Gaussian 
mixture model, GMM) explains the contained features. 
Specifically, it builds a vector from the derivatives of 
the log-likelihood of the image features, given the mix¬ 
ture coefficients, means, and diagonal covariances of 
a GMM. Note that this vector has much more entries 
than a histogram of occurrences using the same GMM 
as a codebook, allowing to build highly discrimina¬ 
tive representations already for small numbers of GMM 
mixture components (typically only a few hundred). 

Both sparse coding and Fisher vector represen¬ 
tations haven proven to be very effective for im¬ 
age classification in connection with spatial pyramids, 
and have been successfully applied also in large-scale 
settings, such as classification on ImageNet [33.33] 
(Sect. 33.2.8). 

Spatial Pooling 

Apart from the classic pyramid scheme [33.27] in 
which histograms of different pyramid levels are con¬ 
catenated, retaining the maximum value of a particular 
bin over a spatial region or even multiple pyramid levels 
(max-pooling) has gained increasing attention [33.31], 
in particular in connection with CNNs (Sect. 33.2.6). 
Max-pooling has the advantage that it promotes invari¬ 
ance to local translations and scale changes, and has 
been found to be present in the human visual cortex 
(VI) [33.34], 

In addition to pooling features in the two- 
dimensional (2-D) image plane, more object-centric 
pooling techniques have been developed in the context 
of fine-grained categorization [33.35, 36] (Sect. 33.2.7). 

Classification 

The last stage in an SPM pipeline is typically given 
by a discriminative classifier that is trained on a train¬ 
ing set of fixed length vectors created from the SPM 
representations. Depending on the sophistication of the 
feature coding step, support vector machines with non¬ 
linear histogram intersection or chi-square [33.27] or 


linear kernels [33.30-32] achieve state-of-the-art per¬ 
formance. Further extensions include the use of multi¬ 
ple kernels with learned contribution weights [33.37]. 
When evaluated as part of a classification cascade, mul¬ 
tiple kernel learning (MKL) can also be used in a sliding 
window object class detector [33.38]. 

33.2.3 Flexible Part Models 

While orderless bags of visual features (Sect. 33.2.1) or 
spatial pyramids with a fixed grid layout (Sect. 33.2.2) 
have proven to be effective object class representations 
for classification, they both fail to explicitly capture the 
flexible nature of physical objects that are composed 
of multiple parts - as an example, consider the human 
face with eyes, eyebrows, mouth, nose, and ears: ob¬ 
viously, the precise shape and spatial arrangement of 
these parts will vary from person to person, but varia¬ 
tions in shape and arrangement are constrained to lie 
within a plausible range (i. e., the parts are not un¬ 
ordered as in a bag-of-words representation). It also 
seems intuitive that the constraints that govern the range 
of plausible variations should be soft, and not bound to 
a fixed binning or a fixed spatial grid (as in spatial pyra¬ 
mid matching). In this section, we review a family of 
flexible part models that implement this intuition. 

Constellation Model 

The constellation model [33.39— 41] represents an ob¬ 
ject class as a flexible arrangement of a fixed number 
of parts. It consists of two components: first, a model 
of the appearance of each part, and second, a model 
of the relative spatial layout of the parts. It is formu¬ 
lated in a fully probabilistic way, such that each part 
is characterized by a probabilistic, generative model of 
its appearance, and the spatial layout of the parts is 
governed by a joint Gaussian distribution that captures 
how the location (and scale) of each part is expected to 
vary with the location of each other part. Interestingly, 
the definition of what constitutes a part is driven en¬ 
tirely by the training data: parts are chosen and placed 
such that the likelihood of the training data, given the 
model, is maximized during training. This can be seen 
as an instance of a missing data problem (the asso¬ 
ciation between features and parts is unknown), and 
solved by means of the expectation maximization (EM) 
algorithm. 

Unfortunately, the representative power of the joint 
Gaussian spatial distribution comes at a cost: the fully 
connected dependency structure demands examining 
each of the exponentially many combinations of plac¬ 
ing all parts in an image in order to find the global 
optimum - the exact inference problem is known to be 
intractable. As a consequence, the original constellation 
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model [33.40,41] is limited to a few dozen candidates 
per part, per image, limiting its generality. An alterna¬ 
tive approach to solving the exact inference problem is 
to resort to an approximation, e.g., by sampling likely 
constellations from part detection proposals [33.42]. In 
this way, the number of part candidates can be scaled 
up to several thousands. 

Due to its probabilistic nature, the constellation 
model has also been extended to a full Bayesian 
model [33.43], in which the object class model it¬ 
self is governed by a prior distribution over models. 
This abstraction allows to separate characteristics of 
one particular object class (e.g., the particular shape 
of a horse’s head) from properties that are shared by 
classes (e.g., all quadrupeds are four legged). These 
shared properties can be re-used in multiple object class 
detectors (e.g., for horses, cows, dogs, etc.), facilitating 
learning from limited training data. The underlying idea 
of re-use of information is known as transfer learning in 
the machine learning literature, and has been success¬ 
fully applied for image classification [33.43] as well as 
object class detection. 

Implicit Shape Model 

The implicit shape model (ISM) [33.44] relaxes the as¬ 
sumption of the constellation model that the locations 
and scales of all parts are mutually dependent. Instead, 
it limits the dependency structure to a star topology, in 
which part locations and scales are conditionally inde¬ 
pendent, given the object center (i. e., given the object 
center, revealing information about the location of one 
part does not alter the expectation of where to observe 
any other part). While this model clearly constitutes an 
approximation of the physical world, it greatly simpli¬ 
fies the inference problem of finding the configuration 
with maximum score (there exists an algorithm that is 
exact and efficient). 

Specifically, the ISM represents an object class as 
a collection of local feature matches - the supply of 
matchable features is given by a codebook of visual 
words, as it is also used in bag-of-words (Sect. 33.2.1) 
or spatial pyramid representations (Sect. 33.2.2). In 
contrast to those, however, each ISM codebook entry 
also maintains a list of feature matches from a training 
set of images: each match specifies where the match oc¬ 
curred relative to the object center. As a result, this list 
of occurrences constitutes a nonparametric probability 
density estimate of the location of the object center, 
given the matched feature - in other words, each feature 
can cast votes for likely object center positions (e.g., 
a wheel feature can cast two votes: one to the left of 
it, assuming that it is the front wheel of a car, and one 
to the right, assuming it’s the back wheel). This voting 


procedure can directly be translated into a recognition 
algorithm: at recognition time, all test image features 
cast their votes into a three-dimensional (3-D) voting 
space ( x , y, and scale). All votes are accumulated, and 
since multiple features are likely to agree on the true 
object center, the maxima of this voting space can be 
considered candidate object detections (note that this 
voting can be considered a generalized Hough trans¬ 
form). In practice, continuous voting space maxima are 
found using a mean-shift algorithm. 

In order to increase its discriminative power, 
the basic ISM model is extended by a verification 
step [33.44], in which object detection candidates are 
verified based on how consistent they are with re¬ 
spect to foreground-background segmentation. To that 
end, feature occurrences stored as part of the code¬ 
book are enriched with segmentation mask fragments 
(which have to be provided as additional supervision 
during training). At recognition time, the consistency 
of the cast segmentation mask votes is quantified based 
on minimum description length (MDL). In addition, it 
has been suggested to learn weights that regulate the 
contributions of feature matches to the final detection 
hypotheses in a discriminative way (i. e., optimized to 
distinguish object and background firings). This dis¬ 
criminative Hough transform [33.45] further improves 
the performance of the ISM and constitutes the basis 
for the poselet framework [33.46], in which a multitude 
of small HOG [33.47] detectors (Sect. 33.2.4) are com¬ 
bined in order to detect people. 

Since, in theory, the ISM voting space can accom¬ 
modate arbitrary quantities in addition to object center 
and scale, there have been extensions to predict the 
walking poses of pedestrians [33.48] as well as pre¬ 
dicting the viewpoint from which the detected object 
is imaged [33.49, 50]. 

Pictorial Structures 

While being the first to conceptualize the idea of objects 
being composed of flexible parts, the pictorial structures 
model [33.39,51] is typically understood as a simpli¬ 
fied version of the constellation model. The full mutual 
dependency structure between parts is relaxed to a tree 
(or a mixture of trees in more advanced incarnations) - 
note that the star-shaped ISM (Sect. 33.2.3) is hence 
a special case of a pictorial structures model with tree- 
depth one. In contrast to the constellation model, where 
parts are defined in a data-driven way, the pictorial 
structures model is typically associated with a seman¬ 
tic notion of parts. An intuitive example is the set of 
human body parts (head, torso, upper arm, lower arm, 
etc.) in connection with the natural kinematic tree struc¬ 
ture (Fig. 33.3). 
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Fig.33.3a,b Human body 
pose estimation with the 
pictorial structures model, 
(a) Image likelihoods for in¬ 
dividual parts; (b) final pose 
estimate using spatial layout 
prior 


In fact, current state-of-the-art systems for people 
detection and human body pose estimation are typically 
built upon the pictorial structures model [33.52]. Here, 
pairwise relations between body parts express the an¬ 
gular constraints of the human body as well as the prior 
probability of observing people in various poses (e.g., it 
is unlikely to observe extreme angles at certain joints, 
Fig. 33.3b). In order to compensate for the weaker spa¬ 
tial model in comparison to the constellation model, the 
pictorial structures model typically relies on a strong 
model for the appearance of each part that is trained dis- 
criminatively (distinguishing the part from background. 
Fig. 33.3a) rather than in a generate way (explaining the 
part’s appearance). Examples of discrimative part ap¬ 
pearance models include AdaBoost [33.23] classifiers 
trained on dense shape context features [33.52]. Also 
note that part appearance and spatial layout models are 
trained independently (joint training has been realized 
to bear large potential for improvement in the context 
of the deformable part model [33.53] which we will re¬ 
view in Sect. 33.2.5). 

The use of semantic parts naturally demands to 
provide additional supervision in the form of part anno¬ 
tations during training; human body parts are typically 
annotated with a fixed size relative to the size of the 
person. For recognition, inference in the tree-structured 
probabilistic graphical model can be carried out exactly 
and efficiently by max-product or sum-product be¬ 
lief propagation, depending on whether the maximum 
a posteriori (MAP) or marginal solution is preferred. 

Extensions to the pictorial structures model in¬ 
clude learning the tree structure instead of using the 
natural kinematic tree [33.54], introducing additional 
lateral edges in addition to the tree edges [33.55], 
and combining pictorial structures models with non- 
parametric representations of partial human poses 
(poselets [33.46]). 

33.2.4 Histogram of Oriented Gradients 

At the opposite end of the spectrum from the 
bag-of-words (Sect. 33.2.1) or flexible part models 
(Sect. 33.2.3) are approaches that represent an object 


class as a rigid template of local features with a fixed 
spatial layout. Conceptually, this corresponds to the 
finest level of a spatial pyramid in which each feature 
gets assigned its own exclusive set of histogram bins 
(with values equal to the feature itself). The by far most 
successful rigid template detector is the histogram of 
oriented gradients (HOG) [33.47] detector, and we will 
discuss its properties in detail in this section. Origi¬ 
nally developed for people detection, the HOG feature 
has evolved into one of the most frequently used fea¬ 
tures in computer vision to date, and also constitutes 
the basis for the successful deformable part model 
(DPM) [33.53] described in Sect. 33.2.5. 

Feature Extraction 

The HOG feature can be seen as an extension of 
SIFT [33.9] from a single circular interest region to 
a rectangular detection window - at its core, it bins gra¬ 
dient directions into histograms, weighting their con¬ 
tributions by their respective magnitudes. Like SIFT, it 
has been carefully engineered and its parameters empir¬ 
ically tuned for performance (more recently, learning 
features end-to-end has received renewed attention, 
Sect. 33.2.6). 

The first step for the computation of a HOG fea¬ 
ture is to divide the detection window into a fixed grid 
of cells: a cell is the basic spatial unit of measurement 
for the HOG feature. The second step is to gamma- 
normalize the pixel content of each cell, in order to 
increase robustness to lighting changes. Third, the im¬ 
age is smoothed and image gradients computed using 
discrete derivative masks. Fourth, gradients are binned, 
such that each gradient casts weighted votes (by mag¬ 
nitude) according to nine different gradient orientation 
bins. Votes are interpolated bilinearly in both orien¬ 
tation and position. And fifth, the per-cell histograms 
are contrast normalized with respect to a larger block 
of surrounding cells in order to compensate for lo¬ 
cal variations in lighting and foreground-background 
contrast. Crucially, blocks are formed such that they 
overlap, which means that each cell contributes mul¬ 
tiple times to the final feature descriptor, normalized in 
different ways, resulting in an over-complete represen- 
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Fig. 33.4 Sliding window object class 
detection 



Fig.33.5a,b The deformable part 
model with multiple mixture com¬ 
ponents. (a) Mixture component for 
a frontal view of object class car. 

(b) Mixture component for a side view 
of object class car (after [33.56]) 


tation. This step has been found to be essential for good 
performance. 

Sliding Window Object Detection 
Similar to pure bag-of-words and spatial pyramid-based 
detectors, the HOG detector is typically run in a sliding 
window fashion over a test image, over different loca¬ 
tions and scales (Fig. 33.4). That is, HOG features are 
computed for a detection window of fixed size, resulting 
in a feature vector that can then be fed into a discrimi¬ 
native classifier, such as a linear support vector machine 
(SVM). 

Bootstrapping 

One of the key ingredients for good detection per¬ 
formance of the HOG detector [33.47] is a technique 
termed as bootstrapping )not to be confused with a tech¬ 
nique of the same name known from statistics, in 
which a dataset is repeatedly subsampled). An initially 
trained model is refined by re-training it on an ex¬ 
tended set of negative training images, similar in spirit 
to boosting [33.23]. Specifically, the initial model is run 
on a validation set, and the false positive detections 
(wrongly detected objects) are added to the negative 
training set. As a result, the decision boundary of the re¬ 
trained classifier is pushed further away from these hard 
negatives. Note that this procedure is not guaranteed to 
always lead to an improvement, but often improves per¬ 
formance substantially in practice. 

Extensions 

Due to its simple grid structure, the HOG feature lends 
itself to being combined with other feature channels ex¬ 


tracted from a similar spatial grid. Notably, it has been 
successfully combined with optical flow features for 
people detection in video [33.57]. 

33.2.5 The Deformable Part Model 

The section provides an introduction to one of the 
most successful computer vision algorithms to date, the 
DPM [33.53]. Its success is partly due to its consistently 
high performance for detecting a wide variety of ob¬ 
ject classes, but also due to the fact that a high quality 
implementation has been made publicly available from 
the start. The original implementation has constantly 
been improved and updated, and even found its way 
into OpenCV [33.58]. The HOG [33.47] implementa¬ 
tion at its core has found wide spread use throughout 
the computer vision community. 

Model Structure and Inference 
The DPM can be seen as a proper generalization of 
the fully rigid HOG [33.47] template to a flexible, 
part-based object class representation (Fig. 33.5). It 
generalizes HOG in two different ways that both in¬ 
crease its ability to handle intraclass variation: first, it 
divides the single HOG template into a collection of 
multiple templates that can move relative to one another 
and capture small variations in object shape (hence the 
name Deformable Part Model). Specifically, it includes 
a low-resolution root template for the entire object plus 
a number of high-resolution parts that are constrained 
to occur not too far from their expected positions 
relative to the root - this constitutes a star-shaped 
probabilistic graphical model (PGM) in full analogy 
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to the implicit shape model [33.44] (Sect. 33.2.3) or 
a special case of a tree-structured pictorial structures 
model [33.39] (Sect. 33.2.3). While this is less powerful 
than a full constellation model [33.41] in terms of rep¬ 
resentation, it has proven very effective in connection 
with a discriminative learning regime. 

Second, the DPM is formulated as a mixture model 
with multiple components, where each component 
takes care of another aspect of the object class of inter¬ 
est, such as a particular viewpoint or a particular context 
in which the object occurs (e.g., a person sitting on 
a chair). Each component consists of a star-model with 
root and multiple part templates as described earlier, 
trained to respond the strongest to its dedicated aspect - 
this separation of concerns leads to a simpler learning 
problem than trying to handle all aspects with a single 
component. 

For recognition, each of the DPM components pro¬ 
duces a score as follows: first, all of the component’s 
templates (root and parts) are convolved with the im¬ 
age in a sliding window fashion, as in the original 
HOG [33.47] work (Sect. 33.2.4), resulting in dense 
score maps over image locations and scales. Second, 
root and part scores are combined with spatial defor¬ 
mation terms that reflect the star-shaped spatial de¬ 
pendency structure, resulting in new score maps over 
locations and scales - note that this inference step 
can be carried out by max-product belief propagation, 
which is exact and efficient due to the star topology, 
and implemented as an instance of a generalized dis¬ 
tance transform [33.59]. And third, for each image 
location and scale, the maximum response over all com¬ 
ponents is kept as the final detection score (which is 
then further processed by a standard, greedy nonmax¬ 
ima suppression). 

Learning 

It has been argued that much of the DPM’s excellent 
detection performance is due to the specific way in 
which parameters are learned from training data, in ad¬ 
dition to the added expressive power in comparison to 
a single, rigid HOG template. This learning also consti¬ 
tutes the major difference between the DPM and earlier 
flexible part models such as the ISM [33.44] or the pic¬ 
torial structures model [33.39] (Sect. 33.2.3) that are 
structurally equivalent, but less effective in terms of 
performance. 

In contrast to these earlier models, where local part 
appearance and spatial layout models are trained sep¬ 
arately, all DPM parameters are learned jointly, such 
that a single binary classification objective is opti¬ 
mized (distinguish between the object class of interest 
and background). Crucially, this includes the placement 
of parts relative to the root template as well as the 


selection of the mixture component that is deemed re¬ 
sponsible; both are treated as latent variables at training 
(and test) time. As a consequence, all aspects of the 
DPM are optimized towards the single goal of object 
class detection, without being convoluted by potentially 
suboptimal choices of part placement based on appear¬ 
ance similarity (ISM) or semantic meaning (pictorial 
structures). 

The joint optimization problem is phrased as a gen¬ 
eralization of the linear SVM [33.21] formulation of 
a single HOG template, termed latent support vector 
machine (LSVM) [33.60]. It is based on the following 
intuition: assuming that the latent variables (part place¬ 
ments and mixture component selection) are observed, 
all model parameters (HOG template and part displace¬ 
ment weights) can be learned using standard linear 
SVM procedures (the resulting objective function is 
convex, assuming the use of a hinge loss). Conversely, 
assuming known model parameters, the optimal values 
of all hidden variables can be found exactly and effi¬ 
ciently (using the distance transform for part placement 
and exhaustive search over components). This intuition 
gives rise to an EM-like alternation of two steps, which 
can be shown to converge to a local optimum of the 
original, nonconvex objective function. 

Naturally, a good initialization is needed in order 
to reach good local optima during learning. The DPM 
suggests a clustering of training examples based on as¬ 
pect ratio, which has proven to work surprisingly well 
in practice (clustering based on appearance typically re¬ 
sults in small improvements but is much more costly 
in terms of computation time). As for the HOG de¬ 
tector [33.47], bootstrapping (here called hard negative 
?nining) allows to escape bad local optima. 

While the concept of jointly optimizing part appear¬ 
ance and placement has already existed in the constel¬ 
lation model [33.41], the DPM achieves this goal by 
largely different technical means, resulting in better per¬ 
formance: (1) it uses sliding-window HOG templates 
with linear SVM classifiers instead of a generative 
model over local interest point descriptors to represent 
local appearance, and (2) it relaxes the fully connected 
probabilistic graphical model topology to a star, making 
exact inference tractable and efficient for learning. The 
DPM has achieved excellent performance on a wide va¬ 
riety of benchmark datasets, including PASCAL VOC 
(visual object class) [33.61] (Sect. 33.2.8). 

Multiview and 3-D DPMs 

While the original DPM [33.53] achieves remarkable 
performance on standard detection benchmarks, it is 
inherently limited to predicting 2-D bounding boxes. 
While this might be sufficient for some applications 
(such as counting the number of objects in an image), it 
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is clearly not enough for others: suppose an autonomous 
driving scenario in which vehicles are to be tracked 
across multiple frames, and their spatial extent to be 
estimated in order to avoid collisions - here, a 3-D rep¬ 
resentation of each vehicle is required. 

To that end, the DPM has been extended [33.56, 
62, 63] to output richer object hypotheses than 2-D 
bounding boxes, in the form of additional viewpoint 
estimates (providing a better notion of 3-D object ex¬ 
tent than a 2-D bounding box alone) and 3-D parts that 
are consistent across viewpoints (providing a basis for 
establishing correspondences across multiple frames 
in a sequence). Providing these richer outputs can be 
achieved by extending the original DPM in two differ¬ 
ent ways: first, the classification problem solved during 
learning can be reformulated as a structured output pre¬ 
diction problem [33.62]. In this way, the model can 
be optimized for the prediction of other quantities than 
a binary object-background label, such as 2-D bounding 
box overlap or angular viewpoint estimates, through the 
use of different loss functions. In its simplest variant, 
viewpoint estimation can be implemented by a binning 
of angles, and dedicating a single DPM mixture com¬ 
ponent to each angular bin. While this model is still 
inherently view-based, it has been shown to deliver 
excellent performance in both 2-D bounding box local¬ 
ization and viewpoint estimation (note that viewpoint 
labels have to be available during training in addition to 
bounding box annotations). 

Second, instead of parameterizing the DPM parts 
in the image plane, an object-centric 3-D coordinate 
system can be used, such that parts from different mix¬ 
ture components refer to the same 3-D object portion 
(Fig. 33.6a). Part deformations are then governed by 
3-D displacement distributions (Fig. 33.6b), and object 
as well as part appearance can be captured in a continu- 


a) 





Fig.33.6a ,b Part representation in the 3-D deformable part 
model (3-D DPM). (a) 3-D/2-D part coordinate systems; 
(after [33.62]) (b) 3-D/2-D part spatial distributions (af¬ 
ter [33.63]) 


ous appearance model (Fig. 33.7a). The result is a 3-D 
object class representation [33.63] that can synthesize 
previously unseen viewpoints at test time (by inter¬ 
polating between a number of supporting viewpoints 
for which training data is available) and thus estimate 
viewpoints up to arbitrary granularity. The learning of 
this 3-D object class representation is facilitated by us¬ 
ing 3-D computer-aided design (CAD) models of the 
object class of interest in addition to real-world im¬ 
ages (Fig. 33.7b). During training, the CAD models 
serve both to establish a 3-D coordinate system and to 
generate artificial training data for different viewpoints 
through rendering. Figure 33.8 shows example detec¬ 
tions obtained by the 3-D 2 PM. 

Computational Considerations 
Interestingly, it has been realized that the major com¬ 
putational bottleneck of the original DPM [33.53] at 
recognition time is the convolution of HOG templates 
with image features (and not probabilistic graphical 
model inference as one might think). As a consequence, 
there have been attempts to minimize the number of 
HOG convolutions by various means, e.g., by part shar¬ 
ing parts across components and object classes [33.64], 
re-using parts of templates by decomposing them into 
an overcomplete set of dictionary elements [33.65] or 
cascaded detector evaluation [33.66]. Likewise, there 
have been attempts to reduce the training time for the 
DPM by exploiting HOG feature statistics to re-use 
computation across models [33.67, 68]. 

33.2.6 Convolutional Neural Networks 

While artificial neural networks (ANNs) [33.69] had 
largely been neglected in favor of support vector ma¬ 
chines, random forests, and other learners until around 
2010, they have recently returned to the scene, with 
quite remarkable success. Among a number of artificial 



Fig.33.7a,b Continuous appearance model (a) in the 
3-D 2 PM, trained from rendered 3-D CAD models (af¬ 
ter [33.63]) (b) and real images (not shown) 
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neural network architectures that had been suggested 
already in the early days of computer vision, the convo¬ 
lutional neural network (CNN) [33.70] has emerged as 
a particularly promising candidate for becoming a stan¬ 
dard tool for visual recognition tasks. 

One of the key properties of the CNN is its specific 
structure (Fig. 33.9), which is both amenable to deal¬ 
ing with multidimensional input (such as a color image) 
and effective for preventing overfitting when learned 
from limited training data. As an ANN, a CNN de¬ 
fines an artificial neural network with multiple layers 
on an input image, such that image pixels provide the 
input to the neurons of the first layer, and the outputs 
of each layer of neurons provide the inputs to the next 
layer (the final layer can provide a task specific output, 
such as an encoding of different image classes). Now, 
the topology of the CNN is restricted in two specific 
ways: first, neurons in one layer can only be connected 
to a fixed spatial region of neurons in the preceding 
layer (the receptive field) - this dramatically reduces the 
total number of connections in the network and hence 
parameters of the model that have to be learned. And 
second, most layers of neurons (typically the lower lay¬ 
ers) are formed by replicating local groups of neurons 
multiple times, including the weights associated with 
their connections to the preceding layer ( weight shar¬ 
ing) - this further reduces the number of parameters of 
the model, and encourages the network to learn a repre¬ 
sentation for the input that is invariant to translation. As 
a result of their specific structure, CNNs are less prone 
to overfitting than fully connected ANNs, and can be 
successfully trained using gradient-based methods and 
back-propagation of errors. 


CNNs for Image Classification 
CNNs have recently achieved remarkable performance 
in large-scale image classification [33.71], fueled 
by the availability of large-scale datasets like Ima- 
geNet [33.33] that provide thousands of images per 
object class. The success of CNNs in image classifica¬ 
tion is mostly attributed to their ability to learn useful 
image representations end-to-end ( deep learning) rather 
than relying on engineered features, such as SIFT [33.9] 
or HOG [33.47], While the latter have been carefully 
designed and empirically tuned over many years, they 
are general purpose and not specifically adapted to 
the classification task at hand, and hence suboptimal. 
A recent study provides further insights into the op¬ 
eration of individual CNN components (hidden layers 
and classifiers) through visualization [33.74], resulting 
in a further boost in performance. 

Region Proposals for Detection 
Since the evaluation of a CNN is computationally ex¬ 
pensive due to convolutions, a CNN is typically applied 
to preselected candidate regions rather than in a dense 
sliding window fashion for detection. Candidate regions 
can be generated either by a specialized detector trained 
for the object class of interest (such as HOG [33.47], 
Sect. 33.2.4) that is weaker but also much faster to 
evaluate than the CNN, or by a more general atten¬ 
tion mechanism that generates bounding boxes that are 
likely to contain an object instance of any class. Among 
various methods that generate proposal regions with 
a high degree of objectness [33.75] ([33.76] for a recent 
discussion and analysis of region proposal methods), 
selective search [33.77] has been successfully used in 



Fig. 33.8 3-D 2 PM example detections (car, bicycle, human head, cell phone) 
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Fig. 33.9 The CNN model of (after [33.71]) in the implementation of Caffe (after [33.72, 73]) 
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connection with CNN-based, object class specific clas¬ 
sifiers for detection [33.78]. Selective search is built 
upon a superpixel-based oversegmentation of an image, 
and suggests a greedy procedure to group adjacent su¬ 
perpixels into regions of high objectness. The resulting 
detector [33.79] has been shown to outperform previ¬ 
ous results in generic object class detection by a large 
margin (notably outperforming the previous best re¬ 
sults of the DPM [33.53], Sect. 33.2.5, on PASCAL 
VOC [33.61], Sect. 33.2.8). Curiously, it is not nec¬ 
essary to train this detector end-to-end for the task of 
object class detection for good performance - instead, 
the lower layers of the CNN can be pretrained for classi¬ 
fication on ImageNet as in [33.71], and later fine-tuned 
for detection. Both the CNN pretrained on ImageNet 
and the resulting state-of-the-art object class detector 
are publicly available under the name Caffe [33.72] and 
likely to find their way into the toolboxes of many com¬ 
puter vision practitioners. 

Structured Outputs 

While artificial neural networks can represent any func¬ 
tion in theory, effective learning of ANNs with more 
complex, structured outputs than single class labels or 
continuous values remains a challenge. Instead of learn¬ 
ing a full network end-to-end, structured outputs are 
typically obtained by feeding the outputs of the last 
layer of neurons into an existing structured model, such 
as multiple body joint regressors for human body pose 
estimation [33.80]. 

33.2.7 Fine-Grained Categorization 

In recent years, there has been a growing interest in dis¬ 
tinguishing objects not only with respect to basic-level 
categories [33.5] (such as car, bicycle, and person), but 
also on a finer level of granularity (such as different car 
models or plant and animal species). This fine-grained 
categorization is deemed particularly challenging: the 
distinction between two categories is often determined 
by small visual differences (such as the shape of a car’s 
headlight): these differences are easily outweighed by 
appearance differences due to change in lighting or 
viewpoint (a car looks largely different from the front 
and from the side), bearing the risk of overfitting to ir¬ 
relevant details. 

Feature Selection 

A first strategy to identifying subtle discriminative fea¬ 
tures while at the same time preventing overfitting 
is to combine classical feature selection techniques 
with randomization, e.g., in the form of random for¬ 
est classifiers [33.35,81], Here, randomization acts as 


a regularizer that prevents multiple trees to latch onto 
the same potentially irrelevant feature, hence reducing 
the risk of overfitting. Feature selection can also be per¬ 
formed with humans in the loop, using crowd-sourcing 
and gamification [33.82], 

Pose Normalization 

A second, orthogonal strategy in fin-grained categoriza¬ 
tion is to devise factored object class representations 
that explicitly capture different modes of variation in 
object appearance, thereby separating variations caused 
by category membership from variations caused by 
other factors, such as viewpoint changes. The main idea 
is to establish a common frame of reference for the 
extraction of visual features (e.g., a local coordinate 
system for the dense extraction of SIFT [33.9] features), 
such that the same object instance will always result in 
(roughly) the same visual features being extracted, irre¬ 
spective of viewpoint, lighting, etc. 

Specifically, such a frame of reference can be estab¬ 
lished by first detecting the object of interest in a test 
image and also localizing its semantic parts. Visual fea¬ 
tures are then extracted and described relative to the 
estimated object position and detected parts, ensuring 
that a feature on the beak of a bird will only ever 
be compared to a corresponding feature on the beak 
of another bird, and not on its tail (assuming perfect 
detection, viewpoint estimation, and part localization). 
This kind of part-based pose normalization has been 
demonstrated to be effective for bird species categoriza¬ 
tion [33.83,84], 

Instead of localizing object parts in 2-D, some 
works establish a common reference frame by esti¬ 
mating a rough 3-D geometry of the object of interest 
prior to feature extraction, in addition to estimating 
its viewpoint. This allows not only to describe fea¬ 
ture locations relative to the object surface rather than 
the 2-D image plane (effectively performing the pool¬ 
ing in 3-D [33.36]), but also to rectify image patches 
with respect to the estimated surface normals, improv¬ 
ing the robustness of the resulting feature descriptors 
with respect to small viewpoint changes. This strategy 
has proven effective in connection with a prebuilt bird 
model [33.35] as well as in connection with predicted 
3-D CAD model matches for fine-grained car catego¬ 
rization [33.36]. 

33.2.8 Datasets 

Since visual object class recognition approaches rely on 
machine learning algorithms at their core, they are very 
much dependent on the availability of sufficient train¬ 
ing data - it has in fact been argued that major advances 
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in the field might have occurred in response of obtain¬ 
ing more data rather than by developing more powerful 
algorithms [33.85]. At the same time, datasets are of 
utmost importance in order to quantify and compare 
the performance of different methods under controlled 
conditions, and checking the sanity of newly developed 
algorithms. To that end, benchmark datasets for visual 
object class recognition typically consist of a collec¬ 
tion of images split into training, validation, and test 
sets, plus a set of ground truth annotations, and often 
also evaluation scripts that measure the performance 
of a recognition algorithm relative to the provided 
annotations. 

Since each dataset provides only a small sample of 
the visual world, there is an inherent danger of being 
biased [33.85] with respect to which images are se¬ 
lected to be part of a dataset (and which ones are not), 
and by what means images are captured. As a result, 
new recognition algorithms should always be tested and 
cross-trained on multiple datasets in order to prevent 
overfitting to the characteristics of a particular dataset 
rather than learning a concept of interest. The following 
paragraphs give a small, by no means complete, selec¬ 
tion of some of today’s influential datasets for visual 
object class recognition. 

PASCAL VOC 

The PASCAL visual object classes challenge [33.61] 
stands for a succession of datasets that have been re¬ 
leased on a yearly basis from 2005 to 2012. Each dataset 
consists of multiple challenges (image classification, ob¬ 
ject class detection, segmentation, human body pose es¬ 
timation) that are defined on a given set of images. No- 

33.3 Discussion and Conclusions 

In this chapter, we have discussed visual object class 
recognition as one of the most fundamental prob¬ 
lems in computer vision, with a focus on presenting 
the most promising models available to practitioners 
today. The selection of methods that we have re¬ 
viewed is by no means complete - in favor of going 
into some depth for the most common techniques, 
we have skipped over object representations based 
on manifold learning [33.90] as well as representa¬ 
tions based on grammars [33.91,92], to mention just 
a few. 

As we have seen, practically all available models are 
inherently based on some notion of object parts that are 
governed by certain topological constraints. Naturally, 
the choice of which part-based model to pick is not an 
easy one, as it depends on many different factors that 
should be carefully considered. 


tably, the set of images is steadily increased, such that 
images of year n are re-used in year n + 1. PASCAL VOC 
counteracts dataset overfitting by not releasing test set 
ground truth annotations to the public, but instead main¬ 
taining an evaluation server that runs a limited number 
of automated tests for a particular recognition algorithm. 
PASCAL VOC images have been equipped with addi¬ 
tional annotations, e.g., in the form of aligned 3-D CAD 
models (3-D-PASCAL [33.86]). 

SUN and COCO 

The scene understanding (SUN) [33.87] and common 
objects in context (COCO) [33.88] datasets have been 
built in an effort to capture entire visual scenes rather 
than objects in isolation. They are meant as benchmarks 
for methods that reason jointly about multiple objects 
in a scene rather than performing independent classifi¬ 
cation or detection. 

ImageNet 

ImageNet [33.33] is one of the largest image datasets 
to date, consisting of over 14 million internet im¬ 
ages that have been annotated with object labels from 
the WordNet [33.89] hierarchy. Like PASCAL VOC, 
it constitutes the basis for multiple challenges that 
are defined on the dataset, ranging from classifica¬ 
tion (1000 classes) over object class detection to fine¬ 
grained categorization. Due to its massive size, the 
collection of the ImageNet dataset has been performed 
mostly by means of crowd sourcing. ImageNet con¬ 
stitutes the basis for pretraining some of the most 
successful CNN-based image classifiers to date [33.71] 
(Sect. 33.2.6). 


33.3.1 Task 

First of all, the appropriate choice of model is depen¬ 
dent on the application scenario and the resulting de¬ 
sired recognition output: different methods can generate 
outputs of different granularity, ranging from image- 
level class labels (basic bag-of-words, Sect. 33.2.1, spa¬ 
tial pyramid matching, Sect. 33.2.2) over 2-D bound¬ 
ing box locations (bag-of-words and spatial pyramid 
matching run in sliding window fashion, histogram of 
oriented gradient detectors. Sect. 33.2.4, flexible part 
models. Sect. 33.2.3, basic deformable part models, 
Sect. 33.2.5) to multiview detections and viewpoint 
estimates (multiview ISM, Sect. 33.2.3, 3-D DPM, 
Sect. 33.2.5). 

Secondly, an application scenario typically comes 
with a certain set of object classes of interest, which 
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are in turn characterized by rather different visual prop¬ 
erties. Rigid objects (like cars or mugs) tend to be 
handled well by rigid templates (like histogram of ori¬ 
ented gradients. Sect. 33.2.4) or deformable part models 
(Sect. 33.2.5), while articulate objects (like cats and 
dogs) can be more reliably represented using looser 
spatial models like bag of words (Sect. 33.2.1), or by 
explicitly modeling their kinematic tree structure (pic¬ 
torial structures, Sect. 33.2.3). 

Finally, it should be noted that the distinction be¬ 
tween object classes itself is quite arbitrary when draw¬ 
ing strict borders between any two classes. In reality, 
some classes are inherently more similar than others 
(e.g., dogs and horses are more similar than dogs and 
cars, since dogs and horses are both quadrupeds). Look¬ 
ing at multiple levels of object categorization rather 
than individual classes, it becomes a desired prop¬ 
erty that objects from the same super-ordinate cate¬ 
gory, such as quadrupeds, be classified as more similar 
than objects from different super-ordinate categories. 
In other words, an object categorization system should 
degrade gracefully: if the object itself is not correctly 
recognized, then we want it to be assigned at least to 
a similar category. We thus need a way of representing 
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the relationship and grading similarities between object 
categories [33.93]. 

33.3.2 Data 
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(Sect. 33.2.1) are very efficient, modeling deforma¬ 
tions and interdependence of parts (flexible part models. 
Sect. 33.2.3, deformable part models. Sect. 33.2.5) 
comes at the cost of inferring latent quantities in the 
model. Notably, this inference often has to be per¬ 
formed repeatedly during learning, underlining the im¬ 
portance of efficient inference procedures. 
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34. Visual Servoing 


Francois Chaumette, Seth Hutchinson, Peter Corke 


This chapter introduces visual servo control, using 
computer vision data in the servo loop to control 
the motion of a robot. We first describe the ba¬ 
sic techniques that are by now well established 
in the field. We give a general overview of the 
formulation of the visual servo control problem, 
and describe the two archetypal visual servo con¬ 
trol schemes: image-based and pose-based visual 
servo control. We then discuss performance and 
stability issues that pertain to these two schemes, 
motivating advanced techniques. Of the many 
advanced techniques that have been developed, 
we discuss two-and-a-half-dimensional (2.5-D), 
hybrid, partitioned, and switched approaches. 
Having covered a variety of control schemes, we 
deal with target tracking and controlling motion 
directly in the joint space and extensions to under¬ 
actuated ground and aerial robots. We conclude 
by describing applications of visual servoing in 
robotics. 
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Visual servo (VS) control refers to the use of computer 
vision data to control the motion of a robot. The vision 
data may be acquired from a camera that is mounted di¬ 
rectly on a robot manipulator or on a mobile robot, in 


which case motion of the robot induces camera motion, 
or the camera can be fixed in the workspace so that it 
can observe the robot motion from a stationary configu¬ 
ration. Other configurations can be considered, such as 
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for instance several cameras mounted on pan-tilt heads 
observing the robot motion. The mathematical devel¬ 
opment of all these cases is similar, and in this chapter 
we will focus primarily on the former, so-called eye-in¬ 
hand. , case. 


Visual servo control relies on techniques from image 
processing, computer vision, and control theory. In the 
present chapter, we will deal primarily with the issues 
from control theory, making connections to previous 
chapters when appropriate. 


34.1 The Basic Components of Visual Servoing 


The aim of all vision-based control schemes is to mini¬ 
mize an error e(t), which is typically defined by 

e(t) = s(m(t),a) — s* . (34.1) 

This formulation is quite general, and it encom¬ 
passes a wide variety approaches, as we will see below. 
The parameters in (34.1) are defined as follows. The 
vector m(t) is a set of image measurements (e.g., the im¬ 
age coordinates of interest points, or the parameters of 
a set of image lines or segments). These image measure¬ 
ments are used to compute a vector of k visual features, 
s(m(t),a), in which a is a set of parameters that rep¬ 
resent potential additional knowledge about the system 
(e.g., true or approximate camera intrinsic parameters 
or a model of the object to be tracked). The vector s* 
contains the desired values of the features. Note that 
the order of the desired and actual values in (34.1) is 
reversed with respect to the common convention for 
feedback control systems. 

For now, we consider the case of a fixed goal pose 
and a motionless target, i. e., s* is constant, and changes 
in .v depend only on camera motion. Further, we con¬ 
sider here the case of controlling the motion of a camera 
with six degrees of freedom (e.g., a camera attached to 
the end effector of a six-degree-of-freedom arm). We 
will treat more general cases in later sections. 

Visual servoing schemes mainly differ in the way 
that s is designed. In Sects. 34.2 and 34.3, we de¬ 
scribe classical approaches, including image-based vi¬ 
sual servo control (IB VS), in which s consists of a set 
of features that are immediately available in the image, 
and pose-based visual servo control (PBVS), in which s 
consists of a pose, which must be estimated from image 
measurements. Note that in the older visual servoing 
literature PBVS is named position-based, rather than 
pose-based, visual servoing [34.1,2]. We also present 
in Sect. 34.4 several more-advanced methods. 

Once s is selected, the design of the control scheme 
can be quite simple. Perhaps the most straightforward 
approach is to design a velocity controller. To do this, 
we require the relationship between the time variation 
of s and the camera velocity. Let the spatial velocity of 
the camera be denoted by v c = (v c , co c ) where v c is the 


instantaneous linear velocity of the origin of the camera 
frame and co c is the instantaneous angular velocity of 
the camera frame. The relationship between s and v c is 
given by 

s = L s u c , (34.2) 

in which L s e M. kx6 is called the interaction matrix re¬ 
lated to s [34.3]. The term feature Jacobian is also used 
somewhat interchangeably in the visual servo litera¬ 
ture [34.2], but in the present chapter we will use this 
latter term to relate the time variation of the features to 
the robot’s joint velocity (Sect. 34.8). 

Using (34.1) and (34.2) we immediately obtain the 
relationship between camera velocity and the time vari¬ 
ation of the error 

e = L, e v c , (34.3) 

where L e = L s . Considering v c as the input to the robot 
controller, and if we would like, for instance, to design 
for an exponential and decoupled decrease of the error 
(i. e., e = — Xe) then using (34.3) we obtain as controller 

v c = — AL+e, (34.4) 

where L+ g R 6xk is the Moore-Penrose pseudo¬ 
inverse of L e , that is, L+ = (LjL e )~*Lj when k > 6 
and L e is of full rank 6. When k = 6, if det L e 0 it is 
possible to invert L„, giving the control v c = —XLf 1 e. 
When k < 6 and L t , is of full rank k, L+ is given by 

L+=Lj(L e Lj)- 1 . 

When L c is not full rank, the numerical value of L+ 
can be obtained from the singular value decomposi¬ 
tion of L e . In all cases, control scheme (34.4) allows 
\\e — XL, e L,f e\\ and ||u c || to be minimal. Note that 
the desired behavior e = —Xe is obtained only when 
L C L+ = I/., where I*, is the k x k identity matrix, that 
is, only when L e if of full rank k, k < 6. 

In real visual servo systems, it is impossible to know 
perfectly in practice either L e or L+. So an approxima¬ 
tion or an estimation of one of these two matrices must 
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be realized. In the sequel, we denote both the pseudo¬ 
inverse of the approximation of the interaction matrix 
and the approximation of the pseudo-inverse of the in¬ 
teraction matrix by the symbol L+. Using this notation, 
the control law is in fact 

v c = -AL+e = -AL+(s-s*). (34.5) 

Closing the loop and assuming that the robot con¬ 
troller is able to realize perfectly v c , that is insert¬ 
ing (34.5) into (34.3), we obtain 

e = — XL e L^ e . (34.6) 

This equation characterizes the actual behavior of 
the closed-loop system, which is different from the de¬ 


sired one ( e = —Xe) when ever 
L e L+ /1, . 

It is also the basis of the stability analysis of the system 
using Lyapunov theory. 

What we have presented above is the basic design 
implemented by most visual servo controllers. All that 
remains is to fill in the details. How should s be cho¬ 
sen? What then is the form of L s ? How should we 
estimate L+ ? What are the performance characteristics 
of the resulting closed-loop system? These questions 
are addressed in the remainder of the chapter. We first 
describe the two basic approaches, IBVS and PBVS, 
whose principles were proposed more than 20 years 
ago [34.1], We then present more-recent approaches 
that have improved their performance. 


34.2 Image-Based Visual Servo 

Traditional image-based control schemes [34.1,4] use 
the image-plane normalised coordinates of a set of 
points to define the vector s. The image measure¬ 
ments m are usually the pixel coordinates of the set 
of image points (although this is not the only possible 
choice), and the parameters a in the definition of s = 
s(m,a) in (34.1) are nothing but the camera intrinsic 
parameters to go from image measurements expressed 
in pixels to the features. 

34.2.1 The Interaction Matrix 


A three-dimensional world point with coordinates X = 
( X , Y, Z) in the camera frame projects into the image 
plane of a conventional perspective camera as a two- 
dimensional point with normalised coordinates x = 
(x, y). More precisely we have 


X u — c„ 
Z fa 
Y V — Cy 


(34.7) 


where m = (u,v) gives the coordinates of the image 
point expressed in pixel units, and a = (c u ,c v ,f, a) is 
the set of camera intrinsic parameters as defined in 
Chap. 32: c„ and c v are the coordinates of the prin¬ 
cipal point, / is the focal length, and a is the ratio of 
the pixel dimensions. The intrinsic parameter f defined 
in Chap. 32 has been assumed to be 0 here. In this 
case, we take s = x = ( jc , y), the image plane coordi¬ 
nates of the point. The details of the imaging geometry 


and perspective projection can be found in many com¬ 
puter vision texts, including [34.5-7], 

Taking the time derivative of the projection equa¬ 
tions (34.7), we obtain 

. _X XZ _X-xZ 

Z Z ~. . Z . . (34.8) 

Y YZ _ Y-yZ 

' ~ Z~^Z? ~ Z 

We can relate the velocity of the 3-D point to the camera 
spatial velocity using the well-known equation 


X = — v x — a>yZ + a> z Y 


X = —v c -a c xX<^ 


• Y = —Vy — co z X + (l) x Z 
Z = —v z — a> x Y + a>yX , 


(34.9) 


where v c = (v x , Vy, v z ) and u> c = (co x , co y , co z ). Insert¬ 
ing (34.9) into (34.8), grouping terms, and using (34.7) 
we obtain 


— V XV 

x = ~Y~ + '—+ - (1 + x 2 )a>y + yco z 

-Vy yv z . 

v =— + ■— + (1 +y )0J x — xycOy — xco z , 

(34.10) 


which can be written 
x = L x v c , 


(34.11) 
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where the interaction matrix L A is given by 

(y 0 % -(1+* 2 ) y\ 

L x = Z , Z 

-1 y , 

\ 0 ~Y y l +y ~ x ) 

(34.12) 

In the matrix L x , the value Z is the depth of the point 
relative to the camera frame. Therefore, any control 
scheme that uses this form of the interaction matrix 
must estimate or approximate the value of Z. Similarly, 
the camera intrinsic parameters are involved in the com¬ 
putation of x and y. Thus Ly cannot be directly used 

in (34.4), and an estimation or an approximation Ly 
must be used, as in (34.5). We discuss this in more de¬ 
tail below. 

To control the six degrees of freedom, at least 
three points are necessary (i. e., we require k > 6). If 
we use the feature vector x = (xi, xt. x?), by merely 
stacking interaction matrices for three points we 
obtain 



In this case, there will exist some configurations 
for which L x is singular [34.8]. Furthermore, there ex¬ 
ist four distinct camera poses for which e = 0 , i. e., 
four global minima exist for the error function ||e||, 
and it is impossible to differentiate them [34.9]. For 
these reasons, more than three points are usually 
considered. 


34.2.2 Approximating 

the Interaction Matrix 

There are several choices available for constructing the 

estimate to be used in the control law. One popular 
scheme is of course to choose 



if L e = L x is known, that is if the current depth Z of 
each point is available [34.2]. In practice, these param¬ 
eters must be estimated at each iteration of the control 
scheme. The basic IB VS methods use classical pose- 
estimation methods (Chap. 32 and the beginning of 
Sect. 34.3). Another popular approach is to choose 



where L c * is the value of L t , for the desired position e = 

e* = 0 [34.3]. In this case, L+ is constant, and only the 
desired depth of each point has to be set, which means 
no varying 3-D parameters have to be estimated during 
the visual servo. Finally, the choice 



has been proposed [34.10]. Since L e is involved in this 
method, the current depth of each point also has to be 
available. 

We illustrate the behavior of these control schemes 
with an example. The goal is to position the cam¬ 
era so that it observes a square centered in the image 
(Fig. 34.1). We define s to include the x and y co¬ 
ordinates of the four points forming the square. Note 



Fig. 34 . 1 a-c Example of positioning task: (a) the desired camera pose with respect to a simple target, (b) the initial 
camera pose, and (c) the corresponding initial and desired image of the target 
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a) b) c) 




Fig.34.2a-c IBVS system behavior using s = (*1 , yi ,..., JC4, V4) and L+ = I/t: (a) image point trajectories including 
the trajectory of the center of the square, which is not used in the control scheme, (b) v c components (cm/s and deg/s) 
computed at each iteration of the control scheme, and (c) the 3-D trajectory of the camera optical center expressed in the 
desired camera frame R. c * (cm) 



Fig.34.3a-c IBVS system behavior using s = (x\ , x^.y^) and L^“ = L/“ (refer to Fig. 34.2a-c for description) 


that the initial camera pose has been selected far away 
from the desired pose, particularly with regard to the 
rotational motions, which are known to be the most 
problematic for IBVS. In the simulations presented in 
the following, no noise or modeling errors have been 
introduced in order to allow comparison of different be¬ 
haviors in perfect conditions. The accompanying videos 
show experimental results obtained using an Adept 
Viper robot arm and the ViSP library [34.11], The 
videos all show the same task being performed from the 
same initial conditions, and only the control approach 
varies. 

The results obtained by using 


L+=Lj 


are given in Fig. 34.2 and ItSfll'Jt'fcU. Note that de¬ 
spite the large displacement that is required the system 
converges. However, neither the behavior in the image, 
nor the computed camera velocity components, nor the 
3-D trajectory of the camera present desirable proper¬ 
ties far from the convergence (i. e., for the first 30 or so 
iterations). 


The results obtained using 


LJ = L 


-T + 


are given in Fig. 34.3 and l-EfiM'Uill'lTB. In this case, the 
trajectories of the points in the image are almost straight 
lines, which means that the points never leave the cam¬ 
era’s field of view. However the behavior induced in the 
camera frame is even less satisfactory than for the case 
of 

I + — 1 + 

The large camera velocities at the beginning of the 

servo indicate that the condition number of L+ is high 
at the start of the trajectory, and the camera trajectory is 
far from a straight line. 

The choice 

+ 




provides good performance in practice. Indeed, as can 
be seen in Fig. 34.4, the camera velocity compo¬ 
nents do not include large oscillations, which provides 


845 


Part C | 34.2 
























Part C | 34.2 


846 Part C I Sensing and Perception 



Fig. 34.4 IB VS system behavior using s = (jfi.yi_, .* 4 , >’ 4 ) and L+ = (L e /2 + L*/2) + (refer to Fig. 34.2a-c for 

description) 


a smooth trajectory in both the image and in 3-D space 


VIDEO 61 


34.2.3 A Geometrical Interpretation of I6VS 

It is quite easy to provide a geometric interpretation of 
the behavior of the control schemes defined above. The 
example illustrated in Fig. 34.5 corresponds to a pure 
rotation around the optical axis from the initial config¬ 
uration (shown in blue) to the desired configuration of 
four coplanar points parallel to the image plane (shown 
in red). 

As explained above, using L+ in the control scheme 
attempts to ensure an exponential decrease of the er- 



Fig. 34.5 Geometrical interpretation of IBVS: going from 
the blue position to the red one. In green , image motion 
when L^“ is used in the control scheme; in blue, when Ljt 
is used; and in black when (L e /2 + L„* /2)~*~ is used (see 
the text for more details) 


ror e. This means that, when .r and y image point coor¬ 
dinates compose this error, the points’ trajectories in the 
image follow straight lines from their initial to their de¬ 
sired positions, when this is possible. This leads to the 
image motion plotted in green in the figure. The cam¬ 
era motion to realize this image motion can be easily 
deduced and is indeed composed of a rotational motion 
around the optical axis, but combined with a retreat¬ 
ing translational motion along the optical axis [34.12], 
This unexpected motion is due to the choice of the fea¬ 
tures and the form of the third and sixth columns in the 
interaction matrix, which induces a coupling between 
the features and the two degrees of freedom involved 
( v z and co z ). If the rotation between the initial and de¬ 
sired configurations is very large, this phenomenon is 
amplified, and leads to a particular case for a rotation 
of tc radians where no rotational motion at all will be 
induced by the control scheme [34.13]. On the other 
hand, when the rotation is small, this phenomenon al¬ 
most disappears. To conclude, the behavior is locally 
satisfactory (i. e., when the error is small), but it can be 
unsatisfactory when the error is large. As we will see 
below, these results are consistent with the local asymp¬ 
totic stability results that can be obtained for IBVS. 

If instead we use L t (t in the control scheme, the im¬ 
age motion generated can easily be shown to be the blue 
one plotted in Fig. 34.5. Indeed, if we consider the same 
control scheme as before but starting from s* to reach s, 
we obtain 

v c = -ALjj (s* -s) , 

which again induces straight-line trajectories from the 
red points to the blue ones, causing the image motion 
plotted in brown. Going back to our problem, the con¬ 
trol scheme computes a camera velocity that is exactly 
the opposite one 

v c = —AI/t(s —s*) , 
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and thus generates the image motion plotted in red at the 
red points. Transformed to the blue points, the camera 
velocity generates the blue image motion and corre¬ 
sponds once again to a rotational motion around the 
optical axis, combined now with an unexpected forward 
motion along the optical axis. The same analysis can 
be done as before for the case of large or small errors. 
We can add that, as soon as the error decreases signifi¬ 
cantly, both control schemes get closer, and tend to the 
same one (since L e = L f * when e = e*) with a nice be¬ 
havior characterized with the image motion plotted in 
black and a camera motion composed of only a rotation 
around the optical axis when the error tends towards 
zero. 

If we instead use 



it is intuitively clear that considering the mean of L e 
and L c * generates the image motion plotted in black, 
even when the error is large. In all cases but the rotation 
of jr radians, the camera motion is now a pure rotation 
around the optical axis, without any unexpected trans¬ 
lational motion. 

34.2.4 Stability Analysis 

We now consider the fundamental issues related to 
the stability of IB VS. To assess the stability of the 
closed-loop visual servo systems, we will use Lya¬ 
punov analysis. In particular, consider the candidate 
Lyapunov function defined by the squared error norm 
£ = | ||e(t) || 2 , whose derivative is given by 

£ = e T e = —Ae T L c L +e 

since e is given by (34.6). The global asymptotic stabil¬ 
ity of the system is thus obtained when the following 
sufficient condition is satisfied 

L e L+>0. (34.13) 

If the number of features is equal to the number 
of camera degrees of freedom (i. e., k = 6), and if the 
features are chosen and the control scheme designed 

so that L e and L+ are of full rank 6, then the condi¬ 
tion (34.13) is satisfied if the approximations involved 

in L+ are not too coarse. 

As discussed above, for most IBVS approaches we 
have k > 6. Therefore the condition (34.13) can never 
be ensured since 

L e L+ g R kxk 


is at most of rank 6, and thus has a nontrivial null 

space. In this case, configurations such that e e ker L+ 
correspond to local minima. Reaching such a local 
minimum is illustrated in Fig. 34.6. As can be seen 
in Fig. 34. 6d, each component of e has a nice exponen¬ 
tial decrease with the same convergence speed, causing 
straight-line trajectories to be realized in the image, but 
the error reached is not exactly zero, and it is clear 
from Fig. 34.6c that the system has been attracted to 
a local minimum far away from the desired config¬ 
uration. Thus, only local asymptotic stability can be 
obtained for IBVS. 

To study local asymptotic stability when k > 6, let 

us first define a new error e' with e' = L+ e. The time 
derivative of this error is given by 

e' = L+e + L+e = (l+L, + o) v c , 

where O G K 6x6 is equal to 0 when e = 0, what¬ 
ever the choice of L+ [34.14]. Using the control 
scheme (34.5),w e obtain 

e' = -X (L+L e + (?) e ', 

which is known to be locally asymptotically stable in 
a neighborhood of e = e* = 0 if 

L+L c >0, (34.14) 

where L+L e G M 6x6 . Indeed, only the linearized sys¬ 
tem 

e' = -AL+L e e' 

has to be considered if we are interested in the local 
asymptotic stability [34.15]. 

Once again, if the features are chosen and the con¬ 
trol scheme designed so that L e and L+ are of full 
rank 6, then condition (34.14) is ensured if the approx¬ 
imations involved in L+ are not too coarse. 

To end the demonstration of local asymptotic sta¬ 
bility, we must show that there does not exist any 

configuration e ^ e* such that e G ker L+ in a small 
neighborhood of e* and in a small neighborhood of 
the corresponding pose p*. Such configurations corre¬ 
spond to local minima where v c = 0 and e =/= e* ,\ I such 
a pose p would exist, it is possible to restrict the neigh¬ 
borhood around p* so that there exists a camera velocity 
v to reach p* from p. This camera velocity would im¬ 
ply a variation of the error e = L e v. However, such 

a variation cannot belong to ker since L+L e > 0. 
Therefore, we have v c = 0 if and only if e = 0. i. e., 
e = e*\ in a neighborhood of p*. 
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Fig.34.6a-e IB VS reaching a local minimum using s = (x\,y\ ,... ,M, V4) and Lj*“ = L+: (a) the initial configuration, 
(b) the desired one, (c) the configuration reached after the convergence of the control scheme, (d) the evolution of the 
error e at each iteration of the control scheme, and (e) the evolution of the six components of the camera velocity v c 


Even though local asymptotic stability can be en¬ 
sured when k > 6, we recall that global asymptotic 
stability cannot be ensured. For instance, as illustrated 
in Fig. 34.6, there may exist local minima correspond¬ 
ing to configurations where e e ker L+, which are 
outside of the neighborhood considered above. Deter¬ 
mining the size of the neighborhood in which stability 
and the convergence are ensured is still an open issue, 
even if this neighborhood is surprisingly quite large in 
practice. 

34.2.5 IBVS with a Stereo Vision System 

It is straightforward to extend the IBVS approach to 
a multicamera system. If a stereo vision system is used, 
and a world point is visible in both left and right images 
(Fig. 34.7), it is possible to use as visual features 

s=x s = (*,,x r ) = (xi,yi,x r ,y r ) 

i. e., to represent the point by just stacking in s the x and 
y coordinates of the observed point in the left and right 


images [34. 16]. However, care must be taken when con¬ 
structing the corresponding interaction matrix since the 
form given in (34.11) is expressed in either the left or 
right camera frame. More precisely, we have 

Ui = Ljcj V\ , 

|± r = L Xr v r , 

where V\ and v r are the spatial velocity of the left 
and right camera, respectively, and where the analyti¬ 
cal form of L XI and L Vr are given by (34.12). 

By choosing a sensor frame rigidly linked to the 
stereo vision system, we obtain 



where the interaction matrix related to x s can be de¬ 
termined using the spatial motion transform matrix V 
defined in Chap. 02 to transform velocities expressed in 
the left or right cameras frames to the sensor frame. We 
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recall that V is given by 

v-C [ 'r R ) • 


instead of their Cartesian coordinates x = (.r, y). They 
are given by 


(34.15) 


P = 


\/x 2 +y 2 , 9 = arctan 


y 

X 


where [f] x is the skew-symmetric matrix associated to 
the vector t and where (R.f) e SE( 3) is the rigid-body 
transformation from the camera to the sensor frame. 
The numerical values for these matrices are directly 
obtained from the calibration step of the stereo vision 
system. Using this equation, we obtain 



Note that L, Xs e ]R 4x6 is always of rank 3 because of 
the epipolar constraint that links the perspective projec¬ 
tion of a 3-D point in a stereo vision system (Fig. 34.7). 
Another simple interpretation is that a 3-D point is rep¬ 
resented by three independent parameters, which makes 
it impossible to find more than three independent pa¬ 
rameters using any sensor observing that point. 

To control the six degrees of freedom of the sys¬ 
tem, it is necessary to consider at least three points, as 
the rank of the interaction matrix considering only two 
points is 5. 

Using a stereo vision system, since the 3-D coordi¬ 
nates of any point observed in both images can be easily 
estimated by a simple triangulation process it is possi¬ 
ble and quite natural to use these 3-D coordinates in 
the features set s. Such an approach would be, strictly 
speaking, a position-based approach, since it would re¬ 
quire 3-D parameters in s. 

34.2.6 IBVS with Cylindrical Coordinates 
of Image Points 

In the previous sections, we have considered the 
Cartesian coordinates of image points. As proposed 
in [34.17] it may be useful to consider instead the 
cylindrical coordinates y = (p, 9 ) of the image points 



from which we deduce 

. xx + yy ■ xy-yx 
P = -, 8 =-. 

P P 2 

Using (34. 11 ) and then substituting x by p cos 8 and y 
by p sin 8, we obtain immediately 

' ~z z~ / ( 1 + p2 ' )i - ( 1 + p 2 ) c 0> 
Ly= A Z£ 0 - - -1 ’ 

\pZ pZ p p ) 

(34.16) 

where c = cos 9 and 5 = sin 9. Note that 9 is not de¬ 
fined when the image point lies at the principal point 
(where x = y = p = 0). It is thus not surprising that the 
interaction matrix L y is singular in that case. 

If we go back to the example depicted in Fig. 34.5, 
the behavior obtained using cylindrical coordinates 
will be the expected one, that is a pure rotation 
around the optical axis, by using either L+, L~t or 
(L e /2 + L e »/2) + in the control scheme. This is due to 
the form of the third and sixth columns of the interac¬ 
tion matrix (34.16), which leads to a decoupled system. 

34.2.7 IBVS with Other Geometrical Features 

In the previous sections, we have only considered image 
point coordinates in s. Other geometrical primitives can 
of course be used. There are several reasons to do so. 
Firstly, the scene observed by the camera cannot always 
be described merely by a collection of points, in which 
case the image processing provides other types of mea¬ 
surements, such as a set of straight lines or the contours 
of an object. Secondly, richer geometric primitives may 
ameliorate the decoupling and linearizing issues that 
motivate the design of partitioned systems (Sect. 34.4). 
Finally, the robotic task to be achieved may be ex¬ 
pressed in terms of virtual linkages (or fixtures) be¬ 
tween the camera and the observed objects [34.18, 19], 
sometimes expressed directly by constraints between 
primitives, such as point-to-line [34.20] (which means 
that an observed point must lie on a specified line). 

It is possible to determine the interaction matrix 
related to the perspective projection of a large class 
of geometrical primitives, such as segments, straight 
lines, spheres, circles, and cylinders. The results are 
given in [34.3,18]. Recently, the analytical form of the 
interaction matrix related to any image moments cor¬ 
responding to planar objects has been computed. This 


Fig. 34.7 A stereo vision system 
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Fig. 34.8 Unified imaging model of Geyer and Daniilidis 
(after [34.7]) 


makes it possible to consider planar objects of any 
shape [34.21], If a collection of points is measured in 
the image, moments can also be used [34.22]. In both 
cases, moments allow the use of intuitive geometrical 
features, such as the center of gravity or the orienta¬ 
tion of an object. By selecting an adequate combination 
of moments, it is then possible to determine partitioned 
systems with good decoupling and linearizing proper¬ 
ties [34.21,22], 

Note that, for all these features (geometrical prim¬ 
itives, moments), the depth of the primitive or of the 
object considered appears in the coefficients of the 
interaction matrix related to the translational degrees 
of freedom, as was the case for the image points. 
An estimation of this depth is thus generally neces¬ 
sary (Sect. 34.6). In a few situations, for instance with 
a suitable normalization of moments [34.22], only the 
constant desired depth appears in the interaction matrix, 
which makes estimating depth unnecessary. 

34.2.8 Non-Perspective Cameras 

The vast majority of cameras we use, and our own eyes, 
are characterized by a perspective projection model 
which closely approximate the ideal pinhole imaging 
model. Such cameras have a narrow field of view, typ¬ 
ically less than one half hemisphere. For robotics it is 
often advantageous to have a large field of view and 
this can be achieved using a fisheye lens camera or 
a catadioptric (lens and mirror system) camera (often 
referred to as a panoramic camera). For these non¬ 
perspective sensors the interaction matrix of any visual 
feature has a different form to those discussed above, 
such as (34.12) and (34.16) for an image point. 


Rather than determine the interaction matrix of 
visual features expressed in the image plane of non¬ 
perspective cameras, we can transform the images from 
these cameras to the view that would be seen by an ideal 
spherical camera. The spherical model projects world 
points onto a unit sphere, the intersection of the unit 
sphere with the ray from the world point to the center of 
the sphere. Such an ideal camera has the largest possible 
field of view. The unified imaging model [34.23] shown 
in Fig 34.8b provides a general mechanism to project 
a world point to the image plane of a large class of 
cameras. To be precise it includes all central projection 
cameras and this includes perspective and some cata¬ 
dioptric cameras with particular mirror shapes, but in 
practice it is a very good approximation to non-central 
cameras including fisheye and general catadioptric sys¬ 
tems. The mechanism of this unified model can also be 
used to reproject points from these varied image planes 
to a spherical camera. 

Referring to Fig. 34.8 the world point P is rep¬ 
resented by the vector X = ( X , 7, Z) in the camera 
frame, and is projected onto the surface of the unit 
sphere at the point x s = (jq, y 5 , z s ) with 

X Y Z 

R y R R 


where R = VX 2 + Y 2 + Z 2 is the distance from the 
sphere center to the world point. 

The interaction matrix of jc s is derived in essentially 
the same manner as for the perspective camera, and it 
can be shown to be [34.24] 


L x = 


(hh 1 -13) kvj 


(34.17) 


Note that R can be expressed as a function of the point 
depth Z by using R = Z y/l + x 2 s +Vj. Therefore, the 
general spherical model does not add any supplemen¬ 
tary unknown in the interaction matrix. 

A particular advantage of the spherical model is that 
for pure camera rotation the shape of an object is in¬ 
variant, which eases determining visual features that are 
only linked to translational motions. 


34.2.9 Direct Estimation 


In the previous sections, we have focused on the ana¬ 
lytical form of the interaction matrix. It is also possible 
to estimate its numerical value directly using either an 
offline learning step, or an online estimation scheme. 

All the methods proposed to estimate the inter¬ 
action matrix numerically rely on the observation of 
a variation of the features due to a known or measured 
camera motion. More precisely, if we measure a fea¬ 
ture’s variation As due to a camera motion Av c , we 
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have from (34.2) 

L s At; c = As , 

which provides k equations while we have k x 6 un¬ 
known values in L s . Using a set of N independent 
camera motions with N > 6, it is thus possible to es¬ 
timate L s by solving 

L S A=B , 

where the columns of A e R 6xN and B e M* XA? are, re¬ 
spectively, formed from the set of camera motions and 
the set of corresponding features variations. The least- 
square solution is of course given by 

L S =BA + . (34.18) 

Methods based on neural networks have also been de¬ 
veloped to estimate L s [34.25, 26]. It is also possible to 
estimate the numerical value of L+ directly, which in 
practice provides a better behavior [34.27]. In this case, 
the basic relation is 

L+ As = Av c , 

which provides six equations. Using a set of N measure¬ 
ments, with N > k, we now obtain 

L+ =AB + . (34.19) 


34.3 Pose-Based Visual Servo 

Pose-based control schemes (PBVS) [34.1,32,33] use 
the pose of the camera with respect to some reference 
coordinate frame to define s. Computing this pose from 
a set of measurements in one image necessitates the 
camera intrinsic parameters and the 3-D model of the 
object observed to be known. This classic computer vi¬ 
sion problem is called the 3-D localization problem. 
While this problem is beyond the scope of the present 
chapter, many solutions have been presented in the lit¬ 
erature [34.34, 35] and its basic principles are recalled 
in Chap. 32. 

It is then typical to define s in terms of the parame¬ 
terization used to represent the camera pose. Note that 
the parameters a involved in the definition (34.1) of s 
are now the camera intrinsic parameters and the 3-D 
model of the object. 

It is convenient to consider three coordinate frames: 
the current camera frame jp c , the desired camera 
frame (Ft*, and a reference frame (To attached to the 


In the first case (34.18), the six columns of L„ are esti¬ 
mated by solving six linear systems, while in the second 
case (34.19), the k columns of L+ are estimated by 
solving k linear systems, which explains the difference 
in the results. 

Estimating the interaction matrix online can be 
viewed as an optimization problem, and consequently 
a number of researchers have investigated approaches 
that derive from optimization methods. These methods 
typically discretize the system equation (34.2), and use 
an iterative updating scheme to refine the estimate of L s 
at each stage. One such online and iterative formulation 
uses the Broyden update rule given by [34.28, 29] 

L s (t + 1) = L s (f) 

where a defines the update speed. This method has been 
generalized to the case of moving objects in [34.30]. 

The main interest of using such numerical estima¬ 
tions in the control scheme is that it avoids all the 
modeling and calibration steps. It is particularly use¬ 
ful when using features whose interaction matrix is not 
available in analytical form. For instance, in [34.3 1], the 
main eigenvalues of the principal component analysis 
of an image have been considered in a visual servoing 
scheme. The drawback of these methods is that no the¬ 
oretical stability and robustness analysis can be made. 


object. We adopt here the standard notation of using 
a leading superscript to denote the frame with respect 
to which a set of coordinates is defined. Thus, the co¬ 
ordinate vectors c t 0 and c t a give the coordinates of the 
origin of the object frame expressed relative to the cur¬ 
rent camera frame, and relative to the desired camera 
frame, respectively. Furthermore, let R = c R c be the 
rotation matrix that gives the orientation of the current 
camera frame relative to the desired frame. 

We can define s to be (t, 6u), in which t is a trans¬ 
lation vector, and 9u gives the angle/axis parameteriza¬ 
tion for the rotation. We now discuss two choices for t , 
and give the corresponding control laws. 

If t is defined relative to the object frame J a , we 
obtain 

s = Ct 0 ,9u),s* =( c *t 0 ,0 ), 

and 


e = (% 


t a , 6u). 
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Fig. 34.9 PBVS system behavior using s = ( c f 0 > Ou) (refer to Fig. 34.2a-c for description) 


In this case, the interaction matrix related to e is given 
by 

L '=(t '!!;';)• 

in which 1 3 is the 3 x 3 identity matrix and L$„ is given 
by [34.36] 

6 , , / sine 9 \ r ,, 

L du = I3 + x Mx + ( 1 -:—tttt ) [m]' x , 

2 V sinc“61/2 / 

(34.21) 


motion to follow a geodesic with an exponential de¬ 
creasing speed and causes the translational parameters 
involved in s to decrease at the same speed. This ex¬ 
plains the nice exponential decrease of the camera 
velocity components in Fig. 34.9. Furthermore, the tra¬ 
jectory in the image of the origin of the object frame 
follows a straight line (here the center of the four points 
has been selected as this origin). On the other hand, 
the camera trajectory does not follow a straight line 
(i <s3>wimim). 

Another PBVS scheme can be designed by using 
s = ( c t c , 9u). In this case, we have s* = 0, e = s, and 


where sineje is the sinus cardinal defined such that 
x sine x = sin x and sine 0 = 1 . 

Following the development in Sect. 34.1, we obtain 
the control scheme 

v c = —XL~ 1 e 



Note the decoupling between translational and rota¬ 
tional motions, which allows us to obtain a simple 
control scheme 


since the dimension k of s is six, that is, the number of 
camera degrees of freedom. By setting 


v c = — AR t c *t c 
o) c = —X 9u 


(34.25) 



%] x L 

L eu 



we obtain after simple developments 

j v c = -X - %] + [ c f D ] x 9u 
I « c = —X611 


(34.22) 


(34.23) 


since ~Lq u is such that 

Lgi = . 

Ideally, that is, if the pose parameters are per¬ 
fectly estimated, the behavior of e will be the expected 
one (e = —Xe). The choice of e causes the rotational 


In this case, as can be seen in Fig. 34.10 and 
in l<£z>MlltHLlTB, if the pose parameters involved 
in (34.25) are estimated perfectly, the camera trajec¬ 
tory is a straight line, while the image trajectories are 
less satisfactory than before. Some particular configu¬ 
rations can be found which will lead to some points 
leaving the camera field of view during the robot 
motion. 

The stability properties of PBVS seem quite attrac¬ 
tive. Since L@„ given in (34.21) is nonsingular when 
9 ^ 2 kit, 'ik G Z*, we obtain from (34.13) the global 

asymptotic stability of the system since T t .T)T 1 = If,, 
under the strong hypothesis that all the pose parame¬ 
ters are perfect. This is true for both methods presented 
above, since the interaction matrices given in (34.20) 
and (34.24) are full rank when L^,, is nonsingular. 
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Fig. 34.10 PBVS system behavior using s = { c t c , 6u) (refer to Fig. 34.2a-c for description) 
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Fig.34.11a-c Two different camera poses (a,c) that provide almost the same image of four coplanar points shown over¬ 
laid in (b) 


With regard to robustness, feedback is computed us¬ 
ing estimated quantities that are a function of the image 
measurements and the system calibration parameters. 
For the first method presented in Sect. 34.3 (the analy¬ 
sis for the second method is analogous), the interaction 
matrix given in (34.20) corresponds to perfectly esti¬ 
mated pose parameters, while the real one is unknown 
since the estimated pose parameters may be biased due 
to calibration errors, or inaccurate and unstable due 
to noise [34.13]. The true positivity condition (34.13) 


should in fact be written 

LjLT 1 > 0 , (34.26) 

where Lr 1 is given by (34.22) but where L% is un¬ 
known, and not given by (34.20). Indeed, even small 
errors in computing the position of points in the im¬ 
age can lead to pose errors which will significantly 
impact the accuracy and the stability of the system 
(Fig. 34.11). 
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34.4 Advanced Approaches 

34.4.1 Hybrid VS 

Suppose we have access to a control law for co c , such as 
the one used in PBVS ((34.23) or (34.25)) 

0 ) c = —A 611 . (34.27) 

How could we use this in conjunction with IBVS? 

Considering a feature vector s t and an error e, de¬ 
voted to control the translational degrees of freedom, 
we can partition the interaction matrix as follows 

= Ls, tt c 

= (L v L») 

— L v v c + L w w c . 

Now, setting e t = —Xe u we can solve for the desired 
translational control input as 

— = — L v v c + L co(*) c , 

=> v c =-L+(Ae t + L w w c ) . (34.28) 

We can think of the quantity (Ae t +L M (o c ) as 
a modified error term, one that combines the orig¬ 
inal error with the error that would be induced by 
the rotational motion due to oj c . The translational 
control input v c = — LjT ( Xe t + L M co c ) will drive this 
error to zero. The method known as 2.5-D visual 
servo [34.36] was the first to exploit such a parti¬ 
tioning in combining IBVS and PBVS. More pre¬ 
cisely, in [34.36], s t has been selected as the coor¬ 
dinates of an image point, and the logarithm of its 
depth, so that L v is a triangular always invertible 
matrix. More precisely, we have s t = (jc, logZ), s* = 
(x*,logZ*), e t = (x — x*, log p z ) where pz=Z/Z*, 
and 



/ xy -(1+x 2 ) y \ 
1 +y 2 -xy -x 

\ —y X 0) 


Note that the ratio p z can be obtained directly from the 
partial pose estimation algorithm that will be described 
in Sect. 34.6. 

If we come back to the usual global representation 
of visual servo control schemes, we have e = (e t , 6u) 


and L, e given by 



from which we immediately obtain the control 
law (34.27) and (34.28) by applying (34.5). 

The behavior obtained using this choice for s t is 
shown in Fig. 34.12 and Here, the point 

that has been considered in s t is the center of gravity 
x g of the target. We note the image trajectory of that 
point, which is a straight line as expected, and the nice 
decreasing of the camera velocity components, which 
makes this scheme very similar to the first PBVS one. 

As for stability, it is clear that this scheme is 
globally asymptotically stable in perfect conditions. 
Furthermore, thanks to the triangular form of the inter¬ 
action matrix L e , it is possible to analyze the stability 
of this scheme in the presence of calibration errors us¬ 
ing the partial pose-estimation algorithm that will be 
described in Sect. 34.6 [34.37]. Finally, the only un¬ 
known constant parameter involved in this scheme, that 
is Z*, can be estimated online using adaptive tech¬ 
niques [34.38], 

Other hybrid schemes can be designed. For in¬ 
stance, in [34.39], the third component of s t is different 
and has been selected so that all the target points remain 
in the camera field of view as far as possible. Another 
example has been proposed in [34.40]. In that case, s 
is selected as s = ( c t c ,x g ,6u z ) which provides with 
a block-triangular interaction matrix of the form 



where L' and I/ M can easily be computed. This scheme 
is such that, under perfect conditions, the camera tra¬ 
jectory is a straight line (since c t c is a part of s), and 
the image trajectory of the center of gravity of the ob¬ 
ject is also a straight line (since x g is also a part of 
s). The translational camera degrees of freedom are 
devoted to realize the 3-D straight line, while the rota¬ 
tional camera degrees of freedom are devoted to realize 
the 2-D straight line and also compensate the 2-D mo¬ 
tion of x 2 due to the translational motion. As can be seen 
in Fig. 34.13 and l<s>MMSI3i, this scheme is particu¬ 
larly satisfactory in practice. 

Finally, it is possible to combine 2-D and 3-D 
features in different ways. For instance, in [34.41], 
it has been proposed to use in s the 2-D homoge¬ 
neous coordinates of a set of image points expressed 
in pixels multiplied by their corresponding depth: 
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Fig. 34.12 2.5-D VS system behavior using s = (x g , log (Z g ), 611 ) (refer to Fig. 34.2a-c for description) 





a) 



b) c) 




Fig. 34.13 2.5-D VS system behavior using s = ( c *t c , x g , 6 u z ) (refer to Fig. 34.2a-c for description) 


s= (u\Zi, v\Z\,Z\, ■ • • , m„Z„, v n Z n , Z n ). As for classi¬ 
cal IB VS, we obtain in this case a set of redundant 
features, since at least three points have to be used to 
control the six camera degrees of freedom (here k> 9). 
However, it has been demonstrated in [34.42] that this 
selection of redundant features is free of attractive local 
minima. 

34.4.2 Partitioned VS 

The hybrid visual servo schemes described above have 
been designed to decouple the rotational motions from 
the translational ones by selecting adequate visual fea¬ 
tures defined in part in 2-D, and in part in 3-D (which is 
why they have been called 2.5-D visual servoing). This 
work has inspired some researchers to find features that 
exhibit similar decoupling properties but using only fea¬ 
tures expressed directly in the image. More precisely, 
the goal is to find six features such that each is related 
to only one degree of freedom (in which case the inter¬ 
action matrix is a diagonal matrix). The Grail is to find 
a diagonal interaction matrix whose elements are con¬ 
stant, as near as possible to the identity matrix, leading 
to a pure, direct, and simple linear control problem. 

The first work in this area partitioned the interaction 
matrix to isolate motion related to the optic axis [34.12]. 


Indeed, whatever the choice of s, we have 
s = L s v c 

— L xyV xy T I V 

— $xy + S z 

in which L x> . includes the first, second, fourth, and 
fifth columns of L s , and L. includes the third and 
sixth columns of L s . Similarly, = (v x , Vy, a> x , co y ) 
and v z = (v z , co z ). Here, s z = L z v z gives the component 
of s due to the camera motion along and rotation about 
the optic axis, while = L x> , v xy gives the component 
of s due to velocity along and rotation about the camera 
x andy axes. 

Proceeding as above, by setting e = —Xe we obtain 
— Xc = c = s = Lj tyVxy + L-tt. , 
which leads to 

= -L+[Ae(f) + L,'tt-.] . 

As before, we can consider [Ae(t) + L z i; z ] as a modified 
error that incorporates the original error while taking 
into account the error that will be induced by v z . 
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Given this result, all that remains is to choose s and 
v z . As for basic IBVS, the coordinates of a collection 
of image points can be used in s, while two new image 
features can be defined to determine v z : 

• Define a , with 0 < a < 2 tt, as the angle between the 
horizontal axis of the image plane and the directed 
line segment joining two feature points. 

It is clear that a is closely related to the rotation 
around the optic axis. 


• Define a 2 to be the area of the polygon defined by 
these points. Similarly, a 1 is closely related to the 
translation along the optic axis. 

Using these features, v z has been defined in [34.12] 
as 

U =A„ z ln^, 

\co z =Xco z (a*-a). 


34.5 Performance Optimization and Planning 


In some sense, partitioned methods represent an effort 
to optimize system performance by assigning distinct 
features and controllers to individual degrees of free¬ 
dom. In this way, the designer performs a sort of 
offline optimization when allocating controllers to de¬ 
grees of freedom. It is also possible to explicitly design 
controllers that optimize various system performance 
measures. We describe a few of these in this section. 

34.5.1 Optimal Control 

and Redundancy Framework 

An example of such an approach is given in [34.43] 
and [34.44], in which linear quadratic Gaussian (LQG) 
control design is used to choose gains that mini¬ 
mize a linear combination of state and control inputs. 
This approach explicitly balances the trade-off between 
tracking errors (since the controller attempts to drive 
s — s* to zero) and robot motion. A similar control 
approach is proposed in [34.45] where joint limit avoid¬ 
ance is considered simultaneously with the positioning 
task. 

It is also possible to formulate optimality criteria 
that explicitly express the observability of robot motion 
in the image. For example, the singular value decompo¬ 
sition of the interaction matrix reveals which degrees of 
freedom are most apparent and can thus be easily con¬ 
trolled, while the condition number of the interaction 
matrix gives a kind of global measure of the visibility 
of motion. This concept has been called resolvabil¬ 
ity in [34.46] and motion perceptibility in [34.47]. By 
selecting features and designing controllers that max¬ 
imize these measures, either along specific degrees of 
freedom or globally, the performance of the visual servo 
system can be improved. 

The constraints considered to design the control 
scheme using the optimal control approach may be con¬ 
tradictory in some cases, leading the system to fail due 
to local minima in the objective function to be min¬ 


imized. For example, it may happen that the motion 
produced to move away from a robot joint limit is ex¬ 
actly the opposite of the motion produced toward the 
desired pose, which results in a zero global motion. To 
avoid this potential problem, it is possible to use the gra¬ 
dient projection method, which is classical in robotics. 
Applying this method to visual servoing has been pro¬ 
posed in [34.3, 19]. The approach consists of projecting 
the secondary constraints e s onto the null space of the 
vision-based task e so that they have no effect on the 
regulation of e to 0 

e s = L t e + G - 

where e g is the new global task considered and 

p c = (i 6 -l+C) 

is such that 

L e P e e s = 0 , We s . 

Avoiding the robot joint limits using this approach has 
been presented in [34.48]. However, when the vision- 
based task constrains all the camera degrees of freedom, 
the secondary constraints cannot be considered since, 
when L e is of full rank 6, we have P e e s = 0, Ve s . In this 
case, it is necessary to insert the constraints into a global 
objective function, such as navigation functions that are 
free of local minima [34.49, 50]. 

34.5.2 Switching Schemes 

The partitioned methods described previously attempt 
to optimize performance by assigning individual con¬ 
trollers to specific degrees of freedom. Another way to 
use multiple controllers to optimize performance is to 
design switching schemes that select at each moment 
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Fig. 34.14 Image feature trajectories for a rotation of 160° 
about the optical axis using a switched control scheme (ini¬ 
tial point positions in blue, and desired points position in 
red) 

in time which controller to use based on criteria to be 
optimized. 

A simple switching controller can be designed us¬ 
ing an IB VS and a PBVS controller as follows [34.51]. 
Let the system begin by using the IB VS controller. 
Consider the Lyapunov function for the PBVS con¬ 
troller given by £p = ^||ep(f)|| 2 , with ep(t) = ( c t a — 
c t o ,0u). If at any time the value of this Lyapunov 
function exceeds a threshold yp, the system switches 
to the PBVS controller. While using the PBVS con¬ 
troller, if at any time the value of the Lyapunov function 
£1 for the IB VS controller exceeds a threshold, L\ = 
5 lki(01| 2 > Yi, the system switches to the IBVS con¬ 
troller. With this scheme, when the Lyapunov function 
for a particular controller exceeds a threshold, that con¬ 
troller is invoked, which in turn reduces the value of 
the corresponding Lyapunov function. If the switching 
thresholds are selected appropriately, the system is able 
to exploit the relative advantages of IBVS and PBVS, 
while avoiding their shortcomings. 

An example of such a system is shown in Fig. 34.14 
for the case of a rotation by 160° about the optical axis. 
Note that the system begins in IBVS mode and the fea¬ 
tures initially move on straight lines toward their goal 


positions in the image. However, as the camera retreats, 
the system switches to PBVS, which allows the camera 
to reach its desired position by combining a rotational 
motion around its optic axis and a forward translational 
motion, producing the circular trajectories observed in 
the image. 

Other examples of temporal switching schemes can 
be found, such as the one developed in [34.52], to en¬ 
sure the visibility of the target observed. 

34.5.3 Feature Trajectory Planning 

It is also possible to treat the optimization problem 
offline, during a planning stage, if we have sufficient 
knowledge of the system and world. In this case, several 
constraints can be taken into account simultaneously, 
such as obstacle avoidance [34.53], joint limit and oc¬ 
clusions avoidance, and ensuring the visibility of the 
target [34.54]. The feature trajectories s*(t ) that allow 
the camera to reach its desired pose while ensuring that 
the constraints are satisfied are determined using path 
planning techniques, such as the well-known potential 
field approach [34.54] or linear matrix inequality opti¬ 
mizations [34.55]. 

Coupling path planning with trajectory following 
also allows the robustness of the visual servo with re¬ 
spect to modeling errors to be significantly improved. 
Indeed, modeling errors may have large effects when 
the error s—s* is large, but have little effect when s—s* 
is small. Once the desired features trajectories s*(t) 
such that s*(0) = s(0) have been designed during the 
planning stage, it is easy to adapt the control scheme to 
take into account the fact that s* is varying, and to make 
the error s—s* remain small. More precisely, we now 
have 

e = s—s =L e v c —s , 

from which we deduce, by selecting as usual e = —Xe 
as the desired behavior, 

v c = -AL+e + L+s* . 

The new second term of this control law anticipates the 
variation of s*, removing the tracking error it would 
produce. We will see in the 34.8 that the form of the 
control law is similar when tracking of a moving target 
is considered. 
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34.6 Estimation of 3-D Parameters 

All the control schemes described in the previous sec¬ 
tions use 3-D parameters that are not directly available 
from the image measurements. As for IBVS, we recall 
that the range of the object with respect to the camera 
appears in the coefficients of the interaction matrix re¬ 
lated to the translational degrees of freedom. Noticeable 
exceptions are the schemes based on a numerical esti¬ 
mation of L e orofL+ (Sect. 34.2.8). Another exception 

is the IBVS scheme that uses the constant matrix Ljt 
in the control scheme, in which only the depth for the 
desired pose is required, which is not so difficult to ob¬ 
tain in practice. As for PBVS and hybrid schemes that 
combine 2-D and 3-D data in e, 3-D parameters ap¬ 
pear both in the error e and in the interaction matrix. 
A correct estimation of the 3-D parameters involved is 
thus important for IBVS since they will have an effect 
on the camera motion during the task execution (they 
appear in the stability conditions (34.13) and (34.14)), 
while a correct estimation is crucial in PBVS and hy¬ 
brid schemes since they will have also an effect on the 
accuracy of the pose reached after convergence. 

If a calibrated stereo vision system is used, all 3-D 
parameters can be easily determined by triangulation, 
as mentioned in Sect. 34.2.5 and described in Chap. 32. 
Similarly, if a 3-D model of the object is known, all 3-D 
parameters can be computed from a pose-estimation al¬ 
gorithm. However, we recall that such an estimation can 
be quite unstable due to image noise (Sect. 34.3). As 
already said, IBVS does not require full pose estima¬ 
tion, simply the range of the object with respect to the 
camera. When image points are involved in s, the range 
is expressed as the scalar depth Z or distance R of the 
corresponding world points, which appears in the inter¬ 
action matrix (34.12), (34.16) and (34.17). This can be 
considered as a parameter estimation problem, that is, 
estimating the 3-D parameters from knowledge of the 
analytical form of the interaction matrix, and measure¬ 
ments of camera motion and visual feature position and 
velocity [34.56-58], 

It is also possible to estimate 3-D parameters by us¬ 
ing the epipolar geometry that relates the images of the 
same scene observed from different viewpoints. Indeed, 
in visual servoing, two images are generally available: 
the current one and the desired one. Given a set of 
matches between the image measurements in the cur¬ 
rent image and in the desired one, the fundamental 
matrix, or the essential matrix if the camera is cali¬ 
brated, can be recovered [34.6], and then used in visual 
servoing [34.59]. Indeed, from the essential matrix, the 
rotation and the translation up to a scalar factor between 
the two views can be estimated. However, near the con¬ 


vergence of the visual servo, that is, when the current 
and desired images are similar, the epipolar geometry 
becomes degenerate and it is not possible to estimate 
accurately the partial pose between the two views. For 
this reason, using homography is generally preferred. 

Let x, and x* denote the homogeneous image co¬ 
ordinates for a point in the current and desired images. 
Then x, is related to x* by 

xi = H,x* 

in which H, is a homography matrix. 

If all feature points lie on a 3-D plane, then there 
is a single homography matrix H such that x, = Hx* 
for all i. This homography can be estimated using the 
position of four matched points in the desired and the 
current images. If all the features points do not belong 
to the same 3-D plane, then three points can be used to 
define such a plane and five supplementary points are 
needed to estimate H [34.60]. 

Once H is available, it can be decomposed as 

H = R+— n* T , (34.29) 

d* 

in which R is the rotation matrix relating the orienta¬ 
tion of the current and desired camera frames, n* is the 
normal to the chosen 3-D plane expressed in the de¬ 
sired frame, d* is the distance to the 3-D plane from 
the desired frame, and t is the translation between cur¬ 
rent and desired frames. From H, it is thus possible to 
recover R, t/d*, and n. In fact, two solutions for these 
quantities exist [34.61], but it is quite easy to select the 
correct one using some knowledge about the desired 
pose. It is also possible to estimate the depth of any 
target point up to a common scale factor [34.54]. The 
unknown depth of each point that appears in classical 
IBVS can thus be expressed as a function of a sin¬ 
gle, constant parameter whatever the number of points. 
Similarly, the pose parameters required by PBVS can 
be recovered up to a scalar factor as for the transla¬ 
tion term. The PBVS schemes described previously can 
thus be revisited using this approach, with the new er¬ 
ror defined as the translation up to a scalar factor and 
the angle/axis parameterization of the rotation. This ap¬ 
proach has also been used for the hybrid visual servoing 
schemes described in Sect. 34.4.1. In that case, using 
such homography estimation, it has been possible to an¬ 
alyze the stability of hybrid visual servoing schemes in 
the presence of calibration errors [34.36]. Finally, it is 
also possible to directly use the homography in the con¬ 
trol scheme, avoiding thus its decomposition as a partial 
pose [34.62], 
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34.7 Determining s* and Matching Issues 


All visual servo methods require knowledge of the 
desired feature values s* which implicitly define con¬ 
straints on the desired camera or robot pose with respect 
to the target. Three common approaches are employed. 
The first is when the task is directly specified as a de¬ 
sired value of some features to be reached. This is the 
case for instance when a target has to be centered in 
the image. This is also the case when the task is speci¬ 
fied as a particular pose to reach and a PBVS is chosen. 
In that case however, the camera will reach its desired 
pose only if the camera is perfectly calibrated. Indeed, 
a coarse camera calibration will induce a biased estima¬ 
tion of the pose, which will make the final pose different 
from the desired one. 

For IBVS and hybrid schemes, the second approach 
is to use knowledge of the object and camera projec¬ 
tion models to compute s* for the desired relative pose. 
Once again, the accuracy of the system directly depends 
on the camera calibration, since the camera intrinsic pa¬ 
rameters are involved to compute s*. 

Finally, the third approach is to simply record the 
feature values s* when the camera or robot has the de¬ 
sired pose with respect to the target. This is usually 
done during an off-line teaching step. When it is pos¬ 
sible in practice, this approach is very efficient since 
the positioning accuracy does not depend anymore on 
the camera calibration. This is still true for PBVS since, 
even if the pose estimated from the desired image is bi¬ 
ased due to calibration errors, the same biased pose will 
be estimated once the robot will have converged so that 
the final image acquired by the camera will be the de¬ 
sired one. 


So far we have not commented on the matching is¬ 
sues involved in visual servo methods. Two cases can 
be differentiated in function of the nature of the com¬ 
ponents of s. When some components of s come from 
a pose estimation, it is necessary to match the measure¬ 
ments in the image (usually some image points) to the 
model of the object (usually some world points). Incor¬ 
rect association will lead to an erroneous estimation of 
the pose. In all other cases, the calculation of the er¬ 
ror vector e defined in (34.1) necessitates a matching 
between the measurements m(t) in the current image 
and m* in the desired image. For instance, in all IBVS 
examples presented above, the camera observes four 
image points and we need to determine which of the 
four desired points to associate with each observed 
point so that the visual features s are correctly asso¬ 
ciated to their desired value s*. Incorrect association 
will lead to incorrect final camera pose and possibly 
a configuration which can not be achieved from any real 
camera pose. If we want to use the epipolar geometry or 
estimate a homography matrix, a similar matching pro¬ 
cess is necessary (Sect. 34.6). 

This matching process is a classical computer vi¬ 
sion problem. Note that it may be particularly difficult 
for the very first image, especially when the robot dis¬ 
placement to achieve is large, which generally implies 
large disparities between the initial and desired images. 
Once the association has been correctly performed for 
the very first image, the matching is greatly simplified 
since it transforms to a visual tracking problem where 
the results obtained for the previous image can be used 
as initialization for the current one. 


34.8 Target Tracking 

We now consider the case of a moving target and a con¬ 
stant desired value s* for the features, the generaliza¬ 
tion to varying desired features s*(t) being immediate. 
The time variation of the error is now given by 

de 

e = L e v c + — , (34.30) 

dt 

where the term de/dt expresses the time variation of e 
due to the generally unknown target motion. If the con¬ 
trol law is still designed to try to ensure an exponential 
decoupled decrease of e (that is, once again e = —Ae), 
we now obtain using (34.30) 


where de/dt is an estimation or an approximation of 
de/dt. This term must be introduced into the control law 
to compensate for the target motion. 

Closing the loop, that is, inserting (34.31) 
into (34.30), we obtain 

-4: ""T: de de 

e = -AL c L+e - L C L+ — + — . (34.32) 

dt dt 

Even if L e L+ > 0, the error will converge to zero only 
if the estimation of de/dt is sufficiently accurate so that 


(34.31) 



de 
Tt ’ 


v, 


"LA-L 


(34.33) 
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otherwise tracking errors will be observed. Indeed, by 
just solving the scalar differential equation e = —Xe + 
b, which is a simplification of (34.32), we obtain e{t) = 
e(0) exp(— Xt) + b/X, which converges towards b/X. On 
one hand, setting a high gain X will reduce the tracking 
error, but on the other hand, setting the gain too high 
can make the system unstable. It is thus necessary to 
make b as small as possible. 

Of course, if the system is known to be such that 
de/dt = 0 (that is, the camera observes a motionless 
object, as described in Sect. 34.1), no tracking error 
will appear with the most simple estimation given by 
de/dt = 0. Otherwise, a classical method in automatic 
control to cancel tracking errors consists of compensat¬ 
ing the target motion through an integral term in the 
control law. In this case, wehave 



j 


where fi is the integral gain that has to be tuned. This 
scheme allows the tracking errors to be canceled only if 


the target has a constant velocity. Other methods, based 
on feedforward control, estimate the term de/dt directly 
through the image measurements and the camera veloc¬ 
ity, when it is available. Indeed, from (34.30), we obtain 


where e can, for instance, be obtained as e(t) = [e(t) — 
e(t — At)]/At, At being the duration of the control 
loop. A Kalman filter [34.63] or more-elaborate filter¬ 
ing methods [34.64] can then be used to improve the 
estimated values obtained. If some knowledge about 
the target velocity or the target trajectory is available, 
it can of course be used to smooth or predict the mo¬ 
tion [34.65-67]. For instance, in [34.68], the periodic 
motion of the heart and breathing are compensated for 
an application of visual servoing in medical robotics. 
Finally, other methods have been developed to remove 
the perturbations induced by the target motion as fast 
as possible [34.43], using for instance predictive con¬ 
trollers [34.69]. 


34.9 Eye-in-Hand and Eye-to-Hand 
in the Joint Space 

In the previous sections, we have considered the six 
components of the camera velocity as the input of the 
robot controller. As soon as the robot is not able to real¬ 
ize this motion, for instance, because it has fewer than 
six degrees of freedom, the control scheme must be ex¬ 
pressed in the joint space. In this section, we describe 
how this can be done, and in the process develop a for¬ 
mulation for eye-to-hand systems. 

In the joint space, the system equations for both the 
eye-to-hand and eye-in-hand configurations have the 
same form 

ds 

s = J s «+tr- (34.34) 

at 

Here, J s e R tx " is the feature Jacobian matrix, which 
can be linked to the interaction matrix, and n is the num¬ 
ber of robot joints. 

For an eye-in-hand system (Fig. 34.15a), ds/dt is 
the time variation of s due to a potential object motion, 
and J s is given by 

Jv = L s c X N i{q) , (34.35) 


Systems Controlled 

• C X N is the spatial motion transform matrix (as de¬ 
fined in Chap. 02 and recalled in (34.15)) from the 
vision sensor frame to the end-effector frame. It is 
usually a constant matrix (as long as the vision sen¬ 
sor is rigidly attached to the end-effector). Thanks 
to the robustness of closed-loop control schemes, 
a coarse approximation of this transform matrix is 
sufficient in visual servoing. If needed, an accurate 
estimation is possible through classical hand-eye 
calibration methods [34.70]. 

• J(?) is the robot Jacobian expressed in the end- 
effector frame (as defined in Chap. 02) 

For an eye-to-hand system (Fig. 34.15b), ds/dt is 
now the time variation of s due to a potential vision sen¬ 
sor motion and J s can be expressed as 

J s = -L s C Z W W J( 9 ) , (34.36) 

= —L s C X 0 °J(<7) • (34.37) 

In (34.36), the classical robot Jacobian N J(q) expressed 
in the end-effector frame is used but the spatial motion 
transform matrix C X N from the vision sensor frame to 
the end-effector frame changes all along the robot mo- 


where: 
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the global asymptotic stability is given by 


J C J+ > 0 . 


(34.39) 


Fig.34.15a,b (a) Eye-in-hand system, (b) eye-to-hand sys¬ 
tem: system schematic (top) and opposite image motion 
produced by the same robot motion ( bottom ) 

tion, and it has to be estimated at each iteration of the 
control scheme, usually using pose-estimation methods. 

In (34.37), the robot Jacobian °J(<7) is expressed 
in the robot reference frame, and the spatial motion 
transform matrix c Xq from the vision sensor frame to 
that reference frame is constant as long as the cam¬ 
era does not move. In this case, which is convenient 
in practice, a coarse approximation of c Xq is usually 
sufficient. 

Once the modeling step is finished, it is quite easy to 
follow the procedure that has been used above to design 
a control scheme expressed in the joint space, and to 
determine the sufficient condition to ensure the stability 
of the control scheme. We obtain, considering again e = 
s — s* , and an exponential decoupled decrease of e 

„ "4; "T de 

q = -XJ+e-J+— . (34.38) 

at 

If k = n, considering as in Sect. 34.1 the Lyapunov 
function £ = i||e(f)|| 2 , a sufficient condition to ensure 


If k > n, we obtain similarly to Sect. 34.1 

J+ 3e > 0 (34.40) 

to ensure the local asymptotic stability of the system. 
Note that the actual extrinsic camera parameters appear 

in J e while the estimated ones are used in J+. It is thus 
possible to analyze the robustness of the control scheme 
with respect to the camera extrinsic parameters. It is 
also possible to estimate directly the numerical value 
of J e or J+ using the methods described in Sect. 34.2.8. 
Finally, to remove tracking errors, we have to ensure 

that 


J,J+ 


de 
d1 


de 
~dt ' 


Let us note that, even if the robot has six degrees of 
freedom, it is generally not equivalent to first compute 
v c using (34.5) and then deduce q using the robot in¬ 
verse Jacobian, and to compute directly q using (34.38). 
Indeed, it may occur that the robot Jacobian .J(q) is 
singular while the feature Jacobian J s is not (that may 
occur when k < n). Furthermore, the properties of the 
pseudo-inverse ensure that using (34.5), ||t> c || is mini¬ 
mal while using (34.38), ||^r|| is minimal. As soon as 

J+ / J + (#W C I +. 


the control schemes will be different and will induce 
different robot trajectories. The choice of the state space 
is thus important. 


34.10 Under Actuated Robots 

Many useful robots are under actuated, that is, they 
cannot move instantaneously in all directions because 
of the number or configuration of their actuators 
(Chap. 52). For instance, a quadrotor flying robot is un¬ 
der actuated since it has only four actuators while the 
dimension of its configuration space is six. Many other 
useful robots are subject to non-holonomic constraints 
(Chap. 49). leading to similar motion inability. A car for 
instance has only two degrees of freedom (for velocity 
and steering) while the dimension of its configuration 
space is three (position and orientation in the ground 
plane). 


A consequence of under actuation is that time vary¬ 
ing manoeuvres may be required in order to achieve 
particular goal states. For example, if a quadrotor has 
to move forward, it must first change its attitude, pitch¬ 
ing down, so that a component of its thrust vector is 
able to accelerate the vehicle forward. For a visual servo 
system this can be problematic since the initial attitude 
change will affect the value of the visual features even 
before the vehicle has moved. In fact the attitude change 
increases the error and this is ultimately destabilising. 

A common and expedient solution [34.71] is to 
de-rotate the image, that is, to use information from 
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a non-vision attitude sensor such as an inertial mea¬ 
surement unit (IMU) (Chap. 29) to correct the feature 
coordinates as if they had been viewed by a vir¬ 
tual camera whose optical axis has a constant ori¬ 
entation in space, typically, straight down. As dis¬ 
cussed in Sect. 34.2.8, the spherical projection model 
is well suited for such image transformation, thanks 
to its invariance properties with respect to rotational 
motion. 

For a non-holonomic vehicle, there are several so¬ 
lutions. In the non-general case where the vehicle is 
able to follow a smooth path from its initial to goal 


configuration, a controller based on visually estimated 
relative pose (range, heading angle and lateral offset) 
can be used (Chap. 49), following a PBVS strategy. 
Particular controllers based on the epipolar geometry 
or the trifocal tensor have also been designed [34.72, 
73]. For the more general case, switching control laws 
can be used. If possible in practice, another common 
solution is to add a controlled DOF between the vehicle 
and the camera in order to bypass the non-holonomic 
constraint through redundancy. It is thus possible to 
control the full camera configuration, but not the robot 
one. 



Fig. 34.16 Few applications 
of visual servoing: Gaze 
control for target tracking, 
navigation of a mobile 
robot to follow a wall using 
an omnidirectional vision 
sensor, grasping a ball with 
a humanoid robot, assembly 
of MEMS and film of a dia¬ 
logue within the constraints 
of a script in animation 
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34.11 Applications 

Applications of visual servoing in robotics are numer¬ 
ous. It can be used as soon as a vision sensor is available 
and a task is assigned to a dynamic system to con¬ 
trol its motion. A non exhaustive list of examples are 
(Fig. 34.16): 

• The control of a pan-tilt-zoom camera for target 
tracking 

• Grasping using a robot arm 

• Locomotion and dextrous manipulation with a hu¬ 
manoid robot 

• Micro or nano manipulation of microelectrome¬ 
chanical system (MEMS) or biological cells 

• Accurate positioning of a parallel robot 


34.12 Conclusions 

In this chapter we have only considered velocity con¬ 
trollers, which are convenient for most classical robot 
arms. However, the dynamics of the robot must of 
course be taken into account for high-speed tasks. 
Features related to the image motion [34.75], image 
intensity [34.76], or coming from other vision sensors 
(RGB-D sensors, ultrasonic probes [34.77], etc.) neces¬ 


• Pipe inspection by an underwater autonomous vehi¬ 
cle 

• Autonomous navigation of a mobile robot in indoor 
or outdoor environment 

• Aircraft landing 

• Autonomous satellite rendezvous 

• Biopsy using ultrasound probes or heart motion 
compensation in medical robotics 

• Virtual cinematography in animation. 

Software are available to help any user interested 
in developing visual servoing applications. We rec¬ 
ommend the Robotics, Machine Vision Toolbox for 
Matlab [34.74] and the ViSP C++ library [34.11], 


sitate reconsideration of the modeling issues to select 
adequate visual features. Finally, fusing visual features 
with data coming from other sensors (force sensor, 
proximity sensors, etc.) at the level of the control 
scheme will allow new research topics to be addressed. 
The end of fruitful research in the field of visual servo 
is thus nowhere yet in sight. 
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35. Multisensor Data Fusion 


Hugh Durrant-Whyte, Thomas C. Henderson 


Multisensor data fusion is the process of combining 
observations from a number of different sensors 
to provide a robust and complete description of 
an environment or process of interest. Data fusion 
finds wide application in many areas of robotics 
such as object recognition, environment mapping, 
and localization. 

This chapter has three parts: methods, ar¬ 
chitectures, and applications. Most current data 
fusion methods employ probabilistic descriptions 
of observations and processes and use Bayes' rule 
to combine this information. This chapter sur¬ 
veys the main probabilistic modeling and fusion 
techniques including grid-based models, Kalman 
filtering, and sequential Monte Carlo techniques. 
This chapter also briefly reviews a number of 
nonprobabilistic data fusion methods. Data fu¬ 
sion systems are often complex combinations 
of sensor devices, processing, and fusion algo¬ 
rithms. This chapter provides an overview of key 
principles in data fusion architectures from both 
a hardware and algorithmic viewpoint. The ap¬ 
plications of data fusion are pervasive in robotics 
and underlythe core problem of sensing, estima¬ 
tion, and perception. We highlight two example 
applications that bring out these features. The 
first describes a navigation or self-tracking ap¬ 
plication for an autonomous vehicle. The second 
describes an application in mapping and environ¬ 
ment modeling. 
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The essential algorithmic tools of data fusion 
are reasonably well established. However, the 
development and use of these tools in realistic 
robotics applications is still developing. 


35.1 Multisensor Data Fusion Methods 


The most widely used data fusion methods employed in challenges. In particular, autonomy is most often the 
robotics originate in the fields of statistics, estimation, goal and so results must be presented and interpreted in 
and control. However, the application of these meth- a form from which autonomous decisions can be made, 
ods in robotics has a number of unique features and for recognition or navigation, for example. 
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In this section we review the main data fusion meth¬ 
ods employed in robotics. These are very often based 
on probabilistic methods, which are indeed now con¬ 
sidered the standard approach to data fusion in all 
robotics applications [35.1]. Probabilistic data fusion 
methods are generally based on Bayes’ rule for com¬ 
bining prior and observation information. Practically, 
this may be implemented in a number of ways: through 
the use of the Kalman and extended Kalman filters, 
through sequential Monte Carlo methods, or through 
the use of functional density estimates. Each of these 
is reviewed. There are a number of alternatives to 
probabilistic methods. These include the theory of evi¬ 
dence and interval methods. Such alternative techniques 
are not as widely used as they once were, however 
they have some special features that can be advan¬ 
tageous in specific problems. These, too, are briefly 
reviewed. 


35.1.1 Bayes' Rule 

Bayes’ rule lies at the heart of most data fusion meth¬ 
ods. In general, Bayes’ rule provides a means to make 
inferences about an object or environment of interest 
described by a state x, given an observation z. 

Bayesian Inference 

Bayes’ rule requires that the relationship between x 
and z be encoded as a joint probability or joint prob¬ 
ability distribution P(x,z) for discrete and continuous 
variables, respectively. The chain rule of conditional 
probabilities can be used to expand a joint probability 
in two ways 

P(x,z) = P(x | z)P(z) = P(z | x)P(x). (35.1) 


Rearranging in terms of one of the conditionals, Bayes’ 
rule is obtained 


P(* I 2) 


P(z I x)P(x) 
P(z ) 


(35.2) 


The value of this result lies in the interpretation of the 
probabilities P(x \ z), P(z | x). and P(x). Suppose it is 
necessary to determine the various likelihoods of dif¬ 
ferent values of an unknown state x. There may be prior 
beliefs about what values of x might be expected, en¬ 
coded in the form of relative likelihoods in the prior 
probability P(x). To obtain more information about the 
state x an observation z is made. These observations are 
modeled in the form of a conditional probability P(z | x) 
that describes, for each fixed state x, the probability 
that the observation z will be made, i. e., the probabil¬ 
ity of z given x. The new likelihoods associated with 


the state x are computed from the product of the orig¬ 
inal prior information and the information gained by 
observation. This is encoded in the posterior probabil¬ 
ity P(x | z), which describes the likelihoods associated 
with x given the observation z. In this fusion process, 
the marginal probability P(z) simply serves to normal¬ 
ize the posterior and is not generally computed. The 
marginal P(z ) plays an important role in model valida¬ 
tion or data association as it provides a measure of how 
well the observation is predicted by the prior, because 
P(z) = / P(z | x)P(x) dx. The value of Bayes’ rule is 
that it provides a principled means of combining ob¬ 
served information with prior beliefs about the state of 
the world. 

Sensor Models 

and Multisensor Bayesian Inference 
The conditional probability P(z | x) serves the role of 
a sensor model and can be thought of in two ways. 
First, in building a sensor model, the probability is 
constructed by fixing the value of x = x and then ask¬ 
ing what probability density P(z \ x = x) on z results. 
Conversely, when this sensor model is used and ob¬ 
servations are made, z = z is fixed and a likelihood 
function P(z = z | x) on x is inferred. The likelihood 
function, while not strictly a probability density, models 
the relative likelihood that different values of x gave rise 
to the observed value of z. The product of this likelihood 
with the prior, both defined on x, gives the posterior or 
observation update P(x | z). In a practical implementa¬ 
tion of (35.2), P(z | x) is constructed as a function of 
both variables (or a matrix in discrete form). For each 
fixed value of x, a probability density on z is defined. 
Therefore, as x varies, a family of likelihoods on z is 
created. 

The multisensor form of Bayes’ rule requires con¬ 
ditional independence 

P(z \, • • • , z„ | x) = P(zi | x) ■ • • P(z n | x) 

n 

= I“[ P(Zi | x) , (35.3) 

/= 1 

so that 

n 

P(x\Z' l ) = CP(x)Y[P(.Zi\x), (35.4) 

(=1 

where C is a normalizing constant. Equation 35.4 is 
known as the independent likelihood pool [35.2]. This 
states that the posterior probability on x given all ob¬ 
servations Z", is simply proportional to the product of 
prior probability and individual likelihoods from each 
information source. 
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The recursive form of Bayes’ rule is 


P(x | Z k ) 


P(z k \x)P(x\Z k - 1 ) 

P{z k \Z k ~ l ) 


rule can be used to write (35.6) in terms of a sen¬ 
sor model P(zk \ x k ) and a predicted probability den- 
(35 5 ) sit y P( Xk I Z k ~ 1 , U k ,Xo) based on observations up to 
time k — 1 as 


The advantage of (35.5) is that we need to compute and 
store only the posterior density P(x | Z k ~ l ), which con¬ 
tains a complete summary of all past information. When 
the next piece of information P(zk \ x) arrives, the pre¬ 
vious posterior takes on the role of the current prior and 
the product of the two becomes, when normalized, the 
new posterior. 

Bayesian Filtering 

Filtering is concerned with the sequential process of 
maintaining a probabilistic model for a state which 
evolves over time and which is periodically observed 
by a sensor. Filtering forms the basis for many prob¬ 
lems in tracking and navigation. The general filtering 
problem can be formulated in Bayesian form. This is 
significant because it provides a common representa¬ 
tion for a range of discrete and continuous data fusion 
problems without recourse to specific target or observa¬ 
tion models. 

Define x, as the value of a state of interest at time t. 
This may, for example, describe a feature to be tracked, 
the state of a process being monitored, or the location 
of a platform for which navigation data is required. For 
convenience, and without loss of generality, time is de¬ 
fined at discrete (asynchronous) times 4 = k. At a time 
instant k , the following quantities are defined: 

• Xk'- The state vector to be estimated at time k. 

• up. A control vector, assumed known, and applied 
at time A: — 1 to drive the state from Xk— 1 to xk at 
time k. 

• Zk'- An observation taken of the state Xk at time k. 

In addition, the following sets are also defined: 

• The history of states: 

X k = {*0, *!,■•■ ,Xk} = {X k ~\xk}. 

• The history of control inputs: 

U k = {u u u 2 ,--- ,««.} = {U k ~\uk}. 

• The history of state observations: 

Z k = {zuZ2,--- ,z k } = {Z k -\zk}- 

In probabilistic form, the general data fusion prob¬ 
lem is to find the posterior density 

P(x k \Z k ,U k ,Xo) (35.6) 

for all times k given the recorded observations and 
control inputs up to and including time k together (pos¬ 
sibly) with knowledge of the initial state jcq. Bayes’ 


P(x k \Z k ,U k ,x 0 ) 

_ P(zk I Xk)P(x k \z k ~\u k ,x 0 ) 

P(z k \Z k -\V k ) • (35 ' 7> 

The denominator in (35.7) is independent of the state 
and following (35.4) can be set to some normalizing 
constant C. The sensor model makes use of the condi¬ 
tional independence assumption from (35.3). 

The total probability theorem can be used to rewrite 
the second term in the numerator of (35.7) in terms of 
the state transition model and the joint posterior from 
time step k — 1 as 


{x k \Z k ~ l 

,U k ,x 0 ) 

- I p <«• 

Xk-1 

| Z k ~\U k ,x 0 ) dx k - 

= /'■<* 

l**-l 

,Z k ~\U k ,x 0 ) 

x P {x k - 

1 1 Z k 

~ l ,U k ,x 0 ) d^-i 

-- J P(x k | 

Xk-1 

• u k ) 

X P (x k - 

1 1 z k - 

~ 1 ,U k ~ l ,x 0 ) dx^—i . 


where the last equality implies that the future state de¬ 
pends only on the current state and the control exerted 
at this time. The state transition model is described in 
terms of a probability distribution in the form P{x k \ 
Xk— 1 , Uk)- That is, the state transition may reasonably 
be assumed to be a Markov process in which the next 
state Xk depends only on the immediately proceeding 
state Xk— 1 and the applied control 14 , and is indepen¬ 
dent of both the observations and preceding states. 

Equations (35.7) and (35.8) define a recursive so¬ 
lution to (35.6). Equation (35.8) is the time update or 
prediction step for the full Bayes data fusion algorithm. 
A graphical description of this equation is shown in 
Fig. 35.1. Equation (35.7) is the observation update 
step for the full Bayes data fusion algorithm. A graph¬ 
ical description of this equation is shown in Fig. 35.2. 
The Kalman filter, grid-based methods, and sequential 
Monte Carlo methods, to be described, are specific im¬ 
plementations of these general equations. 

35.1.2 Probabilistic Grids 

Probabilistic grids are conceptually the simplest way of 
implementing Bayesian data fusion methods. They can 
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Fig. 35.1 Time update step for the full Bayes filter. At a time k— 1, knowledge of the state Xk—i is summarized in 
a probability distribution P(xk— 1 ). A vehicle model, in the form of a conditional probability density P(xk | then 

describes the stochastic transition of the vehicle from a state Xk—i at a time k— 1 to a state xk at a time k. Functionally, this 
state transition may be related to an underlying kinematic state model in the form Xk = f(xk—\,Uk). The figure shows 
two typical conditional probability distributions P(xk \ Xk— 1 ) on the state xi given fixed values of xt—i . The product 
of this conditional distribution with the marginal distribution P(xk—i), describing the prior likelihood of values of Xk, 
gives the the joint distribution P(xk,Xk— 1 ) shown as the surface in the figure. The total marginal density P(xk) describes 
knowledge of Xk after state transition has occurred. The marginal density P(xk) is obtained by integrating (projecting) the 
joint distribution P(xk,Xk— 1 ) over all Xk-i- Equivalently, using the total probability theorem, the marginal density can 
be obtained by integrating (summing) all conditional densities P(Xk \ Xk— 1 ) weighted by the prior probability P(xk— 1 ) of 
each Xk— 1 . The process can equally be run in reverse (a retroverse motion model) to obtain P(xk— 1 ) from P(xk) given 
a model P(xk— 1 | Xk) 


be applied both to problems in mapping [35.3,4] and 
tracking [35.5]. 

In mapping applications, the environment of in¬ 
terest is divided into a grid of equally sized spatial 
cells. Each cell is indexed and labeled with a prop¬ 
erty, thus the state Xy may describe a two-dimensional 
world indexed by ij and having the property x. Inter¬ 
est is focused on maintaining a probability distribution 
on possible state values P( Xy) at each grid cell. Typi¬ 
cally, in navigation and mapping problems, the property 
of interest has only two values O and E, occupied and 


empty, respectively, and it is then usual to assume that 
P(X)j = O) = 1 — P(xy = E). However, there is no par¬ 
ticular constraint on the property encoded by the state 
Xij which could have many values (green, red, blue, for 
example) and indeed be continuous (the temperature at 
a cell, for example). 

Once the state has been defined, Bayesian meth¬ 
ods require that a sensor model or likelihood function 
for the sensor be established. In theory, this requires 
specification of a probability distribution P(z \ Xy = Xy) 
mapping each possible grid state Xy to a distribution on 
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Fig. 35.2 Observation update for the full Bayes filter. Prior to observation, an observation model in the form of the 
conditional density P(zk \ Xk) is established. For a fixed value of Xk, equal to xi or X2 for example, a density function 
P(Zk I Xk = xi) or P(zk \ Xk = X2) is defined describing the likelihood of making the observation Zk- Together the density 
P(Zk | Xk) is then a function of both Zk and x*-. This conditional density then defines the observation model. Now, in 
operation, a specific observation Zk = *1 is made and the resulting distribution P(zk = x 1 | Xk) defines a density function 
(now termed the likelihood function) onx*. This density is then multiplied by the prior density P(xJ) and normalized to 
obtain the posterior distribution P(xk \ Zk) describing knowledge in the state after observation 


observations. Practically, however, this is implemented 
simply as another observation grid so that for a specific 
observation z = z ( taken from a specific location), a grid 
of likelihoods on the states Xjj is produced in the form 
P(z = z | Xij) = A(Xij). It is then trivial to apply Bayes’ 
rule to update the property value at each grid cell as 

P + (Xij) = CA(Xjj)P(Xij) , Vi,j , (35.9) 

where C us a normalizing constant obtained by sum¬ 
ming posterior probabilities to one at node ij only. 
Computationally, this is a simple pointwise multipli¬ 
cation of two grids. Some care needs to be taken that 
the two grids appropriately overlap and align with each 
other at the right scale. In some instances it is also valu¬ 
able to encode the fact that spatially adjacent cells will 


influence each other; that is, if we knew the value of 
the property (occupancy, temperature, for example) at 
ij we will have some belief also of the value of this 
property at adjacent nodes i + 1, /, i,j + 1, etc. Differ¬ 
ent sensors and the fusion of different sensor outputs is 
accommodated simply by building appropriate sensor 
models A(Xjj). 

Grids can also be used for tracking and self-tracking 
(localization). The state Xy in this case is the location of 
the entity being tracked. This is a qualitatively different 
definition of state from that used in mapping. The prob¬ 
ability P(Xjj) must now be interpreted as the probability 
that the object being tracked occupies the grid cell ij. In 
the case of mapping, the sum of property probabilities at 
each grid cell is one, whereas in the case of tracking, the 
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sum of location probabilities over the whole grid must 
sum to one. Otherwise, the procedure for updating is 
very similar. An observation grid is constructed which, 
when instantiated with an observation value, provides 
a location likelihood grid 

P(z = z | xij) = A(x t j ) . 

Bayes’ rule is then applied to update the location proba¬ 
bility at each grid cell in the same form as (35.9) except 
that now the normalization constant C is obtained by 
summing posterior probabilities over all ij grid cells. 
This can become computationally expensive, especially 
if the grid has three or more dimensions. One major 
advantage of grid-based tracking is that it is easy to 
incorporate quite complex prior information. For exam¬ 
ple, if it is known that the object being tracked is on 
a road, then the probability location values for all off¬ 
road grid cells can simply be set to zero. 

Grid-based fusion is appropriate to situations where 
the domain size and dimension are modest. In such 
cases, grid-based methods provide straightforward and 
effective fusion algorithms. Grid-based methods can 
be extended in a number of ways; to hierarchical 
(quad-tree) grids, or to irregular (triangular, pentago¬ 
nal) grids. These can help reduce computation in larger 
spaces. Monte Carlo and particle filtering methods 
(Sect. 35.1.4) may be considered as grid-based meth¬ 
ods, where the grid cells themselves are sample of the 
underlying probability density for the state. 

35.1.3 The Kalman Filter 

The Kalman filter is a recursive linear estimator that 
successively calculates an estimate for a continuous val¬ 
ued state, that evolves over time, on the basis of periodic 
observations of the state. The Kalman filter employs an 
explicit statistical model of how the parameter of in¬ 
terest x(t) evolves over time and an explicit statistical 
model of how the observations z(t) that are made are re¬ 
lated to this parameter. The gains employed in a Kalman 
filter are chosen to ensure that, with certain assumptions 
about the observation and process models used, the re¬ 
sulting estimate x(t) minimizes mean-squared error and 
is thus the conditional mean x(t) = E[jc(t) | Z'\. an av¬ 
erage, rather than a most likely value. 

The Kalman filter has a number of features which 
make it ideally suited to dealing with complex multisen¬ 
sor estimation and data fusion problems. In particular, 
the explicit description of process and observations al¬ 
lows a wide variety of different sensor models to be 
incorporated within the basic algorithm. In addition, 
the consistent use of statistical measures of uncertainty 
makes it possible to quantitatively evaluate the role each 


sensor plays in overall system performance. Further, the 
linear recursive nature of the algorithm ensures that its 
application is simple and efficient. For these reasons, 
the Kalman filter has found widespread application in 
many different data fusion problems [35.6-9]. 

In robotics, the Kalman filter is most suited to prob¬ 
lems in tracking, localization, and navigation, and less 
so to problems in mapping. This is because the algo¬ 
rithm works best with well-defined state descriptions 
(positions, velocities, for example), and for states where 
observation and time-propagation models are also well 
understood. 

Observation and Transition Models 
The Kalman filter may be considered a specific instance 
of the recursive Bayesian filter of (35.7) and (35.8) for 
the case where the probability densities on states are 
Gaussian. The starting point for the Kalman Filter algo¬ 
rithm is to define a model for the states to be estimated 
in the standard state-space form 

x(t) = F(r)jt(r) + B (t)u(t) + G (t)v(t) , (35.10) 

where x(t) is the state vector of interest, u(t ) is a known 
control input, v(t) is a random variable describing un¬ 
certainty in the evolution of the state, and F(r), B(t), 
and G(f) are matrices describing the contribution of 
states, controls, and noise to state transition [35.7]. An 
observation (output) model is also defined in standard 
state-space form 

Z(t) = H (t)x(t) + D(/) w(t) , (35.11) 

where z(t) is the observation vector, w{t) is a random 
variable describing uncertainty in the observation, and 
where H(t) and D(/) are matrices describing the contri¬ 
bution of state and noise to the observation. 

These equations define the evolution of a contin¬ 
uous-time system with continuous observations being 
made of the state. However, the Kalman Filter is al¬ 
most always implemented in discrete time 4 = k. It is 
straightforward [35.8] to obtain a discrete-time version 
of (35.10) and (35.11) in the form 

x(k) = F (k)x(k- 1) + B(k)u(k) + G(k)v(k) , 

(35.12) 

z(k) = H(k)x(k) + D(k) w(k) . (35.13) 

A basic assumption in the derivation of the Kalman 
filter is that the random sequences v(k) and w(k) de¬ 
scribing process and observation noise are all Gaussian, 
temporally uncorrelated, and zero-mean 

E[v(/t)] = E[u>(k)] = 0, Vk, 


(35.14) 
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with known covariance 

E[v(i)v T (j)] = SijQ(i) , E[w (i)w T (J)] = Sij R(0 . 

(35.15) 

It is also generally assumed that the process and obser¬ 
vation noises are also uncorrelated 

E[tt (i)w J (/)] = 0 , Vi'j. (B5.16) 

These are equivalent to a Markov property requiring 
observations and successive states to be conditionally 
independent. If the sequences v (k) and w (k) are tem¬ 
porally correlated, a shaping filter can be used to whiten 
the observations, again making the assumptions re¬ 
quired for the Kalman filter valid [35.8]. If the process 
and observation noise sequences are correlated, then 
this correlation can also be accounted for in the Kalman 
filter algorithm [35.10]. If the sequence is not Gaussian, 
but is symmetric with finite moments, then the Kalman 
filter will still produce good estimates. If however, the 
sequence has a distribution which is skewed or other¬ 
wise pathological, results produced by the Kalman filter 
will be misleading and there will be a good case for us¬ 
ing a more-sophisticated Bayesian filter [35.5]. 

Filtering Algorithm 

The Kalman filter algorithm produces estimates that 
minimize mean-squared estimation error conditioned 
on a given observation sequence and so is the condi¬ 
tional mean 

x(i |./) = E[x(0 | z(l), • • • , z(j)] = E[x(0 | Z J ]. 

(35.17) 

The estimate variance is defined as the mean-squared 
error in this estimate 

E(i|7') = E{[jc(0-i0'|7)]^(0-^(t|7')] r |Z 7 } . 

(35.18) 

The estimate of the state at a time k given all informa¬ 
tion up to time k is written x(k \ k). The estimate of the 
state at a time k given only information up to time k— 1 
is called a one-step-ahead prediction (or just a predic¬ 
tion) and is written x{k \ k — 1). 

The Kalman filter algorithm is now stated with¬ 
out proof. Detailed derivations can be found in many 
books on the subject, for example [35.7, 8]. The state is 
assumed to evolve in time according to (35.12). Obser¬ 
vations of this state are made at regular time intervals 
according to (35.13). The noise processes entering the 
system are assumed to obey (35.14-35.16). It is also 


assumed that an estimate x(k— 1 | k— 1) of the state 
x(k — 1) at time k — 1 based on all observations made up 
to and including time k— 1 is available, and that this es¬ 
timate is equal to the conditional mean of the true state 
x(k— 1) conditioned on these observations. The condi¬ 
tional variance P(k— 1 | k— 1) in this estimate is also 
assumed known. The Kalman filter then proceeds re¬ 
cursively in two stages (Fig. 35.3). 

Prediction. A prediction x(k\k— 1) of the state at 
time k and its covariance P(k \ k— 1) is computed ac¬ 
cording to 

x(k | k- 1) = F (k)x(k- 1 | k- 1) + B(k)u(k) , 

(35.19) 

P(k \k-l) = F(k)P(k - 1 | k- l)F T (k) 

+ G(k)Q(,k)G [ (k) . (35.20) 

Update. At time k an observation z(k ) is made and the 
updated estimate x(k \ k) of the state x(k), together with 
the updated estimate covariance P(k \ k), is computed 
from the state prediction and observation according to 

x(k | k) = x(k | k— 1) + W (k)[z(k) 

-H(k)x(k|k-1)] , (35.21) 

P(k | k) =P(k\k- 1) - W(k)S(k)W T (k) , (35.22) 

where the gain matrix W (k) is given by 

W (k)=P(k | k— l)H(k)S~ l (k) , (35.23) 

where 

S (k) = R(k) + H(k)P(k | k- l)H(k) (35.24) 

is the innovation covariance. The difference between 
the observation z{k) and the predicted observation 
H(k)x(k | k— 1) is termed the innovation or resid¬ 
ual v(k) 

v(k)=z(k)-H(k)x(k\k- 1) . (35.25) 

The innovation is an important measure of the devi¬ 
ation between the filter estimates and the observation 
sequence. Indeed, because the true states are not usu¬ 
ally available for comparison with the estimated states, 
the innovation is often the only measure of how well the 
estimator is performing. The innovation is particularly 
important in data association. 

The Extended Kalman Filter 
The extended Kalman filter (EKF) is a form of the 
Kalman filter that can be employed when the state 
model and/or the observation model are nonlinear. The 
EKF is briefly described in this section. 
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True state 


Estimation 
of state 


State covariance 
computation 



Fig. 35.3 Block diagram of the Kalman filter cycle (after Bar-Shalom and Fortmann [35.7]) 


The state models considered by the EKF are de¬ 
scribed in state-space notation by a first-order nonlinear 
vector differential equation or state model of the form 

x{t) =f[x(t), u(t), v(t),t\ , (35.26) 

where/[•••,•,■] is now a general nonlinear mapping of 
state and control input to state transition. The observa¬ 
tion models considered by the EKF are described in 
state-space notation by a nonlinear vector function in 
the form 

z(t) = h[x(t),u(t), w(t), t\ , (35.27) 

where h[-, •, ■, •] is now a general nonlinear mapping of 
state and control input to observations. 

The EKF, like the Kalman filter, is almost always 
implemented in discrete time. By integration and with 
appropriate identification of discrete time states and ob¬ 
servations, the state model is written 

x(k ) = f[x(k— 1), u(k), v(k ;), fc] , (35.28) 

and the observation model as 

z(k) = h[x{k). w(k)] . (35.29) 

Like the Kalman filter, it is assumed that the noises v(k) 
and w(k) are all Gaussian, temporally uncorrelated, 


and zero-mean with known variance as defined in 
(35.14-35.16). The EKF aims to minimize the mean- 
squared error and thereby compute an approxima¬ 
tion to the conditional mean. It is assumed therefore 
that an estimate of the state at time k — 1 is avail¬ 
able which is approximately equal to the conditional 
mean, x(k- 1 | k- 1) « E[x(k- 1) | Z* -1 ]. The EKF 
algorithm will now be stated without proof. Detailed 
derivations may be found in any number of books 
on the subject. The principle stages in the deriva¬ 
tion of the EKF follow directly from those of the 
linear Kalman filter with the additional step that the 
process and observation models are linearized as a Tay¬ 
lor series about the estimate and prediction, respec¬ 
tively. The algorithm has two stages: prediction and 
update. 

Prediction. A prediction x(k\k— 1) of the state at 
time k and its covariance P(k \ k— 1) is computed ac¬ 
cording to 

x(k | k— 1) = f[x(k— 1 | k— 1),m(/c)] , (35.30) 

P(k | k- 1) = yf x (k)P(k- 11 k- 1)V T /,(k) 

+ Yf v (k)Q(k)V T f v (k) . (35.31) 
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Update. At time k an observation z(k) is made and the 
updated estimate x(k \ k) of the state x(k), together with 
the updated estimate covariance P(k \ k) is computed 
from the state prediction and observation according to 

x(k | k) =x(k | k— 1 ) 

+ W (k){z(k)-h[x(k | k- 1)]} , (35.32) 

P(k | k) =P(k\k- 1)- W(k)S(k)W T (k) , (35.33) 

where 

W(£) =P(k\ k-l)W T h x (k)S~ 1 (k) , (35.34) 

and 

S(k) = Yh w (k)R(k)Y T h w (k) 

+ Vh x (k)P(k\ k- 1 )V T h x (k) , (35.35) 

and where the Jacobian \f.(k ) is evaluated at x(k — 
1) =x(k— 1 | k— 1) and Vh.(k) is evaluated atx(k) = 
x(k | k— 1 ). 

A comparison of (35.19-35.24) with (35.30-35.35) 
makes it clear that the EKF algorithm is very similar to 
the linear Kalman filter algorithm, with the substitutions 
F (k) —> Vf x (k) and H(k) —¥■ Vh x {k) being made in the 
equations for the variance and gain propagation. Thus, 
the EKF is, in effect, a linear estimator for a state error 
which is described by a linear equation and which is 
being observed according to a linear equation of the 
form of (35.13). 

The EKF works in much the same way as the linear 
Kalman filter with some notable caveats: 

• The Jacobians \f x (k ) and Vh x (k) are typically not 
constant, being functions of both state and time step. 
This means that unlike the linear filter, the covari¬ 
ances and gain matrix must be computed online as 
estimates and predictions are made available, and 
will not in general tend to constant values. This 
significantly increases the amount of computation 
which must be performed online by the algorithm. 

• As the linearized model is derived by perturbing 
the true state and observation models around a pre¬ 
dicted or nominal trajectory, great care must be 
taken to ensure that these predictions are always 
close enough to the true state that second order 
terms in the linearization are indeed insignificant. 
If the nominal trajectory is too far away from the 
true trajectory then the true covariance will be much 
larger than the estimated covariance and the filter 
will become poorly matched. In extreme cases the 
filter may also become unstable. 

• The EKF employs a linearized model that must be 
computed from an approximate knowledge of the 


state. Unlike the linear algorithm this means that the 
filter must be accurately initialized at the start of op¬ 
eration to ensure that the linearized models obtained 
are valid. If this is not done, the estimates computed 
by the filter will simply be meaningless. 

The Information Filter 

The information filter is mathematically equivalent to 
a Kalman filter. However, rather than generating state 
estimates x(i \ j) and covariances P(i \ j) it uses infor¬ 
mation state variables y(i \ j) and information matri¬ 
ces Y (i | j), which are related to each other through the 
relationships 

y(j U) = P~' I j)x{i |./) , 

Y(i\j) = P~\i\j). (35.36) 

The information filter has the same prediction-update 
structure as the Kalman filter. 

Prediction. Apredictiony(£ | k— 1) of the information 
state at time k and its information matrix Y(k \ k— 1) is 
computed according to (Joseph form [35.8]) 

y(k \k-l)= (1- S2G t )F ~ T y(k~ 1 I k — 1) 

+ Y(fc| k- l)Bw(yt) , (35.37) 

Y(fc|fc-l) = M(fc)-flEtt T , (35.38) 

respectively, where 

M(k) = F _t Y (k- 1 | k- 1)F 1 , 

Z =G T M(fc)G + Q~ 1 , 

and 

9. =M(4)GZ~ 1 . 

It should be noted that E, whose inverse is required to 
compute 9 , has the same dimensionality as the process 
driving noise, which is normally considerably smaller 
than the state dimension. Further, the matrix F ~ 1 is the 
state-transition matrix evaluated backwards in time and 
so must always exist. 

Update. At time k an observation z{k) is made and 
the updated information state estimate y (k \ k) together 
with the updated information matrix Y(k\k) is com¬ 
puted from 

y(k | k) =y(k | k— \ ) + H(k)R-'(k)z(k) , (35.39) 

Y(k | k) = Y(k\k- 1) + H (fc) R —1 (k) H T (k). 

(35.40) 
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We emphasize that (35.38) and (35.37) are math¬ 
ematically identical to (35.19) and (35.20), and 
that (35.39) and (35.40) are mathematically identi¬ 
cal to (35.21) and (35.22). It will be noted that 
there is a duality between information and state space 
forms [35.10]. This duality is evident from the fact 
that ft and E in the prediction stage of the informa¬ 
tion filter play an equivalent role to the gain matrix W 
and innovation covariance S in the update stage of the 
Kalman filter. Furthermore, the simple linear update 
step for the information filter is mirrored in the simple 
linear prediction step for the Kalman filter. 

The main advantage of the information filter over 
the Kalman filter in data fusion problems is the rela¬ 
tive simplicity of the update stage. For a system with n 
sensors, the fused information state update is exactly the 
linear sum of information contributions from all sensors 
as 

n 

y(k | k) = y(k | k— l)+J2^)RT l (k)Zi(k ), 

/= 1 
n 

Y (k | k) = Y(k |£-1) + . 

1= 1 

(35.41) 

The reason such an expression exists in this form is 
that the information filter is essentially a log-likelihood 
expression of Bayes’ rule, where products of likeli¬ 
hoods (35.4) are turned into sums. No such simple ex¬ 
pression for multisensor updates exists for the Kalman 
filter. This property of the information filter has been 
exploited for data fusion in robotic networks [35.11, 
12 ] and more recently in robot navigation and localiza¬ 
tion problems [35.1]. One substantial disadvantage of 
the information filter is the coding of nonlinear models, 
especially for the prediction step. 

When to Use a Kalman or Information Filter 
Kalman or information filters are appropriate to data 
fusion problems where the entity of interest is well 
defined by a continuous parametric state. This would 
include estimation of the position, attitude, and veloc¬ 
ity of a robot or other object, or the tracking of a simple 
geometric feature such as a point, line or curve. Kalman 
and information filters are inappropriate for estimating 
properties such as spatial occupancy, discrete labels, or 
processes whose error characteristics are not easily pa¬ 
rameterized. 

35.1.4 Sequential Monte Carlo Methods 

Monte Carlo (MC) filter methods describe probability 
distributions as a set of weighted samples of an underly¬ 


ing state space. MC filtering then uses these samples to 
simulate probabilistic inference usually through Bayes’ 
rule. Many samples or simulations are performed. By 
studying the statistics of these samples as they progress 
through the inference process, a probabilistic picture of 
the process being simulated can be built up. 

Representing Probability Distributions 
In sequential Monte Carlo methods, probability dis¬ 
tributions are described in terms of a set of support 
points (state-space values) x', i = 1, • • • ,N, together 
with a corresponding set of normalized weights w‘, 
i = 1, • • ■ , N, where w’ = 1. The support points and 
weights can be used to define a probability density func¬ 
tion in the form 

N 

P(x) ^ w'S(x — x') . (35.42) 

;=i 

A key question is how these support points and weights 
are selected to obtain a faithful representation of the 
probability density P(x). The most general way of 
selecting support values is to use an importance den¬ 
sity q{x). The support values x 1 are drawn as samples 
from this density; where the density has high proba¬ 
bility, more support values are chosen, and where the 
density has low probability, few support support vectors 
are selected. The weights in (35.42) are then computed 
from 

w‘ oc . (35.43) 

q(x‘) 

Practically, a sample x' is drawn from the impor¬ 
tance distribution. The sample is then instantiated in 
the underlying probability distribution to yield the 
value P(x = x'). The ratio of the two probability values, 
appropriately normalized, then becomes the weight. 

There are two instructive extremes of the impor¬ 
tance sampling method: 

1. At one extreme, the importance density could be 
taken to be a uniform distribution, so the support 
values x' are uniformly distributed on the state space 
in a close approximation to a grid. The probabilities 
q(x') are also therefore equal. The weights com¬ 
puted from (35.43) are then simply proportional 
to the probabilities w 1 oc P(x = x'). The result is 
a model for the distribution which looks very like 
the regular grid model. 

2. At the other extreme, we could choose an impor¬ 
tance density equal to the probability model q(x) = 
P(x). Samples of the support values x' are now 
drawn from this density. Where the density is high 
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there will be many samples, where the density is 
low there will be few samples. However, if we sub¬ 
stitute q(x') = P(x') into (35.43), it is clear that the 
weights all become equal w' = h. A set of samples 
with equal weights is known as a particle distribu¬ 
tion. 

It is, of course, possible to mix these two representa¬ 
tions to describe a probability distribution both in terms 
of a set of weights and in terms of a set of support 
values. The complete set of samples and weights de¬ 
scribing a probability distribution {*', w I }P =l is termed 
a random measure. 

The Sequential Monte Carlo Method 
Sequential Monte Carlo (SMC) filtering is a simulation 
of the recursive Bayes update equations using sample 
support values and weights to describe the underlying 
probability distributions. 

The starting point is the recursive or sequential 
Bayes observation update given in (35.7) and (35.8). 
The SMC recursion begins with an a posterior proba¬ 
bility density represented by a set of support values and 
weights {x' k _ x , in the form 

N k -1 

P { x k—\ | Z k ~ l ) = J2 W k- 1 S ( X k~\ ~4-l) ' 

i=l 

(35.44) 

The prediction step requires that (35.44) is substituted 
into (35.8) where the joint density is marginalized. 
Practically however, this complex step is avoided by im¬ 
plicitly assuming that the importance density is exactly 
the transition model as 

?*(**) = p (x‘k K-i) • < 35 - 45 ) 

This allows new support values x' k to be drawn on 
the basis of old support values x‘ k _ t while leaving the 
weights unchanged w k = wj c _ 1 . With this, the predic¬ 
tion becomes 

Nt 

P(x k I Z k ~ l ) = w k-I s ( x k ~ x 'k) ■ (35.46) 

i= 1 

The SMC observation update step is relatively straight¬ 
forward. An observation model P{z k \ x k ) is defined. 
This is a function on both variables, z k and x k , and 
is a probability distribution on z k (integrates to unity). 
When an observation or measurement is made, Zk = Zk, 
the observation model becomes a function of state xk 
only. If samples of the state are taken x k = x' k , i = 


1 • ■ • , Nk, the observation model P(z k = Zk I x k = x [) 
becomes a set of scalars describing the likelihood that 
the sample x' k could have given rise to the observation 
Zk- Substituting these likelihoods and (35.46) into (35.7) 
gives 

P(x k \Z k ) 

Nk 

= c'^2 w k-\ p (z* = Zk I Xk = x‘ k ) S (x k - x ‘ k ) . 

;=i 

(35.47) 

This is normally implemented in the form of an updated 
set of normalized weights 

= “ ,; -. f ( a = a '*,-=*;> A , (35.48) 

Eji 1 K-iP (zt = Zk I X k = x? k ) 

and so 

Nk 

P (x k I Z k ) = ^2 w k 8 ( x k-x'k) • (35.49) 

Note that the support values in (35.49) are the same as 
those in (35.46), only the weights have been changed 
by the observation update. 

The implementation of the SMC method requires 
the enumeration of models for both the state transi¬ 
tion P(x k | Xr—i) and the observation P(z k \ x k )• These 
need to be presented in a form that allows instantia¬ 
tion of values for z k , x k , and x k —\. For low-dimensional 
state spaces, interpolation in a lookup table is a viable 
representation. For high-dimensional state spaces, the 
preferred method is to provide a representation in terms 
of a function. 

Practically, (35.46) and (35.49) are implemented as 
follows. 

Time Update. A process model is defined in the usual 
state-space form as x k = f(x k — 1 , w k — 1 , k), where w k is 
an independent noise sequence with known probability 
density P(w k ). The prediction step is now implemented 
as follows: N k samples w' k , i= l,-- - ,N k are drawn 
from the distribution P(w k ). The N k support values 
x‘ k _ | together with the samples w' k are passed through 
the process model as 

x i k =f(x i k _ l ,w[_ v k), (35.50) 

yielding a new set of support vectors x' k . The weights for 
these support vectors w' k _ l are not changed. In effect, 
the process model is simply used to do N k simulations 
of state propagation. 
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Observation Update. The observation model is 
also defined in the usual state-space form as Zk = 
h(xk, Vk, k), where Vk is an independent noise sequence 
with known probability density P{v k ). The observation 
step is now implemented as follows. A measurement 
Zk = Zk is made. For each support value x' k , a likelihood 
is computed as 

A (x' k ) = P (z k =z k \x k = x[) . (35.51) 

Practically, this requires that the observation model be 
in an equational form (such as a Gaussian) which al¬ 
lows computation of the likelihood in the error between 
the measured value Zk and the observations predicted by 
each particle h(x‘ k , k). The updated weights after obser¬ 
vation are just 

wj c (Xwj c _ l P(z k =Z k \x , k ) . (35.52) 

Resampling 

After the weights are updated it is usual to resample 
the measure w' k }^ = j. This focuses the samples in 
on those areas that have most probability density. The 
decision to resample is made on the basis of the effective 
number, N t s of particles in the sample, approximately 
estimated from 


The sampling importance resampling (SIR) algorithm 
resamples at every cycle so that the weights are always 
equal. One of the key problems with resampling is that 
the sample set fixates on a few highly likely samples. 
This problem of fixating on a few highly likely particles 
during resampling is known as sample impoverishment. 
Generally, it is good to resample when A e ff falls to some 
fraction of the actual samples (say 1 /2). 

When to Use Monte Carlo Methods 
Monte Carlo (MC) methods are well suited to problems 
where state-transition models and observation mod¬ 
els are highly nonlinear. This is because sample-based 
methods can represent very general probability densi¬ 
ties. In particular multimodal or multiple hypothesis 
density functions are well handled by Monte Carlo 
techniques. One caveat to note however is that the mod¬ 
els P(Xk | x k — 1 ) and P(zk \ Xk) must be enumerable in all 
cases and must typically be in a simple parametric form. 
MC methods also span the gap between parametric and 
grid-based data fusion methods. 

Monte Carlo methods are inappropriate for prob¬ 
lems where the state space is of high dimension. In 
general the number of samples required to faithfully 


model a given density increases exponentially with 
state-space dimension. The effects of dimensionality 
can be limited by marginalizing out states that can be 
modeled without sampling, a procedure known as Rao- 
Blackwellization. 

35.1.5 Alternatives to Probability 

The representation of uncertainty is so important to the 
problem of information fusion that a number of alter¬ 
native modeling techniques have been proposed to deal 
with perceived limitations in probabilistic methods. 

There are three main perceived limitations of prob¬ 
abilistic modeling techniques: 

1. Complexity: the need to specify a large number of 
probabilities to be able to apply probabilistic rea¬ 
soning methods correctly. 

2. Inconsistency: the difficulties involved in specifying 
a consistent set of beliefs in terms of probability and 
using these to obtain consistent deductions about 
states of interest. 

3. Precision of models: the need to be precise in the 
specification of probabilities for quantities about 
which little is known. 

4. Uncertainty about uncertainty: the difficulty in as¬ 
signing probability in the face of uncertainty, or 
ignorance about the source of information. 

There are three main techniques put forward to ad¬ 
dress these issues: interval calculus, fuzzy logic, and 
the theory of evidence (Dempster-Shafer methods). We 
briefly discuss each of these in turn. 

Interval Calculus 

The representation of uncertainty using an interval to 
bound true parameter values has a number of potential 
advantages over probabilistic techniques. In particular, 
intervals provide a good measure of uncertainty in situ¬ 
ations where there is a lack of probabilistic information, 
but in which sensor and parameter error is known to be 
bounded. In interval techniques, the uncertainty in a pa¬ 
rameter x is simply described by a statement that the 
true value of the state x is known to be bounded from 
below by a, and from above by b; x e [a, b\. It is im¬ 
portant that no other additional probabilistic structure 
is implied; in particular the statement x e [a, b] does not 
necessarily imply that x is equally probable (uniformly 
distributed) over the interval [a, b\. 

There are a number of simple and basic rules for 
the manipulation of interval errors. These are described 
in detail in the book by Moore [35.13] (whose analysis 
was originally aimed at understanding limited-precision 
computer arithmetic). Briefly, with a, b,c,d e M, addi- 
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tion, subtraction, multiplication, and division are de¬ 
fined by the following algebraic relations 

[a, h] + [c, d] = [a + c, b + d\ , 

[a, b\ — [c, d\ = [a — d, b — c] . (35.53) 

[a, b] x [c, d] = [min(ac, ad, be, bd), 

ma x(ac,ad,bc,bd)] , (35.54) 

0 $ [c, d\ . 

(35.55) 

It can be shown that interval addition and multiplication 
are both associative and commutative. Interval arith¬ 
metic admits an obvious metric distance measure 


[n, b] 
[c.r/] 


= [a, b] x 


a member of this set 


JA ^ im(x) 


1 if j t e A 
0 if xtf A 


For example X may be the set of all aircraft. The set JA 
may be the set of all supersonic aircraft. In the fuzzy 
logic literature, this is known as a crisp set. In contrast, 
a fuzzy set is one in which there is a degree of mem¬ 
bership, ranging between 0 and 1. A fuzzy membership 
function / 44 (A) then defines the degree of membership 
of an element x e X to the set JA. For example, if X is 
again the set of all aircraft, JA may be the set of all fast 
aircraft. Then the fuzzy membership function pb^ (x) as¬ 
signs a value between 0 and 1 indicating the degree of 
membership of every aircraft x to this set. Formally 


d([a,b\, [c,d\) = max(|fl-c|, \b-d\) . (35.56) JA ^ pb A f-> [0, 1] . 


Matrix arithmetic using intervals is also possible, but 
substantially more complex, particularly when matrix 
inversion is required. 

Interval calculus methods are sometimes used for 
detection. However, they are not generally used in data 
fusion problems since: 

1. It is difficult to get results that converge to anything 
of value (it is too pessimistic), and 

2. It is hard to encode dependencies between variables 
which are at the core of many data fusion problems. 

Fuzzy Logic 

Fuzzy logic has found widespread popularity as 
a method for representing uncertainty particularly in 
applications such as supervisory control and high-level 
data fusion tasks. It is often claimed that fuzzy logic 
provides an ideal tool for inexact reasoning, particu¬ 
larly in rule-based systems. Certainly, fuzzy logic has 
had some notable success in practical application. 

A great deal has been written about fuzzy sets and 
fuzzy logic (e.g. [35.14] and the discussion in [35.15, 
Chap. 11]). Here we briefly describe the main defini¬ 
tions and operations without any attempt to consider the 
more advanced features of fuzzy-logic methods. 

Consider a universal set consisting of the ele¬ 
ments x; X = {x}. Consider a proper subset JA C X 
such that 

JA = {x | x has some specific property} . 

In conventional logic systems, we can define a mem¬ 
bership function pb A (x) (also called the characteristic 
function which reports if a specific element teX is 


Composition rules for fuzzy sets follow the compo¬ 
sition processes for normal crisp sets, for example, 

JA n S ^ iMnn(x) = min[/t A (jf), fi B (x)] , 

JA U ® ^ IIaub(x) = m&x[ii A (x), p. B (x)] ■ 

The normal properties associated with binary logic now 
hold: commutativity, associativity, idempotence, dis- 
tributivity, De Morgan’s law, and absorption. The only 
exception is that the law of the excluded middle is no 
longer true 

JJ lul/x, J4nM^0. 

Together these definitions and laws provide a system¬ 
atic means of reasoning about inexact values. 

The relationship between fuzzy set theory and prob¬ 
ability is still hotly debated. 

Evidential Reasoning 

Evidential reasoning (often called the Dempster-Shafer 
theory of evidence after the originators of these ideas) 
has seen intermittent success particularly in automated 
reasoning applications. Evidential reasoning is quali¬ 
tatively different from either probabilistic methods or 
fuzzy set theory in the following sense. Consider a uni¬ 
versal set X. In probability theory or fuzzy set theory, 
a belief mass may be placed on any element x- t e X and 
indeed on any subset JA C X. In evidential reasoning, 
belief mass can not only be placed on elements and sets, 
but also sets of sets. Specifically, while the domain of 
probabilistic methods is all possible subsets X, the do¬ 
main of evidential reasoning is the power set 2 X . 

As an example, consider the mutually exclusive 
set X = {occupied, empty}. In probability theory we 
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might assign a probability to each possible event, for 
example, P(occupied) = 0.3, and thus P(empty) = 0.7. 
In evidential reasoning, we construct the set of all sub¬ 
sets 

2 } = {{occupied, empty}, {occupied}, {empty}, 0} , 
and belief mass is assigned to all elements of this set as 

m({occupied, empty}) = 0.5 , 

7 w({occupied}) = 0.3 , 
m({empty}) = 0.2 , 
m(0) = 0.0 , 

(the null set 0 is assigned a belief mass of zero for nor¬ 
malization purposes). The interpretation of this is that 
there is a 30% chance of occupied, a 20% chance of 
empty, and a 50% chance of either occupied or empty. 
In effect, the measure placed on the set containing 
both occupied and empty is a measure of ignorance 
or inability to distinguish between the two alternatives. 
See [35.16] for a more-detailed example of applying the 
evidential method to certainty-grid navigation. 

Evidential reasoning thus provides a method of cap¬ 
turing ignorance or an inability to distinguish between 


alternatives. In probability theory, this would be dealt 
with in a very different manner by assigning an equal 
or uniform probability to each alternative. Yet, stating 
that there is a 50% chance of occupancy is clearly not 
the same as saying that it is unknown whether it will 
occupied or not. The use of the power set as the frame 
of discernment allows a far richer representation of be¬ 
liefs. However, this comes at the cost of a substantial 
increase in complexity. If there are n elements in the 
original set X, then there will be 2" possible subsets on 
which a belief mass will be assigned. For large n, this is 
clearly intractable. Further, when the set is continuous, 
the set of all subsets is not even measurable. 

Evidential reasoning methods provide a means of 
assigning, and combining belief masses on sets. Meth¬ 
ods also exist for obtaining related measures called 
support and plausibility which, in effect, provide upper 
and lower probability bounds in agreement with Demp¬ 
ster’s original formulation of this method. 

Evidential reasoning can play an important role in 
discrete data fusion systems, particularly in areas such 
as attribute fusion and situation assessment, where in¬ 
formation may be unknown or ambiguous. Its use in 
lower-level data fusion problems is challenging as the 
assignment of belief mass to the power set scales expo¬ 
nentially with state cardinality. 


35.2 Multisensor Fusion Architectures 


The multisensor fusion methods described in the pre¬ 
vious section provide the algorithmic means by which 
sensor data and their associated uncertainty models can 
be used to construct either implicit or explicit mod¬ 
els of the environment. However, a multisensor fusion 
system must include many other functional compo¬ 
nents to manage and control the fusion process. The 
organization of these is termed a multisensor fusion ar¬ 
chitecture. 

35.2.1 Architectural Taxonomy 

Multisensor systems architectures can be organized in 
various ways. The military community has developed 
a layout of functional architectures based on the joint 
directors of the laboratories (IDL) model for multisen¬ 
sor systems. This approach views multisensor fusion 
in terms of signal, feature, threat and situation anal¬ 
ysis levels (so-called JDL levels). The assessment of 
such systems is specified in terms of tracking perfor¬ 
mance, survivability, efficiency and bandwidth. Such 
measures are not generally appropriate in robotics ap¬ 
plications and so the JDL model is not discuss this 


further here (see [35.17, 18] for details). Other classifi¬ 
cation schemes distinguish between low- and high-level 
fusion [35.19], or centralized versus decentralized pro¬ 
cessing or data versus variable [35.20]. 

A general architectural framework for multisensor 
robotic systems has been developed and described in 
detail by Makarenko [35.21], and we will base our 
discussion on this approach. A system architecture is 
defined as follows. 

Meta-Architecture 

A set of high-level considerations that strongly char¬ 
acterize the system structure. The selection and or¬ 
ganization of the system elements may be guided by 
aesthetics, efficiency, or other design criteria and goals 
(for example, system and component comprehensibil¬ 
ity, modularity, scalability, portability, interoperability, 
(de)centralization, robustness, fault tolerance). 

Algorithmic Architecture 

A specific set of information fusion and decision¬ 
making methods. These methods address data hetero¬ 
geneity, registration, calibration, consistency, informa- 
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tion content, independence, time interval and scale, and 
relationships between models and uncertainty. 

Conceptual Architecture 

The granularity and functional roles of components 
(specifically, mappings from algorithmic elements to 
functional structures). 

Logical Architecture 

Detailed canonical component types (i. e., object- 
oriented specifications) and interfaces to formalize in¬ 
tercomponent services. Components may be ad hoc 
or regimented, and other concerns include granularity, 
modularity, reuse, verification, data structures, seman¬ 
tics, etc. Communication issues include hierarchical 
versus heterarchical organization, shared memory ver¬ 
sus message passing, information-based characteriza¬ 
tions of subcomponent interactions, pull/push mech¬ 
anisms, subscribe-publish mechanisms, etc. Control 
involves both the control of actuation systems within 
the multisensor fusion system, as well as control of 
information requests and dissemination within the sys¬ 
tem, and any external control decisions and commands. 

Execution Architecture 

Defines mapping of components to execution elements. 
This includes internal or external methods of ensur¬ 
ing correctness of the code (i. e., that the environment 
and sensor models have been correctly transformed 
from mathematical or other formal descriptions into 
computer implementations), and also validation of the 
models (i. e., ensure that the formal descriptions match 
physical reality to the required extent). 

In any closed-loop control system, sensors are used 
to provide the feedback information describing the cur¬ 
rent status of the system and its uncertainties. Building 
a sensor system for a given application is a system 
engineering process that includes the analysis of sys¬ 
tem requirements, a model of the environment, the 
determination of system behavior under different con¬ 
ditions, and the selection of suitable sensors [35.22]. 
The next step in building the sensor system is to assem¬ 
ble the hardware components and develop the necessary 
software modules for data fusion and interpretation. 
Finally, the system is tested, and the performance is 
analyzed. Once the system is built, it is necessary to 
monitor the different components of the system for 
the purpose of testing, debugging, and analysis. The 
system also requires quantitative measures in terms 
of time complexity, space complexity, robustness, and 
efficiency. 

In addition, designing and implementing real-time 
systems are becoming increasingly complex owing to 
many added features such as graphical user interfaces 


(GUIs), visualization capabilities, and the use of many 
sensors of different types. Therefore, many software 
engineering issues such as reusability and the use of 
commercial off-the-shelf (COTS) components [35.23], 
real-time issues [35.24—26], sensor selection [35.27], 
reliability [35.28-30], and embedded testing [35.31] are 
now receiving more attention from system developers. 

Each sensor type has different characteristics 
and functional descriptions. Consequently, some ap¬ 
proaches aim to develop general methods of modeling 
sensor systems in a manner that is independent of the 
physical sensors used. In turn, this enables the per¬ 
formance and robustness of multisensor systems to be 
studied in a general way. There have been many at¬ 
tempts to provide the general model, along with its 
mathematical basis and description. Some of these 
modeling techniques concern error analysis and fault 
tolerance of multisensor systems [35.32-37], Other 
techniques are model based, and require a priori knowl¬ 
edge of the sensed object and its environment [35.38- 
40]. These help fit data to a model, but do not al¬ 
ways provide the means to compare alternatives. Task- 
directed sensing is another approach to devising sensing 
strategies [35.41— 43]. General sensor modeling work 
has had a considerable influence on the evolution of 
multisensor fusion architectures. 

Another approach to modeling sensor systems is to 
define sensori-computational systems associated with 
each sensor to allow design, comparison, transforma¬ 
tion, and reduction of any sensory system [35.44]. In 
this approach, the concept of an information invariant 
is used to define a measure of information complexity. 
This provides a computational theory allowing analysis, 
comparison, and reduction of sensor systems. 

In general terms, multisensor fusion architectures 
may be classified according to the choice along four in¬ 
dependent design dimensions: 

1. Centralized-decentralized, 

2. Local-global interaction of components, 

3. Modular-monolithic, and 

4. Heterarchical-hierarchical. 

The most prevalent combinations are: 

• Centralized, global interaction, and hierarchical, 

• Decentralized, global interaction, and heterarchical, 

• Decentralized, local interaction, and hierarchical, 

• Decentralized, local interaction, and heterarchical. 

In some cases explicit modularity is also desirable. 
Most existing multisensor architectures fit reasonably 
well into one of these categories. These categories make 
no general commitment to the algorithmic architecture. 
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If the algorithmic architecture is the predominant fea¬ 
ture of a system, it will be characterized as part of 
multisensor fusion theory in Sect. 35.1; otherwise, it 
merely differentiates methods within one of the four 
meta-architectures. 

35.2.2 Centralized, Local Interaction, 
and Hierarchical 

Centralized, local interaction and hierarchical archi¬ 
tectures encompass a number of system philosophies. 
Least representationally demanding is the subsumption 
architecture initially proposed by Braitenberg [35.45] 
and popularized by Brooks [35.46]. The subsumption 
multisensor architecture defines behaviors as the basic 
components, and employs a layered set of behaviors 
to embody one program (monolithic). Any behavior 
may utilize the output of other behaviors, and may also 
inhibit other behaviors. The hierarchy is defined by 
the layers, although this is not always clear-cut. The 
major design philosophy is to develop behaviors di¬ 
rectly from perception-action loops without recourse to 
brittle, environment representations. This leads to ro¬ 
bustness in operation, but a lack of composite behavior 
predictability. 

A more sophisticated (representationally) behavior- 
based system is the distributed field robot architecture 
(DFRA) [35.47]. This is a generalization of the sen¬ 
sor fusion effects (SFX) architecture [35.48]. This ap¬ 
proach exploits modularity, and aims to achieve both 
behavior-based and deliberative action, reconfigurabil¬ 
ity and interoperability through the use of Java, Jini, and 
XML, fault tolerance, adaptability, longevity, consistent 
interfaces, and dynamic components. The algorithmic 
architecture is based on fuzzy logic controllers. Experi¬ 
ments have been demonstrated on outdoor mobile robot 
navigation. 

Other similar architectures of this type include per¬ 
ception action networks Lee and Ro [35.49, 50], while 
Draper et al. [35.51] focuses on types of information 
needed to perform tasks (higher-level integration); see 
also [35.52], 

Another approach to this type of sensor fusion is to 
use artificial neural networks. The advantage is that the 
user, at least in principle, does not need to understand 
how sensor modalities relate, nor model the uncertain¬ 
ties, nor in fact determine the structure of the system 
more than to specify the number of layers in the net¬ 
work and the number of nodes per layer. The neural 
network is presented with a set of training examples, 
and must determine through the weights on the neuron 
connections the optimal mapping from inputs to desired 
outputs (classifications, control signals, etc.) [35.53, 
54], 


Various other methods exist; for example, Hager 
and Mintz [35.42,43] defines a task-oriented approach 
to sensor fusion based on Bayesian decision theory and 
develops an object-oriented programming framework. 
Joshi and Sanderson [35.55] describe a 

methodology for addressing model selection and 
multisensor fusion issues using representation size 
(description length) to choose (1) model class and 
number of parameters, (2) model parameter resolu¬ 
tion (3) subset of observed features to model, and 
(4) correspondence to map features to models. 

Their approach is broader than an architecture and 
uses a minimization criterion to synthesize a multi¬ 
sensor fusion system to solve specific two-dimensional 
(2-D) and three-dimensional (3-D) object recognition 
problems. 

35.2.3 Decentralized, Global Interaction, 
and Heterarchical 

The major example of the decentralized, global interac¬ 
tion meta-architecture is the blackboard system. There 
have been many examples of blackboard systems de¬ 
veloped for data fusion applications. For example, the 
SEPIA system of Berge-Cherfaoui and Vachon [35.56] 
uses logical sensors (see below) in the form of modu¬ 
lar agents that post results to a blackboard. The overall 
architectural goals for blackboards include efficient col¬ 
laboration and dynamic configuration. Experiments are 
reported on an indoor robot moving from room to 
room. 

The MESSIE (multi expert system for scene inter¬ 
pretation and evaluation) system [35.57] is a scene in¬ 
terpretation system based on multisensor fusion; it has 
been applied to the interpretation of remotely sensed 
images. A typology of the multisensor fusion concepts 
is presented, and the consequences of modeling prob¬ 
lems for objects, scene, and strategy are derived. The 
proposed multispecialist architecture generalized the 
ideas of their previous work by taking into account 
the knowledge of sensors, the multiple viewing notion 
(shot), and the uncertainty and imprecision of models 
and data modeled with possibility theory. In particular, 
generic models of objects are represented by concepts 
independent of sensors (geometry, materials, and spa¬ 
tial context). Three kinds of specialists are present in 
the architecture: generic specialists (scene and conflict), 
semantic object specialists, and low-level specialists. 
A blackboard structure with a centralized control is 
used. The interpreted scene is implemented as a matrix 
of pointers enabling conflicts to be detected very easily. 
Under the control of the scene specialist, the conflict 
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specialist resolves conflicts using the spatial context 
knowledge of objects. Finally, an interpretation system 
with SAR (synthetic aperture radar)/SPOT sensors is 
described, and an example of a session concerned with 
bridge, urban area, and road detection is shown. 

35.2.4 Decentralized, Local Interaction, 
and Hierarchical 

One of the earliest proposals for this type of architecture 
is the real-time control system (RCS) [35.58]. RCS is 
presented as a cognitive architecture for intelligent con¬ 
trol, but essentially uses multisensor fusion to achieve 
complex control. RCS focuses on task decomposition 
as the fundamental organizing principle. It defines a set 
of nodes, each comprised of a sensor processor, a world 
model, and a behavior generation component. Nodes 
communicate with other nodes, generally in a hierar¬ 
chical manner, although across-layer connections are 
allowed. The system supports a wide variety of algorith¬ 
mic architectures, from reactive behavior to semantic 
networks. Moreover, it maintains signals, images, and 
maps, and allows tight coupling between iconic and 
symbolic representations. The architecture does not 
generally allow dynamic reconfiguration, but maintains 
the static module connectivity structure of the specifica¬ 
tion. RCS has been demonstrated in unmanned ground 
vehicles [35.59]. Other object-oriented approaches have 
been reported [35.34, 60]. 

An early architectural approach which advocated 
strong programming semantics for multisensor systems 
is the logical sensor system (LSS). This approach ex¬ 
ploits functional (or applicative) language theory to 
achieve that. 

The most developed version of LSS is instrumented 
LSS (ILLS) [35.22]. The ILLS approach is based on 
the LSS introduced by Shilcrat and Henderson [35.61], 
The LSS methodology is designed to specify any sen¬ 
sor in a way that hides its physical nature. The main 
goal behind LSS was to develop a coherent and effi¬ 
cient presentation of the information provided by many 
sensors of different types. This representation provides 
a means for recovery from sensor failure, and also facil¬ 
itates reconfiguration of the sensor system when adding 
or replacing sensors [35.62]. 

ILSS is defined as an extension to LSS, and is com¬ 
prised of the following components (Fig. 35.4): 

1. ILS name: uniquely identifies a module 

2. Characteristic output vector (COV): strongly typed 

output structure, with one output vector and zero or 

more input vectors 

3. Commands: input commands to the module, and 

output commands to the other modules 


4. Select function: a selector that detects the failure of 
an alternate and switches to another alternate if pos¬ 
sible 

5. Alternate subnets: alternative ways of producing the 
COV out ; it is these implementations of one or more 
algorithms that carry the main functions of the mod¬ 
ule 

6 . Control command interpreter ( CCI): interpreter of 
the commands to the module 

7. Embedded tests: self-testing routines that increase 
robustness and facilitate debugging 

8 . Monitors: modules that check the validity of the re¬ 
sulting CO Vs 

9. Taps: hooks on the output lines to view different 
COV values. 

These components identify the system behavior 
and provide mechanisms for online monitoring and 
debugging. In addition, they provide handles for mea¬ 
suring the runtime performance of the system. Monitors 
are validity check stations that filter the output and 
alert the user to any undesired results. Each moni¬ 
tor is equipped with a set of rules (or constraints) 
that governs the behavior of the COV under different 
conditions. 

Embedded testing is used for online checking and 
debugging purposes. Weller et al. proposed a sensor¬ 
processing model with the ability to detect measure¬ 
ment errors and to recover from these errors [35.31]. 
This method is based on providing each system module 
with verification tests to verify certain characteristics in 
the measured data, and to verify the internal and output 
data resulting from the sensor-module algorithm. The 
recovery strategy is based on rules that are local to the 
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Fig. 35.4 Instrumented logical sensor module 
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Table 35.1 Canonical components and the roles they play. Multiple S in the same row indicate that some interrole 
relationships are internalized within a component. Frame does not participate in information fusion or decision making 
but is required for localization and other platform-specific tasks (after [35.21]) 
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different sensor modules. ILSS uses a similar approach 
called local embedded testing, in which each module is 
equipped with a set of tests based on the semantic defi¬ 
nition of that module. These tests generate input data to 
check different aspects of the module, then examine the 
output of the module using a set of constraints and rules 
defined by the semantics. These tests can also take input 
from other modules to check the operation of a group 
of modules. Examples are given of a wall-posed esti¬ 
mation system comprised of a Labmate platform with 
a camera and sonars. Many extensions have been pro¬ 
posed for LSS [35.63,64]. 

35.2.5 Decentralized, Local Interaction, 
and Heterarchical 

The best example of this meta-architecture is the active 
sensor network (ASN) framework for distributed data 
fusion developed by Makarenko et al. [35.21,65]. The 
distinguishing features of the various architectures are 
now described. 

Meta-architecture 

The distinguishing features of ASN are its commit¬ 
ment to decentralization, modularity, and strictly lo¬ 
cal interactions (this may be physical or by type). 
Thus, these are communicating processes. Decentral¬ 
ized means that no component is central to operation 
of the system, and the communication is peer to peer. 
Also, there are no central facilities or services (e.g., for 
communication, name and service lookup or timing). 
These features lead to a system that is scalable, fault 
tolerant, and reconfigurable. 

Local interactions mean that the number of commu¬ 
nication links does not change with the network size. 
Moreover, the number of messages should also remain 
constant. This makes the system scalable as well as re- 
configurable. 

Modularity leads to interoperability derived from 
interface protocols, reconfigurability, and fault toler¬ 
ance: failure may be confined to individual modules. 


Algorithmic Architecture 

There are three main algorithmic components: belief fu¬ 
sion, utility fusion, and policy selection. Belief fusion is 
achieved by communicating all beliefs to neighboring 
platforms. A belief is defined as a probability distribu¬ 
tion of the world state space. 

Utility fusion is handled by separating the individ¬ 
ual platform’s partial utility into the team utility of 
belief quality and local utilities of action and commu¬ 
nication. The downside is that the potential coupling 
between individual actions and messages is ignored be¬ 
cause the utilities of action and communication remain 
local. 

The communication and action policies are chosen 
by maximizing expected values. The selected approach 
is to achieve point maximization for one particular state 
and follows the work of Durrant-Whyte et al. [35.11, 
66 ]. 

Conceptual Architecture 

The data types of the system include: 

1. Beliefs : Current world beliefs 

2. Plans'. Future planned world beliefs 

3. Actions'. Future planned actions. 

The definition of component roles leads to a natural 
partition of the system. 

The information fusion task is achieved through the 
definition of four component roles for each data type: 
source, sink, fuser, and distributor. (Note that the data 
type action does not have fuser or distributor component 
roles.) 

Connections between distributors form the back¬ 
bone of the ASN framework, and the information ex¬ 
changed is in the form of their local beliefs. Similar 
considerations are used to determine component roles 
for the decision-making and system configuration tasks. 

Logical Architecture 

A detailed architecture specification is determined 
from the conceptual architecture. It is comprised of 
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six canonical component types as described in Ta¬ 
ble 35.1 [35.21]. 

Makarenko then describes how to combine the com¬ 
ponents and interfaces to realize the use cases of the 
problem domain in ASN. 

Execution Architecture 

The execution architecture traces the mapping of log¬ 
ical components to runtime elements such as pro¬ 
cesses and shared libraries. The deployment view shows 
the mapping of physical components onto the nodes 


35.3 Applications 

Multisensor fusion systems have been applied to a wide 
variety of problems in robotics (see the references 
for this chapter and l<s>MltiiHiI£Uh |<s>JEZEEaEB3i and 
but the two most general areas are 
dynamic system control and environment modeling. Al¬ 
though there is some overlap between these, they may 
generally be characterized as: 

• Dynamic system control : The problem is to use ap¬ 
propriate models and sensors to control the state 
of a dynamic system (e.g., industrial robot, mobile 
robot, autonomous vehicle, surgical robot). Usu¬ 
ally such systems involve real-time feedback con¬ 
trol loops for steering, acceleration, and behavior 
selection. In addition to state estimation, uncer¬ 
tainty models are required. Sensors may include 
force/torque sensors, gyros, global positioning sys¬ 
tem (GPS), position encoders, cameras, range find¬ 
ers, etc.. 

• Environment modeling : The problem is to use ap¬ 
propriate sensors to construct a model of some 
aspect of the physical environment. This may be 
a particular object, e.g., a cup, a physical part, 
a face, or a larger part of the surroundings, e.g., 
the interior of a building, part of a city or an ex¬ 
tended remote or underground area. Typical sensors 
include cameras, radar, 3-D range finders, infrared 
(IR), tactile sensors and touch probes (CMMs), 
etc. The result is usually expressed as geometry 
(points, lines, surfaces), features (holes, sinks, cor¬ 
ners, etc.), or physical properties. Part of the prob¬ 
lem includes the determination of optimal sensor 
placement. 

35.3.1 Dynamic System Control 

The EMS-Vision system [35.67] is an outstanding ex¬ 
emplar of this application domain. The goal is to 


of the physical system. The source code view ex¬ 
plains how the software implementing the system is 
organized. At the architectural level, three items are 
addressed: execution, deployment, and source code 
organization. 

The experimental implementation of the ASN 
framework has proven to be flexible enough to ac¬ 
commodate a variety of system topologies, platform, 
and sensor hardware, and environment representations. 
Several examples are given with a variety of sensors, 
processors, and hardware platforms. 


develop a robust and reliable perceptual system for au¬ 
tonomous vehicles. The development goals, as stated by 
the EMS-Vision team, are: 

• COTS components 

• Wide variety of objects modeled and incorporated 
into behaviors 

• Inertial sensors for ego-state estimation 

• Peripheral/foveal/saccadic vision 

• Knowledge and goal driven behavior 

• State tracking for objects 

• 25 Hz real-time update rate. 

The approach has been in development since the 
1980s. Figure 35.5 shows the first vehicle to drive fully 
autonomously on the German autobahn for 20 km and 
at speeds up to 96 km/h. 

Information from inertial and vision sensors is com¬ 
bined to produce a road scene tree (Fig. 35.6). A four- 


Fig. 35.5 First fully autonomous vehicle on German autobahn 
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Platform base 



Fig. 35.6 EMS-Vision road scene tree 



Fig. 35.7 EMS-Vision guidance system 












































































































Multisensor Data Fusion I 35.3 Applications 887 


□ 



Switch 


S \ 


Image- 
proc. 
PC 1 


L 




Image- 
proc. 
PC 2 


Fast ethernet 


Image- 
proc. 
PC 3 


Video 


SCI-Net 


Switch 


Behavior 

PC 


Aii k 


Link or CAN 


Parallel 


Radar 


VaPM only 



Ethernet 


Gateway 

PC 


Link 


1 

f 


1 r 

Platform- 


Serial 

Vehicle- 

subsystem 



subsystem 




Transputers 


GPS 

Inertial 

sensors 

Convent, sensors 

Actuators 


+ Accel. 

sensors 
+ Rate 
sensors 
+ Angle 
sensors 

+ Odometry 
+ Steering angle 
+ Throttle position 
+ Brake pressure 
+ Motor revolution 

+ Throttle 
+ Brake 
+ Stearing 


Fig. 35.8 EMS-Vision hardware layout (CAN: controller area network) 




Fig. 35.9 Mathematical structure of a decentralized data fusion node 


Part C | 35.3 













































































































Part C | 35.3 


888 Part C 


Sensing and Perception 



Fig.35.10a-i A synopsis of the ANSER II autonomous network and its operation, (a-c) Main system components: (a) air vehicle, 
(b) ground vehicle, (c) human operative, (d-e) The perception process: (d) top three dimensions of features discovered from 
ground-based visual sensor data along with the derived mixture model describing these feature properties, (e) sector of the 
overall picture obtained from fusing air vehicle (UAV), ground vehicle (GV), and human operator (HO) information. Each set 
of ellipses corresponds to a particular feature and the labels represent the identity state with highest probability, (f-i) Sequential 
fusion process for two close landmarks: (f) a tree and a red car, (g) bearing-only visual observations of these landmarks are 
successively fused, (h) to determine location and identity (i). Note the Gaussian mixture model for the bearing measurement 
likelihood 


dimensional (4-D) generic object representation is built 
which includes background knowledge of the object 
(e.g., roads), its behavioral capabilities, object state and 
variances, and shape and aspect parameters. Figure 35.7 
shows the 4-D inertial/vision multisensor guidance sys¬ 
tem, while Fig. 35.8 shows the hardware aspects. 

In summary, the EMS-Vision system is an interest¬ 
ing and powerful demonstration of multisensor fusion 
for dynamic system control. 


35.3.2 ANSER II: Decentralized Data Fusion 

Decentralized data fusion (DDF) methods were initially 
motivated by the insight that the information or canon¬ 
ical form of the conventional Kalman filter data fusion 
algorithm could be implemented by simply adding in¬ 
formation contributions from observations as shown 
in (35.41). As these (vector and matrix) additions are 
commutative, the update or data fusion process can 
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be optimally distributed amongst a network of sen¬ 
sors [35.11, 12,68]. The aim of the ANSER II project 
was to generalize the DDF method to deal with non- 
Gaussian probabilities for observations and states, and 
to incorporate information from a diversity of sources 
including uninhabited air and ground vehicles, terrain 
databases, and human operatives. 

The mathematical structure of a DDF sensor node 
is shown in Fig. 35.9. The sensor is modeled directly 
in the form of a likelihood function. Once instanti¬ 
ated with an observation, the likelihood function is 
input to a local fusion loop which implements a lo¬ 
cal form of the Bayesian time and observation update 
of (35.7) and (35.8). Network nodes accumulate proba¬ 
bilistic information from observation or communication 
and exchange mutual information (information gain) 
with other nodes in the network [35.21]. This mutual 
information is transmitted to and assimilated by other 
nodes in the network in an ad hoc manner. The result is 
that all nodes in the network obtain a single integrated 
posterior-probability-based all-node observations. 

The ANSER II system consists of a pair of au¬ 
tonomous air vehicles equipped with infrared and vi¬ 
sual sensors, a pair of unmanned ground vehicles 
equipped with visual and radar sensors, and additional 
information provided by geometric and hyperspectral 
databases, along with information input by human op¬ 
eratives [35.69]. The likelihood functions for single¬ 


35.4 Conclusions 

Multisensor data fusion has progressed greatly in the 
last few decades; further advances in the field will be 
documented in the robotics and multisensor fusion and 
integration conference and journal literature. Robust ap¬ 
plications are being fielded based on the body of theory 
and experimental knowledge produced by the research 
community. Current directions of interest include: 

1. Large-scale, ubiquitous sensor systems, 

2. Bio-based or biomimetic systems, 

3. Medical in situ applications, 

4. Wireless sensor networks. 

Representative large-scale examples include intelli¬ 
gent vehicle and road systems, as well as instrumented 
contexts such as cities. Biological principles may pro¬ 
vide fundamentally distinct approaches to the exploita¬ 
tion of dense, redundant, correlated, noisy sensors, 
especially when considered as part of a Gibbsian frame¬ 
work for behavioral response to environmental stimuli. 


sensor features are obtained through a semisupervised 
machine learning method [35.70]. The resulting prob¬ 
abilities are modeled in the form of a mixture of 
Gaussians. Each platform then maintains a bank of 
decentralized, non-Gaussian Bayesian filters for the ob¬ 
served features, and transmits this information to all 
other platforms. The net result is that each platform 
maintains a complete map of all features observed by 
all nodes in the network. Multiple observations of the 
same feature, possibly by different platforms, result in 
an increasingly accurate estimate of the feature loca¬ 
tion for all nodes. A corresponding discrete probability 
measure is used for Fig. 35.10 shows a synopsis of the 
operation of the ANSER II system. 

The ANSER II system demonstrates a number of 
general principles in Bayesian data fusion methods, 
specifically the need to model sensors appropriately 
through the likelihood function, and the possibility of 
building very different data fusion architectures from 
the essential Bayesian form. 

35.3.3 Recent Developments 

There have been recent developments in multisensor 
fusion methods: see [35.71-73] for more theoreti¬ 
cal work, vision and biomedical applications [35.74- 
77], tracking and terrain classification [35.78-81], and 
robotics and automation [35.82-87]. 


Another issue here is the development of a theoretical 
understanding of sensor system development, adaptiv¬ 
ity, and learning with respect to the particular context in 
which the system is deployed. 

Further pushing the envelope of both technology 
and theory will permit the introduction of micro- and 
nanosensors into the human body and allow the mon¬ 
itoring and locally adaptive treatment of various ill¬ 
nesses. Finally, a more-complete theoretical framework 
that encompasses system models for wireless sensor 
networks is still required. This should include models 
of the physical phenomena being monitored, as well 
as operational and network issues. Finally, numerical 
analysis of the algorithmic properties of data-driven 
systems with sensor data error sources must be uni¬ 
fied with the analysis of truncation, roundoff, and other 
errors. 

A firm foundation exists upon which to build these 
new theories, systems, and applications. It will be a vi¬ 
brant area of research for years to come. 
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Video-References 


loMliiLiilfcfcl AnnieWay 

available from http://handbookofrobotics.org/view-chapter/35/videodetails/132 
Application of visual odometry for sewer inspection robots 
available from http://handbookofrobotics.org/view-chapter/35/videodetails/638 
Multisensor remote surface inspection 

available from http://handbookofrobotics.org/view-chapter/35/videodetails/639 
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Part D, Manipulation and Interfaces, is separated into 
two subparts; the first half is concerned with manipula¬ 
tion where frameworks of modeling, motion planning, 
and control of grasp and manipulation of an object are 
addressed, and the second half is concerned with in¬ 
terfaces where physical human-robot interactions are 
handled. Humans can achieve grasping and manipula¬ 
tion of an object dexterously through hand-arm coordi¬ 
nation. An optimum control skill for such a redundant 
system is naturally and gradually acquired through ex¬ 
perience in our daily life. Especially, fingers play an 
important role for expressing human dexterity. With¬ 
out dexterous fingers, it is hard for us to handle any 
daily tool, such as a pencil, keyboard, cup, knife, or 
fork. This dexterity is supported with active and pas¬ 
sive compliance as well as the multiple sensory organs 
existing at the fingertip. Such dexterous manipulation 
enables us to clearly differentiate humans from other 
animals. Thus, manipulation is one of the most im¬ 
portant functions for humans. We eventually acquired 
the current shape of finger, the sensory organs, and 
skill for manipulation, through a long history of evo¬ 
lution, over more than six million years. While humans 
and robots are largely different in terms of actuators, 
sensors, and mechanisms, achieving dexterous manipu¬ 
lation like that of a human in a robot is a challenging 
subject in robotics. As we overview current robot tech¬ 
nology, however, we observe that the dexterity of robots 
is still far behind that of humans. With this overview, we 
now provide a brief synopsis of each chapter in the first 
half of Part D. 

Chapter 36, Motion for Manipulation Tasks, dis¬ 
cusses algorithms that generate motion for manipula¬ 
tion tasks at the arm level, especially in an environment, 
by using the configuration space formalism. While in 
previous chapters (6 and 7) the focus was on specific 
algorithmic techniques for robot motion, this chapter 
is focused on a specific application for robot manipu¬ 
lation. The important example of assembly motion is 
discussed through the analysis of contact states and 
compliant motion control. 

Chapter 37, Contact Modeling and Manipulation, 
provides the contact modeling on rigid contact with and 
without friction, and also the modeling on soft con¬ 
tact, such as elastic and viscoelastic contact interfaces. 
Kinematics and mechanics with friction are precisely 
handled under rigid-body contact. The selection ma¬ 
trix H is introduced to understand the force and velocity 
constraints at the contact interface. Pushing manipu¬ 
lation is also addressed by using the concept of the 
friction limit surface. 


Chapter 38, Grasping, discusses based on the clo¬ 
sure property, grasping with many examples, supposing 
multifingered robotic hands. A strong constraint for 
grasping is the unilateral characteristic,where a finger¬ 
tip can push but not pull an object through a contact 
point. The rigid-body model is further extended for 
considering compliance in grasping with robotic hands 
having a low number of degrees of freedom (DOFs). 
Kinematics and closure issues are also addressed under 
this unilateral constraint. 

Chapter 39, Cooperative Manipulators, addresses 
the strategies for controlling both the motion of coop¬ 
erative system and the interaction forces between the 
manipulators and the grasped object when two manip¬ 
ulator arms firmly grasp a common object. It should be 
noted that this chapter allows the bilateral constraint 
where both directional force and moment are permis¬ 
sible. A general cooperative task-space formulation is 
introduced for different classes of multiarm robotic sys¬ 
tems. 

Chapter 40, Mobile Manipulation, focuses on re¬ 
search conducted on an experimental platform that 
combines capabilities in mobility and manipulation. 
Furthermore, it involves the interaction between the 
robots and real-world environment, unstructured envi¬ 
ronments. The main research objective here is to max¬ 
imize task generality of autonomous robotic systems, 
under minimizing the dependence on task-specific, 
hard-coded, or narrowly-relevant information. 

Chapter 41, Active Manipulation for Perception, 
covers perceptual methods in which manipulation is 
an integral part of perception. There are advantages to 
use manipulation rather than vision for perception. For 
example, manipulation can be used to sense in poor- 
visibility conditions, and also to determine properties 
that require physical interaction. This chapter includes 
the methods that have been developed for inference, 
planning, recognition, and modeling in sensing-via- 
manipulation approaches are cover. 

Without dexterity like that of humans, future robots 
will not be able to work instead of humans in en¬ 
vironments where human cannot enter. In this sense, 
the implementation of dexterity into robots is one of 
the highlights of future robot design. Chapters 36- 
41 provide a good hint for enhancing dexterity for 
robots. 

The second half of Part D addresses interfaces 
where humans control a robot or multiple robots 
through direct or indirect contact with robot(s).We now 
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provide a brief synopsis of each chapter in the second 
half of Part D. 

Ch a pte r 42, Haptics, discusses robotics devices that 
allow human operators to experience the sense of touch 
in remote or virtual environment. Two classes of force 
feedback haptic devices are discussed. One is an ad¬ 
mittance device that senses the force applied by the 
operator and constrains the operator’s position to match 
the appropriate deflection of a simulated object or sur¬ 
face; the other is an impedance haptic device that senses 
the position of the operator and then applies a force 
vector to the operator according to the computed behav¬ 
ior of the simulated object or surface. Haptic rendering 
from real time three-dimensional (3-D) image is also 
introduced in this chapter. 

Chapter 43, Telerobotics, starts with a discussion 
on the classification of three different concepts: direct 
control where all slave motions are directly controlled 
by the user via the master interface, shared control 
where task execution is shared between direct con¬ 
trol and local sensory control, and supervisory control 


where the user and slave are connected loosely with 
strong local autonomy. Various control issues such as 
lossy communication with Internet and operation with 
mobile robots, are also addressed. 

Chapter 44, Networked Robots, focuses on the 
framework of computer networks which offer exten¬ 
sive computing, memory, and other resources that can 
dramatically improve performance. The chapter covers 
a wide span from the history of networked robots as it 
evolves from teleoperation to cloud robotics to how to 
build a networked robot. The very recent progress on 
cloud robotics and potential topics for future research 
are included later in the chapter. 

In Haptics (Chap. 42) direct contact between 
human and robot is made, while in both Telerobotics 
(Chap. 43) and Networked Robots (Chap. 44), an 
appropriate distance between the human and robot 
is kept. One of the main issues is how to maintain 
appropriate control performance of the system in the 
presence of humans or a time lag between humans and 
robots. 
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36. Motion for Manipulation Tasks 


This chapter serves as an introduction to Part D 
by giving an overview of motion generation and 
control strategies in the context of robotic ma¬ 
nipulation tasks. Automatic control ranging from 
the abstract, high-level task specification down 
to fine-grained feedback at the task interface are 
considered. Some of the important issues include 
modeling of the interfaces between the robot and 
the environment at the different time scales of 
motion and incorporating sensing and feedback. 
Manipulation planning is introduced as an exten¬ 
sion to the basic motion planning problem, which 
can be modeled as a hybrid system of continu¬ 
ous configuration spaces arising from the act of 
grasping and moving parts in the environment. 
The important example of assembly motion is dis¬ 
cussed through the analysis of contact states and 
compliant motion control. Finally, methods aimed 
at integrating global planning with state feedback 
control are summarized. 
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Manipulation and Interfaces 


36.1 Overview 

Part D of this handbook is concerned with the interfaces 
that connect robots to their environment. We differen¬ 
tiate three such interfaces, depicted in Fig. 36.1. The 
first interface, between the robot and the computer, 
is primarily concerned with the automatic generation 
of motion for performing a task. The second inter¬ 
face is concerned with the physical interaction between 
the robot and the environment. This chapter relates 
both interfaces in the context of manipulation tasks. 
It is focused on automatic specification, planning, and 
execution of motion of the robot manipulator or the 
manipulated object to meet a task goal under the con¬ 
straints of the physical environment. Chapters 37-39, 
40, and 42 address other important issues pertaining to 
the second interface, such as the physical characteris¬ 
tics of contacts between the robot and the environment, 
grasping and active manipulation (i. e., the interaction 
between the robot and the object for manipulation), 
the interaction between cooperative manipulators, and 
the interaction between mobility and manipulation. The 
third interface, described in Chaps. 41, 43, and 44, 
lies between humans and robots. Key issues that need 
to be addressed involve displaying appropriate sensor 
information to humans, or permitting humans to inter¬ 
actively specify the task or motion a robot is supposed 
to perform. 

Manipulation refers to the process of moving or re¬ 
arranging objects in the environment [36. 1]. To perform 
a manipulation task, a robot establishes physical con¬ 
tact with objects in the environment and subsequently 
moves these objects by exerting forces and moments. 
The object to be manipulated can be large or small, 
and may serve various purposes in relation to the ma¬ 
nipulation task (e.g., flipping a switch, opening a door, 
polishing a surface). In the context of automated as¬ 
sembly and industrial manipulation, the object to be 
manipulated is often referred to as a part. The end-effec¬ 
tor generally refers to the link of the manipulator that 
makes contact and applies forces to the part. The end- 
effector is so named as it is often the end link in a se¬ 



Fig. 36.1 Overview of Part D 


rial kinematic chain. However, complex manipulation 
tasks may require the simultaneous exertion of multiple 
forces and moments at different contact points as well 
as the execution of sequences of force applications to 
several objects. 

To understand the challenge of generating the mo¬ 
tion required for the execution of a manipulation task, 
we will consider the classic example of inserting a peg 
into a hole. The mobile manipulator shown in Fig. 36.2 
is commanded to perform the peg insertion manipu¬ 
lation task, whose general problem structure arises in 
a number of contexts such as compliant assembly. For 
simplicity, we will only consider the transfer motion 
(Sect. 36.3) and assume that the robot has already es¬ 
tablished a stable grasp (Chap. 38) and is holding the 
peg. If the clearance between the peg and the hole is 
very small, the peg may not be able to enter the hole 
easily, and often some contact occurs. In order to guide 
the peg successfully into the hole, the robot has to deal 
with the possible contact states between the peg and the 
hole (Sect. 36.4) and select an appropriate sequence of 
control strategies that will result in successful comple¬ 
tion of the task. 

The peg-and-hole example illustrates that success¬ 
ful robotic manipulation involves dealing with models 
of object geometry and contact in the presence of uncer¬ 
tainty. There are various kinds of uncertainty, including 
modeling, actuation, and sensing uncertainty. To plan 
motion strategies in the presence of uncertainty poses 
difficult computational challenges. Historically, motion 
planning for manipulation has been divided into gross 
motion planning and fine motion planning. The former 
involves reasoning about the manipulator motion on the 
macro scale and considers its overall global movement 



Fig. 36.2 A mobile manipulator inserting a peg into a hole 
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strategy, while the latter considers how to deal with un¬ 
certainty to accomplish a task requiring high precision 
robustly. The primary reason for the historical division 
stems from attempts to simplify the various aspects of 
the problem into something computationally tractable. 
Assuming that the uncertainty in actuator position, ob¬ 
ject models, and environment obstacle shapes can be 
bounded, gross motion planning can be formulated pri¬ 
marily in terms of geometry (Sect. 36.3). As the robot or 
the manipulated object makes contact with the environ¬ 
ment, such as in the example of the insertion task, fine 
motion planning techniques are employed to take into 
account contact geometry, forces, friction, and uncer¬ 
tainty effectively (see Mason's book [36.1] and [36.2, 3] 
for an overview and historical perspective on fine mo¬ 
tion strategies for manipulation tasks). 

This peg-and-hole insertion example also illustrates 
that motion in the context of manipulation tasks is 
subject to constraints. These constraints have to be 
maintained to perform the task successfully. The spe¬ 
cific constraints depend on the type of manipulation 
task, but they may include contact constraints, position, 
and force constraints for the point at which the manip¬ 
ulator makes contact with the environment, kinematic 
and dynamic constraints imposed by the mechanism 
and its actuation capabilities, posture constraints that 
specify behavior to be performed in addition to the 
manipulation task, reactive obstacle avoidance in an un- 
predictably changing environment, and global motion 
constraints to ensure that a specific goal location is at¬ 
tained. These motion constraints are imposed by the 
task, by the kinematics of the mechanism performing 
the task, by the actuation capabilities of the mechanism, 
and by the environment. 

To ensure that constraints are satisfied in spite of 
uncertainty, sensory information (see Part C - Sensing 
and Perception) is considered in the context of feed¬ 
back control loops (Chaps. 8 and 47). Depending on the 
type of constraint, this feedback has to be considered at 
various time scales. For example, the exertion of a con¬ 
stant force on an object in the environment requires 
high-frequency feedback at rates up to 1000 Hz. At the 
other end of the time scale, we consider changes to the 
global connectivity of the environment. These changes, 
resulting from opening doors or moving obstacles, for 
example, occur relatively slowly or infrequently and 
feedback about these changes only has to be considered 
a few times per second. Figure 36.3 graphically illus¬ 
trates a task-dependent ordering of motion constraints 
encountered in manipulation tasks and their associated 
feedback requirements. 

This chapter is concerned with algorithms that gen¬ 
erate motion for manipulation tasks. Previous chapters 
already discussed planning algorithms (Chap. 7 and 


control methods (Chap. 8). While in these two former 
chapters the focus was on specific algorithmic tech¬ 
niques for robot motion, the current chapter is focused 
on a specific application, namely robotic manipulation. 
This application dictates motion constraints and their 
feedback requirements that will have to be satisfied dur¬ 
ing the motion for a manipulation task. In contrast, the 
methods discussed in the two aforementioned chapters 
each only address a subset of the motion constraints, as 
indicated in Fig. 36.3. 

This chapter first discusses task-level control in 
Sect. 36.2. Task-level control describes a set of tech¬ 
niques to control a robotic mechanism in terms of 
its contact points with the environment. These contact 
points are controlled to perform motion that accom¬ 
plishes a specific manipulation task. This means that, 
instead of directly controlling the mechanism, task-rel¬ 
evant points on the mechanism, so-called operational 
points, are controlled. This indirect control provides an 
intuitive means to specify desired motion for manipula¬ 
tion tasks. 

Section 36.3 gives an overview of the configura¬ 
tion space formalization for manipulation planning, and 
how it can be cast as a classical motion planning prob¬ 
lem (Chap. 7). Through this formalism, we can gain 
an understanding and intuition regarding how the ge¬ 
ometry and relative positions of the manipulator, the 
manipulated objects, and the obstacles in the workspace 
determine both the geometry and topological structure 
of the manipulation configuration space. As we shall 
see. there are often an infinite number of possible ways 
to accomplish a manipulation task. Thus, the challenge 
in manipulation planning is to develop methods that 
deal effectively with the combinatorics of the search 
over all possible motions. 

Section 36.4 addresses planning motion for assem¬ 
bly tasks, for which the peg-in-hole insertion is a typical 
example. The focus is on motion constrained by con¬ 
tact, called compliant motion. There are two broad 
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Fig. 36.3 Examples of motion constraints encountered in manipu¬ 
lation tasks and their feedback requirements 
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classes of assembly strategies involving compliant mo¬ 
tion; one class is through special mechanism or control, 
called passive compliance, and the other is through ac¬ 
tive reasoning of contact states, called active compliant 
motion. Fine motion planning is discussed within the 
class of active compliant motion. 

Methods from task-level control and manipulation 
planning address aspects of motion in a complemen¬ 
tary fashion. While control methods are able to satisfy 
the feedback requirements of manipulation tasks, they 
often do not account for the overall global progress 


36.2 Task-Level Control 

To perform a manipulation task, a robot has to establish 
contact with the environment. Through contact points 
the robot is able to apply forces and moments to objects. 
By controlling the position and velocity of the contact 
points as well as the forces acting at them, the robot 
causes the desired motion of objects, thereby perform¬ 
ing the manipulation task. The programming of such 
a task is most conveniently accomplished by directly 
specifying positions, velocities, and forces at the con¬ 
tact points, rather than by specifying the joint positions 
and velocities required to achieve them. 

Consider the manipulation task of serving a cup 
of water shown in Fig. 36.2. This task can easily be 
specified by providing a trajectory for the cup’s mo¬ 
tion. To determine a joint space trajectory that achieves 
the same motion, however, would be much more com¬ 
plex. One would have to rely on inverse kinematics 
(Sect. 2.9), which can be computationally challenging. 
More importantly, the task constraints imposed on the 
cup’s motion do not uniquely specify the cup’s trajec¬ 
tory. For example, while the cup is not allowed to tilt, 
it can be delivered to the goal location in any vertical 
orientation. Such task constraints can easily be speci¬ 
fied in terms of the object motion - at the task level - 
but would be very difficult to characterize in terms of 
joint trajectories. Operational space control is there¬ 
fore a natural choice for performing manipulation tasks. 
We will see in Sect. 36.2.3 that the operational space 
framework provides important additional advantages 
over joint space control in the context of redundant 
manipulators. 

The advantages associated with task-level control 
come at the cost of depending on the manipulator Ja¬ 
cobian. For singular configurations of the manipulator 
(Chap. 4), the manipulator Jacobian is not well defined. 
In these configurations, task-level control of a manipu¬ 
lator becomes unstable. Special care has to be applied 
to prevent the manipulator from entering these con¬ 


in accomplishing the manipulation task. Manipulation 
planners, on the other hand, reason about the global 
nature of the task at the high level, but are often com¬ 
putationally too expensive to deal with uncertainty or 
satisfy the high-frequency feedback requirements of 
manipulation tasks. A robust and skillful manipula¬ 
tor must satisfy all feedback requirements while at the 
same time ensuring the accomplishment of the manip¬ 
ulation task. Section 36.5 discusses various techniques 
that offer a unified view of planning and control for ma¬ 
nipulation tasks. 


figurations, or to resort to specialized controllers in 
the proximity of these configurations. Most commonly, 
the manipulator is controlled so as to avoid singular 
configurations. 

36.2.1 Operational Space Control 

Task-level control - also called operational space con¬ 
trol [36.4,5] (Sect. 8.2) - specifies the behavior of 
a robot in terms of operational points rather than joint 
positions and velocities. An operational point is an 
arbitrary point on the robot that is required to per¬ 
form a particular motion or to exert a specified force 
to accomplish a manipulation task. In a serial-chain 
manipulator arm the operational point is most com¬ 
monly chosen to coincide with the end-effector. More 
complex, branching mechanisms, such as humanoid 
robots, but also redundant serial-chain mechanisms, can 
have several operational points. To specify positions, 
velocities, accelerations, forces, and moments at the 
operational point, it is convenient to define a coor¬ 
dinate system with an origin that coincides with the 
operation point. This coordinate frame is called the op¬ 
erational frame. The orientation of the frame should 
be chosen in accordance with the task. For now we 
will ignore the orientation, however, and only consider 
tasks that require positioning of the operational point. 
In Sect. 36.2.2, we will consider more general tasks, 
combining position and force control. 

Let us consider an operational point x on a robot. 
The relationship between the joint velocities q and 
the velocity of the operational point x is given by the 
Jacobian matrix of the mechanism (Sect. 2.8) at the op¬ 
erational point x 

x = i x (q)q ■ (36.i) 

Note that the Jacobian J depends on the placement of 
the operational point x, and on the current configura- 
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tion q of the robot. For simplicity, we will omit this 
dependency from our notation and just write x = iq. 
From (36.1) we can derive an expression for the instan¬ 
taneous torques r acting at the joints when the forces 
and moments described by F is applied at the opera¬ 
tional point x 

T=f(q)F x . (36.2) 

The vector F captures the task to be performed at opera¬ 
tional point x. If the task specifies the complete position 
and orientation of the operational frame, F will be given 
by F = (f x ,fy,f z ,u x ,u y ,u z ), where / and u designate 
forces along and moments about the respective axes. 
(For easy of presentation, we will refer to F as a force 
vector, even if it describes forces and moments. The ex¬ 
ecution of such a task requires a manipulator with at 
least six degrees of freedom. If a task does not spec¬ 
ify one of the components of F, this component can be 
omitted and the corresponding column of the Jacobian 
matrix is dropped (or a manipulator with fewer than six 
degrees of freedom can be used to accomplish the task). 
For example, dropping u, would indicate that the orien¬ 
tation about the z-axis is not specified by the task. The 
motion behavior in the dimensions of the task space that 
have been dropped is unspecified and the robot will float 
in these dimensions [36.4]. In Sect. 36.2.3 we will dis¬ 
cuss how additional behavior can be performed in these 
unspecified dimensions of the task space. 

Similarly to joint space control, we have to account 
for manipulator dynamics to achieve acceptable perfor¬ 
mance in operational space control. Using (36.1), we 
can project the joint space dynamics into operational 
space to obtain 

F x = A(q .. ,x+T(q,q)x + q(q) , (36.3) 

where A is the operational space inertial matrix, T 
captures centrifugal and Coriolis forces in operational 
space, and q compensates for gravity forces. This equa¬ 
tion pertains to a particular operational point. Intuitively 
speaking, the operational space inertia matrix A cap¬ 
tures the resistance of the operational point to accelera¬ 
tion along and about different axes. More details about 
operational space control and its relationship to joint 
space control are provided in Sect. 8.2 (motion control, 
joint space versus operational space control). 

36.2.2 Combined Force and Position Control 

Let us consider the example task of controlling the 
motion of a peg inside a hole. We assume the peg is al¬ 
ready inserted into the hole and rigidly connected to the 
robot’s end-effector. We attach the operational frame to 


a point on the peg so that its z-axis is aligned with the 
desired direction of motion of the peg inside the hole. 
To perform this task, the robot has to control the posi¬ 
tion of the peg along the z-axis of the operational frame 
and it also has to control contact forces between the peg 
and the hole to avoid jamming. Thus, the task of mov¬ 
ing a peg inside a hole requires that position and force 
control be seamlessly integrated. 

To address contact forces within the operational 
space framework, we rewrite (36.3) as 

F x = F c + A (q)F m + T (q, q)x + q(q) , (36.4) 

where F c represents the contact forces acting at the end- 
effector [36.4]. We can replace x with F m because x is 
acting on a dynamically decoupled system, i. e., a sys¬ 
tem that acts like a unit point mass. We can now control 
forces and motion in the operational frame by selecting 
the following control structure 


Fx — Fm~\~ F c , 

(36.5) 

where 


F m = A ( 4 ) SlF' m + T (q, q)x + 1 ](q) , 

(36.6) 

F c = A(q)aF ' c , 

(36.7) 


where S 2 and ft represent complementary task spec¬ 
ification matrices [36.4] that determine along which 
directions the end-effector is position controlled and 
along which it is force controlled. By selecting ft 
appropriately, the combination of position and force 
control can be tailored to the task. In its simplest form, 
ft and ft = 7 — ft are diagonal matrices. The 7-th diag¬ 
onal entry of ft is 1 if the 7-th operational coordinate 
of the end-effector is position controlled and 0 if it is 
force controlled. In this simplest case, positions and 
forces are controlled along axes of the same coordi¬ 
nate frame. The concept of task specification matrix can 
be extended to include differently oriented coordinate 
frames for position and force control [36.4]. 

Once the force F x has been computed as given by 
(36.5), the corresponding joint torques used to control 
the robot are computed based on (36.1). 

36.2.3 Operational Space Control 
of Redundant Mechanisms 

The full expressiveness of task-level control comes to 
bear in the context of redundant manipulators. A manip¬ 
ulator is considered redundant with respect to the task 
it is performing if has more degrees of freedom than 
required by the task. For example, the task of holding 
a cup of water only specifies two degrees of freedom, 
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namely the two rotations about the axes spanning the 
horizontal plane - this task has two degrees of freedom. 
The mobile manipulator shown in Fig. 36.2 has ten de¬ 
grees of freedom, leaving eight redundant degrees of 
freedom with respect to the task. 

The operational space framework for task-level con¬ 
trol of redundant manipulators decomposes the overall 
motion behavior into two components. The first com¬ 
ponent is given by the task, specified in terms of forces 
and moments F tas k acting at an operational point. This 
vectorF is translated into a joint torque based on (36.2): 
t = ,J 1 Tiask- For a redundant manipulator, however, the 
torque vector r is not uniquely specified and we can se¬ 
lect from a set of task-consistent torque vectors. The 
operational space framework performs this selection 
by considering a secondary task, the so-called pos¬ 
ture behavior, making up the second component of the 
overall motion behavior. Posture behavior can be spec¬ 
ified by an arbitrary torque vector r posture . To ensure 
that this additional torque does not affect the task be¬ 
havior (Ftask), the torque T post ure is projected into the 
null space N [36.6] of the task Jacobian J. The null 
space of the Jacobian is the space orthogonal to the one 
spanned by J and will be of rank Nj — k, where Nj is 
the number of degrees of freedom of the manipulator 
and k = rank(J). The torque A T r posture resulting from 
the null space projection is task consistent, i. e., it is 
guaranteed not to affect the behavior of the operational 
point. Since the vector A^fposture may not lie entirely 
within the null space of J, however, the execution of the 
posture behavior cannot be guaranteed. 

The operational space task (J T F tas k) and the posture 
behavior (A T T posture ) are combined to obtain the gen¬ 
eral expression for the torque-level decomposition of 
the overall motion behavior 

r — J F^k . t N Tposture • (36.8) 

The null-space projection N associated with the task Ja¬ 
cobian J can be obtained by 

fV = I — jj , (36.9) 

where I is the identity matrix, and J is the dynamically 
consistent generalized inverse of J, given by 

J = H _1 J T A , (36.10) 

where H is the joint space inertia matrix and A is 
the operational space inertia matrix for the operational 
point for which J is defined. The use of this specific in¬ 
verse to compute the null-space projection results in the 
selection of the task-consistent torque vector that mini¬ 
mizes kinetic energy. 


36.2.4 Combining Mobility 
and Manipulation 

The operational space framework does not distinguish 
between degrees of freedom used for mobility and 
those used for manipulation. The end-effector-centric 
perspective of this approach considers all degrees of 
freedom to be in service to position and move the end- 
effector. An explicit coordination of manipulation and 
mobility for task-level control is not necessary. 

A computationally efficient approach for coordinat¬ 
ing manipulation and mobility was introduced in [36.7). 
A similar, schema-based coordination of manipulation 
and mobility in the presence of dynamic obstacles 
was demonstrated in [36.8]. This approach is based on 
(36.2) and projects the forces that are derived from var¬ 
ious schemas into the configuration space of the robot 
to determine its motion. Another coordination approach 
for manipulation and mobility considers a given end- 
effector path and generates a path of the base that 
maintains manipulability criteria [36.9]. This underly¬ 
ing representation of mobile manipulators can also be 
used to generate obstacle avoidance behavior of the 
base for a given end-effector path [36.10]. 

The coordination of a nonholonomic mobility plat¬ 
form and a cart-pushing task is addressed in [36.11]. 
In [36.12], an analysis of the task space for a mobile 
base with two manipulators is presented. Such a system 
with two manipulator arms can be viewed as a branch¬ 
ing kinematic chain. Section 36.6 discusses extensions 
to the operational space framework to such branching 
mechanisms. 

36.2.5 Combining Multiple Task Behaviors 

The concept of decomposing a redundant robot’s over¬ 
all motion into task and posture behavior can be gener¬ 
alized to an arbitrary number of behaviors [36.13,14], 
Assume a set of n tasks 7), where 7] has higher pri¬ 
ority than Tj if i < j. Every task 7) is associated with 
a force vector F,- and the corresponding joint torque t , . 
These tasks can be performed simultaneously by pro¬ 
jecting the joint torque associated with each task into 
the combined null space of all tasks with higher pri¬ 
ority, just as task and posture behavior were combined 
in the previous section. The null-space projection en¬ 
sures that the execution of task i does not affect the 
execution of tasks with higher priority, given by the 
set prec(i). 

Given a task i, the joint torque consistent with the 
set of higher priority tasks prec(i) can be obtained by 
projecting r, into the combined null space of prec(/), 

T’ilprec(i) = ^ P rec (i) T ‘ ’ (36.11) 
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where the combined null space is computed as 

A^prec(n) — N n —1 N n —2 '' ' N 1 • (36.12) 

We say that T i | prec ( i ) is the torque for task i given the pre¬ 
ceding tasks prec(r'). Applying this torque to the robot 
will not affect the execution of higher-priority tasks. 
The original torque vector t, can be computed based 
on (36.2). 

Given the ability to compute a task torque that is 
consistent with preceding tasks we are able to combine 
an arbitrary number of tasks into a single motion behav¬ 
ior 

T Tl f T2|prec(2) T T 3 ]p rec ( 3 ) T * * * Gi|prec(/i) , 

(36.13) 

where r is the torque used to control the robot. 

Note that Ti is not projected into any null space, 
since it is the task with the highest priority. Task 1 
is therefore performed in the full space defined by 
the robot’s kinematics. If the robot has N degrees 
of freedom, this space has N dimensions. As the 
number of tasks performed on the robot increases, 
the null-space projections (V prec (;) will project the N- 
dimensional torque vectors associated with lower-pri¬ 
ority tasks into a subspace of decreasing dimension. 
Eventually, the null space will be reduced to the triv¬ 
ial case, when all torque vectors are projected onto the 
null vector. This prevents lower-priority tasks from be¬ 
ing executed. The dimensionality of the null space of 
tasks prcc(i) can be computed by counting the nonzero 
singular values [36.6] of (V prec (,), permitting a program¬ 
matic assessment of task feasibility. 

If the execution of a particular task has to be guar¬ 
anteed, it should be associated with task 1 in (36.13). 
Therefore, the highest-priority task is generally used 
for hard constraints that under no circumstances should 
be violated. These constraints may include contact 
constraints, joint limits, and balancing in the case of hu¬ 
manoid robots, for example (Chap. 67). 

The projection of (36.11) maps the torque t, of 
task i into the null space of all preceding tasks. This 
projection also scales the vector, which may signifi¬ 
cantly impair the execution of task i. This effect can 
be reduced by scaling the desired accelerations x at the 
operational point with the task-consistent inertia ma¬ 
trix A i | prec(0 , given by 

-^ilprec(i) = (ji'|prec(;)H J;| pre c(o) ’ (36.14) 

where H is the joint space inertia matrix and 

(36.15) 


is the task Jacobian consistent with all preceding tasks. 
This Jacobian projects operational space forces into 
joint torques that do not cause any acceleration at the 
operational points of tasks in prec(i). The task-consis¬ 
tent inertia matrix A,| prec ( ; ) then captures the inertia 
perceived at the operational point when its motion is 
limited to the subspace consistent with all preceding 
tasks. 

Using the task-consistent inertia matrix, we can ob¬ 
tain the task-consistent operational space dynamics by 
projecting the joint space dynamics into the space de¬ 
fined by the dynamically consistent inverse of J/| pr ec(/) ■ 
Inverting (36.2) and replacing r based on (36.2) we ob¬ 
tain 

jilprecO) (H(9) + C(q. q) + T g (q) = T %rec( ,)) , 

(36.16) 

which yields the task-consistent equivalent to (36.3) 

^j'lprecO') — A-ilprecO)^ ^ilprec(i)^' T ^ilprec(i) > 

(36.17) 

where A,| prec ( i ) is the task-consistent operational space 
inertial matrix, r^p^,-) captures centrifugal and Cori¬ 
olis forces in operational space, and J?;| pr ec(;) compen¬ 
sates for gravity forces. Equation 36.17 can be used 
to control a task at a particular operational point in 
a task-consistent manner, i. e., without causing accel¬ 
eration at the operational points associated with the 
tasks in prec(i). Note that in practice the terms C(q, q) 
and r g (q) can be computed in the joint space; they only 
need to be accounted for once in the case of multiple 
tasks. 

Referring back to Fig. 36.3, we can analyze the ef¬ 
fectiveness of the operational space framework. The 
control of operational points directly addresses force 
and position constraints on motion in manipulation 
tasks. Additional behaviors with lower priority than 
the task can be used to ensure the maintenance of 
kinematic and dynamic constraints, implement pos¬ 
ture behavior, or perform reactive obstacle avoidance 
based on artificial potential fields [36.15]. The con¬ 
trol of all of these aspects of manipulation tasks is 
computationally efficient. As a consequence, the op¬ 
erational space framework is able to maintain these 
motion constraints while satisfying feedback frequency 
requirements. However, the operational space frame¬ 
work does not include the consideration of global 
motion constraints, such as those imposed by the con¬ 
nectivity of the work space. Therefore, the operational 
space framework is susceptible to local minima, simi¬ 
larly to the artificial potential field method. Such local 


Ji|prec(i) — JiA^prec(t) 
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minima can occur for all of the aforementioned mo¬ 
tion constraints and would prevent the robot from 
accomplishing its task. To overcome this problem, 
planning methods have to be applied (Chap. 7 and 
Sect. 36.3). 

The control basis approach [36.16] provides an al¬ 
ternative, compact formalism for combining multiple 
behaviors. Each individual behavior is described by 
a controller, represented by a potential function 0 e 
0, sensors a e E, and effectors r e T. A particular 
controller 0 can be parameterized with sensors and 
effectors. A particular instance of a behavior can be 
written as <p\°. These controllers can be combined us¬ 
ing null-space projections, similar to (36.13). In the 
control basis framework, (36.13) would be written more 
compactly as 

0„ <1 • • • < 03 <1 02 < 0i , (36.18) 


where we abbreviate 0,1^- with 0;. The relation 0; <(pj 
means that 0,- is projected into the null space of 0. We 
say that 0, is executed subject to (pj. If more than two 
behaviors are combined, as shown in (36.18), each con¬ 
troller 0 , is projected into the combined null space of 
all superior tasks. 

The control basis represents a general framework 
for combining controllers and is not limited to the 
notion of task-level control. In addition to the afore¬ 
mentioned formalism for representing combinations of 
controllers, the control basis also encompasses discrete 
structures that define transitions between combinations 
of controllers. Such transitions can be specified by the 
user or learned with techniques from reinforcement 
learning [36.16]. The control basis can compose mul¬ 
tiple task-level controllers to implement manipulation 
tasks that cannot be described in terms of a single task- 
level controller. 


36.3 Manipulation Planning 


In order to appreciate the computational challenges in¬ 
herent to gross motion planning for manipulation, it 
is useful to adopt a mathematical formalism for anal¬ 
ysis. The chapter on planning (Chap. 7) introduced 
the configuration space (C-space), which represents the 
space of allowable transformations of the robot. In gen¬ 
eral, the set of all allowable transformations forms an 
n-dimensional manifold, in which n is the total number 
of degrees of freedom (DOF) of the mechanical sys¬ 
tem. For a robotic manipulator, the configuration space 
is defined by its kinematic structure (Chap. 2). One 
common class of manipulators is serial-chain manipu¬ 
lators, which consist of a series of revolute or prismatic 
joints connected to form a linear kinematic chain. The 
discussion of manipulation planning in this section fo¬ 
cuses on configuration spaces arising from single serial- 
chain manipulators. However, the mathematical for¬ 
mulation generalizes to more complex robots such as 



Fig. 36.4 Simulation of the H6 humanoid robot manipulating an 
object 


humanoids (Fig. 36.4; Chap. 67), whose upper body 
consists of two cooperating manipulator arms attached 
to a movable base (Chap. 39). The model considered 
here greatly simplifies the problems of grasping, stabil¬ 
ity, friction, mechanics, and uncertainties and instead 
focuses on the geometric aspects. In this section, we 
will consider a simple example of a manipulation task 
that involves a classic pick-and-place operation. Sec¬ 
tion 36.4 further considers grasping and contact states 
in the context of assembly operations, and Sect. 36.5 
gives an overview of techniques to unify feedback 
control and planning for more complex manipulation 
tasks. 

36.3.1 Configuration Space Formalism 

The basic motion planning problem is defined as find¬ 
ing a path in the configuration space connecting a start 
and a goal configuration while avoiding static ob¬ 
stacles in the environment (Chap. 7). Manipulation 
planning involves additional complexity because ma¬ 
nipulation tasks fundamentally require contact with 
objects in the environment. When objects are grasped, 
regrasped, and moved, the structure and topology of 
the free configuration space can change drastically. 
However, manipulation planning shares the same basic 
theoretical foundations and computational complexity 
as classical path planning, such as PSPACE-hardness 
(Chap. 7). 

To be more precise, we will utilize a formalization 
for manipulation planning that is primarily due to Alami 
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et al. [36.17], and adapted in LaValle 's book [36.18]. 
Our goal is to gain understanding and intuition re¬ 
garding the search space for manipulation tasks and 
the different situations that can arise. Throughout this 
discussion, bear in mind that the geometry and rel¬ 
ative positions of the manipulator, the manipulated 
objects, and the obstacles in the workspace determine 
both the geometry and topological structure of the 
free configuration space. Within this framework, ma¬ 
nipulation planning can be intuitively placed in the 
context of planning for hybrid systems [36.19]. Hy¬ 
brid systems involve a mixture of continuous vari¬ 
ables and discrete modes, whose transitions are de¬ 
fined by switching functions (for a discussion of hy¬ 
brid systems in a planning context, see Chap. 7 of 
LaValle 's book [36.18]). In the case of manipulation 
planning, the effect of grasping, repositioning, and 
releasing objects in the environment corresponds to 
switching between different continuous configuration 
spaces. 

Let us consider a simple example of a simple pick- 
and-place manipulation task that requires a robot ma¬ 
nipulator SA to reposition a single movable rigid object 
(part) T from its current location in the workspace to 
some desired goal location. Note that accomplishing 
the task only requires that the part T reach the goal 
location, without regard to precisely how the manipu¬ 
lator does so. Because the part cannot move by itself, 
it must either be transported by the robot or be placed 
at rest in some stable intermediate configuration. There 
are some configurations in which the robot is able to 
grasp and move the part to other locations in the en¬ 
vironment. There are also forbidden configurations in 
which the robot or the part collides with obstacles in 
the environment. The solution of a manipulation plan¬ 
ning problem consists of a sequence of subpaths for 
the robot moving alone or moving while grasping the 
part. The primary constraints on the allowable motions 
are: 

1. The robot manipulator must not collide with any 
obstacles in the environment while reaching for, 
grasping, or regrasping the part. 

2. The robot must adequately grasp the part while 
transporting it (for details on grasping and metrics 
for stable grasps see Chap. 38). 

3. The robot manipulator and movable part must not 
collide with any obstacles in the environment while 
the part is being transported. 

4. If the part is not being transported by the robot, it 
must be placed at rest in some stable intermediate 
placement. 

5. The part must end up at the desired goal location in 
the workspace. 


The two modes of operation give rise to the hybrid sys¬ 
tems formulation: either the robot moves alone, or the 
robot moves while grasping the part. In the literature, 
motions of the robot holding the object at a fixed grasp 
are called transfer paths, and motions of the robot while 
the part stays at a stable placement are called transit 
paths [36.17,20,21]. Solving a manipulation planning 
problem involves searching for a connected sequence of 
transit and transfer paths for the manipulator that will 
accomplish the task. For an autonomous robot, the ideal 
is to utilize planning algorithms that will automatically 
produce correct sequences of transfer and transit paths 
separated by grasping and ungrasping operations. 

Admissible Configurations 
Building on the notation introduced in Chap. 7, we 
define a robot manipulator JA with n degrees of free¬ 
dom operating in a Euclidean world or workspace, 
AV = R N , in which N = 2 or N = 3. Let C A represent 
the n-dimensional C-space of JA, and q A be a configura¬ 
tion. C A is called the manipulator configuration space. 
Let T denote a movable part, which is a solid rigid body 
modeled with geometric primitives. We will assume 
that T is allowed to undergo rigid-body transforma¬ 
tions. This gives rise to a part configuration space, 
C p = SE( 2) or C p = SE( 3). Let q p e C p denote a part 
configuration. The area or volume in the workspace 
occupied by the transformed part model is denoted 

by P(q p ). 

We define the combined (robot and part) configura¬ 
tion space C as the Cartesian product 

C = C A xC p , 

in which each configuration q e C is of the form q = 
( q A , q p ). Note that the set of admissible configurations 
for manipulation planning purposes is more restrictive 
than simply Cf ree , the space free of collisions with obsta¬ 
cle regions as defined in Chap. 7. We must incorporate 
the constraints on the allowable motions by remov¬ 
ing inadmissable configurations. Figure 36.5 illustrates 
examples of some of the important subsets of C for ma¬ 
nipulation planning. 

We start by removing all configurations that involve 
collisions with obstacles. Let the subset of configura¬ 
tions in which the manipulator collides with obstacles 
be given by 

We also want to remove configurations for which the 
part collides with obstacles. However, we will allow 
the surface of the part to make contact with obstacle 
surfaces. This happens for example, when the part is 
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Fig. 36.5 Examples of the various configuration space subsets de¬ 
fined for manipulation planning 

resting on a shelf or table, or when a peg-shaped part is 
being inserted into a hole. Therefore, let 

CL = {(/,/) G C|int(P(/)) 00^0} 

denote the open set for which O intersects the interior 
volume int(3 :, (c7 /> )) of the part for any configuration q p . 
If the interior of the part penetrates O, then clearly these 
configurations should be avoided. 

We can now define C \ (C^ bs U C abs ), which is the set 
of all configurations in which both the robot and the part 
do not inappropriately collide with (9. Next consider the 
interaction between S\ and T. The manipulator must be 
allowed to make contact with the surface of the part, but 
penetration is once again not allowed. Therefore, let all 
configurations in which the interior of the part overlaps 
with the robot geometry be defined as 

= {(q A , q p ) G C|A(/) n int(P(/)) / 0} . 

Finally, we are able to define 

c adm = c\(ci s ucluc 0 t), 

which represents the set of configurations remaining af¬ 
ter subtracting away all unwanted configurations. We 
call this the set of admissible configurations. 

Stable and Grasped Configurations 
By definition, all configurations in the set C a( jm are free 
of penetrative collisions between the robot, the part, 
and obstacles in the environment. However, many of 
these configurations define the part as floating in space 
or on the verge of falling. We now consider two im¬ 
portant subsets of C a dm that are used in manipulation 


planning (Fig. 36.5). Let C^ able C C p denote the subset 
of stable part configurations, which are configurations 
at which the part can safely rest without any forces 
being applied by the manipulator. Examples of stable 
configurations include the part resting on a table or in¬ 
serted into a larger assembly of other parts. The criteria 
for inclusion into the set of stable configurations de¬ 
pends on properties such as the part geometry, friction, 
mass distribution, and contacts with the environment. 
Our formalism does not consider these issues directly, 
but assumes that some method for evaluating the sta¬ 
bility of a part configuration is available. Given C^ able , 
we define C sta bie Q C adm to be the corresponding stable 
configurations of the part-robot system 

Cstabie = {(<f - d ) G C a dm 19 G C stab i e } . 

The other important subset of C ad m is the set of all con¬ 
figurations in which the robot is grasping the part and 
is capable of manipulating it according to some defined 
criteria. Let C grasp C C adm denote the set of grasp con¬ 
figurations. For every configuration, (q A ,q p ) G C grasp , 
the manipulator touches the part, which implies that 
jf(? 4 )nf , (/)/0. Just as before, penetration be¬ 
tween the robot and part geometry is not allowed 
because C grasp C C a d m - In general, many configurations 
at which contacts T(q p ) will not necessarily be 

in Cgrasp. The criteria for a configuration to lie in C grasp 
depends on the particular characteristics of the manip¬ 
ulator, the part, and the contact surfaces between them. 
For example, a typical manipulator would not be able 
to pick up a part by making contact at only a sin¬ 
gle point. (For additional information, see Sect. 36.4 
for contact state identification, Chap. 37, and grasp¬ 
ing criteria, such as force closure models in Chaps. 38 
and 19). 

For any robot and part configuration q = (q A ,q p ) G 
C that we consider for manipulation planning we must 
always ensure that either q G C sta bi e or q G C grasp . We 
therefore define Cf ree = C sta bi e U C grasp , to reflect the 
subset of C a dm that is permissible for manipulation plan¬ 
ning. In the context of hybrid systems, this gives rise 
to the two distinct modes for manipulation planning: 
the transit mode and the transfer mode. Recall that in 
the transit mode, the manipulator is not carrying the 
part (i. e., only the robot moves), which requires that 
q G C s t a bie- In the transfer mode, the manipulator car¬ 
ries the part (i. e., both the robot and the part move), 
which requires that q G C grasp . Based on these condi¬ 
tions, the only way the mode can change is if q G 
C s t a bie C C grasp . When the manipulator is in these con¬ 
figurations, it has two available actions: (1) continue 
to grasp and move the part, or (2) release the part in 
its currently stable configuration. In all other configura- 
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tions the mode remains unchanged. For convenience, 
we define Cn-ans = C sta bie FI C grasp to denote the set of 
transition configurations , which are the places in which 
the mode may change and the robot may grasp or re¬ 
lease the part. 

Manipulation Planning Task Definition 
Finally, the basic pick-and-place manipulation planning 
task can now be defined. An initial part configura- 
tion, q{ nix G C£ able , and a goal part configuration, ^ oal G 
^-stable’ t 11 " 6 specified. Recall that this high-level task 
specification is only defined in terms of repositioning 
the part, without regard to how the manipulator actually 
accomplishes the task. Let C; n it c Cf ree be the set of all 
configurations in which the part configuration is 

Cinit = | L/ ' ■ 7 ) G Cfree 19 = Tmit} • 


Figure 36.7 shows a simple redundant robot arm 
originally illustrated in [36.22]. The robot has three rev¬ 
olute joints with parallel axes and joint limits of — tc, n 
radians, providing three degrees of freedom. Since the 
end-effector of the robot can only reach positions in 
a plane, the arm is redundant in this plane and since 
it has only three DOF and limits imposed on the joint 
angles, C can be visualized as a cube with an edge 
length of 2n. Generally, the C-space manifold of a ma¬ 
nipulator with n revolute joints is homeomorphic to an 
n-dimensional hypertorus if no joint limits are present, 
and homeomorphic to an n-dimensional hypercube if 
joint limits are present. 

Recall that obstacles in the workspace are mapped 
to regions in the C-space called C-obstacles (Chap. 7), 
which represent the set of all joint configurations of the 
robot that cause an intersection of the geometry of the 
robot and the obstacle 


We define C al g C free similarly as the set of all config- ^ = , ^ P £ Q ^ n 0 ^ 0} 
urations in which the part conhguration is q g0!i ° os 1 ’ 


Cgoal — IC/' • H IF Cfree I 7 — </goal j ■ 

If the initial and final configurations of the manipula¬ 
tor are specified, then let ^ init = (qf nll , gf nit ) G C in i t and 
<7goai = (<4,ai>9goai) e C g°ai- The objective of the plan¬ 
ner is to compute a path t such that 

T : [0, 1] l-> Cfree . 

^(D) — 7init. tr(l) — 7goal • 

If the initial and final positions of the manipulator are 
unspecified, we implicitly define the objective of the 
planner as computing a path r : [0, 1] i-> Cf ree such that 
r(0) G Cj n i t and r(l) G C goa i. In either case, a solution 
is an alternating sequence of transit paths and transfer 
paths, whose names follow from the mode. In between 
each transfer path, the part is placed in a stable interme¬ 
diate configuration while the manipulator moves alone 
(transit path) in order to regrasp the part. This sequence 
continues until the part is ultimately placed to rest in its 
final goal configuration. This is depicted in Fig. 36.6. 

36.3.2 Example of a Three-DOF 
Planar Manipulator 

For manipulators with three or fewer degrees of free¬ 
dom (DOF), we can construct visualizations of the 
C-space in order to gain an intuition of the structure. 
Throughout this section, we will use the example of 
a three-DOF serial-chain manipulator whose end-effec¬ 
tor operates a planar workspace (R 2 ). 


In general, the structure of C^ bs can be highly com¬ 
plex and involve multiple connected components of C. 
An example visualization of a configuration space 
for the planar three-DOF manipulator corresponding 



Fig. 36.6 Manipulation planning involves searching for 
a sequence of transfer and transit paths in a set of hybrid 
continuous configuration spaces. In this example, transfer 
paths in different C-spaces arise from different ways of 
rigidly grasping a part 


Workspace 




Fig. 36.7 Planar 3 DOF manipulator 
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to a workspace with two differently colored box¬ 
shaped obstacles is given in Fig. 36.8. The C-obstacles 
are illustrated using the same color as the corre¬ 
sponding box-shaped obstacle in the workspace. The 
red, green, and blue coordinate axes correspond to 
the three joint angles ${ 1 , 2 , 3 } of the manipulator, 
respectively. 

Self-collisions induce special C-obstacle regions 
that do not involve environment geometry, but only 
the geometric interference of the links of the robot 
itself. The robot shown in Fig. 36.7 is, by design, 
self-collision-free. Figure 36.9 shows a modified ver¬ 
sion of the manipulator, where self-collision is actu¬ 
ally possible, and a visualization of the corresponding 
C-obstacles. 

C-obstacles can cover large parts of the configu¬ 
ration space and even cause a complete disconnection 
between different components of Cf ree - Figure 36.10 
shows how the addition of only a small post as an 
obstacle can cause a wide range of configurations 
to produce collisions and disconnect regions of Cf rcc : 
the C-obstacle corresponding to the post forms a wall 
of configurations parallel to the $ 2 -$ 3 -plane. This wall 
represents the range of values for $1 for which the 
first link of the manipulator intersects with the post, 
splitting Cf r ee into two disconnected components. Hav¬ 
ing disconnected components Ci, C 2 € Cff ee means that 


Workspace ftpace 



Fig. 36.8 Planar 3 DOF manipulator in a workspace with 
box-shaped obstacles and associated configuration space 
obstacles 



Fig. 36.9 Non-planar 3 DOF manipulator and correspond¬ 
ing self-collision C-obstacle 


there is no collision-free path that starts in Ci and ends 
in C 2 . 

36.3.3 Inverse Kinematics Considerations 

The typical formulation of a path planning problem re¬ 
quires a goal configuration < 7 goa i as input (Chap. 7). For 
the case of manipulation planning, this configuration is 
usually computed from the desired workspace pose of 
the end-effector by an inverse kinematics (IK) solver 
(Sect. 2.7). Apart from special cases, there currently ex¬ 
ist no known analytical methods for solving the inverse 
kinematics of a general redundant mechanism (greater 
than six degrees of freedom). Iterative, numerical tech¬ 
niques (such as Jacobian-based methods or gradient- 
based optimization) are typically used to calculate solu¬ 
tions in this case. Note that the numerical computation 
of inverse kinematics solutions can suffer from poor 
performance or even nonconvergence, depending on the 
quality of the initial guess, the distribution of singular¬ 
ities in the C-space or a combination of these effects 
(Sect. 2.7). 

Most IK solvers are only able to compute one con¬ 
figuration for a given end-effector pose. For a redundant 
manipulator, however, an infinite, continuous range of 
configurations that cause the end-effector to assume the 
desired pose usually exists. In the context of path plan¬ 
ning, this means that the IK solver will select one out of 
an infinite number of configurations as q gm \. Note that 
in some cases, the selected q gm \ might not be collision- 
free, i. e., it is not part of Cf ree ■ Such a configuration can¬ 
not be used as a goal for computing a global reaching 
strategy because no solution exists for the correspond¬ 
ing path planning query in the classical sense (Chap. 7). 

Another possibility is that the selected ^go-d might 
be disconnected from q m \ { . In this case, q g0!l \ is unsuit¬ 
able for planning despite the fact that it is a member 
of Cfree, because it is in a component of Cf ree that is 
disconnected from the component in which q m \ { lies. 
Consider the planar manipulator shown in Fig. 36.7. 



Fig. 36.10 Workspace mapping to a configuration space 
with topologically disconnected components 
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Figure 36.11 shows two possible solutions for a cer¬ 
tain end-effector pose. The corresponding configura¬ 
tions r/ a = (0, jr/2,0) and q b = (n /2, — n /2, tt /2) are 
marked by green spheres in the C-space visualiza¬ 
tion. In this case, the IK solver might select one of 
the two solutions as q gm i. If the same obstacle as 
in Fig. 36.10 is added to the workspace, the two so¬ 
lutions will lie in disconnected components of Cf ree , as 
shown in Fig. 36.12. 

Let ^init = (0,0,0) be the zero configuration, 
marked by the point of intersection of the C-space axes 
in Fig. 36.12. Let the component of Cf ree in which it lies 
be denoted by Ci and the other component by C 2 . Since 
<7b G C 2 and Ci (T C 2 = 0, no solution for the planning 
problem can exist if the IK solver selects q\, as t 7 goa i. 
This means that every planning attempt with this ^ goa i 
will invariably fail. 

Since computing a complete representation of the 
connectivity of C is a problem of similar complexity to 
path planning itself, it is virtually impossible to detect 
a disconnection between < 7 goa i and before the plan¬ 
ning attempt has failed. 

Workspace Gspace 

02 



Fig.36.11a-c Two inverse kinematics solutions for the 
same end-effector pose of a planar 3 DOF arm; (a) solu¬ 
tion 1 , (b) solution 2, (c) solution shown in C space 


Recall that the workspace goal for a classical ma¬ 
nipulation planning problem is defined by a desired 
end-effector pose computed using inverse kinematics. 
Note however, that the purpose of a manipulation task 
may not require moving the end-effector to that par¬ 
ticular location, but rather moving it to any feasible 
location which enables the desired manipulation task to 
be solved. Thus, it becomes important to devise compu¬ 
tational methods and control schemes that are focused 
around the task, and allow some flexibility and freedom 
with regards to the end-effector pose and manipula¬ 
tor configuration. This is one of the motivations and 
advantages of task space or operational space control 
techniques (Sect. 8.2). 

As an example, consider the task of grasping 
a cylindrical part with the planar three-DOF manipula¬ 
tor. Figure 36.11 shows two possible solution configura¬ 
tions yielding the same end-effector pose. The addition 
of obstacles to this workspace may cause one of the 
configurations to become disconnected from the zero 
configuration. These two configurations, however, may 
not be the only valid configurations that allow the ma¬ 
nipulator to grasp the cylindrical part. Figure 36.13a 
shows several other configurations that allow the ma¬ 
nipulator to grasp the part. By interpolating between 
these, an infinite number of different workspace poses 
that provide solutions to the grasping problem is gener¬ 
ated. The inverse kinematics solutions associated with 
these workspace poses are contained in the continuous 
subset C g oai Q C, as shown in Fig. 36.13b, which repre¬ 
sents the actual solution space to the planning problem 
for this reaching task. 

Adding the obstacle from Fig. 36.10 to the 
workspace of this manipulation task produces the con¬ 
figuration space depicted in Fig. 36.14. Note that the 
C-obstacle corresponding to collisions between the ma¬ 
nipulator and the cylinder itself has been disregarded in 
this visualization for clarity. Approximately half of the 


Workspace C space 



Fig.36.12a-c Two inverse kinematics solutions lying in 
disconnected components of Cfr ee due to a small obsta¬ 
cle in the workspace; (a) solution 1 , (b) solution 2, (c) the 
two solutions shown in C space, which are in disconnected 
components 


a) Workspace example solutions b) G space solution region 



Fig. 36.13 (a) Multiple arm configurations that allow the 
manipulator to grasp the cylinder and (b) C-space visual¬ 
ization of the set of solution configurations 
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possible solutions are disconnected from the zero con¬ 
figuration and a quarter are in collision, which leaves 
only one quarter as reachable and therefore suitable 
candidates for q gm i in a classical planning query. 

36.3.4 Continuum Manipulation 

As introduced in Chap. 20, a manipulator with far more 
degrees of freedom than that of an object to be manipu¬ 
lated is a hyperredundant manipulator. Hyperredundant 
manipulators can be further classified into vertebrate¬ 
like rigid-link manipulators, such as snakes, and inver- 
tebrate-like continuum manipulators, such as octopus 
arms or elephant trunks. Many vertebrate-like manip¬ 
ulators inherit the kinematics of articulated arms, and 
like the articulated arms, they are often used as carriers 



Fig. 36.14 Two views of the set of goal configurations 
(green) for grasping a cylindrical part in a disconnected 
C-space 





Fig. 36.15 Different configurations of the OctArm manipulator 


of the actual manipulators: hands or grippers, such as in 
surgical applications. 

Continuum manipulators [36.23], on the other hand, 
consist of deformable sections and are often used for 
whole-arm manipulation without a divide between the 
arm and hand/gripper, taking advantage of their com¬ 
pliance and flexibility. They can be viewed as the 
extreme case of hyperredundancy but are fundamentally 
different from articulated arms. A grasping configura¬ 
tion for a continuum manipulator affects the whole arm 
as it wraps around an object. The smooth and com¬ 
pliant nature of a continuum manipulator allows it to 
gently interact with the object by adapting its shape to 
the object it wraps. The OctArm [36.24], with multi¬ 
ple constant-curvature sections, is representative of the 
general class of continuum manipulators. 

Although each section i of a continuum manipulator 
is deformable, and thus has infinite degrees of freedom 
theoretically, it often has three controllable degrees of 
freedom, which describes the configuration of the sec¬ 
tion as a truncated torus when undeformed or a cylinder 
if the curvature is zero, length curvature /c,-, and the 
orientation angle </>,- between section i — 1 and section i. 
By changing those three variables for each section, the 
configuration of the whole manipulator can be changed. 
Figure 36.15 shows different configurations of a three- 
section OctArm. 

Planning for continuum, whole-arm manipulation 
is significantly different from manipulation and grasp¬ 
ing planning for conventional articulated arms with 
hands/grippers. Here the inverse kinematics problem is 
to find end positions of each section for the manipulator, 
given the configuration variables of the section [36.25]. 
However, planning motion for whole-arm manipulation 
has to consider directly a mixture of configuration vari¬ 
ables and Cartesian-space variables (i. e., the end posi¬ 
tion of each section), because the shapes and grasping 
capabilities of the end-effector (which can be viewed as 
the portion of the continuum manipulator contacting the 
target object) depend on both. 

Given a target object, its location and size decide 
if there exists a configuration that a continuum ma¬ 
nipulator can wrap around the object to grasp it. If 
wrapping around the object along one circle is suf¬ 
ficient, that bounding circle of the object centered at 
position c with radius r c determines the k (k < n) sec¬ 
tions that are needed to wrap it, starting from the tip /z-th 
section backward. The poses of the remaining n — k sec¬ 
tions must satisfy the intersection constraints to form 
feasible whole-arm configurations for grasping [36.26]. 
A feasible grasping configuration can also be deter¬ 
mined heuristically as an integral part of collision-free 
motion planning [36.27]. Multiple grasping solutions 
provide flexibility to avoid obstacles 
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A more practical approach [36.28] is to first make 
a contact between the manipulator and the target object 
at the m-th section ( m < n), and then generate wrapping 
configurations for the m to n sections of the manipula¬ 
tor one by one without overlap to form a tight, spiral 
wrap (i. e., along multiple object bounding circles). In 
a cluttered environment, progressive wrapping of an ob¬ 
ject can be achieved through moving all arm sections 
together led by the tip of the arm to surroun. the ob¬ 
ject [36.29], which requires minimum free space for 
the arm to manuever. Those approaches can generate 
a better force-closure grasp and facilitate sensing-based 
manipulation based on contact detection. 

Finally, strategies for planning continuum manipu¬ 
lation could also benefit whole-arm manipulation with 

36.4 Assembly Motion 

Assembly or an assembly task defines the process of 
putting together manufactured parts to make a complete 
product, such as a machine. It is a major operation in the 
manufacturing process of any product. Assembly au¬ 
tomation with robots aims to reduce cost and increase 
the quality and efficiency of the operation. In environ¬ 
ments hazardous to humans, such as in space, having 
robots perform assembly tasks could save human lives. 
Assembly has long been not only an important but also 
one of the most challenging applications for robotics. 
There are many significant research issues related to the 
broad scope of assembly automation, from design for 
assembly [36.31] to tolerance analysis [36.32], assem¬ 
bly sequence planning [36.33], fixture design [36.34], 
etc. This section is only focused on the issue of robotic 
motion for assembly. 

The concerned assembly motion is that of a robot 
manipulator holding a part and moving it to reach 
a certain assembled state, i. e., a required spatial ar¬ 
rangement or contact against another part. The main 
difficulty of assembly motion is due to the requirement 
for high precision or low tolerance between the parts in 
an assembled state. As a result, the assembly motion has 
to overcome uncertainty to be successful. Compliant 
motion is defined as motion constrained by the contact 
between the held part and another part in the environ¬ 
ment. As it reduces uncertainty through reducing the 
degrees of freedom of the held part, compliant motion 
is desirable in assembly. 

Consider the peg-in-hole insertion example intro¬ 
duced earlier (Fig. 36.2). If the clearance between the 
peg and the hole is very small, the effect of uncertainty 
will most likely cause a downward insertion motion 
of the peg to fail, i. e., the peg ends up colliding with 
the entrance of the hole in some way without reach- 


vertebrate-like hyperredundant manipulators in that the 
planned spatial curves for poses of a continuum manip¬ 
ulator could be used as desired shapes for a hyperredun¬ 
dant manipulator, such as a snake robot, to fit its joint 
configurations. Indeed, curve design and fitting for hy¬ 
perredundant manipulators has long been introduce, for 
planning manipulator poses [36.30]. 

In this section, we have covered the basics of 
manipulation planning, developed a mathematical for¬ 
mulation for the kinds of search spaces that arise, and 
discussed some of the important geometric and topo¬ 
logical considerations, as well as different types of 
manipulators and manipulation. Section 36.6 provides 
an overview of additional topics and extensions with 
pointers to further reading. 


ing the desired assembled state. Therefore, a successful 
assembly motion has to move the peg out of such an 
unintended contact situation and lead it to reach the 
desired assembled state eventually. To make this tran¬ 
sition, compliant motion is preferred. Often a sequence 
of contact transitions via compliant motion is neces¬ 
sary before the desired assembled state can be reached. 
Figure 36.16 shows a typical sequence of contact tran¬ 
sitions for the peg-in-hole task. 

Assembly motion strategies that incorporate com¬ 
pliant motion can be broadly classified into two groups: 
passive compliance and active compliant motion, and 
both groups of strategies require certain informa- 
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Fig. 36.16 A sequence of contact transitions for the peg- 
in-hole insertion task 
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tion characterizing topological contact states between 
parts. 

36.4.1 Topological Contact States 

When a part A contacts a part B, the configuration of 
A (or B) is a contact configuration. Often a set of con¬ 
tact configurations share the same high-level contact 
characteristics. For example, a cup is on the table is 
a high-level description shared by all the contact con¬ 
figurations of the cup when its bottom are on the table. 
Such a description is often what really matters in as¬ 
sembly motion as it characterizes a spatial arrangement 
that could be either an assembled state or just a contact 
state between a part and another part. 

For contacting polyhedral objects, it is common to 
describe a contact state topologically as a set of prim¬ 
itive contacts, each of which is defined by a pair of 
contacting surface elements in terms of faces, edges, 
and vertices. Different contact state representations 
essentially differ only in how primitive contacts are 
defined. One common representation [36.35] defines 
primitive contacts as point contacts in terms of vertex- 
edge contacts for two-dimensional (2-D) polygons, and 
vertex-face and edge-edge contacts for three-dimen¬ 
sional (3-D) polyhedra. Another representation [36.36] 
defines a primitive contact as between a pair of topolog¬ 
ical surface elements (i. e., faces, edges, and vertices). 
Flere a contact primitive can characterize a contact 
region that is either a point, a line segment, or a pla¬ 
nar face, unlike the point-contact notion. From the 
viewpoint of contact identification via sensing, how¬ 
ever, both representations can result in states that are 
different by definition but indistinguishable in identifi¬ 
cation due to uncertainties. Figure 36.17 shows such an 
example. 

The notion of a principal contact [36.37,38] 
presents a high-level primitive contact that is more ro¬ 
bust to recognition. A principal contact (PC) denotes 
a contact between a pair of topological surface elements 
that are not boundary elements of other contacting topo¬ 
logical surface elements. The boundary elements of 
a face are the edges and vertices bounding it, and the 
boundary elements of an edge are the vertices bound¬ 
ing it. Different PCs between two objects correspond 
to different degrees of freedom of the objects, which 
often also correspond to significant differences in the 
contact forces and moments. As shown in Fig. 36.17, 
the indistinguishable states in terms of the other contact 
primitives are grouped as a single contact state in terms 
of PCs. Thus, there are also fewer contact states in terms 
of PCs, leading to a more concise characterization of 
contact states. In fact, every contact state between two 
convex polyhedral objects is described by a single PC. 


36.4.2 Passive Compliance 

Passive compliance refers to strategies that incorpo¬ 
rate compliant motion for error correction during the 
assembly motion without requiring active and explicit 
recognition and reasoning of contact states between 
parts. 

Remote Center Compliance 
In the 1970s, a remote center compliance (RCC) de¬ 
vice was developed to assist high-precision peg-in-hole 
insertion [36.39— 41], The RCC is a mechanical spring 
structure used as a tool attached to the end-effector 
of a robot manipulator to hold a round peg when it 
is inserted into a round hole. The RCC is designed 
to have high stiffness along the direction of insertion 
but high lateral and angular compliances K x and Kq, 
and it projects the center of compliance near the tip 
of the peg (hence the name remote center compli¬ 
ance) to overcome small lateral and angular errors in 
the peg’s position and orientation in response to the 
contact forces applied to the peg by the hole during 
insertion (Fig. 36.18). The large lateral and angular 
compliance of the RCC also helps to avoid wedging and 
jamming, conditions that cause the peg to stick in two- 
point contacts due to contact forces in direct opposition 
(wedging) or ill-proportioned contact forces/moments 
(jamming). Because the RCC is low cost and reliable 
for fast insertions, it has long seen successful industrial 
applications. 

However, the RCC has limitations. It only applies 
to round peg-in-hole insertions, and a particular RCC 
works better for a peg and hole of a particular size. At¬ 
tempts have been made to expand the applicability of 
the RCC. Some attempts try to make certain param- 



Fig.36.17a-c Indistinguishable contact states due to po¬ 
sitional uncertainty of the top part; (a) two objects, 
(b) state 1 , (c) state 2 
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eters of an RCC adjustable to achieve variable com¬ 
pliance or projection (i. e., the position of the remote 
center) [36.42^14]. A spatial RCC (SRCC) [36.45] 
was proposed to achieve square-peg-in-hole insertions. 
A notable difference with this extension is the combi¬ 
natorial explosion of the number of possible contact 
states, given the range of the position and orientation 
uncertainty of the held part that have to be considered 
in the design of the device. Unlike the case of the round 
peg and hole, which is essentially a 2-D problem with 
a handful of different contact states, there are hundreds 
of possible different contact states between a square peg 
and a square hole. From each of these possible contact 
states, a motion leading the held part to the goal assem¬ 
bled state has to be realized by the device. This makes it 
difficult to design an RCC device that works for parts of 
more general and complex shapes with more possible 
contact states between them. This is a major limiting 
factor that prevent more extensions of the RCC to as¬ 
sembly of different parts. 

Admittance Matrix 

As an alternative method to the RCC, a particular form 
of manipulator force control, damping control, was pro¬ 
posed to achieve compliant motion of the held part 
by the manipulator and to correct small location er¬ 
rors of the held part during assembly. This approach 
eliminates the need to build a mechanical device like 
an RCC to achieve error correction. Among the force 
control laws [36.46], damping control is a common 
strategy, where a commanded velocity of the held part 
is modified based on the sensed force caused by con¬ 
tact between the held part and the environment. The 
resulted actual velocity leads to reduction and hopefully 
eventual correction of small position or orientation er¬ 
rors of the held part. Let v be a six-dimensional (6-D) 
vector representing the actual translational and angu- 



Fig. 36.18 (a) Planar representation of RCC. (b) Rota¬ 
tional part of RCC allowing workpiece to rotate, (c) Trans¬ 
lational part of RCC allowing workpiece to translate 


lar velocity of the held part, Do be the six-dimensional 
commanded velocity, and / be a six-dimensional vec¬ 
tor representing the sensed force and moment. A linear 
damping control law is described as 

v = v 0 + Af , 

where A is a 6 x 6 matrix, called an admittance matrix 
or accommodation matrix. 

The effectiveness of such a damping control law 
depends on the existence and finding of a proper ad¬ 
mittance matrix A. There is considerable research on 
the design of a single A that can make an assembly op¬ 
eration successful regardless of what contact states the 
held peg may encounter in the process [36.47-51]. This 
is aimed at cases where a single commanded velocity 
would be sufficient to achieve an assembly operation 
when there were no uncertainty or error, such as certain 
peg-in-hole insertion operations. One main approach 
to design A is based on explicit kinematic and static 
analysis of contact conditions under all possible contact 
states and mating requirements, which result in a set of 
linear inequalities as constraints on A. Learning is also 
used [36.50] to obtain an A that minimizes the force / 
without causing instability. Another approach [36.51] 
applies perturbations to the end-effector during inser¬ 
tion in order to obtain richer force information. 

Learning Control for Assembly 
Another category of approaches is to learn proper con¬ 
trol for a particular assembly operation through stochas¬ 
tic or neural-network-based methods [36.52-56]. The 
essence of most of these approaches is to learn to map 
a reaction force upon the held object caused by contact 
to the next commanded velocity in order to reduce er¬ 
rors and to achieve an assembly operation successfully. 
A more recent approach [36.56] maps fused sensory 
data of pose and vision obtained during human demon¬ 
stration of assembly tasks to compliant motion signals 
for successful assembly. 

A different approach observes assembly tasks per¬ 
formed by human operators through vision [36.57] or in 
a virtual environment [36.58, 59] and generates a mo¬ 
tion strategy necessary for the success of the task that 
consists of a sequence of recognized contact state tran¬ 
sitions and associated motion parameters. 

As a sequence of commanded velocities can be 
generated, unlike RCC or strategies based on a sin¬ 
gle admittance matrix described above, can be applied 
to cases with large uncertainties. However, the learned 
controllers are task dependent. 

More recently, an approach combining vision and 
force sensing for assembly was introduced [36.60] 
based on measuring the deformation of parts through 
vision to adjust desired forces for mating and achieve 
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force control through force sensing and an impedance 
controller. The approach was applied to the task of mat¬ 
ing two power connectors. The effect was similar to 
having an RCC device on the end-effector. 

All of the above assembly motion strategies do 
not require explicit recognition of contact states during 
their execution. 

36.4.3 Active Compliant Motion 

Active compliant motion is characterized by error cor¬ 
rection based on online identification or recognition of 
contact states in addition to feedback of contact forces. 
The capability for active compliant motion allows 
a robot the flexibility to deal with a much broader range 
of assembly tasks with large uncertainties and tasks be¬ 
yond assembly where compliance is required. An active 
compliant motion system generally requires the follow¬ 
ing components: a planner to plan compliant motion 
commands, an identifier to recognize contact states, 
state transitions, and other state information during task 
operation, and a controller to execute compliant motion 
plans based on both low-level feedback provided by 
sensors and high-level feedback provided by the identi¬ 
fier. Research on each component is described below. 

Fine Motion Planning 

Fine motion planning refers to planning fine-scale mo¬ 
tions to make an assembly task successful in spite of 
significant uncertainties. A general approach was pro¬ 
posed [36.61] based on the concept of preimages in 
configuration space (C-space) [36.35] to devise mo¬ 
tion strategies that would not fail in the presence of 
uncertainties. Given a goal state of the held part de¬ 
fined by a region of goal configurations, a preimage of 
the goal state encodes those configurations from which 
a commanded velocity will guarantee that the held part 
reaches the goal state recognizably in spite of location 
and velocity uncertainties (Fig. 36.19). 

Given the initial location of the held part and the 
goal state, the preimage approach generates a motion 


plan in backward chaining by finding the preimage of 
the goal state associated with a commanded velocity 
and then the preimage of the preimage, and so on, until 
a preimage that includes the initial configuration of the 
held part is found. Figure 36.20 shows an example. 

The sequence of commanded velocities associated 
with the sequence of preimages (starting from the ini¬ 
tial preimage that includes the initial location) forms 
the motion plan that guarantees the success of the task. 
Starting from the initial preimage, each subsequent 
preimage in the sequence can be viewed as a subgoal 
state. In this approach, compliant motions are preferred 
wherever possible because they typically produce larger 
preimages [36.61] than pure positioning motions, and 
subgoal states are often contact states. The approach 
was further extended [36.62] to include modeling un¬ 
certainties of objects. 

However, the computability of the approach is a ma¬ 
jor problem. The requirement of both goal reachability 
and recognizability under sensing uncertainties, which 
are intertwined issues, complicates the computation of 
preimages. It has been shown [36.63] that the time com¬ 
plexity of generating a plan can be double exponential 
in nmr, where n is the number of plan steps, m is the en¬ 
vironment complexity (of the physical space), and r is 
the dimension of the configuration space. By separat¬ 
ing reachability and recognizability and restricting the 
recognizability power of the original preimage model, 
computability was improved [36.64, 65]. 

As an alternative, a two-phase approach is to sim¬ 
plify the fine motion planning into (1) global and offline 
nominal path planning assuming no uncertainty and 
(2) local and online replanning to deal with unintended 
contacts due to uncertainties. Different variations of the 
two-phase approach have been proposed [36.66-72], 
The success of such an approach depends on success¬ 
ful online identification of contact states (Sect. 36.4.3). 

Several researchers also studied the representation 
and propagation of uncertainties and constraints that 
have to be satisfied for the success of a task [36.73- 
77], 


a) b) 



Fig.36.19a,b Location and velocity uncertainties (in the 
C-space). (a) The actual configuration of a part could be 
inside an uncertainty ball centered at an observed configu¬ 
ration. (b) The actual velocity of a part could be inside an 
uncertainty cone of a commanded velocity 



Fig.36.20a,b Backward chaining of preimages: P 1 is the 
preimage of the goal region G (a), and Pi is the preimage 

of Pi (b) 
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Compliant Motion Planning 
Compliant motion planning focuses on planning mo¬ 
tions of objects always in contact. Hopcroft and Wil- 
fong [36.78] proved that, if two arbitrary objects in 
contact can be moved to another configuration where 
they are also in contact, then there is always a path 
of contact configurations (on the boundary of con¬ 
figuration space obstacles, or C-obstacles) connecting 
the first and second contact configurations. Therefore, 
not only is compliant motion desirable in many cases, 
but it is always possible given initial and goal contact 
configurations. 

Because compliant motion occurs on the bound¬ 
ary of C-obstacles, planning compliant motion poses 
special challenges not present in collision-free motion 
planning: it requires the exact information of con¬ 
tact configurations on the boundary of C-obstacles. 
Unfortunately computing C-obstacles exactly remains 
a formidable task to date. While there are exact descrip¬ 
tions of C-obstacles for polygons [36.79,80], which 
are three dimensional, there are only approximations of 
C-obstacles for polyhedra [36.81, 82], which are six di¬ 
mensional. If m and n indicate the complexity of two 
polyhedral objects in contact, the complexity of the cor¬ 
responding C-obstacle is 0(m 6 n 6 ) [36.83]. 

Researchers have focused on either reducing the 
dimensionality and scope of the problem or get¬ 
ting around the problem of computing C-obstacles 
altogether. Some researchers have studied compliant 
motion planning on C-obstacles of reduced dimen¬ 
sions [36.72,84,85]. It is more popular to plan com¬ 
pliant motions based on a predetermined graph of 
topological contact states to avoid the problem of 
computing C-obstacles [36.86-89]. In particular, one 
approach [36.87, 88] models an assembly task of polyg¬ 
onal parts as a discrete event system using Petri nets. 
However, contact states and transitions are often gen¬ 
erated manually, which is awfully tedious for even 
assembly tasks of simple geometry [36.45] and is prac¬ 
tically infeasible for complex tasks due to the huge 
number of different contact states. 

Therefore, automatic generation of a contact state 
graph is desirable and even necessary. A method was 
first developed to enumerate all possible contact states 
and their connections between two convex polyhedral 
objects [36.90]. Recall that a contact state between two 
convex polyhedra can be described by a single prin¬ 
cipal contact (Sect. 36.4.1), and any principal contact 
between two topological surface elements of two con¬ 
vex polyhedra describes a geometrically valid contact 
state. These are nice properties and greatly simplify 
the problem of contact state graph generation. In gen¬ 
eral, however, to construct a contact state graph between 
two objects automatically requires the handling of two 


rather difficult issues: 

1. How to generate valid contact states, i. e., how to tell 
if a set of principal contacts corresponds to a geo¬ 
metrically valid contact state, given the geometries 
of objects, and 

2. How to link one valid contact state to another in 
the graph, i. e., how to find the neighboring (or 
adjacency) relations among the regions of contact 
configurations belonging to different contact states 
in the contact configuration space. 

A general and efficient divide-and-merge approach 
was introduced [36.38] for automatically generating 
a contact state graph between two arbitrary polyhedra. 
Each node in the graph denotes a contact state, de¬ 
scribed by a topological contact formation (CF) [36.37] 
as a set of principal contacts and a configuration sat¬ 
isfying the CF. Each edge connects the nodes of two 
neighboring contact states. Figure 36.21 shows an ex¬ 
ample contact state graph between two planar parts. 

The approach handled the above two issues si¬ 
multaneously by directly exploiting both topological 
and geometrical knowledge of contacts in the phys¬ 
ical space of objects and by dividing the problem 
into simpler subproblems of generating and merging 
special subgraphs. Specifically, the approach takes ad¬ 
vantage of the fact that a contact state graph can be 
divided into special subgraphs called the goal-contact 
relaxation (GCR) graphs, where each GCR graph is 
defined by a locally most constrained valid contact 
state, called the seed, and its less-constrained neigh¬ 
boring valid contact states, which is easier to gener¬ 
ate because of several properties. The main properties 
include: 

• Given a valid contact state CS, all of its less 
constrained neighboring contact states can be hy- 
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pothesized topologically from the principal contacts 
in CS;. 

• A hypothesized less constrained neighboring con¬ 
tact state CSj is valid, if and only if there exists 
a compliant motion to relax certain constraints of 
CS, to obtain CSj which does not result in any other 
contact state, called neighboring relaxation. 

• Neighboring relaxation can often be achieved by in¬ 
stantaneous compliant motion. 

With this approach, a contact state graph of several 
hundreds or thousands of nodes and links can be gener¬ 
ated in a few seconds. 

With a contact state graph, the problem of com¬ 
pliant motion planning can be decomposed into two 
simpler subproblems at two levels: (1) high level: graph 
search for state transitions from one node to another 
in a contact state graph, and (2) low level: contact 
motion planning within the set of contact configura¬ 
tions constrained by the same contact state (and the 
same contact formation), called CF-compliant motion 
planning. A general contact motion plan crossing sev¬ 
eral contact states can be considered as consisting of 
segments of CF-compliant motions in different con¬ 
tact states. One approach plans [36.91] CF-compliant 
motions based on random sampling of CF-compliant 
configurations and extending the probabilistic roadmap 
motion planning technique [36.92]. 

Contact State Identification 
Successful execution of a fine-motion or compliant mo¬ 
tion plan depends on correct online identification of 
contact states during execution. Contact state identifi¬ 
cation uses both model information of contact states 
(including topological, geometrical and physical in¬ 
formation) and sensory information of location and 
force/torque of the end-effector. The presence of sens¬ 
ing uncertainties make the identification problem non¬ 
trivial. Approaches for contact state identification are 
different in ways of dealing with uncertainties. 

One class of approaches obtain the mapping be¬ 
tween sensory data (in the presence of sensing uncer¬ 
tainties) and corresponding contact states through learn¬ 
ing. Models for learning include hidden Markov mod¬ 
els [36.93,94], thresholds [36.95,96], neural network 
structures, and fuzzy classifiers [36.97-101]. Training 
data are obtained either in an unsupervised way or by 
human task demonstration. Such approaches are task 
dependent: a new task or environment requires new 
training. 

Another class of approaches are based on analytical 
models for contact states. A common strategy is to pre¬ 
determine the set of configurations, constraints on con¬ 
figurations, or the set of forces/torques or force/torque 


constraints that are possible for each contact state and 
match such information against the sensed data to iden¬ 
tify contact states online. Some approaches do not 
consider the effect of uncertainties [36.102, 103], while 
others consider uncertainties modeled either by uncer¬ 
tainty bounds or by probability distributions [36.36, 89, 
104-108]. An alternative strategy is based on checking 
the distance between contacting elements in Cartesian 
space: either by growing the objects with pose uncer¬ 
tainty and intersecting the obtained regions [36.109], 
or by combining the distance between the objects with 
several other factors such as the instantaneous approach 
direction [36.110]. 

In terms of how sensory data are used, some iden¬ 
tification schemes are rather static because they do not 
consider the prior motion and state identifications be¬ 
fore the current contact state is reached, while others 
use prior history or even use new motion or active 
sensing to help current identification. The latter in¬ 
clude approaches that perform simultaneous contact 
state identification and parameter estimation. In such 
approaches, an analytical contact state model is ex¬ 
pressed as a function of uncertain parameters, such as 
the pose or dimension of an object in contact. At each 
contact state, some of the uncertain parameters can be 
observed or estimated with reduced uncertainty. For 
example, if the held object is at a face-face contact 
with another object, parameters describing the nor¬ 
mals of the contacting faces can be estimated with the 
help of force/torque sensing. During a task execution 
(i. e., the execution of a motion plan), the uncertain 
parameters are estimated along with the identification 
of contact states with increasing accuracy, which in 
turn makes the subsequent contact state identifications 
more accurate. This also improves force control and 
contact state transition monitoring. The increased per¬ 
formance, however, comes at a higher computational 
cost. 

Simultaneous contact state identification and pa¬ 
rameter estimation is most often done by ruling out 
contact state models that are inconsistent with sensed 
data or parameter estimation results [36. 111-1 17]. One 
notable exception [36.118] performs a truly simulta¬ 
neous estimation by describing the system as a hy¬ 
brid joint probability distribution of contact states and 
parameters. 

Active sensing is about deliberate use of applied 
force/torque or motion to better aid contact state iden¬ 
tification and parameter estimation. Earlier research 
focused on simple force strategy [36.1 19] or movability 
tests [36.36, 110]. More recently, methods were intro¬ 
duced [36.120] to design sequences of contact state 
transitions and compliant motion strategies that were 
optimized for active sensing to determine all uncertain 
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geometric parameters while achieving certain goal con¬ 
tact state. 

From a different perspective, an approach was in¬ 
troduced to analyze contact state distinguishability and 
unknown/uncertain parameter identifiability [36.121] 
during the design phase of a contact state identifier or 
parameter estimator. 

36.4.4 Compliant Motion 

Under Manipulator Constraints 

A general compliant motion plan (as the output of 
a compliant motion or fine motion planner) usually con¬ 
sists of a sequence of contact states, and within each 
contact state, a path of contact configurations compliant 
to the contact state and leads to the next contact state in 
the sequence. Each contact configuration is described in 
terms of the parts in contact. However, it takes a robot 
manipulator to move one part against the others, espe¬ 
cially in assembly tasks. Therefore, whether a contact 
state can be formed and whether a compliant motion 
and a contact state transition is possible is subject to the 
constraints of the manipulator. 

Feasibility of Contact States 
and Compliant Motions 

A contact state graph should be re-visited by consid¬ 
ering manipulator constraints [36.122]. Contact config¬ 
urations compliant to a given contact state are sampled 
and applied inverse kinematics to check if there is a fea¬ 
sible manipulator joint-space configuration to realize 
the contact state. If no feasible manipulator config¬ 
uration can be found after a number of samples are 
considered, the contact state is considered at least very 
difficult to reach, and is discarded from the graph as an 
infeasible state. 

Given two feasible adjacent contact states, a com¬ 
pliant motion for their transition should also be checked 
for feasibility under manipulator constraints. The main 
issue here is how to deal with possible singularities 
along the compliant path (of contact configurations). 
Certain singularities can be escaped if the manipula¬ 
tor is redundant (i. e., with more than six degrees of 
freedom) [36.123]. With inescapable singularities, the 
compliant path of the moved object is not feasible and 
has to be either altered to avoid singularity [36.124] 
or abandoned. However, an altered path may no longer 
be compliant to the desired contact states. In practice, 
it is preferred to check the feasibility of a compli¬ 
ant path starting from the goal contact state of an 


assembly task backward to make sure that the feasi¬ 
ble portion leading to the goal as well as the feasible 
contact states encountered are discovered and pre¬ 
served [36.125]. 

Execution of Feasible Compliant Motion Plans 
Depending on whether the manipulator is equipped 
with a force/torque sensor or not, different control 
schemes can be considered for executing a compliant 
motion plan that includes multiple contact state transi¬ 
tions (l<E&»'il>Hi>m) 

If the manipulator has a force/torque sensor, a hy¬ 
brid position/force controller can be used. It requires 
a specification that separates dimensions of force con¬ 
trol from those of position/velocity control. Such spec¬ 
ifications not only have to vary for different contact 
states but may have to vary from one contact config¬ 
uration to another within the same contact state, i. e., 
a control specification is a function of time or configura¬ 
tion along a compliant trajectory in general. Moreover, 
it is far from trivial to make control specifications 
for complex contact states involving multiple principal 
contacts [36.126]. 

An approach was introduced [36. 127] to convert au¬ 
tomatically a compliant path of contact configurations 
across a sequence of contact states into wrench, twist, 
and position control signals w(t), t(t), and p(t) for a hy¬ 
brid controller to execute the plan. The approach was 
experimentally verified successfully. 

More recently, an approach was introduced [36. 128] 
to achieve force control by estimating contact force/ 
torque from the joint torques without using a force 
sensor for the end-effector force/torque or torque sen¬ 
sors for individual joints. By reducing the integral gain 
of the joint PID (proportional-integral-derivative) con¬ 
troller, the approach acts like a high-pass filter to detect 
sharp force/torque changes more effectively, which of¬ 
ten indicates contact state changes. Combining such 
detection with position control, the approach was able 
to execute an assembly motion plan successfully, even 
though compliant motion could not always be main¬ 
tained with position control. 

Other approaches to realize force control without 
a force sensor requires the dynamic model of the ma¬ 
nipulator [36.128-130]. 

However, there is not yet a full integration of 
planning, online contact state identification, online re¬ 
planning (to deal with contact states that are off the 
preplanned path due to uncertainty), and compliant mo¬ 
tion control. 
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36.5 Unifying Feedback Control and Planning 


In previous sections we discussed task-level control 
methods and planning algorithms. Both task-level con¬ 
trol and planning methods are able to address specific 
constraints imposed on robot motion in the context of 
manipulation tasks. However, as illustrated in Fig. 36.3, 
neither of these categories of methods is able to address 
all constraints completely. Control remains susceptible 
to local minima and thus cannot guarantee that a par¬ 
ticular motion will lead to the desired result. Planning 
methods, on the other hand, overcome the problem of 
local minima by projecting possible motions into the 
future to predict if a sequence of motion will lead to 
success. Given certain assumptions, the resulting plan 
can be guaranteed to succeed. However, the compu¬ 
tations associated with this process are generally too 
computationally complex to satisfy the feedback re¬ 
quirements of manipulation tasks. Unfortunately, this 
means that control and motion planning by themselves 
are unable to address the problem of moving robots for 
general manipulations tasks. 

In this final section of this chapter, we will review 
efforts to combine the advantages of control methods 
and planning methods into a single approach to deter¬ 
mine the motion of a robot. These efforts aim to create 
methods that avoid the susceptibility to local minima 
while satisfying the feedback requirements. Early at¬ 
tempts of integrating planning and control occurred in 
the early 1990s. Not all of these efforts are specifically 
directed at manipulation but many contain insights rel¬ 
evant to the topic. In this section, we will review these 
efforts and also discuss some of the more recent re¬ 
search efforts aimed at unifying motion planning and 
feedback control. 

Motion planning (Chap. 7) and motion control 
(Chap. 8) have traditionally been regarded as two dis¬ 
tinct areas of research. However, these areas have many 
features in common. Both areas are concerned with the 
motion of robotic mechanisms, and more importantly 
planning and control methods both determine a repre¬ 
sentation that maps robot state to robot motion. In the 
case of feedback control, this representation is a po¬ 
tential function defined for a given region of the state 
space. The gradient of the potential function at a spe¬ 
cific state encodes the motion command. In contrast, 
motion planning determines motion plans. These mo¬ 
tion plans also encode motions for a set of states. 

Motion planning and feedback control also differ 
in certain aspects. A motion planner generally makes 
stronger assumptions than a controller about the envi¬ 
ronment, the ability to assess its state, and the changes 
that can occur in the environment. Motion planning also 
requires the ability to project the robot’s state into the 


future, given its current state and a particular action. By 
using this ability together with global information about 
the environment, motion planners can determine mo¬ 
tions that are not susceptible to local minima. This pos¬ 
itive characteristic of motion planners, however, results 
in significantly increased computational cost (Chap. 7). 
The large discrepancy in computational requirements 
for planning and control and the resulting divergence of 
computational techniques may explain the current sep¬ 
aration of the two fields. Researchers are beginning to 
attempt to reverse this separation by devising a unified 
theory of planning and control. 

36.5.1 Feedback Motion Planning 

Feedback motion planning combines planning and 
feedback into a single motion strategy. A planner con¬ 
siders global information to compute a feedback motion 
plan free of local minima. Such a feedback motion plan 
can be interpreted as a potential function or vector field 
whose gradient will lead the robot to the goal state 
from any reachable part of the state space [36.18]. Local 
minima-free potential functions are also called naviga¬ 
tion functions [36.131, 132], Given the current state of 
the robot and the global navigation function, feedback 
control is used to determine the robot’s motion. The 
consideration of feedback about the robot’s state in the 
context of a global navigation function reduces the sus¬ 
ceptibility to sensing and actuation uncertainty. 

Feedback motion planning, in principle, can ad¬ 
dress the entire spectrum of motion constraints and their 
feedback requirements (Fig. 36.3). Given a global nav¬ 
igation function that considers all motion constraints, 
the feedback requirements can easily be satisfied, since 
the feedback motion plan already specifies the desired 
motion command for the entire state space. Obviously, 
the major challenge in feedback motion planning is the 
computation of such a navigation function or feedback 
motion plan. The problem becomes particularly diffi¬ 
cult in the context of a manipulation task, since the state 
space (or configuration space) changes each time the 
robot grasps or releases an object in the environment 
(Sect. 36.3). This change makes it necessary to recom¬ 
pute the feedback motion plan. Frequent recomputation 
is also necessary in dynamic environments, where the 
motion of obstacles can repeatedly invalidate a previ¬ 
ously computed feedback motion plan. 

In the remainder of this section, we will review a va¬ 
riety of methods for computing navigation functions. In 
general, the problem of efficiently computing feedback 
motion plans for manipulation tasks remains unsolved. 
We therefore also present methods that do not explicitly 
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consider manipulation. These methods can be divided 
into three categories: (1) exact methods, (2) approx¬ 
imate methods based on dynamic programming, and 
(3) approximate methods based on composing and se¬ 
quencing simpler potential functions. 

The earliest exact methods for the computation of 
navigation functions are applicable to simple environ¬ 
ments with obstacles of specific shapes [36.131-133]. 
Approximate methods based on discretized spaces 
(grids) overcome this limitation but possess an ex¬ 
ponential computational complexity in the number of 
dimensions of the state space. These approximate nav¬ 
igation functions are called numerical navigation func¬ 
tions [36.134]. These navigation functions are often 
used for applications in mobile robotics. Due to the low¬ 
dimensional configuration space associated with mobile 
robots, they can generally solve motion planning prob¬ 
lems robustly and efficiently. 

Some physical processes, such as heat transfer or 
fluid flow, can be described by a specific type of dif¬ 
ferential equation, called harmonic functions. These 
functions possess properties that make them suitable 
as navigation functions [36.135-139]. Navigation func¬ 
tions based on harmonic functions are most commonly 
computed in an approximate, iterative fashion. The 
requirement for iterative computation increases the 
computational cost relative to the simpler numerical 
navigation functions [36.134]. 

More recent methods for the computation of numer¬ 
ical navigation functions consider differential motion 
constraints at a significantly reduced computational 
cost [36.140]. These methods rely on classical numer¬ 
ical dynamic programming techniques and numerical 
optimal control. However, in spite of their reduced 
computational complexity, these methods remain too 
computationally costly to be applied to manipula¬ 
tion tasks with many degrees of freedom in dynamic 
environments. 

Navigation functions can also be computed by com¬ 
posing local potential functions based on global infor¬ 
mation. This is illustrated in Fig. 36.22. The goal is 
to compute a navigation function for the entire config¬ 
uration space C. This is accomplished by sequencing 
overlapping funnels. Each of the funnels represents 
a simple, local potential function. By following the 
gradient of this potential function, motion commands 
can be determined for a subset of the configuration 
space. If the funnels are sequenced correctly, the com¬ 
position of local funnels can yield a global feedback 
plan. This feedback plan can be viewed as a hybrid 
system [36.141] in which the sequencing of funnels 
represents a discrete transition structure, whereas the 
individual controllers operate in a continuous domain. 
The composition of funnels is considered planning. The 


consideration of global information during the planning 
permits to determine a funnel composition that avoids 
local minima. 

One of the earliest methods based on the global 
composition of local funnels was proposed by Choi 
and Latombe [36.142]. In this method, the state space 
of the robot is decomposed into convex regions. The 
connectivity of these regions is analyzed to determine 
global information about the state space. This infor¬ 
mation can be used to combine simple, local potential 
functions, one for each convex regions of state space, 
into a local minima-free potential function. More rigor¬ 
ous approaches based on this idea have been developed. 
These approaches can consider the dynamics of the 
robot and nonholonomic motion constraints [36.143- 
145]. These methods have only been applied to low¬ 
dimensional state spaces and cannot easily be applied 
to manipulation tasks. 

The random neighborhood graph [36.146] is a sam¬ 
pling-based method of computing a decomposition of 
the overall state space. As before, a navigation function 
can be computed by analyzing the global connectiv¬ 
ity of the decomposition and imposing adequate local 
potential functions for each subdivision. A special¬ 
ized method following the same principle for planar 
robots in polygonal environments has also been pro¬ 
posed [36.147]. The idea of composing local potential 
functions has also been applied successfully to a com¬ 
plex robot control task [36.148]. Finally, in [36.149] 
a general and efficient method of computing a smooth 
feedback plan over a cylindrical algebraic decomposi¬ 
tion of configuration space has been proposed. 

The computation of a navigation function over the 
entire configuration space quickly becomes intractable 
as the dimensionality of the space increases. To over¬ 
come this challenge, in particular in the context of 
autonomous mobile manipulation, workspace heuristics 
have been employed to determine a navigation func¬ 
tion efficiently. This navigation function does not cover 
the entire configuration space but only those regions 
heuristically determined to be relevant to the motion 
problem [36.150]. This method of generating feedback 
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plans is able to satisfy the various motion constraints 
depicted in Fig. 36.3 and their respective feedback re¬ 
quirements. 

36.5.2 Augmenting Global Plans 
with Feedback 

The manipulation planning techniques described in 
Sect. 36.3 are not susceptible to local minima, as they 
consider global state space information. Due to the 
computational complexity associated with the consider¬ 
ation of global information, these planning techniques 
are not able to satisfy the feedback requirements of ma¬ 
nipulation tasks. However, the required frequency of 
feedback about global motion are relatively low: the 
global connectivity of the configuration space changes 
relatively infrequently (Fig. 36.3). It would therefore be 
possible to consider feedback for global motion at the 
slow rates the planner can accommodate, while consid¬ 
ering feedback for other motion constraints at higher 
frequencies. To achieve this, global motion plans have 
to be augmented with reactive components that incre¬ 
mentally modify the global plan in response to feedback 
from the environment. As long as the global connec¬ 
tivity information captured by the plan remains valid, 
the incremental modifications can ensure that all other 
motion constraints, ranging from task requirements to 
reactive obstacles avoidance, are satisfied. 

The elastic-band framework [36.151] augments 
global plans with reactive obstacle avoidance. A global 
configuration space path, determined by a planner, is 
covered with local potential functions, each of which is 
derived from the local distribution of obstacles around 
the path. These local potentials cause the path to deform 
so as to maintain a minimum distance from obsta¬ 
cles. Visually, the path behaves as an elastic band 
that is deformed by the motion of obstacles. The lo¬ 
cal potential functions, together with the global path, 
can be viewed as a navigation function for a lo¬ 


cal region of the configuration space. Integrated with 
a global planner and replanner, the elastic-band frame¬ 
work permits real-time obstacle avoidance that is not 
susceptible to local minima. However, the feedback 
frequency for global motion remains limited by the 
global motion planner. Specific task constraints have 
not been integrated into the elastic-band framework; 
consequently, its application to manipulation tasks is 
limited. 

In its original formulation, the elastic-band frame¬ 
work assumed that all degrees of freedom of the robot 
are holonomic. An extended formulation augments mo¬ 
tion paths for nonholonomic platforms with reactive 
components [36.152]. 

The elastic-strip framework [36.153] also augments 
global motion plans with reactive obstacle avoidance. 
In addition to reactive obstacle avoidance, however, 
the elastic-strip framework can accommodate task con¬ 
straints. Similarly to an elastic band, an elastic strip 
covers a global path with local potential functions. In 
contrast to the elastic-band framework, these potential 
functions are based on task-level controllers (Sect. 36.2) 
and therefore allow the task-consistent modification of 
the global path. The elastic-strip framework is therefore 
well suited for the execution of manipulation plans in 
dynamic environments. An elastic strip will be incre¬ 
mentally modified to represent a constraint-consistent 
trajectory, as long as the global information captured 
by the underlying plan remains valid. The elastic-strip 
framework has been applied to a variety of manipula¬ 
tion tasks on a mobile manipulation platform. 

Extending the elastic-band and elastic-strips frame¬ 
works, the elastic-roadmap framework combines re¬ 
active task-level control with efficient global motion 
planning [36. 150]. The elastic roadmap represents a hy¬ 
brid system of task-level controllers that are composed 
into a navigation function, thereby satisfying the motion 
constraints depicted in Fig. 36.3 and their respective 
feedback requirements. 


36.6 Conclusions and Further Reading 


In this chapter, an overview of motion generation and 
control strategies in the context of robotic manipula¬ 
tion tasks has been provided. Issues related to modeling 
the interfaces between the robot and the environment at 
the different time scales of motion and incorporating 
sensing and feedback were considered. Manipulation 
planning was introduced as an extension to the ba¬ 
sic motion planning problem, which can be modeled 
as a hybrid system of continuous configuration spaces 
arising from the act of grasping and moving parts in 


the environment. The important example of assem¬ 
bly motion has been discussed through the analysis of 
contact states and compliant motion control. The op¬ 
erational space framework, as described in Sect. 36.2, 
permits the position and force control of operational 
points on a serial-chain manipulator. A number of ex¬ 
tensions have been proposed in the literature, extending 
the framework to situations with multiple concurrent 
contact points, cooperative manipulation scenarios, and 
branching kinematic chains. For manipulation plan- 
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ning, extensions involving grasp and regrasp planning, 
multiple robots, multiple parts, and movable obstacles 
have been considered. In this section we will briefly 
discuss these extensions and refer to the appropriate 
literature. 

36.6.1 General Contact Models 

The hybrid force/motion control described by Raibert 
and Craig [36.154] has been shown to have shortcom¬ 
ings [36. 155]. A general contact model for dynamically 
decoupled force/motion control in the context of op¬ 
erational space control that overcomes these problems 
has been presented in [36.155]. This general contact 
model has been extended into a compliant motion con¬ 
trol framework to enable force control for multiple 
contact points [36.126] in nonrigid environments, and 
to enable force control for multiple contact points on 
different links of a kinematic chain [36.156]. The oper¬ 
ational space framework for task-level control has been 
applied for the real-time simulation of complex dy¬ 
namic environments [36.157]. Contact points between 
objects in the environment are modeled as operation 
points, resulting in a mathematically elegant framework 
for resolving impulses and contact constraints for mov¬ 
ing bodies. 

36.6.2 Cooperative Manipulation Control 

If multiple task-level controlled robots collaborate to 
manipulate an object, they form a closed kinematic 
chain that is connected through the object. The dy¬ 
namics of the entire system can be described using 
the notion of an augmented object [36.158,159], in 
which the dynamics of the manipulators and the object 
are combined to form a model of the overall system. 
The internal forces that occur at the grasp points dur¬ 
ing motion can be modeled using the virtual linkage 
framework [36.160]. A number of alternative strate¬ 
gies for the cooperative manipulation of multiple robots 
outside of the operational space framework have been 
proposed [36.161-168]. 

36.6.3 Control of Branching Mechanisms 

So far, we have implicitly assumed that the task-con- 
trolled robot consists of a single kinematic chain. This 
assumption does not hold in the case of kinemati¬ 
cally more complex mechanisms, such as humanoid 
robots (Chap. 67). These robots can consist of multi¬ 
ple branching kinematic chains. If we consider the torso 
of a humanoid robot to be the robot’s base, for exam¬ 
ple, then the legs, arms, and the head represent five 
kinematic chains attached to this base. We call such 


a mechanism a branching mechanism if it does not con¬ 
tain any closed kinematic loops. Tasks performed by 
a branching mechanism may require the specification 
of operational points on any one of those branches; for 
example, while the legs perform locomotion the hands 
achieve a manipulation task and the head is oriented to 
maintain visibility of the manipulated object. 

The operational space framework has been extended 
to task-level control of branching mechanisms [36.169]. 
This extension combined operational points and asso¬ 
ciated Jacobians and computes an operational space 
inertia matrix that combines all operational points. This 
matrix can be computed efficiently with an algorithm 
that in practice is linear in the number of operational 
points [36.170,171], More recently, the operational 
space framework has been applied to quadrupedal lo¬ 
comotion [36.172]. 

36.6.4 Nonholonomic Mobile Manipulation 

All previously discussed work on task-level control 
assumes that the degrees of freedom are holonomic 
(Chap. 40). For manipulator arms this is generally 
a valid assumption. The most common type of mobil¬ 
ity platform, however, is based on either differential 
drives, synchro-drives, or Ackerman steerings (point 
to a section), all of which are subject to nonholo¬ 
nomic constraints. The operational space framework 
has been extended to mobile manipulation platforms 
that combine a nonholonomic mobility platform with 
a holonomic manipulator arm [36.173-175], enabling 
the task-level control of a large class of mobile manip¬ 
ulation platforms. 

36.6.5 Learning Models with Uncertainty 

The efficacy of operational space control depends on 
the accuracy of the dynamic model of the robot. In par¬ 
ticular when multiple behaviors are executed using null- 
space projections, modeling errors can have significant 
effects. To overcome the reliance on accurate dynamic 
models, reinforcement learning can be used to learn op¬ 
erational space controllers [36.176]. 

36.6.6 Grasping and Regrasp Planning 

Deciding the intermediate stable configurations for 
a part necessary to complete a manipulation task and 
selecting proper grasps are both difficult problems in 
themselves that operate over a continuum of possible 
solutions. Simeon et al. developed a manipulation plan¬ 
ning framework that considers continuous grasps and 
placements [36.177]. However, selecting from among 
all possible grasp configurations and deciding when 
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to regrasp is still an open research problem. For an 
overview of various grasp quality metrics, see Miller 
and Allen [36.178,179] and Chap. 38. 

36.6.7 Multiple Parts 

The manipulation planning framework nicely general¬ 
izes to multiple parts, Tt- Each part has its 

own C-space, and C is formed by taking the Cartesian 
product of all the part C-spaces with the manipula¬ 
tor C-space. The set C a d m is defined in a similar way, 
but now part-part collisions also have to be removed, 
in addition to part-manipulator, manipulator-obstacle, 
and part-obstacle collisions. The definition of C sta bie re¬ 
quires that all parts be in stable configurations; the parts 
may even be allowed to stack on top of each other. The 
definition of C grasp requires that one part is grasped and 
all other parts are stable. There are still two modes, de¬ 
pending on whether the manipulator is grasping a part. 
Once again, transitions occur only when the robot is in 

Ctrans — C S t a ble Cl Cg raS p. 

36.6.8 Planning for Multiple Robots 

Generalizing to k robots would lead to 2k modes, in 
which each mode indicates whether each robot is grasp¬ 
ing a part. Multiple robots may even be allowed to grasp 
the same part, which leads to the interesting problem of 
planning for closed kinematic chains and cooperative 
motion (Chap. 39). Koga addressed multi-arm manipu¬ 
lation planning where the same object must be grasped 
and moved by several manipulator arms [36.21]. An¬ 
other generalization could allow a single robot to grasp 
more than one part simultaneously. 

36.6.9 Planning 

for Closed Kinematic Chains 

The subspace of C that results from maintaining kine¬ 
matic closure arises when multiple robots grasp the 
same part, or even when multiple fingers of the same 
hand grasp a single part (Chap. 19). Planning in this 
context requires that paths remain on a lower-dimen¬ 
sional variety for which a parameterization is not avail¬ 
able. Planning the motion of parallel mechanisms or 
other systems with loops typically requires maintaining 
multiple closure constraints simultaneously (Chap. 18). 


36.6.10 Planning with Movable Obstacles 

In some cases, the robot may be allowed to reposi¬ 
tion obstacles in the environment rather than simply 
avoid them. Wilfong first addressed motion planning 
in the presence of movable obstacles [36.180]. Wil¬ 
fong proved that the problem is PSPACE-hard even 
in two-dimensional environments where the final po¬ 
sitions of all movable objects are specified. Erdmann 
and Lozano-Perez considered coordinated planning for 
multiple moving objects [36.181]. Alami et al. pre¬ 
sented a general algorithm for the case of one robot 
and one movable object and formulated the space 
of grasping configurations into a finite number of 
cells [36.17]. Chen and Hwang developed a plan¬ 
ner for a circular robot that is allowed to push ob¬ 
jects aside as it moves [36.182]. Stilman and Kuffner 
considered movable obstacles in the context of nav¬ 
igation planning [36.183], and manipulation plan¬ 
ning [36.184]. Nieuwenhuisen et al. also developed 
a general framework for planning with movable obsta¬ 
cles [36.185]. 

36.6.11 Nonprehensile Manipulation 

Lynch and Mason explored scenarios in which grasping 
operations are replaced by pushing operations [36.186]. 
The space of stable pushing directions imposes non- 
holonomic constraints on the motion of the robot, 
which opens up issues related to controllability. Re¬ 
lated issues also arise in the context of part-feed¬ 
ers and manipulation for manufacturing and assembly 
(Chap. 54). 

36.6.12 Assembly Motion Extensions 

The work in assembly motion described in this chapter 
has focused on rigid-part assembly, and the parts con¬ 
sidered are mostly polyhedral or of simple nonpolyhe- 
dral shapes, such as round pegs and holes. More recent 
work on contact state analysis for curved objects can be 
found in [36.187]. An emerging field in assembly is mi¬ 
cro/nanoassembly (l<£ 2 >jKii> 12 J&LJ; Chap. 27). Flexible 
or deformable part assembly has also begun to attract 
attention [36.188-190], Virtual assembly [36.191] for 
simulation and prototyping is another interesting area 
of study. 
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37. Contact Modeling and Manipulation 


Imin Kao, Kevin M. Lynch, Joel W. Burdick 



Robotic manipulators use contact forces to grasp 
and manipulate objects in their environments. 
Fixtures rely on contacts to immobilize work- 
pieces. Mobile robots and humanoids use wheels 
or feet to generate the contact forces that al¬ 
low them to locomote. Modeling of the contact 
interface, therefore, is fundamental to analysis, 
design, planning, and control of many robotic 
tasks. 

This chapter presents an overview of the mod¬ 
eling of contact interfaces, with a particular focus 
on their use in manipulation tasks, including gras¬ 
pless or nonprehensile manipulation modes such 
as pushing. Analysis and design of grasps and 
fixtures also depends on contact modeling, and 
these are discussed in more detail in Chap. 38. 
Sections 37.2-37.5 focus on rigid-body models of 
contact. Section 37.2 describes the kinematic con¬ 
straints caused by contact, and Sect. 37.3 describes 
the contact forces that may arise with Coulomb 
friction. Section 37.4 provides examples of analy¬ 
sis of multicontact manipulation tasks with rigid 
bodies and Coulomb friction. Section 37.5 ex¬ 
tends the analysis to manipulation by pushing. 
Section 37.6 introduces modeling of contact inter¬ 
faces, kinematic duality, and pressure distribution 
and soft contact interface. Section 37.7 describes 
the concept of the friction limit surface and il¬ 
lustrates it with an example demonstrating the 
construction of a limit surface for a soft contact. 
Finally, Sect. 37.8 discusses how these more accu¬ 
rate models can be used in fixture analysis and 
design. 
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37.1 Overview 

A contact model characterizes both the forces that can 
be transmitted through the contact as well as the allowed 


relative motions of the contacting bodies. These char¬ 
acteristics are determined by the geometry of the con- 
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tacting surfaces and the material properties of the parts, 
which dictate friction and possible contact deformation. 

37.1.1 Choosing a Contact Model 

The choice of a contact model largely depends upon the 
application or analysis that must be carried out. When 
appropriate analytical models are used, one can deter¬ 
mine if a manipulation plan or fixture design meets de¬ 
sired functional requirements within the model’s limits. 

Rigid-Body Models 

Many approaches to manipulation, grasp, and fixture 
analysis are based on rigid-body models. In the rigid- 
body model, no deformations are allowed at the points 
or surfaces of contact between two bodies. Instead, con¬ 
tact forces arise from two sources: the constraint of 
incompressibility and impenetrability between the rigid 
bodies, and surface frictional forces. Rigid-body mod¬ 
els are straightforward to use, lead to computationally 
efficient planning algorithms, and are compatible with 
solid-modeling software systems. Rigid-body models 
are often appropriate for answering qualitative ques¬ 
tions such as will this fixture be able to hold my 
workpiece ? and for problems involving stiff parts with 
low to moderate contact forces. 

Rigid-body models are not capable of describing the 
full range of contact phenomena, however; for example, 
rigid-body models cannot predict the individual contact 
forces of a multiple-contact fixture (the static inde¬ 
terminacy problem [37.1,2]). Furthermore, workpieces 
held in fixtures experience non-negligible deformations 
in many high-force manufacturing operations [37.3-6]. 
These deformations, which crucially impact machin¬ 
ing accuracy, cannot be determined from rigid-body 
models. Also, a rigid-body model augmented with 
a Coulomb friction model can lead to mechanics prob¬ 
lems that have no solution or multiple solutions [37.7— 
15]. To overcome the limitations inherent in the rigid- 
body model, one must introduce compliance into the 
contact model. 

Compliant Models 

A compliant contact deforms under the influence of ap¬ 
plied forces. The forces of interaction at the contact are 


derived from the compliance or stiffness model. While 
compliant contact models are typically more compli¬ 
cated, they have several advantages: they overcome the 
static indeterminacy inherent to rigid-body models and 
they predict the deformations of grasped or fixtured 
parts during loading. 

A detailed model of the deformations of real ma¬ 
terials can be quite complex. Consequently, for analy¬ 
sis we often introduce lumped-parameter or reduced- 
order compliance models having a limited number of 
variables. In this chapter we summarize a reduced- 
order quasi-rigid-body approach to modeling that can 
model a variety of compliant materials in a way 
that is consistent with both the solid mechanics lit¬ 
erature and conventional robot analysis and planning 
paradigms. 

Finally, three-dimensional finite-element mod¬ 
els [37.16-18] or similar ideas [37.19,20] can be used 
to analyze workpiece deformations and stresses in fix¬ 
tures. While accurate, these numerical approaches have 
some drawbacks, for example, the grasp stiffness matrix 
can only be found through difficult numerical proce¬ 
dures. Stiffness matrices are often needed to compute 
quality measures that are the basis for optimal grasping 
plans or fixturing designs [37.21,22]. Thus, these nu¬ 
merical approaches are better suited for verifying final 
fixture designs. 

37.1.2 Grasp/Manipulation Analysis 

Once a contact model has been chosen, we can use it 
to analyze tasks involving multiple contacts. If a part is 
subject to multiple contacts, the kinematic constraints 
and force freedoms due to the individual contacts must 
be combined. This combined analysis facilitates ma¬ 
nipulation planning - choosing the contact locations, 
and possibly the motions or forces applied by those 
contacts, to achieve the desired behavior of the part. 
A prime example is the grasping or fixturing problem: 
choosing contact locations, and possibly contact forces, 
to prevent motion of a part in the face of external distur¬ 
bances. This well-studied topic is discussed in greater 
detail in Chap. 38. Other examples include problems 
of partial constraint, such as pushing a part or inserting 
a peg into a hole. 


37.2 Kinematics of Rigid-Body Contact 


Contact kinematics is the study of how two or more 
parts can move relative to each other while respecting 
the rigid-body impenetrability constraint. It also classi¬ 
fies motion in contact as either rolling or slipping. 


Consider two rigid bodies whose position and ori¬ 
entation (configuration) is given by the local coordi¬ 
nate column vectors qi and < 72 , respectively. Writing 
the composite configuration as q = (gj, q\) J , we de- 
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fine a distance function d(q ) between the parts that is 
positive when they are separated, zero when they are 
touching, and negative when they are in penetration. 
When d(q) > 0, there are no constraints on the motions 
of the parts. When the parts are in contact ( d(q ) = 0), 
we look at the time derivatives d, d. etc., to determine 
if the parts stay in contact or break apart as they follow 
a trajectory q(t). This can be determined by the follow¬ 
ing table of possibilities 

d d d ... 


>0 


no contact. 

<0 


infeasible (penetration), 

= 0 

>0 

breaking contact, 

= 0 

<0 

infeasible (penetration), 

= 0 

= 0 >0 

breaking contact, 

= 0 

etc. 

= 0 <0 

infeasible (penetration). 


The contact is maintained only if all time derivatives are 
zero. 

The first two time derivatives are written 


d = 



x d 2 d 

d = q -rr^q + 
dq- 



(37.1) 

(37.2) 


The terms dd/dq and d 2 d/dq 2 carry information about 
the local contact geometry. The former corresponds to 
the contact normal, while the latter corresponds to the 
relative curvature of the parts at the contact. 

If contact is maintained, we can classify the contact 
as slipping or rolling. Analogous to the table above, the 
contact is rolling if and only if there is zero relative tan¬ 
gential velocity, acceleration, etc., between the contact 
points on the parts. If the relative tangential velocity is 
nonzero, the parts are slipping; if the relative velocity is 
zero but relative tangential acceleration or (higher-order 
derivatives) is not, slipping is incipient. 

In this section we focus on a first-order analy¬ 
sis of contact kinematics. A first-order analysis con¬ 
cludes that contact is maintained if d(q) = 0 and d = 0. 
This local linearization of contact kinematics focuses 
on the velocity q and the contact normal in dd/dq; 
higher-order spatial derivatives of the contact geom¬ 
etry (curvature, etc.) are not considered. While this 
is a good starting point, it may occasionally lead to 
erroneous conclusions. For example, Rimon and Bur- 
click [37.23-25] showed that a first-order analysis may 
incorrectly predict mobility of a part in a fixture when 
a second-order analysis shows that it is in fact com¬ 
pletely constrained. 


Analysis of the kinematics of rolling contact of 
parameterized surfaces can be found in [37.26]; see 
also [37.27-30]. 

37.2.1 Contact Constraints 

As described in Chap. 2, a rigid body in space has six 
degrees of freedom, specified by the location of the ori¬ 
gin of a coordinate frame P affixed to the part and the 
orientation of this coordinate frame relative to an in¬ 
ertial frame O fixed in the world. Let °p P e R 3 be the 
position of the part center of mass and °R/> e 50(3) be 
the rotation matrix describing the orientation of the part 
relative to O. The spatial velocity of the part can be writ¬ 
ten as f G R 6 , sometimes called a twist, 

t=(co\ tt T )\ 

where co = ( w x , co y , co z ) T and v = (v*, v y , u : ) T give the 
angular velocity and linear velocity of P in the world 
frame O, respectively, such that 00 satisfies 

°R P = a> x °R p 
and v satisfies 

v = °p P — cox °p P . 

It is worth taking a moment to really understand the 
spatial velocity of a body. It consists of the body’s an¬ 
gular velocity expressed in the world frame O, along 
with the linear velocity of a point as if it were rigidly 
attached to the body but currently at the origin of the 
world frame. This point need not be physically on the 
body. In other words, v is not simply °p P . This nota¬ 
tion will simplify the following expressions, where all 
velocities and forces will be expressed in the common 
world frame O. (Be aware that twists are sometimes de¬ 
fined in a body frame instead.) 

A point contact acting on the part provides a uni¬ 
lateral constraint which prevents the part from locally 
moving against the contact normal. Let x be the loca¬ 
tion of the contact in O. The linear velocity of the point 
on the part in contact is 

Vc = v + co xx . 

(We drop the pre-superscripts O for simplicity; for ex¬ 
ample, °x is written simply as x.) Let u be the unit 
vector normal pointing into the part (Fig. 37.1). The 
first-order condition that the part not move into the uni¬ 
lateral constraint can be written 

vj.u = (v + co xx) 7 u > 0 . 


(37.3) 
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Fig. 37.1 Notation for a part in contact with a manipulator 
or the environment 

In other words, the velocity of the part at C cannot have 
a component in the opposite direction of the contact 
normal. To write this another way, define a general¬ 
ized force or wrench w consisting of the torque m and 
force / acting on the part for a unit force along the con¬ 
tact normal 

w = (tn T ,/ T ) T = [(* x m) t , ri T ] T . 

Then (37.3) can be rewritten as 

t T w> 0. ( 37 . 4 ) 


passes through the origin of the velocity space, and 
the feasible velocity set becomes a cone rooted at the 
origin. Let w, be the constraint wrench of stationary 
contact i. Then the feasible velocity cone is 

V = {t\ t T Wj > 0 Vi}. 

If the Wj span the six-dimensional generalized force 
space, or, equivalently, the convex hull of the w , con¬ 
tains the origin in the interior, then V is the null set, 
the stationary contacts completely constrain the motion 
of the part, and we have form closure, as discussed in 
Chap. 38. 

In the discussion above, each constraint (37.5) di¬ 
vides the part velocity space into three categories: 
a hyperplane of velocities that maintain contact, a half¬ 
space of velocities that separate the parts, and a half¬ 
space of velocities that cause the parts to penetrate. 
Velocities where the contact is maintained can be fur¬ 
ther broken down into two categories: velocities where 
the part slips over the contact constraint, and velocities 
where the part sticks or rolls on the constraint. In the 
latter case, the part velocity satisfies the three equations 

v + to x x = v ex t • (37.7) 


If the external constraint point is moving with linear 
velocity v ext , (37.4) changes to 

t T w>vJ xl u, ( 37 . 5 ) 

which reduces to (37.4) if the constraint is stationary. 

Each inequality of the form (37.5) constrains the ve¬ 
locity of the part to a half-space of its six-dimensional 
velocity space bounded by the hyperplane f T w = vj xt u. 
Unioning the set of all constraints, we get a convex 
polyhedral set of feasible part velocities. A constraint 
is redundant if its half-space constraint does not change 
the feasible velocity polyhedron. For a given twist t, 
a constraint is active if 

t T w = vj xt u ; ( 37 . 6 ) 


Now we can give each point contact i a label m, 
corresponding to the type of contact, called the contact 
label: b if the contact is breaking, f if the contact is 
fixed (including rolling), and s if the contact is slip¬ 
ping, i. e., (37.6) is satisfied but (37.7) is not. The 
contact mode for the entire system can be written as 
the concatenation of the contact labels at the k contacts, 
m\ni2 ... mi. 

37.2.2 Collections of Parts 

The discussion above can be generalized to find the fea¬ 
sible velocities of multiple parts in contact. If parts i 
and j make contact at a point x, where m, points into 
part i and w , = [(jc x m,) t . mJ] t , then their spatial ve¬ 
locities tj and tj must satisfy the constraint 


otherwise the part is breaking contact at that point. In 
general, the feasible velocity polyhedron for a part can 
consist of a six-dimensional interior (where no con¬ 
tact constraint is active), five-dimensional hyperfaces, 
four-dimensional hyperfaces, and so on, down to one¬ 
dimensional edges and zero-dimensional points. A part 
velocity on an n-dimensional facet of the velocity poly¬ 
hedron indicates that 6 — n independent (nonredundant) 
constraints are active. 

If all of the constraints are stationary (v ex t = 0), 
then the boundary of each half-space defined by (37.5) 


(ti-tj) T Wi> 0 ( 37 . 8 ) 

to avoid penetration. This is a homogeneous half-space 
constraint in the composite (t,, tj) velocity space. In 
an assembly of multiple parts, each pairwise contact 
contributes another constraint in the composite part ve¬ 
locity space, and the result is a polyhedral convex cone 
of kinematically feasible velocities rooted at the origin 
of the composite velocity space. The contact mode for 
the entire assembly is the concatenation of the contact 
labels at each contact in the assembly. 
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If there are moving contacts whose motion is pre¬ 
scribed, e.g., robot fingers, the constraints on the motion 
of the remaining parts will no longer be homogeneous. 
As a result, the convex polyhedral feasible velocity 
space is no longer a cone rooted at the origin. 

37.2.3 Graphical Planar Methods 

When a part is confined to move in the v-v-plane, 
the twist t reduces to t = (co x , co y , oj~, v x , v y , v z ) T = 
(0,0, (o z , v x , Vy, 0) T . The point (— Vy/a> z , v x /a> z ) is called 
the center of rotation (COR) in the projective plane, and 
we can represent any planar twist by its COR and rota¬ 
tional velocity a> z . (Note that the case co z = 0 must be 
treated with care, as it corresponds to a COR at infin¬ 
ity.) This is sometimes useful for graphical purposes: 
for a single part constrained by stationary fixtures, at 
least, we can easily draw the feasible twist cone as 
CORs [37.14,31]. 

As an example, Fig. 37.2a shows a planar part stand¬ 
ing on a table and being contacted by a robot finger. 


a) 





I SiSib 

| at infinity 


Fig. 37.2 (a) A part resting on a table and the three con¬ 
tact constraints. (b) The twist cone satisfying the contact 
constraints. The contact normals are shown with their asso¬ 
ciated constraint planes, (c) The equivalent representation 
as CORs, shown in grey. Note that the lines that extend 
off to the left and to the bottom wrap around at infinity 
and come back in from the right and the top, respec¬ 
tively, so this COR region should be interpreted as a single 
connected convex region, equivalent to the twist cone rep¬ 
resentation in (b). (d) The contact modes assigned to each 
feasible motion. The zero velocity contact mode is f f f 


This finger is currently stationary, but we will later set 
it in motion. The finger defines one constraint on the 
part’s motion and the table defines two more. (Note 
that contact points in the interior of the edge between 
the part and the table provide redundant kinematic con¬ 
straints.) The constraint wrenches can be written 

Wi = (0,0,-1,0,1,0) T , 

W2 = (0, 0 , 1 , 0 , 1 , 0) T , 

w 3 = (0,0,1,-1,0,0) T . 

For a stationary finger, the kinematic constraints yield 
the feasible twist cone shown in Fig. 37.2b. This re¬ 
gion can also be easily visualized in the plane by using 
the following method: at each contact, draw the con¬ 
tact normal line. Label all points on the normal ±, 
points to the left of the inward normal +, and points 
to the right —. For each contact constraint, all the points 
labeled + can serve as CORs with positive angular ve¬ 
locity, and all the points labeled — can serve as CORs 
with negative angular velocity, without violating the 
contact constraint. After doing this for all the contact 
normals, keep only the CORs that are consistently la¬ 
beled. These CORs are a planar representation of the 
feasible twist cone (Fig. 37.2c). 

We can refine this method by assigning contact 
modes to each feasible COR. For each contact nor¬ 
mal, label the COR at the contact point f for fixed, 
other CORs on the normal line s for slipping, and all 
other CORs b for breaking contact. The concatenation 
of these labels gives the part’s contact mode for a par¬ 
ticular part motion. In the planar case, the label s on 
the contact normal line can be further refined into s r or 
Si, indicating whether the part is slipping right or left 
relative to the constraint. An s COR labeled + above 
the contact (in the direction of the contact normal) or — 
below the contact should be relabeled s r , and an s COR 
labeled — above the contact or + below should be rela¬ 
beled si (Fig. 37. 2d). 

This method can be used readily to determine if the 
part is in form closure. If there is no COR labeled con¬ 
sistently, then the feasible velocity cone consists of only 
the zero velocity point, and the part is immobilized by 
the stationary contacts. This method also makes it clear 
that at least four contacts are necessary to immobilize 
the part by the first-order analysis (Chap. 38). This is 
a weakness of the first-order analysis - curvature ef¬ 
fects can be used to immobilize a part with three or 
even two contacts [37.24]. This weakness can also be 
seen in Fig. 37. 2d. A pure rotation about the COR la¬ 
beled {+, s r bs r } is actually not feasible, but it would 
be if the part had a small radius of curvature at the con¬ 
tact with the finger. The first-order analysis ignores this 
curvature. 
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37.3 Forces and Friction 

A commonly used model of friction in robotic manip¬ 
ulation is Coulomb’s law [37.32]. This experimental 
law states that the friction force magnitude f in the 
tangent plane at the contact interface is related to the 
normal force magnitude/,, by f < /jf n , where /x is called 
the friction coefficient. If the contact is sliding, then 
ft = fJ-fn, and the friction force opposes the direction of 
motion. The friction force is independent of the speed 
of sliding. 

Often two friction coefficients are defined, a static 
friction coefficient /x s and a kinetic (or sliding) fric¬ 
tion coefficient /x^, where /x s > /Xk- This implies that 
a larger friction force may be available to resist initial 
motion, but once motion has begun, the resisting force 
decreases. Many other friction models have been devel¬ 
oped with different functional dependencies on factors 
such as the speed of sliding and the duration of static 
contact before sliding. All of these are aggregate mod¬ 
els of complex microscopic behavior. For simplicity, we 
will assume the simplest Coulomb friction model with 
a single friction coefficient /x. This model is reasonable 
for hard, dry materials. The friction coefficient depends 
on the two materials in contact, and typically ranges 
from 0.1 to 1. 

Figure 37.3a shows that this friction law can be 
interpreted in terms of a friction cone. The set of all 
forces that can be applied to the disk by the supporting 



Fig. 37.3 (a) A planar friction cone, (b) The correspond¬ 
ing wrench cone, (c) An example composite wrench cone 
resulting from two frictional contacts 


line is constrained to be inside this cone. Correspond¬ 
ingly, any force the disk applies to the support is inside 
the negative of the cone. The half-angle of the cone 
is ft = tan -1 fi, as shown in Fig. 37.4. If the disk slips 
to the left on the support, the force the support applies 
to it acts on the right edge of the friction cone, with 
a magnitude determined by the normal force. 

If we choose a coordinate frame, the force / ap¬ 
plied to the disk by the support can be expressed as 
a wrench w = [(x x/) T ,/ T ] T , where x is the contact lo¬ 
cation. Thus the friction cone turns into a wrench cone, 
as shown in Fig. 37.3b. The two edges of the planar fric¬ 
tion cone give two half-lines in the wrench space, and 
the wrenches that can be transmitted to the part through 
the contact are all nonnegative linear combinations of 
basis vectors along these edges. If W\ and W 2 are ba¬ 
sis vectors for these wrench cone edges, we write the 
wrench cone as 


TVC = {k\W\ + k2U)2 | k\, &2 -2 0} . 

If there are multiple contacts acting on a part, then 
the total set of wrenches that can be transmitted to the 
part through the contacts is the nonnegative linear com¬ 
bination of all the individual wrench cones fV C,-, 

fVC = pos({TVC,}) 

T; kiWi\Wi g )VC„ k t > 0 

i 

This composite wrench cone is a convex polyhedral 
cone rooted at the origin. An example composite 




Fig. 37.4 (a) A spatial friction cone. The half-angle of the 
cone is / = tan" 1 /x. (b) An inscribed pyramidal approx¬ 
imation to the friction cone. A more accurate inscribed 
pyramidal approximation can be used by increasing the 
number of faces of the pyramid. Depending on the appli¬ 
cation, a circumscribed pyramid could be used instead of 
an inscribed pyramid 
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wrench cone arising from two planar frictional contacts 
is shown in Fig. 37.3c. If the composite wrench cone is 
the entire wrench space, then the contacts can provide 
a force-closure grasp (Chap. 38). 

In the spatial case, the friction cone is a circular 
cone, defined by 

fz> 0 , ( 37 . 9 ) 

when the contact normal is in the + 7 -direction 
(Fig. 37.4). The resulting wrench cone, and compos¬ 
ite wrench cone pos({lTC;}) for multiple contacts, is 
a convex cone rooted at the origin, but it is not poly¬ 
hedral. For computational purposes, it is common to 
approximate circular friction cones as pyramidal cones, 
as shown in Fig. 37.4. Then individual and composite 
wrench cones become polyhedral convex cones in the 
six-dimensional wrench space. 

If a contact or set of contacts acting on a part is 
ideally force controlled, the wrench w ext specified by 
the controller must lie within the composite wrench 
cone corresponding to those contacts. Because these 
force-controlled contacts choose a subset of wrenches 
(possibly a single wrench) from this wrench cone, the 
total composite wrench cone that can act on the part 
(including other non-force-controlled contacts) may no 
longer be a homogeneous cone rooted at the origin. This 
is roughly analogous to the case of velocity-controlled 
contacts in Sect. 37.2.1, which results in a feasible twist 
set for the part that is not a cone rooted at the origin. 
Ideal robot manipulators may be controlled by position 
control, force control, hybrid position-force control, or 
some other scheme. The control method must be com¬ 
patible with the parts’ contacts with each other and the 
environment to prevent excessive forces [37.33]. 

37.3.1 Graphical Planar Methods 

Just as homogeneous twist cones for planar problems 
can be represented as convex signed (+ or —) COR 
regions in the plane, homogeneous wrench cones for 
planar problems can be represented as convex signed 
regions in the plane. This is called moment label¬ 
ing [37.14,34]. Given a collection of lines of force in 
the plane (e.g., the edges of friction cones from a set of 
point contacts), the set of all nonnegative linear combi¬ 
nations of these can be represented by labeling all the 
points in the plane with either a + if all resultants make 
nonnegative moment about that point, a — if all make 
nonpositive moment about that point, a ± if all make 
zero moment about that point, and a blank label if there 
exist resultants making positive moment and resultants 
making negative moment about that point. 


The idea is best illustrated by an example. In 
Fig. 37.5a, a single line of force is represented by la¬ 
beling the points to the left of the line with a + and 
points to the right of the line with a —. Points on 
the line are labeled ±. In Fig. 37.5b, another line of 
force is added. Only the points in the plane that are 
consistently labeled for both lines of force retain their 
labels; inconsistently labeled points lose their labels. 
Finally, a third line of force is added in Fig. 37.5c. 
The result is a single region labeled +. A nonnegative 
combination of the three lines of force can create any 
line of force in the plane that passes around this re¬ 
gion in a counterclockwise sense. This representation 
is equivalent to a homogeneous convex wrench cone 
representation. 

37.3.2 Duality of Contact Wrenches 
and Twist Freedoms 

Our discussion of kinematic constraints and friction 
should make it apparent that, for any point contact and 
contact label, the number of equality constraints on the 
part’s motion caused by that contact is equal to the 
number of wrench freedoms it provides. For example, 
a breaking contact b provides zero equality constraints 
on the part motion and also allows no contact force. 
A fixed contact f provides three motion constraints (the 
motion of a point on the part is specified) and three free¬ 
doms on the contact force: any wrench in the interior 
of the contact wrench cone is consistent with the con¬ 
tact mode. Finally, a slipping contact s provides one 
equality motion constraint (one equation on the part’s 
motion must be satisfied to maintain the contact), and 
for a given motion satisfying the constraint, the con¬ 
tact wrench has only one freedom, the magnitude of 
the contact wrench on the edge of the friction cone and 
opposite the slipping direction. In the planar case, the 
motion constraints and wrench freedoms for b, s, and f 
contacts are zero, one, and two, respectively. 

a) b) ; c) 



± \ 


Fig. 37.5 (a) Representing a line of force by moment la¬ 
bels. (b) Representing the nonnegative linear combinations 
of two lines of force by moment labels, (c) Nonnegative 
linear combinations of three lines of force 
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Table 37.1 Duality of the kinematics of contact interface. The force/motion relationships for finger and grasped object 
are dual to each other in the joint space, contact interface, and the Cartesian space of the object. In this table, the 
notation of 6, jCf, x p , and Xb are the displacement in the joint space, the Cartesian space at the contact point on the 
finger, the Cartesian space at the contact point on the object, and the reference frame on the object, respectively, with x tt 
denoting the transmitted components. The S prefix denotes infinitesimal changes in the named coordinates. The forces 
with corresponding subscripts are referenced to the corresponding coordinates as those of the displacement described 
before 


Motion J () SO = Sxr 

(6 x m)(m x 1) (6x1) 


H <SXf = Sx a = H rt-Vp J c (>Xp = rtA'p 

(nx6)(6xl) (/1 x 1 ) (nx6)(6xl) (6x6)(6xl) (6 x1) 



Joints 


Contact 


Force 

Je /r = r 


h 

X 

II 

=/p 


(/«x6)(6xl) (wixl) 


(6x1) (6 x n)(nx\) 

(6xl) 


Object 

Jj / P = / P 

(6x6)(6x 1) (6xl) 


At each contact, the force and velocity constraints 
can be represented by an n x 6 constraint or selection 
matrix, H [37.3], at the contact interface. This con¬ 
straint matrix works like a filter to transmit or deny 
certain components of motion across the contact inter¬ 
face. By the same token, forces/moments applied across 
the contact interface are also filtered by the same con¬ 
straint matrix, H T , in a dual relationship. Three typical 
contact models are illustrated below. Contact forces and 
wrenches are measured in a frame at the contact, with 
the contact normal in the +t;-direction: 

• The point contact without friction model. Only 
a normal force f z can be exerted between the con¬ 
tacting bodies 

fz> 0, m = H T / : = (00 100 0)7;. (37.10) 


tact normal [37.35-39]. We have 

fz> 0; 

/ 1 0 0 0 \ 


( L \ 


0 10 0 


( fi \ 

fy 


0 0 10 


fy 

fz 


0 0 0 0 


fz 

\ m - / 


0 0 0 0 


V m z / 


V 0 0 0 1 / 

(37.12) 

where m z is the moment with respect to the normal 
of contact. The finite contact area assumed by the 
soft contact interface results in the application of 
a friction moment in addition to the traction forces. 
If the resultant force on the tangential plane of con¬ 
tact is denoted as 


• The point contact with friction model includes tan¬ 
gential friction forces, f x and f y , in addition to 
normal force/;, 


( 1 0 0 \ 



0 0 0 / 



(37.11) 


ft= yjf? +fy 

and the moment with respect to the contact normal 
is m z , the following elliptical equation represents 
the relationship between the force and moment at 
the onset of sliding 

/; 2 

7 + TT = 1 , (37.13) 

a 1 b- 

where a = pf z is the maximum friction force, and 
b = (wj;) max is the maximum moment defined in 
equation (37.36). Further readings on this subject 
can be found in [37.35^4-2], 


where f x and f. are the x- and y-components of the 
tangential contact force, and/, is the normal force. 

• The soft finger contact model with a finite contact 
patch allows, in addition to the friction and normal 
forces, a torsional moment with respect to the con- 


As can be appreciated from the three cases above, 
the rows of H are the directions along which contact 
forces are supported. Conversely, relative motions of 
the two objects are constrained along these same di¬ 
rections: H q = 0. Thus, the constraint matrix works 
like a kinematic filter which dictates the components 
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of motions that can be transmitted through the con¬ 
tact interface. The introduction of this matrix makes it 
easy to model the contact mechanics in analysis using 
the dual kinematic relationship between force/moment 
and motion, as described in Table 37.1. For example, 
the constraint filter matrix in (37.12) describes that all 
three components of forces and one component of the 
moment with respect to the normal of contact can be 
transmitted through the contact interface of a soft finger. 

In Chap. 38, when multiple contacts are considered, 
such H matrices associated with each finger can be 
concatenated in an augmented matrix to work in con¬ 


junction with the grasp and Jacobian matrices for the 
analysis of grasping and manipulation. 

Table 37.1 summarizes the dual relationship be¬ 
tween the force/moment and displacement - a result 
that is derived from the principle of virtual work. In Ta¬ 
ble 37.1, J g is the joint Jacobian matrix relating joint 
velocities to fingertip velocities, and J c is the Cartesian 
coordinate transformation matrix relating the contact 
point with the centroidal coordinates of the grasped 
object. In addition, the number of degrees of freedom 
(DOFs) in the task space is n and the number of DOFs 
in the joint spaces is m. 


37.4 Rigid-Body Mechanics with Friction 


The manipulation planning problem is to choose the 
motions or forces applied by manipulator contacts so 
that the part (or parts) moves as desired. This requires 
solving the subproblem of determining the motion of 
parts given a particular manipulator action. 

Let q e R" be local coordinates describing the com¬ 
bined configuration of the system consisting of one or 
more parts and robot manipulators, and let Wj e R 6 
represent a wrench at an active contact i, measured 
in the common coordinate frame O. Let tn a n e R 6i 
be the vector obtained by stacking the w t , u> a n = 
(wj, wj,.... wj) r (where there are k contacts), and 
let A (q) G R" x6i ' be a matrix indicating how (or if) each 
contact wrench acts on each part. (Note that a con¬ 
tact wrench w ,• acting on one part means that there is 
a contact wrench — w, acting on the other part in con¬ 
tact.) The problem is to find the contact forces u> a n and 
the system acceleration q , given the state of the system 
(q, q). the system mass matrix M (q) and resulting Cori¬ 
olis matrix C(q,q), the gravitational forces g (q), the 
control forces r, and the matrix T(^) indicating how the 
control forces r act on the system. (Alternatively, if we 
view the manipulators as position controlled, the ma¬ 
nipulator components of q can be directly specified and 
the corresponding components of the control forces r 
solved for.) One way to solve this problem is to (a) enu¬ 
merate the set of all possible contact modes for the 
current active contacts, and (b) for each contact mode, 
determine if there are wrenches iu a n and accelerations q 
that satisfy the dynamics 

A(<jr)t« a ii + T(q)T - g (q) = M {q)q + C(q,q)q , 

(37.14) 

and that are consistent with the contact mode’s 
kinematic constraints (constraints on q) and friction 
cone force constraints (constraints on tn a u). This for¬ 


mulation is quite general and applies to multiple 
parts in contact. Equation (37.14) may be simpli¬ 
fied by appropriate representations of the configu¬ 
rations and velocities of rigid-body parts (for ex¬ 
ample, using angular velocities of the rigid bod¬ 
ies instead of derivatives of local angular coordi¬ 
nates). 

This formulation leads to some surprising conclu¬ 
sions: there may be multiple solutions to a particular 
problem (ambiguity) or there may be no solutions (in¬ 
consistency) [37.7-14]. This strange behavior arises 
from the status of Coulomb’s law as an approximate 
law, and it disappears for zero friction, or friction that 
is small enough. Despite this weakness of the Coulomb 
friction law, it is a useful approximation. Nonetheless, if 
we would like to prove (under the Coulomb model) that 
a particular desired part motion occurs, we generally 
must also show that no other motion can occur. Other¬ 
wise we have only shown that the desired motion is one 
of the possible outcomes. 

37.4.1 Complementarity 

The conditions that each contact provides an equal 
number of motion constraints and wrench freedoms 
(Sect. 37.3.2) can be written as complementarity con¬ 
ditions. Thus the problem of solving (37.14), subject 
to contact constraints, can be formulated as a comple¬ 
mentarity problem (CP) [37.12, 13,43^-6]. For planar 
problems, or spatial problems with approximate pyra¬ 
midal friction cones, the problem is a linear comple¬ 
mentarity problem [37.47]. For circular spatial friction 
cones, the problem is a nonlinear complementarity 
problem, due to the quadratic constraints describing 
the cones. In either case, standard algorithms can be 
used to solve for possible contact modes and part 
motions. 
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Alternatively, assuming linearized friction cones, 
we can formulate a linear constraint satisfaction pro¬ 
gram (LCSP) for each contact mode (e.g., a linear 
program with no objective function). The contact mode 
places linear constraints on the part accelerations, and 
the solver solves for the part accelerations and the non¬ 
negative coefficients multiplying each of the edges of 
the friction cones, subject to the dynamics (37. 14). Each 
LCSP with a feasible solution represents a feasible con¬ 
tact mode. 

37.4.2 Quasistatic Assumption 

A common assumption in robot manipulation planning 
is the quasistatic assumption. This assumption says that 
parts move slowly enough that inertial effects are neg¬ 
ligible. This means that the right- and left-hand sides 
of (37.14) are zero. With this assumption, we often 
solve for part velocities rather than accelerations. These 
velocities must be consistent with the kinematic con¬ 
straints and force constraints, and forces acting on the 
parts must always sum to zero. 

37.4.3 Examples 

While rigid-body mechanics problems with friction are 
usually solved using computational tools for CPs and 
LCSPs, equivalent graphical methods can be used for 
some planar problems to assist the intuition. As a simple 
example, consider the pipe clamp in Fig. 37.6 [37.14]. 
Under the external wrench iu ext , does the clamp slide 
down the pipe, or does it remain fixed in place? The 
figure uses moment labeling to represent the compos¬ 
ite wrench cone of contact forces that can act on the 
clamp from the pipe. The wrench w ext acting on the 
clamp can be exactly balanced by a wrench in the com- 


tv. 



Fig. 37.6 The moment-labeling representation of the com¬ 
posite wrench cone of contact forces the pipe can apply to 
the pipe clamp. In this figure, the wrench w ext applied to 
the pipe clamp can be resisted by forces within the com¬ 
posite wrench cone 


posite wrench cone. This is evident from the fact that 
the wrench opposing w ext passes around the + labeled 
region in a counterclockwise sense, meaning that it is 
contained in the contact wrench cone. As a result, static 
equilibrium (the ff contact mode) is a feasible solu¬ 
tion for the pipe clamp. To arrive at the same result 
using an LCSP, label the unit wrenches at the edges of 
the friction cones W\ ,..., w 4. Then the clamp can re¬ 
main at rest if there exist coefficients a 1 ,... ,<34 such 
that 

a\ , 02, < 23 , 04 > 0 , 

aiWi + CI2W2 + 03103 + 04104 + to ext = 0 . 

To show that static equilibrium is the only solution, all 
other contact modes must be ruled out. Note that, if 
the friction coefficient at the contacts is too small, the 
clamp will fall under tu ext . 

Analysis of the classic peg-in-hole problem is sim¬ 
ilar to that of the pipe clamp. Figure 37.7a shows an 
angled peg making two-point contact with the hole. If 
we apply the wrench w 1 , the forces from the two-point 
contact cannot resist, so the f f mode contact mode is 
not a possibility, and the peg will continue to move into 
the hole. If we apply the wrench W2, however, the con¬ 
tacts can resist, and the peg will get stuck. This is called 
jamming [37.48]. If there is higher friction at the con¬ 
tacts, as in Fig. 37.7b, then each friction cone may be 
able to see the base of the other, and the feasible contact 
wrenches span the entire wrench space [37.49]. In this 
case, any wrench we apply may be resisted by the con¬ 
tacts. The peg is said to be wedged [37.48]. Whether 
or not the peg will actually resist our applied wrench 
depends on how much internal force acts between the 
contacts. This cannot be answered by our rigid-body 
model; it requires a compliant contact model, as dis¬ 
cussed in Sect. 37.6. 

As a slightly more complex quasistatic example, 
consider a block on a table being pushed by a finger 



external wrench w 1 but will aet stuck under the wrench W 2 . 
(b) The peg is wedged 
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Fig. 37.8 (a) A finger moves left into a planar block resting 
on a table in gravity. The contact friction cones are shown, 
(b) Possible contact forces for the contact label f at the 
leftmost bottom contact, b at the rightmost bottom con¬ 
tact, and s r at the pushing contact. This contact mode fbr 
corresponds to the block tipping over the leftmost contact. 
Note that the contact wrench cone (represented by moment 
labels) can provide a force that exactly balances the grav¬ 
itational force. Thus this contact mode is quasistatically 
possible, (c) The contact wrench cone for the sisif con¬ 
tact mode (block slides left on the table) cannot balance 
the gravitational force. This contact mode is not quasistat¬ 
ically possible, (d) The contact wrench cone for the sibs r 
contact mode (block slides and tips) is not quasistatically 
possible 

that moves horizontally to the left (Fig. 37.8) [37.50, 
51]. Does the part begin to tip over, slide, or both 
tip and slide? The corresponding contact modes are 
fbs r , sisif, and sibs r , as shown in Fig. 37.8. The 
figure also shows the composite contact wrench cones 
using moment labels. It is clear that quasistatic bal¬ 
ance between the gravitational force and the contact 
wrench cone can only occur for the tipping with¬ 
out slipping contact mode fbs r . Therefore, the only 
quasistatic solution is that the block begins to tip 
without sliding, and the speed of this motion is de¬ 
termined by the speed of the finger’s motion. The 
graphical construction can be used to confirm our in¬ 
tuition that tipping of the block occurs if we push 
high on the block or if the friction coefficient at the 
block’s support is high. Try it by pushing a can or 




A P 

1 

• 1 




b) Velocity 



Fig. 37.9 (a) A part supported by a horizontally vibrat¬ 
ing surface, (b) Friction between the part and the surface 
causes the part to always try to catch up to the surface, 
but its acceleration is bounded by ±/zg. The asymmetric 
motion of the surface gives the part an average positive ve¬ 
locity over a cycle 

glass. Quasistatically, the height of the center of mass 
is immaterial. 

A final example is given in Fig. 37.9. A part of 
mass 111 and friction coefficient /z is supported by a hor¬ 
izontal surface that moves periodically in the horizontal 
direction. The periodic motion consists of a large neg¬ 
ative acceleration for a short duration and a smaller 
positive acceleration for a longer duration. The hori¬ 
zontal friction force that can be applied to the part is 
bounded by ±/z;wg, so the part’s horizontal accelera¬ 
tion is bounded by ±/zg. As a result, the part cannot 
keep up with the surface as it accelerates backward. In¬ 
stead, the part slips forward with respect to the surface, 
which means that the maximal friction force acts in the 
negative direction, and the part attempts to slow down 
to the surface’s velocity. Eventually the part catches up 
with the surface as it executes its slow forward acceler¬ 
ation, which is less than /zg. Once the part catches up 
to the surface, it sticks to it, until the next backward ac¬ 
celeration phase. As is evident in Fig. 37.9, the average 
part velocity over a cycle is positive, so the part moves 
forward on the surface [37.52,53]. This idea can be ex¬ 
tended to create a wide variety of frictional force fields 
on a rigid plate vibrated with three degrees of freedom 
in a horizontal plane [37.54, 55] or with a full six de¬ 
grees of freedom [37.56]. 

For further examples of manipulation planning 
using the rigid-body model with Coulomb friction, 
see [37.14] and the references therein. 
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37.5 Pushing Manipulation 

The friction limit surface (Sect. 37.7) is useful for ana¬ 
lyzing pushing manipulation, as it describes the friction 
forces that can occur as a part slides over a support sur¬ 
face. When the part is pushed with a wrench contained 
within the limit surface, friction between the part and 
the support resists the pushing wrench and the part re¬ 
mains motionless. When the part slides quasistatically, 
the pushing wrench w lies on the limit surface, and 
the part’s twist t is normal to the limit surface at w 
(Fig. 37.10). When the part translates without rotating, 
the friction force magnitude is fling , where m is the 
part’s mass and g is gravitational acceleration. The force 
applied by the part to the surface is directed through the 
part’s center of mass in the direction of translation. 

If the pushing wrench makes positive moment about 
the part’s center of mass, the part will rotate counter¬ 
clockwise (CCW), and if it makes negative moment 
about the part’s center of mass, it will rotate clock¬ 
wise (CW). Similarly, if the contact point on the part 
moves along a line that passes around the part’s cen¬ 
ter of mass in a CW (respectively, CCW) sense, then 
the part will rotate CW (CCW). From these two ob¬ 
servations and considering all possible contact modes, 
we can conclude that a part pushed at a point contact 
will rotate CW (CCW) if either (1) both edges of the 
contact friction cone pass CW (CCW) about the cen- 
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Fig. 37.10 (a) Contact between a pusher and a sliding object, 
(b) The composite contact wrench cone, (c) Mapping the wrench 
cone through the limit surface to twists of the object, (d) The unit 
twists that can result from forces in the composite wrench cone 


ter of mass, or (2) one edge of the friction cone and 
the pushing direction at the contact both pass around 
the center of mass in a CW (CCW) sense [37.14,57], 
(Fig. 37.11). 

This observation allows pushing to be used to re¬ 
duce uncertainty in part orientation. A series of pushes 
with a flat fence can be used to completely elimi¬ 
nate the uncertainty in the orientation of a polygo¬ 
nal part [37.57-59]. Bounds on the rate of rotation 
of parts [37.14,60,61] allow the use of a sequence 
of stationary fences suspended above a conveyor belt 
to orient parts by pushing them as they are carried 
along by the conveyor [37.62,63]. Stable pushing plans 
(Fig. 37.12) use pushing motions that are guaranteed 
to keep the part fixed to the pusher as it moves, even 



Fig. 37.11 The pusher’s motion direction votes for counter¬ 
clockwise rotation of the part, but it is outvoted by the two 
edges of the friction cone, which both indicate clockwise 
rotation 



Fig. 37.12 Stable pushes can be used to maneuver a part 
among obstacles 
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in the face of uncertainty in the part’s pressure dis¬ 
tribution [37.14,64,65]. Extensions of this work find 
pushing motions for planar assemblies of parts so that 
they remain fixed in their relative configurations during 
motion [37.66-68], 


The examples above assume that pushing forces and 
support friction forces act in the same plane. Other work 
on pushing has considered three-dimensional effects, 
where pushing forces are applied above the support 
plane [37.69]. 


37.6 Contact Interfaces and Modeling 


Contact interface is a general expression to describe the 
kinematics and kinetics of contacts. Contact applies in 
various contexts in robotics research; thus, it is more 
meaningful to refer to generic contact as an interface, 
which is not limited to fingers in grasping and manip¬ 
ulation. The concept of contact interface extends the 
traditional context of physical contact. It refers to an in¬ 
terface which imparts kinematic filtering as well as the 
duality of force/motion transmitted across the contact 
interface. 

Consequently, a contact interface, whether it is rigid 
or deformable, can be considered as a kinematic fil¬ 
ter which includes two characteristics: (a) motion and 
force transmission, and (b) kinematic duality between 
force/moment and motion. Different contact interfaces 
will be described in the following sections. 

Previous sections of this chapter have assumed rigid 
bodies in contact. In reality, however, all contacts are 
accompanied by some deformation of the objects. Of¬ 
ten this is by design, as in the case of compliant robot 
fingertips. When deformation is non-negligible, elastic 
contact models can be used. 

37.6.1 Modeling of Contact Interface 

Contact modeling depends on the nature of bodies 
in contact, including their material properties, applied 
force, contact deformation, and elastic properties. This 
section discusses different contact models. 

Rigid-Body and Point-Contact Models 
With the rigid-body assumption, as discussed in the pre¬ 
vious sections of this chapter, two models are often 
used: (a) point contact without friction and (b) point 
contact with friction. In the former case, the contact can 
only apply force in the direction normal to the contact. 
In the latter case, a tangential friction force is applied 
in addition to the normal force. The simplest analytical 
model for point contact with friction is the Coulomb 
friction model as presented in (37.9). 

Hertzian Contact Model 

Elastic contact modeling was first studied and formu¬ 
lated more than a century ago by Hertz in 1882 [37.70] 


based on contact between two linear elastic materials 
with a normal force which results in very small con¬ 
tact deformation. This is commonly called the Hertzian 
contact and can be found in most mechanics textbooks, 
such as [37.71,72]. Hertz made two important explicit 
assumptions in order for his contact model to be appli¬ 
cable: 

1. Objects of linear elastic materials in contact, 

2. Small contact deformation compared to the dimen¬ 
sion of object. 

Hertz also conducted experiments using a spheri¬ 
cal glass lens against a planar glass plate to validate the 
contact theory. 

Two relevant results of the Hertzian contact theory 
applied to robotic contact interfaces are summarized as 
follows. The first pertains to the radius of the contact 
area. Hertz [37.70] studied the growth of the contact 
area as a function of the applied normal force N based 
on the linear elastic model. Based on ten experimental 
trials, he concluded that the radius of contact is propor¬ 
tional to the normal force raised to the power of 1/3, 
which is consistent with the analytical results he derived 
based on the linear elastic model. That is, the radius of 
contact, a, is related to the normal force, N, by 

aocN? . ( 37 . 15 ) 

The second result pertains to the pressure distri¬ 
bution over the assumed symmetric contact area - 
a second-order pressure distribution of the nature of an 
ellipse or circle. For a symmetric and circular contact 
area, the pressure distribution is 

N I /r \2 

P(r)=~ Jl-(-) . ( 37 . 16 ) 

n a 1 V 

where N is the normal force, a is the radius of contact, 
and r is the distance from the center of contact with 
0 < r < a. 

Soft Contact Model. A typical contact interface be¬ 
tween a soft finger and contact surface is illustrated in 
Fig. 37.13. In typical robotic contact interfaces, the ma¬ 
terials of the fingertips are not linear elastic. A model 
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that extends linear to nonlinear elastic contact was pre¬ 
sented in [37.38] with a power-law equation which 
subsumes the Hertzian contact theory, 

a = c N y , ( 37 . 17 ) 

where y = n/(2n+ 1) is the exponent of the normal 
force, n is the strain-hardening exponent, and c is 
a constant depending on the size and curvature of the 
fingertip as well as the material properties. Equation 
(37.17) is the new power law that relates the growth of 
the circular contact radius to the applied normal force 
for soft fingers. Note that the equation is derived assum¬ 
ing circular contact area. For linear elastic materials, the 
constant n is equal to 1 with y = 1/3, resulting in the 
Hertzian contact model in (37.15). Therefore, the soft 
contact model in (37.17) subsumes the Hertzian contact 
model. 


Two popular viscoelastic models include: the 
Kelvin-Voigt/Maxwell model (the spring-damper 
model) and the Fung’s model, which will be described 
in the following. 

(1) Kelvin-Voigt/Maxwell model. The Kelvin- 
Voigt’s solid model was constructed by a spring and 
a damper connected in parallel [37.77] as shown in 
Fig. 37.14. The relation between the stress, a, and 
strain, e, can be formulated as 

de 

a = ke + c —. ( 37 . 18 ) 

d t 

Maxwell proposed in 1867 the Maxwell fluid 
model [37.78] as a single set of spring and damper in 
series as shown in Fig. 37.15. The relationship between 
the stress, ct, and strain, e, can be expressed as 


Viscoelastic Soft Contact Model. Typical soft mate¬ 
rials used in robotic fingertip, e.g., rubber, silicone and 
polymers, show viscoelastic properties. Viscoelasticity 
is a physical phenomenon of time-dependent strain and 
stress [37.73]. Specifically in the context of robotic 
grasping and manipulation, there are two types of time 
dependent responses [37.74-76]: 

1 . Relaxation : The evolution of force in grasping while 
the displacement is held constant; 

2. Creep : The evolution of displacement in contact and 
grasping while the external force is held constant. 

Such time-dependent responses would approach 
equilibrium asymptotically. Viscoelastic materials also 
exhibit the properties of: 

1 . Strain history dependence : The response of the ma¬ 
terial depends on prior strain history; and 

2. Energy dissipation: A net energy dissipation asso¬ 
ciated with a complete cycle of loading and unload¬ 
ing. 




Fig. 37.13 An elastic soft fingertip with a hemispherical tip 
making contact with a rigid surface 


1 do o de 
k dt c dt 


( 37 . 19 ) 


The generalized Maxwell model utilizes multiple 
serial spring-damper sets and a spring connected in par¬ 
allel. One can employ the curve-fitting techniques using 
the experimental data to find the modeling parameters 
of springs and dampers. 

However, the issue of this model is often on the 
lack of consistency of the parameters (stiffness con¬ 
stants and damping factors) obtained from the model. 
Such parameters can have large discrepancy in nu¬ 
merical values, often in two orders of magnitude or 
higher, while representing the same material with sup¬ 
posedly similar values of springs and dampers. These 
parameters sometimes can also present unrealistic dif¬ 
ference in scales under different experimental setup 
(e.g., in [37.79,80]). 

(2) Fung's model. A popular viscoelastic model 
in modeling biomedical materials is the Fung’s 
model [37.76], proposed by Fung in 1993. The main 
idea of the model is to represent the reacting force as 
the product of two independent responses: the temporal 
response and the elastic response, while incorporating 




Fig. 37.14 

Kelvin-Voigt 
model 




Fig. 37.15 

Maxwell model 
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the history of the stress response. The model may be 
written as 


n,)= f GO- 


where T(t) is the tensile stress at time f, with a step 
increase of size A in elongation on the specimen, the 
function (A) is the so-called elastic response, and 
G(f), a normalized function of time, is the reduced re¬ 
laxation function. 

Tiezzi and Kao [37.42, 81, 82] simplified this model 
to study the soft contact interface by assuming no past 
stress history, as expressed in (37.21) 

§(S,t) = N M (8)-g(t). ( 37 . 21 ) 

where Cj(8.1) represents the grasping force as a func¬ 
tion of the displacement 8 and time t, N(8) represents 
the elastic response of normal force as a function of the 
displacement (or depression unto the object), and g(t) 
represents the temporal response of relaxation or creep. 
The important property of this model is the separation 
of the spatial response and temporal response as two 
independent functions. It was shown that the viscoelas¬ 
tic model has an important implication on the stability 
of grasping which cannot be captured by rigid or linear 
elastic modeling. 

Other Models. In addition to the aforementioned 
models, other models from different viewpoints were 
proposed, as follows: From the viewpoint of rheo¬ 
logical perspectives [37.83-86]; molecular perspec¬ 
tives [37.73,87-92]; energy perspectives [37.85]; dis¬ 
tributed modeling perspectives [37.93-98] and stress 
wave propagation perspectives [37.99-102]. 


37.6.2 Pressure Distribution at Contacts 


In Sect. 37.6.1, when the Hertzian contact theory is 
considered, the assumed pressure distribution for small 
elastic deformation is given in (37.16). As the radius of 
curvature of the two asperities increases and the ma¬ 
terial properties change to hyperelastic, the pressure 
distribution becomes more uniform [37.38, 103, 104]. 
Generalizing equation (37.16), the pressure distribution 
function for circular contact area with radius a is 


N 

P{r) = C k —- 
it a £ 



( 37 . 22 ) 


where N is the normal force, a is the radius of con¬ 
tact, /• is the radius with 0 < r < a, k determines the 
shape of the pressure profile, and C* is a coefficient 
that adjusts for the profile of pressure distribution over 
the contact area to satisfy the equilibrium condition. 
In (37.22), p{r) is defined for 0 < r < a. By symmetry, 
p(r) = p(—r ) when —a < r < 0, as shown in Fig. 37.16. 
When k becomes larger, the pressure distribution ap¬ 
proaches uniform distribution, as shown in Fig. 37.16. 
It is also required that the integral of the pressure over 
the contact area be equal to the normal force; that is 


Ijc a 

/" <r|d4= / / 

R 0=0 1— 0 


p(r) r dr dd = N . 


( 37 . 23 ) 


The coefficient C* can be obtained by substitut¬ 
ing (37.22) into (37.23). It is interesting to note that 
when (37.23) is integrated, both the normal force and 
radius of contact vanish, leaving only the constant C* 
as follows 


3 k £{|) 

2 r(l)r(t) 


( 37 . 24 ) 



Fig. 37.16 Illustration of pressure distribution with respect to the normalized radius, r/a. The plot shows an axisymmetric 
pressure distribution with k = 2,3,4,5, and 00 . As k increases, the pressure profile becomes more uniform 
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where k = 1, 2, 3,... are typically integer values (al¬ 
though noninteger A' values are also possible) and C() is 
the gamma function [37.105]. The numerical values of 
Ck for a few values of k are listed in Table 37.2 for refer¬ 
ence. A normalized pressure distribution with respect to 
the normalized radius, r/a, is plotted in Fig. 37.16. As 
can be seen from the figure, when k approaches infin¬ 
ity, the pressure profile will become that of a uniformly 
distributed load with magnitude of N/(n a 2 ), in which 
case Ck = 1.0. 

For linear elastic materials, k = 2 can be used, 
although it is found that k =1.8 is also appro- 


37.7 Friction Limit Surface 

Section 37.3 introduced the notion of a wrench cone, 
describing the set of wrenches that can be applied 
through a point contact with friction. Any contact 
wrench is limited to lie within the surface of the cone. 
This gives rise to the general notion of a friction limit 
surface - the surface bounding the set of wrenches that 
can be applied through a given contact or set of con¬ 
tacts. 

In this section we study the particular case of limit 
surfaces arising from planar contact patches [37.7, 107]. 
A planar contact patch occurs when a flat object slides 
on the floor, a soft robot finger presses against a face 
of a polyhedron, or a foot of a humanoid robot pushes 
on the ground. We would like to know what wrenches 
can be transmitted through such a contact. In the rest of 
this section, we focus on the properties of the limit sur¬ 
face of a planar contact patch with a specified pressure 
distribution over the patch. 

For ease of discussion, we will call one of the ob¬ 
jects in contact the part (e.g., the flat object on the floor 
or the robot finger) and the other object a stationary sup¬ 
port. We define a coordinate frame so that the planar 
contact patch is in the z = 0 plane, and let p(r) > 0 be 
the contact pressure distribution between the part and 
support as a function of the location r = (x , y) T . The 
friction coefficient at the contact patch is p.. If the planar 
velocity of the part is t = (co z , v x , ty) T , then the linear 
velocity at r is 

v(r) = (v x - co z y, v\ + a> z x) T , 

and the unit velocity is v(r) = v(r)/\\v(r)\\. The in¬ 
finitesimal force applied by the part to the support at r, 
in the plane of sliding, is 

d fif) = [d f x (r), d/ v (r)] T = jip(r)v(r) . (37.25) 


Table 37.2 Values of the coefficient Ck in the pressure dis¬ 
tribution (37.22) 


k 

Coefficient C* 

k = 2 (circular) 

C 2 = 1.5 

k = 3 (cubic) 

C 3 = 1.24 

k = 4 (quadruple) 

C 4 = 1.1441 

k —> 00 (uniform) 

Coo = 1.0 


priate in some cases [37.106]. For nonlinear elas¬ 
tic and viscoelastic materials, the value of k tends 
to be higher, depending on the properties of the 
material. 


The total wrench the part applies to the support is 



where A is the support area. 

As expected, the wrench is independent of the speed 
of motion: for a given to and all a > 0, the wrench re¬ 
sulting from velocities ato is identical. Viewing (37.26) 
as a mapping from twists to wrenches, we can map the 
surface of the unit twist sphere (||f|| = 1) to a surface 
in the wrench space. This surface is the limit surface, 
and it is closed, convex, and encloses the origin of the 
wrench space (Fig. 37.17). The portion of the wrench 



Fig. 37.17 Contact and coordinates for COR and local in¬ 
finitesimal area dA for numerical integration to construct 
the limit surface of soft fingers 
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space enclosed by this surface is exactly the set of 
wrenches the part can transmit to the support. When the 
part slips on the support (t 0), the contact wrench w 
lies on the limit surface, and by the maximum-work in¬ 
equality, the twist t is normal to the limit surface at w. 
If the pressure distribution p(r) is finite everywhere, 
the limit surface is smooth and strictly convex, and the 
mapping from unit twists to unit wrenches, and vice 
versa, is continuous and one-to-one. The limit surface 
also satisfies the property w(—t) = —w(t). 

37.7.1 The Friction Limit Surface at a Soft 
Contact Interface 

Given a pressure distribution in (37.22) we can numer¬ 
ically construct the corresponding friction limit surface 
by using (37.25) and (37.26). For an infinitesimal con¬ 
tact area, the contact resembles a point contact. Hence, 
the Coulomb’s law of friction can be employed. 

In the following, we will exploit the kinematic 
property of center of rotation (COR) with modeling 
symmetry to formulate equations for the magnitudes of 
friction force and moment on a circular contact patch. 
Figure 37.17 shows a circular contact patch with an in¬ 
stantaneous COR. By moving (or scanning) the COR 
along the Jt-axis, different possible combinations of fric¬ 
tion force and moment can be obtained to construct the 
limit surface. More details can be found in [37.35-38, 
41,41,42, 107-109]. 

The following derivation is for the total friction 
force (f t ) and moment (m z ) on the contact interface for 
the COR at a distance d c along the jc-axis, as shown in 
Fig. 37.17. By varying the COR distance d c from —00 
to 00 , all possible combinations of (f , m z ) can be found 
in order to construct the entire friction limit surface. 

The tangential force over the entire contact area 
can be obtained by integrating shear force on each in¬ 
finitesimal areas, dA, on which the Coulomb’s law of 
friction is observed, over the entire contact area, A. 
When Hertzian contact pressure distribution [37.70] is 
considered, use k = 2 in (37.22). Setting r = {x ,y) T 
and r = ||r||. the total tangential force can be integrated 
using (37.25) to obtain 

ft = (f) = -[ fJL dA • (37 - 27) 

A 

where A denotes the circular contact region in 
Fig. 37.17, / t is the tangential force vector with the 
direction shown in Fig. 37.17, ft, is the coefficient of 
friction, v(r) is the unit vector in the direction of the 
velocity vector v(r) with respect to the COR on the 
infinitesimal area dA at the location r, and p(r) is the 
pressure distribution at distance r from the center of 


contact. Since the pressure along the annular ring with 
distance r from the center is the same, we denote it by 
p(r) instead of p(r). The minus sign denotes the oppo¬ 
site directions of v(r) and / t . Since we are primarily 
interested in the magnitude of the friction force and 
moment, we shall omit this sign in the later derivation 
when magnitudes are concerned. 

Similarly, the moment about the z-axis, or the nor¬ 
mal to the contact area, is 



(37.28) 


where ||r x {)(r)|| is the magnitude of the cross product 
of the vectors r and v(r), whose direction is normal to 
the contact surface. 

The unit vector v(r) is related to the distance d c 
from the origin to the COR chosen on the x-axis from 
Fig. 37.17, and can be written as follows 


v(r) 


1 

y/(x-d c ) 2 +y 2 

1 


-y 

(x - d c ) 


y/(rcos8 —d c ) 2 + (rsin$) 2 


—r sin 6 
(rcos 6 — d c ) 


(37.29) 


Due to symmetry, f x = 0 for all CORs along the x- 
axis; therefore, the magnitude of the tangential force in 
the contact tangent plane is/ t =f y . Substituting equa¬ 
tions (37.22) and (37.29) into (37.27) and (37.28), we 
obtain 


ft = / ^ 


/' 


(r cos 9 — d c ) 
y/r 2 + d 2 — 2rd c cos 6 
1 

N 


C k 


*-(;) 


dA . 


(37.30) 


Similarly, the moment about the axis normal to the 
plane is 


/' 


r 2 — rd c cos ( 


m z = I fi 
A 

X 


yjr 2 + d 2 — 2 rd c cos 9 
1 

N 


C k 


■-© 


dA. 


(37.31) 


In (37.30) and (37.31), polar coordinates are defined 
such that x = r cos 9, y = r sin 9, and dA = rdrdd. 
We also introduce a normalized coordinate 
r 


a 


(37.32) 
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From (37.32), we can write dr = a dr. We also de¬ 
note d c = d c /a, and assume that 41 is constant through¬ 
out the contact area. Substituting the normalized co¬ 
ordinate into equation (37.30) and dividing both sides 
by fiN, we can derive 

/t _c k r f ( r 2 cos 9 — rdf) 

71 0 0 Jr 2 + ~ cos 9 

x(l-r*)idrd0 . (37.33) 

Substituting again r = r/a and dr = adr into (37.31) 
and normalizing with a/iN, we obtain 

m z _ C k f f (r 3 cos 9 - r 2 ~df) 
a ^N 71 I l Jr 2 d-'d 2 — 2rd c cos 9 

x (1 — drdO . (37.34) 

Equations (37.33) and (37.34) can be numerically 
integrated for a distance d c or d c to yield a point on 
the limit surface for a prescribed pressure distribution 
p(r) given by (37.22). Both equations involve elliptic 
integrals whose closed-form solutions may not exist but 
can be evaluated numerically. When the COR distance 
d c varies from —00 to 00 , all possible combinations of 
(ft, m 7 ) can be obtained for plotting the friction limit 
surface. 


m z 

Normalized moment,- tt 

a fiN 



Fig. 37.18 Example limit surface obtained by numerical integration 
and elliptical approximation. The numerical integration is based on 
the pressure distribution, p(r), and the coefficient, C k . In this figure, 
the pressure distributions in (37.22) of both k = 2 and k = 4 are 
used 


37.7.2 Example of Constructing a Friction 
Limit Surface 


When the pressure distribution is fourth order 
with k = 4, the coefficient from Table 37.2 is 
C 4 = (6/v^r)(r(3/4)/r(l/4)) = 1.1441. Equations 
(37.33) and (37.34) can be written as 


2n 1 


— = 0.3642 
jxN 


// 

0 0 


ajiN 


In 1 

03642 / / 
0 0 


(r 2 cos 9 — rdf) 
■Jr 2 + S 2 — 2 rd c cos 9 
x (1 — r 4 )? drd9 . 

(r 3 cos 9 — r 2 d c ) 
Jr 2 d-d 2 - 2 rd c cos 9 

x (1 — r 4 )* drd9 . 


Numerical integration for different values of d c 
yields pairs of (f[/ (jiN), m z /(afiN)). Plots of these 
pairs are shown in Fig. 37.18. 

A reasonable approximation to these numerical re¬ 
sults is given by the following equation of an ellipse 

m z 

Ou-hriax 

where the maximum moment (;nj) ma x is 



j = 1, (37.35) 


(mz) n 


f , N 

/r\ k ~ 

/ I 1 \ r \Ck —j 

1 - -) 

J Jta 2 

V a / 


dA . 


(37.36) 



Fig.37.19a,b Friction limit surface for soft fingers: 

(a) a 3-D ellipsoid representing the limit surface, and 

(b) a section of the ellipsoidal limit surface showing the 
coupled relationship between the force and moment, as 
those obtained in Fig. 37.18 
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obtained by (37.31) with the COR at d c = 0. This de¬ 
fines the quarter-elliptical curves in Fig. 37.18. This 
approximation is the basis for constructing a three- 
dimensional (3-D) ellipsoidal limit surface, as illus¬ 


trated in Fig. 37.19, and is a good model for the soft 
contact described in Sect. 37.6.1. More details can be 
found in [37.35-37, 39-42], 


37.8 Contacts in Grasping and Fixture Designs 


It is often important to relate the force and con¬ 
tact displacement (or deformation of contact inter¬ 
face) in grasping and fixture design in which de¬ 
formable contacts are concerned. Furthermore, such 
a force-displacement relationship is typically nonlin¬ 
ear due to the nature of contact. A linear expression 
such as Hooke’s law cannot capture the instantaneous 
and overall characteristics of force and displacement 
for these contacts. In this section, we formulate and 
discuss such a relationship using the elastic contact 
model. 

From Fig. 37.13 and the geometry of con¬ 
tact, (37.17), relating the contact radius with normal 
force, can be rewritten. Assuming d Rq, the follow¬ 
ing equation can be derived [37.39] 

N = c i d i , (37.37) 

where Cd is a proportional constant, and £ is 



Both Cd and £ can be obtained experimentally. The ex¬ 
ponent £ can also be obtained from y through (37.38) 

Stiffness k s (N/mm) Normal force N (N) 



Fig. 37.20 The typical load-depression curve in (37.37) 
shown with a scale to the right , and the contact stiffness, 
k s , as given by (37.39) with a scale to the left 


if y is already known for the fingertip. The range of 
the exponent in (37.37) is 3/2 < £ < 00 . In (37.37), 
the approach or the vertical depression of the finger¬ 
tip, d, is proportional to the normal force raised to 
the power of 2y (cf. (37.38)) which ranges from 0 
to 2/3. 

A plot of normal force versus displacement of con¬ 
tact is shown in Fig. 37.20 using (37.37) with a nonlin¬ 
ear power-law equation. Experimental data of contact 
between a soft finger and flat surface show consistent 
results when compared with Fig. 37.20 [37.39]. 

37.8.1 Contact Stiffness 
of Soft Fingers 

The nonlinear contact stiffness of a soft finger is defined 
as the ratio of the change in normal force with respect 
to the change in vertical depression at the contact. The 
contact stiffness of soft fingers can be obtained by dif¬ 
ferentiating (37.37) as follows 



Substitute (37.37) with d = ( ^ j into (37.39) to de¬ 
rive 

k s = cj C = cf f N 1 ~ 2y . (37.40) 

Thus, the nonlinear contact stiffness of soft fingers, de¬ 
rived in a succinct form in (37.39), is the product of the 
exponent / and the ratio of the normal force versus the 
approach, N/cl. A typical contact stiffness as a func¬ 
tion of vertical depression is plotted in Fig. 37.20. The 
stiffness shown in the figure increases with the force. 
The expressions for the stiffness, k s , as a function of 
various parameter(s) are summarized in the following 
Table 37.3. 

Table 37.3 Summary of equations of contact stiffness 



m 

f(N) f(N,d) 

k s 


4^ Hi) 
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Table 37.4 Summary of the contact mechanics equations 
for linear elastic (when y = 1/3 or / = 3/2) and nonlinear 
elastic soft fingers 


Description 

Equation for soft 
fingers 

Parameters 

Power law 

a = cN v 

o<y< 5 

Pressure distribution 

p(r) = 

*o)[i-(§)*]* 

Typically k > 1.8 

Contact approach 

N = Cd 

| < f < 00 

Contact stiffness 

II 

Nonlinear 


The change in stiffness can be obtained by differen¬ 
tiating (37.39) to derive the following equations 

^=c d Z(!-l)dt- 2 , (37.41) 

da 


Equation (37.39) suggests that the stiffness of soft con¬ 
tacts always increases because the ratio N/d always 
increases, with constant £ for prescribed fingertip ma¬ 
terial and range of normal force. This is consistent with 
the observation that the contact stiffness becomes larger 
(i. e., stiffer) with larger depression and force. Addition¬ 
ally, (37.42) suggests that the contact stiffness k s always 
increases with the normal force because dk s /dN> 0 
owing to £ > 1.5. Moreover, the change in stiffness with 
respect to the normal load is inversely proportional to 
the vertical depression, d, as derived in (37.42). This 
result asserts that the rate of increase in the contact stiff¬ 
ness will gradually become less and less as the normal 
load and vertical depression increase. 

A summary of the equations for soft fingers is pre¬ 
sented in Table 37.4. The range of the exponent of 


the general power-law equation in (37.17) is 0 < y < 
1/3. The Hertzian contact has an exponent of y = 1/3; 
therefore, the Hertzian contact theory for linear elas¬ 
tic materials is a special case of (37.17). Linear elastic 
materials, such as steel or other metallic fingertips, un¬ 
der small deformation, generally follow the Hertzian 
contact theory fairly well. The power-law equation 
in (37.17) should be employed for fingertips made of 
softer materials, such as soft rubber or silicone or even 
viscoelastic fingertips. 

37.8.2 Application 

of Soft Contact Theory 
to Fixture Design 

The preceding analysis and results can be applied 
in fixturing design and other applications that in¬ 
volve contact with finite areas [37.6]. In fixture de¬ 
sign with soft contacts (e.g., copper surfaces) un¬ 
der relatively large deformation and load, the power- 
law equations in Table 37.4 should be considered 
in place of the Hertzian contact equation. In these 
cases, the Hertzian contact model is no longer ac¬ 
curate and should be replaced. On the other hand, 
if linear elastic materials are used in fixture design 
with relatively small deformation (d/Ro < 5%), the ex¬ 
ponent should be taken as y = 1/3 when applying 
the contact theory. Furthermore, the exponent y was 
found to be material dependent and not geometry de¬ 
pendent in general [37.38]. Once the value of y is 
determined for the material (for example, using a ten¬ 
sile testing machine with the experimental procedure 
in [37.38]), it can be employed for the analysis of fix¬ 
turing design using the relevant equations presented 
herewith. 


37.9 Conclusions and Further Reading 


In this chapter, both rigid-body and elastic models 
of contact interfaces are discussed, including kine¬ 
matic constraints and the duality between the contact 
wrenches and twists. Contact forces that may arise 
with Coulomb friction are described. Multiple-contact 
manipulation tasks with rigid-body and Coulomb fric¬ 
tion are presented. Soft and viscoelastic contact inter¬ 
faces are presented and discussed. The friction limit 
surface is introduced and utilized to analyze pushing 
problems. An example of constructing a friction limit 
surface for soft contact is presented based on the for¬ 
mulation of force/moment of soft contact interface. 
Applications of these contact models in fixture anal¬ 
ysis and design are presented with the modeling of 


contact interface. Many references in the bibliography 
section of this chapter provide further reading on this 
subject. 

Manipulation modes include grasping, pushing, 
rolling, batting, throwing, catching, casting, and 
other kinds of quasistatic and dynamic manipula¬ 
tion [37.110]. This chapter presents an overview of 
contact modeling interfaces, with a particular focus on 
their use in manipulation tasks, including graspless or 
nonprehensile manipulation modes such as pushing. 
Analyses of grasps and fixtures are also presented. Ma¬ 
son’s textbook [37.14] expands on a number of themes 
in this chapter, including the theory of polyhedral con¬ 
vex cones, graphical methods for planar problems, and 
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applications to manipulation planning. Fundamental 
material on polyhedral cones can be found in [37.111] 
and their application to representations of twists and 
wrenches in [37.10, 112-1 14]. Twists and wrenches are 
elements of classical screw theory, which is covered in 


the texts [37.115, 116] and from the point of view of 
robotics in the texts [37.30, 117, 118]. 

In addition, several relevant chapters in this hand¬ 
book provide further reading for theory and applica¬ 
tions, for example, Chaps. 2 and 38. 


Video-References 


l-s&MiLliiili/l Pushing, sliding, and toppling 

available from http://handbookofrobotics.org/view-chapter/37/videodetails/802 
l-gslBTI'Jt'm Horizontal Transport by 2-DOF Vibration 

available from http://handbookofrobotics.org/view-chapter/37/videodetails/803 
Programmable Velocity Vector Fields by 6-DOF Vibration 

available from http://handbookofrobotics.org/view-chapter/37/videodetails/804 
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38. Grasping 


Domenico Prattichizzo, Jeffrey C. Trinkle 


This chapter introduces fundamental models of 
grasp analysis. The overall model is a coupling of 
models that define contact behavior with widely 
used models of rigid-body kinematics and dy¬ 
namics. The contact model essentially boils down 
to the selection of components of contact force 
and moment that are transmitted through each 
contact. Mathematical properties of the complete 
model naturally give rise to five primary grasp types 
whose physical interpretations provide insight for 
grasp and manipulation planning. 

After introducingthe basic models and types of 
grasps, this chapter focuses on the most important 
grasp characteristic: complete restraint. A grasp 
with complete restraint prevents loss of contact 
and thus is very secure. Two primary restraint prop¬ 
erties are form closure and force closure. A form 
closure grasp guarantees maintenance of contact 
as long as the links of the hand and the object 
are well-approximated as rigid and as long as the 
joint actuators are sufficiently strong. As will be 
seen, the primary difference between form clo¬ 
sure and force closure grasps is the latter's reliance 
on contact friction. This translates into requiring 
fewer contacts to achieve force closure than form 
closure. 

The goal of this chapter is to give a thorough 
understanding of the all-important grasp proper¬ 
ties of form and force closure. This will be done 
through detailed derivations of grasp models and 


Mechanical hands were developed to give robots the 
ability to grasp objects of varying geometric and 
physical properties. The first robotic hand designed 
for dexterous manipulation was the Salisbury Hand 
(Fig. 38.1) [38.2], It has three three-jointed fingers; 
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discussions of illustrative examples. For an in- 
depth historical perspective and a treasure-trove 
bibliography of papers addressing a wide range of 
topics in grasping, the reader is referred to [38.1], 


enough to control all six degrees of freedom of an 
object and the grip pressure. The fundamental grasp 
modeling and analysis done by Salisbury provides a ba¬ 
sis for grasp synthesis and dexterous manipulation 
research which continues today. Some of the most ma- 
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ture analysis techniques are embedded in the widely 
used software Grasplt! [38.3] and SynGrasp [38.4,5]. 
Grasplt! contains models for several robot hands and 
provides tools for grasp selection, dynamic grasp sim¬ 
ulation, and visualization. SynGrasp is a MATLAB 
Toolbox that can be obtained from [38.6] and pro¬ 
vides models and functions for grasp analysis with 
both fully and underactuated hands. It can be a useful 
educational tool to get aquainted with the mathemat¬ 


ical framework of robotic grasping described in this 
Chapter. Over the years since the Salisbury Hand was 
built, many articulated robot hands have been devel¬ 
oped. Nearly all of these have one actuator per joint or 
fewer. A notable exception is the DLR hand arm system 
developed by DLR (Deutsches Zentrum fur Luft- und 
Raumfahrt), which has two actuators per joint that drive 
each joint independently with two antagonistic tendons 
(Fig. 38.7) [38.7], 


38.1 Models and Definitions 


A mathematical model of grasping must be capable of 
predicting the behavior of the hand and object under 
various loading conditions that may arise during grasp¬ 
ing. Generally, the most desirable behavior is grasp 
maintenance in the face of unknown disturbing forces 
and moments applied to the object. Typically these 
disturbances arise from inertia forces which become 
appreciable during high-speed manipulation or applied 
forces such as those due to gravity. Grasp maintenance 
means that the contact forces applied by the hand are 
such that they prevent contact separation and unwanted 
contact sliding. The special class of grasps that can be 
maintained for every possible disturbing load is known 
as closure grasps. Figure 38.1 shows the Salisbury 
Hand [38.2, 8], executing a closure grasp of an object by 
wrapping its fingers around it and pressing it against its 
palm. Formal definitions, analysis, and computational 
tests for closure will be presented in Sect. 38.4. 

Figure 38.2 illustrates some of the main quantities 
that will be used to model grasping systems. Assume 
that the links of the hand and the object are rigid and 
that there is a unique, well-defined tangent plane at each 
contact point. Let {IV} represent a conveniently chosen 
inertial frame fixed in the workspace. The frame {B} 



Fig. 38.1 The Salisbury Hand grasping an object 


is fixed to the object with its origin defined relative 
to {N} by the vector p e R 3 , where R 3 denotes three- 
dimensional Euclidean space. A convenient choice for 
p is the center of mass of the object. The position of 
contact point i in {N} is defined by the vector c,- e R 3 . 
At contact point i, we define a frame {C},-, with axes 
0 ,} ({C}i is shown in exploded view). The unit 
vector ii, is normal to the contact tangent plane and 
directed toward the object. The other two unit vec¬ 
tors are orthogonal and lie in the tangent plane of the 
contact. 

Let the joints be numbered from 1 to n q . Denote 
by q = [<7i - ■ ■ <?h,] T e R' ! ‘ i the vector of joint displace¬ 
ments, where the superscript T indicates matrix trans¬ 
position. Also, let t = [ti • • • r„J T g R"« represent joint 
loads (forces in prismatic joints and torques in revolute 
joints). These loads can result from actuator actions, 
other applied forces, and inertia forces. They could 
also arise from interaction at the contacts between the 
object and hand. However, it will be convenient to sep- 
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arate joint loads into two components: those arising 
from contacts and those arising from all other sources. 
Throughout this Chapter, non-contact loads will be de¬ 
noted by r. 

Let u G R"“ denote the vector describing the posi¬ 
tion and orientation of {B} relative to {N}. For planar 
systems n u = 3. For spatial systems, n„ is three plus 
the number of parameters used to represent orienta¬ 
tion; typically three (for Euler angles) or four (for unit 
quaternions). Denote by v = [w T tu T ] T G R" v , the twist 
of the object described in {N}. It is composed of the 
translational velocity of the point p and the 

angular velocity u> G R 3 of the object, both expressed 
in {N}. A twist of a rigid body can be referred to any 
convenient frame fixed to the body. The components of 
the referred twist represent the velocity of the origin of 
the new frame and the angular velocity of the body, both 
expressed in the new frame (Table 38.1). For a rigorous 
treatment of twists and wrenches see [38.9, 10]. Note 
that for planar systems, v G R 2 , w G R, and so n v =3. 

Another important point is u 7^ v. Instead, these 
variables are related by the matrix V as 


ii = Vv , (38.1) 

where the matrix V G R"" x " y is not generally square, 
but nonetheless satisfies V T V = I [38.11], where I is 
the identity matrix, and the dot over the u implies dif¬ 
ferentiation with respect to time. Note that for planar 
systems, V = 1 g R 3x3 . 

Let / G R 3 be the force applied to the object at the 
point p and let m G R 3 be the applied moment. These 
are combined into the object load, or wrench, vector 
denoted by g = [/ T »i T ] T G R' !y , where / and m are ex¬ 
pressed in {N}. Like twists, wrenches can be referred to 
any convenient frame fixed to the body. One can think 
of this as translating the line of application of the force 
until it contains the origin of the new frame, then ad¬ 
justing the moment component of the wrench to offset 
the moment induced by moving the line of the force. 
Last, the force and adjusted moment are expressed in 
the new frame. As done with the joint loads, the object 
wrench will be partitioned into two main parts: contact 
and non-contact wrenches. Throughout this chapter, g 
will denote the non-contact wrench on the object. 

38.1.1 Velocity Kinematics 

The material in this chapter is valid for a wide range 
of robot hands and other grasping mechanisms. The 
hand is assumed to be composed of a palm that serves 
as the common base for any number of fingers, each 
with any number of joints. The formulations given in 
this chapter are expressed explicitly in terms of only 


Table 38.1 Primary notation for grasp analysis 


Notation 

Definition 

«C 

Number of contacts 

n q 

Number of joints of the hand 

n v 

Number of degrees of freedom of object 

n\ 

Number of contact wrench components 

q e R"« 

Joint displacements 

q e R"1 

Joint velocities 

T G R"? 

Non-contact joint loads 

u e R"« 

Position and orientation of object 

v e R" y 

Twist of object 

j€R" v 

Non-contact object wrench 

X e 

Transmitted contact wrenches 

v cc e R" A 

Transmitted contact twists 

{B} 

Frame fixed in object 

{Ch 

Frame at contact i 

m 

Inertial frame 


revolute and prismatic joints. However, most other com¬ 
mon joints can be modeled by combinations of revolute 
and prismatic joints (e.g., cylindrical and planar). Any 
number of contacts may occur between any link and the 
object. 

Grasp Matrix and Hand lacobian 
Two matrices are of the utmost importance in grasp 
analysis: the grasp matrix G and the hand Jacobian J. 
These matrices define the relevant velocity kinemat¬ 
ics and force transmission properties of the contacts. 
The following derivations of G and J will be done un¬ 
der the assumption that the system is three-dimensional 
(n v = 6). Changes for planar systems will be noted 
later. 

Each contact should be considered as two coinci¬ 
dent points; one on the hand and one on the object. The 
hand Jacobian maps the joint velocities to the twists of 
the hand to the contact frames, while the transpose of 
the grasp matrix maps the object twist to the contact 
frames. Finger joint motions induce a rigid body motion 
in each link of the hand. It is implicit in the terminology, 
twists of the hand, that the twist referred to contact i is 
the twist of the link involved in contact i. Thus these 
matrices can be derived from the transforms that change 
the reference frame of a twist. 

To derive the grasp matrix, let denote the angu¬ 
lar velocity of the object expressed in {N} and let tt^ ob j, 
also expressed in {N}, denote the velocity of the point 
on the object coincident with the origin of {C},. These 
velocities can be obtained from the object twist referred 
to {N} as 


(<obA 

UsJ 


p>, 


(38.2) 
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where 


P,= 


( 13X3 

\S(c, -p) 



(38.3) 


I 3 X 3 € R 3x3 is the identity matrix, and S(c,- —p) is the 
cross product matrix, that is, given a three-vector r = 
[r t r y r.] T , S (r) is defined as follows 

( 0 -r z r y \ 

S(r) = I r z 0 -r x I 

\~r y r x 0 / 

The object twist referred to {C}, is simply the vector 
on the left-hand side of (38.2) expressed in {C},. 

Let R, = (riitjdi) G R 3x3 represent the orientation 
of the ;-th contact frame {C},- with respect to the inertial 
frame (the unit vectors and 0 , are expressed in 

{N}). Then the object twist referred to {C},- is given as 

where R, = Blockdiag(R„ R,) = ( ® j e R 6x6 . 

Substituting P^v from (38.2) into (38.4) yields the 
partial grasp matrix G R 6x6 , that maps the object 
twist from {IV} to {C},- 

v/.obj = Gjv , (38.5) 

where 

GT = R*PT. (38 . 6) 


The hand Jacobian can be derived similarly. Let 
w fhnd t ^ le an gular velocity of the link of the hand 
touching the object at contact i, expressed in {TV}, and 
define vf hnd as the translational velocity of contact i 
on the hand, expressed in {A'}. These velocities are re¬ 
lated to the joint velocities through the matrix Z,- whose 
columns are the Pliicker coordinates of the axes of the 
joints [38.9, 10]. We have 


Kj 


Z iq , 


(38.7) 


where Z,- G R 6x,, 9 is dehned as 


Z,= 




(38.8) 


with the vectors d, j, G 1R 3 defined as 


0 3 xl 

% 

S(cj — $j) 7 Zj 


if contact force i does 
not affect joint j , 
if joint j is prismatic , 
if joint j is revolute , 


f0 3 xl 


®3xl 


if contact force i does 
not affect joint j , 
if joint j is prismatic , 
if joint j is revolute , 


where t, j is the origin of the coordinate frame associated 
with the 7 -thjoint andzj is the unit vector in the direction 
of the z-axis in the same frame, as shown in Fig. 38.12. 
Both vectors are expressed in {/V}. These frames may 
be assigned by any convenient method, for example, the 
Denavit-Hartenberg method [38.12]. The z ; -axis is the 
rotational axis for revolute joints and the direction of 
translation for prismatic joints. 

The final step in referring the hand twists to the con¬ 
tact frames is change the frame of expression of n^ hnd 

and w /?hnd to {Qi 

_ ^i, hnd\ too q\ 

Vhnd — R. I m I • (38.9) 

\ w ;h„d/ 

Combining (38.9) and (38.7) yields the partial hand 
Jacobian J, G R 6xn «, which relates the joint velocities 
to the contact twists on the hand 


L.hnd — J/3/ 
where 

J, = RZ,. 


(38.10) 


(38.11) 


To simplify notation, stack all the twists on the hand 


and object into the vectors v c ,hnd G 
R 6, ‘ c as follows 


t> 6 n c 


and v 


c.obj 


v c,£ = 


( v T.t ••• v l,t) • f = {obj,hnd}. 


Now the complete grasp matrix G G R 6x6,! >: and com¬ 
plete hand Jacobian J G R 6,! ‘: Xn <( relate the various ve¬ 
locity quantities as follows 


= G t v . 


W.obj 
t^c.hnd — ,L/ 


where 





(GJ\ 


/ Jl \ 

G t = 


, J = 





<)nj 


(38.12) 

(38.13) 


(38.14) 
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The term complete is used to emphasize that all 6n c 
twist components at the contacts are included in the 
mapping. See Example 1, Part 1 and Example 3, Part 1 
at the end of this chapter for clarification. 

Contact Modeling 

Contacts play a central role in grasping. Contacts al¬ 
lows to impose a given motion to the object or to apply 
a given force through the object. All grasping actions 
go through contacts whose model and control is cru¬ 
cial in grasping. Three contact models useful for grasp 
analysis are reviewed here. For a complete discussion 
of contact modeling in robotics, readers are referred to 
Chap. 37. 

The three models of greatest interest in grasp analy¬ 
sis are known as point-contact-without-friction, hard- 
finger, and soft-finger [38.13]. These models select 
components of the contact twists to transmit between 
the hand and the object. This is done by equating a sub¬ 
set of the components of the hand and object twist 
at each contact. The corresponding components of the 
contact force and moment are also equated, but without 
regard for the constraints imposed by a friction model 
(Sect. 38.4.2). 

The point-contact-without-friction (PwoF) model is 
used when the contact patch is very small and the sur¬ 
faces of the hand and object are slippery. With this 
model, only the normal component of the translational 
velocity of the contact point on the hand (i. e., the 
first component of v, hnd) is transmitted to the object. 
The two components of tangential velocity and the 
three components of angular velocity are not transmit¬ 
ted. Analogously, the normal component of the contact 
force is transmitted, but the frictional forces and mo¬ 
ments are assumed to be negligible. 

A hard finger (HF) model is used when there is 
significant contact friction, but the contact patch is 
small, so that no appreciable friction moment exists. 
When this model is applied to a contact, all three trans¬ 
lational velocity components of the contact point on 
the hand (i. e., the first three components of v, hnd) 
and all three components of the contact force are 
transmitted through the contact. None of the angu¬ 
lar velocity components or moment components are 
transmitted. 

The soft finger (SF) model is used in situations in 
which the surface friction and the contact patch are 
large enough to generate significant friction forces and 
a friction moment about the contact normal. At a con¬ 
tact where this model is enforced, the three translational 
velocity components of the contact on the hand and 
the angular velocity component about the contact nor¬ 
mal are transmitted (i. e., the first four components of 
Whnd)- Similarly, all three components of contact force 


and the normal component of the contact moment are 
transmitted. 


Remark 38.1 

The reader may see a contradiction between the rigid- 
body assumption and the soft-finger model. The rigid- 
body assumption is an approximation that simplifies all 
aspects of the analysis of grasping, but nonetheless is 
sufficiently accurate in many real situations. Without it, 
grasp analysis would be impractical. On the other hand, 
the need for a soft-finger model is a clear admission that 
the finger links and object are not rigid. However, it can 
be usefully applied in situations in which the amount 
of deformation required to obtain a large contact patch 
is small. Such situations occur when the local surface 
geometries are similar. If large finger or body deforma¬ 
tions exist in the real system, the rigid-body approach 
presented in this chapter should be used with caution. 


To develop the PwoF, HF, and SF models, define 
the relative twist at contact i as follows 

(J; -(I, 1 ) ^ j = v,.hml - V/.ohj . 

A particular contact model is defined through the ma¬ 
trix H, e R' u,x6 , which selects nx, components of the 
relative contact twist and sets them to zero 

H/( tYhmi A, 0 bj) = 0 . 

These components are referred to as transmitted de¬ 
grees of freedom (DOF). Define H, as follows 


H, f 

0 

0 

H/m 


where H,p and H,m are the translational and rotational 
component selection matrices. Table 38.2 gives the def¬ 
initions of the selection matrices for the three contact 
models, where vacuous for H,m means that the corre¬ 
sponding block row matrix in (38.15) is void (i. e., it has 
zero rows and columns). Notice that for the SF model, 
H,m selects rotation about the contact normal. 

After choosing a transmission model for each con¬ 
tact, the kinematic contact constraint equations for all 
n c contacts can be written in compact form as 

H(v c? hnd W.obj) — 0 > (38.16) 


Table 38.2 Selection matrices for three contact models 


Model 

n Xi 

H,f 

H;m 

PwoF 

1 

(100) 

Vacuous 

HF 

3 

Ex3 

Vacuous 

SF 

4 

Ex3 

(10 0) 
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where 

H = Blockdiag(H,,..., H„ c ) G R" aX6 " c , 

and the number of twist components nx transmitted 
through the n c contacts is given by nx = Yl'i=i W A;- 
Finally, by substituting (38.12) and (38.13) into 
(38.16) one gets the compact form of the velocity kine¬ 
matic contact constraints 

(J -G T )(j)=0, (38.17) 

where the grasp matrix and hand Jacobian are finally 
defined as 


G t = HG t g R" aX6 , 

J = HJ G r AX "« . (38.18) 

For more details on the construction of H, the grasp 
matrix, and the hand Jacobian, readers are referred 
to [38.14—16] and the references therein. Also, see Ex¬ 
ample 1, Part 2 and Example 3, Part 2. 

It is worth noting that (38.17) can be written in the 
following form 

J<7 = t*cc,hnd = ^cc.obj = G V , (38.19) 

where v cc ,hnd and v CCj0 bj contain only the components of 
the twists that are transmitted by the contacts. To under¬ 
line the central role of contact constraints in grasping, 
it is worth noting that grasp maintenance is defined 
as the situation in which constraints (38.19) are main¬ 
tained over time. The kinematic contact constraint holds 
only if the contact force satisfies the friction constraints 
for the contact models with friction or the unilat¬ 
eral constraint for the contact model without friction 
(Sect. 38.4.2). 

Thus, when a contact is frictionless, contact mainte¬ 
nance implies continued contact, but sliding is allowed. 
However, when a contact is of the type HF, con¬ 
tact maintenance implies sticking contact, since sliding 
would violate the HF model. Similarly, for a SF con¬ 
tact, there may be no sliding or relative rotation about 
the contact normal. 

For the remainder of this chapter, it will be assumed 
that v cc hnd = fcc.obj. so the notation will be shortened 


Planar Simplifications 

Assume that the plane of motion is the (x,y)-plane 
of {N}. The vectors v and g reduce in dimension 


from six to three by dropping components three, four, 
and five. The dimensions of vectors c, and p re¬ 
duce from three to two. The ;-th rotation matrix be¬ 
comes R, = [hi ti\ G R 2x2 (where the third component 
of hi and f, is dropped) and (38.4) holds with R, = 
Blockdiag(R,, 1) G R 3x3 . Equation (38.2) holds with 


P,= 


( 12X2 

S 2 (Ci-p) 


0 


where S 2 is the analog of the cross product matrix for 
two-dimensional vectors, given as 


S2OO = (~r y r x ) . 


Equation (38.7) holds with dij G R 2 and tc,. 7 - g M de¬ 
fined as 


02 x 1 



S(c f 


</) T 


if contact force i does 
not affect the joint j , 
if joint j is prismatic , 
if joint j is revolute , 


0 

0 

1 


if contact force i does 
not affect joint j , 
if joint j is prismatic , 
if joint j is revolute . 


The complete ^rasp matrix and hand Jacobian have re¬ 
duced sizes: G T gR 3 " cX3 and j gR 3 "'^. As far as 
contact constraint is concerned, (38.15) holds with H,p 
and H,m defined in Table 38.3. 

In the planar case, the SF and HF models are equiv¬ 
alent, because the object and the hand lie in a plane. 
Rotations about the contact normals would cause out- 
of-plane motions. Finally, the dimensions of the grasp 
matrix and hand Jacobian are reduced to the follow¬ 
ing sizes: G T G R' uX3 and J G R" AXn «. See Example 1, 
Part 3 and Example 2, Part 1. 

38.1.2 Dynamics and Equilibrium 

Dynamic equations of the system can be written as 
Mhnd(<7)(? + &hnd(*7' T J ^ — ^app 

M obj (t<)v+*obj(M. V)-GA =gapp 

subject to constraint (38.17) , (38.20) 


Table 38.3 Definitions 


Model 

nxi H,f H iM 

PwoF 

1 (10) Vacuous 

HF/SF 

2 12x2 Vacuous 
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Table 38.4 Vectors of contact force and moment compo¬ 
nents, also known as the wrench intensity vector, transmit¬ 
ted through contact i 


Model 

A,- 

PwoF 

(fin) 

HF 

(fn/it/io) T 

SF 

(fmfafvo m; n ) T 


where Mhnd(') and M 0 bj(-) are symmetric, positive def¬ 
inite inertia matrices, Z>hnd(‘.‘) and ft 0 bj(‘.‘) are the 
velocity-product terms, g app is the force and moment 
applied to the object by gravity and other external 
sources, r app is the vector of external loads and ac¬ 
tuator actions, and the vector GA is the total wrench 
applied to the object by the hand. The vector A contains 
the contact force and moment components transmit¬ 
ted through the contacts and expressed in the contact 
frames. Specifically, A = [A| ■ ■ • A JJ T , where 

A i — H i\fmfitfio Mj n Mil Mio\ 

The subscripts indicate one normal (n) and two tangen¬ 
tial (t,o) components of contact force / and moment m. 
For a SF, HF, or PwoF contact. A, is defined as in Ta¬ 
ble 38.4. Finally, it is worth noting that G,A, = G/HJA, 
is the wrench applied through contact i, where G, and 
H, are defined in (38.6) and (38.15). The vector A, is 
known as the wrench intensity vector for contact i. 

Equation (38.20) represents the dynamics of the 
hand and object without regard for the kinematic con¬ 
straints imposed by the contact models. Enforcing 
them, the dynamic model of the system can be written 
as follows 

( J g)H*) 138 211 


subject to .Jr/ = G 1 v = v cc , where 

r = r app —M hnd(q)q-b hnd (q.q) 

g=gapp-M obi (u)v-b obi (u, v) . (38.22) 

One should notice that the dynamic equations 
are closely related to the velocity kinematic model 
in (38.17). Specifically, just as J and G T transmit only 
selected components of contact twists, J T and G in 
(38.20) serve to transmit only the corresponding com¬ 
ponents of the contact wrenches. 

When the inertia terms are negligible, as occurs dur¬ 
ing slow motion, the system is said to be quasi-static. In 
this case, (38.22) becomes 

T — 1 app , 

g = ga PP . (38.23) 

and does not depend on joint and object velocities. Con¬ 
sequently, when the grasp is in static equilibrium or 
moves quasi-statically, one can solve the first equation 
and the constraint in (38.21) independently to compute 
A, q , and v. It is worth noting that such a force/velocity 
decoupled solution is not possible when dynamic ef¬ 
fects are appreciable, since the first equation in (38.21) 
depends on the third one through (38.22). 


Remark 38.2 

Equation (38.21) highlights an important alternative 
view of the grasp matrix and the hand Jacobian. G can 
be thought of as a mapping from the transmitted con¬ 
tact forces and moments to the set of wrenches that the 
hand can apply to the object, while J T can be thought 
of as a mapping from the transmitted contact forces and 
moments to the vector of joint loads. Notice that these 
interpretations hold for both dynamic and quasi-static 
conditions. 


38.2 Controllable Twists and Wrenches 


In hand design and in grasp and manipulation planning, 
it is important to know the set of twists that can be im¬ 
parted to the object by movements of the fingers, and 
conversely, the conditions under which the hand can 
prevent all possible motions of the object. The dual 
view is that one needs to know the set of wrenches that 
the hand can apply to the object and under what con¬ 
ditions any wrench in R 6 can be applied through the 
contacts. This knowledge will be gained by studying the 
various subspaces associated with G and J [38.17]. 

The spaces, shown in Fig. 38.3, are the column 
spaces and null spaces of G, G T , J, and J T . Column 


space (also known as range) and null space will be de¬ 
noted by !R(-) and W(-), respectively. The arrows show 
the propagation of the various velocity and load quan¬ 
tities through the grasping system. For example, in the 
left part of Fig. 38.3 it is shown how any vector q e R' ! « 
can be decomposed into a sum of two orthogonal vec¬ 
tors in 'i-UJ 1 ) and in .'Af(J) and how q is mapped to 
21 (,J) by multiplication by J. 

It is important to recall two facts from linear al¬ 
gebra. First, a matrix A maps vectors from 2£(A T ) to 
21(A) in a one-to-one and onto fashion, that is, the 
map A is a bijection. The generalized inverse A+ of 
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Joint velocities Contact twists Object twists 

q g v cc G R' u v G 



Fig. 38.3 Linear maps relating twists 
and wrenches of a grasping system 


A is a bijection that maps vectors in the opposite direc¬ 
tion [38.18]. Also, A maps all vectors in TV (A) to zero. 
Finally, there is no non-trivial vector that A can map 
into J7V(A t ). This implies that if JAf(G T ) is non-trivial, 
then the hand will not be able to control all degrees of 
freedom of the object’s motion. This is certainly true for 
quasi-static grasping, but when dynamics are important, 
they may cause the object to move along the directions 
in G t ). 

38.2.1 Grasp Classifications 


The four null spaces motivate a basic classification of 
grasping systems defined in Table 38.5. Assuming so¬ 
lutions to (38.21) exist, the following force and velocity 
equations provide insight into the physical meaning of 

the various null spaces 


q = J + f cc + N(j)>', 

(38.24) 

v = (G T )+v cc +N(G T )3/, 

(38.25) 

X =-G+g + N(G)y , 

(38.26) 

X =(J T )+r+N(J T )y . 

(38.27) 


In these equations, A+ denotes the generalized inverse, 
henceforth pseudoinverse, of a matrix A, N(A) denotes 


Table 38.5 Basic grasp classes 


Condition 

Class 

Many-to-one 

A7(J)^0 

Redundant 

<7 ^ **cc 

T —> A 

tN(G T ) ± 0 

Indeterminate 

v -► Vcc 

g x 

JV(G) + 0 

Graspable 

X -+g 

Vcc -► V 

■W(J T ) + 0 

Defective 

A — ► T 

V C c Q 


a matrix whose columns form a basis for IN'(A), and y 
is an arbitrary vector of appropriate dimension that pa¬ 
rameterizes the solution sets. If not otherwise specified, 
the context will make clear if the generalized inverse is 
left or right. 

If the null spaces represented in the equations are 
non-trivial, then one immediately sees the first many- 
to-one mapping in Table 38.5. To see the other many- 
to-one mappings, and in particular the defective class, 
consider (38.24). It can be rewritten with v cc decom¬ 
posed into orthogonal components v rs and i>i ns in 3£(J) 
and 7L(J T ), respectively 

q = J + (v rs + vin S )-|-N(J)y . (38.28) 

Recall that every vector in 2AT (A T ) is orthogonal to ev¬ 
ery row of A+. Therefore J"*" iq,,. = 0. If y and v rs are 
fixed in (38.28), then q is unique. Thus it is clear that 
if JAf (J T ) is non-trivial, then a subspace of twists of the 
hand at the contacts will map to a single joint velocity 
vector. Applying the same approach to the other three 
equations (38.25)—(38.27) yields the other many-to-one 
mappings listed in Table 38.5. 

Equations (38.21) and (38.24)-(38.27), motivate the 
following definitions. 


Definition 38.1 Redundant 

A grasping system is said to be redundant if 1N(J) is 
non-trivial. 


Joint velocities q in 'N (J) are referred to as internal 
hand velocities, since they correspond to finger mo¬ 
tions, but do not generate motion of the hand in the 
constrained directions at the contact points. If the quasi¬ 
static model applies, it can be shown that these motions 
are not influenced by the motion of the object and vice 


versa. 
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Definition 38.2 Indeterminate 

A grasping system is said to be indeterminate if lAf (G T ) 
is non-trivial. 


Object twists v in 5N"(G T ) are called internal object 
twists, since they correspond to motions of the object, 
but do not cause motion of the object in the constrained 
directions at the contacts. If the static model applies, it 
can be shown that these twists cannot be controlled by 
finger motions. 


Definition 38.3 Graspable 

A grasping system is said to be graspable if JsT (G) is 
non-trivial. 


Wrench intensities A in Jsf (G) are referred to as inter¬ 
nal object forces. These wrenches are internal because 
they do not contribute to the acceleration of the object, 
i. e., GA = 0. Instead, these wrench intensities affect 
the tightness of the grasp. Thus, internal wrench inten¬ 
sities play a fundamental role in maintaining grasps that 
rely on friction (Sect. 38.4.2). 


Definition 38.k Defective 

A grasping system is said to be defective if 2AT(J T ) is 
non-trivial. 


Wrench intensities A in JAf(J T ) are called internal 
hand forces. These forces do not influence the hand 
joint dynamics given in (38.20). If the static model 
is considered, it can be easily shown that wrench in¬ 
tensities belonging to 2AT(J T ) cannot be generated by 
joint actions, but can be resisted by the structure of the 
hand. 

See Example 1, Part 4, Example 2, Part 2 and Ex¬ 
ample 3, Part 3. 

38.2.2 Limitations of Rigid-Body 
Formulation 

The rigid-body dynamics equation (38.20) can be 
rewritten with Lagrange multipliers associated with the 
contact constraints as 

( q\ /r-b hnd \ 

v J = I v - * ob j I , (38.29) 

where b c = [d(iq)/dq]q — [3(Gv)/3h]m and 

/M hnd 0 J T \ 

M dyn = 0 M obj -G . 

V J -G T 0 / 


In order for this equation to completely determine the 
motion of the system, it is necessary that matrix M dyn be 
invertible. This case is considered in detail in [38.19], 
where the dynamics of multi-finger manipulation is 
studied under the hypothesis that the hand Jacobian is 
full row rank, i. e., JAf (J T ) = 0. For all manipulation 
systems with non-invertible M dyn , rigid-body dynamics 
fails to determine the motion and the wrench intensity 
vector. By observing that 

N (M dyn ) 

= {(q. v,X) T \'q = 0,v =0,A e JAf (J T ) n JAf(G)} , 

the same arguments apply under quasi-static condi¬ 
tions defined by (38.21) and (38.23). When W(J T ) fl 

G) f= 0, the rigid-body approach fails to solve the 
first equation in (38.21), thus leaving A indeterminate. 


Definition 38.5 Hyperstatic 

A grasping system is said to be hyperstatic if 

JAf (J T ) n JAf (G) 
is non-trivial. 


In such systems there are internal forces (Defini¬ 
tion 38.3) belonging to tN~( J T ) that are not controllable 
as discussed for defective grasps. Rigid-body dynam¬ 
ics is not satisfactory for hyperstatic grasps, since the 
rigid-body assumption leads to undetermined contact 
wrenches [38.20]. 

See Example 3, Part 3. 

38.2.3 Desirable Properties 

For a general purpose grasping system, there are three 
main desirable properties: control of the object twist v, 
control of object wrench g. and control of the internal 
forces. Control of these quantities implies that the hand 
can deliver the desired v and g with specified grip pres¬ 
sure by the appropriate choice of joint velocities and 
actions. The conditions on J and G equivalent to these 
properties are given in Table 38.6. 

We derive the conditions in two steps. First, we 
ignore the structure and configuration of the hand (cap¬ 
tured in J) by assuming that each contact point on each 


Table 38.6 Desirable properties of a grasp 


Task requirement 

Required conditions 

All wrenches possible, g 
All twists possible, v 

! 

rank(G) = n v 

Control all wrenches, g 1 
Control all twists, v ( 


rank(GJ) = rank(G) = n v 

Control all internal forces 

w(G) n w(j T ) = o 
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Fig. 38.4 The Salisbury Hand 


finger can be commanded to move in any direction 
transmitted by the chosen contact model. An important 
perspective here is that v cc is seen as the independent 
input variable and v is seen as the output. The dual 
interpretation is that the actuators can generate any con¬ 
tact force and moment in the constrained directions at 
each contact. Similarly, A is seen as the input and g is 
seen as the output. The primary property of interest un¬ 
der this assumption is whether or not the arrangement 
and types of contacts on the object (captured in G) are 
such that a sufficiently dexterous hand could control its 
fingers so as to impart any twist v e M 6 to the object 
and, similarly, to apply any wrench g e R 6 to the object. 

All Object Twists Possible 

Given a set of contact locations and types, by solv¬ 
ing (38.19) for v or observing the map G on the right 
side of Fig. 38.3, one sees that the achievable object 
twists are those in 3t(G). Those in .ATG 1 ) cannot be 
achieved by any hand using the given grasp. Therefore, 
to achieve any object twist, we must have: TAT(G t ) = 0, 
or equivalently, rank(G) = n v . Any grasp with three 
non-collinear hard contacts or two distinct soft contacts 
satisfies this condition. 

All Object Wrenches Possible 
This case is the dual of the previous case, so we ex¬ 
pect the same condition. From (38.21), one immediately 
obtains the condition JsT (G T ) = 0, so again we have 
rank(G) = n v . 

To obtain the conditions needed to control the var¬ 
ious quantities of interest, the structure of the hand 


cannot be ignored. Recall that the only achievable con¬ 
tact twists on the hand are in 2£(J), which is not 
necessarily equal to R' u . 

Control All Object Twists 

By solving (38.17) for v, one sees that in order to cause 
any object twist v by choice of joint velocities q. we 
must have 3t(GJ) = R" v and H (G T ) = 0. These con¬ 
ditions are equivalent to rank(GJ) = rank(G) = n v . 

Control All Object Wrenches 
This property is dual to the previous one. Analysis of 
(38.21) yields the same conditions: 

rank(GJ) = rank(G) = n v . 

Control All Internal Forces 
Equation (38.20) shows that wrench intensities with no 
effect on object motion are those in JsT (G). In general, 
not all the internal forces may be actively controlled by 
joint actions. In [38.16,21] it has been shown that all 
internal forces in JsT (G) are controllable if and only if 
2AT(G) n N (J T ) = 0. 

See Example 1, Part 5 and Example 2, Part 3. 

Design Considerations 

of the Salisbury Hand 

The Salisbury Hand in Fig. 38.4 was designed to have 
the smallest number of joints that would meet all the 
task requirements in Table 38.6. Assuming HF contacts, 
three non-collinear contacts is the minimum number 
such that rank(G) = n v = 6. In this case, G has six 
rows and nine columns and the dimension of JsT (G) is 
three [38.2, 8]. The ability to control all internal forces 
and apply an arbitrary wrench to the object requires 
that ^Af(G) fl 2AT(J T ) = 0, so the minimum dimension 
of the column space of J is nine. To achieve this, the 
hand must have at least nine joints, which Salisbury 
implemented as three fingers, each with three revolute 
joints. 

The intended way to execute a dexterous manipula¬ 
tion task with the Salisbury Hand is to grasp the object 
at three non-collinear points with the fingertips, forming 
a grasp triangle. To secure the grasp, the internal forces 
are controlled so that the contact points are maintained 
without sliding. Dexterous manipulation can be thought 
of as moving the fingertips to control the positions of 
the vertices of the grasp triangle. 
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38.3 Compliant Grasps 


In this section we extend the rigid-body model to in¬ 
clude compliance. This is needed to design controllers 
that can implement desired compliance behaviors of 
a grasped object when it contacts the environment, 
which can increase the robustness of static grasps and 
dexterous manipulation tasks. It also facilitates the 
analysis of hands designed with flexible mechanical 
elements [38.22,23] and grasp control strategies that 
exploit grasp synergies [38.24,25]. Thanks to compli¬ 
ance, hands can be designed to maintain a secure grasp 
with fewer joints, which provides greater mechani¬ 
cal robustness and reduces the complexity of planning 
grasps. 

On the other hand, reducing the number of DOFs 
in robotic hands demands a compliant design of the 
whole structure to adapt the shape of the hand to dif¬ 
ferent objects and to improve robustness with respect 
to uncertainties [38.26]. Compliance can be passive or 
active. Passive compliance is due to the structural de¬ 
formation of robot components, including joints, while 
active compliance refers to virtual elasticity of actua¬ 
tors, e.g., due to the proportional action of a PD joint 
controller, that can be actively set by changing the con¬ 
trol parameters [38.26-29]. 

In the following, we extend the grasp analysis re¬ 
laxing the rigid-body contact constraints to take into 
account both the compliance and the low number of 
DOFs in robotic hands. 

If the hand structure is not perfectly stiff, as shown 
in Fig. 38.5, the actual vector of joint variables q can 
be different from the reference one q r , given to the joint 
actuator controllers, and their difference is related to the 
joint effort r through a compliance matrix C q e 
by the constitutive equation 

q t -q = C q T. (38.30) 

Note that if the hand structure is perfectly rigid, C q = 0 
and the hand stiffness K (; = is not defined. 

It will be clear in a moment that to deal with 
a low number of DOFs of the hand, a compliant 
model of the contact must be considered. Accord¬ 
ing to Definition 38.4, in case of a low number of 
DOFs, it is very likely that the grasp will be defec¬ 
tive, i. e., with a non-trivial JV(J T ). It is worth noting 
that this typically happens also in power grasps [38.30] 
where the hand envelops the object establishing con¬ 
tacts even with inner limbs. In this case the hand 
Jacobian is a tall matrix with a non-trivial null space 
of its transpose. 

If the system is very defective, it is very likely 
that the grasp will be hyperstatic according to Def¬ 



Fig. 38.5 Compliant joints and compliant contacts, main 
definitions 

inition 38.5 and consequently the rigid-body model 
in (38.21) is under-determined and does not admit 
a unique solution for the contact force vector X as dis¬ 
cussed in Sect. 38.2.2. Note that computing the force 
distribution X is crucial in grasp analysis since it allows 
one to evaluate if the contact constraints are fulfilled, 
and consequently, if the grasp will be maintained. 

The force distribution problem in hyperstatic grasps 
is an under-determined problem of statics under the as¬ 
sumption of rigid contacts of (38.17). To solve the prob¬ 
lem, we need to enrich the model with more information 
on the contact forces. A possibile solution is to sub¬ 
stitute the rigid-body kinematic constraint (38.19) with 
a compliant model of the contact interaction [38.21], 
obtained introducing a set of springs between the con¬ 
tact points on the hand c h and on the object c° as shown 
in Fig. 38.5 

C c A =c h -c‘ 


(38.31) 
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where C c € R' ,;lX ' ,;l is the contact compliance matrix 
which is symmetric and positive definite. The contact 
stiffness matrix is defined as the reciprocal of the con¬ 
tact compliance matrix K c = C^ 1 - 

The analysis is presented in a quasi-static frame¬ 
work: from an equilibrium reference configuration q r 0 , 
qo, To, Ao, «o and go, a small input perturbation is 
applied to the joint references q, o + A q T and to the ex¬ 
ternal load go + A g which moves the grasp system to 
a new equilibrium configuration whose linear approxi¬ 
mation is represented by qo + A q, to + At,Ao + AA, 
«o + Am. 

In the following, for the sake of simplicity, it is 
convenient to refer G and J matrices to the object ref¬ 
erence frame {B} and not to the inertial frame {IV} as 
previously described. Note that this only applies to this 
section on compliance analysis, while the rest of this 
chapter still refers to matrices G and J as described in 
Sect. 38.1.1. 

Let Rfc e R 3x3 represent the orientation of the ob¬ 
ject frame {B} with respect to the inertial frame. Then 
the object twist referred to {B} is given as 

•h.obj = K (^ N ° bJ ) , (38.32) 

where 

R* = Blockdiag(R*. R„) e R 6x6 . 

Then, substituting P^v from (38.2) into (38.32) yields 
the partial grasp matrix G J e R 6x6 , that maps the object 
twist from {TV} to contact frame {C}; referred to {B}. 

Similarly, the hand twists can be expressed to the 
object frame \B\ 


—1 

^i,hnd — 


(v* bn A 

WW 


(38.33) 


and combining (38.33) and (38.7) yields the partial 
hand Jacobian J,- e R 6x "«, which relates the joint ve¬ 
locities to the contact twists on the hand expressed with 
respect to {B}. 

Table 38.7 gives the definitions of the selection ma¬ 
trices for the three contact models when object and hand 
twists are referred to frame {B}. In the planar case, as 
far as contact constraint is concerned, equation (38.15) 
holds with H,f and H,m defined in Table 38.8. 

The contact force variation A A from the equilib¬ 
rium configuration is then related to the relative dis¬ 
placement between the object and the fingers at the 
contact points as 


Table 38.7 Selection matrices for three contact models, 
when the object twists are expressed with respect to {B} 



Table 38.8 Selection matrices for the planar simplified 
case, when the object twists are expressed with respect 
to {B} 


Model 

11 Xi H; F 

H,m 

PwoF 

1 nf 

Vacuous 

HF/SF 

2 [»?,?n T 

Vacuous 


By assuming that the perturbed configuration is suf¬ 
ficiently near to the reference one, the following lin¬ 
earized relationships can be found for g , and r 


Ag = —GAA (38.35) 

Ar = J t AA + K Jtq Aq + Kj, u Au (38.36) 


where 


K j. q = 


ajAc 

dq 


and K/,„ = 


3JAo 

3m 


are the variation of the hand Jacobian with respect to q 
and u variations, respectively. 


Remark 38.3 

In (38.35) and (38.36) both the grasp matrix and hand 
Jacobian are expressed with respect to the object ref¬ 
erence frame, and by neglecting rolling between the 
fingers and the object at the contact points, G becomes 
constant, while J(< 7 , m), in general, depends on both 
hand and object configurations. Matrix K J q represent 
the variability of J(< 7 , u) matrix with respect to hand and 
object configurations. 


Matrices e Rand K JiU e R' , « Xnu are usu¬ 
ally referred to as geometric stiffness matrices [38.28]. 
Furthermore, it can be verified that the matrix K J r/ is 
symmetric [38.27]. 

By substituting (38.36) in (38.30) we get 


JrC 9 J t AA — JrA< 7 , — JA</ — J r C 9 K/ „Am , 

(38.37) 


where 


C c AA = (JA</ —G t Am) . 


J r = J(I + C 9 K 7 , ? ) 


(38.34) 
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Summing (38.34) and (38.37) the following expression 
can be found for the contact forces displacement AA 

AA = K CiC (J R Atf r -G£AM) (38.38) 

where 

g£ = g t + j r c 9 k 7 ,„ 

and 

K c , e = (C c + J R C 9 J T ) _1 . (38.39) 

Matrix K c .,,, represents the equivalent contact stiffness, 
that takes into account both the joint and the contact 
compliance. If the geometrical terms are neglected, i. e., 
K J q = 0 and K / „ = 0, the classical expression K r ,. = 
(C c + JC ? J T ) for the equivalent contact stiffness ma¬ 
trix can be found [38.28]. 

By substituting (38.38) in (38.35), the object dis¬ 
placement can be evaluated as a function of the small 
input perturbations A q r and A g 

Au = (GK Cif G R ) _1 (GK c , e J R A<jr r + A g) . (38.40) 
When Aq r = 0, (38.40) can be rewritten as 

A g = KAu , 

with K = GK^Gj , 

where the term that multiplies the external wrench vari¬ 
ation A g represents the reciprocal of the grasp stiffness 
matrix K. The grasp stiffness evaluates ability of the 

38.4 Restraint Analysis 

The most fundamental requirements in grasping and 
dexterous manipulation are the abilities to hold an ob¬ 
ject and control its position and orientation relative to 
the palm of the hand. The two most useful character¬ 
izations of grasp restraint are force closure and form 
closure. These names were in use as early as 1876 in 
the field of machine design to distinguish between joints 
that required an external force to maintain contact, and 
those that did not [38.31]. For example, some water 
wheels had a cylindrical axle that was laid in a hori¬ 
zontal semi-cylindrical groove split on either side of the 
wheel. During operation, the weight of the wheel acted 
to close the groove-axle contacts, hence the term force 
closure. By contrast, if the grooves were replaced by 
cylindrical holes just long enough to accept the axle, 
then the contacts would be closed by the geometry 
(even if the direction of the gravitational force was re¬ 
versed), hence the term form closure. 


robotic grasp to resist to external load variations applied 
to the object. 

Regarding the force distribution AA, by substituting 
(38.40) into (38.38), the variation of the contact forces 
is evaluated as 

AA = G+ Ag + PAtf r , (38.41) 

where 

G+ = K c , e G£ (GK c , e G £) _1 , 

P= (i-g+g) K c ,J r . 

Matrix Gjj is a right pseudo-inverse of grasp matrix G 
that takes into account both the geometrical effects 
and the hand and contact stiffness. Matrix P maps the 
reference joint variables A q T to the contact force varia¬ 
tion A A. 

It is worth noting that (I — G^“G) is a projector onto 
the null space of G and then each contact force variation 
AA;, = PA<jr r , produced by modifying joint reference 
values, satisfies the equation 

GAA h = 0 , 

and then belongs to the internal force subspace. 

Summarizing, compliant grasps have been analyzed 
with a linearized quasi-static model and the main re¬ 
lationships mapping joint references, the controlled 
inputs, and external disturbances, the wrenches, onto 
object motions and contact forces have been evaluated 
in (38.40) and (38.41), respectively. 


When applied to grasping, form and force clo¬ 
sure have the following interpretations. Assume that 
a hand grasping an object has its joint angles locked 
and its palm fixed in space. Then the grasp has form 
closure, or the object is form-closed, if it is impossi¬ 
ble to move the object, even infinitesimally. Under the 
same conditions, the grasp has force closure, or the 
object is force-closed, if for any non-contact wrench 
experienced by the object, contact wrench intensities 
exist that satisfy (38.20) and are consistent with the 
constraints imposed by the friction models applica¬ 
ble at the contact points. Notice that all form closure 
grasps are also force closure grasps. When under form 
closure, the object cannot move even infinitesimally rel¬ 
ative to the hand, regardless of the non-contact wrench. 
Therefore, the hand maintains the object in equilib¬ 
rium for any external wrench, which is the force closure 
requirement. 
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Roughly speaking, form closure occurs when the 
palm and fingers wrap around the object forming a cage 
with no wiggle room such as the grasp shown in 
Fig. 38.6. This kind of grasp is also called a power 
grasp [38.32] or enveloping grasp [38.33]. However, 
force closure is possible with fewer contacts, as shown 
in Fig. 38.7, but in this case, force closure requires 
the ability to control internal forces. It is also possible 
for a grasp to have partial form closure, indicating that 



Fig. 38.6 The palm and fingers combine to create a very 
secure form closure grasp of a router 


^7 

Y DLR 



Fig. 38.7 Hand and wrist of the tendon driven DLR 
Hand Arm System grasping a tool [38.7]. This grasp has 
a force closure grasp appropriate for dexterous manipu¬ 
lation. (DLR Hand Arm System, Photo courtesy of DLR 
2011) 


only a subset of the possible degrees of freedom are re¬ 
strained [38.34]. An example of such a grasp is shown 
in Fig. 38.8. In this grasp, fingertip placement between 
the ridges around the periphery of the bottle cap pro¬ 
vides form closure against relative rotation about the 
axis of the helix of the threads and also against trans¬ 
lation perpendicular to that axis, but the other three 
degrees of freedom are restrained through force closure. 
Strictly speaking, given a grasp of a real object by a hu¬ 
man hand it is impossible to prevent relative motion of 
the object with respect to the palm due to the compli¬ 
ance of the hand and object. Preventing all motion is 
possible only if the contacting bodies are rigid, as is as¬ 
sumed in most mathematical models employed in grasp 
analysis. 

38.4.1 Form Closure 

To make the notion of form closure precise, introduce 
a signed gap function denoted by 1 /rj(u,q) at each of 
the h c contact points between the object and the hand. 
The gap function is zero at each contact, becomes 
positive if contact breaks, and negative if penetration 
occurs. The gap function can be thought of as distance 
between the contact points. In general, this function is 
dependent on the shapes of the contacting bodies. Let 
u and q represent the configurations of the object and 
hand for a given grasp; then 

■^i(u,q) = 0 Vi=l,...,n c . (38.42) 

The form closure condition can now be stated in terms 
of a differential change di< of u : 



Fig. 38.8 In the grasp depicted, contact with the ridges on 
the bottle cap create partial form closure in the direction 
of cap rotation (when screwing it in) and also in the direc¬ 
tions of translation perpendicular to the axis of rotation. To 
achieve complete control over the cap, the grasp achieves 
force closure over the other three degrees of freedom 
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Definition 38.6 

A grasp (u,q) has form closure if and only if the fol¬ 
lowing implication holds 

f(u + dtt, q) > 0 => dtt = 0 , ( 38 . 43 ) 

where if is the n c -dimensional vector of gap functions 
with r'-th component equal to fi(u,q). By definition, 
inequalities between vectors imply that the inequality 
is applied between corresponding components of the 
vectors. 


Expanding the gap function vector in a Taylor series 
about u, yields infinitesimal form closure tests of var¬ 
ious orders. Let ^ f(u,q), ft = 1,2,3,..., denote the 
Taylor series approximation truncated after the terms of 
order ft in d u. From (38.42), it follows that the first- 
order approximation is 

! 3 f(u,q) 

f(u + du,q) = ——-|( 5 ,^)d«, 

where \(u,q) denotes the partial derivative of f 

with respect to u evaluated at ( u , q). Replacing f with 
its approximation of order /3 in (38.43) implies three 
relevant cases of order ft: 


form closure. Note that first-order form closure only 
involves the first derivatives of the distance functions. 
This implies that the only relevant geometry in first- 
order form closure are the locations of the contacts 
and the directions of the contact normals. The grasp in 
the center has form closure of higher order, with the 
specific order depending on the degrees of the curves 
defining the surfaces of the object and fingers in the 
neighborhoods of the contacts [38.35]. Second-order 
form closure analysis depends on the curvatures of the 
two contacting bodies in addition to the geometric in¬ 
formation used to analyze first-order form closure. The 
grasp on the right does not have form closure of any 
order, because the object can translate horizontally and 
rotate about its center. 

First-Order Form Closure 

First-order form closure exists if and only if the follow¬ 
ing implication holds 


3 f(u, q) 
du 


|(«, 9 )d M > 0 => du = 0 . 


The first-order form closure condition can be written in 
terms of the object twist v 


1. If there exists du such that ^f(ii + du,q) has at 
least one strictly positive component, then the grasp 
does not have form closure of order /J. 

2. If for every non-zero du, ^f (u + d«. q ) has at least 
one strictly negative component, then the grasp has 
form closure of order f. 

3. If neither case i) nor case ii) applies for all r, f(u + 
d u,q)V q < /}, then higher-order analysis is re¬ 
quired to determine the existence of form closure. 

Figure 38.9 illustrates form closure concepts using 
several planar grasps of gray objects by fingers shown 
as black disks. The concepts are identical for grasps of 
three-dimensional objects, but are more clearly illus¬ 
trated in the plane. The grasp on the left has first-order 



Fig. 38.9 Three planar grasps; two with form closure of 
different orders and one without form closure 


G^v>0=>v= 0, ( 38 . 44 ) 

where G„ = df/du\ e M" 0 * 6 . Recall that V is the 
kinematic map defined in (38.1). Also notice that G n 
is the grasp matrix, when all contact points are friction¬ 
less. 

Because the gap functions only quantify distances, 
the product Gj v is the vector of normal components of 
the instantaneous velocities of the object at the contact 
points (which must be non-negative to prevent interpen¬ 
etration). This in turn implies that the grasp matrix is 
the one that would result from the assumption that all 
contacts tire of the type PwoF. 

An equivalent condition in terms of the contact 
wrench intensity vector X n e M" c can be stated as fol¬ 
lows. A grasp has first-order form closure if and only 
if 

J n tV _ 1 v ^ K6 - (38 - 45> 

The physical interpretation of this condition is that equi¬ 
librium can be maintained under the assumption that 
the contacts are frictionless. Note that the components 
of X n are the magnitudes of the normal components of 
the contact forces. The subscript (.) n is used to em¬ 
phasize that A n contains no other force or moment 
components. 
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Since g must be in the range of G n for equilibrium to 
be satisfied, and since g is an arbitrary element of R 6 , 
then in order for condition (38.45) to be satisfied, the 
rank of G n must be six. Assuming rank(G n ) = 6, an¬ 
other equivalent mathematical statement of first-order 
form closure is: there exists A n such that the following 
two conditions hold [38.36] 

G n -A, „ = 0 , 

A n > 0. (38.46) 

This means that there exists a set of strictly compressive 
normal contact forces in the null space of G n . In other 
words, one can squeeze the object as tightly as desired 
while maintaining equilibrium. A second interpretation 
of this condition is that the non-negative span of the 
columns of G n must equal R 6 . Equivalently, the con¬ 
vex hull of the columns of G n must strictly contain the 
origin of R 6 . As will be seen, the span and hull interpre¬ 
tations will provide a conceptual link called frictional 
form closure that lies between form closure and force 
closure. 

The duality of conditions (38.44) and (38.45) can 
be seen clearly by examining the set of wrenches 
that can be applied by frictionless contacts and the 
corresponding set of possible object twists. For this dis¬ 
cussion, it is useful to give definitions of cones and their 
duals. 


Definition 38.7 

A cone C is a set of vectors ^ such that for every g in 
C, every non-negative scalar multiple of ^ is also in C. 


Equivalently, a cone is a set of vectors closed under 
addition and non-negative scalar multiplication. 


Definition 38.8 

Given a cone C with elements g, the dual cone C* 
with elements g *, is the set of vectors such that the dot 
product of g* with each vector in C is non-negative. 
Mathematically 

C* = k*!? 1 ?* >0, Vg eC}. (38.47) 


See Example 4. 

First-Order Form Closure Requirements 
Several useful necessary conditions for form closure 
are known. In 1897 Somov proved that at least seven 
contacts tire necessary to form close a rigid object 
with six degrees of freedom [38.37,38]. Lakshmi- 
narayana generalized this to prove that n v + 1 contacts 
are necessary to form close an object with n v degrees 


of freedom [38.34] (based on Goldman and Tucker 
1956 [38.39]) (Table 38.9). This led to the definition 
of partial form closure that was mentioned above in 
the discussion of the hand grasping the bottle cap. 
Markenscoff and Papadimitriou determined a tight up¬ 
per bound, showing that for all objects whose surfaces 
are not surfaces of revolution, at most n v + 1 contacts 
are necessary [38.40]. Form closure is impossible to 
achieve for surfaces of revolution. 

To emphasize the fact that n v + 1 contacts are nec¬ 
essary and not sufficient, consider grasping a cube with 
seven or more points of contact. If all contacts are on 
one face, then clearly, the grasp does not have form 
closure. 

First-Order Form Closure Tests 
Because form closure grasps are very secure, it is de¬ 
sirable to design or synthesize such grasps. To do this, 
one needs a way to test candidate grasps for form clo¬ 
sure, and rank them, so that the best grasp can be 
chosen. One reasonable measure of form closure can be 
derived from the geometric interpretation of the condi¬ 
tion (38.46). The null space constraint and the positivity 
of A„ represent the addition of the columns of G n scaled 
by the components of X n . Any choice of A n closing 
this loop is in JsT(G n ). For a given loop, if the mag¬ 
nitude of the smallest component of A n is positive, then 
the grasp has form closure, otherwise it does not. Let 
us denote this smallest component by d. Since such 
a loop, and hence d, can be scaled arbitrarily, A„ must 
be bounded. 

After verifying that G n has full row rank, a quanti¬ 
tative form closure test based on the above observations 
can be formulated as a linear program (LP) in the un¬ 
knowns d and A n as follows 

LP1: maximize: d (38.48) 

subject to: G„A n = 0 (38.49) 

IA n -U>0 (38.50) 

d> 0 (38.51) 

l T A n < n c , (38.52) 

where I e M" cX " c is the identity matrix and 1 e R' !c is 
a vector with all components equal to 1. The last in¬ 
equality is designed to prevent this LP from becoming 
unbounded. A typical LP solution algorithm determines 


Table 38.9 Minimum number of contacts n c required to 
form close an object with n v degrees of freedom 


11 y 

n c 

3 (planar grasp) 

4 

6 (spatial grasp) 

7 

fly (general) 

Ylv + 1 
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infeasibility or unboundedness of the constraints in 
Phase I of the algorithm, and considers the result be¬ 
fore attempting to calculate an optimal value [38.41]. If 
LP1 is infeasible, or if the optimal value d* is zero, then 
the grasp is not form closed. 

The quantitative form closure test (38.48)-(38.52) 
has n c + 8 constraints and n c + 1 unknowns. For a typ¬ 
ical grasp with n c < 10 , this is a small linear program 
that can be solved very quickly using the simplex 
method. However, one should note that d* is dependent 
on the choice of units used when forming G n . It would 
be advisable to non-dimensionalize the components of 
the wrenches to avoid dependence of the optimal d on 
one’s choice of units. This could be done by dividing 
the first three rows of G n by a characteristic force and 
the last three rows by a characteristic moment. If one 
desires a binary test, LP1 can be converted into one by 
dropping the last constraint (38.52) and applying only 
Phase I of the simplex algorithm. 

In summary, quantitative form closure testing is 
a two-step process: 

Form Closure Test. 

1. Compute rank(G n ). 

a) If rank(G n ) ^n v , then form closure does not 
exist. Stop. 

b) If rank(G n ) = n v , continue. 

2. Solve LP1. 

a) If d* = 0, then form closure does not exist. 

b) If d* > 0, then form closure exists and d*n c is 
a crude measure of how far the grasp is from 
losing form closure. 

See Example 5, Part 1. 

Variations of the Test. If the rank test fails, then the 
grasp could have partial form closure over as many 
as rank(G n ) degrees of freedom. If one desires to 
test this, then LP1 must be solved using a new G n 
formed by retaining only the rows corresponding to 
the degrees of freedom for which partial form closure 
is to be tested. If d* > 0, then partial form closure 
exists. 

A second variation arises when one knows in ad¬ 
vance that the object is already partially constrained. 
For example, in the case of a steering wheel, the driver 
knows, that relative to her, the steering wheel has only 
one degree of freedom. A form closure grasp suit¬ 
able for driving would be required only to restraint the 
wheel’s rotation about the steering column. In general, 
assume that the object is constrained by a set of bilateral 
constraints, which can be written as 


With these additional constraints, the form closure 
property can be expressed as follows 

G„ v > 0) 

" )=>v=0, (38.54) 

B t v = 0 j 

which can be shown to be equivalent to 

G^/0 > 0=> p = 0 , (38.55) 

where p is an arbitrary vector with length equal to the 
dimension of the null space of B T and G n = A /( G n , 
where A /( is an annihilator of the column space of B. 
One possible way to construct this annihilator is A B = 
(N (B T )) , where N (B T ) is a matrix whose columns 
form a basis for the null space of B T . Also note that 
N(B T )p is a possible twist of the object consistent with 
the bilateral constraints (i. e., the twists that must be 
eliminated by the unilateral constraints). 

The form closure condition for an object partially 
constrained by bilateral constraints can also be stated in 
terms of wrenches 

G n ^n = 0 > 

A n > 0. (38.56) 

Notice that twist conditions (38.44) and (38.55) and 
wrench conditions (38.46) and (38.56) are formally 
analogous, and therefore, the quantitative form clo¬ 
sure test can be applied by substituting G n for G n and 
( n v — rank(B)) for n v . For detailed information on the 
derivations of the constrained form closure conditions, 
please refer to [38.42], 

See Example 5, Part 2. 

Monotonicity. In grasp synthesis, sometimes it is de¬ 
sirable for the grasp metric to increase monotonically 
with the number of contact points, the intuition be¬ 
ing, that adding contacts to a stable grasp will make it 
more stable. The solution d* does not have this prop¬ 
erty. However f ormClosure .m, a Matlab function 
which is available at [38.43], returns d*n c as the metric, 
because it nearly maintains monotonicity when adding 
random contacts to random grasps. This can be demon¬ 
strated by running test_monotonicity .m. 

One can design metrics that are monotonic with 
norms on the set of wrenches that the hand can apply. 
Let the polytope <3 = {g\g = G„A n , 0 < A n < 1} denote 
the wrench space, which is the set of wrenches that can 
be applied by the hand if the contact forces magnitudes 
are limited to one. A metric proposed by Ferrari and 
Canny [38.44] is the radius of the largest sphere in g 
with origin at the origin of the wrench space. The ra¬ 
dius is greater than 0 if and only if the grasp has form 


B t v = 0. 


(38.53) 
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closure. In addition, adding a contact point will always 
produce a new g that is a superset of the original g. 
Thus the radius of the sphere cannot reduce as a result. 
Hence the monotonicity property holds. 

See Example 5, Part 3. 

Planar Simplifications 

In the planar case, Nguyen [38.45] developed a graphi¬ 
cal qualitative test for form closure. Figure 38.10 shows 
a form closure grasp with four contacts. To test form 
closure one partitions the normals into two groups of 
two. Let Ci be the non-negative span of two normals in 
one pair and C 2 be the non-negative span of the other 
pair. A grasp has form closure if and only if Ci and C 2 
or —Ci and —C 2 see each other for any pairings. Two 
cones see each other if the open line segment defined 
by the vertices of the cones lies in the interior of both 
cones. In the presence of more than four contacts, if 
any set of four contacts satisfies this condition, then the 
grasp has form closure. 

Notice that this graphical test can be difficult to ex¬ 
ecute for grasps with more than four contacts. Also, it 
does not extend to grasps of three-dimensional (3-D) 
objects and does not provide a closure measure. 

38.4.2 Force Closure 

A grasp has force closure, or is force-closed, if the grasp 
can be maintained in the face of any object wrench. 
Force closure is similar to form closure, but relaxed to 
allow friction forces to help balance the object wrench. 
A benefit of including friction in the analysis is the 
reduction in the number of contact points needed for 
closure. A three-dimensional object with six degrees 
of freedom requires seven contacts for form closure, 
but for force closure, only two contacts are needed if 
they are modeled as soft fingers, and only three (non- 
collinear) contacts are needed if they are modeled as 
hard fingers. 

Force closure relies on the ability of the hand to 
squeeze arbitrarily tightly in order to compensate for 
large applied wrenches that can only be resisted by 
friction. Figure 38.15 shows a grasped polygon. Con- 



Fig. 38.10 Planar grasps with first-order form closure 


sider applying a wrench to the object that is a pure 
force acting upward along the y-axis of the inertial 
frame. It seems intuitive that if there is enough fric¬ 
tion, the hand will be able to squeeze the object with 
friction forces preventing the object’s upward escape. 
Also, as the applied force increases in magnitude, the 
magnitude of the squeezing force will have to increase 
accordingly. 

Since force closure is dependent on the friction 
models, common models will be introduced before giv¬ 
ing formal definitions of force closure. 

Friction Models 

Recall the components of force and moment transmit¬ 
ted through contact i under the various contact models 
given earlier in Table 38.4. At contact point i, the fric¬ 
tion law imposes constraints on the components of the 
contact force and moment. Specifically, the frictional 
components of A,- are constrained to lie inside a limit 
surface, denoted by £,, that scales linearly with the 
product Hifm, where /X; is the coefficient of friction at 
contact i. In the case of Coulomb friction, the limit sur¬ 
face is a circle of radius nfm. The Coulomb friction 
cone Ji is a subset of M 3 

ft = Mi/in} • (38.57) 

More generally, the friction laws of interest have 
limit surfaces defined in the space of friction compo¬ 
nents, and friction cones J 7 , defined in the space 

of A,-, R' Ui . They can be written as follows 

Ji = { A, er A - | ||A,| \ w <f in } , (38.58) 

where 11A ,■ 11 ^ denotes a weighted quadratic norm of the 
friction components at contact i. The limit surface is 
defined by ||A,-|| M =f n . 

Table 38.10 defines useful weighted quadratic 
norms for the three contact models: PwoF, HF, and SF. 
The parameter /x, is the friction coefficient for the tan¬ 
gential forces, Vi is the torsional friction coefficient, 
and a is the characteristic length of the object that is 
used to ensure consistent units for the norm of the SF 
model. 


Table 38.10 Norms for the three main contact models 


Model 

IIA.IU 

PwoF 

0 

HF 

J_ / f2 , f 2 
//, y Jit ' Jio 

SF 

JTi \/f£ ~F/io + trui l m ' n l 
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Remark 38A 

There are several noteworthy points to be made about 
the friction cones. First, all of them implicitly or ex¬ 
plicitly constrain the normal component of the con¬ 
tact force to be non-negative. The cone for SF con¬ 
tacts has a cylindrical limit surface with circular 
cross section in the (f it ,f io )-plane and rectangular cross 
section in the (f it , m, n )-plane. With this model, the 
amount of torsional friction that can be transmitted 
is independent of the lateral friction load. An im¬ 
proved model that couples the torsional friction limit 
with the tangential limit was studied by Howe and 
Cutkosky [38.46]. 


A Force Closure Definition 
One common definition of force closure can be stated 
simply by modifying condition (38.45) to allow each 
contact force to lie in its friction cone rather than 
along the contact normal. Because this definition does 
not consider the hand’s ability to control contact 
forces, this definition will be referred to as frictional 
form closure. A grasp will be said to have frictional 
form closure if and only if the following condition is 
satisfied 


Definition 38.10 

A grasp has force closure if and only if 
rank(G) = n v , M(G) fi WTJ t ) = 0 , 
and there exists A such that GA = 0 and X G Int(J’). 


The full row rank condition on the matrix G is the 
same condition required for form closure, although G 
is different from G n used to determine form closure. If 
the rank test passes, then one must still find X satisfying 
the remaining three conditions. Of these, the null space 
intersection test can be performed easily by linear pro¬ 
gramming techniques, but the friction cone constraint 
is quadratic, and thus forces one to use non-linear pro¬ 
gramming techniques. While exact non-linear tests have 
been developed [38.47], only approximate tests will be 
presented here. 

Approximate Force Closure Tests 
Any of the friction cones discussed can be approxi¬ 
mated as the non-negative span of a finite number n g 
of generators Sy of the friction cone. Given this, one 
can represent the set of applicable contact wrenches at 
contact i as follows 


GX = -g( 

AcJ ) 


Wge m ,,u , 


where J is the composite friction cone defined 
as: 


J = J lX -xJ„ c 

= {X GR m |A; g Jy, i=l,...,n c }, 

and each J 7 , is defined by (38.58) and one of the models 
listed in Table 38.10. 

Letting Int(jT) denote the interior of the composite 
friction cone, Murray et al. give the following equiva¬ 
lent definition [38.19]: 


Definition 38.9 

(Proposition 5.2, Murray et al.) A grasp has frictional 
form closure if and only if the following conditions are 
satisfied: 

1. rank(G) = n v 

2. 3 A such that GA = 0 and A G Int(jF). 


These conditions define what Murray et al. call 
force closure. The force closure definition adopted here 
is stricter than frictional form closure; it additionally re¬ 
quires that the hand be able to control the internal object 
forces. 


G,A, = S,o-, : , ff; > 0 , 

where S, = [s,i ■ ■ ■ .s',„ g ] and a,- is a vector of non¬ 
negative generator weights. If contact i is frictionless, 
then n g = 1 and S; = [h\ ((c, —p) x n,) T ] T . 

If contact i is of type HF, we represent the friction 
cone by the non-negative sum of uniformly spaced con¬ 
tact force generators (Fig. 38.11) whose non-negative 
span approximates the Coulomb cone with an inscribed 
regular polyhedral cone. This leads to the following def- 



Fig. 38.11 Quadratic cone approximated as a polyhedral 
cone with seven generators 


Part D | 38.4 














Part D | 38.4 


974 


Part D 


Manipulation and Interfaces 


inition of S,- 

(■ 1 

S, = [• HiCos(2kjr/n g ) •••!, (38.59) 

\- ■ ■ Hi sin(2far/n g ) ■••/ 

where the index k varies from one to n g . If one prefers 
to approximate the quadratic friction cone by a circum¬ 
scribing polyhedral cone, one simply replaces Hi in the 
above definition with Hi/ cos (n/n g ). 

The adjustment needed for the SF model is quite 
simple. Since the torsional friction in this model is de¬ 
coupled from the tangential friction, its generators are 
given by [1 0 0 ± Thus Sfor the SF model is 


/ ■■ 

1 

1 

1 


Hi cos(2L7r/n g ) 

0 

0 


Hi sm(2kn/n s ) 

0 

0 

V •• 

0 

bvj 

-bVi 


(38.60) 

where b is the characteristic length used to unify units. 
The set of total contact wrenches that may be applied 
by the hand without violating the contact friction law at 
any contact can be written as 

GA = Scr, <r > 0 , 

where 

S = (Sr,-- - ,S„ g ) 

and 

<r = (o-}---o-J g ) T . 

It is convenient to reformulate the friction con¬ 
straints in a dual form 

F,A, > 0 . (38.61) 

In this form, each row of F, is normal to a face formed 
by two adjacent generators of the approximate cone. For 
an HF contact, row i of F,- can be computed as the cross 
product of Sj and s f _|_ i. In the case of an SF contact, the 
generators are of dimension four, so simple cross prod¬ 
ucts will not suffice. However, general methods exist to 
perform the conversion from the generator form to the 
face normal form [38.39]. 

The face normal constraints for all contacts can be 
combined into the following compact form 

FA > 0 , (38.62) 


Let e, e R" Ai be the first row of H,. Further let 
e= [e 1 ,...,e„ c ]eR' u 
and let 

E = Blockdiagjd,.. ., e,J e R" 1 *' 1 ' . 

The following linear program is a quantitative test for 
frictional form closure. The optimal objective function 
value d* is a measure of the distance the contact forces 
are from the boundaries of their friction cones, and 
hence a crude measure of how far a grasp is from losing 
frictional form closure. 

LP2: maximize: d 

subject to: GA = 0 

FA — Id > 0 

d> 0 
eX <n c . 

The last inequality in LP2 is simply the sum of the mag¬ 
nitudes of the normal components of the contact forces. 
After solving LP2, if d* = 0 frictional form closure 
does not exist, but if d* > 0, then it does. 

If the grasp has frictional form closure, the last step 
to determine the existence of force closure is to verify 
the condition 'N (G) fl 2Af (J T ) = 0. If it holds, then the 
grasp has force closure. This condition is easy to verify 
with another linear program LP3. 

LP3: maximize: d 

subject to: GA = 0 

J T A =0 
EA — Id > 0 
d> 0 
eX < n c . 

In summary, force closure testing is a three-step 
process. 

Approximate Force Closure Test. 

1. Compute rank(G). 

a) If rank(G) f n v , then force closure does not ex¬ 
ist. Stop. 

b) If rank(G) = n v , continue. 

2. Solve LP2: Test frictional form closure. 

a) If d* = 0, then frictional form closure does not 
exist. Stop. 

b) If d* > 0, then frictional form closure exists and 
d* is a crude measure of how far the grasp is 
from losing frictional form closure. 


where F = Blockdiag(Fi__ F„ c ). 







Grasping I 38.5 Examples 975 


3. Solve LP3: Test control of internal force. 

a) If d* > 0, then force closure does not exist. 

b) If d* = 0, then force closure exists. 

Variation of the Test. A variation of the approximate 
force closure test arises when the object is partially 
constrained with the bilateral constraints described by 
(38.53), in such a cases the definitions of frictional form 
closure becomes 

GA =0. 

Aeint(J), (38.63) 

where G = A fi G. A /( is an annihilator of the column 
space of B. Similarly, the force closure condition can 
be stated as 

GA =0, 

A G int( J) , 

M(G) n /A/(J T ) = 0 , (38.64) 

Since the frictional form closure definition (38.63) 
is analogous to Definition 38.9 and the force closure 
definition (38.64) is analogous to Definition 38.10, the 
force closure test can be applied by substituting G for 

38.5 Examples 

38.5.1 Example 1: Grasped Sphere 

Part 1: G and J 

Figure 38.12 shows a planar projection of a three- 
dimensional sphere of radius r grasped by two fin¬ 
gers, which make two contacts at angles G\ and 62- 
The frames {C}i and {C }2 are oriented so that their 
o-directions point out of the plane of the figure (as indi¬ 
cated by the small filled circles at the contact points). 
The axes of the frames {N} and {B} were chosen to 
be axis-aligned with coincident origins located at the 
center of the sphere. The z-axes are pointing out of the 
page. Observe that since the two joint axes of the left 
finger are perpendicular to the (x,y)-plane, it operates 
in that plane for all time. The other finger has three rev¬ 
olute joints. Because its first and second axes, £3 and £ 4 , 
currently lie in the plane, rotation about £3 will cause 
£4 to attain an out-of-plane component and would cause 
the finger tip at contact 2 to leave the plane. 

In the current configuration, the rotation matrix for 
the i-th contact frame is defined as follows 

/ — cos (6j) sin(@,j 0\ 

R, = I — sin(0,) —cos (6i) 0 I . (38.66) 

Vo 01 / 


G, provided that, (n v — rank(B)) is substituted for n v . 
For detailed information on the derivation of these con¬ 
ditions, please refer to [38.42]. 

See Example 1, Part 6. 

Planar Simplifications 

In planar grasping systems, the approximate method de¬ 
scribed above is exact. This is because the SF models 
are meaningless, since rotations about the contact nor¬ 
mal would cause motions out of the plane. With regard 
to the HF model, for planar problems, the quadratic 
friction cone becomes linear, with its cone represented 
exactly as 



Nguyen’s graphical form closure test can be applied 
to planar grasps with two frictional contacts [38.45]. 
The only change is that the four contact normals are re¬ 
placed by the four generators of the two friction cones. 
However, the test can only determine frictional form 
closure, since it does not incorporate the additional in¬ 
formation needed to determine force closure. 


The vector from the origin of {N} to the i-th contact 
point is given by 

c, —p = r (cos((9,) sin(0,) 0) T . (38.67) 

Substituting into (38.3) and (38.6) yields the complete 
grasp matrix for contact i 


-C, 

-Si 

0 

Si 

~Ci 

0 

0 

0 

1 


0 

\ 

0 

0 

rsi 

-C; 

Si 

0 

0 

0 

—rci 

~Si 

-Cj 

0 

0 

—r 

0 

0 

0 

1 / 


(38.68) 

where 0 G R 3x3 is the zero matrix and c, and s, are 
abbreviations for cos(@,) and sin(0,), respectively. The 
complete grasp matrix is defined as: G = (Gi G 2 ) £ 
R 6x12 . 

The accuracy of this matrix can be verified by in¬ 
spection, according to Remark 38.2. For example, the 
first column is the unit wrench of the unit contact nor¬ 
mal; the first three components are the direction cosines 
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of hi and the last three are (c,- —p) x n,. Since n, is 
collinear with (c, —p), the cross products (the last three 
components of the column) are zero. The last three 
components of the second column represent the mo¬ 
ment of tj about the x, y-, and z-axes of {N}. Since f,- 
lies in the (x, y)-plane, the moments with the x- andy- 
axes are zero. Clearly f, produces a moment of — r about 
the z-axis. 

Construction of the hand Jacobian J, for contact i 
requires knowledge of the joint axis directions and the 
origins of the frames fixed to the links of each finger. 
Figure 38.13 shows the hand in the same configuration 
as in Fig. 38.12, but with some additional data needed 
to construct the hand Jacobian. Assume that the origins 
of the joint frames lie in the plane of the figure. In the 
current configuration, the quantities of interest for con¬ 
tact 1, expressed in {C}i are 

Ci-$i = (Z 2 h 0) T , (38.69) 



Fig. 38.12 A sphere grasped by a two-fingered hand with 5 
revolute joints 


Fig. 38.13 Relevant data for the hand Jacobian in 
Fig. 38.12 


Ci-$2 = {h h 0) T , (38.70) 

Zt=Z 2 = (0 0 1) T . (38.71) 

The quantities of interest for contact 2, in JCji are 

C2-^=C2-U = (l4 h 0) T , (38.72) 

C2-^5 = {k 0 0) T , (38.73) 

Z 3 = (0 1 0) T , (38.74) 

Z4fes) = (-! 1 0) T , (38.75) 

Z5(<?3, <?4) = (0 0 l) T . (38.76) 


Generally all of the components of the c — £ and z 
vectors (including the components that are zero in the 
current configuration), are functions of q and u. The de¬ 
pendencies of the z vectors are shown explicitly. 

Substituting into (38.14), (38.11) and (38.8) yields 
the complete hand Jacobian J e M 12x5 



The horizontal dividing line partitions J into Ji (on 
top) and J 2 (on the bottom). The columns correspond 
to joints 1 through 5. The block diagonal structure is 
a result of the fact that finger i directly affects only con¬ 
tact i. 

Example 1, Part 2: G and J 
Assume that the contacts in Fig. 38.12 are both of type 
SF. Then the selection matrix H is given by 

H = 


/ 

1 

0 

0 

0 

0 

0 








0 

1 

0 

0 

0 

0 




0 




0 

0 

1 

0 

0 

0 








0 

0 

0 

1 

0 

0 














1 

0 

0 

0 

0 

0 








0 

1 

0 

0 

0 

0 




0 




0 

0 

1 

0 

0 

0 

V 







0 

0 

0 

1 

0 

0 
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thus the matrices G T G E 8x6 and J G R 8x5 are con¬ 
structed by removing rows 5, 6 , 11, and 12 from G T 
and J 


/-C| 

-Sl 

0 

0 

0 

°\ 

S\ 

-Cl 

0 

0 

0 

—r 

0 

0 

1 

r s\ 

— rci 

0 

0 

0 

0 

“Cl 

~s 1 

0 

-c 2 

-S2 

0 

0 

0 

0 

Si 

-Cl 

0 

0 

0 

—r 

0 

0 

1 

rsi 

—r C2 

0 

V 0 

0 

0 

-ci 

-Sl 

0/ 

( ~ h 

-h 




\ 

h 

h 



0 


0 

0 





0 

0 







0 


0 

0 



0 


0 

k 

0 


Z 4 

#(Z 4 + / 5 ) 

0 

V 


0 


■v/2 

2 

0 / 


(38.77) 


(38.78) 


Notice that changing the contact models is easily ac¬ 
complished by removing more rows. Changing con¬ 
tact 1 to HF would eliminate the fourth rows from G T 
and J. Changing it to PwoF would eliminate the second, 
third and fourth rows of G T and J. Changing the model 
at contact 2 would remove either just the eighth row or 
the sixth, seventh, and eighth rows. 


Example 1, Part 3: Reduce to Planar Case 
The grasp shown in Fig. 38.12 can be reduced to 
a planar problem by following the explicit formulas 
given above, but it can also be done by understand¬ 
ing the physical interpretations of the various rows and 
columns of the matrices. Proceed by eliminating veloc¬ 
ities and forces that are out of the plane. This can be 
done by removing the z-axes from {N} and {B}, and 
the o-directions at the contacts. Further, joints 3 and 4 
must be locked. The resulting G T and J are constructed 
eliminating certain rows and columns. G T is formed 
by removing rows 3, 4, 7, and 8 and columns 3, 4, 
and 5. J is formed by removing rows 3, 4, 7, and 8 and 
columns 3 and 4 yielding 


(-Cl 

-Sl 

°\ 

Sl 

-Cl 


r 

-Cl 

-Sl 

0 


V *2 

-Cl 


7 

(~h 

~h 

°\ 


h 

h 

0 


0 

0 

0 


V 0 

0 

k / 



(38.79) 


(38.80) 


Example 1, Part 4: Grasp Classes 
The first column of Table 38.11 reports the dimen¬ 
sions of the main subspaces of J and G for the sphere 
grasping example with different contact models. Only 
non-trivial null spaces are listed. 

In the case of two HF contact models, all four null 
spaces are non-trivial, so the system satisfies the condi¬ 
tions for all four grasp classes. The system is graspable 
because there is an internal force along the line segment 
connecting the two contact points. Indeterminacy is 
manifested in the fact that the hand cannot resist a mo¬ 
ment acting about that line. Redundancy is seen to exist 
since joint 3 can be used to move contact 2 out of the 
plane of the figure, but joint 4 can be rotated in the op¬ 
posite direction to cancel this motion. Finally, the grasp 
is defective, because the contact forces and the instanta¬ 
neous velocities along the b\ and n 2 directions of con¬ 
tact 1 and 2 , respectively, cannot be controlled through 
the joint torques and velocities. These interpretations 
are borne out in the null space basis matrices below, 
computed using r=l, cos($i) = — 0.8 = — cos( 0 2 ), 
sin(0i) = cos(0 2 ) = -0.6, Z, = 3, Z 2 = 2, Z 3 = 1, Z 4 = 
2, Z 5 = 1, Z 6 = 1, and Z 7 = 0 


N(J) : 


N(G) i 


0 

0 

-0.73 

0.69 

V 0 ) 

( 0.57\ 
-0.42 
0 

0.57 

0.42 

0 


N(G T ) : 


( 0 \ 
0 

0.51 

0.86 

0 

V 0 ) 


N(J T ) = 


/o 

0 

0 

1 

0 

Vo 


0 \ 
0 

-1 

0 

0 

0 / 


(38.81) 


(38.82) 


Table 38.11 Dimensions of main subspaces and classifica¬ 
tions of grasp studied in Example 1 


Models 

Dimension 

Class 

HF,HF 

dim 77 (J) = 1 
dim TV" (G T ) = 1 
dim^V(G) = 1 
dim 277 (J T ) = 2 

Redundant 

Indeterminate 

Graspable 

Defective 

SF,HF 

dimW(J) = 1 
dim^A/"(G) = 1 
dim77(J T ) = 3 

Redundant 

Graspable 

Defective 

HF,SF 

dim 77(G) = 1 
dim 77 (J T ) = 2 

Graspable 

Defective 

SF,SF 

dim 77(G) = 2 
dim77(J T ) = 3 

Graspable 

Defective 
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Notice that changing either contact to SF makes it 
possible for the hand to resist external moments applied 
about the line containing the contacts, so the grasp loses 
indeterminacy, but retains graspability (with squeezing 
still possible along the line of the contacts). However, 
if contact 2 is the SF contact, the grasp loses its re¬ 
dundancy. While the second contact point can still be 
moved out of the plane by joint 3 and back in by joint 4, 
this canceled translation of the contact point yields a net 
rotation about M 2 (this also implies that the hand can 
control the moment applied to the object along the line 
containing the contacts). Changing to SF at contact 2 
does not affect the hand’s inability to move contact 1 
and contact 2 in the b\ and hj directions, so the defec- 
tivity property is retained. 

Example 1, Part 5: Desirable Properties 
Assuming contact model types of SF and HF at con¬ 
tacts 1 and 2, respectively, G is full row rank and 
so 2AT(G t ) = 0 (see Table in part 4 of this example). 
Therefore, as long as the hand is sufficiently dexterous, 
it can apply any wrench in R 6 to the object. Also, if the 
joints are locked, object motion will be prevented. As¬ 
suming the same problem values used in the previous 
part of this problem, the matrix G T is 



Bases for the three non-trivial null spaces are 


N(J T ) 


N(J) 


/0 0 0 \ 

0 0 0 

0 0-1 
1 0 0 

0-10 
0 0 0 

\0 0 0 / 


(38.84) 



/ 0.57 \ 

( 0 \ 


-0.42 

0 


0 

-0.73 

, N(G)« 

0 

0.69 


0.57 

V 0 ) 


0.42 


l 0 ) 


(38.85) 


Since 2£(J) is four-dimensional and !N (G) is 
one-dimensional, the maximum dimension of 71 (J) + 
JsT (G) cannot be more than 5, and therefore, the hand 
cannot control all possible object velocities. For ex¬ 
ample, the contact velocity v cc = (000 0.8 0 0 0) T is 
in 2AT(J t ), and so cannot be controlled by the fingers. 
It is also equal to 0.6 times the third column of G T 
plus the 4th column of G T and therefore is in 2l(G T ). 
Since the mapping between 21(G) and 2£(G T ) is one- 
to-one and onto, this uncontrollable contact velocity 
corresponds to a unique uncontrollable object veloc¬ 
ity, v = (0 0 0.6 1 0 0). In other words, the hand cannot 
cause the center of the sphere to translate in the z- 
direction, while also rotating about the *-axis (and not 
other axes simultaneously). 

On the question of controlling all internal object 
forces, the answer is yes, since JAf(J T ) fl IAf(G) = 0. 
This conclusion is clear from the fact that IAf(G) has 
non-zero values in the first, second, and sixth posi¬ 
tions, while all columns of IAf(J T ) have zeros in those 
positions. 

Example 1, Part 6: Force Closure 
Again assume that contacts 1 and 2 on the grasped 
sphere were modeled as SF and HF contacts, respec¬ 
tively. Under this assumption, G is full row rank, and 
the internal force corresponds to equal and opposite 
contact forces. For frictional form closure to exist, the 
internal force must lie within the friction cones. Choos¬ 
ing r and the sines and cosines of 9\ and 62 as in 
example 1, part 4, frictional form closure can be shown 
to exist if both friction coefficients are greater than 
0.75. For this grasp, since ,7V”(J T ) IT H (G) = 0, fric¬ 
tional form closure is equivalent to force closure. 

The plot in Fig. 38.14 was generated by fixing /z 2 
at a specific value and varying /z 1 from 0.5 to 2.0. 
Notice that for /z 1 < 0.75, force closure does not ex¬ 
ist regardless of the value of /Z 2 . The metric increases 
smoothly until a specific value of /i 1 . From that point 
on, the friction coefficient at contact 2 is the limit¬ 
ing factor. To increase the metric further, /Z 2 must be 
increased. 

38.5.2 Example 2: Grasped Polygon 
in the Plane 

Part 1: G and J 

Figure 38.15 shows a planar hand grasping a polygon. 
Finger 1 (on the right) contains two joints numbered 1 
and 2. Finger 2 contains joints 3-7, which are numbered 
in increasing order moving from the palm distally. The 
inertial frame has been chosen to lie inside the object, 
with its x-axis passing through contacts 1 and 2, and 
collinear with the normal vector of contact 2. 
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Frictional form closure metric 



0 -- 

0.5 1 1.5 2 


1 , friction coefficient at contact 1 


Assuming HF contacts and all joints are active (i. e., 
not locked), J is 



(38.88) 


The first two columns of J T are the torques required to 
produce a unit force in the h\ and t\ directions at con¬ 
tact 1. The horizontal line through the matrix partitions 
the contributions for the first finger (the upper part) and 
second finger. Notice that both J T and G are full column 
rank. 


Fig. 38.14 Plot of force closure metric versus friction co¬ 
efficient on contact 1 



Fig. 38.15 Planar hand with 2 fingers and 7 joints grasping 
a polygonal object 

The rotation matrices are given by 


Ri = 



— 0 . 6 \ 
— 0.8 J 



(38.86) 


Assuming HF contacts, G is given as follows 


/ 

- 0.8 

- 0.6 

1 

0 \ 

G= 

0.6 

- 0.8 

0 

1 

l 

k 

~h 

0 

-k ) 


Notice that the first two columns of G correspond to 
the normal and tangential unit vectors at contact 1. The 
third and fourth columns correspond to contact 2. 


Example 2, Part 2: Grasp Classes 
This example clearly illustrates the physical qualities of 
the various grasp classes without introducing features 
that can cloud the descriptions. 

We now discuss the details of the four grasp classes 
using the previous planar example. During these dis¬ 
cussions it is useful to choose non-dimensional values 
for the parameters in the grasping system. The lengths 
were assumed to have the following values (the results 
are the same, regardless of a particular choice of units, 
so units are not specified) 

Zi = 2.89 , / 2 = 0.75 , I3 = 1.97 , (38.89) 

l 4 =0.80, (5 = 0.80 , If, = 0.90 , (38.90) 

(7 = 1.20, / 8 = 1.35 . (38.91) 


Redundant. Redundancy exists if N(J) is non-triv- 
ial. Assuming that both contacts are hard contacts and 
all the joints are active, rank(J) = 4, so W(J) is three- 
dimensional. A basis for 2AT(J) was obtained using 
Matlab’s null () function 


N (J) 


( 0 

0 

° \ 

0 

0 

0 

-0.49 

-0.31 

-0.27 

0.53 

0.64 

-0.17 

0.49 

-0.50 

-0.02 

-0.49 

0.50 

0.02 

V-0.02 

0.01 

0.95/ 


(38.92) 


Since the first two rows are zero, 'N(J) does not 
include motions of the first finger (on the right of the 
palm). To understand this, assume the object is fixed in 
the plane. Then the first finger cannot maintain sticking 
contact at contact 1 unless its joints are also fixed. 
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The three non-zero columns corresponding to fin¬ 
ger 2 show that there are 3 basis motions of its joints 
that allow the finger contact to stick to the object con¬ 
tact. For example, the first column shows that if joint 3 
moves roughly as much as joints 4, 5, and 6, but in the 
opposite direction as joints 4 and 5 and in the same di- 
rectionas joint 6, while joint 7 is more or less fixed, then 
contact 2 will be maintained. 

Notice that finger 2 contains a parallelogram. Be¬ 
cause of this geometry, one can see that the vector 
(0 0 0 — 1 1 — 1 1) T is an element of H (J). The veloc¬ 
ity interpretation of this vector is that the link of the 
finger connected to the palm, and the link touching the 
object remain fixed in space, while the parallelogram 
moves as a simple four-bar mechanism. Similarly, joint 
actions in 2AT(J) do not affect the contact forces, but 
cause internal hand velocities. Also, notice that since 
/Af (J T ) = 0, the entire space of possible generalized ve¬ 
locities and forces at the contacts can be generated by 
the joints. 

Indeterminate. As noted above, with HF contact 
models, the system is graspable. However, replacing 
the HF models with PwoF models removes the tan¬ 
gent force components in the t\ and directions. 
This effectively removes columns 2 and 4 from G, 
which guarantees that the system will be indeterminate. 
The reduced matrix is denoted by G(i j3 ). In this case 

(^(1,3)) * S 


Nfe) 



(38.93) 


Physically, this basis vector corresponds to moving the 
object such that the point coincident with the origin of 
{N} moves directly downward, while the object rotates 
counter clockwise. Also, if the analogous force and mo¬ 
ment were applied to the object, the frictionless contacts 
could not maintain equilibrium. 


Recall that because the contact model is kinematic, 
there is no consideration of contact friction. However, 
given the direction of the contact normal relative to the 
line of the internal force, one can see that if the co¬ 
efficient of friction is not greater than 0.75, squeezing 
tightly will cause sliding at contact 1, thus violating the 
kinematic contact model. 

Defective. In a defective grasp, 2AT(J t ) f 0. Given 
that the original J is full row rank, the grasp is not de¬ 
fective. However, it can be made defective by locking 
a number of joints and/or changing the hand’s config¬ 
uration so that J is no longer full rank. For example, 
locking joints 4, 5, 6, and 7 makes finger 2 a single¬ 
link finger with only joint 3 active. In this new grasping 
system, ,, 3) is simply the first three rows of the orig¬ 
inal J T given in (38.88), where the subscript is the list 
of indices of active joints. The null space basis vector 
is 


N(J[ 1i2 . 3) ) 


M 

_o_ 

0 


\v 


(38.95) 


This grasp is defective, since there is a subspace 
of contact velocities and forces that cannot be con¬ 
trolled by joint generalized velocities and forces. Since 
only the last component of N(J^ , 3) ) is non-zero, it 
would be impossible for the hand to give the contact 
point 2 on the object a velocity in the G-direction while 
maintaining the contact. This is also clear from the ar¬ 
rangement of joint 3, contact 2, and the direction of the 
contact normal. The dual interpretation is that forces 
in 2AT(J t ) are resisted by the structure and the corre¬ 
sponding joint loads is zero, or equivalently that those 
forces are not controllable by the hand. Notice that if 
the model of contact 2 were changed to point-without- 
friction, then N(J^ 2 3 >) = 0 and the system would no 
longer be defective. 


Graspable. With two HF contact models in force, 
rank(G) = 3, so /AT(G) is one-dimensional and the sys¬ 
tem is graspable. The null space basis vector of the 
grasp matrix is 


N(G) 


/ 0.57\ 
0.42 

0.71 

V 0 / 


(38.94) 


The physical interpretation of this basis vector is two 
opposing forces acting through the two contact points. 


38.5.3 Example 3: Hyperstatic Grasps 

Part 1: G and J 

Figure 38.16 shows a planar projection of a three- 
dimensional sphere of radius / grasped by one finger 
only, with 3 revolute joints, through 3 contacts. The 
frames {C} 1 , {C} 3 and {C} 3 are oriented so that their 
o-directions point out of the plane of the figure (as indi¬ 
cated by the small filled circle). The axes of the frames 
{N} and {B\ were chosen to be axis-aligned with coin¬ 
cident origins located at the center of the sphere. The 
Z-axes are pointing out of the page. Observe that since 
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Fig. 38.16 A sphere grasped by a finger with three revo¬ 
lute joints. The force direction A/, ( dashed line) is a force 
that belongs to both T7V (G) and T7V (J T ) and causes hyper- 
staticity 

the three joint axes of the finger are perpendicular to 
the (x,y)-plane, the grasp operates in that plane for all 
time. 

Assume that the width of all the links of the robotic 
hand is zero. Rotation matrices R, and vectors c, — p for 
i = 1,.... 3, can be computed as in (38.66) and (38.67) 
considering that 6\ = jc , for contact 1 , 62 = it / 2 and 
@3 = 0, for contact 2 and 3, respectively. Finally, the 
complete grasp matrix is 

G t = (G, G 2 G 2 ) T g R 18x6 , 
where G, is as defined in (38.68) 


/ 1 

0 

0 

0 

0 

0 \ 

0 

1 

0 

0 

0 

-1 

0 

0 

1 

0 

/ 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

1 

0 

-1 

0 

0 

0 

0 

1 

0 

0 

0 

0 

-/ 

0 

0 

1 

/ 

0 

0 

0 

0 

0 

0 

-1 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

1 

-1 

0 

0 

0 

0 

0 

0 

-1 

0 

0 

0 

-/ 

0 

0 

1 

0 

-l 

0 

0 

0 

0 

-1 

0 

0 

0 

0 

0 

0 

-1 

0 

0 

0 

0 

0 

0 

1 


Construction of the complete hand Jacobian J, for con¬ 
tact i requires knowledge of the joint axis directions and 


the origins of the frames fixed to the links of each fin¬ 
ger. Assume that the origins of the Denavit-Hartenberg 
(DH) frames lie in the plane of the figure. In the current 
configuration, the quantities of interest for contact 1 , ex¬ 
pressed directly in {TV} are 

ct-?t = (0 / 0)\ 

z\ = (0 0 l) T . 

The quantities of interest for contact 2, in {TV} are 

C 2-$1 = (/ 2 / 0 ) x , 

c 2 -$2 = (/ 0 0 ) T , 

Z] = (0 0 1 ) T , 

Z 2 =( 0 0 1 ) T . 

The quantities of interest for contact 3, in {TV} are 


(N 

II 

•*s^ 

1 

1 

o) T , 

<N 

II 

CN 

1 

0 

-/ 

0) T 

Ci 

u> 

1 

II 

^0 

-/ 

o) T , 

Zl = (0 

0 

1) T > 

Zl = (0 

0 

1) T > 

t^> 

II 

^0 

0 

1) T - 


The complete hand Jacobian J e R 1Sx3 (contact ve¬ 
locities are expressed in {C},) is 


/ ~ l 

0 

0 \ 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

-l 

-1 

0 

-21 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

0 

/ 

-1 

-l 

—21 

-21 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

1 
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The horizontal dividing lines partition J into J 1 (on top), 
J 2 , and J 3 (on the bottom). The columns correspond to 
joints 1 through 3. 

Example 3, Part 2: G and J 
Assume that the 3 contacts in Fig. 38.16 are of type HF. 
Then the selection matrix H is given by 


/I 0 0 0 0 0\ 

H = I 0 0 I 0 0 0 , (38.96) 

\0 0 0 0 I 0/ 

where I and 0 are in R 3x3 , thus matrices G T e R 9x6 
and J G R 9x3 are obtained by removing rows related to 
rotations from G T and J 


/ 1 

0 

0 

0 

0 

0 \ 

0 

1 

0 

0 

0 

-1 

0 

0 

1 

0 

/ 

0 

0 

-1 

0 

0 

0 

0 

1 

0 

0 

0 

0 

-1 

0 

0 

1 

/ 

0 

0 

-1 

0 

0 

0 

0 

0 

0 

-1 

0 

0 

0 

-/ 

\0 

0 

1 

0 

-/ 

0 / 


( l 

0 

°\ 

0 

0 

0 

0 

0 

0 

1 

/ 

0 

21 

0 

0 

0 

0 

0 

1 

l 

l 

21 

21 

0 

Vo 

0 

0/ 


Example 3, Part 3: Grasp Classes 
The first column of Table 38.12 reports the dimen¬ 
sions of the main subspaces of J T and G for the sphere 
grasping example with three hard finger contacts. Only 
non-trivial null spaces are listed. 

The system is defective because there are general¬ 
ized contact forces belonging to the subspace that are 
resisted by the structure, which correspond to zero joint 


Table 38.12 Dimensions of main subspaces and classifica¬ 
tion of grasp given in Example 3 


Dimension 

Class 

dim^\T(J T ) = 6 

Defective 

dimjy(G) = 3 

Graspable 

dim^y(j T )nw(G)= 1 

Hyperstatic 


actions 


N(J T ) = 


0 

0 

0 

0 

0 

1 

1 

0 

0 0 jl 

u 1 

0 

0 

0 

0 

0 

0 

0 

-2 

0 

0 

0 

0 

1 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

A 

0 

A 

0 

A 

0 

A 

1 

A , 


The first three columns represent generalized forces 
acting at the three contact points in a direction perpen¬ 
dicular to the plane of the Fig. 38.16. The fourth column 
corresponds to a contact force applied only along the t\ 
direction. 

System is graspable because the subspace of internal 
forces is three-dimensional; a possible basis matrix is 



( 1 

1 

°\ 


1 

0 

1 


0 

0 

0 


1 

0 

2 

N(G) = 

-1 

0 

0 


0 

0 

0 


0 

1 

0 


0 

0 

-1 


\0 

0 

0/ 


The three force vectors of subspace N(G) are easily 
identified from Fig. 38.16. Note that all forces are ex¬ 
pressed in local contact frames. The first column vector 
of N(G) represents opposed forces at contacts 1 and 2 
along the line joining contacts 1 and 2. The second col¬ 
umn vector parameterizes opposed forces at contacts 1 
and 3 along the line joining contacts 1 and 3. The last 
vector represents forces along direction A/,, shown as 
the (dashed lines) in Fig. 38.16. Note that this direction 
(in wrench intensity space) corresponds to two upward 
friction forces at the left and right contacts and one 
downward with double the magnitude from the center 
of the top link in the work space. 

Finally, the grasp is hyperstatic because 


N(G)nN(J T ) 


f°\ 

1 

0 

2 

0 

0 

0 

-1 


/0. 


\o/ 
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Hyperstatic forces in this subspace, are internal forces 
that cannot be controlled through the hand joints. In 
Fig. 38.16 the internal force A/, that is also in N(J T ) 
is reported. 

The grasp in Fig. 38.16 is an example of power 
grasp, style of grasp mentioned earlier that uses many 
contact points not only on the fingertips but also on the 
links of fingers and the palm [38.8, 28, 33]. 

All power grasps are kinematically defective 
(2AT(J t ) 0) and usually are hyperstatic. According to 

Sect. 38.2.2, rigid-body modeling is not sufficient to 
capture the overall system behavior, because the gen¬ 
eralized contact forces in N (G) d N (J T ) leave the 
dynamics indeterminate. 

Many approaches have been used to overcome 
rigid-body limitation in hyperstatic grasps such as those 
proposed in [38.16,20,21] where visco-elastic contact 
models have been used to solve the force indeterminacy. 
In [38.48], authors found that a sufficient condition for 
hyperstaticity is m > q + 6 where m is the dimension of 
contact force vector. 

38.5.4 Example 4: Duality 

Consider a frictionless disc constrained to translate in 
the plane; (Fig. 38.17). In this problem n v = 2, so the 
space of applied contact forces and object velocities is 


2-D grasp 
of a disc 




Force and 
velocity cones 



L Velocity cone is 
half plane 



3 



Force cone is 
the plane 


2 

Velocity cone is 
the origin 


Fig. 38.17 Case of a translating disc in the plane: Rela¬ 
tionship between frictionless contacts and possible disc 
velocities and net contact forces 


the plane R 2 . In the top pair of pictures, a single (fixed) 
contact point imposes a half space constraint on the in¬ 
stantaneous velocity and limits the force at a frictionless 
contact to the ray. Both the ray and the (dark gray) half 
space are defined by the contact normal pointing into 
the object. Notice that the ray and half space are dual 
cones. When two contacts are present, the (light gray) 
force cone becomes the non-negative span of the two 
contact normals and the velocity cone is its dual. With 
the addition of the third contact, the grasp has form clo¬ 
sure as indicated by the degeneration of the velocity 
cone to the origin and the expansion of the force cone 
to become equal to the plane. 

It is important to point out that the discussion of 
the dual cones applies to three-dimensional bodies after 
replacing the contact normals with the columns of G. 

38.5.5 Example 5: Form Closure 

Parti: Unilateral Constraints 
Form closure of a spatial object requires seven unilat¬ 
eral contacts, which is difficult to illustrate. Therefore, 
the only form closure example analyzed in this chap¬ 
ter is the planar problem shown in Fig. 38.18. In the 
plane only four unilateral contacts are needed. In this 
problem, even though the fourth contact is at a vertex 
of the object, the contact normal is still well-defined. 
The angle a of the finger is allowed to vary, and it can 
be shown that form closure exists if a lies in the in¬ 
terval: 1.0518 < a < y. Notice that a critical value of 
a occurs when the lower edge of C 2 contains contact 
point 3 (a « 1.0518) and contact point 2 (a = y). Be¬ 
yond these angles, the cone C 1 and C 2 can no longer see 
each other. 

Choosing the frame for analysis with origin at the 
fourth contact point, the grasp matrix for this example 




Fig. 38.18 Plot of closure metrics versus angle of contact 
for 1.04 < a < 1.59 
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Fig. 38.19 Planar grasp with first-order form closure if 
1.052 < a < f 

is 

/0 0 1 — cos(a)\ 

G = [ 1 1 0 — sin(a) 1 . (38.97) 

V-/ 0 \l 0 j 

Form closure was tested for a range of angles. The 

blue curve in Fig. 38.19 is the LP1 metric ( d**n c 

returned by the function formClosure.m, available 
from [38.43]), which indicates that the grasp farthest 
from losing form closure has a 1.222 radians, which 
is the configuration shown in Fig. 38.18. For compari¬ 
son purposes, the Ferrari-Canny metric (scaled to have 
the same maximum as the LP1 metric) is plotted in red. 
They agree on the angles for which form closure exists, 
but differ on the optimal a. 

Example 5, Part 2: Bilateral Constraints 
Let us next consider form closure with a mix of bilateral 
and unilateral constraints using (38.54). For example, 
if contact 1 is treated as a bilateral constraint and the 
remaining are considered to be unilateral, G„ and B are 

/ 0 \ /0 1 — cos(a)\ 

B = I 1 J , G n = I 1 0 — sin(o!) 1 . 

V-z./ \0 \l 0 ) 

(38.98) 

Additional contacts can be converted to bilateral by 
moving their corresponding columns from G n to B. Fig¬ 
ure 38.20 shows five plots of the form closure metric 
LP1. In the legend, 4 refers to the grasp with all four 
contacts treated as unilateral, 5 means that contact 1 
has been converted to bilateral as defined by the previ¬ 
ous equation, 6 means that contacts 1 and 2 have been 


LP1 metric with bilateral constraints 



Fig. 38.20 First-order form closure metric for Fig. 38.18 
with progressively more contacts converted to bilateral 




Fig. 38.21 Comparison of the monotonicity properties of 
the LP1 and Ferrari-Canny form closure metrics 


converted to bilateral, etc. The plots show an important 
property of the LP1 metric; the maximum value is equal 
to the number of unilateral contacts (bilateral contacts 
count as two unilateral contacts), which is attained for 
curves 7 and 8. For this particular problem, these max¬ 
ima are achieved for all a. 

Example 5, Part 3: Monotonicity 
If one were to zoom in on curves 4 and 5 near a = 1.5, 
one would see that LP1 is not monotonic as constraints 
are added. That is, when contact 1 is converted from 
unilateral (curve 4) to bilateral (curve 5), for some val¬ 
ues of a, the LP1 metric is smaller despite the additional 
constraint. To demonstrate monotonicity, the mono¬ 
tonic Ferrari-Canny metric and the non-monotonic LP1 
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metric were computed as a function of a for a sequence 
of grasps. Starting with the original four contacts, five 
unilateral contacts were added: {(0, —1.3/), (—0.5/, 0), 
(0,1.75/), (—/, —0.6/), (-0.7/, 0)}. Figure 38.21 shows 
the metrics with 4, 5, 6, 7, 8, and 9 contacts, as indicated 
by the legend. 

The LP1 metric does not increase monotonically as 
contacts are added for every a. Near a = 1.2 adding 


the fifth contact reduces the metric slightly. Also, near 
a = 1.55, adding the eighth contact reduces the metric. 
By contrast, the Ferrari-Canny metric is monotonic, al¬ 
though not strictly monotonic; the plots labeled 6 and 
7 are identical for all a and there are other intervals of 
a where the metric is constant as contacts are added. 
For example, near a = 1.35, the grasps with 5, 6, and 7 
contacts all have the same metric value. 


38.6 Conclusion and Further Reading 

A great deal of understanding of grasping systems can t 
be derived from the simple linear kinematic, dynamic, 1 
and contact models presented in this chapter. The most ; 
widely used grasp classifications and closure proper¬ 
ties can all be derived from these models under the ; 
rigid-body assumption. Linearizing these models leads 1 
to metrics and tests that can be computed efficiently t 
using computational linear algebra and linear program- 1 
rning techniques. Grasp synthesis tools built on these t 
tests take object and hand models as inputs and return < 
a set of possible grasp configurations as outputs (see for / 
example [38.3]). In-depth discussions of grasp kinemat- ( 
ics and grasp classifications can be found in [38.15,20, t 
20,34,48-52], 1 

One has to wonder what insights have been lost as 1 
a result of the simplifying assumptions made in this 1 
chapter. For the interested reader, there are a host of ( 
papers that analyze grasping systems under more so- 1 
phisticated assumptions. In general, bodies are curved 1 
and compliant [38.17,28,35,53-55] and contact fric¬ 
tion models are not quite as simple as the linearized t 
ones so widely adopted. For example, if a contact has 1 
to resist a moment about its normal, its effective tan- t 
gential friction coefficient is reduced [38.46,56]. In < 
this chapter, the quadratic Coulomb friction cone was 1 
approximated by a polyhedral cone. The analysis prob- 1 
lems are more difficult when using the quadratic cone, ( 
but they are quite tractable [38.47, 57]. 1 

In principle, a properly designed grasping system i 
could be controlled to maintain all contacts, but worldly 1 
realities can lead to unwanted slipping or twisting. This 1 
leads us back to the topic of grasp stability, which i 
is too often equated to grasp closure. However, grasp I 
closure is really equivalent to the existence of equilib- ( 
rium, which is a necessary, but not sufficient, condition j 
for stability. The common definition of stability out- t 
side of the field of robotic grasping requires that when 1 
a system is deflected from an equilibrium point, the sys¬ 
tem returns to this point. From this perspective, and t 
under the assumption of no slipping at the contact 
points, it is known that all closure grasps are stable, but 1 


the converse is not necessarily true [38.53,54,58,59]. 
However, stability analysis when contacts slide is still 
an open question. 

Given a stable grasp, another important consider¬ 
ation not discussed in this chapter is that of grasp 
force distribution problem, i. e., finding good actuator 
torques and contact forces to balance a given external 
load applied to the object. This problem was studied in 
the context of walking machines first by McGhee and 
Orin [38.60] and later by several others [38.61,62]. Ku¬ 
mar and Waldron applied similar techniques to force 
distribution problems in grasping [38.63]. Work by Han 
et al. and Buss et al. solved the force distribution prob¬ 
lem with non-linear friction cone constraints by taking 
advantage of convex optimization techniques [38.47, 
57, 64]. In power grasps, this problem of finding a good 
distribution is difficult, because the space of control¬ 
lable contact wrenches is severely restricted by the large 
number of contacts [38.21, 32, 33, 65]. 

Grasp synthesis largely depends on the structure of 
the robotic hands that are often very complex systems 
with many degrees of freedom, sensors and actua¬ 
tors, which are necessary to adapt to many different 
objects and tasks. An important research area, not dis¬ 
cussed in this chapter, is that of designing simplified 
hands coupling some of the degrees of freedom, re¬ 
ducing the number of effective inputs, and leading to 
more efficient, simpler and reliable designs [38.23, 66]. 
A reduction of independent inputs is observed also in 
human hand movement data, where few variables, de¬ 
fined as postural synergies, explain most of the variance 
in the configurations of the hand while grasping dif¬ 
ferent objects [38.25,67]. The reduction of the number 
of independently controlled inputs in the hand affects 
grasp properties, and in particular the ability of the hand 
to dexterously controlling grasp forces and in-hand ma¬ 
nipulation as discussed in [38.68-71] 

All of the above considerations implicitly assume 
that a grasp has been achieved, which is no easy task. 
The bulk of today’s research in robotic grasping can 
be fairly characterized as focused on grasp acquisition. 
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In other words, the problem is to move the hand from 
a state of no contact with the object to one in which 
a satisfactory grasp has been achieved. When a robot 
identifies an object to grasp, its knowledge of the ob¬ 
ject’s pose and geometry are not perfect. Even if they 
were, the robot’s control system could not move the 
hand perfectly to the desired grasping points. The hand 
will bump the object accidentally, altering its pose, pos¬ 
sibly leading to grasp failure. 

Areas of current research cover methods that ex¬ 
ploit some detailed aspects of planned or sensed con¬ 
tact interactions that occur before achieving the final 
grasp and those that try to be robust to pre-grasp con¬ 
tact interactions. In the first category are quasistatic 
push-grasping, dynamic grasp acquisition, and percep¬ 
tion. Push-grasping seeks out contact prior to wrapping 
the fingers in order to allow the object to settle into 
a good position against the palm. It has, so far, been 
applied to objects that can slide stably across a hor¬ 
izontal surface cluttered with object that are not to 
be grasped [38.72]. Dynamic planning combines a dy¬ 
namic model of grasping that includes intermittent con¬ 
tact to design optimal controllers and grasping actions 


simultaneously [38.73]. The performance of both of the 
above in real-time implementations could be improved 
by new grasp perception methods, which estimate the 
pose of the object relative to the hand [38.74-76]. 

Impedance controllers are being developed to re¬ 
duce the negative effects of unexpected contacts be¬ 
tween the hand and object as the grasp is being 
formed [38.77] and between the object and environ¬ 
ment during object transport tasks [38.78]. Independent 
contact regions are surface patches on the object which 
have the property: if a contact point is stabilized any¬ 
where inside each region, then the grasp will have 
force closure [38.79,80]. With this approach, a small 
amount of jostling will not cause grasp failure. Perhaps 
caging takes the independent regions idea to the ex¬ 
treme. Here the goal is to find a configuration of the 
hand that loosely surrounds the object with the addi¬ 
tional condition that the object cannot escape without 
deforming itself or the hand [38.81, 82]. The challenges 
in caging are in finding a pre-cage configuration and 
a finger motion plan that impose the least restrictive 
accuracy requirements on the robots perception and 
control systems. 


Video-References 


Grasp analysis using the MATLAB toolbox SynGrasp 

available from http://handbookofrobotics.org/view-chapter/04/videodetails/551 
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39. Cooperative Manipulation 


Fabrizio Caccavale, Masaru Uchiyama 


This chapter is devoted to cooperative manipula¬ 
tion of a common object by means of two or more 
robotic arms. The chapter opens with a historical 
overview ofthe research on cooperative manipula¬ 
tion, ranging from early 1970s to very recent years. 
Kinematics and dynamics of robotic arms cooper¬ 
atively manipulating a tightly grasped rigid object 
are presented in depth. As for the kinematics and 
statics, the chosen approach is based on the so- 
called symmetric formulation; fundamentals of 
dynamics and reduced-order models for closed 
kinematic chains are discussed as well. A few spe¬ 
cial topics, such as the definition of geometrically 
meaningful cooperative task space variables, the 
problem of load distribution, and the definition of 
manipulability ellipsoids, are included to give the 
reader a complete picture of modeling and evalua¬ 
tion methodologies for cooperative manipulators. 
Then, the chapter presents the main strategies 
for controlling both the motion ofthe coopera¬ 
tive system and the interaction forces between 
the manipulators and the grasped object; in de¬ 
tail, fundamentals of hybrid force/position control, 
proportional-derivative (PD)-type force/position 
control schemes, feedback linearization tech¬ 
niques, and impedance control approaches are 
given. In the last section further reading on ad¬ 
vanced topics related to control of cooperative 
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robots is suggested; in detail, advanced nonlinear 
control strategies are briefly discussed (i.e., intel¬ 
ligent control approaches, synchronization control, 
decentralized control); also, fundamental results 
on modeling and control of cooperative systems 
possessing some degree of flexibility are briefly 
outlined. 


It was not long after the emergence of robotics tech¬ 
nologies that multi-arm robot systems began to be 
investigated by robotics researchers. In the early 1970s, 
research on this topic had already started. This inter¬ 
est was mainly due to typical limitations in applications 
of single-arm robots. It has been recognized, in fact, 
that many tasks that are difficult or impossible to ex¬ 
ecute by a single robot become feasible when two 
or more manipulators are employed in a cooperative 


way. Such tasks include, for instance, carrying heavy 
or large payloads, the assembly of multiple parts with¬ 
out using special fixtures, and handling of objects that 
are flexible or possess extra degrees of freedom. Re¬ 
search on this subject has been aimed at solving existing 
problems and opening up a new stream of applica¬ 
tions in flexible manufacturing systems as well as in 
poorly structured environments (e.g., outer space and 
undersea). 
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39.1 Historical Overview 

Examples of research work in the early days include 
that by Fujii and Kurono [39.1], Nakano et al. [39.2], 
and Takase et al. [39.3]. Already in those pieces 
of work important key issues in the control of 
multi-arm robots were investigated: master/slave con¬ 
trol, force/compliance control, and task-space control. 
In [39.1] Fujii and Kurono proposed to adopt a com¬ 
pliance control concept for the coordination of multiple 
manipulators; they defined a task vector with respect 
to the object frame and controlled the compliance ex¬ 
pressed in the same coordinate frame. An interesting 
feature of the work by Fujii and Kurono [39.1] and 
by Takase et al. [39.3] is that force/compliance con¬ 
trol was implemented by exploiting back-drivability of 
the actuators, without using force/torque sensors. The 
importance of this technique in practical applications, 
however, was not recognized at that time, and more 
complex approaches using force/torque sensors lured 
people in robotics. Nakano et al. [39.2,4] proposed 
a master/slave force control approach for the coordina¬ 
tion of two arms carrying an object cooperatively and 
pointed out the necessity of force control for coopera¬ 
tive multiple robots. 

In the 1980s, based on several fundamental theo¬ 
retical results for single-arm robots, strong research on 
multi-arm robotic systems was renewed [39.5], Def¬ 
inition of task vectors with respect to the object to 
be handled [39.6], dynamics and control of the closed 
kinematic chain formed by the multi-arm robot and the 
object [39.7,8], and force control issues, such as hy¬ 
brid position/force control [39.9-12], were explored. 
Through this research work, a strong theoretical back¬ 
ground for the control of multi-arm robots has been 
formed, providing the basis for research on more ad¬ 
vanced topics from the 1990s to today. 

How to parameterize the constraint forces/moments 
on the object based on the dynamic model of the whole 
cooperative system has been recognized as a critical 
issue; in fact, this parametrization leads to the defi¬ 
nition of task variables for the control and hence to 
an answer to one of the most frequently asked ques¬ 
tions in the field of multi-arm robotics: how to control 
simultaneously the trajectory of the object, the me¬ 
chanical stresses (internal forces/moments) acting on 
the object, load sharing among the arms, and even the 
external forces/moments on the object. Force decom¬ 
position may be a key to solving these problems and 
has been studied by Uchiyama and Dauchez [39.11, 
12] and Walker et al. [39.13] as well as Bonitz and 
Hsia [39.14]. In detail, devising a geometrically clear 
parametrization of the internal forces/moments acting 
on the object has been recognized as an important 


problem, solved in [39. 15, 16]. Several cooperative con¬ 
trol schemes based on the sought parameterizations 
have been designed, including control of motion and 
force [39.11, 12,17—19] and impedance/compliance 
control [39.20-22]. Other approaches include adaptive 
control [39.23,24], kinematic control [39.25], task- 
space regulation [39.26], joint-space control [39.27, 
28], and coordinated control [39.29]. 

Also, the definition of user-oriented task-space vari¬ 
ables for coordinated control [39.26, 30] and the devel¬ 
opment of meaningful performance measures [39.31- 
34] have been fruitfully investigated in 1990s. 

Load sharing among the arms is also an inter¬ 
esting issue on which many papers have been pub¬ 
lished [39.35-40]. Load sharing may be exploited both 
for optimal load distribution among arms and for robust 
holding of the object, when the object is held by the 
arms without being grasped rigidly. In both cases, any¬ 
how, this becomes a problem of optimization and can 
be solved by either heuristic [39.41] or mathematical 
methods [39.42]. 

Other research efforts have been focused on coop¬ 
erative handling of multibodied or even flexible ob¬ 
jects [39.43-45]. Control of multi-flexible-arm robots 
has been investigated [39.46-48], since the merits of 
flexible-arm robots can be exploited in cooperative 
systems [39.49], i. e., lightweight structure, intrinsic 
compliance and hence safety, etc. 

Robust holding of an object in the presence of slip¬ 
page of end-effectors on the object may be achieved as 
well, if the slippage is detected correctly [39.50], 

A more recent control framework for cooperative 
systems is so-called synchronization control [39.51, 
52]; in this class of approaches the control prob¬ 
lem is formulated in terms of suitably defined errors 
accounting for the motion synchronization between 
the manipulators involved in the cooperative task. As 
for the nonlinear control of cooperative manipula¬ 
tion systems, efforts have been spent on intelligent 
control (e.g., [39.53-55]) as well as on the investi¬ 
gation of control strategies in the presence of partial 
state feedback [39.56]. The problem of controlling un¬ 
deractuated cooperative manipulators is tackled, e.g., 
in [39.57], 

The problem of the implementation of cooperative 
control strategies on conventional industrial robots has 
attracted the increasing interest of the research commu¬ 
nity. In fact, the control units of industrial robots do 
not present all the features needed to implement non¬ 
linear torque control schemes, while the integration of 
force/torque sensing in standard industrial robot control 
units is often cumbersome and tends to be avoided in in- 
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dustry for many reasons: unreliability, cost, etc. Hence, 
the rebirth of the early methods, where force sensors 
were not used (Fujii and Kurono [39.1], lnoue [39.58]), 
has become attractive for industrial settings. Hybrid po¬ 
sition/force control without using force/torque sensors 
has been successfully implemented [39.59]. Interesting 
results on the implementation of effective cooperative 
control strategies for industrial robots are presented, 
e.g., in [39.60], where a design approach that makes 
use of tool-based coordinate systems, trajectory gen¬ 
eration, and distributed control of multiple robots is 
presented. Also, a task planning approach for multi-arm 
industrial robotic work-cells has been recently proposed 
in [39.30], together with a language for cooperative task 
programming. 

Another interesting aspect, related to the reliability 
and safety of cooperative manipulation systems, is in¬ 
vestigated in [39.61], where the use of nonrigid grippers 
is considered to avoid large internal forces and, at the 
same time, to achieve safe manipulation of the object, 
even in the presence of failures and unpredicted con¬ 
tacts with the external environment. 

It is worth mentioning the strict relationship be¬ 
tween issues related to grasping of objects by fin- 
gers/hands (widely described in Chap. 38) and those 
related to cooperative manipulation. In fact, in both 
cases, multiple manipulation structures grasp a com¬ 
monly manipulated object. In multifingered hands only 
some motion components are transmitted through the 


contact point to the manipulated object (unilateral con¬ 
straints), while cooperative manipulation via robotic 
arms is achieved by rigid (or near-rigid) grasp points 
and interaction takes place by transmitting all the mo¬ 
tion components through the grasping points (bilateral 
constraints). While many common problems between 
the two fields can be tackled in a conceptually similar 
way (e.g., kinetostatic modeling, force control), many 
other are specific of each of the two application fields 
(e.g., form and force closure for multifingered hands). 
Interestingly, a common frame for both cooperative and 
multifingered manipulation has been proposed [39.25]; 
namely rolling/sliding of the grasp points on the ob¬ 
ject is taken into account by modeling the contacts via 
rotational/prismatic joints; then, the desired manipula¬ 
tor/finger joints trajectories are derived from the desired 
object motion by using numerical inverse kinematics 
algorithms. In [39.62-64], contributions to modeling 
and control of cooperative manipulation systems in the 
presence of different types of manipulators/object con¬ 
tact can be found. Moreover, since cooperative manipu¬ 
lators can be often viewed as closed-chain mechanisms, 
a strong link with research related to parallel manipula¬ 
tors (Chap. 18) exists, especially from a modeling point 
of view. 

Finally, it is worth mentioning the important issue of 
cooperative transportation and manipulation of objects 
via multiple mobile robotic system [39.65,66], which 
is currently subject of investigation. 


39.2 Kinematics and Statics 


Consider a system composed by M manipulators, each 
equipped with Nj joints (i = 1,... ,M). Let /?, denote 
the (3x1) vector of the position of the i-th end-effector 
coordinate frame, Twith respect to a common base 
frame, 'T ; let R, denote the (3x3) matrix which ex¬ 
presses the orientation of Tj with respect to the base 
frame T. 

Both pi and R, can be expressed as a function of the 
(Ni x 1) vector of the joints variables of each manipula¬ 
tor qi through the direct kinematics equations 


j Pi =pMi ), 

( R = R/(<7,) • 


(39.1) 


Of course, the orientation of the end-effector frame may 
be expresses in terms of a minimal set of angles, i. e., 
a set of three Euler angles 0,. In this case, direct kine¬ 
matics can be expressed in terms of an operational space 
(pseudo-)vector x t as follows 


Pi(Qi)\ 


The linear /), and angular a>, velocity of the i-th end- 
effector can be collected in the (6x1) generalized 
velocities vector Vj = (pj coj) . Then, the differential 
direct kinematics can be expressed as 

v i = J i (q i )q i , (39.3) 

where the (6 x Ni) matrix J, is the so-called geometric 
Jacobian of the i-th manipulator (Chap. 2). When the 
velocity is expressed as the derivative of the operational 
space vector, differential kinematics takes a formally 
similar form 

3 ktiqd. T j • ,, nil 

Xi = —r- qi = Ja iiqoqi , (39.4) 

dqi 

where the (6 x Ai,) matrix Ja, is the so-called ana¬ 
lytical Jacobian of the i-th manipulator (Chap. 2). 

Let us consider the (6x1) vector of the generalized 
forces acting at the i-th end-effector 



Xj = ki(qi) 


(39.2) 


(39.5) 
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where /, and zi, denote the force and moment, re¬ 
spectively. By invoking the principle of virtual work, 
a relation dual to (39.3) can be derived 

T, = J J(qi)hj, (39.6) 

where r, is the (IV, x 1) vector of the forces/torques act¬ 
ing at the joints of the ;-th manipulator. 

For the sake of simplicity, consider a system of two 
cooperative robots (Fig. 39.1) manipulating a common 
object. Let C be a fixed point of the object (e.g., the cen¬ 
ter of mass of the object), whose position in the base 
frame is given by the vector pc', moreover, let Tc be 
a coordinate frame attached to object with origin in C. 
The virtual stick [39.11, 12] is defined as the vector r, 
(i = 1, 2) which determines the position of Tc with re¬ 
spect to Tj (i = 1,2); it is assumed that r, behaves as 
a rigid stick fixed to the i-th end-effector. Flence, each 
virtual stick, expressed in the frame Tj (or 'T 2 ), is a con¬ 
stant vector if the commonly grasped object is rigid and 
tightly grasped by each manipulator. In this case, the 
direct kinematics of each manipulator can be expressed 
in terms of a virtual end-effector frame Ts,; = Tc, hav¬ 
ing the same orientation of Tc and origin located in 
Ps,i = Pi + r i = Pc- Hence, the position and orientation 
at the tip of each virtual stick are given by (i = 1,2) 

Ps.i =Pc • Rs,; = Rc • 

The set of Euler angles corresponding to Rs,; will be 
denoted by the (3 x 1) vector 0s,/- Hereafter, it is as¬ 
sumed that the object grasped by the two manipulators 
can be considered rigid (or nearly rigid) and tightly (or 
nearly tightly) attached to each end-effector. Hence, the 
displacements between the above defined frames can be 
considered null or negligible. Otherwise, if the manip¬ 
ulated object is subject to deformation (e.g., in the case 



Fig. 39.1 Grasp geometry for a two-manipulator coopera¬ 
tive system manipulating a common object 


of a flexible object) and/or the grasp is not tight (e.g., 
in the case of compliant grippers) the above defined 
coordinate frames, will be subject to nonnegligible dis¬ 
placements with respect to each other. 

Let hsj denote the vector of generalized forces act¬ 
ing at the tip of the z-th virtual stick; it can be easily 
verified that the following equation holds 

'■*•' = (-Sfr.) £)*' = «*• (M ' 7) 

where O; and 1/ denote, respectively, the null matrix and 
the identity matrix of (Z x /) dimensions, and S(r,) is the 
(3 x 3) skew-symmetric matrix operator performing the 
cross product. It is worth noticing that W; is always full 
rank. 

By invoking the virtual works principle, a relation 
dual to (39.7) can easily be derived 

«i = (o, S if)»sj=w>s.<. M 

where is the generalized velocity vector of the vir¬ 
tual stick endpoint. When r, = 0, it is W, = 16; in other 
words, if the end-effector kinematics of each manipu¬ 
lator is referred to the corresponding virtual stick (or 
the object reduces to point), the forces and velocities at 
the two end-effectors coincide with their counterparts 
referred at the virtual sticks. 

39.2.1 Symmetric Formulation 

The kinetostatic formulation proposed by Uchiyama 
and Dauchez [39.12], i. e., the so-called symmetric for¬ 
mulation, is based on kinematic and static relationships 
between generalized forces/velocities acting at the ob¬ 
ject and their counterparts acting at the manipulators 
end-effectors (or at the virtual sticks endpoints). 

Let us define first the external forces as the (6x1) 
vector of generalized forces given by 

he = Z*s,i + Z*s ,2 = Ws hs , (39.9) 

with Ws = (16 16) and hs = (hj { ZzJ 2 ) T ; in other 
words, he represents the vector of generalized forces 
causing the object’s motion. From (39.7) and (39.9) 
it follows that he can be expressed in terms of end- 
effector forces as well 

/z E = W; /?, + W 2 h 2 = W/z , (39.10) 

with W = (Wi W 2 ) and h = (/z| ZzJ) T . It can be recog¬ 
nized that Ws (W) is a (6 x 12) matrix, having a six¬ 
dimensional (6-D) range space and a six-dimensional 
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null space, describing the geometry of the grasp, and 
thus is usually termed the grasp matrix. 

The inverse solution to (39.9) yields 

h s = wlh E + \ s h l = V s ho, (39.11) 

where Wl denotes the Moore-Penrose pseudoinverse 
ofW s 



Vs is a matrix whose columns are a basis of the null 
space of Ws, e.g., 

*-("£)■ 13,131 

ho = (h\ , and 

Us = (W| V s ) . (39.14) 

The second term on the right-hand side of (39.1 1), i. e., 
Vsh|, represents a vector of generalized forces, referred 
at the tip of the virtual sticks, which lies in the null 
space of Ws; thus, such forces do not contribute to 
external forces. Hence, the (6x1) vector h\ is a gen¬ 
eralized force which does not contribute to the object’s 
motion. Therefore, it represents internal loading of the 
object (i. e., mechanical stresses) and is termed the in¬ 
ternal forces vector [39.12]. A similar argument can be 
used for equation (39.10), whose inverse solution is 

h = W^e + V/tj = \Jh 0 , (39.15) 


where 

U= (W t V). 


(39.16) 


In [39.13] it has been shown that the first term on the 
right-hand side of (39.15) represents only contributions 
to external forces if the pseudoinverse of the grasp ma¬ 
trix is properly defined, i. e., 


W f = 


( ^3 

|S(ri) 

\h 

Us(r 2 ) 


o 3 \ 

5I3 

O 3 

\h) 


(39.17) 


As Vs in (39.11), the columns of V span the null space 
of W, and can be chosen as [39.31] 


( -I3 o 3 \ 

-S(n) -i 3 

I3 O3 

V S(r 2 ) I 3 / 


(39.18) 


A different parametrization of the inverse solutions 
to (39.9) and (39.10) can be expressed, respectively, as 

*s = wJ*e + (i 12 -WJWs)As (39.19) 

and 

h = W f h E + (li 2 -W t w)/t* , (39.20) 

where (h*) is an arbitrary (12 x 1) vector of general¬ 
ized forces acting at the tip of the i-th virtual stick (r-th 
end-effector) projected onto the null space of Ws (W) 
via Ii 2 - WjW s (Ii 2 - W+ W). 

By exploiting the principle of virtual work, the 
mappings between generalized velocities, dual to those 
derived above, can be established. In detail, the map¬ 
ping dual to (39.11) is 


tt 0 = U^ s , (39.21) 

where v$ = (ttj y t) T , Vo = (v E vJ) T . The vector 
v E can be interpreted as the absolute velocity of the ob¬ 
ject, while Vi represents the relative velocity between 
the two coordinate frames T 3,1 and Tf g j2 attached to the 
tips of the virtual sticks [39. 12]; this vector is null when 
the manipulated object is rigid and rigidly grasped. In 
a similar way, from (39.15), the following mapping can 
be devised 

•u 0 = U T tt , (39.22) 

where v = (vj u,) T . 

Moreover, a set of position and orientation vari¬ 
ables corresponding to v E and Vy can be defined as 
follows [39.12,25] 

Pe = ^(Ps.i +Ps, 2 ) . Pi =Ps.2-Ps.i , (39.23) 

Re = RiR'(£ 21 , # 21 /2) , R; 1 = R 2 , (39.24) 

where R, = R 1 R 2 is the matrix expressing the ori¬ 
entation of T 2 with respect to the axes of T), while 
£21 and &21 are, respectively, the equivalent unit vec¬ 
tor (expressed with respect to T)) and rotation angle 
that determine the mutual orientation expressed by Rj. 
Hence, Re expresses a rotation about k\ j by an angle 
which is half the angle needed to align T"> with Ty. 

In turn, if the orientation variables are expressed in 
terms of Euler angles, a set of operational space vari¬ 
ables can be defined as 

»-(s)- -te) 


(39.25) 


Part D | 39.2 







Part D | 39.2 


994 Part D 


Manipulation and Interfaces 


where 

0e = 2^s,i+0S.2) > 01 = 0S,2“0s,l • (39.26) 

It must be remarked, however, that the definitions 
in (39.26) keep a clear geometric meaning only if 
the orientation displacements between the virtual stick 
frames are small. In this case, as shown in [39.1 1, 12], 
the corresponding operational space velocities, xe and 
Xi, correspond to Ve and V\, respectively, with good 
approximation. Otherwise, if the orientation displace¬ 
ments become large, the variables defined in (39.26) 
do not represent any meaningful quantities, and other 
orientation representations have to be adopted, e.g., the 
unit quaternion (see Chap. 2 on kinematics for a gen¬ 
eral introduction to quaternions and Sect. 39.3 for an 
application to cooperative robots kinematics). 

Finally, by using (39.15) and (39.22), together with 
equations (39.3) and (39.6), the kinetostatic mappings 
between the force/velocities at the object and their 
counterparts in the joint space of the manipulators can 
be given 

r=f 0 h 0 , (39.27) 

v 0 = Jo?, (39.28) 

where r = (r| r?) T , q = (< 7 } q\) T > and 

j « =i,tj ' j =(i °‘) |39 - mi 

Formally similar mappings can be established in terms 
of operational space velocities i E andij [39.11, 12] and 
the corresponding operational space forces/moments. 

39.2.2 Multifingered Manipulation 

Hereafter some connections between the field of multi¬ 
arm cooperative manipulation, described in this chapter, 
and multifingered manipulation, described in Chap. 38, 
are briefly outlined, focusing on the kinetostatics of the 
two classes of manipulation systems. 

Both in the case of multi-arm cooperative systems 
and of multifingered manipulation, two or more ma¬ 
nipulation structures grasp a commonly manipulated 
object. 

Cooperative manipulation via multiple robotic arms 
is achieved by rigidly grasping the object (e.g., via rigid 


fixtures), and thus interaction takes place by exchanging 
both forces and moments at the grasp points. In other 
words, all the translational and rotational motion com¬ 
ponents are transmitted through the grasp points. 

On the other side, when a multifingered hand ma¬ 
nipulates an object, only some components of the mo¬ 
tion are transmitted through the contact points. This is 
effectively modeled via properly defined constraint ma¬ 
trices, whose expression depends on the type of contact. 
In other words, constraint matrices act as filters select¬ 
ing the components of the motion transmitted through 
each contact. In fact, as shown in Chap. 38, it is con¬ 
venient to think of the object-finger contact point as 
twofold: a point on the tip of the finger and a point on 
the object. Hence, two generalized velocity vectors are 
defined for the i-th contact point (both expressed with 
respect to frame Tj): the velocity of the contact point 
on the hand, v' h ; , and the velocity of the contact point 
on the object v' 0 r The corresponding dual generalized 
force vectors are h’ h ; and h' 0 ; , respectively. A contact 
model is then defined via the (m,- x 6) constraint ma¬ 
trix H,, selecting m, velocity components transmitted 
through the contact (v \ ; ), i. e., 

<i = H/<,. = H ; < ; . ( 39 . 30 ) 

The relation dual to (39.30) is 

HX,- =*w = <i. < 39 - 31 > 

where h \ ; is the vector of the transmitted generalized 
forces. Hence, (39.10) can be rewritten as 

h E = W,R,HXi +W 2 R 2 HT/r t 2 2 , ( 39 . 32 ) 

where R, = diagjR,. R,}. Hence, a conceptually simi¬ 
lar kinetostatic analysis can be developed, leading to 
the concept of external and internal forces (and related 
kinematic quantities) as well. 

However, it must be remarked that, while in the case 
of cooperative manipulators, tightly grasping the object, 
internal stresses are usually undesirable effects (unless 
controlled squeezing of a deformable object has to be 
achieved), in multifingered hands suitably controlled in¬ 
ternal forces are useful to guarantee a firm grasp even 
in the presence of external loading applied to the object 
(see the problem of form and force closure discussed 
in Chap. 38). 
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39.3 Cooperative Task Space 

The symmetric formulation reviewed in Sect. 39.2.1 
defines the fundamental variables and the primary re¬ 
lationships needed to describe the kinematics and the 
statics of the cooperative manipulators. However, the 
symmetric formulation was not originally conceived 
for task and motion planning of cooperative robotic 
systems, since this formulation effectively describes 
a cooperative system in terms of forces and veloci¬ 
ties at the virtual sticks, while the user is expected 
to plan the task in terms of geometrically meaningful 
variables describing the position and the orientation of 
a set of relevant coordinate frames. Thus, starting from 
equations (39.23) and (39.24), further research efforts 
have been devoted to define alternative kinematic for¬ 
mulations, explictly aimed at planning the coordinated 
motion of general multi-arm systems. A notable exam¬ 
ple of such alternative formulations is the task-oriented 
formulation for the definition of the cooperative task 
space originally proposed in [39.25,26]. By starting 
from (39.23) and (39.24), this formulation defines di¬ 
rectly the task variables in terms of the absolute and 
relative motion of the cooperative system, which can 
be directly computed from the position/orientation of 
the end-effector coordinate frames. 

Let us define as absolute coordinate frame, T" a , the 
frame whose position in the base frame is given by the 
position vector p a ( absolute position ) 

Pa = y(Pl+P2) ■ (39.33) 

The orientation of T” a with respect to the base frame ( ab¬ 
solute orientation ) is defined by means of the rotation 
matrix R a 

R a = R 1 R 1 (&2J, # 21 / 2 ) . (39.34) 

Of course, the sole absolute variables cannot 
uniquely specify the cooperative motion, since 12 vari¬ 
ables are required, e.g., for a two-arm system. Hence, in 
order to devise a complete description of the system’s 
configuration, the position/orientation of each manip¬ 
ulator relative to the other manipulators in the system 
must be considered. In detail, for a two-arm system, the 
relative position between the manipulators is defined as 

Pr=P2-Pl, (39.35) 

while the relative orientation between the two end- 
effector frames is expressed via the following rotation 
matrix 

(39.36) 


The variables R a , p r , and R t define the cooper¬ 
ative task space. Remarkably, R a and R* coincide with 
R e and R| , respectively. 

It is worth pointing out a useful feature of the above 
defined cooperative task space formulation. In fact, it 
can be recognized that definitions (39.33)-(39.36) are 
not based on any special assumption on the nature of 
the manipulated object and/or the grasp. In other words, 
the cooperative task space variables can be effectively 
used to describe cooperative systems manipulating non- 
rigid objects and/or characterized by a non-rigid grasp. 
Also, the same task-space variables can be used to 
describe a pure motion coordination task, i. e., when 
the arms are required to perform a coordinated motion 
without physically interacting via a commonly manipu¬ 
lated object. When the manipulators hold a rigid object 
(or a deformable object for which deformations are not 
commanded), then relative position and orientation are 
to be kept constant. Otherwise, if a relative motion be¬ 
tween the end-effectors is allowed, then p\ and Rj may 
vary according to the actual relative motion. 

As shown in [39.25,26], absolute 

Pa = -(Pi +P 2 ) , « a = ^(«i +« 2 ) , (39.37) 

and relative 

Pr = (P2-Pi), (o r = co 2 -«i, (39.38) 

linear and angular velocities can be readily derived, as 
well as their dual variables (i. e., absolute/relative forces 
and moments) 

/a =/i T f 2 , M a = «i+«2 , (39.39) 

/r = ^(f2-/i) , H r =^(«2-«l). (39.40) 

Kinetostatic mappings, analogous to those derived 
in the previous section, can be established between lin¬ 
ear/angular velocities (force/moments) and their coun¬ 
terparts defined at the end-effectors (or joints) level of 
each manipulator [39.25, 26]. 

Remarkably, the variables defined by the symmetric 
formulation and those used in the task-oriented formu¬ 
lation are related via simple mappings. In fact, forces 
(angular velocities, orientation variables) always coin¬ 
cide in the two formulations, while moments (linear 
velocities, position variables) coincide only when the 
object reduces to a point, or when the kinematics of 
each manipulator is referred to the tip of the corre¬ 
sponding virtual stick. 


R 1 = RR 2 = R\ 
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As shown in the following example, in the case of 
planar cooperative systems, the definition of the coop¬ 
erative task-space variables is straightforward. 

Example 39.1 (Cooperative Task-Space Variables for 
a Planar Two-Arm System) 

For a planar two-arm system the i-th end-effector coor¬ 
dinates can be defined via the (3x1) vector 



where p , is the (2x1) vector of the ;-th end-effector 
position in the plane and <p, is the angle describing its 
orientation (i. e., the rotation of the end-effector frame 
about an axis orthogonal to the plane). Hence, the task- 
space variables can be readily defined as 

*a = ~(x\ +* 2 ) , (39.41) 

x r =x 2 — Jti , (39.42) 

since the orientation of each end-effector is simply rep¬ 
resented by an angle. 


In the spatial case, the definition of the orientation 
variables in terms of Euler angles, as in (39.34) and 
(39.36), is somewhat critical; in fact, since T "1 and T 2 do 
not coincide in general, orientation displacements are 


likely to be large and definitions analogous to (39.26) 
are not correct. Hence, geometrically meaningful ori¬ 
entation representations have to be adopted, e.g., the 
unit quaternion (Chap. 2). In detail, by following the 
approach in [39.26], the orientation variables can be de¬ 
fined as follows. Let 



denote the unit quaternion extracted from R ! (k ] 21 , 
# 2 i/ 2 ); let, also, £1 = {rj u «i} and £ 2 = {r) 2 . * 2 } de¬ 
note the unit quaternions extracted from Ri and R 2 , 
respectively. Then, the absolute orientation can be ex¬ 
pressed in terms of quaternion product as follows 

£a = {>?a, fa}=£l*£k> (39.44) 

while the relative orientation can be expressed as the 
quaternion product 

Sr 1 = {'fc.«r 1 }=2r I *£2. < 39 - 45 > 

where = { 771 , — 61 } (i. e., the conjugate of <?i) rep¬ 
resents the unit quaternion extracted from Rf. 

Finally, it is worth mentioning that a more gen¬ 
eral cooperative task-space formulation is developed 
for different classes of multi-arm robotic systems 
in [39.30]. 


39.4 Dynamics and Load Distribution 

The equations of motion of the ;-th manipulator in a co¬ 
operative manipulation system are given by 

Mi(qi)qi + Ci(qi.qi) = r, -J J(qj)hj , (39.46) 

where M,(^,) is the symmetric positive-definite inertia 
matrix and c,(< 7 ,, < 7 ,) is the vector of the forces/torques 
due to the centrifugal, Coriolis, gravity, and friction ef¬ 
fects. The model can be expressed in compact form as 

M(q)q + c(q.q) = r - J T (q)h , (39.47) 

where the matrices are block-diagonal (e.g., M = 
blockdiag{Mi,M 2 }) and the vectors are stacked (e.g., 

9 = (<?! ?D T )- 

The object’s motion is described by the classical 
Newton-Euler equations of rigid body 

M^Re)^ + Ce(Re. we)«e = he = Wh , (39.48) 

where Me is the inertia matrix of the object and 
Ce collects the nonlinear components of the inertial 


forces/moments (i. e., gravity, centrifugal and Coriolis 
forces/moments). 

The above equations must be completed by impos¬ 
ing the closed-chain constraints arising from the kine¬ 
matic coupling between the two manipulators through 
the commonly manipulated rigid object. The constraints 
can be expressed by imposing a null internal velocity 
vector in the mapping (39.21) 

v I = VsWs = Ws,i-Vs,2 =0, (39.49) 

which can be expressed, by using (39.8) and (39.22), 
in terms of end-effector velocities (where the notation 
Wf T stands for (W?) -1 ) 

\ T v = -W7 t w 2 = 0, (39.50) 

and, finally, in terms of joint velocities 

V T J (q)q = Wr T Ji (?i)?i - W 2 ~ T J 2 (q 2 )q 2 = 0 . 

(39.51) 
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Equations (39.47), (39.48), and (39.51) represent 
a constrained dynamical model of the cooperative sys¬ 
tem in the joint space; Ah + N 2 generalized coordinates 
(i. e., q 1 and qi) are related to each other by the six alge¬ 
braic closed-chain constraints (39.51). This implies that 
the total number of degrees of freedom is N\ + N 2 — 6 
and the model has the form of a set of differential- 
algebraic equations. 

39.4.1 Reduced-Order Models 

The above derived dynamic model incorporates a set 
of closed-chain constraints, which reduces the number 
of independent generalized coordinates to N 1 +N 2 — 6. 
Hence, it is expected that, by eliminating six equations 
via the constraints (39.51), a reduced-order model can 
be obtained. Early work on reduced-order modeling 
of closed chains can be, e.g., found in [39.67]. Later, 
in [39.68, 69], this problem has been tackled by first de¬ 
riving, from (39.47), (39.48), and (39.51), a joint space 
model of the whole closed chain in the form 

M c (q)q + c c (q,q) = D c (?)r , (39.52) 

where Me, Dc, and cq depend on the dynamics of the 
manipulators and the object, as well as on the geometry 
of the grasp. The above model can be integrated, once 
the joint torques vector r is specified over an assigned 
time interval, to solve for the joint variables q (for¬ 
ward dynamics). However, the model cannot be used 
to find r from assigned q , q , and q ( inverse dynamics), 
since the ((Ah + Nf) x (N\ + Nf)) matrix Dc is not full 
rank [39.69], and thus the inverse dynamics problem 
turns out to be underspecified. 

In order to find a reduced-order model, composed 
by Ni + N 2 — 6 equations, a ((Ah + Ah — 6) x 1) pseu¬ 
dovelocity vector has to be considered 

v = B(q)q. (39.53) 

where the ((Ah + Ah — 6) x (Ah + Ah)) matrix B(^) is 
selected so that (A T (</) B T (<jr)) T is nonsingular and 

A (q) = Wj V T ( 9 ) . 

Then, the reduced-order model can be written in terms 
of the variables q , v, and v as 

'L T (q)M c (q)T(q) v + E T (q)c R (q, v) = E t ( 9 )t , 

(39.54) 

where E is an ((Ah + Ah) x (Ah + Ah — 6)) matrix such 
that 

(b) '=< nE| - 


and Cr depends on cq, E , and II [39.69]. The reduced- 
order model can be used for computing the forward 
dynamics; in this case, however, the problem of ex¬ 
pressing a reduced set of pseudocoordinates related 
to v in the numerical integration has to be consid¬ 
ered. Since E T is nonsquare, the inverse dynamics 
problem still admits infinitely many solutions in terms 
of r. However, this does not prevent the application 
model (39.54) to cooperative manipulators control (e.g., 
the decoupled control architecture proposed in [39.68, 
69]). 

39.4.2 Load Distribution 

The problem of load sharing in multi-arm robotic sys¬ 
tems is that of distributing the load among the arms 
composing the system (e.g., a strong arm may share the 
load more than a weak one). This is possible because 
a multi-arm system has redundant actuators; in fact, if 
a robotic arm is equipped with the number of actuators 
strictly needed for supporting the load, no optimization 
of load distribution is possible. In this section, this prob¬ 
lem is presented according to the results in [39.36-42]. 

We can introduce a load-sharing matrix in the 
framework adopted for presenting the kinematics of 
the cooperative manipulators. By replacing the Moore- 
Penrose inverse in equation (39.11) with a suitably 
defined generalized inverse, W(T, the following expres¬ 
sion can be devised for the generalized forces at the tip 
of the virtual sticks 

h s = Wfh E + \ s h[, (39.55) 

where 



and the matrix L is the so-called load sharing matrix. 
It can be easily proved that the nondiagonal elements 
of L only yield a h$ vector in the null space of Ws, 
that is, the space of internal forces/moments. Therefore, 
without losing generality, let us choose L such that 

L = diag {A} , (39.57) 

where the vector A = (A 1 ,..., Ag) T collects a set of 
constants A, representing the load-sharing coefficients. 

The problem is that of properly tuning the load¬ 
sharing coefficients to ensure correct cooperative ma¬ 
nipulation of the object by the arms. In order to answer 
this question, it must be noticed that, by combining 
equations (39.11) and (39.55), the following relation 
can be obtained 

hi = V f s (w^-W l)h E + h[, 


( 39 . 58 ) 
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which, bearing in mind that only h E and are really 
existing forces/moments, indicates that: 

• hi, h[, and A,- can be thought of as artificial pa¬ 
rameters, introduced for better understanding of the 

manipulation process. 

• h[ and A, are not independent; the concept of inter¬ 
nal forces/moments and the concept of load sharing 

are mathematically mixed with each other. 

Therefore, tuning the load-sharing coefficients or 
choosing suitable internal forces/moments is strictly 
equivalent from the mathematical (and, also, from the 
performance) point of view. Only one variable among 
hi, h [, and A is independent; hence, any of those redun¬ 
dant parameters, to be optimized for load sharing, can 
be exploited. This is more generally stated in [39.39, 
40]. In [39.42], internal forces/moments h\ are tuned, 
both for the sake of simplicity and for consistency with 
the adopted control laws. 

A very relevant problem related to load sharing is 
that of robust holding, i. e., the problem of determin¬ 
ing forces/moments applied to object by the arms, /is, 
required to keep the grasp even in the presence of dis¬ 
turbing external forces/moments. This problem can be 
solved by tuning the internal forces/moments (or the 
load-sharing coefficients, of course) and is addressed, 
e.g., in [39.41], where conditions to keep the grasp are 
expressed via end-effector forces/moments. Namely, by 
replacing hs in (39.55 ) into the equations expressing the 
grasp conditions, a set of linear inequalities for both h[ 
and A are obtained 

A L /ij + B l A < c L , (39.59) 

39.5 Task-Space Analysis 

As for single-arm robotic systems, a major issue in 
cooperative manipulation is that of task-space perfor¬ 
mance evaluation via the definition of suitable manipu- 
lability ellipsoids. These concepts have been extended 
to multi-arm robotic systems in [39.31]. Namely, by 
exploiting the kinetostatic formulation in Sect. 39.3, ve¬ 
locity and force manipulability ellipsoids are defined by 
regarding the whole cooperative system as a mechanical 
transformer from the joint space to the cooperative task 
space. Since the construction of force/velocity ellip¬ 
soids involves nonhomogeneous quantities (i. e., forces 
and moments, linear and angular velocities), special 
attention must be paid to the definition of such con¬ 
cepts [39.14,70,71], Also, as for the ellipsoids involv¬ 
ing internal forces, a major issue is that of physically 
meaningful parametrization of the internal forces (see, 
e.g., the work in [39.15,16]). 


where Al and B L are 6 x 6 matrices and c L is a (6 x 1) 
vector. In [39.41], a solution A for the inequality is ob¬ 
tained heuristically. The above inequality can be trans¬ 
formed into another inequality expressed with respect 
to hi, of course; however, A is fitter to such heuristic 
algorithm because it can be understood intuitively. The 
same problem may be solved mathematically, by intro¬ 
ducing an objective function to be optimized; in this 
way, the problem is recast in the framework of math¬ 
ematical programming. To this purpose, let us choose 
a quadratic cost function of h\ to be minimized 

f = hjQhi, (39.60) 

where Q is a (6 x 6) positive-definite matrix. The above 
cost function can be seen as a (pseudo)energy to be 
dissipated by the joint actuators: i. e., the arms dis¬ 
sipate electrical energy in the actuators to yield the 
internal forces/moments hi. A solution to the quadratic 
programming problem (39.60) can be found, e.g., 
in [39.42], 

For further insights on the problem of robust hold¬ 
ing in the framework of multifingered manipulation, the 
reader is referred to Chap. 38. 

Interestingly, in [39.37] the problem of load distri¬ 
bution is formulated in such a way to take into account 
manipulators dynamics at the joint level. This approach 
allows the expression of the load-sharing problem di¬ 
rectly in terms of joints actuators torques; at this level, 
different subtask performance indexes can be used, 
in a similar way as was done to solve the inverse 
kinematics of redundant manipulators, to achieve load 
distribution solutions. 


In detail, by following the approach in [39.31], the 
external force manipulability ellipsoid can be defined 
by the following scalar equation 

*e(JeJe)Ae = 1, (39.61) 

where Je = W +T J. The external velocity manipulabil¬ 
ity ellipsoid is defined dually by the following scalar 
equation 

^(JeJe) 1 v E = 1 . (39.62) 

In the case of dual-arm systems the internal force ma¬ 
nipulability ellipsoid can be defined as 


hi (Ji J?) *1 = 1 . 


(39.63) 
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where Ji = V T J. Also, the internal velocity ellipsoid 
can be defined, via kinetostatic duality, as 

vj (j,J^) ' ft, = 1 • (39.64) 

The internal force/velocity ellipsoids can be defined in 
the case of cooperative systems composed of more than 
two manipulators by considering one pair of interacting 
end-effectors at a time [39.31]. 

The manipulability ellipsoids can be seen as per¬ 
formance measures aimed at determining the arms’ 
attitude to cooperate in a given system’s configuration. 
Also, as for the single-arm systems, the manipulability 


39.6 Control 

When a cooperative multi-arm system is employed for 
the manipulation of a common object, it is important to 
control both the absolute motion of the held object and 
the internal stresses applied to it. Hence, most of the 
control approaches to cooperative robotic systems can 
be classified as force/motion control schemes, in that 
they decompose the control action in a motion control 
loop, aimed at tracking of the desired object motion, 
and a force control loop, aimed at controlling the inter¬ 
nal loading of the object. 

Early approaches to the control of cooperative sys¬ 
tems are based on the master/slave concept [39.2]. 
Namely, the cooperative system is decomposed into: 

• A master arm, which is in charge of imposing the 
absolute motion of the object; hence, the master arm 
is position controlled so as to achieve accurate and 
robust tracking of position/orientation reference tra¬ 
jectories, in the face of external disturbances (e.g., 
forces due to the interaction with the other coop¬ 
erating arms); in other words, the master arm is 
controlled so as to have a stiff behavior. 

• The slave arms, which are force controlled so as to 
achieve a compliant behavior with respect to the in¬ 
teraction forces; hence, it is expected that the slave 
arms are capable of following (as smoothly as pos¬ 
sible) the motion imposed by the master arm. 

A natural evolution of the above approach is the 
so-called leader-follower [39.67], where the follower 
arms reference motion is computed via closed-chain 
constraints. 

However, such approaches suffered from imple¬ 
mentation issues, mainly due to the fact that the com¬ 
pliance of the slave arms has to be very large, so as to 
follow the motion imposed by the master arm smoothly. 
Also, a difficulty arises when the roles of master and 


ellipsoids can be used to determine optimal postures for 
redundant multi-arm systems. 

Besides the above described approach, two other 
main approaches have been proposed to analyze the 
manipulability of cooperative multi-arm systems: the 
task-oriented manipulability measure [39.32] and poly¬ 
topes [39.33]. Moreover, in [39.34] a systematic ap¬ 
proach to perform dynamic analysis of multi-arm sys¬ 
tems is presented. Namely, the concept of dynamic ma¬ 
nipulability ellipsoid is extended to multi-arm systems, 
and indexes of the system’s capability of generating ob¬ 
ject accelerations along assigned task-space directions 
are devised. 


slave have to be assigned to the arms for a given coop¬ 
erative task, since the master/slave modes may need to 
be dynamically changed during the task execution. 

Hence, a more natural non-master/slave approach 
has been pursued later, where the cooperative system 
is seen as a whole. Namely, the reference motion of the 
object is used to determine the motion of all the arms in 
the system and the interaction forces acting at each end- 
effector are fed back so as to be directly controlled. To 
this aim, the mappings between the forces and veloc¬ 
ities at the end-effector of each manipulator and their 
counterparts at the manipulated object are to be consid¬ 
ered in the design of the control laws. 

39.6.1 Hybrid Control 

In [39.11,12] a non-master/slave approach has been 
proposed, based on the well-known scheme proposed 
by Raibert and Craig for the robot/environment inter¬ 
action control of single-arm systems (Chap. 9). The 
operational space vector for the hybrid position/force 
control is defined via the following variables 

(39.65) 

where xe, *1 are the operational space vectors, defined 
by specifying the orientation via a minimal set of ori¬ 
entation angles (e.g., Euler angles). The generalized 
forces vector to be considered is 



The organization of the control scheme is shown 
diagrammatically in Fig. 39.2. The suffixes d and m 
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Fig. 39.2 A hybrid position/force control scheme 


represent the desired value and the control command, 
respectively. The command vector r m to the actuators 
of the two arms is given by two contributions 

Tm = T p + r h . (39.67) 

The first term, r p , is the command vector for the posi¬ 
tion control and is given by 

t p = K x J~ 1 G x (s)SB(xo,d —Xo) , (39.68) 

while Th is the command vector for the force control 

r h = K h jjGh(s)(I —S)(/«o,d—M • (39.69) 

The matrix B transforms the errors on the orienta¬ 
tion angles into equivalent rotation vectors. The matrix 
J s is the Jacobian matrix that transforms the joint 
velocity q into the task-space velocity Vq. The ma¬ 
trix operators G x (s) and G|,(.v) represent position and 
force control laws, respectively. The gain matrices K x 
and Kh are assumed to be diagonal; their diagonal 
elements convert velocity and force commands into ac¬ 
tuator commands, respectively. The matrix S selects the 
position-controlled variables; it is diagonal and its di¬ 
agonal entries take the values of 1 or 0 ; namely, the 
i-th workspace coordinate is position controlled if the 
t-th diagonal element of S is 1, while it is force con¬ 
trolled if it is 0. Finally, I is the identity matrix having 
the same dimensions as S, while q and li are the vectors 
of measured joint variables and measured end-effector 
generalized forces, respectively. 

39.6.2 PD Force/Motion Control 

In [39. 17] a Lyapunov-based approach is pursued to de¬ 
vise force/position PD-type control laws. Namely, the 
joints torques inputs to each arm are computed as the 
combination of two contributions 

T m = T p + r h , (39.70) 

where r p is a PD-type term (eventually includ¬ 
ing a feedback/feedforward model-based compensation 


term), taking care of position control, while Th is in 
charge of internal forces/moments control. 

Namely, the PD and model-based terms can be com¬ 
puted at the joint level 

T p = K p e q - K d <jr + £ + ,J T W + gE , (39.71) 

where e q = q< j — q. q,\ is the vector of desired joint vari¬ 
ables, K p and Kj are positive-definite matrix gains, 
g is the vector of the gravitational forces/torques acting 
at the manipulators joints, gg is the vector of gravity 
forces/moments at the manipulated object. 

Since the cooperative task is usually assigned in 
terms of absolute and relative motion, the equivalent 
end-effector desired trajectories are to be computed by 
using the closed-chain constraints, as in the following 
example. 

Example 39.2 (Computation of Desired Trajectories 
for a Planar Two-Arm System) 

For the planar two-arm system, the desired trajectories 
defining the cooperative task are assigned by specify¬ 
ing the absolute, jr a ,d( 0 > and relative, x r d(?), desired 
motion. Then, the corresponding end-effectors desired 
trajectories can be computed as 

*i,d(0 = *a.d(0 - 2*r,d(0 , (39.72) 

*2,d(0 = *a,d(f) + -* r ,dOj , (39.73) 

where (39.41) and (39.42) have been exploited. 

In the spatial case (39.72) and (39.73) do not 
hold, since the absolute and relative orientation must 
be expressed in terms of geometrically meaningful 
quantities, e.g., via (39.44) and (39.45). However, the 
same approach may be pursued at the expense of 
a slightly more complex expressions of the above for¬ 
mulas [39.26]. 


Once, the desired position/orientation for each end- 
effector has been obtained, the inverse kinematics of 
each arm has to be computed to provide the desired 
joint trajectories, q j, to the control loop; to this aim, 
e.g., numerical inverse kinematics algorithms may be 
employed (Chaps. 2 and 10). 

Also, a PD-type control law can be expressed in 
terms of end-effector variables, i. e., 

T p =J I (K f 0-K v v)-K i q+g+i r Yf*g E , (39.74) 

where e is the tracking error computed in terms of end- 
effector position/orientation variables, v is the vector 
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collecting the end-effector velocities, and K p , K v , and 
K d are positive-definite matrix gains. 

Finally, the same concept may be exploited to de¬ 
sign a PD control law directly in the object space 

r p = J T W + (V E - K v v E ) - K d q + g + J T W^ E , 

(39.75) 

where e e is the tracking error computed in terms of ob¬ 
ject absolute position/orientation variables, Ve is the 
object’s generalized velocity vector, and K p , K v , and 
K d are positive-definite matrix gains. 

The internal force control term can instead be de¬ 
signed as follows 


T h = J T V/t ljC , 


(39.76) 


where 

/t I ,c = Al,d + G h (i)(/ti,d-/ii) , (39.77) 

Gh(s) is a matrix operator representing a strictly proper 
linear filter, such that I — Gh (s) has zeros only in the 
left half plane, and /t E d is the vector of desired internal 
forces; the internal force vector can be computed from 
the vector of measured end-effector forces as h\ = V' h. 
A particularly simple choice, ensuring a null error at 
steady state, is given by 

Gh(s)= -Kh, 
s 

where Kh is a positive-definite matrix. Remarkably, 
when an infinitely rigid object/grasp is considered, 
preprocessing of the force error via a strictly proper 
filter is needed to ensure closed-loop stability [39.17]; 
e.g., if a simple proportional feedback is adopted (i. e., 
Gh(j) = Kh), the closed loop will be unstable in the 
presence of an arbitrary small time delay, unless a small 
(namely, smaller than 1) force gain is adopted. In prac¬ 
tice, the closed kinematic chain will be characterized 
by some elasticity (e.g., due to grippers, end-effector 
force/torques sensors, joints); in this case, the product 
between the control gain Kh and the stiffness of the flex¬ 
ible components has to be chosen sufficiently small to 
ensure stability. 

An interesting extension of the above approach has 
been given in [39.26,28], where kinetostatic filtering 
of the control action is performed so as to filter all 
the components of the control input which contribute 
to internal stresses at the object. Namely, the control 
law (39.71) can be modified by weighting the propor¬ 
tional term (K p c q ) via the filtering matrix 

0 = J T (W t W + V5]V t )J“ T , 


where the (6 x 6 ) diagonal matrix E = diagjer,-} weights 
the components of ,J T K p e q in each direction of the 
subspace of the internal forces via the constant values 
0 < 07 < 1. In detail, if 51 = 06 then all these compo¬ 
nents are completely canceled from the control action, 
while the choice E = I 5 leads to the control law (39.71) 
without kinetostatic filtering. In a similar way, the con¬ 
trol laws (39.74) and (39.75) can be modified so as 
to introduce proper kinetostatic filtering of K p e and 
K p e E , respectively. 

39.6.3 Feedback Linearization Approaches 

A further improvement of the PD-plus-gravity com¬ 
pensation control approach has been achieved by in¬ 
troducing a full model compensation, so as to achieve 
feedback/feedforward linearization of the closed-loop 
system. The feedback linearization approach formu¬ 
lated at the operational space level is the base for the 
so-called augmented object approach [39.72,73]. In 
this approach the system is modeled in the operational 
space as a whole, by suitably expressing its inertial 
properties via a single augmented inertia matrix Mo- 
Hence, the dynamics of the cooperative system in the 
operational space can be written as 

M 0 (xe)*e + CoC-TAe) = h Y . (39.78) 

In (39.78), Mo and Co are the operational space terms 
modeling, respectively, the inertial properties of the 
whole system (manipulators and object) and the Cori¬ 
olis, centrifugal, friction, and gravity terms. 

In the framework of feedback linearization (formu¬ 
lated in the operational space), the problem of control¬ 
ling the internal forces can be solved, e.g., by resorting 
to the virtual linkage model [39.16] or according to the 
scheme proposed in [39.29], i. e.. 


T = J T W' [Mo (x E ,d + K v e E + K p c E ) + Co] 


+ J T V Ai,d + K h /(Aw—*!) 


£ "/< 


(39.79) 


The above control law yields a linear and decoupled 
closed-loop dynamics 


ce + K v e E + K p ce — 0 , 

i„ + K h f hi dt = 0. (39.80) 

where hi = h\ A [ — h\. Hence, the closed-loop dynamics 
guarantees asymptotically vanishing motion and force 


errors. 
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39.6.4 impedance Control 

An alternative control approach can be pursued based 
on the well-known impedance concept (Chap. 9). In 
fact, when a manipulation system interacts with an ex¬ 
ternal environment and/or other manipulators, large val¬ 
ues of the contact forces and moments can be avoided 
by enforcing a compliant behavior, with suitable dy¬ 
namic features, of the robotic system. Impedance con¬ 
trol schemes have been proposed in the case of co¬ 
operative manipulation to control object/environment 
interaction forces [39.21] or internal forces [39.22], 
More recently, an impedance scheme for the control 
of both external forces and internal forces has been 
proposed [39.74] (in l<Ef'll'lf'lM some experimental 
results of impedance control ar a dual-arm cooperative 
system are documented). 

In detail, the impedance scheme in [39.21] enforces 
the following mechanical impedance behavior between 
the object displacements and the forces acting on object 
due to interaction with the environment 

M E a E + D E i} E + K eeE = /i env , (39.81) 

where: 

• €]■ represents the vector of the object’s displace¬ 
ments between the desired and actual pose 

• v E is the difference between the object’s desired 
and actual generalized velocities 

• a E is the difference between the object’s desired and 
actual generalized accelerations, 

and /j en v is the generalized force acting on object due 
to the interaction with the environment. The impedance 
dynamics is characterized in terms of given positive- 
definite mass (Me), damping (D E ), and stiffness (K E ) 
matrices to be properly chosen so as to achieve the de¬ 
sired compliant behavior of the object. 

As for the pose displacements used in (39.81), spe¬ 
cial attention has to be paid to the orientation variables. 


Example 39.3 (External Impedance for a Planar 
Two-Arm System) 

For the planar two-arm system the quantities in (39.81) 
can be defined in a straightforward way. Namely, 

eE=*E,d-*E, (39.82) 

VE=x E ,d-x E , (39.83) 

«e =-*E,d-*E , (39.84) 

where x E is the (3x1) vector collecting the object’s 
position and the orientation, while x E ,d represents its 
desired counterpart. Hence, M E , D E , and Ke are (3 x 3) 
matrices. 


In the spatial case, orientation displacements cannot 
be defined as in (39.82), and geometrically meaning¬ 
ful orientation representations (i. e., rotation matrices 
and/or angle/axis representations) or in terms of oper¬ 
ational space variables (i. e., differences between oper¬ 
ational space vectors) have to be adopted [39.74]. 

The impedance scheme in [39.22], enforces a me¬ 
chanical impedance behavior between the z-th end- 
effector displacements and the internal forces, i. e., 

M u a ; + D u Vi + K u e, = Hu , (39.85) 

where: 

• €j is the vector expressing the displacement of the 
r-th end-effector between the desired and actual pose 

• Vj is the vector expressing the difference between 
the desired and actual velocities of the z-th end- 
effector 

• a, is the vector expressing the difference between 
the desired and actual accelerations of the z'-th end- 
effector, 

and hu is the contribution of the z'-th end-effector to 
the internal force, i. e., the z-th component of the vector 
VV' h. Again, the impedance dynamics is characterized 
in terms of given positive-definite mass (Mj,), damping 
(Du), and stiffness (Kj ,-) matrices, to be properly cho¬ 
sen so as to achieve a suitable compliant behavior of the 
end-effectors with respect to internal forces. 

Example 39.k (Internal Impedance for a Planar 
Two-Arm System) 

For the planar two-arm system the quantities in (39.85) 
can be defined in a straightforward way as well 


H 

1 

* 

II 

(39.86) 

V, =x lM -x l , 

(39.87) 

= %i,d %i » 

(39.88) 


where x, is the (3x1) vector collecting the position 
in the plane and the orientation angle of the z-th end- 
effector, while Xj'd represents its desired counterpart. 
Again, M Ei , D Ei , and K El are (3 x 3) matrices. 


As for the spatial case, orientation displacements 
in (39.85) cannot be defined as in (39.86), and geo¬ 
metrically meaningful orientation representations (i. e., 
rotation matrices and/or angle/axis representations) or 
in terms of operational space variables (i. e., differ¬ 
ences between operational space vectors) have to be 
adopted [39.74]. 

The above two approaches have been combined 
in [39.74], where two control loops are designed to en¬ 
force an impedance behavior both at the object level 
(external forces) and at the end-effector level (internal 
forces). 
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In this chapter, the fundamentals of cooperative manip¬ 
ulation have been presented. A historical overview of 
the research on cooperative manipulation has been pro¬ 
vided. The kinematics and dynamics of robotic arms 
cooperatively manipulating a tightly grasped rigid ob¬ 
ject are considered. Special topics, such as the definition 
of a cooperative task space and the problem of load dis¬ 
tribution, have also been touched on. Then, the main 
control approaches for cooperative systems have been 
discussed. A few advanced topics related to control of 
cooperative robots, i. e., advanced nonlinear control as 
well as the modeling and control of cooperative systems 
including flexible elements, will be briefly outlined in 
the following. 

Some of the first attempts to cope with uncer¬ 
tainties and disturbances in cooperative robots control 
focused on adaptive strategies [39.23,24], in which 
the unknown parameters are estimated online, based 
on a suitable linear-in-the-parameters model of the un¬ 
certainties. In detail, the approach in [39.23] is aimed 
at controlling the object motion, the interaction force 
due to the contact object/environment, and the in¬ 
ternal forces; the adaptive control law estimates the 
unknown model parameters, both of the manipulators 
and of the object, on the basis of suitable error equa¬ 
tions. In [39.24], the adaptive control concept is used 
to design a decentralized scheme, i. e., a centralized 
coordinator is not used, for redundant cooperative ma¬ 
nipulators. 

A recently developed control framework for coop¬ 
erative systems is the so-called synchronization con¬ 
trol [39.51,52]; in this approach the control problem 
is formulated in terms of suitably defined errors ac¬ 
counting for motion synchronization between the ma¬ 
nipulators involved in the cooperative task. Namely, the 
key idea in [39.51] is to ensure tracking of the desired 
trajectory assigned to each manipulator, while syn¬ 
chronizing its motion with other manipulators motion. 
In [39.52], the problem of synchronizing the motion of 
multiple manipulators systems is solved by using only 
position measurements; the synchronization controller 
consists of a feedback control law and a set of nonlinear 
observers, and synchronization is ensured by suitably 
defining the coupling errors between the manipulators’ 
motions. 

Recently, research efforts have been spent on intel¬ 
ligent control [39.53-55]. Namely, in [39.53], a semi- 
decentralized adaptive fuzzy control scheme, with Jfoo 
performance in motion and internal force tracking, 
is proposed; the controller of each robot consists of 
two parts: a model-based adaptive controller and an 
adaptive fuzzy logic controller; the model-based adap¬ 


tive controller handles the nominal dynamics, includ¬ 
ing a purely parametric uncertainties model, while the 
fuzzy logic controller is aimed at counteracting the 
effect of unstructured uncertainties and external dis¬ 
turbances. In [39.54], a decentralized adaptive fuzzy 
control scheme is proposed, where the control law 
makes use of a multi-input multi-output fuzzy logic 
engine and a systematic online adaptation mechanism. 
The approach has been further extended in [39.55]. 

Finally it is worth mentioning the efforts invested in 
the investigation of control strategies using partial state 
feedback, i. e., only joints position and end-effector 
forces are fed back to the controller. A recent contri¬ 
bution was provided by the work in [39.56], in which 
a decentralized control algorithm that achieves asymp¬ 
totic tracking of desired positions and forces by using 
a nonlinear observer for velocities was proposed. 

Flexibility in a cooperative system may arise, e.g., 
due to the use of nonrigid grippers to grasp the ma¬ 
nipulated object. In fact, the adoption of compliant 
grippers allows large internal forces to be avoided and, 
at the same time, achieves safe manipulation of the ob¬ 
ject, even in the presence of failures and unpredicted 
contacts with the external environment. In detail, the 
work in [39.61] develops a non-model-based decentral¬ 
ized control scheme. Namely, a proportional-derivative 
(PD) position feedback control scheme with gravity 
compensation, is designed; the PD scheme is capable 
of regulating the position/orientation of the manipu¬ 
lated object and, simultaneously, achieves damping of 
the vibrations induced by the compliant grippers; also, 
a hybrid scheme is adopted to control internal forces 
along the directions in which the compliance of grip¬ 
pers is too low to ensure limited internal stresses at the 
manipulated object. 

Other research efforts have been focused on 
handling multibodied objects, or even flexible ob¬ 
jects [39.43-45]. Those objects are difficult to handle 
and, therefore, assembly of those objects in manu¬ 
facturing industry is not automated. Also, cooperative 
control of multi-flexible-arm robots has been inves¬ 
tigated [39.46-48]. Once the modeling and control 
problem is solved (Chap. 11), the flexible-arm robot is 
a robot with many merits [39.49] : it is lightweight, com¬ 
pliant, and hence safe, etc. Combining control methods 
for flexible-arm robots, such as vibration suppression, 
with the cooperative control methods presented in this 
chapter is straightforward [39.46]. Automated object- 
retrieval operation with a two-flexible-arm robot has 
been demonstrated in [39.47, 48] and in l<s>MMSEH. 

Finally, cooperative transportation and manipu¬ 
lation of objects via multiple mobile manipulators 
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can be considered still an open research subject. In 
fact, although notable research results have been al¬ 
ready devised [39.65,75-77], the foreseen use of 
robotic teams in industrial settings (hyperflexible 
robotic workcells) and/or in collaboration with hu¬ 
mans (robotic co-worker concept) raises new chal¬ 


lenges related to autonomy and safety of such systems. 
A recently emerged application scenario is the co¬ 
operative transportation of objects via multiple aerial 
robots [39.66] (see l^s&JKMIEEai for an example of co¬ 
operative transportation via multiple unmanned aerial 
vehicles). 


Video-References 




Cooperative grasping and transportation of objects using multiple UAVs 
available from http://handbookofrobotics.org/view-chapter/39/videodetails/66 
Impedance control for cooperative manipulators 

available from http://handbookofrobotics.org/view-chapter/39/videodetails/67 
Cooperative capturing via flexible manipulators 

available from http://handbookofrobotics.org/view-chapter/39/videodetails/68 
Cooperative grasping and transportation of an object using two industrial manipulators 
available from http://handbookofrobotics.org/view-chapter/39/videodetails/69 
Control of cooperative manipulators in the operational space 
available from http://handbookofrobotics.org/view-chapter/39/videodetails/70 
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40. Mobility and Manipulation 


Oliver Brock, Jaeheung Park, Marc Toussaint 


Mobile manipulation requires the integration of 
methodologies from all aspects of robotics. Instead 
of tackling each aspect in isolation, mobile manip¬ 
ulation research exploits their interdependence to 
solve challenging problems. As a result, novel views 
of long-standing problems emerge. In this chap¬ 
ter, we present these emerging views in the areas 
of grasping, control, motion generation, learning, 
and perception. All of these areas must address 
the shared challenges of high-dimensionality, 
uncertainty, and task variability. The section on 
grasping and manipulation describes a trend to¬ 
wards actively leveraging contact and physical and 
dynamic interactions between hand, object, and 
environment. Research in control addresses the 
challenges of appropriately coupling mobility and 
manipulation. The field of motion generation in¬ 
creasingly blurs the boundaries between control 
and planning, leading to task-consistent motion 
in high-dimensional configuration spaces, even 
in dynamic and partially unknown environments. 
A key challenge of I earning for mobile manipulation 
consists of identifying the appropriate priors, and 
we survey recent learning approaches to percep¬ 
tion, grasping, motion, and manipulation. Finally, 
a discussion of promising methods in perception 
shows how concepts and methods from navigation 
and active perception are applied. 
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What exactly is mobile manipulation? To find an an¬ 
swer to this question, we must consider the historical 
development of this research area. The term mobile ma¬ 
nipulation was coined in the late 1980s to early 1990s, 
when research labs began to mount robot manipulators 
on mobile platforms [40.1,2], Back then, the term was 
intended to capture just that: research was conducted on 
an experimental platform that combines capabilities in 


mobility and manipulation. A series of mobile manipu¬ 
lation platforms is shown in Fig. 40.1. 

With the introduction of such platforms it soon became 
apparent that combining mobility with manipulation 
was a game changer. Mobility enables manipulators to 
leave the lab. They now have to face the complexi¬ 
ties of the real world. And in the real world, much of 
the research that had been successful in the carefully 
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controlled environments in research labs proved to be 
brittle or inadequate. This traditional research often re¬ 
lied on strong assumptions about the environment to be 
successful - an untenable assumption in unpredictable, 
nonstationary everyday environments. 

In the context of mobile manipulation research, con¬ 
trolled environments, such as factory floors and specific 
experimental setups, are often referred to as structured 
environments. In contrast, unstructured environments 
are environments that have not been modified specif¬ 
ically to facilitate the execution of a task by a robot. 
These two types of environments are opposites on 
a continuous scale with many possible intermediates. 
And, of course, structured environments are still laden 
with uncertainty just the same as unstructured environ¬ 
ments still contain a significant structure that can be 
exploited by the robot. 

In unstructured environments, it becomes a neces¬ 
sity for the robot to perform a wide variety of tasks, 
instead of a single specific task. Any specific task, such 
as retrieving a book from the library, might require the 
robot to solve additional challenges, such as opening 
doors, operating elevators, moving a chair out of the 
way to approach the shelf, or asking for the location 
of the stacks. Today, no robotic system possesses such 
capabilities. 

Researchers in mobile manipulation have learned 
that the progress achieved in traditional research ar¬ 
eas of robotics cannot simply be combined to produce 
the level of competency they aim for. This insight 
motivated the founding of mobile manipulation as a re¬ 
search area. The research in this area seeks to develop 
robotic systems capable of autonomous task execution 
in unstructured or minimally structured environments. 
Clearly, the transition from single-purpose lab demon¬ 
stration - a common practice in robotics - toward such 


mobile manipulation systems is a gradual one. Mobile 
manipulation research therefore gradually increases the 
autonomy, robustness, and task generality of robotic 
systems, while at the same time gradually reducing the 
dependence on prior information about the environment 
and the task. 

The objective of mobile manipulation research is to: 

1. Maximize task generality of autonomous robotic 

systems, while at the same time 

2. Minimizing the dependence on task-specific, hard¬ 
coded, or narrowly-relevant information. 

Some people argue that this is exactly what main 
stream robotics is doing and what most of this Hand¬ 
book is about. And they might be right. But over the last 
20 years, the mobile manipulation community, forced 
by failures of robotic systems in the real world, has be¬ 
gun to explore alternatives and extensions to the body 
of robotics knowledge collected in this tome. It is these 
alternatives or extensions that this chapter is about. In 
each of the subsequent sections, our goal is to showcase 
work in mobile manipulation that opens up possible 
novel directions for future advances toward robots op¬ 
erating in real-world environments. For the areas of 
grasping and manipulation, mobility, control, motion 
generation, learning, and perception, we will analyze 
how the main challenges of the field change when sys¬ 
tems are operating in the real world. We will show 
how each of these fields has developed and advanced 
to increasingly address the challenges of unstructured 
environments. 

Before we embark on this journey, it is worthwhile 
to agree on the challenges that are in fact encoun¬ 
tered when robots are supposed to perform a variety 
of tasks in an unconstrained and uncontrolled envi- 



Fig.40.1a-d Some exemplary mobile manipulation platforms developed throughout the decades (a) Herbert (MIT, 
1985), (b) Stanford assistent mobile manipulator (1995), (c) PR2 (Willow Garage, 2009), (d) Herb (CMU, 2013) 
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ronment. These challenges are common to all of the 
subdisciplines we discuss in the following sections. 

First, the necessity to perform a variety of tasks, 
instead of a single one, requires that the robot be 
equipped with general means of mobility, manipulation, 
and perception. Its capabilities must be the union of 
the minimum capabilities required for each task. This 
entails the need for versatile sensing and dexterous ma¬ 
nipulation. This, in turn, implies that the sensor input 
of the robot is high dimensional and so is the configu¬ 
ration space of the robot. The first challenge therefore 
consists of dealing with the high dimensionality and the 
inherent complexity of the state spaces required for task 
generality. To make matters worse, task generality most 
often requires system dynamics to be hybrid, i. e., ex¬ 
hibit discrete as well as continuous behavior. 

Second, by the definition of mobile manipulation, 
the robot cannot rely on a complete, continuously, and 
globally updated world model. While some part of the 
robotics community attempts to structure the world to 
become such a world model - by using RFID tags or 
by placing sensors everywhere - many in the mobile 
manipulation community believe it to be more effective 
and more insightful to make robots smarter. Not hav¬ 
ing a complete world model means that the robot has 
to use perception to acquire relevant information about 

40.1 Grasping and Manipulation 

Robots accomplish tasks by grasping, manipulating, 
and placing objects. Several chapters in this Handbook 
are concerned with associated mechanisms, methods, 
and concepts: Chapter 19 discusses robot hands, the 
most versatile type of end-effector for grasping and 
manipulation. Chapter 37 discusses manipulation in 
general and Chap. 38 presents the state of the art in 
grasping. Important for grasping and manipulation are 
also force and tactile sensors, which are discussed in 
Chap. 28. While these four chapters describe the well- 
established foundation of grasping and manipulation, in 
this chapter we want to take a higher level perspective, 
assess the suitability of these approaches in the context 
of mobile manipulation, and speculate about possible 
novel approaches. 

Both grasping and manipulation exemplify the chal¬ 
lenges for mobile manipulation we laid out above: 
uncertainty, high dimensionality, and task variability. 
Uncertainty is relevant as minor inaccuracies in the 
object model, minor errors in motion execution, and 
small differences in positioning relative to the ob¬ 
ject can cause grasp failure. The high dimensionality 
of grasping results from the many degrees of free¬ 
dom of the hand as well as from those of the objects 


the world; it means that the robot has to address the un¬ 
certainty in its sensing, but also the uncertainty in its 
world model which might change unbeknownst to the 
robot. The second challenge therefore consists of ap¬ 
propriately handling the uncertainty inherent to sensing 
and actuation and the uncertainty caused by a dynamic 
world. 

Third, in the real world, objects exhibit large vari¬ 
ability from the perspective of their appearance, even 
when they perform exactly the same function. This 
means that there is a layer of indirection through ap¬ 
pearance when a robot wants to operate autonomously 
in the real world. There are so many different kinds 
of door handles, they differ significantly in appearance, 
but perform the same function. The challenge therefore 
is to cope with the variability in the world by revealing 
the functional principles underlying the world’s appear¬ 
ance. A natural way of addressing variability are skills 
that generalize to novel situations. 

As we will see in subsequent sections, attempts to 
address these challenges by the robotics community 
have produced innovative and successful concepts, al¬ 
gorithms, mechanisms, and integrated systems. Some 
of them were not covered in previous chapters, as they 
do not (yet?) belong to the canon of robotic wisdom. 
Maybe this chapter can contribute to changing this. 


in the environment. Finally, grasping and manipula¬ 
tion occur in a large number of variations, each in¬ 
volving vastly different interactions with objects. It 
is therefore not surprising that decades of research 
on grasping and manipulation have made significant 
progress but so far were unable to create skills suf¬ 
ficiently robust for autonomous operation in the real 
world. 

In this section, we describe a recent development 
in the field of grasping and manipulation away from 
the foundational techniques laid out elsewhere in this 
Handbook toward novel approaches that seem better 
suited for addressing the manipulation challenges in 
mobile manipulation. 

40.1.1 Problem Description 

First, a word on defining the problem. Traditionally, 
grasping and manipulation are viewed as two distinct 
problems. But there is signififant ambiguity with re¬ 
spect to these terms in the literature [40.3]. In this 
section, we limit ourselves to grasping and manipula¬ 
tion with robot hands. And in this context, we define 
the following terms: 
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Grasping refers to gaining reliable control over the 
extrinsic degrees of freedom of an object through the 
robot’s degrees of freedom, i. e., the robot grasps an ob¬ 
ject so that it can change the object’s extrinsic degrees 
of freedom by changing its own, excluding the degrees 
of freedom of the hand, which remain static once the 
grasp is achieved. To perform grasping, the overall de¬ 
grees of freedom of the robot have traditionally been 
divided into two parts: those required to impart forces 
on an object (usually addressed using grasp planning 
approaches) and the remaining ones that can be used to 
reposition the object, once a grasp is obtained (general 
motion and manipulation planning). Note, however, that 
in contrast to this traditional decomposition, coordina¬ 
tion of arm and hand degrees of freedom are usually 
required during grasping. 

By dexterous grasping we refer to grasping while 
optimizing the grasp posture for (a) robustness and 
(b) suitability for the execution of a specific task. In 
the relevant literature, the overall degrees of freedom 
are still most commonly divided into those responsible 
for the grasp and those able to change the pose of the 
grasped object. 

By dexterous in-hand manipulation, we refer to the 
ability to actuate the extrinsic degrees of freedom of the 
object using all degrees of freedom of the arm and the 
hand so that the object moves relative to a reference 
frame placed at the hand’s wrist [40.4], 

Classical approaches to all three of these problems 
rely on precise models of the environment and of the 
hand, carefully reason about contact points and states, 
and attempt to determine and precisely execute a de¬ 
tailed plan. We would like to point to the emergence of 
a novel approach to all three of these problems. Since 
this novel approach, however, has been mainly explored 
in the context of grasping at this early point in time, we 
will focus our discussion on grasping. But we will also 
make the point that in this new way of thinking about 
grasping and manipulation both of these problems be¬ 
come similar, if not identical. 

40.1.2 Evaluation of the State of the Art 

The earliest and still most fundamental considerations 
of grasping are based on the notions of force and form 
closure (Chap. 38). 

Force and Form Closure 

These notions express the effect of a set of disembod¬ 
ied contact points, i. e., ignoring the body on which that 
contact point lies, such as the finger, on the ability of an 
object to move. They reflect a static view of grasping, 
in which the physical interactions that will invariably 
occur during the grasp are not considered. 


This line of research continues to be active and suc¬ 
cessful. as evidenced by a large number of sophisticated 
and capable grasp planners and simulators [40.5]. The 
resulting grasps, however, often do not result in success¬ 
ful real-world grasps. These failures are a consequence 
of model uncertainty and motion execution noise. The 
assumptions made during planning, i. e., that the disem¬ 
bodied contact points can be realized precisely by the 
hand, more often than not do not hold. 

Interactions Between Hand and Object 
There is a very simple strategy to overcome many of the 
theoretical failures of the approaches based on precise 
models and the notion of force closure - and this strat¬ 
egy is used tacitly by all grasping research: just keep 
closing the hand, even if you think you have attained the 
right configuration for your contact points. This strat¬ 
egy effectively leverages mechanical compliance of the 
hand to adapt the hand’s configuration to the object’s 
shape. This adaptation of shape has several positive ef¬ 
fects for grasping: 

1. Sensing and actuation uncertainty are compensated, 

2. Large surface areas are established, and 

3. Grasping forces are distributed and balanced. 

These effects lead to an increase in grasp success 
and grasp quality. It is fair to say that this is a standard 
trick in the grasp planning community. 

This insight is also strongly driving contemporary 
hand design. To maximize the effect of shape adapta¬ 
tion, Rodriguez and Mason [40.6] optimize finger shape 
to yield similar contact configuration irrespective of the 
size of the object. A different way of increasing shape 
adaptability is to include compliant components in the 
hand design [40.7]. The shape deposition manufactur¬ 
ing (SDM) hand [40.8], the Velo gripper [40.9], the 
i-HY hand [40.10], and the Pisa/UT SoftHand [40.11] 
achieve shape adaptability through underactuation by 
coupling degrees of freedom of the hand, adapting the 
shape of the hand to the object while equalizing contact 
forces. An extreme case of this is the positive pressure 
gripper [40. 12], which presses a bag filled with granular 
material onto the object to be grasped. The bag adapts to 
the shape of the object. When the air is evacuated from 
the bag, the granular material jams, forming a gripper 
perfectly adapted to the shape of the object. 

From this brief analysis it is fair to say that the 
shape adaptability between hand and object is routinely 
leveraged in grasp planning, simply by closing the hand 
until a certain grasping force is reached. Also, shape 
adaptability has emerged as an important design crite¬ 
rion for hands, as it very obviously improves grasping 
performance. 
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Interactions Between Hand, Object, 
and Environment 

Features in the environment, provided by other objects 
or support surfaces, for example, may constrain the mo¬ 
tion of hand and object. This is most evident for support 
surfaces, such as tables and floors. These constraints, 
when used properly, can aid grasping. Furthermore, it 
might be the case that the necessary perceptual infor¬ 
mation for leveraging such constraints is often easier to 
obtain than the information required for the successful 
execution of an unconstrained grasp. 

Recent research in robotic grasping leverages en¬ 
vironmental constraints in the suggested manner, e.g., 
to position the hand relative to the object [40.13], to 
cage objects [40.13,14], or to fixate an object during 
planar sliding [40.13,15], Some pregrasp manipulation 
relies on environmental constraints to improve grasp 
success. For example, Chang et al. [40.16] rotate pan 
handles into a specific orientation prior to grasping 
by exploiting the pan’s friction and remote center of 
mass. 

But, in addition to external contacts, environmental 
constraints can also be created by exploiting gravity, in¬ 
ertia, or dynamic motion of the arm and hand. In this 
way, in-hand manipulation is accomplished by using 
what has been called extrinsic dexterity [40.17]. 

All of the aforementioned grasp strategies rely on 
multiple interactions between hand, object, and envi¬ 
ronment prior to attaining the final grasp posture. These 
phases often are designed to reduce uncertainties in spe¬ 
cific variables relevant to grasp success. 

The idea of environmental constraints appears in 
early work by Mason et al. [40. 18, 19]. Here, the intrin¬ 
sic mechanics of the task environment are exploited to 
eliminate uncertainty and to achieve robustness. 

The study of environmental constraint exploitation 
by humans so far has been limited to replicating in¬ 
stances of observed behavior on robots [40.16,20]. For 
example, Kaneko et al. [40.20] extracted a set of grasp¬ 
ing strategies from observations of a human subject. 
These strategies include interactions with environmen¬ 
tal constraints. 

We believe that this recent trend toward the ex¬ 
ploitation of environmental constraints represents an 
important opportunity to improve robotic grasping ca¬ 
pabilities. To take full advantage of this opportunity, we 
should understand the strategies humans employ, trans¬ 
fer them to robotic systems, and develop robotic hands 
tailored to this exploitation [40.21]. This will be dis¬ 
cussed in the next section. 

Insights from Human Grasping 
Interestingly, the study of human grasping paralleled 
the development in the robotics community. Early stud¬ 


ies of human grasping followed the static view captured 
by force closure concepts. This is reflected in grasp 
taxonomies, classifying grasp according to the final 
hand posture attained after the grasp process is com¬ 
pleted [40.22,23]. 

Even the early work on postural synergies, which 
has had a profound impact on robotics, initially only 
considered synergies of static grasp postures [40.24]. 
These studies do not capture the dynamic processes and 
the exploitation of environmental constraints we believe 
to be crucial for robust grasping. Nevertheless, these 
studies of human grasping have led to significant ad¬ 
vances in robotic grasping [40.11]. 

The nature of hand-object interaction has also 
been studied in humans. The effect of shape adapt¬ 
ability is well known for human hands and studies 
have elucidated the degree to which humans vary their 
behavior to take advantage of it. Christopoulos and 
Schrater [40.25] showed that humans react to pose un¬ 
certainty of a cylinder and orient their hand to align it, 
presumably to be able to maximize the benefits of shape 
adaptability. However, other experiments may point at 
the fact that humans also rely on more complex interac¬ 
tions with the environment for grasping under difficult 
conditions. When the vision of humans is impaired, 
they fail more often at first grasp attempts of isolated 
(environmental-constraint free) objects [40.26]. The de¬ 
gree of the demonstrated effect seems surprising. We 
believe that in this specific experiment, it is due to 
the lack of environmental constraints exploitable for 
grasping. 

More recently, studies of human grasping have 
shown the deliberate use of environmental constraints 
during grasping. For example, humans increase their 
use of environmental constraints when their vision is 
impaired [40.21], This seems to indicate that the use 
of environmental constraints will play a crucial role in 
achieving robust and task-general grasping and manip¬ 
ulation in robotics. 

Robot Hands 

Many highly capable robotic hands exist. A histori¬ 
cal overview, collecting robotic hands from over five 
decades, was compiled by Controzzi et al. [40.27]. 
An analysis of robot hand designs with respect to 
grasping capabilities was recently presented by Greben- 
stein [40.28]. As the notion of compliance is central to 
our hand design, we will limit our discussion to hands 
designs that deliberately include this concept. 

We distinguish two main approaches for designing 
compliant hands. Compliance can be achieved using 
active control, and implemented on a fully actuated 
or even hyper-actuated systems, where every degree 
of freedom can be controlled. An impressive exam- 
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Fig.40.2a-e Various hand designs leveraging compliance to improve performance, (a) Awiwi hand (DLR), (b) SDM hand 
(Havard), (c) Pisa-IIT hand, (d) i-HY hand (iRobot, Havard, Yale), (e) RBO hand z (TU Berlin) 


pie of this type of hand is the Awiwi hand [40.28], 
the ShadowRobot Shadow Dexterous Hand, and the 
SimLab Allegro Hand [40.29]. These hands achieve 
dexterity through accurate control, which comes at the 
price of mechanical complexity, making them difficult 
and costly to build and prone to failure. 

An alternative approach is to make hands compliant 
by including elastic or flexible materials (passive com¬ 
pliance). Building a passively compliant joint is much 
cheaper than an actively controlled one, in terms of 
costs, volume and system complexity. Passive compli¬ 
ance can easily absorb impact forces - a desirable prop¬ 
erty for an end-effector designed to establish contact 
with the world. The cost of adding additional (passive) 
degrees of freedom is low, compared to actively compli¬ 
ant hands. The resulting ability to passively adapt to the 
shape of an object greatly enhances grasp success and 
grasp quality. At the same time, the hand can be under¬ 
actuated, effectively offloading control to the physical 
material. 

A pioneering work in grasping with passive compli¬ 
ance was the soft gripper by Hirose and Umetani [40.7]. 
Recently, a whole range of grippers and hands were 
built using passive compliance, such as the FRH-4 
hand [40.30], the SDM hand and its successor [40.10, 


31,32], the starfish gripper [40.33], the THE Second 
Hand and the Pisa-IIT Soft Hand [40.34], the Positive 
Pressure Gripper [40.12], the RBO Hands [40.13,35], 
and the Velo Gripper [40.9]. A different source of in¬ 
spiration was taken by Giannaccini et al. [40.36], who 
built an octopus-inspired compliant gripper. Some of 
these hands are shown in Fig. 40.2. 

The practical realization of underactuated hands 
is matched by theoretical approaches to analyze and 
evaluate their dexterity [40.37,38]. However, these ap¬ 
proaches require accurate knowledge of grasp posture, 
contact point locations, and contact forces. Given to¬ 
day’s sensor technologies, this information is difficult 
to obtain in physical implementations. 

The inclusion of compliance into the design of 
robotic hands has led to significant improvements in 
power-grasping of objects. Given the insights obtained 
from human grasping (see the previous section), it is 
plausible that these improvements are the result of these 
hands’ capabilities to engage in beneficial contact inter¬ 
actions with the environment. While these benefits have 
been demonstrated extensively in the context of power 
grasping, little work has examined the beneficial effects 
of compliance and underactuation on the dexterity of 
robotic hands [40.35]. 
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40.1.3 Toward Robust Grasping 
and Manipulation 

The key to robust grasping and manipulation - an im¬ 
portant prerequisite for mobile manipulation - appears 
to be the deliberate and purposeful use of interactions 
between hand, object, and environment. This is sup¬ 
ported by evidence from human grasping as well as 
from the design features of successful robotic hands. 
If this claim is true, it fundamentally changes the game 
in grasping and manipulation. The notion of a specific 
contact point configuration, which has for the last 50 
years been of pivotal importance, is superseded by de¬ 
liberate constraint exploitations. The traditional, static 
view of grasping is replaced with a dynamic view that 

40.2 Control 

Mobile manipulators are robotic systems with capabili¬ 
ties for both manipulation and mobility. A typical setup 
of mobile manipulator, therefore, consists of manipu¬ 
lator and mobile robot. Traditionally, mobile manip¬ 
ulators were implemented as wheeled mobile robots, 
providing mobility on two-dimensional flat surfaces. 
Mobility, however, can be provided by other types of 
robots depending on the operating environment. Legged 
robot systems operate well in uneven or rough terrain. 
Aerial or underwater robots allow operation in environ¬ 
ments with totally different characteristics. 

Controlling such mobile manipulators involves con¬ 
trol for the subsystems. The control issues of manipu¬ 
lators are explained and discussed earlier in Chaps. 8 
and 9. Modeling and control of various types of mobile 


emphasizes motion of hand, object, and environment 
in contact with each other. Rather than placing fin¬ 
gers carefully at exactly the right place, this new way 
of thinking requires fingers to slide across surfaces, to 
comply to object shapes, and to balance and maintain 
contact forces through intelligent hardware design. 

Especially in hardware design, there has been sig¬ 
nificant progress. The newest generation of hands 
are built for the exploitation of constraints: with¬ 
out sensing and explicit control, they support shape 
adaptation and compliant contact maintenance. One 
of the future challenges will be the development of 
grasp planning methods and perceptual capabilities 
to leverage the advantages afforded by these novel 
hands. 


platforms will be introduced later in detail in Chaps, 
from 48 to 52 in Part E. In these chapters, we would like 
to discuss unique control problems in mobile manipu¬ 
lation, which arise when the manipulator and mobile 
platform operate simultaneously. 

Control of mobile manipulators also has to address 
the three challenges we stated initially: high dimen¬ 
sionality, uncertainty, and task variabilty. The mobile 
manipulator was first implemented to provide mobil¬ 
ity to the manipulator. One of the main purposes is 
to increase the manipulation workspace of the robot 
(Fig. 40.3). This mobile platform naturally adds more 
degrees of freedom to the system by providing mo¬ 
bility. The additional degrees of freedom increase the 
dimensionality of the control problem. The robot be- 



Fig.40.3a-c Increased workspace and redundancy of mobile manipulators, (a) A manipulator on a fixed base; (b) in¬ 
creased work space of a mobile manipulator; (c) redundancy due to the added mobility 
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comes a redundant system because the specified task 
to the robot is still the same, while the overall degrees 
of freedom have been increased. That is, there are infi¬ 
nite number of configurations that can execute the same 
task. 

Second, uncertainty becomes severe in control of 
mobile manipulators due to the mobile platform. The 
uncertainty is larger when controlling a mobile plat¬ 
form compared to control of a manipulator. The reasons 
for the higher uncertainty are imperfect information 
or model about the operating environment, such as 
ground and underwater. Additionally mobile platforms 
with only internal sensors do not have access to per¬ 
fect state information because drift raises uncertainty 
while moving. However, mobile manipulators need to 
solve the same manipulation tasks as stationary ma¬ 
nipulators. They have to overcome this uncertainty to 
operate reliably. 

Third, the variability of the mobile platform exists 
depending on the operating environment, although the 
high dimensionality and uncertainty are the common 
factors across the various mobile platforms. The differ¬ 
ent issues due to various mobile platforms are also to be 
discussed later in this chapter. 

40.2.1 Problem Description 

Control of mobile manipulators must provide capabil¬ 
ities for manipulation and mobility. Often times, these 
two are treated completely separate. The robot moves 
to a certain location using its mobile platform. Then, 
the manipulator performs specific tasks without mov¬ 
ing the platform. In this approach, the mobile platform 
considers the manipulator as a static load. The mobile 
platform is then considered as a fixed base for the ma¬ 
nipulator. There is no major difficulty implementing 
this approach, unless manipulator or mobile platform 
is unable to hold its position steadily. 

The main problem to be discussed in this chapter 
arises when both manipulator and mobile base oper¬ 
ate simultaneously. Although the previously mentioned 
approach of a separately controlling manipulator and 
a platform increases the workspace of the robot, it does 
not exploit the full capabilities of mobile platforms. 
Instead, the advantage of mobile manipulator can be 
maximized when the manipulator operates while the 
bases moves in coordination. This kind of operation will 
not only reduce the total operation time but will also in¬ 
crease task variability. 

In this chapter, therefore, we will discuss how to 
control mobile manipulators by operating both sub¬ 
systems at the same time. During this operation, the 
whole system becomes redundant due to the added de¬ 
grees of freedom (DOF) from the mobile platform. How 


to exploit this increased number of DOF is one im¬ 
portant problem in mobile manipulation. Secondly, the 
increased uncertainty due to the mobile platform re¬ 
quires more robust control strategies than that only for 
manipulator. Finally, various issues for different types 
of mobile platforms will be discussed. 

40.2.2 Evaluation of the State of the Art 

First mobile manipulators were implemented as 
wheeled robots and wheels are still the most popular 
tool to enable a manipulator to move. Most of the issues 
that have been dealt with in wheeled mobile manipula¬ 
tor can be shared with other types of mobile platforms. 
Therefore, we start discussing wheeled mobile plat¬ 
forms in this chapter and discuss issues specific to other 
types of mobile platforms afterward. 

Redundancy Resolution and Control 
One of the main issues in controlling mobile manip¬ 
ulators has been how to use the additional DOFs to 
execute manipulation tasks in a stable way. An early 
study [40.39] investigated how to obtain solutions to the 
inverse kinematics problem that prevent the robot from 
tipping over. 

Other approaches exploited the increased number 
of DOFs to avoid obstacles, increase manipulability, 
handle singularities, or achieve lower priority tasks. 
In [40.40], a preferred region to maximize the manip¬ 
ulability of the robotic arm is defined and the nonholo- 
nomic mobile platform is controlled to be within the 
region. The same concept of a preferred region is also 
applied to the force control of a mobile manipulator 
in [40.41]. These approaches are typical examples for 
approaches that coordinate with the manipulator and 
base motion, but control the subsystems separately. 

The redundancy of mobile manipulators allows 
dealing with multiple tasks simultaneously. In the 
configuration approach [40.42,43], additional tasks or 
specifications are defined to be simultaneously con¬ 
trolled with the end-effector. To obtain solutions online, 
a weighted damped least-squares approach was pre¬ 
sented in [40.44]. The configuration approach is further 
developed to be used both for holonomic and nonholo- 
nomic bases in [40.45]. 

In [40.46], the unified control of the arm and holo¬ 
nomic base was achieved by applying the operational 
space control framework to the mobile manipulator. 
The coordination of the arm and base can be imple¬ 
mented in the null-space of the task control, which does 
not affect the control of the end-effector. Later, this ap¬ 
proach was extended to deal with nonholonomic bases 
in [40.47]. It uses a complete model of the system with 
nonholonomic constraints to devise one unified con- 
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trailer for the whole system. The redundancy was used 
to deal with internal and external constraints. Another 
model-based control is derived in [40.48] for a car-like 
and differentially-driven system. 

Similarly, an approach of exploiting redundancy for 
avoiding singularities and maximizing manipulability is 
proposed in [40.49] using an event-based planner and 
a nonlinear feedback controller with the dynamic model 
of the robot. 

As can be seen from the above literature, the is¬ 
sue of redundancy resolution in mobile manipulator has 
been often dealt with as an extension from the control 
of redundant systems but with the special case of non- 
holonomic constraints. 

Uncertainty Due to Mobile Platform 
Another main problem arising when controlling mobile 
manipulators, is the additional uncertainty due to the 
mobile platform. This is because the mobile platform in 
real world moves on the imperfect ground, which cre¬ 
ates a large uncertainty on the manipulation although 
it can be negligible for navigation. Early work in the 
nineties studied how to control the manipulator when 
the base is under disturbance due to the ground con¬ 
dition in [40.50]. The disturbance was assumed to be 
unknown and considered as rotational motion only. 

Also, model-based approaches are not always suited 
for the control of mobile manipulator because accurate 
models of mobile platforms are harder to obtain than 
models of manipulators. The extended Jacobian trans¬ 
pose algorithm is proposed in [40.51] for controlling 
the manipulator to overcome the excessive end-effector 
error or instability due to unmodeled vehicle dynamics. 
Similarly, since precise modeling of the interaction is 
difficult, decentralized control is proposed in [40.52], It 
regards the interaction force from each system as un¬ 
known disturbance. 

In [40.53], the control system is divided into mobile 
platform and manipulator, and two low-level controllers 
are used to control each system. The dynamic inter¬ 


action force from the base is modeled as an unknown 
disturbance in manipulator control, and an adaptive 
controller was designed to deal with this disturbance. 
A redundancy resolution scheme avoids singularities. 
On the other hand, the effects of dynamic interactions 
between manipulator and wheeled base on the perfor¬ 
mance of the end-effector task are studied in [40.54]. 

In another aspect, there have been approaches with¬ 
out modeling interaction or dynamics of each system. 
A robust damping controller is proposed in [40.55] for 
the motion control subject to kinematic constraints but 
without any knowledge of dynamic parameters. 

Neural network-based control also has been pro¬ 
posed in this respect. In [40.56], a neural network-based 
joint space control is developed for a mobile manipula¬ 
tor with unknown dynamic model and disturbance. Two 
neural network controllers are applied to manipulator 
and vehicle, respectively. The radial basis function net¬ 
work with weight adaptation is used in [40.57] for on¬ 
line estimation of nonlinear dynamics. The algorithm 
is applied to a simulation of a two-link manipulator on 
a mobile platform. Adaptive robust motion/force con¬ 
trol has been developed for both holonomic and non- 
holonomic constrained mobile manipulators in [40.58]. 
It ensures stability and the boundedness of tracking 
errors in the presence of uncertainties in dynamic pa¬ 
rameters and disturbances. 

Sliding mode control and adaptive neural network 
control are combined together in [40.59]. Multilayered 
perceptrons are applied to estimate the dynamic model 
as one system, and the adaptive control is designed in 
the task space control. A sliding mode approach with 
neural network-based control is proposed in [40.60] for 
omnidirectional wheeled mobile manipulators. 

Control Issues on Various Types 
of Mobile Platform 

Although wheeled vehicles are one of the most gen¬ 
eral types of platform to provide mobility, there is an 
increasing tendency of using other types (Fig. 40.4). 




Fig. 40.4 Mobile manipulators in various environments 
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Legged locomotion systems like humanoids and 
quadrupeds are becoming popular. The interesting new 
aspect of these systems is that the platform for loco¬ 
motion can actively participate in manipulation. That 
is, the legs of the humanoid system can provide not 
only locomotion but also assist in manipulation. This 
is especially enabled by using the whole-body control 
framework [40.61-64], This control framework consid¬ 
ers the robotic system as a whole and utilize all the 
joints for both manipulation and locomotion. This is es¬ 
pecially justified in legged robots because the mobile 
platform and manipulator use the same type of actuator. 
Other platforms use different types of actuation for base 
and manipulator. 

Also, quadruped robots such as Robosimian from 
NASA JPL uses the leg parts to increase the workspace 
and dexterity of the manipulation. The legged sys¬ 
tem can provide more capability of manipulation than 
wheeled mobile platform because it can provide the full 
three-dimensional (3-D) motion of the base part such 
that the manipulation part can have much larger manip¬ 
ulation capability. The whole-body control framework 
is also applied to a quadrupedal robot in [40.65]. The 
robot does not yet have manipulator in this system, 
but the control framework can support the manipulation 
control if needed. 

Mobile manipulators operating underwater are be¬ 
coming more prevalent. The main difference to wheeled 
mobile platform is the effect of hydrodynamic forces 
during motion. An efficient dynamic simulation algo¬ 
rithm is developed for underwater mobile manipulator 
in [40.66], where various hydrodynamic forces can 
be modeled and incorporated. In [40.67], the hydro- 
dynamic effect between the manipulator and vehicle 
is modeled and compensated. The demonstrated re¬ 
sults improved performance in controlling both vehicle 
and manipulator. The underwater mobile manipulator is 
modeled using Kane’s method in [40.68], which also 
incorporates the major hydrodynamic forces such as 
added mass, profile drag, fluid acceleration, and buoy¬ 
ancy. 

Underwater mobile manipulators also have redun¬ 
dant degrees of freedom. In [40.69], redundancy created 
by the underwater vehicle is utilized to achieve sec¬ 
ondary tasks, such as reducing energy consumption and 
maximizing dexterity, while executing the main task at 
the end-effector. A task-priority redundancy resolution 
technique is applied for this purpose. The restoring mo¬ 
ment is minimized in [40.70] in redundancy resolution 
so that the performance of the coordinated motion con¬ 
trol is improved. 

Similar to other mobile platforms, the uncertainty 
due to the underwater vehicle is one of the major 
difficulties. To deal with this issue, an adaptive con¬ 


trol approach is proposed in [40.71] that is robust in 
the presence of uncertainties in the vehicle and envi¬ 
ronment. An iterative learning algorithm is developed 
for the hydrodynamic effects in [40.72]. An adap¬ 
tive tracking control is proposed in [40.73] that keeps 
the advantage of model-based control and has a mod¬ 
ular structure. An observer-controller strategy deals 
with the difficulty of obtaining precise velocity mea¬ 
surements in [40.74]. This significantly improves the 
chattering of the output actuator caused by noise and 
quantization. 

Aerial mobile manipulation has been actively in¬ 
vestigated in recent years. Aerial mobile manipulation 
has different characteristics in another sense: stability 
and load capability. The dynamics of the manipulator 
or interactions with the environment can create stability 
problem in aerial mobile manipulation. Also, interac¬ 
tion forces and load are limited due to the capability of 
lift provided by the aerial platform. 

An aerial mobile manipulator system with three 
2-DOF arms is developed in [40.75]. The interaction 
between the manipulators and quadrotors is modeled so 
that the reaction forces and torques during flight and 
manipulation are compensated for stable flight. 

A Cartesian impedance control is implemented 
in [40.76] and [40.77], while the task redundancy due 
to the added DOF from the aerial vehicle is used for 
secondary tasks in [40.76]. Hybrid force and motion 
control of tool operation in quadrotors are proposed 
in [40.78]. The dynamics of quadrotors are transformed 
into the tool-tip position and decomposed into the tan¬ 
gential and normal directions with respect to the contact 
surface. Then, a stabilizing controller is designed for the 
hybrid motion and force control. 

40.2.3 Toward Control 

in Mobile Manipulation 

Control of mobile manipulators is one of the most in¬ 
vestigated topics since the beginning of the concept of 
mobile manipulation. Although the issues discussed so 
far in this chapter are still very important and need to 
be further investigated, interaction with humans and the 
environment is another important aspect to handle to 
enable mobile manipulators to come into our daily life. 
In this respect, force control at the end-effector of the 
mobile manipulator has been studied in [40.79, 80]. The 
issue of contact transition has been studied for non- 
holonomic mobile manipulators in [40.81]. A control 
strategy is proposed to deal with impulsive contact force 
on the mobile manipulator in [40.82]. 

On the other hand, the use of compliant actuators 
such as series elastic actuators (SEA) and variable stiff¬ 
ness actuators (VS As) is expected to become more 
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popular. The topic of compliant actuators is covered ex¬ 
tensively Chap. 21. The SEA has been developed for 
the purpose of safety and energy efficiency in the com¬ 
promise of precise control. The VS A can overcome 
the disadvantage of SEA with additional actuation for 
changing stiffness. 

Despite the complexity of implementing these actu¬ 
ators in terms of both hardware and algorithms, they are 
expected to be used more because there is an increasing 
need for safe robot-human or robot-environment inter¬ 
action. The application of compliant actuators to mobile 


40.3 Motion Generation 

Methods to generate the motion of robots are funda¬ 
mental to robotics. They are treated extensively in this 
Handbook (Chaps. 7, 8 as well as the entire Part E). As 
in the previous sections of this chapter, we will consider 
the state of the art in motion generation in the con¬ 
text of mobile manipulation. We will identify aspects of 
motion in mobile manipulation currently not fully ad¬ 
dressed and will speculate about how these gaps might 
be closed. 

In the context of mobile manipulation, the gener¬ 
ation of robot motion poses a number of challenges, 
related to those laid out in the introduction to this 
chapter. Robotic systems able to perform a variety 
of tasks must possess versatile motion capabilities. 
This is usually realized through a large number of 
degrees of freedom, leading to high-dimensional mo¬ 
tion generation problems. The humanoid robot Justin, 
for example, possesses 58 DOFs; the latest version 
of the Honda ASIMO has 57 DOFs. In addition to 
the high dimensionality of the associated configuration 
spaces, uncertainty represents a major challenges when 
generating robot motion. For applications in mobile ma¬ 
nipulation, one simply cannot assume that a precise 
and complete world model is available at all times. 
The generation of motion therefore must consider the 
uncertainties present in the robot’s world model. Fur¬ 
thermore, it is unrealistic to assume that the entire 
world will be perceivable all at once. This implies 
that the robot’s world model will always be partial 
and potentially wrong in important ways. All this is 
compounded by the fact that the robots sensors and 
actuators themselves are prone to uncertainty. It is 
therefore apt to state that motion generation in mo¬ 
bile manipulation is subject to two of the problems we 
identified in the introduction: high dimensionality and 
uncertainty. 

In this section, we will examine major branches of 
research in motion generation and analyze the degree 


manipulators would provide new challenges to the con¬ 
trol issues of dealing with uncertainty and stability in 
addition to the precision of the manipulator itself. The 
compliance on the joints will provide larger uncertainty 
in the manipulator. This uncertainty affects not only the 
control of the manipulator but also the control of the 
mobile platform. The effect on the mobile base will be 
the stability problem in addition to the precision in con¬ 
trol. Nevertheless, the introduction of soft/compliant 
actuators will be one of the important future directions 
to enable safe interaction in mobile manipulation. 


to which they are poised to address the challenges of 
motion generation in manipulation. But before that, of 
course, we must discuss what exactly these challenges 
are. 

40.3.1 Problem Description 

What are the requirements for robot motion in the con¬ 
text of mobile manipulation? Just as in the classical 
motion planning problem, a robot must be able to move 
from one place to another. This requires information 
about the spatial extent of objects and the resulting 
global connectivity of world. Algorithms that address 
this most basic requirement of robot motion are dis¬ 
cussed extensively in Chap. 7. 

In addition to global, goal-oriented motion, a mo¬ 
bile manipulator must also be able to maintain task 
constraints during motion: holding a glass of water 
upright, pointing a camera in an inspection task, or 
coordinating motion with another robot during cooper¬ 
ative manipulation. At the same time, it must be able 
to react quickly to unforeseen changes in the environ¬ 
ment; global motion planning might not be fast enough 
and therefore has to be complemented by reactive ob¬ 
stacle avoidance. 

While performing this task-consistent, global, and 
reactive motion, the safe and efficient movements of 
a mobile manipulator may also depend on kinematic, 
dynamic, or postural constraints. Postural constraints 
may be used to perform additional tasks or simply to 
move in the most energy economical posture to extend 
battery life. 

Finally, all of the motion requirements (global, 
task-consistent, reactive, joint-limit avoiding, posture- 
optimizing) must be achieved and maintained in the 
presence of various sources of uncertainty. In mobile 
manipulation, there will always be uncertainty about the 
world model, which might be incomplete or partially 
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wrong; changes to the world model may happen without 
knowing the robot. And, of course, there is uncertainty 
in regards to sensing, affecting the robot’s knowledge 
of its own state or the ability to predict the outcome of 
its actions. 

As we will see, most existing areas of research in 
motion generation today cannot address all the afore¬ 
mentioned requirements at the same time. Furthermore, 
these motion requirements also necessitate sensing ca¬ 
pabilities. It is therefore difficult to treat motion gen¬ 
eration for mobile manipulation independently from 
sensing and perception. 

40.3.2 Evaluation of the State of the Art 

Sampling-based motion planning is the de facto stan¬ 
dard for motion planning in robotics; the state of the 
art in that field is described in detail in appendix of 
Chap. 7. Here, we want to show some work that ex¬ 
tends standard methods, such as probabilistic roadmaps 
(PRMs) and rapidly-exploring random trees (RRTs), to 
address some of the motion requirements of mobile 
manipulation. 

Motion Planning 

Motion planners have been devised to account for the 
changes in the configuration space when robots ma¬ 
nipulate objects by picking and placing them, possibly 
handing them off to another robot in order to fulfill 
a motion task [40.83]. This enables complex sequences 
of repeated reach and grasps in very tight environments 
to achieve a planning objective. 

Other planners enable the combination of global 
motion with task constraints [40.84, 85]. The resulting 
planners generate globally goal-directed motion while 
maintaining task constraints at the robot’s end-effector. 
This enables collision-free motion while maintaining 
a fixed orientation of the end-effector, for example. 
This is realized by identifying lower dimensional, task- 
consistent manifolds in configuration space and confin¬ 
ing possible motions to those manifolds. This is usually 
achieved through iterative optimization of sampled con¬ 
figurations, adding the computational complexity of 
global motion planning. 

Planners are also able to devise global motion 
that is consistent with kinodynamic constraints and 
with actuation limits of robotic platforms. Kinody¬ 
namic planning addresses, for example, constraints im¬ 
posed by nonholonomicity or by the dynamics of the 
robot [40.86]. Actuation limits become important when 
a robot is unable to lift a heavy load with its arm 
fully extended. In such a situation, a planner can gen¬ 
erate a motion to slide the object closer to a region 
of the workspace where the mechanical advantage of 


the robot kinematics enables the desired motion objec¬ 
tive [40.87], 

Global motion planning is computationally expen¬ 
sive. Therefore, it generally cannot operate at reactive 
rates. As a result, they are unable to satisfy the motion 
requirement of reactive obstacle avoidance. Of course, 
ideally one would determine a global motion plan at 
high frequencies; then planners would be able to satisfy 
global and reactive motion requirements. To overcome 
the provable computational complexity of planning, 
one has to resort to approximation algorithms. In other 
words, one has to trade completeness for computational 
efficiency. While there are probably many ways of ac¬ 
complishing this, which may be the subject of future 
research, one possibility presented in the literature is the 
balancing of exploration and exploitation [40.88]. By 
replacing the exploration of motion planners with ex¬ 
ploitation on those areas of configuration space where 
helpful information is available, it is possible to speed 
up planning by more than one order of magnitude. 

Common to all of the motion planning approaches 
in this section is that they ignore uncertainty. They 
assume a perfect geometric world model and perfect 
execution capabilities. These assumptions generally do 
not hold in mobile manipulation. Hence, it will be dif¬ 
ficult to extend traditional motion planning approaches 
so as to address the entire spectrum of motion require¬ 
ments described above. 

Further information regarding motion generation 
for mobile manipulation, in particular in the context of 
humanoid robots, has been collected in a special vol¬ 
ume [40.89]. 

Feedback Motion Planning 
Feedback motion planning [40.86, 90] addresses a par¬ 
ticular type of uncertainty: actuation uncertainty. With 
this type of uncertainty, the robot might end up in 
a different state than it intended. An appropriate plan 
must therefore contain the correct action to take in ev¬ 
ery possible configuration the robot might end up in. 
Such a plan is called a navigation function. Given such 
a navigation function, the robot can perform Markovian 
gradient descent until the goal is reached. 

Feedback motion planning also assumes a perfect 
knowledge of the world. The ability to address actuation 
uncertainty comes at a significant computational cost. 
Instead of having to compute a single path, an implicit 
representation of all paths has to be determined. 

Trajectory Modification 

There is an appealing complementarity between motion 
planning and control methods. While the former are 
good at addressing global motion constraints, the latter 
excel at reactive behavior, including obstacle avoid- 
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ance, force control, maintaining task constraints, etc. If 
the advantages of these two approaches could be com¬ 
bined, one would have made significant progress toward 
motion generation for mobile manipulation. 

The simplest way of achieving this objective is 
through consecutive invocation: first determine a path 
consistent with global motion requirements, then apply 
control methods to the entire path to continuously opti¬ 
mize in response to changes in the environment or task 
constraints. This latter part, applying control to an entire 
path or trajectory as opposed to a single robot configu¬ 
ration, is called incremental trajectory modification. 

All methods in this category perform optimization 
of a function, encoding various desirable properties 
of the trajectory, through some kind of gradient de¬ 
scent. The earliest method, called elastic bands [40.91], 
applies repulsive forces from obstacles and attractive 
internal forces to a discretization of the configuration 
space trajectory, leading to a behavior reminiscent of 
elastic material. This method was extended in the elas¬ 
tic strip framework [40.92] by applying these forces 
directly in the workspace to a discretization of the vol¬ 
ume swept by the robot along its configuration space 
trajectory. Approaching obstacles deform the trajectory; 
when obstacles recede, the trajectory is shortened by the 
internal forces. 

In contrast to the local reaction forces computed by 
the elastic band and elastic strip approaches, other ap¬ 
proaches optimize trajectories globally in accordance 
with various criteria of optimality. This optimization 
problem can be cast as approximate probabilistic in¬ 
ference [40.93], then using message passing algorithms 
to find the maximum likelihood trajectory for a given 
objective. Alternatively, the optimization problem can 
be solved using the Hamiltonian Monte Carlo algo¬ 
rithm [40.94], where covariant functional gradient de¬ 
scent techniques perform local optimization. A stochas¬ 
tic variation of this method seems to be able to deal 
better with local minima [40.95]. 

Even though trajectory modification methods are 
often times labeled as motion planning methods, they 
do not perform global planning. They have been shown 
to work well in many realistic scenarios but can fail 
when gradient information is insufficient to find a so¬ 
lution. As a result, trajectory modification can serve as 
a powerful component for motion generation, but they 
cannot address all requirements of motion generation in 
mobile manipulation. 

Integrating Planning and Control 
Traditionally, global motion planners produce paths or 
trajectories that then get passed to the robot for ex¬ 
ecution. At execution time, these trajectories can be 
modified, for example, by the methods described in the 


previous section. It is striking, however, that a plan¬ 
ner determines an entire trajectory from start to goal, 
knowing well that in dynamic environments most of the 
trajectory will be invalidated within seconds due to the 
motion of obstacles. 

To fully leverage the complementarity of global 
planning and control mentioned in the previous section, 
it might be necessary to shift the boundary between 
these two fields. This would imply that, instead of 
computing a detailed trajectory, global motion planning 
would only provide as much information as is required 
to avoid local minima during control. This leads to the 
idea of sequencing local controllers, which can be visu¬ 
alized as funnels, in such a way that a global motion 
task can be achieved [40.96]. Referring to Fig. 40.5, 
each controller can be viewed as a local plan for region 
of the configuration space (in the image, the config¬ 
uration space is the imagined plane above which the 
funnels are arranged). Each funnel has an exit (the con¬ 
verged state of the controller) that leads into the opening 
of the next funnel. Arrangements of these funnels rep¬ 
resent robust motion plans as each funnel (controller) 
inherently rejects uncertainty. 

Such a shift of the boundary between planning 
and control is realized in the Elastic Roadmap ap¬ 
proach [40.97]. It combines global planning with trajec¬ 
tory modification to yield an approach to motion gen¬ 
eration capable of generating global, task-consistent, 
reactive, and constraint-respecting motion. It does so, 
however, by deliberately trading completeness for com¬ 
putational efficiency. While this approach works well 
in practices, no performance guarantees can be made. 
Also, this approach is not able to address uncertainty; it 
still assumes that the world and its changes are perfectly 
known. 

Reasoning About Uncertainty 
Partially observable Markov decision processes 
(POMDP) are a general framework to formalize 



Fig. 40.5 Representing a motion plan as a sequence of 
funnels 
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problems involving actions that perform uncertain 
transition (uncertain outcome) between uncertain (not 
fully known) states (Chap. 15). POMDPs are so general 
that they can easily characterize the type of motion 
planning problem occurring in mobile manipulation. 
A downside of their generality and expressive power 
is the computational complexity of finding an exact 
solution, which is often intractable. To overcome this 
problem, one must resort to approximations. 

Point-based POMDP solvers [40.98] provide such 
an effective approximation for the case that there is 
an uncertainty about the robot’s state. The key idea 
of point-based solvers is to sample only parts of the 
solution space, avoiding regions unlikely to contain 
a solution [40.99]. Effectively, these methods exploit 
additional knowledge about the specific problem to 
guide the search for a solution. 

In the context of motion generation, the roadmaps 
generated by traditional sampling-based motion plan¬ 
ners can provide such knowledge for finding solutions 
to a specific motion planning problem [40.100,101] 
or to any motion planning query in a given environ¬ 
ment [40.4], 

Considering uncertainty not only for the robot’s 
state but also about the world model makes the prob¬ 
lem even more difficult. Here also, it becomes necessary 
to approximate the state space of the world, leading 
to discrete approximations of the corresponding state 
space [40.102, 103]. 

While all the methods mentioned so far success¬ 
fully address uncertainty, their computational complex¬ 
ity prevents them from being reactive. This means 
that all initial assumptions about the world, including 
assumptions about uncertainty distributions (encoded 
in the POMDP), must be correct and remain correct 
throughout the entire motion. In the context of mobile 
manipulation, it is difficult to think that even the initial 
assumptions could be correct. 

If one does not want to rely on fixed assumptions 
about uncertainty, for example in the state transition 
function, one has to rely on sensing. For example, 
a robot might be able to reduce the uncertainty about 
its own state sufficiently well to be able to avoid rea¬ 
soning about this type of uncertainty. But this in itself 
is a very strong assumption. By making such strong 
assumptions, it is possible to reduce the complexity 
of addressing uncertainty in a meaningful way, lead¬ 
ing to a polynomial time to address uncertainty in the 
context of mobile manipulation [40. 104]. While this ap¬ 
proach achieves reactivity, task consistency, and is able 


to address other constraints, it is incomplete and no per¬ 
formance guarantees exist. 

40.3.3 Toward Motion Generation 
for Mobile Manipulation 

The literature contains a number of powerful techni¬ 
cal tools and conceptual approaches that have proven 
effective in the context of motion generation: sampling- 
based motion planners, planning for feedback, opti¬ 
mization for incremental trajectory modification, se¬ 
quencing of controllers, shifting the boundary between 
planning and control, and approximative solutions to 
POMDP-based formalization of motion planning prob¬ 
lems. The use of assumptions to improve the computa¬ 
tional efficiency is inherent to all of them - no surprise, 
given that even simple motion planning problems are 
already PSPACE-complete. 

For each of the aforementioned motion require¬ 
ments in mobile manipulation, when considered in 
isolation, an adequate solution exists. But no method 
or approach addresses all of them in coherent fash¬ 
ion with characterizable performance. Nevertheless, 
in practice, motion generation capabilities exist for 
many interesting applications in mobile manipula¬ 
tion. 

With respect to the goals laid out in the intro¬ 
duction to this chapter, this discussion has exposed, 
however, that all approaches still depend on detailed 
prior knowledge about the environment. Most of this 
prior knowledge is difficult to obtain in the context of 
mobile manipulation. For example, accurate geometric 
models of the environment are unlikely to be obtain¬ 
able at a global scale. And even if they were attainable, 
they will likely be outdated very quickly and would re¬ 
quire constant updating. It might be even more difficult 
to obtain adequate characterizations of uncertainty for 
arbitrary motion problems. 

In response to these difficulties, it is highly likely 
that the close integration of real-world sensing into 
motion generation will become a more active area of re¬ 
search. This sensing must be used to alleviate the needs 
for prior world models. This development might par¬ 
allel the shifting the boundary between planning and 
control, only this time between planning and sensing. 
Instead of performing planning on very detailed and all 
encompassing world models, it might be more practical 
to plan in a coarse and approximate world world, leav¬ 
ing it to perception to fill in the details as they become 
relevant. 
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We mentioned high dimensionality, uncertainty, and 
variability as core challenges for robotic systems that 
leave prestructured lab environments. Learning is the 
core approach to cope with the variability of tasks 
a robot may need to perform in unstructured environ¬ 
ments. 

The topic of learning is orthogonal to all of the other 
topics discussed in this section: Learning methods are 
successfully applied to grasping and manipulation, mo¬ 
bility, control, motion generation, and perception. We 
will highlight representative approaches in these areas 
below, but first we want to discuss in more general 
terms the relation between the areas of machine learn¬ 
ing and robotics. 

A core question is whether existing machine learn¬ 
ing methods can without need for modification be 
applied in robotic domains. Or whether genuinely novel 
issues are raised in the context of robot learning that 
necessarily go beyond core machine learning research. 
This is fundamentally related to the question Is there 
a generic priori 

In the Bayesian view, learning means computing the 
posterior 


P(M\D) 


P(D\M)P(M) 

P(D) 


(40.1) 


A similar discussion could refer to the choice of hy¬ 
pothesis space and regularization. But for simplicity we 
take the Bayesian view as a reference. The machine¬ 
learning community has made great progress in two 
respects: First, it developed a rapidly growing set of 
general and fundamental modeling tools to formalize 
learning problems, including standard parametric and 
nonparametric models, graphical models, infinite (hier¬ 
archical Dirichlet process) models, models combining 
probabilities with first-order logic, random set models, 
etc. 

Second, the community has developed a large set 
of methods to solve learning problems, i. e., compute 
the posterior or optimize discriminative models, includ¬ 
ing a wide range of probabilistic inference methods 
and leveraging the tight relations to the field of (con¬ 
strained) optimization. 

With this range of modeling formalisms and com¬ 
putational methods available, what problems are left? 
Interestingly, a core issue remains the concrete spec¬ 
ification of the prior P(M) or hypothesis space itself. 
An example is the current research on deep represen¬ 
tations: Weston et al. [40.105], for instance, argue that 
deep learning can concisely be understood as a form of 
regularization of intermediate representations, includ¬ 


ing auto-encoders as special case but generalizing it 
to arbitrary choices of embeddings. Such a regular¬ 
ization is one-to-one with the choice of model prior 
in the Bayesian formulation of learning methods. The 
large majority of (non-deep) regression and classifica¬ 
tion models are linear in some features (we include 
Hilbert space features here). The regularization is typ¬ 
ically generic: Lt or L\ on the respective parameters. 
Therefore, the model prior is primarily determined by 
the choice of features. 

Researchers generally agree that a core chal¬ 
lenge of machine learning is in the specification of 
the prior P(M ) - respectively the choice of fea¬ 
tures [40.106, 107]. However, is the question about 
the right prior a genuine machine learning question? 
In other terms: Does there exist a generic prior that 
might perform well in any domain, or is a good choice 
of prior inherently domain specific, requiring expert 
knowledge? 

Related to this question is the idea of learning the 
prior - or learning the right features. Of course, in the 
strict sense, a prior must not be data based. When dis¬ 
cussing the learning of features, people usually presume 
a hyperprior that implies a class of potential features. 
However, it is questionable whether there exist generic 
hyperpriors over feature classes that would allow sys¬ 
tems to learn arbitrary features suited in any domain. 
Some aspects of priors seem generic and are discussed 
extensively within machine learning: L\ or Li regu¬ 
larization to enforce sparsity and penalize complexity; 
metric or topological embedding to enforce the preser¬ 
vation of information in internal (deep) representations. 
However, the structural complexity of suitable features 
in real-world domains seems - in the eyes of a roboti¬ 
cist - beyond such generic properties. 

In this light, it is not surprising that the success 
of learning methods within robotics often relies on an 
insightful way to model the problem using machine 
learning formalisms combined with a good choice of 
features. Both depends on the roboticist’s expert knowl¬ 
edge and his understanding of appropriate priors in real 
world domains, contrasting the idea that generic ma¬ 
chine learning and priors could be applied for learning 
in robotic manipulation. 

40.4.1 Problem Description 

Perhaps a reason for the great success of machine learn¬ 
ing is that its problems are - in constrast to robotics - 
rather straightforward to define rigorously. In most 
cases a learning problem is defined in terms of mini¬ 
mizing a loss or risk (often approximated using cross 
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validation), computing a posterior, or maximizing an 
expected return. Feature and hyperparameter selection 
are subsumed in these objectives. 

To make this more concrete, we choose to detail 
one specific machine learning approach: conditional 
random fields (CRF) [40.108], CRFs have become 
a default starting point for addressing many learn¬ 
ing problems, especially in robotics. And CRFs are 
a very general way of modeling problems, as we dis¬ 
cuss briefly in the following, and should therefore be 
a standard tool for roboticists. 

A CRF could also be named a conditional graphical 
model or conditional factor graph. It describes a struc¬ 
tured probability distribution over labels y where the 
factors that define this distribution additionally depend 
on some input x. The following summarizes the essen¬ 
tials of CRFs: 


where the second term is an Li (Ridge) or L\ (Lasso) 
regularization and the first the neg-log-likelihood on 
data D = { (xi, >’,)}f = |. The optimum is found using 
a Newton method, using the following gradient and 
Hessian terms: 

L(D.f) = - log/i(>',|x,) 

i 

= - y) T P - zte, P)\, 

i 

(40.7) 

VpZ(x.P) = ^2,p(y\x) <p(x,y ) , (40.8) 

y 

%Z(x,P) = I*) 4>(x,y) <P(x,y) r 

y 

-yjpZ[VpZ\ T . (40.9) 


• We consider output variables y = (y\,... ,yi) and 
input variables x. A CRF is defined by a joint func¬ 
tion over x and y of the form 

k 

/try) = ^2(pj(x,y a j)Pj = cp(x,y) T p , (40.2) 

i= i 

which describes the predicted output as 

y* (x) = argmax/O, v) . (40.3) 

Each feature <pj(x, ya/) depends on a subset ysj of 
output variables, dj C {1,..., /}. The function / is 
also called (neg-) energy or discriminative function. 
CRFs can be trained based on a probabilistic inter¬ 
pretation or with a hinge loss. 

• In the probabilistic interpretation [40.108], CRFs 
define a conditional probability distribution over y, 


JC,y) 

& _ f(x,y)-Ztx,0) 

(40.4) 

k 

g —z( x,0) J - J c <l>j(x,ydj)0j 

(40.5) 


7=1 


which is a Boltzmann distribution of / with log- 
partition function 

Z(x, f ) = log fJ'C.y’) 
y' 

ensuring the conditional normalization. This is 
trained by minimizing 


(40.6) 


Computing p(y\x,) (or the factor marginals 
P(ydj\ x d) for a ll x i is analogous to the E-step of 
expectation maximization; the parameter update 
of f in each Newton iteration is analogous to 
the M-step. For arbitrary structures, the Hessian 
term yf x p(y\x)<p(x,y)<p(x,y) 1 may be infeasible 
to compute exactly. Still, Newton methods with 
approximate Hessian may be significantly faster 
than gradient-based methods. 

• In the discriminative interpretation [40.109], the 
CRF is trained by minimizing a hinge loss 


min 

0,6 


P 2 + Cj^$i 

1=1 


s.t. :f(xi,yi) —f(xj, y) > 1 & > 1 , 

(40.10) 


using either perceptron or linear programming 
methods similar to how SVMs use them [40.109]. 
This approach is also called structured output SVM. 

• Kernelization works for both types of training. Fur¬ 
ther, in the probabilistic interpretation, the ridge 
regularization can be replaced by a Gaussian 
prior P(P) = N(P | 0, cr 2 /A) and a fully Bayesian 
predictive posterior P(y\x, D) can be computed (just 
as for Bayesian ridge regression and Gaussian pro¬ 
cesses). 

• CRFs are a very general class of learning tool. We 
list special cases, where the options concern the 
special case input and output spaces, probabilistic or 
hinge loss training, kernelization, bavesianization, 
and of course the choice of features (explained be¬ 
low): 

SVM = CRF(kernel. hinge, y e {0, 1}, </>c) 
logistic regression = CRF(prob, ye {0, 1}, (pc) 


P* = argming L(D, P) + X\\P\\ , 
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Gaussian process (GP) = CRF(prob, bayes, kernel, 
y e ]R, <p R ). 

GP classification = CRF(prob, bayes, kernel. 
y e {0.1}, <t>c) 

factor graph (MRF) = CRF(prob, x = 0) 
hidden Markov model (HMM) = CRF(prob, y = 
(y u ...,y T ), {/,}) 

ridge regression = CRF(prob, yel, <p R ) 

The regression case is somewhat special, see below. 
• We give three feature examples for the cases of mul¬ 
ticlass classification, Markov chains (like HMMs), 
and the somewhat special case of regression. 

For a three-class classification, one chooses 


4>c(y,x ) 


< [>’ = \}(p(x) > 
\y = 2\<p{x) 

K |y = 3]0W } 


6l 


3 k 


where ye {1,2, 3}, [ expr.] e {0,1} is the indicator 
function and </>(x) e R k an arbitrary (e.g., polyno¬ 
mial) input feature vector. The effect of the in¬ 
dicators is that for each class, a different subset 
of /3-elements define the function/. 

For a Markov chain y = (yi,..., y T ) with binary y t , 
one chooses 


4>t(y t +uyt,x) 


t [y, = 0 av, + i = 0] > 
bt =0Ay f+ i = 1] 
bt = lAy,+ i =0] 
bt = 1 Ay ;+ 1 = 1] 

V bt = 1 ]<Pt(x) > 


el 


4+k 


where f } l: 4 determines the transition probabilities, 
and / 5 : 5 -|—i—i plays the same role as in the logistic 
regression of y, on input features 4> t (x) e IR/. If x = 
(xi,..., Xj ) is also a time series and if we choose 
input features </>,(x) = <p,(x t ) to depend only on x t , 
we are in the case of HMMs. The great strengths 
of Markov chain-CRFs over HMMs is that /,(x) 
may depend arbitrarily on the whole of x. In the (un¬ 
usual) regression case, we may choose 


<Mt. x) 


-\y 2 t ° 2 \ 

<t>(x)y/o 2 ) 


e m 1+<: , 


which leads to y*(x) = <p(x) T p and p(y\x) = 2AT(y | 
y* (x), a 2 ) with a partition function independent of x 
and /. Therefore, a single Newton step leads to the 
well-known optimal parameters of ridge regression. 
GPs are the bayesian kernel variant of this. 


An extreme special case is logistic regression, where 
the graphical model only concerns a single binary out¬ 
put variable y, the distribution over which depends on the 
input x. On the other side, also any kind of standard (non¬ 
conditional) graphical model overy is an extreme special 
case of a CRF that lacks the dependence on any exter¬ 
nal x. Conditional Markov chains (e.g., in linguistics), 
Markov random fields (in computer vision), and many 
other models can be viewed as instances of CRFs. 

Given this concrete framework of CRFs, the prob¬ 
lem - from the robotics perspective - essentially be¬ 
comes to propose an appropriate graphical model struc¬ 
ture over y, and to choose appropriate features cf>(x,y). 
Both choices together correspond to the choice of hy¬ 
pothesis space or prior as discussed above. 

40.4.2 Evaluation of the State of the Art 

Perhaps the best developed area for the application of 
learning methods in robotics is perception. 

Learning for Perception 
and Scene Understanding 
Saxena et al. developed a series of methods that very 
successfully employ the general framework of CRFs 
in various perception tasks. In [40.110] they train 
CRFs to predict the depth in two-dimensional (2-D) 
views; [40.111] learns CRFs to classify a grasp- 
affordance in the 2-D view; [40.112] learns CRFs 
(or SVMs) to classify the place- affordance of loca¬ 
tions; [40.113] learns CRFs to label and thereby an¬ 
ticipate the actions and movements of demonstrators. 
These works are exemplary for the power of current 
machine-learning methods for perception - but also in 
that their success relies on the creative choice of prob¬ 
lem formalization and features. As an example, we 
would like to mention the unique features chosen to 
be indicative of concavity and place stability to predict 
cup-like place locations in [40.111]. Could such types 
of features ever be discovered autonomously by more 
advanced machine learning methods? 

Also, for 3-D point cloud data the definition of 
clever features turned out to be essential for basic reg¬ 
istration and classification problems [40.114]. 

With respect to our discussion of learning fea¬ 
tures, in standard image classification tasks there has 
recently been great success in using generic (sparse 
coding) priors to learn features that significantly outper¬ 
form hand-coded features (like robust features (SURF) 
and histograms of oriented features (HOG) [40.115]. 
Such learned features correspond to local 2-D or 3-D 
appearance patterns (which define the codebook); the 
histogram of such patterns turns out to be an excellent 
basis for image classification. It seems promising that 
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typical 2-D CRF approaches to classify affordances - 
as those mentioned above - can be extended to also use 
learned local features. However, there may be limita¬ 
tions to such generically learned features. For instance, 
naively they do not respect any natural invariances 
(such as affine or illumination invariance; this can be 
fixed by artificially incorporating such invariances in 
the dataset). Using merely sparse coding priors, they are 
unlikely to reflect stronger priors (e.g.. Gestalt laws, or 
kinematic understanding of rigid body scenes) of natu¬ 
ral physical environments. For instance, the concavity 
feature for placement prediction seems in strong con¬ 
trast to such learned features. 

As a conclusion with respect to perception, the 
existing machine learning machinery, including novel 
feature learning methods have shown great success. 
For specific mobile manipulation tasks (like affordance 
prediction) this often relied on well chosen problem for¬ 
malization. As is the case also in the following topics, 
a core question for future research are novel ideas for 
feature classes and priors, that are generic and support 
the learning of features for manipulation tasks, but in¬ 
corporate stronger physical world priors than existing 
approaches. 

Learning for Grasping 

The ability to grasp is essential to manipulation and ap¬ 
proaches to learning to grasp therefore deserves special 
attention. Learning to grasp has been addressed from at 
least two perspectives. 

First, on the perceptual side, several authors 
have proposed methods to directly classify 2-D or 
3-D appearance patterns that predict a grasp affor¬ 
dance [40. 111]. On the positive side, this implies a very 
close perception-action cycle; the percept is directly 
translated to a potential pregrasp location without need 
to estimate a 3-D shape of the object or even a notion of 
objects and shapes. Such approaches provide a sort of 
heat map of potential affordances, e.g., for exploration 
or to reduce the number of options for manipulation. 
But they lack the generality of grasp approaches that 
take into account the goal of applying a certain force 
or motion to an object - but classically require a shape 
estimate. 

Second, concerning the grasp motion itself, a ba¬ 
sic approach is to employ policy search methods to 
improve a parametric dynamic movement primitive 
(DMP) model of the grasp motion [40.1 16]. 

Learning for Motion Generation 
There exists extensive work on learning methods for 
control, e.g., in the form of system identification, 
(model-based and model-free) reinforcement learning 
and learning from demonstration. It is beyond the scope 


of this chapter to cover this. Instead, here we focus on 
learning in the context of path finding or trajectory plan¬ 
ning for manipulation. 

A core issue of motion generation for manipulation 
is the question of the right task space. For instance, 
the work by Cakmak and Thomaz on learning from hu¬ 
man interaction [40.117] suggests that the question of 
what is the actual task space should be in the center of 
communication. In a certain formalization, the choice 
of the task space of a motion is perfectly analogous to 
the choice of features in machine learning: The map¬ 
ping from a geometric problem setting to an appropriate 
motion can be formalized as a CRF [40.118], where 
the optimal motion maximizes a discriminative function 
(or neg-cost function). Typically such a cost function 
is a sum of squares of features, where each feature 
captures some error in some nonlinear task space. The 
problem of finding appropriate task spaces - the right 
features of motion - is therefore analogous to the prob¬ 
lem of finding appropriate features in machine learning. 
And as for generic machine learning, the core questions 
are: what is a general class of potential features (of po¬ 
tential task spaces), and what is a possible prior over 
this class. 

Jetchev and Toussaint's task space retrieval (TRIC) 
approach [40.1 19] considers a combinatorial set of po¬ 
tential task dimensions that refer to various absolute 
and relative coordinates of objects and uses a Lasso 
prior to select those features that best explain a set 
of demonstrated trajectories. While this successfully 
extracts the ground-truth task spaces w.r.t. which the 
given trajectories have been optimized, the approach 
still seems limited by the crude set of allowed fea¬ 
tures. Relative coordinates (w.r.t. object centers) can¬ 
not express more meaningful and better generalizing 
concepts, e.g., fitting one piece into another. Further, 
features of manipulation motions might refer to the tem¬ 
porary interaction with contacts or other environmental 
constraints [40.120,121]. It should be the subject of 
future research how such aspects could span a well- 
formalized set of features that leads to an appropriate 
prior over manipulation motions and respective opti¬ 
mization methods. 

The above discussion views motion as being mod¬ 
eled by a cost (or discriminative) function. This is close 
to the view of motion as being modeled by an attrac¬ 
tor (Lyapunov or potential) function; a cost function 
implies such a potential in the form of the cost-to- 
go function, and shifting the modeling directly to the 
cost-to-go function avoids the optimization step. The 
core issue of finding the right features for modeling 
the potential remains unchanged. TRIC [40.1 19] in fact 
models on the level of the cost-to-go instead of the cost 
function. 
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However, these views on modeling motion seem or¬ 
thogonal to sample-based path finding approaches, as 
they traditionally operate only in configuration space 
and employ a discrete notion of feasible/infeasible and 
goal region. 

Learning Sequential Manipulation 
from Demonstration 

It is fairly recent that learning has also been applied 
on the level of higher level sequential manipulation. 
Niekum et al. [40.122] present a complete system to 
analyze a demonstrated sequential manipulation and re¬ 
play it in a robust way on a robot, including its ability to 
search for alternative steps if some manipulation failed. 
Again, the choice of representation is the heart of this 
method: in [40.122] skill trees are used to represent the 
learned policy. Alexandrova et al. [40.123] extend such 
approaches by proposing an explicit user interface that 
allows the demonstrator to correct the robot’s interpre¬ 
tation of the task and e.g., explicitly communicate the 
relevant task spaces. 

Learning sequential manipulation is structurally one 
of the most interesting research challenges as it even¬ 
tually requires choices of representations and priors 
on all levels, from perception and motion generation 
to the higher level representation of action and coop¬ 
eration policies. The abovementioned work does first 
steps in the context of learning from demonstration. 
But much research needs to be done to enable robots 
to autonomously acquire skills that would allow them 
to control and manipulate the state of their environment 
as dexterously as we humans do. 

40.5 Perception 

The need for perception is one of the main culprits 
in making problems in manipulation high dimensional. 
Technological as well as biological agents possess 
many sensors to perceive their environment. Humans, 
for example, possess on the order of 300 million nerve 
endings that produce a rich, continuous stream of in¬ 
formation about the world, which is preprocessed and 
ultimately routed to the brain. The same is true in mo¬ 
bile manipulation: it is not uncommon for robots to be 
equipped with several cameras, each having millions of 
pixels, and each contributing to the high dimensionality 
of the input space. 

To illustrate this point: A simple, black and white 
(not gray scale!) camera with only one thousand pixels 
can sense many, many more distinct images (namely, 
2 1000 ss 10 310 ) than there are atoms in the observable 
universe (about 10 so ) or than nanoseconds have elapsed 
since the big bang (about 4.354 x 10 26 ). Even if ev- 


40.4.3 Toward Learning 

in Mobile Manipulation 

We above raised the question whether existing ma¬ 
chine learning methods can without need for mod¬ 
ification be applied in robotic domains, or whether 
genuinely novel issues are raised in the context of 
robot learning that necessarily go beyond core machine 
learning research. The above examples give two an¬ 
swers to this: First, standard machine learning tools 
like CRFs are widely and very successfully applied 
within the area mobile manipulation, employing off- 
the-shelf algorithms for training. Second, however, the 
above examples all show that the actual crux is in 
the modeling of the learning problem itself, in partic¬ 
ular in the choice of features, the used regularization 
and the chosen structure of the output. These are is¬ 
sues that require substantial domain expertise. In some 
cases, modern approaches to feature learning or fea¬ 
ture selection are indeed successful - particularly im¬ 
pressive are novel sparse coding principles to train 
perceptual features [40.115]. However, in many cases 
successful learning for mobile manipulation requires 
that a very specific feature space has been chosen based 
on the roboticist’s insight (e.g., the potential operational 
spaces in TRIC, or the shape concavity feature to pre¬ 
dict placement). 

If we had a more fundamental theory of real-world 
mobile manipulation problems, we could hope that such 
a theory would also inform us on what are promising 
and general feature spaces and priors for learning in 
manipulation. 


ery atom in the universe would generate one such 
image every nanosecond, all possible images would 
not have been generated by the time the universe 
dies its postulated heat death in 10 100 years. Biologi¬ 
cal agents can cope with this high dimensionality, of 
course, because there is substantial structure in the 
world, i. e., many of the possible images simply do 
not occur in nature or some of the variability in the 
data is not relevant for the task at hand and can be 
ignored. 

In this section, we will survey different success¬ 
ful approaches to perception for mobile manipulation 
that leverage this structure in perceptual data. We will 
mostly focus on visual perception, reflecting the em¬ 
phasis of ongoing research activities. The foundations 
of perception in robotics are presented in great detail 
in Part C of this Handbook. Chapter 42 also contains 
relevant approaches to perception. 
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40.5.1 Problem Description 

Perception is the process of interpreting sensory data 
to enable useful actions. In the context of robotics, this 
may be - given the state of the art - the most suitable, 
albeit very broad definition of the perception prob¬ 
lem. It differs appropriately from the definition most 
commonly used in psychology in that the goal of per¬ 
ception is not solely to gain an understanding of the 
world, but also to use such an understanding to en¬ 
able task-directed actions. This modification in fact can 
be seen as a simplification of the original perception 
problem, as only those aspects of the environment must 
be perceived that are relevant to a particular action. 
And, in fact, there is evidence from the human per¬ 
ceptual system that such a limitation does indeed take 
place, and sometimes leads to odd perceptual deficien¬ 
cies [40.124], 

In the context of robotics, the earliest perception 
work followed the psychologists’ definition of percep¬ 
tion. It focused on the use of visual perception to derive 
an exact 3-D model of the thing being perceived from 
its two-dimensional optical projection onto a camera 
sensor [40.125]. However, this problem is mathemat¬ 
ically ill-posed (three-dimensional shape information 
cannot be reconstructed unambiguously from its two- 
dimensional projection) and, as we argued above, also 
high dimensional. To enable robust and competent per¬ 
ception in the context of mobile manipulation, it there¬ 
fore seems sensible to attempt to reduce the complexity 
of perception, lending further support to our definition 
of perception above. 

40.5.2 Evaluation of the State of the Art 

Much of the work we will discuss in the remainder 
of this section follows the robotics-specific view of 
the perception problem given above. However, we will 
start out by surveying impressive advances made to¬ 
ward the objective of reconstructing three-dimensional, 
geometric models of the environment. Only the fu¬ 
ture can tell us if these methods are well suited to 
supporting the kind of complex and general manip¬ 
ulation tasks encountered in mobile manipulation. In 
the subsequent sections, we will then cover alternative 
approaches. 

3-D Mapping and Localization 
Recently, there has been remarkable progress in dense 
3-D mapping and visual SLAM. What originated with 
DTAM [40.126] lead to current systems like LSD- 
SLAM [40.127] which efficiently and rather robustly 
estimates a 3-D environment model in real-time, with¬ 
out GPU, from monocular video. 


Efficient Dense and Multiresolution Mapping 
Recent multiscale surfel models [40.128,129] using 
RGB-D sensors are an alternative to directly pixel- 
based reconstruction methods and provide a similarly 
dense 3-D model. In [40.130] and [40.131], a 3-D 
representation. Octree, is proposed having multiple res¬ 
olutions based on a probabilistic approach. This 3-D 
multiresolution method can provide flexible occupancy 
mapping depending on the demand of the high or low 
resolutions. Saarinen et al. [40.132] proposed normal 
distributions transform occupancy map (NDT-OM), 
which has the compactness from NDT maps and the 
robustness from occupancy maps. This algorithm sup¬ 
ports for multiresolution maps and it efficiently enables 
mapping in dynamic environments. 

These developments imply that, leaving system in¬ 
tegration problems aside, we may today assume to 
have access to precise 3-D mesh models of the envi¬ 
ronment. In the concrete mobile manipulation context 
3-D information is needed with high resolution only 
when the robot is near the object or the environment. 
Therefore, practical techniques for multiresolution map 
are developed based on the distance from the robot. 
Droeschel et al. [40.133, 134] show 3-D multiresolution 
maps where sparse resolution is used for the environ¬ 
ment far away and high resolution is used for the object 
to be interacted soon. 

Mapping and Tracking with Moving Objects 
Most methods for 3-D mesh estimation assume a static 
environment (but moving sensor). The segmentation 
into objects and dealing with moving objects - or rather 
exploiting the movement of objects - is left aside. 
However, fortunately recent approaches adopt a more 
integrated view on perception by jointly addressing the 
problems of dense 3-D reconstruction, moving object 
segmentation and tracking, and kinematic joint per¬ 
ception [40.135, 136], similarly as has previously been 
done on the basis of key points. We think that such 
integrated perception systems, fusing visual evidences 
and priors on multiple levels of representations (from 
low-level dense pixels to high-level rigid body hypothe¬ 
ses) should be the core effort of future research in this 
area. 

Other approaches deal with general dynamic envi¬ 
ronments by efficiently updating the map continuously. 
Especially, in the real world problem, there are al¬ 
ways moving obstacles and the environment itself can 
change over time. A incremental update algorithm is 
proposed in [40.137] for grid-based spatial representa¬ 
tions for dynamic environment due to moving obstacles. 
Distance maps, Voronoi diagrams, and configuration- 
space collision maps are updated with the algorithm for 
the changes in the environment. In [40.138], velocity 
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occupancy space is proposed by adding the velocity ob¬ 
stacle concept into occupancy grid. This is to enable the 
robot to efficiently navigate through uncertain moving 
obstacles. 

Indoor/Outdoor 

Finally, the usage of a map can be different for indoor 
or outdoor applications. The outdoor map is mainly 
used for navigation whereas the indoor map is for both 
navigation and manipulation. The map for outdoor or 
field applications needs to cover wider range of area 
for navigation and the mobile platforms do not require 
high resolution maps compared to indoor applications. 
Indoor applications, however, typically require higher 
resolution even for navigation because there are more 
obstacles and less free space. Because of this space 
restriction, the navigation route needs to be planned 
carefully considering the obstacles. Manipulation tasks 
are more involved in the indoor applications such as 
cleaning, connecting hoses, and locking a valve, which 
tasks must require much higher resolution than typical 
navigation tasks. 

For such different applications, the maps for indoor 
and outdoor can be quite different. Thus, in [40.139] 
pictures are used for the classification of the cur¬ 
rent environment. Specifically, Hue color component 
and color temperature values from each image are 
used with k-nearest neighbor (KNN) classifier to clas¬ 
sify the environment into indoor or outdoor. Payne 
and Singh [40.140] proposed a simple system for the 
indoor-outdoor classification, which can be used as 
a real-time system. It is based on the assumption that 
indoor images contain more straight lines that outdoor 
environments. 

Differently from indoors, the outdoor map often 
requires terrain information. In [40.141], the system 
classify if the current environment is indoor or outdoor, 
then terrain mapping is used for outdoor environment. It 
also provides seamless integration of mapping between 
indoor and outdoor environment. 

Active and Interactive Perception 
The success of the 3-D mapping approaches presented 
in the previous section is enabled by camera motion. 
This motion provides a continuous stream of incre¬ 
mentally changing two-dimensional projections of the 
three-dimensional scene. This enriched perceptual in¬ 
formation helps resolve the ambiguities of the projec¬ 
tion from 3-D to 2-D, thus overcoming limitations of 
static vision approaches. 

Perceptual algorithms that deliberately change the 
sensor to optimize the perceptual processes are called 
active. Among active perception methods, active vision 
is probably the most prominent one [40.142-144]. But 


active haptic perception methods also hold promise as 
a perceptual approach for mobile manipulation. Active 
haptic perception has been used for object localiza¬ 
tion [40.145, 146], contact point localization [40.147], 
and for estimating the object’s shape [40.148]. 

The concept of active perception can be extended to 
include interactions with the environment. Now the ob¬ 
server is able to manipulate the environment to reveal 
additional perceptual information, either by removing 
obstructions or by creating sensory signals that would 
otherwise not be observable. The class of perceptual al¬ 
gorithms relying on motion of the sensor as well as the 
deliberate application of forces to the environment is 
referred to as interactive perception. 

Interactive perception eliminates the traditional 
boundary between perception and manipulation, com¬ 
bining both into a tightly coupled feedback loop. 
Manipulation now becomes an integral part of per¬ 
ception - in addition to being an objective in itself. 
At the same time, perception of task-relevant informa¬ 
tion is now critically enabled by manipulation. This 
combined perception-and-manipulation process links 
inseparably the goals of achieving a manipulation task 
and of acquiring perceptual information. These two 
goals must be balanced appropriately, as it is not 
desirable to perceive the world without progress to¬ 
ward the task, and it most often is not possible to 
achieve the task without perceiving something about 
the world first. Now, in addition to perception and 
manipulation, we must also consider aspects of ma¬ 
chine learning: how do we balance progress toward 
the task (exploitation) with the goal of acquiring suffi¬ 
cient information to reach the goal robustly and reliably 
(exploration)? These and many related questions are 
current topics of research in this emerging field at the 
intersection of perception, manipulation, and machine 
learning. 

Using interactive perception (or perceptual manipu¬ 
lation, if one prefers), it is possible to separate piles of 
objects so that they can be perceived, manipulated, and 
sorted [40.149-151], It is also possible to interactively 
search for objects by removing obstructions [40.152]. 
Using interactive perception, it becomes possible to 
reveal sensory information about the articulations of 
objects in the environment, i. e., about their inherent de¬ 
grees of freedom [40.153]. This ability is a perceptional 
prerequisite for mobile manipulation, as the degrees of 
freedom of an object are generally linked to its func¬ 
tion. For example, by pushing on a door handle and 
observing its motion, it is possible to perceive the rev¬ 
olute joint connecting the handle to the door. Operating 
this revolute joint is necessary to open the door. 

Interactive perception approaches based purely on 
vision data exist for planar objects [40.154] and three- 


Part D | 40.5 



Part D | 40.5 


1028 Part D 


Manipulation and Interfaces 


dimensional articulated objects [40.155,156]. Their 
performance can be improved significantly by relying 
on RGB-D data [40.157,158]. Interactive perception 
methods have been integrated with object databases 
so as to recognize previously seen objects and the as¬ 
sociated kinematic model [40.159]. Online interactive 
perception enables the integration of interactive per¬ 
ception directly into the manipulation process [40.153]. 
This permits to detect failures, monitor progress, and 
identify the successful completion of a manipulation 
action. It is also possible to categorize novel types of 
articulation with interactive perception [40.160, 161]. 
Interactive perception can include additional modali¬ 
ties, such as proprioception and audition, for example, 
to identify objects [40.162]. 

One of the challenges remains the selection of ap¬ 
propriate actions for interactive perception. There are 
many possible ways to interact with the environment 
but only few of them will reveal relevant informa¬ 
tion. The selection of promising actions can be learned 
from experience [40.163] or by entropy-driven explo¬ 
ration [40.164]. The information obtained from such 
exploration can be represented in relational knowledge 
about the world [40.165]. 

The work discussed above has demonstrated that 
interactive perception is a promising approach to per¬ 
ception and manipulation in the context of mobile 
manipulation. It has resulted in robust perception that 
relies on few assumptions about the world, and in par¬ 
ticular does not rely on the existence of prior geometric 
models. It also has resulted in robust manipulation, 
supported by the most task-relevant perceptual feed¬ 
back. Interactive perception eliminates the boundaries 
between perception, action, and learning. While this 
seems like a complication, the close integration of these 
different aspects of robot behavior lead to novel so¬ 
lutions of long-standing challenges in perception and 
manipulation. 

Affordance Detection 

An affordance can be viewed as an opportunity of an 
agent to perform an action in its environment. The 
term dates back to 1977, when it was introduced by 
psychologist Gibson [40.166]. Since then the term has 
been used inconsistently with slight variations of mean¬ 
ing [40.167], 

Here, we regard an affordance as an opportunity to 
instantiate a behavioral capability of the agent. Such an 
instantiation of a pushing action, for example, selects 
the objects involved (the robot’s hand and a door han¬ 


dle within reach) and determines other free parameters 
of the behavior (pushing down on one side of the door 
handle). 

It is appealing to focus perception of affordances as, 
by definition, this will focus perception only on those 
aspects of the environment that must be known to in¬ 
stantiate and successfully execute a certain behavior. 

Research on affordance perception is intimately 
linked with research on affordance learning [40.168- 
170]. A modern view perspective is to consider affor¬ 
dance learning for manipulation as a statistical rela¬ 
tional learning problem [40.171]. 

More low-level affordance perception research con¬ 
siders the problem of labeling scene segments by their 
affordance of certain actions, often using conditional 
random fields (CRFs) to train the labeled segmentation. 
This has extensively been addressed in the case of learn¬ 
ing to detected grasp affordances [40.172-175]. 

40.5.3 Toward Robust Perception 

Perception is one of the great challenges in mobile 
manipulation. We recall that the objective of mobile 
manipulation research is to a) maximize task general¬ 
ity of autonomous robotic systems, while at the same 
time b) minimizing the dependence on task-specific, 
hard-coded, or narrowly-relevant information. Given 
these objectives, it becomes clear that the complex and 
diverse information required for task generally must 
be acquired through perception. And perception must 
also compensate for the envisaged reduction of a pri¬ 
ori knowledge available to the agent. Perception plays 
a central role in realizing mobile manipulation. 

In this section, we surveyed some approaches de¬ 
signed to meet the challenges posed by the perception 
requirements of mobile manipulation. While a mul¬ 
titude of promising approaches exist, today there is 
no generally accepted framework or philosophy for 
machine perception. Evidence from human perception 
may suggest that such a common framework exists. 
Some results seem to imply that there are some shared 
working principles between vision and audition [40.3]. 
Irrespective of this, there is strong evidence that human 
perception is built to support action. This becomes ap¬ 
parent already in the retina, which is far from being 
a general sense organ; it contains highly specialized 
circuitry tailored for task-relevance [40.176]. Maybe 
this too can be a clue for how to address percep¬ 
tion in robotics: Maybe we need much better low-level 
features? 
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While mobile manipulation requires the integration of 
methodologies from almost all aspects of robotics - 
we mentioned grasping, control, motion generation, 
learning and perception - it also suggests alterna¬ 
tive approaches to the state of the art in these ar¬ 
eas of research. Instead of considering each research 
area as independent, research in mobile manpulation 
exploits their interdependence when this simplifies 
the overall problem. Active and interactive percep¬ 
tion, i. e., combining manipulation and 3-D percep¬ 
tion, is one example that departs from the goals of 
the individual subareas: (1) general 3-D reconstruc¬ 
tion from passive data and (2) manipulation planning 


given a 3-D environment model. Instead, these ap¬ 
proaches view both as an integrated and potentially 
simpler problem. 

In the present survey we only briefly covered the 
various aspects and their interdependencies. Which of 
the approaches will turn out to be the most successful 
for the integrated overall goal of mobile manipulation is 
an ongoing debate. An interesting starting point for fur¬ 
ther reading are the various workshops that focus on the 
integrated nature of mobile manipulation and robotics 
as a whole. A list of these workshops is maintained on 
the Website of the Technical Committee on Mobile Ma¬ 
nipulation. 
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Learning dexterous grasps that generalize to novel objects by combining hand and contact models 
available from http://handbookofrobotics.org/view-chapter/40/videodetails/650 
Atlas whole-body grasping 

available from http://handbookofrobotics.org/view-chapter/40/videodetails/651 
Handle localization and grasping 

available from http://handbookofrobotics.org/view-chapter/40/videodetails/652 
Catching objects in flight 

available from http://handbookofrobotics.org/view-chapter/40/videodetails/653 
Avian-inspired grasping for quadrotor micro UAVs 

available from http://handbookofrobotics.org/view-chapter/40/videodetails/654 
A compliant underactuated hand for robust manipulation 
available from http://handbookofrobotics.org/view-chapter/40/videodetails/655 
Yale Aerial Manipulator - Dollar Grasp Lab 

available from http://handbookofrobotics.org/view-chapter/40/videodetails/656 
Exploitation of environmental constraints in human and robotic grasping 
available from http://handbookofrobotics.org/view-chapter/40/videodetails/657 
Adaptive synergies for a humanoid robot hand 

available from http://handbookofrobotics.org/view-chapter/40/videodetails/658 
Universal gripper 

available from http://handbookofrobotics.org/view-chapter/40/videodetails/660 
DLR's Agile Justin plays catch with Rollin' Justin 

available from http://handbookofrobotics.org/view-chapter/40/videodetails/661 
Atlas walking and manipulation 

available from http://handbookofrobotics.org/view-chapter/40/videodetails/662 
Dynamic robot manipulation 

available from http://handbookofrobotics.org/view-chapter/40/videodetails/664 
CHOMP trajectory optimization 

available from http://handbookofrobotics.org/view-chapter/40/videodetails/665 
Motor skill learning for robotics 

available from http://handbookofrobotics.org/view-chapter/40/videodetails/667 
Policy learning 

available from http://handbookofrobotics.org/view-chapter/40/videodetails/668 
Autonomous robot skill acquisition 

available from http://handbookofrobotics.org/view-chapter/40/videodetails/669 
State representation learning for robotics 

available from http://handbookofrobotics.org/view-chapter/40/videodetails/670 
Extracting kinematic background knowledge from interactions using task-sensitive relational learning 
available from http://handbookofrobotics.org/view-chapter/40/videodetails/671 
DART: Dense articulated real-time tracking 

available from http://handbookofrobotics.org/view-chapter/40/videodetails/673 
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Reaching in clutter with whole-arm tactile sensing 

available from http://handbookofrobotics.org/view-chapter/40/videodetails/674 
Adaptive force/velocity control for opening unknown doors 
available from http://handbookofrobotics.org/view-chapter/40/videodetails/675 
Interactive perception of articulated objects 

available from http://handbookofrobotics.org/view-chapter/40/videodetails/676 
A day in the life of Romeo and Juliet (Mobile Manipulators) 
available from http://handbookofrobotics.org/view-chapter/40/videodetails/776 
Flight stability in aerial redundant manipulator 

available from http://handbookofrobotics.org/view-chapter/40/videodetails/782 

HERMES, A humanoid experimental robot for mobile manipulation and exploration services 

available from http://handbookofrobotics.org/view-chapter/40/videodetails/783 

Task consistent Obstacle avoidance for mobile manipulation 

available from http://handbookofrobotics.org/view-chapter/40/videodetails/784 

Handling of a single object by multiple mobile robots based on caster-like dynamics 

available from http://handbookofrobotics.org/view-chapter/40/videodetails/785 

Rolling Justin - a platform for mobile manipulation 

available from http://handbookofrobotics.org/view-chapter/40/videodetails/786 
Combined mobility and manipulation - operational space control of free-flying space robots 
available from http://handbookofrobotics.org/view-chapter/40/videodetails/787 
Mobile robot helper 

available from http://handbookofrobotics.org/view-chapter/40/videodetails/788 
Free-floating autonomous underwater manipulation: Connector plug/unplug 
available from http://handbookofrobotics.org/view-chapter/40/videodetails/789 
Development of a versatile underwater robot - GTS ROV ALPHA 
available from http://handbookofrobotics.org/view-chapter/40/videodetails/790 
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41. Active Manipulation for Perception 


Anna Petrovskaya, Kaijen Hsiao 


This chapter covers perceptual methods in which 
manipulation is an integral part of perception. 
These methods face special challenges due to data 
sparsity and high costs of sensing actions. How¬ 
ever, they can also succeed where other perceptual 
methods fail, for example, in poor-visibility con¬ 
ditions or for learning the physical properties of 
a scene. 

The chapter focuses on specialized methods 
that have been developed for object localization, 
inference, planning, recognition, and modeling in 
active manipulation approaches. We conclude with 
a discussion of real-life applicationsand directions 
for future research. 
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41.1 Perception via Manipulation 

In this chapter, we focus on perceptual methods that 
rely on manipulation. As we will see, sensing-vw- 
manipulation is a complex and highly intertwined 
process, in which the results of perception are used 
for manipulation and manipulation is used to gather 
additional data. Despite this complexity, there are 
three main reasons to use manipulation either to¬ 
gether or instead of the more commonly used vision 
sensors. 

First, manipulation can be used to sense in poor- 
visibility conditions: for example, sensing in muddy 
water or working with transparent objects. In fact, due 
to its high accuracy and ability to sense any hard sur¬ 
face regardless of its optical properties, contact sensing 
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is widely used for accurate localization of parts in 
manufacturing. Second, manipulation is useful for de¬ 
termining properties that require physical interaction, 
such as stiffness, mass, or the physical relationship of 
parts. Third, if the actual goal is to manipulate the 
object, we might as well use data gathered from ma¬ 
nipulation attempts to improve both perception and 
subsequent attempts at manipulation. 

Sensing-vra-manipulation also faces significant 
challenges. Unlike vision sensors, which provide 
a whole-scene view in a single snapshot, contact sensors 
are inherently local, providing information only about 
a very small area of the sensed surface at any given 
time. In order to gather additional data, the manipu- 
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lator has to be moved into a different position, which 
is a time-consuming task. Hence, unlike vision-based 
perception, contact-based methods have to cope with 
very sparse data and very low data acquisition rates. 
Moreover, contact sensing disturbs the scene. While in 
some situations this can be desirable, it also means that 
each sensing action can increase uncertainty. In fact, 
one can easily end up in a situation where the infor¬ 
mation gained via sensing is less than the information 
lost due to scene disturbance. 

Due to these challenges, specialized perceptual 
methods had to be developed for perception-via- 
manipulation. Inference methods have been developed 
to extract the most information from the scarce data and 
to cope with cases in which the data are insufficient to 
fully solve the problem (i. e., under-constrained scenar¬ 


41.2 Object Localization 

Tactile object localization is the problem of estimating 
the object’s pose - including position and orientation - 
based on a set of data obtained by touching the object. 
A prior geometric model of the object is assumed to be 
known and the sensing is performed using some type of 
contact sensor (wrist force/torque sensor, tactile array, 
fingertip force sensor, etc.). The problem is typically 
restricted to rigid objects, which can be stationary or 
moving. 

Tactile object localization involves several compo¬ 
nents, including manipulator control methods, model¬ 
ing, inference, and planning. During data collection, 
some form of sensorimotor control is used for each 
sensing motion of the manipulator. For information on 
manipulator control, the reader is encouraged to refer¬ 
ence Chaps. 8 and 9. With few exceptions, the sensing 
motions tend to be poking motions rather than fol¬ 
lowing the surface. This is due to the fact that robots 
are usually unable to sense with all parts of the ma¬ 
nipulator, and, hence, the poking motions are chosen 
to minimize the possibility of accidental non-sensed 
contact. 

Due to the properties of sensing-via-contact, spe¬ 
cial methods for modeling, inference, and planning 
had to be developed for tactile localization. Model¬ 
ing choices include not only how to model the object 
itself, but also models of the sensing process and 
models of possible object motion. We discuss mod¬ 
els of sensing and motion in Sect. 41.2.2, but leave 
an in-depth discussion of object modeling techniques 
until Sect. 41.3. We also cover specialized inference 
and planning methods in Sect. 41.2.3 and 41.2.5, 
respectively. 


ios). Planning methods have been developed to make 
the most efficient sensing decisions based on the little 
data available and taking into account time, energy, and 
uncertainty costs of sensing actions. Modeling methods 
have been developed to build the most accurate models 
from small amounts of data. 

In this chapter, the material is organized by per¬ 
ceptual goal: localizing an object (Sect. 41.2), learning 
about an object (Sect. 41.3), and recognizing an object 
(Sect. 41.4). Since the chapter topic is broad, to keep it 
concise, we provide a tutorial-level discussion of object 
localization methods, but give a birds-eye view of the 
methods for the other two perceptual goals. However, 
as we point out along the way, many object localiza¬ 
tion methods can be reused for the other two perceptual 
goals. 


41.2.1 Problem Evolution 

Attempts to solve the tactile object localization date 
back to early 80s. Over the years, the scope of the prob¬ 
lem as well as methods used to solve it have evolved as 
we detail below. 

Early Methods 

Early methods for tactile object localization generally 
ignore the sensing process uncertainties and focus on 
finding a single hypothesis that best fits the measure¬ 
ments. For example, Gaston and Lozano-Perez used 
interpretation trees to efficiently find the best match for 
3 DOF (degrees of freedom) object localization [41.1]. 
Crimson and Lozano-Perez extended the approach to 
6 DOF [41.2]. Faugeras and Hebert used least squares 
to perform geometrical matching between primitive sur¬ 
faces [41.3]. Shekhar et al. solved systems of weighted 
linear equations to localize an object held in a robotic 
hand [41.4]. Several methods use geometric constraints 
together with kinematic and dynamic equations to esti¬ 
mate or constrain the pose of known planar objects in 
plane by pushing them with a finger [41.5] or parallel- 
jaw gripper [41.6], or by tilting them in a tray [41.7]. 

Workpiece Localization 

Single hypothesis methods are also widely used to solve 
the workpiece localization problem in manufacturing 
applications for dimensional inspection [41.8], ma¬ 
chining [41.9], and robotic assembly [41.10]. In these 
applications, the measurements are taken by a coordi¬ 
nate measurement machine (CMM) [41.11] or by on- 
machine sensors [41.12]. Workpiece localization makes 
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a number of restrictive assumptions that make it inap¬ 
plicable to autonomous robot operation in unstructured 
environments. One important restriction is that there is 
a known correspondence between each measured data 
point and a point or patch on the object surface (called 
home point or home surface, respectively) [41.13], In 
semiautomated settings, the correspondence assump¬ 
tion is satisfied by having a human direct the robot to 
specific locations on the object. In fully automated set¬ 
tings, the object is placed on the measurement table 
with low uncertainty to make sure each data point lands 
near the corresponding home point. 

Further restrictions include assumptions that the 
data are sufficient to fully constrain the object, the ob¬ 
ject does not move, and there are no unmodeled effects 
(e.g., vibration, deformation, or temperature variation). 
All of these parameters are carefully controlled for in 
structured manufacturing environments. 

The workpiece localization problem is usually 
solved in least squares form using iterative optimiza¬ 
tion methods, including the Hong-Tan method [41.14], 
the Variational method [41.15], and the Menq 
method [41.16]. Since these methods are prone to get¬ 
ting trapped in local minima, low initial uncertainty 
is usually assumed to make sure the optimization al¬ 
gorithm is initialized near the solution. Some attempts 
have been made to solve the global localization problem 
by rerunning the optimization algorithm multiple times 
from prespecified and random initial points [41.17]. Re¬ 
cent work has focused on careful selection of the home 
points to improve localization results [41.18-20] and 
on improving localization efficiency for complex home 
surfaces [41.21,22]. 

Bayesian Methods 

In the last decade, there has been increased interest in 
Bayesian state estimation for the tactile object local¬ 
ization problem [41.23-27]. These methods estimate 
the probability distribution over all possible states (the 
belief), which captures the uncertainty resulting from 
noisy sensors, inaccurate object models, and other ef¬ 
fects present during the sensing process. Thus, estima¬ 
tion of the belief enables planning algorithms that are 
resilient to the uncertainties of the real world. Unlike 
workpiece localization, these methods do not assume 
known correspondences. In contrast to single hypoth¬ 
esis or set-based methods, belief estimation methods 
allow us to better describe and track the relative likeli¬ 
hoods of hypotheses in the under-constrained scenario, 
in which the data are insufficient to fully localize the 
object. These methods can also work with moving ob¬ 
jects and answer important questions such as: have we 
localized the object completely? and where is the best 
place to sense next?. 


All Bayesian methods share a similar framework. We 
start with a general definition of a Bayesian problem 
(not necessarily tactile object localization), and then, 
explain how this formulation can be applied to tactile 
localization. 

General Bayesian Problem 
For a general Bayesian problem, the goal is to infer the 
state X of a system based on a set of sensor measure¬ 
ments T) := | Di ). Due to uncertainty, this information 
is best captured as a probability distribution 

bel(X) := p(X\D) , (41.1) 

called the posterior distribution or the Bayesian belief. 
Figure 41.1a shows a Bayesian network representing all 
the random variables involved and all the relationships 
between them. 

In a dynamic Bayesian system, the state changes 
over time, which is assumed to be discretized into small 
time intervals (Fig. 41.1b). The system is assumed to 
evolve as a Markov process with unobserved states. The 
goal is to estimate the belief at time t 

bel t (X t ):=p(X t \V l ,...,V t ). (41.2) 

The behavior of the system is described via two prob¬ 
abilistic laws: (i) the measurement model p(D\X) cap¬ 
tures how the sensor measurements are obtained and (ii) 
the dynamics model p(X,\X t —\) captures how the sys¬ 
tem evolves between time steps. 

For brevity, it is convenient to drop the arguments 
in bel(X) and bel,(X t ), and simply write bel and bel t , 
but these two beliefs should always be understood as 
functions of X and X u respectively. 

Tactile Localization in Bayesian Form 
Tactile object localization can be formulated as an 
instance of the general Bayesian problem. Here, the 


a) Static case b) Dynamic case 



Fig.41.1a,b Bayesian network representation of the relationships 
between the random variables involved in a general Bayesian prob¬ 
lem. The directional arrows are read as causes, (a) In the static case, 
a single unknown state X causes a collection of measurements {At}- 
(b) In the dynamic case, the state X, changes over time and at each 
time step causes a set of measurements V, 
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Fig.41.2a,b Examples of objects (a) and their polygonal mesh models (b). Five objects are shown: cash register, toy 
guitar, toaster, box, and door handle. The first three models were constructed by collecting surface points with the 
robot’s end effector. The last two were constructed from hand measurements made with a ruler. The door handle model 
is 2-D. Model complexity ranges from 6 faces (for the box) to over 100 faces (for the toaster) (after [41.28]) 


robot needs to determine the pose X of a known ob¬ 
ject O based on a set of tactile measurements T). 
The state is the 6 DOF pose of the object - includ¬ 
ing position and orientation - in the manipulator co¬ 
ordinate frame. A number of state parameterizations 
are possible, including matrix representations, quater¬ 
nions, Euler angles, and Rodrigues angles, to name 
a few. For our discussion here, assume that the state 
X := ( x , v, z, a, /3, y), where ( x , y, z) is the position and 
(a, /3, y) are orientation angles in Euler representation. 

The measurements D : = { D k } are obtained by 
touching the object with the robot’s end effector. The 
end effector can be a single probe, a gripper, or a hand. 
Thus, it may be able to sense multiple contacts D k si¬ 
multaneously, for example, in a single grasping attempt. 
We will assume that for each touch D k , the robot is able 
to measure the contact point and also possibly sense 
the normal of the surface (perhaps as the direction of 
reaction force). Under these assumptions, each mea¬ 
surement D k := (D^ os ,D" or ) consists of the measured 
Cartesian position of the contact point D pos and the 
measured surface normal D™. If measurements of sur¬ 
face normals are not available, then D k := D pos . 

Measurement Model 

In order to interpret tactile measurements, we need to 
define a model of the object and a model of the sens¬ 
ing process. For now, assume that the object is modeled 
as a polygonal mesh, which could be derived from 
a CAD model or a 3-D scan of the object, or built 
using contact sensors. Examples of objects and their 
polygonal mesh representations are shown in Fig. 41.2. 
Other types of object models can also be used. We pro¬ 
vide an in-depth look at object modeling techniques in 
Sect. 41.3. 

Typically, the individual measurements D k in a data 
set T) are considered independent of each other given 
the state X. Then, the measurement model factors over 


the measurements 

p(V\X) = Y\p(D k \X). (41.3) 

k 

Sampled Models. Early Bayesian tactile localization 
work used sampled measurement models. For example, 
Gadeyne and Bruyninckx used numerical integration to 
compute the measurement model for a box and stored 
it in a look-up table for fast access [41.23]. Chhatpar 
and Branicky sampled the object surface by repeatedly 
touching it with the robot’s end-effector to compute the 
measurement probabilities (Fig. 41.3) [41.24]. 

Proximity Model. One common model for the inter¬ 
pretation of tactile measurements is the proximity mea¬ 
surement model [41.29]. In this model, the measure¬ 
ments are considered to be independent of each other, 
with both position and normal components corrupted by 
Gaussian noise. For each measurement, the probability 
depends on the distance between the measurement and 
the object surface (hence the name proximity). 

b> 

■■9F*. S 

Fig.41.3a r b Sampled measurement model for a key lock, 
(a) The robot exploring the object with a key. (b) The re¬ 
sulting lock-key contact C-space (after [41.24]) 
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Since the measurements contain both contact coor¬ 
dinates and surface normals, this distance is taken in 
the 6-D space of coordinates and normals (i. e., in the 
measurement space). Let O be a representation of the 
object in this 6-D space. Let o := (o pos , o nor ) be a point 
on the object surface, and D be a measurement. Define 
d\\(b, D ) to be the Mahalanobis distance between o and 
D 


du(o,D) 


11 O pos _Z)P0S||2 11 O nor — ZD nor 11 2 


(41.4) 


where <7p 0S and a„ or are Gaussian noise variances of 
position and normal measurement components, respec¬ 
tively. In the case of a sensor that only measures posi¬ 
tion (and not surface normal), only the first summand 
is used inside the square root. The distance between 
a measurement D and the entire object O is obtained 
by minimizing the Mahalonobis distance over all object 
points o 

c/m(O.D) := mindM(o, D) . (41.5) 

See) 


Let Ox denote the object in state X. Then, the mea¬ 
surement model is computed as 


Negative Information. The models we have de¬ 
scribed so far do not take into account negative infor¬ 
mation, i. e., information about the absence rather than 
presence of measurements. This includes information 
that the robot was able to move through some parts of 
space without making contact with the object. Nega¬ 
tive information is very useful for active exploration 
strategies and has been taken into account by [41.26] 
and [41.25], Although adding negative information 
makes the belief more complex (e.g., discontinuous), in 
most cases, it can be superimposed on top of the belief 
computed using one of the above measurement models. 

Dynamics Model 

Free-standing objects can move during probing, and 
hence, a dynamics model is needed to describe this pro¬ 
cess. In most situations, little is known about possible 
object motions, and thus, a simple Gaussian model is 
assumed. Hence, p(X t \X t — ]) is a Gaussian with mean at 
X,—i and variances and cr^ ng along metric and angu¬ 
lar axes respectively. If additional properties of object 
motion are known, then a more informative dynam¬ 
ics model can be used. For example, if we know the 
robot shape and the magnitude and direction of the con¬ 
tact force, we can use Newtonian dynamics to describe 
object motion. This motion may be further restricted, 
if it is known that the object is sliding on a specific 
surface. 


p( D\X) = )]exp 


(Ox,O k ) 

k 


(41.6) 


In the above equation and throughout the chapter, rf de¬ 
notes the normalization constant, whose value is such 
that the expression integrates to 1. shows 

the proximity model being used to localize a door han¬ 
dle. 


Integrated Proximity Model. A variation of the prox¬ 
imity model is called the integrated proximity model. 
Instead of assuming that the closest point on the object 
caused the measurement, it considers the contribution 
from all surface points to the probability of the mea¬ 
surement [41.25]. This is a much more complex model 
that in general can not be computed efficiently. More¬ 
over, for an unbiased application of this model, we need 
to compute a prior over all surface points, i. e., how 
likely each surface point is to cause a measurement. 
This prior is usually nonuniform and highly depen¬ 
dent on the object shape, the manipulator shape, and 
the probing motions. Nevertheless, in some cases this 
model can be beneficial as it is more expressive than 
the proximity model. 


41.2.3 Inference 

Once the models are defined and the sensor data are 
obtained, the next step is to estimate the resulting prob¬ 
abilistic belief. This process is called inference and 
can be viewed as numerical estimation of a real-valued 
function over a multidimensional space. We can distin¬ 
guish two cases: a static case where the object is rigidly 
fixed and a dynamic case where the object can move. 
The dynamic case is solved recursively at each time step 
using a Bayesian filter. The static case can be solved 
either recursively or in a single step by combining all 
the data into a single batch. The other important dif¬ 
ferentiator is the amount of initial uncertainty. In low 
uncertainty problems, the pose of the object is approx¬ 
imately known and only needs to be updated based on 
the latest data. This type of problem usually arises dur¬ 
ing tracking of a moving object, in which case it is 
known as pose tracking. In global uncertainty problems, 
the object can be anywhere within a large volume of 
space and can have arbitrary orientation. This case is 
often referred to as global localization. Global local¬ 
ization problems arise for static objects, when the pose 
of the object is unknown, or as the initial step in pose 
tracking of a dynamic object. It is also the fallback 
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whenever the pose tracking method fails and uncer¬ 
tainty about the object pose becomes large. 

The belief tends to be a highly complex function 
with many local extrema. For this reason, parametric 
methods (such as Kalman filters) do not do well for this 
problem. Instead, the problem is usually solved using 
nonparametric methods. The complexity of the belief is 
caused directly by properties of the world, the sensors, 
and models thereof. For more insight into causes of be¬ 
lief complexity and appropriate estimation methods, see 
the discussion of belief roughness in [41.28, Chap. 1], 

We start by describing basic nonparametric meth¬ 
ods, which are efficient enough to solve problems 
with up to three degrees of freedom (DOFs). Then, in 
Sect. 41.2.4, we describe several advanced methods that 
have been developed to solve the full 6 DOF problem in 
real-time. 

Basic Nonparametric Methods 
Nonparametric methods typically approximate the be¬ 
lief by points. There are two types of nonparametric 
methods: deterministic and Monte Carlo (i. e., nonde- 
terministic). The most common deterministic methods 
are grids and histogram filters (HF). For these meth¬ 
ods, the points are arranged in a grid pattern with each 
point representing a grid cell. The most common Monte 
Carlo methods are importance sampling (IS) and parti¬ 
cle filters (PF). For Monte Carlo methods, the points 
are sampled randomly from the state space and called 
samples or particles. 

Static Case 

Via Bayes rule, the belief bel(X) can be shown to be 
proportional to p(D\X)p(X). The first factor is the mea¬ 
surement model. The second factor, bel(X) := p(X ), is 
called the Bayesian prior, which represents our belief 
about X before obtaining measurements T). Hence, with 
this notation, we can write 

bel = r]p(D\X) bel. (41.7) 

In the most common case, where it is only known that 
the object is located within some bounded region of 
space, a uniform prior is most appropriate. In this case, 
the belief bel is proportional to the measurement model 
and the equation (41.7) simplifies to 

bel = r}p(D\X) . (41.8) 

However, if an approximate object pose is known ahead 
of time (e.g., an estimate produced from a vision sen¬ 
sor), then it is common to use a Gaussian prior of some 
covariance around the approximate pose. 

Grid methods compute the unnormalized belief 
p(D\X) bel at the center of each grid cell and then nor¬ 


malize over all the grid cells to obtain an estimate of the 
belief bel. 

Monte Carlo methods use importance sampling. For 
uniform priors, the particles are sampled uniformly 
from the state space. For more complex priors, the parti¬ 
cles have to be sampled from the prior distribution bel. 
The importance weight for each particle is set to the 
measurement model p(D\X) and the entire set of im¬ 
portance weights is normalized so that the weights add 
up to 1. 

Dynamic Case 

Given the measurement model p(D\X), the dynamics 
model p(X t \X t —\), and measurements {Di, ..., D,} up 
to time t, the belief can be computed recursively us¬ 
ing a Bayesian filter algorithm, which relies on the 
Bayesian recursion equation 

bel, = ij p(T),\X,) bel,. (41.9) 

To compute (41.9), the Bayesian filter alternates two 
steps: (a) the dynamics update, which computes the 
prior 

bel, = i] Jp(X,\X,—i)bel,—idX,—i, (41.10) 

and (b) the measurement update, which computes the 
measurement model p(D,\X,). 

A grid-based implementation of the Bayesian filter 
is called a histogram filter. Histogram filters compute 
(41.9) for each grid cell separately. At the start of time 
step t, the grid cell stores its estimate of the belief 
bel ,—1 from the previous time step. During the dynam¬ 
ics update, the value of the prior bel, at this grid cell is 
computed using (41.10), where the integral is replaced 
by a summation over all grid cells. During the measure¬ 
ment update, the measurement model is computed at 
the center of the grid cell and multiplied by the value 
of the grid cell (i. e., the prior). Then, values of all grid 
cells are normalized so they add up to 1. Since during 
dynamics update, a summation over all grid cells has to 
be carried out for each grid cell, the computational com¬ 
plexity of histogram filters is quadratic in the number of 
grid cells. 

A Monte Carlo implementation of the Bayesian fil¬ 
ter is called a particle filter. The particle set representing 
the prior bel, is produced during the dynamics update 
as follows. The particles are resampled (see [41.30] for 
a tutorial on resampling) from the belief at the previous 
time step, bel,— 1 , and then moved according to the dy¬ 
namics model with the addition of some motion noise. 
This produces a set of particles representing the prior 
bel,. The measurement update is performed using im¬ 
portance sampling as in the static case above. Note that 



Active Manipulation for Perception I 41.2 Object Localization 1043 


unlike for the histogram filters, there is no per-particle 
summation during the dynamics update, and thus, the 
computational complexity of particle filters is linear in 
the number of particles. 

41.2.4 Advanced Inference Methods 

Although suitable for some problems, basic nonpara- 
metric methods have a number of shortcomings. There 
are two main issues. 

1. While a rigid body positioned in space has 6 DOFs, 
the basic methods can only be used for localiza¬ 
tion with up to 3 DOFs. Beyond 3 DOFs, these 
methods become computationally prohibitive due 
to exponential growth in the number of particles 
(or grid cells) required. The reason for the expo¬ 
nential blow-up is as follows. In order to localize 
the object, we need to find its most likely poses, 
i. e., the peaks of bel,. These peaks (also called 
modes) are very narrow, and. hence, the space needs 
to be densely populated with particles in order to 
locate them. The number of particles required to 
achieve the same density per unit volume goes up 
exponentially with the number of DOFs. Hence, the 
computational complexity of nonparametric meth¬ 
ods also grows exponentially with the number of 
DOFs. This is known as the curse of dimensionality. 

2. Another important issue is that the performance 
of basic nonparametric methods actually degrades 
with increase in sensor accuracy. This problem is 
especially pronounced in particle filters. Although 
it may seem very unintuitive at first, the reason is 
simple. The more accurate the sensor the narrower 
the peaks of the belief. Thus, more and more par¬ 
ticles are required to find these peaks. We can call 
this problem curse of accurate sensor. 

Several more advanced methods have been devel¬ 
oped to combat the shortcomings of basic methods. 
Below, we detail one such method, called scaling se¬ 
ries, and briefly discuss a few others. 

Scaling Series 

Scaling series is an inference algorithm designed to 
scale better to higher dimensional problems than the 
basic methods [41.28, Chap. 2]. In particular, it is 
capable of solving the full 6 DOF localization prob¬ 
lem. Moreover, unlike for basic methods, the accu¬ 
racy of scaling series improves with increase in sen¬ 
sor accuracy. For this reason, this algorithm may be 
preferable to basic methods even in low-dimensional 
problems. 

Scaling series represents the belief by broad parti¬ 
cles. Each particle is thought of as a representative for 


an entire 5-neighborhood, that is, the volume of space of 
radius 5 surrounding the sample point. Instead of fixing 
the total number of particles (as in the basic methods), 
scaling series adjusts this number as needed to obtain 
good coverage of main peaks. 

Peak width can be controlled using annealing, 
which means that for a given temperature r, the mea¬ 
surement model is raised to the power of 1/r. Hence, 
for r = 1, the original measurement model is obtained, 
whereas for r > 1 the measurement model is heated- 
up. Annealing broadens the peaks of bel t , making them 
easier to find. However, it also increases ambiguity and 
decreases accuracy (Fig. 41.4). To make sure that ac¬ 
curacy is not compromised, scaling series combines 
annealing with iterative refinements as we detail below. 

The Static Case Algorithm. Let us first consider the 
case, where the prior bel is uniform. Scaling series starts 
by uniformly populating the space with very broad 
particles (i. e., with large 8). It evaluates the annealed 
measurement model for each particle and prunes out 
low probability regions of space. Then, it refines the 
estimate with slightly narrower particles, and repeats 
again and again until a sufficiently accurate estimate is 
obtained. Peak width is controlled during iterations us¬ 
ing annealing, to make sure that each particle is able 
to represent its 5-neighborhood well. As the value of 
8 decreases during iterations, the density of particles 
per volume of 5-neighborhood is kept constant to main¬ 
tain good particle density near peaks. We can think of 
5 as peak width, which decreases (due to annealing) 
with iterations. At each iteration, the 5-neighborhood 
of each peak will have the same fixed number of 
particles. 

The formal algorithm listing is given in Algo¬ 
rithm 41.1, where Ss denotes a 5-neighborhood 
and d i m X is the dimensionality of the state 
space. The algorithm relies on three subroutines. 
Even_Density_Cover samples a fixed number 



Fig.41.4a,b True (a) and annealed (b) belief for localization of the 
cash register [41.28]. The cash register model is shown as a wire 
frame. The small colored squares represent high likelihood parti¬ 
cles. Note that annealing makes the problem much more ambiguous 
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of particles from each 5-neighborhood of a volume 
of space. It can be easily implemented using rejec¬ 
tion sampling. Importance_Weights computes 
normalized importance weights using the annealed 
measurement model p(D\X) l ' z . The Prune sub¬ 
routine prunes out low probability regions. This step 
can be done either by weighted resampling or by 
thresholding on the weights. 

The algorithm returns an approximation of the be¬ 
lief represented by a weighted particle set X, where 
the weights are set according to the measurement 
model, which in the case of uniform prior is propor¬ 
tional to the belief (41.8). Extension to the case of 
nonuniform prior can be done by simply multiplying 
the resulting weights by the prior. Sampling from the 
prior at the start of scaling series may also be useful. 
See for an example, where scaling series 

is successfully applied to 6-DOF localization of a cash 
register. 


Algorithm 47.7 Scaling series algorithm for belief 
estimation 

Input: 

Vq - initial uncertainty region, 

D - data set, 

M - number of particles per <5-neighborhood, 

<5* - terminal value of S. 

1: So <— Radius(Vo) 

2: zoom <— 2 —1 / dlmX 

3: TV <r- [ \og 2 (VoIume(Ss 0 )/Volume(Ss f ))i 
4: for n = 1 to TV do 
5: S„ <— zoom ■ S n - 1 

6: (<5„/<5*) 2 

7: X n <— Even_Density_Cover (V n —i,M) 

8: f/V n <— Importance_Weights (X„, r„, T>) 

9: X n <- Prune (X n , f/V n ) 

10: V n <— Union_Delta_Neighborhoods 

(X n ,8n) 

11: end for 

12: X <— Even_Density_Cover (Vn,M) 

13: TV Importance_Weights(X, 1, D) 

Output: 

(X, TV) - a weighted particle set approximating the 
belief. 


Dynamic Case. Scaling series can be extended to 
the dynamic case using the same technique as in the 
histogram filter. During the measurement update, the 
measurement model is estimated using scaling series 
with a uniform prior. This produces a set of weighted 
particles X t . During the dynamics update, the impor¬ 
tance weights are adjusted to capture the motion model 
via the Bayesian recursion equation (41.9). To do this, 


for each particle X, in X r , the importance weight is 
multiplied by the prior bel t (X,). Like in the histogram 
filter, the prior at a point X, is computed by replacing 
the integral in (41.10) with a summation, except now 
the summation is done over all particles in X,—\. For 
other versions of dynamic scaling series, refer to [41 .28, 
Chapter 2], 

Other Advanced Inference Methods 
Other advanced methods have been used to solve 
the tactile object localization problem, including the 
annealed particle filter (APF) [41.25], the GRAB 
algorithm [41.31], and the manifold particle filter 
(MPF) [41.32], 

The APF algorithm is similar to scaling series as 
it also uses particles and iterative annealing. This al¬ 
gorithm was originally developed for articulated object 
tracking based on vision data [41.33,34], Unlike scal¬ 
ing series, APF keeps the number of particles constant 
at each iteration and the annealing schedule is derived 
from the particle set itself based on survival rate. Due 
to these properties, APF handles poorly in multimodal 
scenarios [41.35], which are prevalent in tactile object 
localization. 

The GRAB algorithm is based on grids and mea¬ 
surement model bounds. It performs iterative grid re¬ 
finements and prunes low-probability grid cells based 
on measurement model bounds. Unlike the majority of 
inference methods, GRAB is able to provide guaranteed 
results as long as the measurement model bounds are 
sound. It is well-suited for problems with many discon¬ 
tinuities, for example, whenever negative information is 
used in the measurement model. However, for smooth 
measurement models (such as the proximity model), 
GRAB has been shown to be slower than scaling se¬ 
ries [41.28]. 

Similarly to the dynamic version of scaling se¬ 
ries, the MPF algorithm samples particles from the 
measurement model and weighs them by the dynam¬ 
ics model. However, to sample from the measure¬ 
ment model, MPF draws a set of particles from the 
contact manifold, defined as the set of object states 
that contact a sensor without penetrating. Then, MPF 
weights these new particles by propagating prior par¬ 
ticles forward using the dynamics model and applying 
kernel density estimation. Since the contact manifold 
is a lower dimensional manifold than the full state 
space of object poses, the MPF requires fewer parti¬ 
cles than a traditional PF. It is not yet known if the 
improvement is sufficient to handle the full 6 DOF 
problem. However, it has been shown that MPF is 
capable of handling accurate sensors [41.32]. While 
MPF provides a method for sampling from the contact 
manifold for a single contact, it is unclear how to ex- 
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tend this approach to the case of multiple simultaneous 
contacts. 

41.2.5 Planning 

In order to localize an object, we need a strategy for 
gathering the data. Since each touch action can take 
significant time to execute, it is particularly important 
to accomplish the task with as few touches as possible. 
In this section, we will cover how to generate candi¬ 
date actions and how to select which action to execute, 
with the goal of either localizing the object or grasping 
it with a high probability of task success. 

Candidate Action Generation 
The first step in selecting actions for tactile localiza¬ 
tion is to generate a set of candidate motions. While 
the object’s pose is uncertain, we would like to avoid 
knocking over the object or damaging the robot. For this 
reason, actions are generally guarded motions, where 
the robot stops moving upon detecting contact. For 
multifingered robots, it may be advantageous to close 
the fingers upon detecting contact, in order to generate 
more than one contact point per action. 

Candidate motions can be automatically generated 
in a similar manner to randomized or heuristic grasp 
planning methods (such as [41.36] or [41.37]). This ap¬ 
proach is particularly useful if the goal is to generate 
multiple fingertip contacts, because any motion that is 
likely to grasp the object is also likely to result in infor¬ 
mative contacts. 

One option is to generate a small pool of good 
candidate motions based on the object shape, and to ex¬ 
ecute them relative to the current most-likely state of 
the object, as in [41.38]. Another option is to generate 
a larger pool of motions that are fixed with respect to the 
world, as in [41.39]. Finally, we can generate paths that 
specifically target disambiguating features. For exam¬ 
ple, Schneiter and Sheridan [41.40] show how to select 
paths that are guaranteed to disambiguate among re¬ 
maining object shape and pose hypotheses, or if none 
such are possible, how to select paths that at least guar¬ 
antee that something will be learned. 

Bayesian Formulation of Planning 
Once a pool of candidate actions has been created, the 
problem of selecting an optimal action to take next can 
be formulated as a partially observable Markov deci¬ 
sion process (POMDP). Details on POMDPs can be 
found in Chap. 14. Briefly, a POMDP consists of a set 
of states X := {X,}, a set of actions JA := {Aj}, a set of 
observations D := { I)i }. Both measurement model and 
dynamics model from Sect. 41.2.2 are now conditioned 
on the action A taken. This change is clearly visible 



Fig. 41.5 Bayesian network representation of the planning 
problem. Note how it differs from Fig. 41.1b. Measure¬ 
ments T), now depend on both the state X, and the action 
chosen A t . The state X, evolves based on both the prior 
state X,—i and the chosen action A, 

in the dynamic Bayesian network representation of the 
problem: compare Fig. 41.5 to Fig. 41.1b. Hence, the 
dynamics model is now p{X t \X t —\,A t ), which is often 
called the transition model in the context of POMDPs. It 
represents the probability distribution of state X t , given 
that we were in state X,—\ and executed action A,. Sim¬ 
ilarly, the measurement model is now p(D\X,A). For 
problems where the state X is continuous, the state 
space is typically discretized using either a grid or a set 
of particles. 

For tactile object localization, the states are the ac¬ 
tual object poses, the actions are our candidate robot 
motions (which may or may not touch the object), and 
the observations can include tactile and/or propriocep¬ 
tive information gained while executing actions. We do 
not actually know the true underlying object pose, but 
at any given time f, we can estimate the uncertain be¬ 
lief bel t about the object pose using methods described 
in Sect. 41.2.3. In the context of POMDPs, the belief is 
often referred to as the belief state. 

A POMDP policy specifies an action A, for every 
possible belief. Intuitively, there is an optimal policy 
that would get the robot to the goal state most efficiently 
for any possible starting state. This optimal policy can 
be found using value or policy iteration algorithms. 
However, for the problem of tactile localization, these 
algorithms are typically computationally prohibitive for 
any practical discretization of the state space. Hence, 
this problem is usually solved using methods that ap¬ 
proximate the optimal policy for a subset of states. 
For instance, online replanning using one-step or multi- 
step lookahead is nonoptimal but usually provides good 
results. 

For a subset of tactile localization problems, one- 
step lookahead has been shown to be nearly opti¬ 
mal [41.41], Problems in this class satisfy two as¬ 
sumptions: (1) the object is stationary and does not 
move when touched (e.g., a heavy appliance), and 
(2) the actions being considered are a fixed set of 
motions regardless of the current belief. With these as- 
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sumptions, tactile localization can be formulated as an 
adaptive submodular information-gathering problem. 
Adaptive submodularity is a property of information¬ 
gathering problems where actions have diminishing 
returns, and taking additional actions never results in 
loss of information. 

One-Step Lookahead 

One way to select the next action is to consider the 
expected effect of each action on the belief about the 
object pose. Figure 41.6 shows examples of the effect 
of different actions on different beliefs. 

Based on the possible object poses according to the 
current belief bel t , we need to consider the possible out¬ 
comes of each action A. However, the same action can 
result in different outcomes depending on the actual ob¬ 
ject pose, which is not known to us exactly. 

Simulation Experiments. Since we can not deter¬ 
mine the effect of action A exactly, we approximate it 
by performing a series of simulation experiments. Let 
X, be the current set of state hypotheses. For histogram 
filters, this can be the entire set X. For particle filters, 
it is the current set of particles. For each particular ob¬ 
ject pose X G X,, we can use a geometric simulation 
to determine where the hand will stop along a guarded 
motion A, and what observation D we are likely to see. 
Then, we can perform a Bayesian update of bel, to ob¬ 
tain the simulated belief bel AD that would result if we 
chosen action A and observed D. Let D A denote the set 
of all simulated observations we obtained for an action 
A using these experiments. 



Fig.41.6a-d Selecting an action using a KLD-based utility func¬ 
tion on examples of localizing a door handle. For each example, the 
starting belief bel t is shown on the left. The action chosen A := Obest 
and the resulting belief he/,+i after executing the action are shown 
on the right. Color shows the probability of the state, ranging from 
high (red) to low (blue), (a) Sharp prior along the z-axis. (b) Sharp 
prior diagonal z-y-axis. (c) Sharp prior along the y-axis. (d) Wide 
prior along the diagonal z-y-axis (after [41.39]) 


To completely account for all possible outcomes, 
we need to simulate all possible noisy executions of 
action A and all possible resulting noisy observations 
D. However, restricting our consideration to the set of 
noise-free simulations works well in practice. 

Each action A may also have an associated cost 
C(A,D), which represents how expensive or risky it is 
to execute A. Note, that the action cost may depend on 
the observation D. For example, if the cost is the amount 
of time it takes to execute A, then depending on where 
the contact is sensed, the same guarded motion will stop 
at a different time along the trajectory. 

Utility Function. In order to decide which action to 
take, we need a utility function U(A) that allows us to 
compare different possible actions. The utility function 
should take into account both the usefulness of the ac¬ 
tion as well as its cost. 

To localize the object, we want to reduce the un¬ 
certainty about its pose. Hence, we can measure action 
usefulness in terms of uncertainty of the resulting be¬ 
lief. Intuitively, the lower the uncertainty, the higher 
the usefulness of an action. The amount of uncertainty 
in a belief can be measured using entropy, which for 
a probability distribution q(X) is defined as 

H(q) :=-Jq(X) log q(X) dX . (41.11) 

For each simulated belief bel AD , we define its utility to 
be a linear combination of its usefulness (i. e., certainty) 
and cost 

U(M a , d ) := —H(bel A D ) — /3C(A, D) , (41.12) 

where /3 is a weight that trades off cost and useful¬ 
ness [41.39], Since we do not know what observation 
we will get after executing action A, we define its util¬ 
ity to be the expected utility of the resulting belief based 
on all possible observations 

U(A):=E D [u(tel A ' D )\. (41.13) 

Based on the simulation experiments, this expectation 
can be estimated as a weighted sum over all the experi¬ 
ments 

U(A) ^2 w D U(bel AiD ) , (41.14) 

oex>A 

where the weight Wq is the probability of the state X that 
was used to generate the observation D. More specif¬ 
ically, for a grid representation of state space, each 
D e D a was generated using some X e X,, and thus, Wd 
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is the belief estimated for the grid point X. If X, is rep¬ 
resented by a set of weighted particles, then Wd is the 
weight of particle X e X, used to generate the observa¬ 
tion D. 

With this utility function in mind, the best action to 
execute is the one with the highest utility 

A best := argmax[t/(A)j . (41.15) 

Since the utility function depends on negative entropy, 
methods that maximize this utility function are often 
called entropy minimization methods. 

Alternative Utility Measures. Other measures of use¬ 
fulness are also possible, such as reduction in average 
variance of the belief or Kullback-Leibler divergence 
(KLD) of the simulated belief from the current belief. 
However, KLD is only applicable for problems where 
the object is stationary as this metric does not support 
a changing state. An example of action selection using 
KLD metric is shown in Fig. 41.6. 

If the goal is to successfully grasp the object, not 
just to localize it, then it makes more sense to select ac¬ 
tions that maximize probability of grasp success rather 
than simply minimizing entropy. Success criteria for 
grasping can, for instance, be expressed geometrically 
as ranges in particular dimensions of object pose uncer¬ 
tainty that are required for a task to succeed. For a given 
belief, we can add up the probabilities correspond¬ 
ing to the object poses (states) for which executing 
a desired, task-directed grasp would succeed open- 
loop, and maximize that value rather than minimizing 
entropy. Action selection can still be done using one- 
step-lookahead. Using this metric instead of the one in 
(41.12) simply allows us to concentrate on reducing 
uncertainty along important dimensions, while pay¬ 
ing less attention to unimportant dimensions [41.42]. 
For example, the axial rotation of a nearly cylindrical 
object is unimportant if it only needs to be grasped 
successfully. 

Multi-step Planning in Belief Space 
While in some situations one-step-lookahead methods 
give good results, in other situations it may be ad¬ 
vantageous to look more than one step ahead. This is 
especially true if actions take a long time to execute, 
and/or if the plan can be stored and reused for multiple 
manipulation attempts. 

Multi-step lookahead can be performed by con¬ 
structing a finite-depth search tree (Fig. 41.7 for il¬ 
lustration). Starting with the current belief bel t , we 
consider possible first-step actions Aj,..., A 3 and sim¬ 
ulate observations D u ,... ,D 22 . For each simulated 


observation, we perform a belief update to obtain the 
simulated beliefs Bn,, Bn - For each of these simu¬ 
lated beliefs, we consider possible second-step actions 
(of which only A 4 ,...,Ag are shown). For each of 
the second-step actions, we again simulate observations 
and compute simulated beliefs. This process is repeated 
for third-step actions, fourth-step actions, and so on, un¬ 
til the desired number of steps is reached. The example 
in Fig. 41.7 shows the search tree for two-step looka¬ 
head. 

After the tree is constructed from the top down to 
the desired depth, scoring is performed from the bottom 
up. Leaf beliefs are evaluated using the chosen utility 
metric. In the example in Fig. 41.7, the leafs shown 
are S 41 ,..., Bfj. For each leaf, we show the entropy 
H and cost /3C. The utility of each corresponding ac¬ 
tion A 4 ,..., Ag is then computed by taking expectation 
over all observations that were simulated for that ac¬ 
tion. Note that we have to take expectations at this level 
because we have no control over which of the simu¬ 
lated observations we will actually observe. However, 
at the next level, we do have control over which action 
to execute, and we select the action with the maxi¬ 
mum utility [41.42]. We compute the utility of beliefs 
fin,.... fi 32 by subtracting their cost from the utility of 
the best action among their children. For example, 

U(B 22 ) = U(A 6 )-PC(A 2 ,D 22 ) . (41.16) 

In this manner, expectation and maximization can be 
carried out for multiple levels of the tree until the top 
is reached. Maximization at the very top level selects 
the most optimum action to execute next. Bold lines 
in Fig. 41.7 show the possible action sequences this 
algorithm would consider optimal depending on the 
possible observations made. 

When searching to a depth greater than 1, branch¬ 
ing on every possible observation is likely to be pro¬ 
hibitive. Instead, we can cluster observations based on 
how similar they are into a small number of canon¬ 
ical observations and only branch on these canonical 
observations. 

Performing a multistep search allows us to include 
any desired final grasps as actions. These actions can re¬ 
sult in early termination if the success criterion is met, 
but can also function as possible information-gathering 
actions if the criterion is not met. It also allows us to rea¬ 
son about actions that may be required to make desired 
grasps possible, such as reorienting the object to bring 
it within reach. As shown in [41.42], a two-step looka¬ 
head search is usually sufficient. Searching to a depth of 
3 generally yields little additional benefit. One full se¬ 
quence of information-gathering, grasping, and object 
reorientation using a multistep search with a depth of 2 
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Fig. 41.7 Part of a depth-2 search tree through belief space, for three actions at each level and two canonical observa¬ 
tions for each action; bel t is the current belief. ,A 6 are actions. Dn,... ,D(a are simulated observations. And 

Bn,... ,B (,2 are simulated beliefs. We can compute the entropy H and action cost C at the leafs, and thus compute 
the utility metric U for each leaf. Then, working upward from the bottom, we can compute the expected utility U of 
actions A 4 ,..., A(, by taking a weighted sum of their children. We then select the action with maximum utility {A(, in this 
example) and use its utility U(A(,) to compute the utility at node B 22 using (41.16). The same operation is repeated for 
the upper levels, and at the top level we finally select the action with the highest utility as the best action to be performed 
(A 2 in this example) (after [41.42]) 


for grasping a power drill is shown in Fig. 41.8, and in 


VIDEO 77 


Other Planning Methods 

There are many other possible methods for selecting 
actions while performing Bayesian state estimation for 
tactile object localization and grasping. As long as we 
continue to track the belief with one of the inference 
methods described above, even random actions can 
make progress toward successfully localizing the ob¬ 
ject. In this section, we will describe a few significantly 
different approaches to action selection for Bayesian 
object localization. 

The planning methods we have described so far 
used either guarded motions or move-until-contact, to 
avoid disturbing or knocking over the object. However, 
one can instead use the dynamics of object pushing 


to both localize and constrain the object’s pose in 
the process of push-grasping, as in Dogar and Srini¬ 
vasa [41.43]. 

Also, instead of using entire trajectories as ac¬ 
tions, we could consider smaller motions as actions 
out of which larger trajectories could be built. In this 
case, one- or two-step lookahead will be insufficient 
to get good results. Instead, we would need to con¬ 
struct plans consisting of many steps. In other words, 
we would need a much longer planning horizon. The 
challenge is that searching through belief space as de¬ 
scribed above is exponential in the planning horizon. In 
Platt et al. [41.44], the authors get around this problem 
by assuming the current most likely state is true and 
searching for longer horizon plans that will both reach 
the goal and also differentiate the current most likely 
hypothesis from other sampled, competing states. If we 
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Fig.41.8a-i An example sequence of grasping a powerdrill based on a depth-2 search through belief space. The right- 
hand side of each pair of images shows the grasp just executed, while the left-hand side shows the resulting belief. 
The dark blue flat rectangle and the red point-outline of the powerdrill show the most likely state, while the light blue 
rectangles show states that are 1 standard deviation away from the mean in each dimension (x,y, 6). Panels (a-c) and 
(e-g) show information-gathering grasps being used to localize the drill, panel (d) shows the robot reorienting the drill 
to bring the desired grasp within reach, and panels (h) and (i) show the robot using the desired final grasp on the drill, 
lifting, and pulling the trigger successfully (after [41.42]) 


monitor the belief during execution and replan when¬ 
ever it diverges by more than a set threshold, the goal 
will eventually be reached because each hypothesis will 
be either confirmed (and thus the goal reached) or dis¬ 
proved. 

Finally, while the above methods can be used for 
planning for what we termed global localization (i. e., 

41.3 Learning About an Object 

While the previous section focused on localization of 
known objects, in this section, we discuss methods for 
learning about a previously unknown object or environ¬ 
ment. The goal is to build a representation of the object 
that can later be useful for localization, grasp planning, 
or manipulation of the object. A number of properties 
may need to be estimated, including shape, inertia, fric¬ 
tion, contact state, and others. 

41.3.1 Shape of Rigid Objects 

In this set of problems, the goal is to construct a 2-D or 
a 3-D model of the object’s geometric shape by touch¬ 
ing the object. Due to low data acquisition rate, touch- 
based shape reconstruction faces special challenges as 
compared to methods based on dense 3-D cameras or 
scanners. These challenges dictate the choice of object 
representation and exploration strategy. 


object localization under high uncertainty), if the ob¬ 
ject needs to be localized in an even broader context, as 
when trying to locate an object somewhere in a kitchen, 
then we may have to plan to gather information with 
higher level actions (e.g., opening cabinets). Such ac¬ 
tions could be reasoned about using symbolic task 
planning, as in Kaelbling and Lozano-Perez [41.45]. 


While most of the work on shape reconstruction fo¬ 
cuses on poking motions with the end-effector, some 
methods rely on techniques such as rolling the object 
in between planar palms equipped with tactile sen¬ 
sors. In these methods, shape of the object can be 
reconstructed from the contact curves traced out by the 
object on the palms. These methods were first devel¬ 
oped for planar 2-D objects [41.46], and later extended 
to 3-D [41.47], 

Representation 

The shape reconstruction process is heavily dependent 
on the chosen representation. Since the data are sparse, 
a simpler shape representation can significantly speed 
up the shape acquisition process. 

Shape Primitives. The simplest representation is 
a primitive shape: plane, sphere, cylinder, torus, tetrahe- 
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Fig. 41.9 A combination of primitive 
shapes and polygonal mesh is used 
for modeling objects during an 
underwater mapping experiment 
(after [41.48]) 


dra, or cube. In this case, only a few parameters need to 
be estimated based on the gathered data. For example, 
Slaets et al. estimate the size of a cube by performing 
compliant motions with the cube pressed against a ta¬ 
ble [41.49]. 

Patchwork of Primitives. If one primitive does not 
describe the whole object well, a patchwork of several 
primitives may be used. Parts of the object are repre¬ 
sented with subsurfaces of different primitives, which 
are fused together [41.48] (Fig. 41.9), In 

this case, a strategy for refining the fusion boundaries 
between primitives is needed and additional data may 
be gathered for this specific purpose. 

Super-quadrics. A slightly more complex primitive 
shape is a super-quadric surface, which gives more flex¬ 
ibility than a simple primitive. Super-quadric surfaces 
are surfaces described in spherical coordinates by 

( c t cos 6 > (co 1 ) cos e2 (tu 2 )\ 
c y cos ei (ft>i) sin 62 (<w 2 ) I . (41.17) 
c z sin e ‘ (a>i) ) 

The parameters c x ,c y , and c z describe the extent of the 
super-quadric along the axes x, y, and z, respectively. 
By adjusting the exponents and e 2 , we can vary the 
resulting shape anywhere from an ellipse to a box. For 
example, setting ei,e 2 « 0 results in box shapes; 61 = 
1 , e 2 « 0 results in cylindrical shapes; ei, f 2 = 1 results 
in ellipses. Hence, by estimating c x , c y ,c z ,e 1 , and e 2 , we 
can model a rich variety of shapes [41.50]. 

Polygonal Mesh. When none of the parametric rep¬ 
resentations capture the object’s shape well enough, 
a polygonal mesh can be used. A polygonal mesh (or 
poly-mesh) can represent arbitrary shapes as it con¬ 
sists of polygons (typically, triangles) linked together 
to form a surface (Fig. 41.2). The simplest method is 
to link up collected data points to create faces (i. e., the 
polygons) of the mesh. However, this method tends to 


under-estimate the overall size of the object because 
data points at object comers are rarely gathered. This 
effect is present even for dense 3-D sensors, but espe¬ 
cially noticeable here due to sparsity of the data. A more 
accurate representation can be obtained by collecting 
several data points for each polygonal face and, then, 
intersecting the polygonal faces to obtain the corners of 
the mesh. However, this tends to be a more involved, 
manual process [41.28]. 

Point Cloud. Objects can also be represented as 
clouds of gathered data points. In this case, the accuracy 
of the representation directly depends on the density of 
the gathered data and, hence, these methods typically 
spend a lot of time collecting data [41.51, 52]. 

Splines. Two-dimensional objects can be represented 
by splines. For example. Walker and Salisbury used 
proximity sensors and a planar robot to map smooth 
shapes placed on a flat surface [41.53] (Fig. 41.10). 



Fig. 41.10 A 2-D object is explored and represented by 
splines (after [41.53]) 
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Volumetric Grid Maps. Objects can also be repre¬ 
sented by volumetric grids. For 2-D objects, this rep¬ 
resentation is called an occupancy grid map. For 3-D 
objects, it is called a voxel grid map. Each grid cell 
represents a small part of the surface (or space) and 
records a binary value (e.g., whether the cell is occupied 
or not). This value is estimated based on gathered data 
using probabilistic methods similar to the methods used 
for mobile robot mapping with a 2-D laser [41.48,54] 
(Fig. 41.11). For additional information on occupancy 
grid maps see Chap. 45. 

Data Gathering 

Different types of data can be collected during mapping. 
Tactile array images can be very informative for map¬ 
ping planar objects. Sensed end-effector contact gives 
3-D data points on the object surface. Link contacts can 
also be estimated and used for mapping even without 
sensory skin (Sect. 41.3.4 for more details). Moreover, 
the volume swept by the robot arm provides negative 
information, that is, the absence of obstacles in this 
volume. All of this information can be used both for 
mapping and for planning the next most informative 
move. 

Guided Exploration. In many applications, it is ac¬ 
ceptable to have a human operator guide the collection 
of data. This method allows for more accurate model 
building because humans can select the best points to 
sense on the object. Humans can also build variable 
accuracy models, where more data are collected near 
areas/features of interest and less data are collected 
elsewhere. 

Autonomous Exploration. When autonomous 
model-building is required, a number of exploration 
strategies exist. The simplest approach is to select 
the sensing locations randomly. However, when using 
random sampling, the density of the gathered data can 
vary. An exhaustive strategy can ensure that points 
are gathered with a specified density. A triangular 
lattice arrangement has been shown to be the most 


optimal for such a strategy; however, this method takes 
considerable time. The most optimal decision about 
the next sensing location can be made by considering 
the amount of information that can be gained. These 
techniques are similar to the techniques described in 
Sect. 41.2.5. Several primitive and advanced explo¬ 
ration strategies for tactile mapping are compared 
in [41.48]. 

41.3.2 Articulated Objects 

In all the prior sections, we were concerned with rigid 
objects, which tend to be simpler to model. The next 
step up in terms of complexity are objects comprised 
of several rigid parts, which can move with respect to 
each other. These objects are called articulated objects. 
For many objects, the motion of connected parts with 
respect to each other is restricted to only one or two 
DOFs by prismatic or revolute joints. Examples of ar¬ 
ticulated objects include simple objects such as doors, 
cabinet drawers, and scissors, as well as more complex 
objects such as robots and even humans (Fig. 41.12). 

Manipulation can play an important role in mod¬ 
eling novel articulated objects. By applying a force to 
one part of an articulated object, we can observe how 
the other parts move and thus infer relationships be¬ 
tween parts. Observation can be performed via the sense 
of touch or via some other sense, for example, vision. 
For example, if a robot arm pulls/pushes a door com¬ 
pliantly, its trajectory allows the robot to determine the 
width of the door [41.57]. Alternatively, if a robot arm 
pushes one handle of a pair of scissors, the location of 
the scissors joint can be inferred by tracking visual fea¬ 
tures with an overhead camera (Fig. 41.13). 

A kinematic structure consisting of several revolute 
and prismatic joints can be represented as a relational 
model, which describes relationships between object 
parts. In order to build a relational model of a multijoint 
object, a robot needs an efficient exploration strategy 
for gathering the data. The exploration task can be de¬ 
scribed using a Markov decision process, similar to 
the ones discussed in Sect. 41.2.5. Since the goal is 



Fig. 41.11 Reconstruction of letters from tactile images. Pixels are colored by probability of occupancy in log-odds form. This 
probability can be seen to be high on the letters themselves, to be low in the surrounding area where measurements were taken, 
and to decay to the prior probability in the surrounding un-sensed area (after [41.54]) 
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Fig. 41.12 Robot manipulating a variety of common articulated objects (from left to right): a cabinet door that opens to the right, 
a cabinet door that opens to the left, a dishwasher, a drawer, and a sliding cabinet door (after [41.55]) 



Fig.41.13a,b Robot exploring an 
articulated object (after [41.56]) 

(a) a robot arm pushes one handle of 
a pair of scissors to infer the location 
of its joint, (b) the view from the 
robot's camera with tracked visual 
features shown as green dots 


to learn relational models, these Markov decision pro¬ 
cesses (MDPs) are called relational Markov decision 
processes (RMDPs). With the aid of RMDPs, robots 
can learn efficient strategies for exploring previously 
unknown articulated objects, learning their kinematic 
structure, and later manipulating them to achieve the 
goal state [41.56]. 

Some kinematic structures do not consist entirely 
of revolute and prismatic joints. A common exam¬ 
ple is garage doors, which slide in and out along 
a curved trajectory. General kinematic models can be 
represented by Gaussian processes, which can capture 
revolute, prismatic and other types of connections be¬ 
tween parts [41.55], 

41.3.3 Deformable Objects 

Even more complex than articulated objects are de¬ 
formable objects, which can be moved to form arbi¬ 
trary shapes. The simplest of these are one-dimensional 
(1-D) deformable objects, such as ropes, strings, and 
cables. These objects are typically modeled as line 
objects with arbitrary deformation capability (except 
for stretching). A number of approaches can be found 
in [41.58]. 

A more complex type of deformable objects are pla¬ 
nar deformable objects, such as fabric or deformable 
plastic sheets. While these objects can be represented as 
networks of nodes, modeling interaction between these 
nodes can get expensive and is not always necessary. 
For example, Platt et al. developed an approach to map 
planar deformable objects by swiping them between 


the robot’s fingers [41.59] (Fig. 41.14). These maps 
can later be used for localization during subsequent 
swipes, in a manner similar to indoor robot localization 
on a map. 

The most complex deformable objects are 3-D de¬ 
formable objects. This category includes a great variety 
of objects in our everyday environments: sponges, so¬ 
fas, and bread are just a few examples. Moreover, in 
medical robotics, tissues typically have to be mod¬ 
eled as 3-D deformable objects. Since the stiffness and 
composition of these objects can vary, they need to 
be modeled as networks of nodes, where different in¬ 
teraction can take place between distinct nodes. Such 



Fig. 41.14 Robot exploring a 2-D deformable object (af¬ 
ter [41.59]) 















Active Manipulation for Perception I 41.3 Learning About an Object 1053 



Fig.41.15a-c Model of a 3-D de¬ 
formable object. The object at rest (a). 
The object under influence of an 
external force (b,c) (after [41.60] 


models typically require a lot of parameters to describe 
even reasonably small objects. Although, some effi¬ 
ciency improvements can be obtained by assuming that 
subregions of the object have similar properties. De¬ 
spite the challenges, 3-D deformable object modeling 
has many useful applications, for example, a surgery 
simulator or 3-D graphics. Burion et al. modeled 3-D 
objects as a mass-spring system (Fig. 41.15), where 
mass nodes are interconnected by springs [41.60]. By 
applying forces to the object, we can observe the de¬ 
formation and infer stiffness parameters of the springs 
(i. e., elongation, flexion, and torsion). 

41.3.4 Contact Estimation 

Up until now, we have assumed that the contact location 
during data gathering was somehow known. However, 
determining the contact point is not as straightforward 
for robots as it is for humans, because robots are not 
fully covered in sensory skin. Thus, during interaction 
between a robot and its environment, nonsensing parts 
of the robot can come in contact with the environment. 
If such contact goes undetected, the robot and/or the 
environment may be damaged. For this reason, touch 
sensing is typically performed by placing sensors at the 
end-effector and ensuring that the sensing trajectories 
never bring nonsensing parts of the robot in contact with 
the environment. 

Clearly, end-effector-only sensing is a significant 
constraint, which is difficult to satisfy in real-world 
conditions. Work on whole-body sensory skin is on¬ 
going [41.61], yet for now only small portions of 
the robot’s surface tend to be covered with sensory 
skin. 

In lieu of sensory skin, contact can be estimated 
from geometry of the robot and the environment via 
active sensing with compliant motions. Once a con¬ 
tact between the robot and the environment is detected 
(e.g., via deviation in joint torques or angles), the robot 
switches to the active sensing procedure. During this 
procedure, the robot performs compliant motions (ei¬ 
ther in hardware or software), by applying a small force 
toward the environment and moving back and forth. In¬ 
tuitively, the robot’s body carves out free space as it 
moves, and thus, via geometrical reasoning, the shape 


of the environment and contact points can be deduced. 
However, from a mathematical perspective, this can be 
done in several different ways. 

One of the earliest approaches was developed by 
Kaneko and Tanie, who formulated the self-posture 
changability (SPC) method [41.63]. In SPC, the inter¬ 
section of finger surfaces during the compliant motions 
is taken to be the point of contact. This method works 
well in areas of high curvature of the environment (e.g., 
comers), but can give noisy results in areas of shallow 
curvature. In contrast to SPC, the space sweeping algo¬ 
rithm by Jentoft and Howe marks all points on finger 
surfaces as possible contact, and then gradually mles 
out the possibilities [41.62] (Fig. 41.16). The space 
sweeping algorithm provides less noisy estimates in ar¬ 
eas of shallow curvature, but can be more susceptible 
to even the smallest disturbances of the environment’s 
surface. 

Mathematically, contact estimation can also be for¬ 
mulated as a Bayesian estimation problem. If the robot 
moves using compliant motions as described above, the 



Fig. 41.16 (a) A two-link compliant finger used for contact 
sensing. (b) The resulting model built by the space sweep¬ 


ing algorithm (after [41.62]) 
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sensor data are the joint angles of the robot. The state 
is the shape and position of the environment, repre¬ 
sented using one of the rigid object models described 
in Sect. 41.3.1. Then, each possible state can be scored 
using a probabilistic measurement model, for example, 
the proximity model described in Sect. 41.2.2, and the 
problem can be solved using Bayesian inference meth¬ 
ods [41.64]. 

41.3.5 Physical Properties 

Up until now, we have focused on figuring out the shape 
and articulation of objects. While there are some advan¬ 
tages to doing so with tactile data as opposed to visual 
data, visual data can also be used to generate shape and 
articulation models. However, there are many proper¬ 
ties of objects that are not easily detectable with vision 
alone, such as surface texture and friction, inertial prop¬ 
erties, stiffness, or thermal conductivity. 

Friction and Surface Texture 
Surface friction, texture, and roughness can be esti¬ 
mated by dragging a fingertip over the object. The 
forces required to drag a fingertip over a surface at 
different velocities can be used to estimate surface 
friction parameters [41.65-67]. Accelerometer or pres¬ 
sure data is useful for estimating surface roughness (by 
the amplitude and frequency of vibration while slid¬ 
ing) [41.67,68], or more directly identifying surface 
textures using classification techniques such as sup¬ 
port vector machines (SVMs) or ^-nearest neighbours 
(&-NNs) [41.69]. The position of the fingertip being 
dragged across the surface is also useful for estimating 
surface roughness and for mapping out small surface 
features [41.65]. Static friction is also often estimated 
as a by-product of trying to avoid object slip during 
a grasp, based on the normal force being applied at 
the moment when slip happens [41.70,71], Finally, the 
support friction distribution for an object sliding on 
a surface can be estimated by pushing the object at 
different locations and seeing how it moves [41.53, 
72], 

41.4 Recognition 

Another common goal of touch-based perception is to 
recognize objects. As with more typical, vision-based 
object recognition, the goal is to identify the object from 
a set of possible objects. Some methods assume known 
object shapes and try to figure out which object geom¬ 
etry matches the sensor data best. Other methods use 
feature-based pattern recognition to identify the object, 
often with a bag-of-features model that does not rely 


Inertia 

Atkeson et al. [41.73] estimate the mass, center of mass, 
and moments of inertia of a grasped object, using mea¬ 
surements from a wrist force-torque sensor. Objects 
that are not grasped but that are sitting on a table can 
be pushed; if the pushing forces at the fingertips are 
known, the object’s mass as well as the center of mass 
(COM) and inertial parameters in the plane can be 
estimated. Yu et al. [41.74] do so by pushing objects us¬ 
ing two fingertips equipped with force/torque sensors 
and recording the hngertip forces, velocities, and accel¬ 
erations. Tanaka and Kushihama [41.75] estimate the 
object’s mass by pushing the object with known contact 
force and watching the resulting motion with a camera. 

Stiffness 

Identifying the stiffness (or the inverse, compliance) 
of an object is useful in terms of modeling the ma¬ 
terial properties of an object, but more directly, it is 
also useful for preventing the robot from crushing or 
damaging delicate objects. Gently squeezing an ob¬ 
ject and measuring the deflection seen for a given 
contact normal force is one method often used to es¬ 
timate stiffness [41.68,76,77]. Omatae t al. [41.78] use 
a piezoelectric transducer and a pressure sensor element 
in a feedback circuit that changes its resonant frequency 
upon touching an object, to estimate the object’s acous¬ 
tic impedance (which varies with both object stiffness 
and mass). Burion et al. [41.60] use particle filters to es¬ 
timate the stiffness parameters of a mass-spring model 
for deformable objects, based on the displacements seen 
when applying forces to different locations on the ob¬ 
ject. 

Thermal Conductivity 

Thermal sensors are useful for identifying the ma¬ 
terial composition of objects based on their thermal 
conductivity. Fingertips with internal temperature sen¬ 
sors [41.68,79] can estimate the thermal conductivity 
of the object by heating a robot hngertip to above room 
temperature, touching the hngertip to the object, and 
measuring the rate of heat loss. 


on geometric comparisons to overall object shape. Fi¬ 
nally, there are methods that use object properties such 
as elasticity or texture to identify objects. 

41.4.1 Object Shape Matching 

Any of the methods for shape-based localization can 
also be used for recognition, by performing localiza- 
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tion on all potential objects. In terms of the equations 
used in Sect. 41.2, if we are considering a limited num¬ 
ber of potential objects with known mesh models, we 
can simply expand our state space to include hypothe¬ 
ses over object identity and pose, instead of just pose. If 
we add an object identifier (ID) to each state in the state 
vector X, and use the appropriate mesh model for that 
object ID for the measurement model for each state, all 
of the equations used in the previous sections still hold. 
The object ID for the combined object shape and pose 
that best matches the collected sensor data is then our 
top object hypothesis. 

Nonprobabilistic methods for localization and ob¬ 
ject recognition often involve geometrically pruning 
objects from a set of possible object shapes/poses 
that do not match sensed data. For instance, Gaston 
and Lozano-Perez [41.1] and Crimson and Lozano- 
Perez [41.2] recognize and localize polyhedral objects 
by using the positions and normals of sensed contact 
points in a generate-and-test framework: generate fea¬ 
sible hypotheses about pairings between sensed contact 
points and object surfaces, and test them for consis¬ 
tency based on constraints over pairs of sensed points 
and their hypothesized face locations and normals. Rus¬ 
sell [41.80] classify tactile array images as point, line, 
or area contacts, and use the time-sequence of contacts 
seen when rolling the object over the array to prune in¬ 
consistent object shapes. 

Other methods recognize objects by forming 
a model of the object, then matching the new 
model with database objects. For instance, Allen and 
Roberts [41.81] build superquadric models of the ob¬ 
ject from contact data, then match the superquadric 
parameters against a database of object models. Caselli 
et al. [41.82] build polyhedral representations of objects 
(one based on contact points and normals that repre¬ 
sents the space that the true object must fit inside, and 
one based on just contact points that the true object 


must envelop), and use the resulting representations to 
prune incompatible objects in a database of polyhedral 
objects. 

41.4.2 Statistical Pattern Recognition 

Many methods for tactile object recognition use tech¬ 
niques from statistical pattern recognition, summariz¬ 
ing sensed data with features and classifying the re¬ 
sulting feature vectors to determine the identity of the 
object. For instance, Schopfer et al. [41.84] roll a tactile 
array over objects along a straight line, compute fea¬ 
tures such as image blob centroids and second or third 
moments over the resulting time series of tactile im¬ 
ages, and use a decision tree to classify objects based 
on the resulting features. Bhattacharjee et al. [41.85] 
use fc-nearest-neighbors on a set of features based on 
time series images from a forearm tactile array to iden¬ 
tify whether an object is fixed or movable, and rigid or 
soft, in addition to recognizing the specific object. 

Tactile array data is very similar to small image 
patches in visual data, and so techniques for object 
recognition using tactile arrays can look much like ob¬ 
ject recognition techniques that use visual data. For 
instance, Pezzementi et al. [41.86] and Schneider and 
Sturm [41.83] both use a Bag-of-Features model on 
fingertip tactile array data to recognize objects. The 
process for tactile data is almost the same as for vi¬ 
sual images: for each training object (image), a set of 
tactile array images (2-D image patches) is obtained 
(Fig. 41.17), features (which are vectors of descriptive 
statistics) based on each tactile image are computed, 
and the features for all objects in the training set are 
clustered to form a vocabulary of canonical features 
(Fig. 41.18). Each object is represented as a histogram 
of how often each feature occurs. When looking at 
a new unknown object, the features computed from the 
tactile array images are similarly used to compute a his- 
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Fig. 41.17 Some objects and their associated tactile images for both left and right fingertips (after [41.83]) 
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Fig. 41.18 Vocabulary of tactile images created using unsupervised clustering. Only the left finger’s image is shown for 
each pair (after [41.83]) 


Part D | 41.4 



Part D | 41.4 


1056 Part D 


Manipulation and Interfaces 


togram, and then the new histogram is compared against 
those of the objects in the database to find the best 
match. The one notable difference in the two domains 
is that tactile data is much more time-consuming and 
difficult to obtain than appropriate image patches: in 
2-D visual object recognition, an interest point detec¬ 
tion algorithm can quickly produce hundreds of salient 
patches from a given 2-D image, whereas for tactile ob¬ 
ject recognition, each tactile array requires touching the 
object, which in turn requires a planner (such as the 
ones described in Sect .41.2.5) to decide how to gather 
appropriate tactile data. 

Tactile object exploration also allows one to gather 
and use information besides just tactile image patches: 
for instance. Gorges and Navarro [41.87] envelop ob¬ 
jects in a multifingered hand, and use the resulting 
joint angles as features that directly relate to object 
dimensions, alongside additional features computed 
from tactile image patterns. In [41.88], Pezzementi and 
Hager combine techniques from vision-based object 
recognition with the type of Bayesian recognition-by- 
localization described in Sect. 41.4.1, using pairs of 
fingertip tactile image features along with their posi¬ 
tions and surface normals to update a belief state over 
object poses and identities. 

41.4.3 Recognition by Material Properties 

Tactile object recognition sometimes has significant 
advantages over purely visual object recognition, par¬ 
ticularly in cases where objects are visually similar 
but have different material properties detectable by 
tactile sensors. A number of recent methods classify ob¬ 
jects using properties such as elasticity, surface texture, 
and thermal conductivity. Any of the methods listed 
in Sect. 41.3.5 can be used to identify objects, using 
many of the techniques described in Sect. 41.4.2. For 
instance, surface texture and roughness can be used to 
distinguish objects, as in [41.69] or [41.67], as can ther¬ 
mal conductivity, as in [41.79], 

More specific properties can also be used to clas¬ 
sify objects of particular types: for instance, Chitta 
et al. [41.89] identify bottles and also classify them as 
open or closed, full or empty by squeezing and rolling 
them from side to side and using decision tree clas¬ 
sifiers on the resulting tactile measurements. Drimus 
et al. [41.90] also identify objects by squeezing them, 
but using k-nearest neighbor on the time series data 
of tactile measurements after performing dynamic time 
warping to align them. 


41.4.4 Combining Visual and Tactile Sensing 

Of course, visual object recognition also has advan¬ 
tages over purely tactile methods, and so combining 
data from both modalities can lead to even better object 
recognition and localization. For instance, Allen [41.91] 
uses sparse stereo data to provide regions of interest 
to explore with tactile sensors, then fuses the stereo 
data with the resulting tactile points to create sur¬ 
face and feature descriptions, which are then matched 
against objects in the database; object hypotheses are 
further verified with tactile exploration. Boshra and 
Zhang [41.92] recognize polyhedral objects by using 
tactile features and visual vertices and edges to for¬ 
mulate constraints in a constraint-satisfaction problem. 
Nakamura [41.93] grasps objects and learns object cat¬ 
egories based on visual data (in the form of SIFT 
descriptors), audio information obtained while shaking 
(using a codebook of audio features based on mel- 
frequency cepstral coefficients), and haptic information 
obtained while squeezing (using hardness estimates 
based on finger pressure sensor readings); the learning 
and inference are done using probabilistic latent seman¬ 
tic analysis (PLSA). 

Segmenting an object from a visual scene (which 
makes further efforts at recognition much easier) is 
also made easier by combining visual and tactile sens¬ 
ing; by pushing on an object and seeing which parts 
of the visual scene move together, objects can be seg¬ 
mented both from the background and from each other. 
For instance, Gupta and Sukhatme [41.94] use RG- 
B-D data along with manipulation primitives such as 
spreading motions to segment and sort Duplo bricks, 
and Hausman et al. [41.95] segment textureless ob¬ 
jects in RGB-D data by performing visual segmen¬ 
tation and then resolving ambiguities using pushing 
motions. 

Work on categorizing objects by their affordances 
or functions often also uses both visual and tactile data. 
For instance, Sutton et al. [41 .96] uses both shape-based 
reasoning (using shape derived from range images) as 
well as interaction-based reasoning (using data from 
the robot physically interacting with the object) to 
predict and confirm object affordances such as grasp¬ 
ing, containing, and sitting (being suitable for humans 
to sit upon). Bekiroglu [41.97] performs inference on 
a Bayesian network using visual, proprioceptive, and 
tactile data to assess whether an object/grasp combina¬ 
tion will be both stable and also good for a task such as 
pouring, hand-over, or dishwashing. 
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41.5 Conclusions 

This chapter described perceptual methods that rely on 
manipulation. Although these methods are generally 
more complex than vision-based perception, they are 
useful in low-visibility conditions, in determining prop¬ 
erties that require manipulation, and in situations where 
perception is needed for subsequent manipulation tasks. 
Perception-via-manipulation faces special challenges 
due to the low data acquisition rate and the fact that 
each sensing action disturbs the scene. To address these 
challenges, special inference methods have been devel¬ 
oped to cope with situations where data are insufficient 
to fully constrain the problem (Sect. 41.2.3). Special 
planning methods had to be applied to minimize the 
number of sensing actions and reduce the possibility 
of significant scene disturbance or non-sensed contacts 
(Sect. 41.2.5). These challenges also dictate the choice 
of object models (Sect. 41.3), and play a role in object 
recognition methods (Sect. 41.4). 

41.5.1 Real-Life Applications 

Perception-via-manipulation is already used in a num¬ 
ber of real-world applications and an even greater 
number of applications will likely benefit from it in the 
near future. In manufacturing, touch-based localization 
of parts for machining or assembly is often preferred 
to optical methods because it is not affected by optical 
properties of objects (e.g., transparency or reflectivity). 
At present, low uncertainty in part positioning is re¬ 
quired prior to touch-based localization, which leads to 
high instrumentation and reconfiguration costs. How¬ 
ever, the new localization methods presented in this 
chapter can cope with high uncertainty and thus en¬ 
able flexible manufacturing lines capable of producing 
a variety of parts without reconfiguration. In the future, 
fully automated in-home manufacturing may even be¬ 
come possible. 

Medical robotics is another mature real-world appli¬ 
cation, in which robotic contact with the environment 
(i. e., the patient) is not only unavoidable, it is in fact 
required for many of the procedures. Visibility is often 
poor, and sensing with devices that emit sound, light, or 
other waves can be harmful to the patient. Under these 
conditions, sensing-via-contact can sometimes be the 
most viable option. The environment is inherently de¬ 
formable, and thus, medical robotics has been a strong 
driver for research on manipulation and perception of 
deformable objects. 

Another area in which robots are indispensable is 
underwater robotics. It is not always safe or convenient 
for humans to be performing tasks deep underwater, 


and hence, remotely operated robots (ROVs) are a nat¬ 
ural choice. ROVs even made international news during 
recovery operations from the BP oil spill disaster in 
the spring of 2010. Since visibility underwater is often 
poor, perception-via-contact plays an even more impor¬ 
tant role. 

Beyond the more established robotic applications, 
as the fields of service robotics and disaster recovery 
develop, the necessity of sensing-via-contact becomes 
more and more obvious. In these applications, robots 
often have to work in clutter and in close proximity to 
humans. Under these conditions, visibility can be poor, 
yet at the same time, unsensed contact can be disas¬ 
trous. 

41.5.2 Future Directions of Research 

Inspired by the many existing and potential applica¬ 
tions, the field of perception-by-manipulation is evolv¬ 
ing and receiving more attention. Despite the chal¬ 
lenges, there are bountiful opportunities for further 
fruitful research: 

• The vast majority of touch-based perception work 
deals with rigid objects, whereas many objects in 
our environments are articulated and/or deformable. 
These types of objects need to be studied in more 
detail. 

• For rigid objects, touch-based perception of moving 
objects, object piles, and objects in clutter has been 
under-explored. 

• Whole-body contact estimation needs further atten¬ 
tion, with or without sensory skin. The challenges 
here are significant, but good solutions would in¬ 
crease the safety of robots in service applications 
and other unstructured environments. 

• Last but not least, while Bayesian methods have 
found wide acceptance in other areas of robotics, 
touch-based perception has yet to fully benefit from 
these methods. Thus, application of Bayesian meth¬ 
ods to any of the above listed areas is particularly 
promising. 

41.5.3 Further Reading 

For a broader view, a lot of related material can be 
found in other chapters throughout this handbook. Most 
sensing-via-manipulation is done using contact or near¬ 
contact (i. e., close-range) sensors. These sensors are 
described in Chap. 28. A few approaches use classical 
vision sensors in conjunction with manipulation (as in 
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Sect. 41.3.2, for example). Vision sensors and methods 
are covered in Chaps. 32 and 34. Manipulator design 
and control are covered in Chaps. 4, 8, and 9. Other 


relevant chapters include 7, 14 and 47 for planning 
methods, 37, 45, and 46 for modeling, 33 for recogni¬ 
tion, 5 and 35 for inference. 
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42. Haptics 


Blake Hannaford, Allison M. Okamura 


The word haptics, believed to be derived from 
the Greek word haptesthai, means related to the 
sense of touch. In the psychology and neuroscience 
literature, haptics is the study of human touch 
sensing, specifically via kinesthetic (force/position) 
and cutaneous (tactile) receptors, associated with 
perception and manipulation. In the robotics 
and virtual reality literature, haptics is broadly 
defined as real and simulated touch interac¬ 
tions between robots, humans, and real, remote, 
or simulated environments, in various com¬ 
binations. This chapter focuses on the use of 
specialized robotic devices and their correspond¬ 
ing control, known as haptic interfaces, that allow 
human operators to experience the sense of touch 
in remote (teleoperated) or simulated (virtual) 
environments. 
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Haptic technology is intimately connected with robotics 
through its reliance on dexterous mechatronic devices 
and draws heavily on the theoretical foundations of 
manipulator design, actuation, sensing, and control. In 
this chapter, we begin with motivation for haptic in¬ 
terface design and use, including the basic design of 
a haptic interface, information about human haptics, 
and examples of haptic interface applications. Next, we 
review concepts in the mechatronic design of kines¬ 


thetic haptic interfaces, including sensors, actuators, 
and mechanisms. We then examine the control aspect of 
kinesthetic haptic interfaces, particularly the rendering 
of virtual environments and stable and accurate display 
of forces. We next review tactile displays, which vary 
widely in their design due to the many types of tactile 
information that can be presented to a human opera¬ 
tor. Finally, we provide resources for further study of 
haptics. 
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42.1 Overview 

Haptics is the science and technology of experienc¬ 
ing and creating touch sensations in human operators. 
Imagine trying to button a coat, shake someone’s hand, 
or write a note without the sense of touch. These sim¬ 
ple tasks become extremely difficult to perform without 
adequate haptic feedback. To improve human operator 
performance in simulated and teleoperated environ¬ 
ments, haptic interfaces seek to generate a compelling 
sensation that the operator is directly touching a real 
environment. 

Haptic interfaces attempt to replicate or enhance the 
touch experience of manipulating or perceiving a real 
environment through mechatronic devices and com¬ 
puter control. They consist of a haptic device (HD, 
a manipulandum with sensors and actuators) and a con¬ 
trol computer with software relating human operator in¬ 
puts to haptic information display. While the low-level 
design of haptic interfaces varies widely depending on 
the application, their operation generally follows the 
haptic loop shown in Fig. 42.1. First, the haptic device 
senses an operator input, which may be position (and 
its derivatives), force, muscle activity, etc. Second, the 
sensed input is applied to a virtual or teleoperated en¬ 
vironment. For a virtual environment, the effect of the 
operator’s input on virtual objects and the subsequent 
response to be displayed to the operator are computed 
based on models and a haptic rendering algorithm. In 
teleoperation, a manipulator that is remote in space, 
scale, or power attempts to track the operator’s input. 
When the manipulator interacts with its real environ¬ 
ment, haptic information to be relayed to the operators 
is recorded or estimated. Finally, actuators on the haptic 
device are used to physically convey touch sensations 
to the human operator. Based on the haptic feedback, 
whether through unconscious or conscious human con¬ 
trol, or simply system dynamics, the operator input is 
modified. This begins another cycle of the haptic loop. 

Despite the simplicity of the concept of haptic 
display, there exist many challenges to developing com¬ 
pelling haptic interfaces. Many of these are addressed 
through fundamental robotics theory and an under¬ 
standing of human haptic capabilities. In general, haptic 
interface performance specifications are based on hu¬ 
man sensing and motor control characteristics. One ma¬ 
jor challenge in artificially generating haptic sensations 
is that the human operator’s motion should be unre¬ 
stricted when there is no contact with a virtual or remote 
object. Haptic devices must allow the human operator to 
make desired motions, thus requiring back-drivability 
and sufficient degrees of freedom of motion. A variety 
of robotic designs are used in haptic devices, includ¬ 
ing exoskeletons, actuated grippers, parallel and serial 


manipulators, small-workspace mouse-\ike devices, and 
large-workspace devices that capture whole arm, and 
even whole body, movement. Another challenge is that 
humans integrate kinesthetic (force/position) and cu¬ 
taneous (tactile) information with motion and control 
cues to form haptic perceptions. Haptic devices would 
ideally include both force and tactile displays, although 
this has been rarely done due to size and weight limi¬ 
tations of actuators. Because of the human’s sensitivity 
to high-frequency information, for many haptic inter¬ 
faces and applications, this loop must repeat at a high 


a) 


Human operator and haptic device 



Desired haptic 
display output 



f 


Measured 
operator input 




Fig. 42.1 (a) The haptic loop of a generic haptic inter¬ 
face. A haptic device senses human operator input, such 
as position or force, and the system applies this input to 
a virtual or teleoperated environment. The response of the 
environment to be relayed to the human operator is com¬ 
puted through models, haptic rendering, sensing, and/or 
estimation. Finally, actuators on the haptic device display 
corresponding touch sensations to the human operator, 
(b) The ideal result is that the human operator feels that 
he or she is interacting directly with a real environment 
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frequency, typically 1 kHz. Not only does a high update 
rate provide realistic (nondiscretized) touch sensations 
to the human operator, it also typically helps to maintain 
system stability. Controls analysis for haptic devices 
must consider both the continuous nature of the phys¬ 
ical dynamics and the discrete nature of the computer 
control. 

Before we examine the various components of hap¬ 
tic interfaces in detail, it is useful to motivate their 
design through a review of human haptics and applica¬ 
tions of haptics. The remainder of this section is devoted 
to those topics. 

42.1.1 Human Haptics 

Two functions of the human nervous system play a pri¬ 
mary role in haptics: kinesthesia (the internal sensing of 
forces and displacements inside muscles, tendons, and 
joints), and tactile sensing (the sensation of deforma¬ 
tions of the skin). 

Anatomy and Physiology 

Haptics incorporates both, and is associated with an 
activity such as manipulation or exploration. Most 
of this chapter will address systems meant to inter¬ 
act primarily with the kinesthetic modality. Devices 
specifically aimed at tactile perception are described 
in Sect. 42.5. Even if tactile stimuli are not explicitly 
generated by a haptic device, tactile receptors are still 
stimulated, and are known to respond to frequencies as 
high as 10 000 Hz [42.1] and displacements as small as 
2—4 |xm [42.2—4]. 

Kinesthesia is mediated by muscle spindles, which 
transduce stretch of muscles, and Golgi tendon or¬ 
gans, which transduce joint rotation, especially at the 
extremes of motion. In principle, these and similar re¬ 
ceptors could be stimulated directly to produce haptic 
sensations. For example, a vibration applied to a muscle 
tendon creates a strong sensation of muscle lengthen¬ 
ing and corresponding joint motion in humans [42.5, 
6]. Research in peripheral nerve stimulation for prosthe¬ 
sis control has demonstrated that electrodes implanted 
within individual fascicles of peripheral nerve stumps 
in amputees can be stimulated to produce sensations of 
touch or movement referred to the amputee’s phantom 
hand [42.7], 

Psychophysics 

At the next level up from physiology and anatomy, 
psychophysics [42.8], the science of the physical capa¬ 
bilities of the senses, has been a rich source of design 
data for haptic device development. Its chief contribu¬ 
tion has been methodologies that haptics researchers 
have applied to answer questions about what capa¬ 


bilities are needed in haptic devices. These sensory 
capabilities could then be translated into design require¬ 
ments. Some of the chief psychophysical methods that 
have been fruitfully applied to haptics include threshold 
measurement by the method of limits and adaptive up- 
down methods. However, perception at threshold is not 
100% reliable. Perception accuracy tends to depend on 
the strength of the stimulus, and the tradeoff between 
hit rate and false alarms depends strongly on the prob¬ 
ability that a stimulus will be present in a given time 
interval (P stim ). 

A more general notion is the receiver operating 
curve (Fig. 42.2, borrowed by psychophysics from 
radar theory), which plots the probability of a subject 
response given the existence of a stimulus, versus the 
probability of response given no stimulus. The curve 
is generated by measuring both probabilities at several 
different values of P, tim . The ideal response is the point 
(0,1): 100% response for stimuli and 0% response for 
nonstimuli. Human response is near this point for stim¬ 
uli much above threshold, but declines to a rounded 
curve and eventually the 45° line as response below 
threshold becomes equal to chance. 

Another relevant concept from psychophysics is the 
just noticeable difference (JND), commonly expressed 
as a percentage. This is the magnitude of a relative 
change in a stimulus, such as a force or displacement 
applied to the finger that is just perceivable by subjects. 


Hit rate (p (yes|stimulus)) 



Fig. 42.2 Receiver operating curve (ROC). The ROC en¬ 
codes the tradeoffs made by a subject between risk of false 
alarm and risk of missing a valid stimulus. Each point en¬ 
codes a specific tradeoff between these two risks observed 
when stimuli are presented with a specified probability. 
ROCs for stronger signals tend towards the upper left hand 
comer (after [42.8] with permission from Lawrence Erl- 
baum and Associates, Mahwah) 
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For example, Jones [42.9] has measured JNDs of 6% 
for force applied to the human finger over a range of 
0.5-200 N. 

Psychology: Exploratory Procedures 
In influential research starting in the 1980s, Leder- 
man and Klatzky defined stereotyped hand motions 
called exploratory procedures (EPs), which are charac¬ 
teristic of human haptic exploration [42.10-12]. They 
placed objects into the hands of blindfolded subjects, 
and videotaped their hand motions. Their initial exper¬ 
iments [42.11] showed that the EPs used by subjects 
could be predicted based on the object property (tex¬ 
ture, mass, temperature, etc.) that the subjects needed 
to discriminate. They also showed that the EPs cho¬ 
sen by subjects were the ones best able to discriminate 
that property. Furthermore, when asked to answer spe¬ 
cific questions about objects (Is this an eating utensil, 
further a fork?), subjects used a two-stage sequence in 
which a more general lifting EP preceded more specific 
EPs [42.12], 

Lederman and Klatzky’s eight EPs (Fig. 42.3) and 
the property for which they are optimal are: 

1. Lateral motion (texture) 

2. Pressure (hardness) 

3. Static contact (temperature) 

4. Unsupported holding (weight) 

5. Enclosure (global shape, volume) 

6. Contour following (exact shape, volume) 

7. Part motion test (part motion) 

8. Function testing (specific function). 

Each of these EPs is a bimanual task involving con¬ 
tact with all interior surfaces of the hand, motion of the 
wrist and various degrees of freedom of the hand, and 
tactile and temperature sensors in the skin (e.g., EPs 1 
and 3), and kinesthetic sensors in the arm (EP 4). A hap¬ 
tic device capable of supporting all of these EPs is far 
beyond today’s state of the art. However, the signifi¬ 
cance of these results for the design of haptic interface 
is great, since they allow us to derive device require¬ 
ments from EPs. 

42.1.2 Application Examples 

The most common haptic device encountered by the 
general population is a vibration display device that 
provides haptic feedback while an operator plays 
a video game. For example, when the operator drives 
off the virtual road or bumps into a virtual wall, the 
hand controller shakes to imply driving over a rough 
surface or displays an impulse to represent the shock of 
hitting a hard surface. We examine two more pragmatic 


examples, medical simulators and computer-aided de¬ 
sign (CAD) systems, in detail below. In addition, we 
review several commercially available haptic devices. 
Although haptic interfaces are not yet in widespread 
commercial use outside of entertainment, they are being 
integrated into numerous applications where the poten¬ 
tial benefits are clear enough to justify the adoption of 
new technologies. A variety of novel and creative ap¬ 
plications are being developed regularly in numerous 
fields, including: 

• Assistive technology 

• Automotive 

• Design 

• Education 

• Entertainment 

• Human-computer interaction 

• Manufacturing/assembly 

• Medical simulation 

• Micro/nanotechnology 

• Molecular biology 

• Prosthetics 

• Rehabilitation 

• Scientific visualization 

• Space 

• Surgical robotics. 

Medical Simulations 

A major example driving much of today’s haptic vir¬ 
tual environment research is simulation for training of 
hands-on medical procedures. Medical invasive thera- 


a) b) 



Fig.42.3a-d Four of the eight human exploratory 
procedures (EPs). (a) Lateral motion (texture); (b) 
pressure (hardness); (c) static contact (temperature); 
(d) unsupported holding (weight) (after Lederman and 
Klatzky [42.11]) 
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peutic and diagnostic procedures, ranging from drawing 
blood samples to surgery, are potentially dangerous 
and painful for the patient and require the student to 
learn hands-on skills mediated by haptic information 
pathways [42.13]. Simulators both with and without 
haptic feedback aim to replace supervised learning di¬ 
rectly on human patients or on animals. Simulators 
have proven highly effective in developing minimally 
invasive surgery skills [42.14], especially when haptic 
feedback is provided in early training [42. 15]. Expected 
benefits of training with haptic simulators include: 

• Reduced risk to patients both during and immedi¬ 
ately after training 

• Increased ability to simulate unusual conditions or 
medical emergencies. 

• Ability to collect physical data during the training 
process and provide specific and directed feedback 
to the student. 

• Increased training time per unit of instructor effort. 

Approaches for simulator designs, specific medical 
applications, and training evaluation methods have also 
been widely studied in the last two decades, e.g., [42.16, 
17]. However, the costs of this technology are still 
high. In addition, it is not always clear which improve¬ 
ments in simulator technology, such as haptic device 
performance or accuracy of soft-tissue modeling, lead 
to improved clinical performance and, ultimately, pa¬ 
tient outcomes. 

Computer-Aided Design 

The Boeing Company [42.18] has studied the use of 
haptic interfaces for solving advanced problems in 
CAD. One such problem is verification of the ability 
to efficiently maintain a complex system such as an 
aircraft. In the past, mechanics could verify the pro¬ 
cedures (such as change-out of parts) on a physical 
prototype. However, this analysis is difficult or impos¬ 
sible to perform visually on an advanced CAD system. 
The VoxMap Pointshell system (Fig. 42.4) was devel¬ 
oped to allow test extraction of parts with a haptic 
interface. Force sensations from the haptic interface re¬ 
produce for the operator the physical constraints of the 
part bumping into elements of the complex workplace. 
If the operator can remove the part in the haptic in¬ 
terface, it is verified that this part can be maintained 
without undue disassembly of the aircraft. This capa¬ 
bility has been proved useful in actual design activities. 

Commercially Available Haptic Devices 
and Systems 

There are a wide variety of haptic devices available 
from companies, although many researchers build their 


own haptic devices for special purposes. At the time 
of this writing, one of the most popular commercially 
available haptic devices is the Geomagic Touch [42.19], 
formerly known as the Phantom Omni from SensAble 
Technologies (Fig. 42.5). In the 1990s and 2000s, Sens- 
Able developed the Phantom line of stylus-type haptic 
devices. The Phantom Premium [42.20], a higher fi¬ 
delity, larger workspace device, has also been used 
widely in haptics research. The high price of hap¬ 
tic devices (compared to visual displays) restricts the 
development of some commercial applications. The Ge¬ 
omagic Touch (Phantom Omni), which is an order of 
magnitude less expensive than the Phantom Premium, 
has gained popularity among haptics and robotics re- 



Fig. 42.4 Boeing computer-aided design (CAD) applica¬ 
tion for assembly and maintenance verification of com¬ 
plex aircraft systems. Boeing researchers developed the 
Voxmap/Pointshell software for haptically rendering very 
complex models in six degrees of freedom at high rates 
(courtesy of Bill McNeely, Boeing Phantom Works) 



Fig. 42.5 The Phantom Omni device from SensAble Tech¬ 
nologies, now marketed as the Geomagic Touch. This 
relatively low-cost device senses motion in six degrees of 
freedom from the stylus and can apply forces in the x, y, 
and 7 directions to the stylus tip (courtesy SensAble Tech¬ 
nologies, Inc., Woburn) 
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searchers. In 2007, Novint Technologies [42.21] re¬ 
leased the Novint Falcon, an inexpensive 3-DOF (three- 
degree-of-freedom) haptic device that is in turn an 
order of magnitude less expensive than the Phantom 
Omni. 

Immersion has aimed at the mass market and con¬ 
sumer segments with a wide variety of haptics-based 
products, many of them involving a single degree of 
freedom. For example, they have licensed technology 
to makers of various video games, as well as mobile 
phone manufacturers, in the form of vibratory feedback 
in handheld devices and haptic-enabled steering wheels 

42.2 Haptic Device Design 

There are two broad classes of haptic devices: ad¬ 
mittance and impedance devices. Admittance devices 
sense the force applied by the operator and constrain 
the operator’s position to match the appropriate deflec¬ 
tion of a simulated object or surface in a virtual world. 
In contrast, an impedance haptic device senses the po¬ 
sition of the operator, and then applies a force vector 
to the operator according to computed behavior of the 
simulated object or surface. 

Robots of the impedance type are back-drivable, 
have low friction and inertia, and have force-source 
actuators. A commonly used impedance haptic de¬ 
vice in robotics-related research is the Phantom Pre¬ 
mium [42.20,23]. Robots of the admittance type, such 
as typical industrial robots, are non-back-drivable and 
have velocity-source actuators. The velocity is con¬ 
trolled with a high-bandwidth low-level controller, and 
is assumed to be independent of applied external forces. 
Some commercially available haptic devices, such as 
the HapticMaster [42.24], do operate under admittance 
control. While such closed-loop force control has been 
used for haptic display, more commonly designers have 
opted for mechanisms specially designed for open-loop 
force control to achieve simultaneously low cost and 
high bandwidth. 

The choice of admittance or impedance architecture 
has many profound implications in the design of the 
software and hardware system. For a variety of reasons, 
including cost, the majority of haptic devices imple¬ 
mented today are of the impedance type. Because the 
preponderance of systems today are impedance devices 
and limitations on space, we limit our subsequent dis¬ 
cussion to that class. 

42.2.1 Mechanisms 

Creating high-fidelity haptic sensations in the operator 
requires attention to mechanism design (Chap. 5). The 


for driving games. Immersion also has a medical divi¬ 
sion selling medical simulators with haptic feedback. 

Software for haptic rendering has also become 
widely available, through both commercial sources and 
research groups. Most companies that sell haptic de¬ 
vices also provide a standard development kit (SDK) 
with haptic rendering capability. In addition, not-for- 
profit open-source projects such as Chai3D [42.22] aim 
to make rendering algorithms from different groups 
publicly available, shortening application development 
time and allowing direct comparison of algorithms for 
benchmarking purposes. 


requirements for impedance haptic devices are similar 
to those for designing manipulators suitable for force 
control. Desirable mechanism attributes for open-loop 
force control include low inertia, high stiffness, and 
good kinematic conditioning throughout a workspace 
designed to effectively match the appropriate human 
limb, primarily the finger or arm. The weight of the 
mechanism should be minimized, as it is perceived 
by the operator as weight and inertia of the virtual 
or teleoperated environment. Kinematic singularities 
(Chaps. 2, 5, and 18) are detrimental to haptic inter¬ 
faces because they create directions in space in which 
the end-point cannot be moved by the human operator 
and thus impose disturbances on the illusion of haptic 
contact with virtual objects. High transmission ratios 
must be avoided as they introduce significant amounts 
of friction. This constraint requires haptic interfaces to 
make high demands on actuator performance. 

Measures of Mechanism Performance 
The ideal haptic device can move freely in any 
direction and is free of singular configurations as 
well as the bad effects of operating in their neigh¬ 
borhood. Traditionally, kinematic performance has 
been derived from the mechanism’s Jacobian ma¬ 
trix, J(p,q), using some of the following well-known 
measures: 

• Manipulability [42.25]: the product of the singular 

values of Up, q) 

• Mechanism isotropy [42.26]: the ratio of smallest to 

the largest singular value of J (p,q) 

• Minimum force output [42.25,27,28]: maximizing 

the force output in the worst direction. 

Dynamics can also be introduced into the cost 
function using measures such as dynamic manipulabil¬ 
ity [42.29]. This is still an active area of research and 
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there is no consensus yet on which dexterity measure is 
most appropriate for haptic devices. 

Kinematic and Dynamic Optimization 
This aspect of design requires synthesis of mechanisms 
that match the workspace of the (most often) human 
finger or arm while simultaneously avoiding kinematic 
singularities. 

A haptic device workspace is defined to match that 
of the targeted human limb. This can be assisted by the 
use of anthropometric data [42.30]. The performance 
goals, such as low inertia and avoidance of kinematic 
singularities, must be formalized into a quantitative 
performance measure which can be computed for any 
candidate design. Such a measure must account for: 

• Uniformity of kinematic conditioning throughout 

the target workspace 

• Favoring designs with lower inertia 

• Guaranteeing that the target workspace is reachable. 

The measures of mechanism performance defined 
above operate at a single point in space and thus must 
be integrated over the entire workspace to derive a fig¬ 
ure of merit for a proposed haptic device design. For 
example, if S is the set of all joint angles 0, such that 
the end effector is inside the target workspace, one such 
measure is 

M=minW(0), (42.1) 

where W(0) is a measure of design performance. 
The performance measure should include a link length 
penalty such as 

. W{0) , , 

M = mm—-—, (42.2) 

s P 

in order to avoid solutions which have excessive size, 
compliance, and mass of long links. We could search 
a large family of mechanism designs to maximize M. 
For example, if a design has five free parameters (typi¬ 
cally link lengths and offsets), and we study ten possible 
values for each parameter, 10 5 designs must be evalu¬ 
ated. 

Available computing power has grown much faster 
than the complexity of realizable mechanisms on the 
human scale (as measured by their DOF). Thus, brute- 
force search of design spaces is often sufficient, and 
sophisticated optimization techniques are not necessary. 

Grounded Versus Ungrounded Devices 
Most current devices that provide kinesthetic feedback 
are physically grounded, that is, forces felt by the opera¬ 
tor are with respect to the operator’s ground, such as the 


floor or desktop. Ungrounded haptic feedback devices 
are more mobile and can operate over larger workspaces 
compared to grounded devices, which enables them to 
be used in large-scale virtual environments. A number 
of ungrounded kinesthetic feedback devices have been 
developed, for example [42.31-33]. Comparisons have 
been made between the performance of ungrounded and 
grounded haptic displays [42.34], Some ungrounded 
devices provide tactile rather than kinesthetic sensa¬ 
tions, and these are described in Sect. 42.5. 

42.2.2 Sensing 

Haptic devices require sensors to measure the state 
of the device. This state may be modified by the op¬ 
erator’s applied position/force, the haptic control law, 
and/or device and environment dynamics. The opera¬ 
tor’s input is sensed in the form of an applied position 
or an applied force. Sensing requirements for haptics 
are similar to those of other robotic devices (Chap. 29) 
so only haptics-specific sensing issues are discussed 
here. 

Encoders 

Rotary optical quadrature encoders are typically used 
as position sensors on the joints of haptic devices. They 
are often integrated with rotary motors, which serve 
as actuators. The underlying sensing mechanism for 
encoders is described in Sect. 29.1. The required res¬ 
olution of an encoder for a haptic device depends on 
the ratio between the angular distance of a single en¬ 
coder tick to the end-point motion in Cartesian space. 
The resolution of the selected position encoder has 
effects beyond simple spatial resolution of the end¬ 
point, including the maximum stiffness that can be 
rendered (Sect. 42.4) without unstable or nonpassive 
behavior [42.35]. 

Many haptic applications, such as the rendering 
of virtual environments with damping (in which force 
is proportional to velocity), require velocity measure¬ 
ment. Velocity is typically obtained by numerical differ¬ 
entiation of the position signal obtained by an encoder. 
An algorithm for velocity estimation must be selected 
which is free of noise but minimizes phase lag at the 
frequencies of interest [42.36]. Thus, an alternative 
method is to use specialized hardware that measures the 
time between encoder ticks in order to compute the ve¬ 
locity [42.37], 

Force Sensors 

Force sensors are used in haptic devices as the operator 
input to an admittance-controlled device, or as a mecha¬ 
nism for canceling device friction and other undesirable 
dynamic properties in an impedance-controlled device. 
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When a force sensor such as a strain gauge or load 
cell measures the operator’s applied force, care must be 
taken to thermally isolate the sensor, since thermal gra¬ 
dients in the sensor caused by body heat can affect force 
readings. 

42.2.3 Actuation and Transmission 

Haptic devices are differentiated from traditional com¬ 
puter input devices by actuators that are controlled to 
provide appropriate haptic sensations to the human op¬ 
erator. The performance of the haptic device depends 
heavily on the actuator properties and the mechanical 
transmission between the actuator and the haptic inter¬ 
action point (HIP). 

Requirements for Haptics 
The primary requirements for actuators and mechan¬ 
ical transmission in impedance-type haptic devices 
are: low inertia, low friction, low torque ripple, back- 
driveability, and low backlash. In addition, if the design 
is such that the actuator itself moves as the user’s posi¬ 
tion changes, a higher power-to-weight ratio is desired. 
Although closed-loop force control has been used for 
haptic display in impedance devices, most often the 
mechanism is designed to have sufficiently low friction 
and inertia so that open-loop force control is accurate 
enough. 

One common mechanical transmission for haptic 
devices is the capstan drive (Fig. 42.6), which consists 
of smooth cables wrapped around pulleys of differing 
diameter to provide a gear ratio. A no-slip, high-friction 
contact between the cable and the pulleys is maintained 
through several wraps of the cable. The capstan drive 



Fig. 42.6 This version of the Haptic Paddle [42.38] in¬ 
cludes an encoder for position sensing, a single-axis load 
cell for force sensing, and a brushed motor with capstan 
transmission for actuation 


minimizes friction forces felt by the human operator be¬ 
cause it prevents translational forces on motor and joint 
axes. 

Current amplifiers are typically used to create a di¬ 
rect relationship between the voltage output by the 
computer via a digital-to-analog (D/A) converter and 
the torque output by the motor. The effect of actuator 
and amplifier dynamics and D/A resolution on sys¬ 
tem stability is typically negligible in comparison to 
position sensor resolution and sampling rate for most 
haptic devices. Actuator or amplifier saturation can pro¬ 
duce undesirable behavior, particularly in multi-DOF 
haptic devices where a single saturated motor torque 
may change the apparent geometry of virtual objects. 
The force vector, and thus the corresponding actuator 
torques, must be scaled appropriately if any actuator is 
saturated. 

42.2.4 An Example Device 

As an illustrative example, we will provide detailed 
design information for a simple 1-DOF haptic device 
known as the Haptic Paddle [42.38]. This section is 
meant to provide a concrete description of the types 
of components that are used in kinesthetic haptic de¬ 
vices, and the device can also be constructed following 
the instructions provided by Johns Hopkins University. 
Many widely available haptic devices share the com¬ 
mon working principles of this device and differ chiefly 
in kinematic details arising from a greater number of 
DOF. 

The haptic paddle shown in Fig. 42.6 is equipped 
with two sensors: a position a position encoder and 
a force sensor. A 500-counts-per-turn Hewlett-Packard 
HEDS 5540 encoder that is mounted directly on the 
motor. The quadrature process yields 2000 counts per 
revolution; the capstan transmission gear ratio and lever 
arm result in a position resolution of 2.24 x 10“ 5 m at 
the haptic interaction point (HIP). An optional load cell 
is used to measure the applied operator force. A plastic 
cap thermally insulates the load cell. In this device, the 
load cell can be used to minimize the effect of friction 
through a control law that attempts to zero the applied 
operator force when the HIP is not in contact with a vir¬ 
tual object. 

The Haptic Paddle shown uses a brushed DC (di¬ 
rect current) motor with an aluminum pulley attached 
to the shaft. Like many commercial haptic devices, it 
uses a capstan drive: a cable is wrapped several times 
around the motor pulley and attached at each end of 
the large partial pulley. In one instantiation, the output 
of the digital-to-analog (D/A) converter from the mi¬ 
croprocessor is passed through a current amplifier that 
gives a current through the motor that is proportional 
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to the D/A voltage. This gives direct control of applied 
torque on the motor. The resulting system has a force 
felt at the driving point that is proportional to the out¬ 


42.3 Haptic Rendering 

Haptic rendering (in impedance systems) is the process 
of computing the force required by contacts with vir¬ 
tual objects based on measurements of the operator’s 
motion. This section describes haptic rendering for vir¬ 
tual environments. Haptic feedback for teleoperators is 
described in Chap. 43. 

An important property of haptic systems is that 
their timing constraints are quite severe. To illustrate 
this point, tap a pencil on the table top. You hear 
a sound which is an audio representation of the con¬ 
tact dynamics between pencil tip and table top. Tactile 
receptors in the human fingers are known to respond up 
to 10kHz [42.1]. To realistically render this type of con¬ 
tact between hard surfaces would require response well 
into the audio range of frequencies (up to 20 kHz) and 
thus sampling times of around 25 |xs. Even with special¬ 
ized designs, haptic devices do not have this bandwidth, 
and such high fidelity is not usually a goal of haptic 
rendering. Achieving stability in any type of hard con¬ 
tact requires very high sampling rates. In practice, most 
haptic simulation systems are implemented with at least 
1000 Hz sampling rate. This can be reduced to the low 
hundreds of Hertz if the virtual environment is limited 
to soft materials. 

Basic Haptic Rendering 

The computational process of haptic rendering can be 
formulated into the following seven sequential steps 
for each cycle (Fig. 42.7). The rendering cycle must 
typically be completed in under 1 ms for stability and 
realism: 

1. Sensing (Sect. 42.2.2) 

2. Kinematics 

3. Collision detection 

4. Determining surface point 

5. Force calculation 

6. Kinematics 

7. Actuation (Sect. 42.2.3). 

Kinematics 

The position and velocity measurements acquired by 
sensors are typically in joint space. These must be con¬ 
verted through the forward kinematics model and the 
Jacobian matrix (Chap. 2) to the Cartesian position and 
velocity of the operator’s hand or fingertip. In some 


put voltage during static operation. When the system is 
moving, the force applied to the operator may differ due 
to human and device dynamics. 


applications the operator is virtually holding a tool or 
object whose shape is represented in the virtual environ¬ 
ment but whose position and orientation are determined 
by the operator. In whose others, the operator’s finger¬ 
tip or hand is represented by a point which makes only 
point contact with objects in the virtual environment 
(VE). We refer to a virtual handheld object as a vir¬ 
tual tool and, following [42.39], we refer to the single 
end-point as the haptic interaction point (HIP). 

Collision Detection 

For the point contact case, the collision detection soft¬ 
ware must determine if the position of the HIP at the 
current instant of time represents contact with a virtual 
object. In practice this usually means to determine if the 
HIP is penetrating or inside the object surface. The ob¬ 
ject surface is represented by a geometric model such 
as polygons or splines. 

Although there is an extensive literature on colli¬ 
sion detection in computer graphics, there are unique 
aspects of the collision detection problem for haptics. 
In particular, speed of computation is paramount and 
worst-case speed, as opposed to average speed, is what 
counts. Solutions that evaluate in constant time are pre- 



Fig. 42.7 Schematic diagram of the haptic rendering cy¬ 
cle for an impedance haptic display system. Virtual object 
moves in the virtual environment according to operator’s 
displacement of the haptic device. Joint displacements (©) 
sensed in the device (1) are processed through kinematics 
(2), collision detection (3), surface point determination (4), 
force calculation (5), kinematics (6), and actuation (7) 
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ferred. Section 42.3.1 addresses collision detection and 
haptic rendering for complex environments. 

If the HIP is found to be outside all objects, then 
a force of zero is returned. 

Determining Surface Point 
Once it is determined that the HIP is inside a surface, 
the force to be displayed to the operator must be com¬ 
puted. Many researchers have used the idea of a virtual 
spring connecting the HIP to the nearest point on the 
surface as a model of interpenetration and force gener¬ 
ation [42.39-41]. Basdogan and Srinivasan named this 
point the intermediate haptic interaction point (IHIP). 
However all of these authors realized that the closest 
surface point is not always the most faithful model of 
contact. For example, as the HIP moves laterally below 
the top surface of a cube (Fig. 42.8), eventually it be¬ 
comes close enough to the edge that the closest surface 
point becomes the side of the cube. In this situation, the 
algorithm needs memory to keep the IHIP on the top 
surface and generate an upward force at all times or the 
operator is suddenly ejected out the side of the cube. 

Force Calculation 

Force is commonly computed by using the spring model 
(Hooke’s Law) 

f=kx , ( 42 . 3 ) 

where x is the vector from the HIP to the IHIP, and k > 
0. When k is sufficiently large, the object surface will 
feel like a wall perpendicular to x. This virtual wall, 
or impedance surface, is a fundamental building block 



Fig. 42.8 Illustration of subtle aspects of haptic rendering 
of contact force. The operator fingertip trajectory enters 
object surface moving down and to right. Haptic interac¬ 
tion points (HIP) are shown at times 1-4 (solid circles, 
Pq—P 4 ). Intermediate haptic interaction points (IHIP) are 
shown when the HIP is inside the object (open circles). 
At position P 4 , the algorithm must not render force based 
on the closest surface point or the operator will be ejected 
from the side of the object (feeling unnatural force tangen¬ 
tial to the top surface) 


of most haptic virtual environments. Because the vir¬ 
tual wall is only displayed if a collision between the 
HIP and virtual object is detected, the wall is a uni¬ 
lateral constraint, governed by a nonlinear switching 
condition. As will be described in the following section, 
haptic virtual environments with complex geometries 
are often formed using a polygonal mesh in which each 
polygon is essentially a virtual wall. A virtual surface 
may also be allowed to deform globally, while the local 
interaction with the operator is governed by a virtual 
wall. Virtual fixtures, which are often constructed from 
virtual walls, can be overlaid on haptic feedback teleop¬ 
erators to assist the operator during a teleoperated task 
(Chap. 43). 

The pure stiffness model described above can be 
augmented to provide other effects, particularly through 
the use of the virtual coupling described in Sect. 42.3.2. 
Damping can be added perpendicular or parallel to the 
surface. In addition. Coulomb or other nonlinear fric¬ 
tion may be displayed parallel to the surface. To provide 
a more realistic display of hard surfaces, vibrations can 
also be displayed open loop at the moment of colli¬ 
sion between the HIP and the surface, as is described 
in Sect. 42.5. 

Kinematics 

The computed force in Cartesian space must then be 
transformed into torques in the actuator space. Typi¬ 
cally the calculation is 

t=J T /, ( 42 . 4 ) 

where r is the torque command to the actuators, / is 
the desired force vector, and J T is the transpose of the 
haptic device Jacobian matrix (Chap. 2). If the haptic 
device has no dynamics and the actuators are perfect, 
the exact desired force is displayed to the operator. 
However, real device dynamics, time delays, and other 
nonidealities result in applied operator forces that differ 
from the desired forces. 

42.3.1 Rendering Complex Environments 

Today’s computer power is sufficient that a variety 
of relatively simple algorithms can effectively render 
haptics for simple virtual environments, say consist¬ 
ing of a handful of simple geometric primitives such 
as spheres, cubes, and planes. However, the challenge 
is to scale these algorithms to complex environments 
such as those we are used to seeing in computer 
graphic renderings consisting of 10 5 — 10 7 polygons. 
A variety of approaches have been tried in the liter¬ 
ature for efficient rendering of complex scenes. Zilles 
and Salisbury [42.40] found planar constraints due to 
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the nearby surface polygons and solved for the clos¬ 
est IHIP point using Lagrange multipliers. Ruspini and 
Khatib [42.42] added force shading and friction mod¬ 
els. Ho et al. [42.39] used a hierarchy of bounding 
spheres to determine the initial contact (collision) point, 
but thereafter searched neighboring surfaces, edges, and 
vertices (referred to as geometric primitives) of the cur¬ 
rent contacted triangle to find the closest point for the 
IHIP. Gregory et al. [42.43] imposed a discretization 
of three-dimensional (3-D) space onto the hierarchy to 
speed up the detection of initial contact with the surface. 
Johnson et al. [42.44] performed haptic rendering on 
moving models based on local extrema in distance be¬ 
tween the model controlled by the haptic device and the 
rest of the scene. Lin and Otaduy [42.45, 46] used level- 
of-detail representations of the objects for performing 
multiresolution collision detection, with the goal of 
satisfying real-time constraints while maximizing the 
accuracy of the computed proximity information. 

Alternative algorithms and efficiencies can be de¬ 
rived by not representing the object surface as polygons. 
Thompson and Cohen [42.47] derived the mathematics 
for computing the interpenetration depth for surfaces 
directly from non-uniform rational B-spline (NURBS) 
models. McNeely et al. [42.18] took the extreme ap¬ 
proach of voxelizing space at the millimeter scale. Each 
voxel contained a precomputed normal vector, stiffness 
properties, etc. 1000Hz rendering was achieved with 
very complex CAD models containing millions of poly¬ 
gons (in the graphical equivalent representation), but 
large amounts of memory and precomputation are re¬ 
quired. The performance of this algorithm is sufficiently 
high that it can be used to render hundreds of contact 
points simultaneously. This allows the operator to hold 
an arbitrarily shaped tool or object. The tool/object is 
populated by points surrounding its surface and the re¬ 
sultant force and moment on the tool is the sum of 
interaction forces computed on all of the surface points. 

For surgical simulation, researchers have focused 
on the modeling and haptic rendering of the interaction 
between surgical instruments and organs. Researchers 
have attempted to model virtual tissue behavior in 


a wide variety of ways, which can be broadly classified 
as: 

1. Linear elasticity based 

2. Nonlinear (hyperelastic) elasticity-based finite- 
element (FE) methods 

3. Other techniques that are not based on FE methods 
or continuum mechanics. 

While most conventional linear and nonlinear FE al¬ 
gorithms cannot be run in real time, methods such 
as preprocessing can allow them to run at haptic 
rates [42.48]. Many researchers rely on data acquired 
from real tissues to model organ deformation and 
fracture accurately. Major challenges in this field in¬ 
clude the modeling of connective tissue supporting the 
organ, friction between instruments and tissues, and 
topological changes occurring during invasive surgical 
procedures. 

42.3.2 Virtual Coupling 

So far we have rendered forces by computing the length 
and direction of a virtual spring and applying Hooke’s 
Law (42.3). This spring is a special case of a virtual 
coupling [42.41] between the HIP and the IHIP. The 
virtual coupling is an abstraction of the interpenetration 
model of force rendering. Instead of viewing the objects 
as compliant, we assume them to be rigid but connect 
them to the operator through a virtual spring. This im¬ 
poses an effective maximum stiffness (that of the virtual 
coupling). 

Problems with stable contact rendering (Sect. 42.4) 
often require more sophisticated virtual couplings than 
a simple spring. For example, damping can be added, 
generalizing the force rendering model of (42.3) to 

f = kx + bx. ( 42 . 5 ) 

The parameters k and b can be empirically tuned for 
stable and high-performance operation. More-formal 
design methods for the virtual coupling are covered in 
Sect. 42.4. 


42.4 Control and Stability of Force Feedback Interfaces 


Introduction to the Problem 
The haptic rendering system depicted in Fig. 42.7 is 
a closed-loop dynamical system. It is a challenge to ren¬ 
der realistic contact forces yet retain the stable behavior 
of human-environment contact in the natural world. In¬ 
stability in haptic interfaces manifests itself as buzzing, 
bouncing, or even wildly divergent behavior. The worst 


case for impedance devices is during attempted contact 
with stiff objects. Empirically, instability is frequently 
encountered when developing haptic interfaces with 
stiff virtual objects, but this instability can be elimi¬ 
nated by reducing the stiffness of the virtual object or 
by the operator making a firmer grasp on the haptic 
device. 
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Problem Description in Classical Control Terms 
Although linear theory is of very limited use in haptic 
control applications, it can be applied to a basic anal¬ 
ysis of the factors affecting instability [42.49]. Such 
a highly simplified model of an impedance device is 
shown in Fig. 42.9. G\ (s) and Gzls) represent dynamics 
of the haptic device for both operator position sensing 
and force display respectively. Assume that the virtual 
environment and human operator/user (HO) can each 
be represented by a linear impedance such as 


ZvE 

Zho 


Fve(s) 
Xye(s) 
Fho(s) 
2Cho (s) 


(42.6) 

(42.7) 


Then the loop gain of the closed-loop system from the 
human operator and back again is 


G,(s) 


Gds)G 2 (s) 


Z V e(s) 

Zho(s) 


(42.8) 


Sampling 

Colgate et al. [42.41] incorporated consideration of dis¬ 
crete time sampling behavior in the stability analysis. 
They considered the problem of implementing a virtual 
wall of stiffness 

H(z) = K + B , (42.9) 

Tz 

where K is the virtual wall stiffness, B is the virtual wall 
damping coefficient, z is the ^-transform variable, and 
T is the sampling time. They further modeled the haptic 
device (HD) in continuous time as 

Zhd(s) = —-—~ , (42.10) 

ms + b 

where m and b are the mass and damping of the haptic 
device, respectively. They derived the following condi¬ 
tion for passivity of the device 

KT 

b>— + \B\, (42.11) 


Stability in the classical sense is assessed by applying 
the magnitude and phase criteria of Nyquist to G/(s). 
Increasing Zve (corresponding to stiffer or heavier vir¬ 
tual objects) increases the magnitude of G/(s) and thus 
destabilizes the system while a firmer grasp by the hu¬ 
man operator, which increases the magnitude of Zho, 
has a stabilizing effect. Similar arguments apply to 
phase shifts that might be present in any part of the 
system. 


showing a significant stabilizing effect of high sampling 
rates and also of high mechanical damping in the haptic 
device. 

Quantization 

Additional factors include delays due to numerical in¬ 
tegration schemes and quantization. These contributing 
factors to instability have been termed energy leaks by 
Gillespie and Cutkosky [42.50]. 


Limitations of Linear Theory 
Although the model of Fig. 42.9 illustrates some quali¬ 
tative features of haptic interface stability, linear contin¬ 
uous time theory is of little use in designing methods to 
stabilize the loop. Interesting virtual environments are 
nonlinear. Furthermore, they can rarely be linearized 
because applications often simulate discontinuous con¬ 
tact, for example, between a stylus in free space and 
a hard surface. A second feature is digital implementa¬ 
tion, which introduces sampling and quantization - both 
of which have significant effects. 



Fig. 42.9 Highly simplified linear model of haptic render¬ 
ing to highlight some stability issues 


Passivity 

Interesting virtual environments are always nonlinear 
and the dynamic properties of a human operator are im¬ 
portant. These factors make it difficult to analyze haptic 
systems in terms of known parameters and linear con¬ 
trol theory. One fruitful approach is to use the idea 
of passivity to guarantee stable operation. Passivity is 
a sufficient condition for stability, and is reviewed more 
completely in Chap. 43 on telerobotics. There are many 
similarities between the control of haptic interfaces and 
bilateral teleoperation. 

The major problem with using passivity for design 
of haptic interaction systems is that it is overly conser¬ 
vative, as shown in [42.35]. In many cases, performance 
can be poor if a fixed damping value is used to guar¬ 
antee passivity under all operating conditions. Adams 
and Hannaford derived a method of virtual coupling 
design from two-port network theory which applied to 
all causality combinations and was less conservative 
than passivity based design [42.51]. They were able to 
derive optimal virtual coupling parameters using a dy¬ 
namic model of the haptic device and by satisfying 
Lewellyn’s absolute stability criterion, an inequality 
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composed of terms in the two-port description of the extended the analysis to nonlinear environments and 
combined haptic interface and virtual coupling system, extracted a damping parameter to guarantee stable op- 
Miller et al. derived another design procedure which eration [42.52-54], 


42.5 Other Types of Haptic Interfaces 

While kinesthetic (force feedback) haptic interfaces 
have the closest relationship to robotics, there are a va¬ 
riety of other types of haptic interfaces which are 
usually classified as tactile displays. Tactile displays 
are used to convey force, contact and shape informa¬ 
tion to the skin. They purposely stimulate cutaneous 
receptors, with little effect on kinesthetic sensation. 
This is in contrast to kinesthetic displays, which must 
inherently provide some cutaneous sensations through 
physical contact with a tool or thimble, but whose 
primary output is force or displacement to the limbs 
and joints. Tactile displays are usually developed for 
a specific purpose, such as display of contact events, 
contact location, slip/shear, texture, and local shape. 
Special purpose tactile displays can be targeted at dif¬ 
ferent types of cutaneous receptors, each with its own 
frequency response, receptive field, spatial distribution, 
and sensed parameter (e.g., local skin curvature, skin 
stretch, and vibration) and each receptor type is associ¬ 
ated with different exploratory procedures described in 
Sect. 42.1.1. 

In contrast, tactile displays that render contact infor¬ 
mation for virtual reality or teleoperation have proven 
far more challenging. Accurate recreation of the lo¬ 
cal shape and pressure distribution at each fingertip 
requires a dense array of actuators. Devices specifi¬ 
cally aimed at tactile perception are an active area of 
research but most have not reached the stage of ap¬ 
plications or commercial distribution, with the notable 
exception of Braille displays for the blind. In this sec¬ 
tion, we describe the various types of tactile displays, 
their design considerations and specialized rendering 
algorithms, and applications. 

42.5.1 Vibrotactile Feedback 

Vibrotactile feedback is a popular method of providing 
tactile feedback. It can be used as a stand-alone method 
for haptic feedback or as an addition to a kinesthetic dis¬ 
play. Vibrating elements, such as piezoelectric materials 
and small voice-coil motors, are lighter than the actua¬ 
tors used in kinesthetic devices, and can often be added 
to kinesthetic devices with little impact on existing 
mechanisms. In addition, high-bandwidth kinesthetic 
displays can be programmed to display open-loop vi¬ 
brations through their normal actuators. The sensitivity 


for human vibration sensing ranges from DC to over 
1 kHz, with peak sensitivity around 250 Hz. 

We first consider the use of vibrations to convey 
impact or contact events - a technique that straddles 
kinesthetic and tactile feedback. When humans touch 
an environment, fast-acting sensors embedded in the 
skin record the minute vibrations occurring from this 
interaction. As described in Sect. 42.3, conventional 
approaches to haptic display usually consist of design¬ 
ing a virtual model with simple geometry, then using 
a first-order stiffness control law to emulate a surface. 
However, such first-order models often lack the realism 
of higher-order effects such as impact. With com¬ 
mon haptic rendering algorithms, surfaces feel squishy 
or unrealistically smooth. One solution to improving 
the realism of such environments is to add higher- 
order effects such as textures and contact vibrations. 
These effects can use a library of surface models based 
on ad hoc analytical descriptions [42.55], which are 
sometimes tuned using qualitative operator feedback, 
physical measurements (reality-based models created 
from empirical data) [42.56, 57], or a combination of 
the two [42.58]. At the instant collision is detected 
between the HIP and a surface of the virtual object, 
the appropriate waveform is called out of the library, 
scaled according to the context of the motion (such 
as velocity or acceleration), and played open loop 
through an actuator. That actuator may be the same 
one simultaneously displaying lower-frequency force 
information, as shown in Fig. 42.10, or it might be 
a separate transducer. Kuchenbecker et al. [42.59] con¬ 
sidered the dynamics of the haptic device to display 
the most accurate vibration waveforms possible, and 
compared a number of different vibration waveform 
generation techniques meant to convey impact super¬ 
imposed on force feedback in virtual environments. 
Most of the vibration feedback methods performed 
similarly in terms of realism, and they were also signif¬ 
icantly more realistic than conventional force feedback 
alone. 

Vibration feedback can also be used to provide 
information about patterned textures, roughness, and 
other phenomena that have clear vibratory signals. This 
type of vibration feedback is often termed vibrotac¬ 
tile feedback. In teleoperated environments, Kontarinis 
and Howe [42.60] showed that damaged ball bear- 
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ings could be identified through vibration feedback, 
and Dennerlein et al. [42.61] demonstrated that vibra¬ 
tion feedback improved performance over no haptic 
feedback, for a telemanipulation task inspired by un¬ 
dersea field robotics. In these teleoperator systems, 
vibration-sensitive sensors such as accelerometers and 
piezoelectric sensors are used to pick up and, in most 
cases, directly provide the vibration signals as inputs 
to a vibrotactile actuator. In virtual environments, Oka- 
mura et al. [42.62] displayed vibrations modeled based 
on textures and puncture of a membrane. Similar to 
the event-based haptics above, the vibration wave¬ 
forms were modeled based on earlier experiments and 
played open loop during interaction with the virtual 
environment. 

Finally, vibration feedback has been used as 
a method of sensory substitution, to convey direction, 
attention, or other information, e.g., [42.63-65]. In this 
case, the strength and clarity of the signal, not real¬ 
ism, is the goal. Vibration frequencies near the peak 
human sensitivity are most effective, and there exist 
commercially available vibrotactile actuators (or fac¬ 
tors) suitable for such applications, e.g., Engineering 
Acoustic, Inc.’s C2 Tactor [42.66]. Elements within an 
array of tactors can be selectively turned on and off 
to evoke the sensory saltation phenomenon, in which 
a pattern of brief pulses on a series of tactors is per¬ 
ceived not as successive taps at different locations, but 
as a single tap that is traveling or hopping over the 
skin. 

Position 



Force Transient 



Fig. 42.10 Event-triggered open-loop force signals super¬ 
impose on traditional penetration-based feedback forces, 
providing vibration feedback that improves the realism of 
hard surfaces in virtual environments (after [42.59] with 
permission) 


42.5.2 Contact Location, Slip, 
and Shear Display 

In early work related to robotic dexterous manipula¬ 
tion, it was found that knowledge of the contact point 
between a robot hand and a grasped object is essen¬ 
tial for manipulation. Without this knowledge, the robot 
will easily drop the object due to rapid accumulation 
of grasp errors. While many robotics researchers and 
some companies have developed tactile array sensors 
capable of measuring contact location, pressure distri¬ 
bution, and local object geometry (Chap. 28), practical 
methods for display of this information to the human 
operator of a virtual or teleoperated environment have 
proven much more difficult. We begin our discussion 
of contact display by considering contact location, slip, 
and shear display, which have the common goal of dis¬ 
playing the motion of a single area of contact relative 
to the skin (almost invariably on a finger). Arrays of 
pins that rise and fall to create a pressure distribution 
on the skin have been the most popular method to date 
for displaying contact information, but we will address 
those designs in the following section on local shape, 
since their primary advantage is the display of spatially 
distributed information. Instead, we will focus here on 
tactile devices that are designed to specifically address 
the problem of contact location and motion. 

As an example of contact location display, 
Provancher et al. [42.67] developed a system that ren¬ 
ders the location of the contact centroid moving on 
the user’s fingertip. The tactile element is a free-rolling 
cylinder that is normally suspended away from the fin¬ 
gertip, but comes into contact with the skin when the 
operator pushes on a virtual object. The motion of 
the cylinder over the skin is controlled by sheathed 
push-pull wires. This allows the actuators to be placed 
remotely, creating a lightweight, thimble-sized pack¬ 
age that can be unobtrusively mounted on a kinesthetic 
haptic device. An experiment demonstrated that human 
operators performed similarly during real manipulation 
and virtual manipulation (using the tactile display) in an 
object curvature discrimination task. In addition, oper¬ 
ators were able to use the device to distinguish between 
manipulations of rolling and anchored but rotating vir¬ 
tual objects. 

42.5.3 Slip and Shear 

Humans use slip and incipient slip widely during ma¬ 
nipulation tasks [42.68]. To reproduce these sensations 
for experiments to characterize human slip sensation, 
researchers have created stand-alone 1-DOF slip dis¬ 
plays [42.69-71]. Webster et al. [42.71] created a 2- 
DOF tactile slip display, which uses an actuated ro- 
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tating ball positioned under the user’s fingertip. The 
lightweight, modular tactile display can be attached to 
a multi-DOF kinesthetic interface and used to display 
virtual environments with slip. Experimental results 
demonstrate that operators complete a virtual manipula¬ 
tion task with lower applied forces using combined slip 
and force feedback in comparison with conventional 
force feedback alone. Skin stretch can also be integrated 
with a slip display to provide information about pre-slip 
conditions. For example, Tsagarakis et al. [42.72] de¬ 
veloped a lightweight device that uses a V configuration 
of miniature motors to provide sensations of relative 
lateral motion (direction and velocity) onto the opera¬ 
tor’s fingertips. Generation of two-dimensional (2-D) 
slip/stretch is achieved by coordinating the rotational 
speed and direction of the two motors. 

In terms of tactile device kinematics, slip and shear 
displays can be quite similar. However, the goal of 
shear, or skin stretch, displays is to maintain a no-slip 
condition such that shear forces/motions can be accu¬ 
rately controlled with respect to the human operator, 
typically on the finger. The use of shear (tangential skin 
stretch) is motivated by perceptual experiments demon¬ 
strating that the human fingerpad is more sensitive to 
tangential displacement compared to normal displace¬ 
ment [42.73]. A variety of tangential skin stretch de¬ 
vices have been designed, with applications in wearable 
haptics, high-fidelity tactile rendering, and teleopera¬ 
tion. Hayward and Cruz-Hernandez [42.74] developed 
a tactile device consisting of closely packed piezoelec¬ 
tric actuators, which generates a programmable stress 
field within the fingerpad. Single skin-stretch tactors 
have also been used to convey two-dimensional di¬ 
rectional information (for navigation) [42.75] and to 
enhance perception of virtual environments (e.g., fric¬ 
tion) [42.76] alone or in combination with kinesthetic 
haptic interfaces. A similar approach has been used 
for 3-DOF skin stretch, in combination with cutaneous 
normal force [42.77,78]. In a different form factor, re¬ 
searchers have developed rotational skin stretch devices 
that can act as a substitute for natural proprioceptive 
feedback [42.79, 80]. 

42.5.4 Local Shape 

Most tactile devices for local shape display consist of 
an array of individual pin elements that move nor¬ 
mal to the surface. Often, a layer of elastic material 
is used to cover the pins so that the operator con¬ 
tacts a smooth surface rather than the pins directly. 
Other systems use individual elements that move later¬ 
ally, and some substitute electrodes for moving parts to 
form an array of electrocutaneous elements. A number 
of researchers have used psychophysical and percep¬ 


tual experimental results to define design parameters 
such as number of pins, spacing, and amplitude of pin- 
based tactile displays. A commonly used metric is the 
two-point discrimination test, which defines the mini¬ 
mum distance between two contact points on the skin 
at which they are perceived as two, rather than one 
point. This discrimination limit varies widely for skin 
on different parts of the body, with the fingertips hav¬ 
ing one of the smallest (usually cited as less than 1 mm, 
although this depends on the shape and size of the 
contacts) [42.81, 82]. Moy et al. [42.83] quantified sev¬ 
eral perceptual capabilities of the human tactile system 
based on predicted subsurface strain and psychophysi¬ 
cal experiments that measured amplitude resolution, the 
effects of shear stress, and the effects of viscoelastic¬ 
ity (creep and relaxation) on tactile perception for static 
touch. They found find that 10% amplitude resolution 
is sufficient for a teletaction system with a 2 mm elas¬ 
tic layer and 2 mm tactor spacing. A different type of 
experiment examines the kind of tactile information rel¬ 
evant to a particular application. For example, Peine and 
Howe [42.84] found that sensed deformation of the fin¬ 
gerpad, and not changes in pressure distribution, were 
responsible for localizing lumps in a soft material, such 
as a tumor in tissue. 

We will now highlight a few distinctive designs 
of array-type tactile displays. A number of actuator 
technologies have been applied to create tactile arrays, 
including piezoelectric, shape-memory alloy (SMA), 
electromagnetic, pneumatic, electrorheological, micro¬ 
electromechanical system (MEMS), and electrotactile. 
Further reading on tactile display design and actuation 
is available in review papers, including [42.85-90]. 

We will first consider two ends of the spectrum in 
complexity/cost of the pin-based approach. Killebrew 
et al. [42.91] developed a 400-pin, 1cm 2 tactile stim¬ 
ulator to present arbitrary spatiotemporal stimuli to the 
skin for neuroscience experiments. Each pin is under in¬ 
dependent computer control and can present over 1200 
stimuli per minute. While not practical for most haptic 
applications due to the size and weight of the actua¬ 
tion unit, it is the highest-resolution tactile display built 
to date and can be used to evaluate potential designs 
for lower-resolution displays. Wagner et al. [42.92] cre¬ 
ated a 36-pin, 1 cm 2 tactile shape display that uses 
commercially available radio-controlled (RC) servomo¬ 
tors. The display can represent maximum frequencies 
of 7.5—25 Hz, pending on the amount of pin deflec¬ 
tion, and is shown in Fig. 42.11. Howe et al. [42.93, 
94] have also explored the use of shape-memory al¬ 
loys for pin actuation, for the application of remote 
palpation. 

In contrast to pins that move normal to the sur¬ 
face, recent tactile array designs have incorporated 
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pins that move laterally. First introduced by Hayward 
and Cruz-Hernandez [42.95], the most recent com¬ 
pact, lightweight, modular design [42.96] uses a 6 x 10 
piezo bimorph actuator array with a spatial resolution 
of 1.8 mm x 1.2 mm. The force of the individual ac¬ 
tuators provides sufficient skin pad motion/stretch to 
excite mechanoreceptors [42.97]. A pilot test demon¬ 
strated that subjects could detect a virtual line randomly 
located on an otherwise smooth virtual surface, and the 
device has also been tested as a Braille display [42.98]. 
Another lateral stretch display and its evaluation is de¬ 
scribed in [42.99]. Other novel approaches to tactile 
display include sending small currents through the skin 
or tongue using an electrocutaneous array [42.100] and 
the application of air pressure to stimulate only superfi¬ 
cial mechanoreceptors [42.101]. 

42.5.5 Surface Displays 

In recent years, the concept of surface displays, which 
modulate surface friction in order to display changing 
shear forces as a user moves the finger over the sur¬ 
face. One example device that uses slip and friction 
to display compelling tactile sensations is the TPaD 



Fig. 42.11 (a) A low-cost 36-pin tactile display using RC 
servomotor actuation. (b) A closeup of the 6x6 dis¬ 
play, showing a sine wave grating (after [42.92] with 
permission) 


(tactile pattern display) [42.102]. Ultrasonic frequency, 
low amplitude vibrations of a flat plate create a film 
of air between the plate and a human finger touching 
the plate, thereby reducing friction. The 33 kHz vibra¬ 
tion of the plate cannot be perceived by the human. 
The amount of friction reduction varies with vibration 
amplitude, allowing indirect control of shear forces on 
the finger during active exploration. Finger position and 
velocity feedback enables haptic rendering of spatial 
texture sensations. This work has been expanded into 
a variety of different surface displays, including de¬ 
vices that generate active forces [42.103] and those that 
use electrostatic forces to modulate friction [42.104, 
105], 

42.5.6 Temperature 

Because the human body is typically warmer than ob¬ 
jects in the environment, thermal perceptions are based 
on a combination of thermal conductivity, thermal ca¬ 
pacity, and temperature. This allows us to infer not 
only temperature difference, but also material compo¬ 
sition [42.106]. Most thermal display devices are based 
on thermoelectric coolers, also known as Peltier heat 
pumps. Thermoelectric coolers consist of a series of 
semiconductor junctions connected electrically in series 
and thermally in parallel. The thermoelectric cooler is 
designed to pump heat from one ceramic faceplate to 
the other, but if used in reverse, a temperature gradient 
across the device produces a proportional potential; as 
a measure of relative temperature change. The designs 
of haptic thermal displays mostly use off-the-shelf com¬ 
ponents, and their applications are typically straightfor¬ 
ward, enabling identification of objects in a virtual or 
teleoperated environment by their temperature and ther¬ 
mal conductivity. 

Ho and Jones [42.107] provide a review of haptic 
temperature display, as well as promising results sug¬ 
gesting that a thermal display is capable of facilitating 
object recognition when visual cues are limited. Al¬ 
though numerous systems have integrated thermal dis¬ 
play with other types of haptic display, the Data Glove 
Input System designed by Caldwell et al. [42.108, 109] 
was one of the first to do so. Their haptic interface pro¬ 
vides simultaneous force, tactile, and thermal feedback. 
The Peltier device used for thermal display contacts the 
dorsal surface of the index finger. Subjects achieved 
a 90% success rate in identifying materials such as 
a cube of ice, a soldering iron, insulating foam, and 
a block of aluminum, based only on thermal cues. The 
study of human temperature perception is particularly 
interesting, including issues such as spatial summation 
and the psychological relevance of temperature display. 
For example, in prosthetic limbs, temperature display 
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may be useful not only for practical reasons such as personal comfort, such as feeling the warmth of a loved 
safety and material identification, but also for reasons of one’s hand. 


42.6 Conclusions and Further Reading 


Haptic technology, which attempts to provide com¬ 
pelling sensations to human operators in virtual and 
teleoperated environments, is a relatively new, but fast¬ 
growing and dynamic area of research. The field relies 
not only on fundamental foundations from robotics and 
control theory, but also on fields in the human sci¬ 
ences, particularly neuroscience and psychology. To 
date, commercial success of haptics has been in the ar¬ 
eas of entertainment, medical simulation, and design, 
although novel devices and applications are regularly 
appearing. 

There exist many books on the topic of haptic 
technology, most of them compendiums from work¬ 
shops or conferences on the subject. One of the earliest 
books on haptics, by Burden [42.1 10], provides a thor¬ 
ough review of applications and haptic devices up 
to 1996. A book specifically focused on haptic ren¬ 
dering, designed for potential use as a textbook, has 
been edited by Lin and Otaduy [42.111]. In addition, 
we recommend the following useful articles: Hayward 
and MacLean [42.112, 113] describe the fundamentals 
of constructing experimental haptic devices of modest 


complexity, the software components needed to drive 
them, and the interaction design concepts important to 
creating usable systems. Hayward et al. [42.114] also 
provide a tutorial on haptic devices and interfaces. Sal¬ 
isbury et al. [42.115] describe the basic principles of 
haptic rendering. Hayward and MacLean [42.116] de¬ 
scribes a number of tactile illusions that can inspire 
creative solutions for haptic interface design. Robles- 
De-La-Torre [42.117] underscores the importance of 
haptics with compelling examples of humans who have 
lost the sense of touch. 

Finally, there exist two journals that are specific to 
the field of haptics: Haptics-e [42.1 18] and IEEE Trans¬ 
actions on Haptics (first issue expected 2008). Several 
conferences are specifically devoted to haptics: Euro¬ 
haptics and the Symposium on Haptic Interfaces for 
Virtual Environment and Teleoperator Systems are held 
separately in even years, and on the odd years become 
a single conference, World Haptics. The IEEE (Institute 
of Electrical and Electronics Engineers) technical com¬ 
mittee on haptics [42.119] provides information about 
relevant publication forums. 
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43. Telerobotics 


Gunter Niemeyer, Carsten Preusche, Stefano Stramigioli, Dongjun Lee 


In this chapter we present an overview of the field 
of telerobotics with a focus on control aspects. 
To acknowledge some of the earliest contribu¬ 
tions and motivations the field has provided to 
robotics in general, we begin with a brief historical 
perspective and discuss some of the challenging 
applications. Then, after introducing and classify¬ 
ing the various system architectures and control 
strategies, we emphasize bilateral control and 
force feedback. This particular area has seen in¬ 
tense research work in the pursuit of telepresence. 
We also examine some of the emerging efforts, 
extending telerobotic concepts to unconventional 
systems and applications. Finally, we suggest some 
further reading for a closer engagement with the 
field. 
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43.1 Overview and Terminology 

Telerobotics is perhaps one of the earliest aspects and 
manifestations of robotics. Literally meaning robotics 
at a distance, it is generally understood to refer to 
robotics with a human operator in control or human- 
in-the-loop. Any high-level, planning, or cognitive de¬ 
cisions are made by the human user, while the robot 
is responsible for their mechanical implementation. In 
essence, the brain is removed or distant from the body. 

Herein the term tele, which is derived from the 
Greek and means distant, is generalized to imply a bar¬ 
rier between the user and the environment. This barrier 
is overcome by remote-controlling a robot at the en¬ 


vironment, as indicated in Fig. 43.1. Besides distance, 
barriers may be imposed by hazardous environments or 
scaling to very large or small environments. All barri¬ 
ers prevent the user from physically reaching or directly 
interacting with the environment. 

While the physical separation may be very small, 
with the human operator and the robot sometimes oc¬ 
cupying the same room, telerobotic systems are often 
at least conceptually split into two sites. The local site 
encompasses the human operator and all elements nec¬ 
essary to support the system’s connection with the user, 
which could be joysticks, monitors, keyboards, or other 
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Operator-site Remote-site 



Fig. 43.1 Overview of a telerobotic system (after [43.1], adapted 
from [43.2]) 


input/output devices. The remote site contains the robot, 
supporting sensors and control elements, and the envi¬ 
ronment to be manipulated. 

To support its operation, telerobotics integrates 
many areas of robotics. At the remote site, to operate 
the robot and execute the human’s commands, the sys¬ 
tem may control the motion and/or forces of the robot. 
We refer to Chaps. 7 and 8 for detailed descriptions of 
these areas. Also, sensors are invaluable (Chap. 5), in¬ 
cluding force sensors (Chap. 28) and others (Part C). 
Meanwhile, at the local site information is often dis¬ 
played haptically (Chap. 41). 

A recent addition to telerobotics is the use of 
computer networks to transmit information between 
the sites. The ubiquity of network access is allow¬ 
ing remote control from anywhere on demand. Chap¬ 
ter 44 discusses some of these developments, de¬ 
tailing network infrastructure and focusing on vi¬ 
sual, often web-based, user interfaces, avoiding the 
need for specialized mechanical I/O (input/output) 
devices. Computer networks also allow new multi¬ 
lateral telerobotic architectures. For example, mul¬ 
tiple users may share a single robot or a single 
user may control multiple robots (Sect. 43.5.2). Un¬ 
fortunately, computer networks often see transmis¬ 
sion delays and can introduce nondeterministic effects 
such as variable delay times and data losses. These 
effects can easily destabilize force feedback loops 
and require particular countermeasures (Sects. 43.4.4- 
43.4.6). 

We should also point out the relation between 
telerobotics and human exoskeletons, as described in 
Chap. 69. Exoskeletons are also controlled by a human 
operator, leaving all planning and high-level challenges 
to the user, and their control systems share many as¬ 
pects with telerobotics. However, the two sites are 
physically combined in an exoskeleton as the user di¬ 
rectly touches and interacts with the robot. In this 
chapter, we will disallow any such direct mechanical 
connection. 


The inclusion of the human operator makes teler¬ 
obotics very attractive to handle unknown and unstruc¬ 
tured environments. Applications are plentiful (Part F) 
and range from space robotics (Chap. 55) to dealing 
with hazardous environments (Chap. 58), from search 
and rescue situations (Chap. 60), to medical systems 
(Chap. 63) and rehabilitation (Chap. 64). 

Before proceeding, we define some basic termi¬ 
nology. Indeed many other terms are used nearly 
synonymously with telerobotics, in particular teleop¬ 
eration and telemanipulation. Telerobotics is the most 
common, emphasizing a human’s (remote) control of 
a robot. Teleoperation stresses the task-level operations, 
while telemanipulation highlights object-level manipu¬ 
lation. 

Within telerobotics, a spectrum of control architec¬ 
tures is used. Direct control or manual control falls 
at one extreme, indicating that the user is controlling 
the motion of the robot directly and without any auto¬ 
mated help. At the other extreme, super\’isory control 
implies that user’s commands and feedback occur at 
a very high level and the robot requires substantial in¬ 
telligence or autonomy to fulfill its function. Between 
the two extrema lie a variety of shared control archi¬ 
tectures, where some degree of autonomy or automated 
help is available to assist the user. 

In practice, many systems involve at least some 
level of direct control and accept the user’s motion 
commands via a joystick or similar device in the user in¬ 
terface. The joystick is an instrumented mechanism and 
can itself be viewed as a robot. The local and remote 
robots are called master and slave respectively, while 
the system is referred to as a master-slave system. To 
provide direct control, the slave robot is programmed to 
follow the motions of the master robot, which is posi¬ 
tioned by the user. It is not uncommon for the master 
robot (joystick) to be a kinematic replica of the slave, 
providing an intuitive interface. 

Some master-slave systems provide force feedback, 
such that the master robot not only measures motions 
but also displays forces to the user. The user inter¬ 
face becomes fully bidirectional and such telerobotic 
systems are often called bilateral. The human-master 
interactions are a form of human-robot interaction 
(Chap. 69). The field of haptics (Chap. 41) also dis¬ 
cusses bidirectional user interfaces, involving both mo¬ 
tion and force, though more commonly to interface the 
user with virtual instead of remote environments. We 
should note that both motion and force may become the 
input or output to/from the user, depending on the sys¬ 
tem architecture. 

Finally, telepresence is often discussed as an ul¬ 
timate goal of master-slave systems and telerobotics 
in general. It promises to the user not only the abil- 























Telerobotics I 43.2 Telerobotic Systems and Applications 1087 


ity to manipulate the remote environment, but also to 
perceive the environment as if encountered directly. 
The human operator is provided with enough feed¬ 
back and sensations to feel present in the remote site. 
This combines the haptic modality with other modali¬ 
ties serving the human senses of vision, hearing or even 
smell and taste. See videos l<ss>)KZn]]EEBi, 
and and l<£»KHiE!E9 for some early 

and recent results aiming for this telepresence. We fo¬ 
cus our descriptions on the haptic channel, which is 
created by the robotic hardware and its control sys¬ 
tems. The master-slave system becomes the medium 
through which the user interacts with the remote en¬ 


vironment and ideally they are fooled into forgetting 
about the medium itself. If this is achieved, we say that 
the master-slave system is transparent. 

While bilateral master-slave systems have held the 
biggest promise for telepresence and intuitive opera¬ 
tions, they have also posed some of the largest stabil¬ 
ity and control problems. Especially considering force 
feedback from sensors at the remote site, these systems 
close multiple interwoven feedback loops and have to 
deal with large uncertainties in the environment. They 
have received heavy research attention and will there¬ 
fore be a repeated focus in some of our following 
discussions. 


43.2 Telerobotic Systems and Applications 


Telerobotic systems, like most robotic devices, are typ¬ 
ically designed for specific tasks and according to 
explicit requirements. As such, many unique systems 
have evolved, of which we present an overview for dif¬ 
ferent applications. We begin with a short historical 
perspective, then describe different applications with 
various robot designs and user interfaces. 

43.2.1 Historical Perspective 

Teleoperation enjoys a rich history and dates back to nu¬ 
clear research by Raymond C. Goertz in the 1940s and 
1950s. In particular, he created systems for humans to 
handle radioactive material from behind shielded walls. 
The first systems were electrical, controlled by an array 
of on-off switches to activate various motors and move 
various axes [43.3]. Without any feel, these manipu¬ 
lators were slow and somewhat awkward to operate, 
leading Goertz to build pairs of mechanically linked 
master-slave robots [43.3, 4]. Connected by gears, link¬ 
ages, and cables, these systems allowed the operator to 
use natural hand motions and transmitted forces and vi¬ 
brations through the connecting structure (Fig. 43.2). 
Unfortunately they limited the distance between the 
operator and environment and required the use of kine¬ 
matically identical devices. Goertz quickly recognized 
the value of electrically coupled manipulators and laid 
the foundations of modem telerobotics and bilateral 
force-reflecting positional servos [43.5]. 

At the beginning of the 1960s the effects of time 
delay on teleoperation started to become a topic of re¬ 
search [43.6,7]. To cope with this problem the concept 
of supervisory control was introduced [43.2] and in¬ 
spired the next years of development. In the late 1980s 
and early 1990s theoretical control came into play with 
Lyapunov-based analysis and network theory [43.8— 


13]. Using these new methods, bilateral control of 
telerobotic systems became the vital research area it is 
today (Sect. 43.4). The growth of the Internet and its 
use as a communication medium has further fueled this 
trend, adding the challenges of nondeterministic time 
delay. 

On the hardware side, the Central Research Labora¬ 
tory model M2 of 1982 was the first telerobotic system 
which realized force feedback while separating master 
and slave electronics. It was developed together with the 
Oak Ridge National Laboratory and was used for some 
time for a wide range of demonstration tasks includ¬ 
ing military, space or nuclear applications. The National 
Aeronautics and Space Administration (NASA) tested 
the M2 system to simulate the ACCESS space truss 
assembly with excellent results (Fig. 43.3). The ad- 



Fig. 43.2 Raymond C. Goertz used electrical and mechan¬ 
ical teleoperators in the early 1950s to handle radioactive 
material (courtesy Argonne National Labs) 
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Fig. 43.3 The telerobotic system CRL Model M2 is used 
to verify the assembly of space truss structures (1982) 
(courtesy Oak Ridge National Laboratory) 


vanced servomanipulator (ASM) was developed from 
the M2 to improve the remote maintainability of ma¬ 
nipulators and intended as a foundation for telerobotic 
systems [43.14], 

Also driven by the nuclear application, bilateral 
servomanipulators for teleoperation were developed in 
France at the Commission de Energie Atomique (CEA) 
by Vertut and Coiffet [43.15]. With the MA 23 they 
demonstrated telerobotic operation including computer- 
assisted functionalities to improve the operator’s perfor¬ 
mance [43.16]. The assistance included software jigs 
and fixtures or virtual walls and restrictions [43.17] 
(Sect. 43.3.2). 

For space applications a dual-arm force reflecting 
telerobotic system was developed by Bejczy at the Jet 
Propulsion Laboratory (JPL) [43.18] and 
This system was the first use of kinematically and dy¬ 
namically different master and slave robots. It required 
control in the Cartesian space coordinates of the opera¬ 
tor’s hand and slave robot’s tool. Figure 43.4 shows the 
master control station with its two back-drivable hand 
controllers. The system was used to simulate teleopera¬ 
tion in space. 

In 1993 the first telerobotic system was flown in 
space with the German Spacelab Mission D2 on board 



Fig. 43.4 JPL ATOP control station (early 1980s) (JPL 
No. 19902Ac, courtesy NASA/JPL-CALTECH) 




Fig. 43.5 ROTEX, the first remotely controlled robot in 
space (1993). Telerobot in space and ground operator sta¬ 
tion (courtesy German Aerospace Center, DLR) 


the Space Shuttle Columbia. The robot technology ex¬ 
periment (ROTEX), shown in Fig. 43.5, demonstrated 
remote control of a space robot by means of local 
sensory feedback, predictive displays, and teleopera¬ 
tion [43.19]. In this experiment the round trip delay 
was 6—7 s, such that it was not feasible to include force 
feedback into the control loop. 

Throughout the 1980s and 1990s, as nuclear power 
activities began to decline, interests expanded into new 
areas including medicine and undersea operations. Ef- 






















Telerobotics I 43.2 Telerobotic Systems and Applications 1089 



Voice over IP 


Voice over IP 


Robot command station 


High-speed 
: iberoptic service 
10 Mb/s 


Robot 


Broadcast-quality 
video monitor 


Endoscopic camera 


Computer 


Videoconference 


Videoconference 


“5 

J 

amputcr 

> 

5 


Fig. 43.6 Operation Lindberg. 
The first transcontinental teler¬ 
obotic surgery (2001) (courtesy 
M. Ghodoussi) 


forts were accelerated by the availability of increasing 
computer power as well as the introduction of novel 
hand controllers, e.g., the PHANToM device [43.20], 
popularized by haptic applications in virtual reality 
(Chap. 41). 

Simultaneously, surgery was seeing the trend to¬ 
ward minimally invasive techniques, highlighted by 
the first laparoscopic cholecystectomy (removal of the 
gallbladder) in 1987. Several groups saw the potential 
for telerobotics and pursued telesurgical systems. Most 
noteworthy are the Telepresence Surgery System de¬ 
veloped at the Stanford Research Institute (now SRI 
International) in 1987 [43.21], the Laparoscopic As¬ 
sistant Robotic System (LARS) created at the IBM 
Watson Research Center [43.22], the teleoperated surgi¬ 
cal instrument Falcon designed at MIT (Massachusetts 
Institute of Technology) [43.23], and the Robot As- 




Fig. 43.7 Intuitive Surgical Inc. makes the da Vinci teler¬ 
obotic system, which is used in minimally invasive surgery 
(courtesy 2008 Intuitive Surgical, Inc.) 


sisted Microsurgery (RAMS) workstation developed at 
JPL [43.24], 

In 1995 Intuitive Surgical Inc. was founded to lever¬ 
age several of these concepts, leading to the da Vinci 
telesurgical system [43.25] and its introduction to mar¬ 
ket in 1999. Meanwhile Computer Motion started with 
a voice-controlled robot moving an endoscopic cam¬ 
era [43.26] and extended those capabilities into the 



Fig. 43.8 tEODor, a telerobotic system for disarming 
of explosives (courtesy telerob Gesellschaft fur Fem- 
hantierungstechnik mbH, Ostfildem, Germany) 
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Fig. 43.9 ROKVISS, a telerobotic system providing stereo vision 
and haptic feedback to the ground operator (courtesy German 
Aerospace Center, DLR) 


ZEUS system. In 2001 a surgeon in New York (USA) 
used a ZEUS system to perform the first transatlantic 
telesurgical laparoscopic cholecystectomy on a patient 
located in Strasbourg (France) [43.27], as depicted in 
Fig. 43.6. The system did not include force feedback, 
so the surgeon had to rely on visual feedback only. 

In this perspective we have given reference only to 
the systems that may be seen as milestones within the 
history of telerobotics. Other systems, which have been 
developed and added value to the research field, unfor¬ 
tunately could not be mentioned here. 

43.2.2 Applications 

Telerobotic systems have been motivated by issues 
of human safety in hazardous environments (e.g., nu¬ 
clear or chemical plants), the high cost of reaching 
remote environments (e.g., space), scale (e.g., power 
amplification or position scaling in micromanipulation 
or minimally invasive surgery), and many others. Not 
surprisingly, after their beginning in nuclear research, 
telerobotic systems have evolved to many fields of ap¬ 
plication. Nearly everywhere a robot is used, telerobotic 
systems can be found. The following are some of the 
more exciting uses. 

In minimally invasive surgery telerobots allow pro¬ 
cedures to be performed through small incisions, reduc¬ 
ing the trauma to the patient compared to traditional 
surgery [43.28]. The da Vinci system, made by Intu¬ 
itive Surgical Inc. [43.25] and shown in Fig. 43.7, is the 
only commercially available device at present. Other ef¬ 


forts, however, have included computer motion [43.26] 
and endoVia Medical [43.29] on the commercial side, 
as well as the University of Washington [43.30], lohns 
Hopkins University [43.31], the German Aerospace 
Center [43.32], and many others (l<s*iUli!*li«). 

Protecting the operator from having to reach into 
a hazardous environment, telerobotic systems are widely 
used in nuclear or chemical industry. Some systems have 
been developed for the maintenance of high-voltage 
electrical power lines, which can be safely repaired 
without service interruption by a human operator using 
a telerobotic system. Disarming of explosives is another 
important task. Many systems like the telerob explo¬ 
sive ordnance disposal and observation robot (tEODor) 
shown in Fig. 43.8 or PackBot, made by (Robot [43.33], 
are used by police and military to disarm mines and other 
explosives. Similar vehicles are remote controlled for 
search and rescue in disaster zones [43.34], 

Space robotics is a classic application, in which 
distance is the dominating barrier, as discussed in 
Chap. 55. The NASA rovers on Mars are a famous 
example. Due to the time delay of several minutes, 
the rovers are commanded using supervisory control, 
in which the human operator is defining the goal of 
a movement and the rover achieves the goal by local 
autonomy using sensory feedback directly [43.35]. 

In orbital robotics the German technology experi¬ 
ment ROKVISS (robot component verification on the 
international space station (ISS)) was the most ad¬ 
vanced telerobotic system [43.36]. Launched in 2004 
and operational through 2010, it was installed outside 
the Russian module of the international space station. It 
validated advanced robot components in the slave sys¬ 
tem, including torque sensors and stereo video cameras, 
in real space conditions. Using a direct communication 
link between the space station and the operator station 
at DLR (German Aerospace Center), the time delay 
was kept at about 20 ms allowing a bilateral control 
architecture with high-fidelity force feedback to the op¬ 
erator [43.37] (Fig. 43.9). This technology is leading to¬ 
ward robotic service satellites, called Robonauts, which 
can be controlled remotely from the ground to help real 
astronauts during extravehicular activities (EVA) or to 
perform repair and maintenance tasks [43.38]. 


43.3 Control Architectures 

Compared to plain robotic systems, in which a robot 
executes a motion or other program without further 
consultation of a user or operator, telerobotic systems 
provide information to and require commands from 
the user. Their control architectures can be described 
by the style and level of this connection, as shown in 


Fig. 43.10. Organized in a spectrum, the three main cat¬ 
egories are: 

• Direct control 

• Shared control 

• Supervisory control. 
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Fig. 43.10 Different concepts for 
telerobotic control architectures 


In practice, however, control architectures often in¬ 
clude parts of all strategies. 

Direct control implies no intelligence or autonomy 
in the system, so that all slave motion is directly con¬ 
trolled by the user via the master interface. This may 
incorporate sensory feedback to the user in a bilat¬ 
eral configuration. If the slave motion is controlled by 
a combination of direct user commands and local sen¬ 
sory feedback or autonomy, the architecture is denoted 
as shared control. It is similarly shared if user feedback 
is augmented from virtual reality or by other automatic 
aids. In supervisory control user commands and feed¬ 
back occur at a higher level. The connection is more 
loose and the slave has to rely on stronger local auton¬ 
omy to refine and execute tasks. The following explains 
the architectures in reverse order, leading to a detailed 
treatment of direct and bilateral control in Sect. 43.3.3, 
which introduces the basic ideas for Sect. 43.4. 

43.3.1 Supervisory Control 

Supervisory control, introduced by Ferell and Sheridan 
in 1967 [43.2], is derived from the analog of supervis¬ 
ing a human subordinate staff member. The supervisor 
gives high-level directives to and receives summary 
information from, in this case, the robot. Sheridan de¬ 
scribes this approach in comparison with manual and 
automatic robot control [43.39]: 

Human operators are intermittently programming 
and continually receiving information from a com¬ 
puter that itself closes an autonomous control loop 
through artificial effectors and sensors. 

In general, supervisory control techniques will al¬ 
low more and more autonomy and intelligence to shift 
to the robot system. Today simple autonomous control 


loops may be closed at the remote site, with only state 
and model information being transmitted to the opera¬ 
tor site. The operator supervises the telerobotic system 
closely and decides exactly how to act and what to do. 
A specific implementation of supervisory control is the 
telesensor programming approach, which is presented 
hereafter. See also for another implemen¬ 

tation of supervisory control for space operation. 

Telesensor Programming 

Developed for space applications with large communi¬ 
cation delays, the telesensor programming (TSP) ap¬ 
proach has been characterized as a task-level-oriented 
programming technique and sensor-based teaching by 
showing [43.40,41]. In essence, operators interact with 
a complex simulation of the robot and remote environ¬ 
ment, in which they can test and adjust tasks. The tasks, 
consisting of robot and environment signals and config¬ 
uration parameters, are then uploaded to the remote site. 
The approach presumes that the sensor systems provide 
sufficient information about the actual environment so 
that the tasks can be executed autonomously. Specifica¬ 
tions and high-level planning remain the responsibility 
of the human operator. 

Figure 43.11 shows the structure of a TSP imple¬ 
mentation, consisting of two control loops working in 
parallel. One loop controls the real (remote) system, 
which contains internal feedback for local autonomy. 
The other loop establishes a simulation environment 
which is structurally equivalent to the real system, with 
a few exceptions. Most importantly, any signal delay 
which may result from communication to the remote 
system, in particular in space applications, is not du¬ 
plicated in the simulation. This makes the simulation 
predictive with respect to the real system. A second 
exception is the display of internal variables in the sim- 
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Fig. 43.11 The concept of telesensor programming as demonstrated during the ROTEX mission 


ulation, which cannot be observed (measured) in the 
real system. This gives the operator or task planner 
more insight into what is happening or may happen in 
the system in response to commands. Communication 
between the two loops occurs via a common model data 
base which delivers a priori knowledge for execution on 
the remote system and a posteriori knowledge for model 
updating in the simulated world. 

Unique tools are necessary to implement the func¬ 
tionality required for such a telerobotic control system. 
First a sophisticated simulation system has to be pro¬ 



Fig. 43.12 An example for the shared control concept in telerobotic 
surgery 


vided to emulate the real robot system. This includes the 
simulation of sensory perception within the real envi¬ 
ronment. Also, the operator needs an efficient interface 
to set up task descriptions, to configure the task control 
parameters, to decide what kind of sensors and control 
algorithms should be used, and to debug an entire job 
execution phase. 

For telerobotic systems with large time delays of 
a few seconds or more, e.g., in space and undersea 
applications, such a sensor-based task-directed pro¬ 
gramming approach has advantages. It is not feasible 
for human operators to handle the robot movements 
directly under delayed visual feedback. Only a predic¬ 
tive simulation allows the operator to telemanipulate 
the remote system [43.42]. In addition, the use of force 
reflecting hand controllers to feed back force signals 
from the simulated predicted world can improve the 
operator’s performance [43.43]. Finally, an interactive 
supervisory user interface makes it possible to config¬ 
ure the environmental and control parameters. 

43.3.2 Shared Control 

Shared control tries to combine the basic reliability and 
sense of presence achievable by direct control with the 
smarts and possible safety guarantees of autonomous 
control ([43.44,45] and This may occur 

in various forms. For example, the slave robot may need 


















































































Telerobotics I 43.3 Control Architectures 


to correct motion commands, regulate subsets of joints 
or subtasks, or overlay additional commands. 

With large communication delays, a human opera¬ 
tor may only be able to specify gross path commands, 
which the slave must fine-tune with local sensory in¬ 
formation [43.46]. We may also want the slave to as¬ 
sume control of subtasks, such as maintaining a grasp 
over long periods of time [43.47]. And in surgical ap¬ 
plications, shared control has been proposed to com¬ 
pensate for beating heart movements (Fig. 43.12). The 
sensed heart motion is overlaid on the user commands, 
so the surgeon can operate on a virtually stabilized pa¬ 
tient [43.48]. 

A special application of shared control is the use 
of virtual fixtures ([43.49-51] and l<s>M3HiES3i). Vir¬ 
tual elements, such as virtual surfaces, virtual velocity 
field, guide tubes, or other appropriate objects, are su¬ 
perimposed into the visual and/or haptic scene for the 
user. These fixtures can help the operator perform tasks 
by limiting movement into restricted regions and/or 
influencing movement along desired paths. Control is 
thus shared at the master site, taking advantage of pre¬ 
knowledge of the system or task to modify the user’s 
commands and/or to combine them with autonomously 
generated signals. 

Capitalizing on the accuracy of robotic systems 
while sharing control with the operator, telerobotic sys¬ 
tems with virtual fixtures can achieve safer, faster and 
more intuitive operation. Abbott et al. describe the ben¬ 
efits by comparison to the common physical fixture of 
a ruler [43.50]: 

A straight line drawn by a human with the help of 
a ruler is drawn faster and straighter than a line 
drawn freehand. Similarly, a [master] robot can ap¬ 
ply forces or positions to a human operator to help 
him or her draw a straight line. 

Based on the nature of the master robot and its con¬ 
troller, the virtual fixtures may apply corrective forces 
or constrain positions. In both cases, and in contrast to 
physical fixtures, the level and type of assistance can be 
programmed and varied. 

43.3.3 Direct and Bilateral Teleoperation 

To avoid difficulties in creating local autonomy, most 
telerobotic systems include some form of direct control: 
they allow the operator to specify the robot’s motions. 
This may involve commanding either position or ve¬ 
locity or acceleration. We begin our discussions with 
the later two options, which are generally implemented 
unilaterally without force feedback to the user. We then 
focus on position control, which is more suited to bilat¬ 
eral operation. We will assume a master-slave system, 


i. e., the user is holding a joystick or master mechanism 
serving as an input device. 

Unilateral Acceleration or Rate Control 
For underwater, airborne, or space applications, a slave 
robot may be a vehicle actuated by thrusters. Direct 
control thus requires the user to power the thrusters, 
which in turn accelerates the vehicle. For other ap¬ 
plications. the user may be required to command the 
rate or velocity of the vehicle or slave robot. In both 
scenarios, the input device is commonly a joystick, 
often spring centered, where the acceleration or rate 
commands are proportional to the joystick displace¬ 
ment. For six degree-of-freedom (DOF) applications, 
i. e., when the slave needs to be controlled in translation 
and orientation, a six-dimensional (6-D) space mouse 
can be used. Alternatively two joysticks may separately 
command translation and orientation. 

Acceleration and rate control are very attractive 
when the master and slave robots are fundamentally 
different, for example if the slave robot can reach an 
effectively unbounded workspace. Unfortunately, basic 
implementations can require considerable effort for the 
operator to reach and hold a given target location. As ex¬ 
pected, users can more accurately position a system un¬ 
der rate control than under acceleration control [43.52]. 
Indeed acceleration control necessitates users to regu¬ 
late a second-order system versus a first-order system 
for rate control. Assuming the slave has local position 
feedback available, a control system is often incorpo¬ 
rated locally, such that the user may specify position 
commands and is relieved from the dynamic control 
problem. We refer to Sect. 43.5.1 for some emerging de¬ 
velopments for bilateral control of mobile robots. 

Position Control and Kinematic Coupling 
Assuming that the slave is under position control, we 
can consider a kinematic coupling between master and 
slave, i. e., a mapping between master and slave posi¬ 
tions. In particular, we must remember that the master 
mechanism moves in the master workspace, while the 
slave robot moves in the slave workspace. The mapping 
connects these two spaces, which are nearly always 
somewhat different. 

Clutching and Offsets. Before discussing how the 
two robots are coupled, we must understand that they 
are not always coupled. For example, before the system 
is turned on, master and slave robots may, for whatever 
reason, be placed in some initial position/configuration. 
We have three options of how to engage the system: 

1. First autonomously move one or both robots so they 

come to the same position 
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2. Wait until someone (the user) externally moves one 
robot to match the location of the other, or 

3. Connect the two robots with some offset. 


Instead kinematically dissimilar robots are com¬ 
monly connected at their tips. If x is a robot’s tip 
position, we have 


Once connected, most systems also allow a tempo¬ 
rary disconnection between the two sites. The reason is 
twofold: to allow the user to rest without affecting the 
slave state and to allow a shift between the two robots. 
The later is most important if the workspaces of both 
robots do not perfectly overlap. This is much like pick¬ 
ing up your mouse off your mouse pad to reposition 
without moving the cursor. In telerobotics the process 
is called clutching or sometimes indexing. If clutching 
is allowed, or both robots are not constrained to start 
at the same location, the system must allow for offsets 
between the two robots. 

When clutched or disconnected, most systems hold 
the slave at rest or allow it to float in response to en¬ 
vironment forces. It is also possible for the slave to 
retain its preclutching momentum and continue mov¬ 
ing, similar to kinetic scrolling popularized in smart¬ 
phones [43.53]. 

Kinematically Similar Mechanisms. The simplest 
scenario involves a master and slave mechanism that are 
kinematically equivalent if not entirely identical. In this 
case, the two robots can be connected at a joint level. 
With q denoting joint values and subscripts “m” refer¬ 
ring to the master, “s” to the slave, “offset” to a shared 
offset, and “d” to a desired value, we can write 

q sd — q m T (/offset > 

^Imd — (7s ^offset • (43.1) 


Wd — Tti T -^offset > 

■^md —*s -^offset • (43.3) 

If orientations are also connected, with R describing 
a rotation matrix, we have 

Rsd =: R|n Rofiset , 

Rmd = RsR 0 ff se t 1 (43.4) 

where the orientational offset is defined as slave relative 
to master 

Roffset = R^Rs • (43.5) 

Again velocities and angular velocities may be con¬ 
nected if needed and do not require offsets. 

Finally note that most telerobotic systems use 
a video camera at the remote site and a monitor at 
the local site. To make the connection appear natural, 
the slave position and orientation should be measured 
relative to the camera, while the master position and ori¬ 
entation should be measured relative to the user’s view. 

Scaling and Workspace Mapping. Kinematically 

dissimilar master-slave robots are commonly also of 

different size. This means not only do they require 
clutching to fully map one workspace to another, but 
they often necessitate motion scaling. And so (43.3) be¬ 
comes 


At the instance the two robots are to be connected 
or reconnected, the offset is computed as 


(7offset — (7s 4m ■ 


(43.2) 


Tsd — FAti T ^offset » 

_ (*s -^offset) 

■And — - 

/X 


(43.6) 


Most kinematically similar master-slave systems have 
the same workspace at both sites and do not allow 
clutching. By construction the offset is then always 
zero. 

Depending on the controller architecture, the joint 
velocities may be similarly related, taking derivatives 
of (43.1). An offset in velocities is not necessary. 

Kinematically Dissimilar Mechanisms. In many 
cases, the master and slave robots differ. Consider that 
the master is connected to the human user and thus 
should be designed accordingly. Meanwhile the slave 
works in some environment and may have a very differ¬ 
ent joint configuration and different number of joints. 
As a result, connecting the robots joint by joint may not 
be feasible or appropriate. 


The orientation, however, typically should not be 
scaled. The scale yt may be set to either map the two 
workspaces as best possible, or to provide the most 
comfort to the user. 

If force feedback is provided, as described below, an 
equivalent force scale may be desired. This will prevent 
distortion of the remote environmental conditions, such 
as stiffness or damping, by the scaling. In addition to 
the motion and force scalings, it is also possible to di¬ 
rectly achieve power scaling between master and slave 
systems [43.51]. 

Beyond linear scaling, several research efforts have 
created nonlinear or time-varying mappings, which de¬ 
form the workspaces. These may effectively change the 
scale in the proximity of objects [43.54] or drift the off¬ 
set to best utilize the master workspace [43.55]. 
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Local Position and Advanced Control. By construc¬ 
tion we are now assuming that the slave follows a posi¬ 
tion command. This necessitates a local slave controller 
to regulate its position. In particular for kinematically 
dissimilar mechanisms, this will be a Cartesian tip po¬ 
sition controller (Chap. 7). 

If the slave robot has redundancies or possesses 
a large number of DOFs, these may be controlled either 


automatically to optimize some criterion or manually 
with additional user commands. Indeed some emerging 
applications are coordinating multiple users to con¬ 
trol such complex systems. This is particularly relevant 
when the kinematic dissimilarity becomes extreme and 
has received considerable research attention. We refer 
here to Chap. 11 and especially Sect. 43.5 for appropri¬ 
ate techniques and new developments. 


43.4 Bilateral Control and Force Feedback 


In pursuit of telepresence and to increase task perfor¬ 
mance, many master-slave systems incorporate force 
feedback. That is, the slave robot doubles as a sensor 
and the master functions as a display device, so that the 
system provides both forward and feedback pathways 
from the user to the environment and back. Figure 43.13 
depicts the common architecture viewed as a chain of 
elements from the user to the environment. 

The bilateral nature of this setup makes the control 
architecture particularly challenging: multiple feedback 
loops form and even without environment contact or 
user intervention, the two robots form an internal closed 
loop. The communications between the two sites often 
inserts delays into the system and this loop, so that sta¬ 
bility of the system can be a challenging issue [43.56]. 

To present force information without stability prob¬ 
lems, it is possible to use alternate displays, such as 
audio or tactile devices [43.57]. Meanwhile, the com¬ 
bination of vibrotactile methods with explicit force 
feedback can increase high-frequency sensations and 
provide benefits to the user [43.58]. Tactile shape sens¬ 
ing and display also extends the force information 
presented to the user [43.59]. 

In the following we discuss explicit force feedback. 
We first examine the basic architectures before dis¬ 
cussing stability and some advanced techniques. 

43.4.1 Position/Force Control 

Two basic architectures couple the master and slave 
robots: position-position and position-force. We as¬ 
sume that the robot tips are to be connected by the 
equations of Sect. 43.3.3 and give the control laws for 


translation. Control of orientation or joint motions fol¬ 
lows equivalent patterns. 

Position-Position Architecture 
In the simplest case, both robots are instructed to track 
each other. Both sites implement a tracking controller, 
often a proportional-derivative (PD) controller, to fulfill 
these commands, 

A'm — A m (.V n -V m( |) B„ (im Find) . 

Fs — F s (x s -^sd) F & (jc s -Vsd) • (43.7) 

If the position and velocity gains are the same 
(Aj n = K s = K, B m = B s = B), then the two forces are 
the same and the system effectively provides force feed¬ 
back. This may also be interpreted as a spring and 
damper between the tips of each robot, as illustrated in 
Fig. 43.14. If the two robots are substantially different 
and require different position and velocity gains, some 
suitable force, position or power scalings, as explained 
in Sect. 43.3.3, may be utilized. 

Note we have assumed the slave is under impedance 
control and back-drivable. If the slave is admittance 
controlled, i. e., it accepts position commands directly, 
the second part of (43.7) is unnecessary. 

Also note that by construction the user feels the 
slave’s controller forces, which include forces asso¬ 
ciated with the spring-damper and slave inertia in 
addition to environment forces. Indeed while moving 
without contact, the user will feel the inertial and other 
dynamic forces needed to move the slave. Furthermore, 
if the slave is not back-drivable, i. e., does not easily 
move under environment forces, the environment force 



Human Master Master _ . Slave 

operator manipulator controller Communications controller 


Telerobot 


Environ¬ 

ment 


Fig. 43.13 A typical bilateral tele¬ 
operator can beviewed as a chain of 
elements reaching from user to envi¬ 
ronment (CPU - central processing 
unit) 
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Fig. 43.14 A position-position architecture effectively 
creates a spring and damper between the two robots 


may be entirely hidden from the user. Naturally this 
defeats the purpose of force feedback. In these cases, 
a local force control system may be used to render the 
slave back-drivable. Alternatively, a position-force ar¬ 
chitecture may be selected. 

Position-Force Architecture 
In the above position-position architecture, the user 
was effectively presented with the slave’s controller 
force. While this is very stable, it also means the 
user feels the friction and inertia in the slave robot, 
which the controller is actively driving to overcome. In 
many scenarios this is undesirable. To avoid the issue, 
position-force architectures place a force sensor at the 
tip of the slave robot and feedback the force from there. 
That is, the system is controlled by 

— T’sensor 1 

F s — K s (x, x s d) B s (x s Xsd) . (43.8) 

This allows the user to only feel the external forces 
acting between the slave and the environment and 
presents a more clear sense of the environment. How¬ 
ever, this architecture is less stable: the control loop 
passes from master motion to slave motion to envi¬ 
ronment forces back to master forces. There may be 
some lag in the slave’s motion tracking not to mention 
any delay in communications. Meanwhile the loop gain 
can be very high: a small motion command can turn 
into a large force if the slave is pressing against a stiff 
environment. In combination, stability may be compro¬ 
mised in stiff contact and many systems exhibit contact 
instability in these cases. 

43.4.2 Passivity and Stability 

The two basic architectures presented in Sect. 43.4.1 
clearly illustrate one of the basic tradeoffs and chal¬ 


lenges in force feedback: stability versus performance. 
Stability issues arise because any models of the system 
depend on the environment as well as the user. Both 
these elements are difficult to capture and, if we assume 
we want to explore unknown environments, impossible 
to predict. This issue makes a stability analysis very dif¬ 
ficult. A common tool that avoids some of these issues 
is the concept of passivity. Although passivity provides 
only a sufficient (not a necessary) condition for stability, 
it incorporates the environmental uncertainly very well. 

Passivity is an intuitive tool that examines the en¬ 
ergy flows in a system and makes stability assertions if 
energy is dissipated instead of generated. Three rules 
are of importance here. First, a system is passive if 
and only if it cannot produce energy. That is the out¬ 
put energy from the system is limited by the initial and 
accumulated energy in the system. Second, two passive 
systems can be combined to form a new passive system. 
Third, the feedback connection of two passive systems 
is stable. 

In the case of telerobotics, we generally assume that 
the slave robot will only interact with passive environ¬ 
ments, that is, that the environments do not contain 
active motors or the like. Without the human opera¬ 
tor, stability can therefore be assured if the system is 
also passive, without needing an explicit environment 
model. 

On the master side the operator closes a loop and 
has to be considered in the stability analysis. In gen¬ 
eral, the master robot will be held by the user’s hand 
and arm. A variety of models and parameters describe 
the human arm dynamics, mainly in the form of a mass- 
damper-spring system. In [43.60] we find a summary 
of model parameters used by different authors. For an 
impedance-controlled haptic interface, found in many 
systems, the operator adds some physical damping and 
a light touch (Fhuman ^ 0) is often one of the more chal¬ 
lenging scenarios [43.61,62]. As such some analyses 
ignore the human operator entirely. But even in gen¬ 
eral, humans seem to achieve stable interactions with 
all passive systems and passivity is usually considered 
sufficient for stable human-telerobot interactions. 

To apply passivity, we take the system originally de¬ 
picted in Fig. 43.13 and describe it as two-port elements 
in Fig. 43.15. Each port describes the energy flow be¬ 
tween subsystems, where power is the product of a pair 



Fig. 43.15 A teleoperator can be analyzed as a chain of-two port elements connecting the one-port operator to the one- 
port environment 
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of collocated physical variables. We can choose a sign 
convention, for example declaring positive power to 
flow toward the remote environment. Then at the first 
boundary, the positive power flow is the product of mas¬ 
ter velocity x m times applied (human) force inhuman 

Fleft —-i* m ^Munnan • (43.9) 

Meanwhile at the last boundary, the positive power flow 
is the product of the slave velocity i s times the environ¬ 
ment force F em (which ultimately opposes the human 
force) 

bright = -^s ^env • (43.10) 


general stability test for telerobotic systems or compo¬ 
nents [43.63]. 

Passive controllers are also limited as they can¬ 
not hide the dynamics of the slave robot. In the 
above position-position architecture, the user will feel 
the forces associated with the slave inertia. In con¬ 
trast the position-force architecture hides the slave 
inertia and friction from the user. As such, when 
the user inserts kinetic energy into the master with¬ 
out feeling any resistance, the system itself creates 
and injects the kinetic energy for the slave. This vi¬ 
olates passivity and provides another insight as to 
why the architecture suffers from potential stability 
problems. 


Therefore the entire telerobotic system is passive if 

t t 

J P input d/ = j (P left P right) tit 
0 0 

t 

— j (■^'nr^'human W F env j d t 


> /-store (0) 


(43.11) 


Obviously, the ideal teleoperator (i m = ir s , F| lumi[ n = 
Fem) is passive. These definitions can also be gener¬ 
alized to six dimensional twists and wrenches. 

To simplify the analysis, we can examine the pas¬ 
sivity of each two-port element or subsystem and then 
deduce the overall passivity. The master and slave 
robots are mechanical elements and hence passive. The 
controllers of a position-position architecture mimic 
a spring and damper, which are also passive ele¬ 
ments. So without delay and ignoring discretization, 
quantization, and other unmodelled effects, a simple 
position-position architecture is passive. The following 
sections will address some of these limitations and thus 
create controllers that are passive under more circum¬ 
stances. 

While powerful to handle uncertainty, passivity 
can be overly conservative. Many controllers are over¬ 
damped if every subsystem is passive. In contrast, the 
combination of an active and a passive subsystem may 
be passive and stable and show less dissipation. This is 
particularly true for the cascaded arrangement of two- 
port elements in the telerobotic system of Fig. 43.15. 
From network theory, the Llewellyn criterion speci¬ 
fies when a possibly active two-port connected with 
any passive one-port becomes passive. This two-port is 
then labeled unconditionally stable, as it will be sta¬ 
ble in connection to any two passive one-ports. The 
Llewellyn criterion may hence be used as a more 


43.4.3 Transparency and Multichannel 
Feedback 

Both basic architectures can be captured by the 
general teleoperator control system described by 
Lawrence [43.13], and later expanded by Hashtrudi- 
Zaad and Salcudean [43.63] and shown in Fig. 43.16. 
Ideally a system will equalize both the operator and 
environment forces as well as the master and slave mo¬ 
tions. Therefore, it is desirable to measure both force 
and velocity (from which position may be integrated or 
vice versa) at both sites. With this complete informa¬ 
tion, the slave may, for example, start moving as soon 



Fig. 43.16 In general, a controller will use both position 


and force information from both master and slave robot 
(after [43.63], adapted from [43.13]) 
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as the user applies a force to the master, even before the 
master itself has moved. 

Following these concepts derived in [43.13], we can 
examine the relationships between velocity and force, 
in the form of impedances and admittances. Note we 
do this in a single degree of freedom, assuming that all 
degrees of freedom may be treated independently. The 
environment will exhibit some impedance Z e (s ) that is 
not known in advance and relates the environment force 
to the slave’s velocity 

F e (s) = Z e (s)v s (s) . (43.12) 

If we describe the teleoperator in whole as a two-port 
with a hybrid matrix formulation 

(Fh(s) \ = ( H n (s ) H n (s)\ ( v s (s) \ 

\H 2 i (s) H 21 {s)) \-F e (s)J ’ ' ■ ' 


and admittance matrices, the system can tolerate de¬ 
lays [43.75]. Scattering matrices relate the sum of 
velocity and force to their difference, so that passivity 
becomes a condition on the system gain, which is unaf¬ 
fected by the delay. 

43.4.5 Wave Variables 

Building on the realization that delay communications 
can be active and that wave phenomena circumvent the 
issue, wave variables provide an encoding scheme that 
is tolerant of delay [43.76]. Consider the power flow¬ 
ing through the system and separate the power moving 
forward and returning. 

P — X F — —U U V V — P f orwar d Rretum , 

(43.15) 


then the user will perceive the impedance 


Z,o(s) 


F h (s) 

V m (s ) 


(Hn-H 12 Z e )(H 2 l -H 22 Z e r l ■ 

(43.14) 


Transparency describes how close the user’s per¬ 
ceived impedance comes to recreating the true environ¬ 
ment impedance. 

For a detailed treatment of passivity in telerobotics, 
impedance and admittance interpretations and designs, 
and transparency, we refer to some of the seminal works 
in [43.11-13,63-66], 

43.4.4 Time Delay and Scattering Theory 


When delays occur in the communications between the 
local and remote site, even position-position architec¬ 
tures can suffer from serious instabilities [43.67,68]. 
This can be traced to the communications block in 
Fig. 43.15, where the power entering the left side and 
exiting the right side do not add up. Rather energy may 
be generated inside the block, which feeds the instabil¬ 
ity [43.9]. 

Several approaches to operate under delay have 
been studied [43.69], in particular shared compli¬ 
ant control [43.70] and the addition of local force 
loops [43.71]. The use of the Internet for communica¬ 
tion, adding variability to the delay, is also an area of 
interest [43.72,73]. This further evokes issues of data 
reduction [43.74], 

Here we note that natural wave phenomena are 
bilateral passive elements that tolerate delay. If the 
control system is described in the frequency domain 
and scattering matrices are used in place of impedance 


where the forward and returning power are by con¬ 
struction nonnegative. This leads to the definition of the 
wave variables 


bx + F bx — F 

u = —7=- , v = —, 

V2 b V2b 


(43.16) 


where u is the forward-moving and v the returning 
wave. 

If velocity and force signals are encoded into wave 
variables, transmitted across the delay, and decoded at 
the far site, the system remains passive regardless of de¬ 
lay. In fact, in the wave domain, passivity corresponds 
to a wave gain of less than or equal to unity. No require¬ 
ments are placed on phase and so lag does not destroy 
stability. 

The wave impedance b relates velocity to force and 
provides a tuning knob to the operator. Large b val¬ 
ues mean the system increases force feedback levels 
at the cost of feeling high inertial forces. Small values 
of b lower any unwanted sensations, making it easy to 
move quickly, but also lower the desirable environment 
forces. Ideally the operator would lower b when there 
is no risk of contact and raise b when contact is immi¬ 
nent. The concept of scattering has also been extended 
to a coordinate free context [43.77], 

Recent developments are incorporating both posi¬ 
tion-position and position-force architectures within 
the wave frame work, so the resulting systems are stable 
with any environment, stable with any delay, yet main¬ 
tain the feedback of high-frequency forces that help the 
operator identify happenings at the remote site [43.78], 
To improve performance and assist the operator, es¬ 
pecially across the Internet, predictors may also be 
incorporated [43.79]. 
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43.4.6 Teleoperation 

with Lossy Communication 

The Internet provides an affordable and ubiquitously 
accessible communication medium. However, it can 
also introduce nondeterministic effects and lossy con¬ 
nections due to time-varying delays, packet losses, and 
packet reordering. How to achieve passive bilateral tele¬ 
operation over such lossy communication network has 
been an active research topic in telerobotics. 

Many telerobotic systems have relied on the 
position-position architecture (43.7) with extra damp¬ 
ing injection. This leads to the following proportional- 
derivative (PD) control law, with a simple structure and 
explicit position feedback, 

F m — B d x m (t) B[.v m (r) x s (t h)] 
--K[* m (f)-X s (f-Ti)] 

F s = -B d x s (t) -B[x s (t)-x m (t- x 2 )j 

— K[x s {t) — * m (f — T 2 )] , (43.17) 

where B d ,K,B are the stabilizing absolute damping and 
the PD control gains, and ti , T 2 > 0 are the communi¬ 
cation delays from slave to master and from master to 
slave, respectively. 

The usage of this controller across lossy commu¬ 
nication was yet hampered (or at least reserved) due 
to the lack of theoretical guarantees for its passivity 
and stability. Anecdotes say that with sufficiently large 
damping and small delays and PD gains, the closed- 
loop system remains stable. This anecdotal observation 
was justified in [43.80] for the case of constant delay, 
that is, the PD-like controller (43.17) is passive if the 
following condition is met, 

Ti + t2 

B d >^-^K, (43.18) 

where (t) + t 2)/2 is the upper-bound of the round-trip 
delay, which can typically be estimated easily. Without 
human or environment forcing, master and slave posi¬ 
tions will also converge to each other. This result was 
extended in [43.81] for the case of time-varying delay. 
Passivity of the PD-like controller (43.17) is guaranteed 
if 

B d > --- K and |r,(f)| < 1 , (43.19) 

where the second condition means the delay r,(t) nei¬ 
ther grows nor decreases faster than time t. We refer 
to [43.81] for cases with asymmetric controller gains 
and to [43.82, 83] for extensions to general digital lossy 
communication networks. 


With a fixed structure, the PD controller (43.17) 
has to be tuned to the worst-case conditions accord¬ 
ing to (43.18) and (43.19). It may thus exhibit drastic 
performance degradation for severely variable commu¬ 
nications. To overcome such fixed structure limitation, 
several passivity-enforcing flexible control techniques 
have been recently proposed. 

The technique of passivity-observer/passivity-con- 
troller (PO/PC) was originally devised for haptic device 
control [43.84] and has been extended for the bilateral 
teleoperation with digital network with time-varying 
delay [43.85-87]. Each PO does real-time bookkeep¬ 
ing of energy flows at the master and slave sites. The 
PC is activated to dissipate energy whenever a passivity 
violation is detected. The passive set-position modula¬ 
tion (PSPM) framework was proposed in [43.88], where 
the desired set-position signal received from a general 
digital lossy communication network is real-time mod¬ 
ulated as close to the original set-position signal as pos¬ 
sible, yet, only to the extent permissible by the available 
energy in the system. Passification of a desired con¬ 
trol force utilizing the device’s physical damping was 
addressed in [43.89] under the name of energy bound¬ 
ing algorithm (EBA). The idea of modulating a control 
signal or action under the passivity constraint was also 
adopted in the two-layer approach [43.90], where the 
transparency-layer is designed for best performance, 
while the passivity-layer superimposes constraints to 
enforce passivity. 

Overall, the PO/PC approach may be considered 
as corrective (i. e., detect violation of passivity first 
and then apply passifying action) whereas the PSPM, 
EBA and the two-layer approaches are preventive (i. e., 
modulate/bound control action to prevent passivity vi¬ 
olation). Interestingly, all the PO/PC, PSPM, EBA and 
two-layer approaches share some common characteris¬ 
tics: 1) they transmit energy packets along with other 
information over the communication network to replen¬ 
ish energy levels and enable useful work; and 2) each of 
these approaches activates their passifying action only 
when necessary, thus, can significantly improve con¬ 
trol performance compared to fixed-structure teleopera¬ 
tion controllers, while also robustly enforcing passivity 
against a variety of communication imperfectness. 

43.4.7 Port-Based Approaches 

From Sect. 43.4.2 we know that energy flows provide 
a great description of telerobotic systems that physi¬ 
cally interact with unknown environments and a human 
user. Indeed all physical interaction dynamics are fun¬ 
damentally bound to energy exchanges. Passivity is 
a well suited analysis tool and assures stability if the 
system energy is bounded. All possible instabilities 
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Energy balance: Hj(t) = H m (t) + H c (t) + H s (t) 
Passivity condition: H T (t) < P m (t) + P s (t ) 


Master robot 


Master 

controller 


Master 

channel 


Slave 

controller 


Slave 

robot 


HJt) 


H c (f) 


H s (t) 


PJf) 


P s (t) 


Fig. 43.17 Energy balance of 
the telemanipulation chain 
(after [43.90]) 


can be traced to unsupervised energy injections via the 
actuators. In Sect. 43.4.6 we also saw methods that 
explicitly monitor energy flows. Here we describe port- 
based approaches designed explicitly around energy 
exchanges. This can be advantageous to handle non¬ 
linear system dynamics, discretization, time-varying 
delays, and other issues. 

The concept of power ports and energy flows 
(Sect. 43.4.2) allows a precise analytical formulation 
of physical systems. This approach, which stems from 
concepts used in bond-graphs, enables a pure energy 
based analysis of complex nonlinear physical sys¬ 
tems [43.91]. It has also provided new perspectives on 
modeling and robot control [43.92]. 

To handle sampling issues and prevent instabilities 
due to limited sampling rates, consider an energy- 
consistent discretization [43.93]. Rather than measuring 
power flow at discrete intervals, consider the energy 
flow that occurs continuously over the entire sampling 
interval. An exact measure of the energy transfer be¬ 
tween sample time kT and the following sample time 
(k+ 1) T’ is 


AE k = 


(i+l)7 

J r(t)q(t)dt. 

kT 


(43.20) 


Assume an electric motor with an ideal current ampli¬ 
fier and without any commutation effects is generating 
the torque as commanded by a zero order hold (ZOH) 
digital to analog converter. Further assume the position 
sensor is collocated and synchronized to the ZOH tran¬ 
sitions. The torque may then be considered constant 
during the interval, so that 


(H-i)r 


AE,= 


J ?k,k+\q(t)dt. 

kT 


(43.21) 


Taking the torque out of the integral, we realize that 
AE k = x k ,k+i[q(k+ l)-q(k)\ . (43.22) 


This simple result has far reaching consequences for 
interfacing the digital and physical world and can 


be easily calculated even if the sampling time varies 
\aa mmiMlU And some of the assumptions may be re¬ 
laxed with suitable adaptation. Using this more precise 
measurement of energy flow leads to more consistent 
energy book-keeping. 

As it is shown in Fig. 43.17, we can now track 
energy in the digital world, associating individual avail¬ 
able energy reservoirs H m and H s with the master and 
slave controllers. The communication channel transmits 
both data and energy packets (EPs), where EPs contain 
only information about an energy quanta. An EP is only 
sent if the transmitter has sufficient available energy, 
which is then decreased by the quanta. An arriving EP 
injects the quanta into the receiver’s available energy. 
In this way the total virtual energy in the system will 
never increase. If the communication protocol allows 
an EP to be lost, this will remove energy similar to dis¬ 
sipation. This process is independent of any constant or 
variable delay. 

The port-based paradigm allows any nonlinear con¬ 
trol algorithm. But any applied control force will 
have an energetic consequence and will be allowed 
if and only if the associated available energy level 
is sufficiently high. Following the classification of 
Sect. 43.4.6, this paradigm is preventive. It prevents 
energy generation without needing to dissipate unex¬ 
pected energy appearances. 

Strategies are needed to address the exchange of 
energy between master and slave. A simple protocol 
introduced in [43.90] continually transmits EPs with 
a percentage of the locally stored energy. It can be 
shown that this will result in an equal distribution of 
energy between the master and slave controller. Also, 
if necessary, a small damper can be superimposed at 
the master side to extract energy from the human as 
needed. 

With the port-based paradigm, the data communi¬ 
cation (related to transparency) and the energy commu¬ 
nication (related to passivity) are split: controllers may 
be nonlinear, energy and data transmission may be in¬ 
dependent, and there is basically no restriction on the 
kind of controller which can be implemented. The fact 
that energy and data are separated gives the name to the 
two-layer approach [43.90]. 
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Historically, telerobotics research has focused on a con¬ 
ventional setup with two fixed-based robotic manipula¬ 
tors serving as the master and slave devices. Recently, 
there have been substantial efforts to extend the teler- 
obotic theories and frameworks to more unconventional 
scenarios. Here we summarize some recent results on 
these emerging applications. The summary is by no 
means exhaustive and, to be consistent with the chap¬ 
ter, focuses on controls aspects and providing a stable 
bilateral user interface. 

43.5.1 Telerobotics for Mobile Robots 

Mobile robots are useful slave devices if the task covers 
a large spatial area. Flying robots, in particular, can op¬ 
erate in three-dimensional space without being bound to 
the ground. For mobile robot teleoperation, force feed¬ 
back may be used to convey proprioceptive information 
of the slave robot (e.g., velocity), or haptic feedback of 
virtual (or real) objects in the remote environment. 

A key difference of mobile and flying robot teleop¬ 
eration compared to a conventional setup is kinematic 
dissimilarity [43.94]: the workspace of the master de¬ 
vice is bounded while the workspace of the slave robot 
is unbounded. This suggests to couple the master posi¬ 
tion to the slave velocity, as with rate-control described 
in Sect. 43.3.3. A direct coupling between master po¬ 
sition and slave velocity, however, cannot be addressed 
by the standard passivity framework (Sect. 43.4.2). The 
master position and the slave velocity possess differ¬ 
ent relative degrees with respect to the torque. One way 
to circumvent this difficulty is to utilize so called r- 
variable 

r := q + Xq . (43.23) 

That is, by utilizing inverse-dynamics similar to adap¬ 
tive control design [43.95, 96] or by injecting some PD- 
type local state feedback with redefined output r [43.97] 
(Fig. 43.18), it is possible to render the master device to 
be passive with this r-variable replacing the velocity q 
in its original passivity relation (43.9). This implies that 
we can couple the r-variable and the slave’s velocity 
passively, just like standard passivity-based controllers. 



Fig. 43.18 

Feedback r- 
passivation by 
using PD-type 
state feedback 


Meanwhile the r-variable contains the master position 
information. 

We can also achieve passive mobile robot tele¬ 
operation with time-varying delays by extending the 
port-based ideas presented in Sect. 43.4.7 with the con¬ 
cept of a virtual vehicle [43.98]. This virtual vehicle 
is a simulated slave system, evolving in a gravitation- 
free field and having a finite energy tank. Commands 
from the master are only allowed to transfer energy 
from the tank to accelerate the vehicle or to return en¬ 
ergy by decelerating the vehicle. This creates a closed 
energy system for the slave. A passive behavior can 
be achieved with a viscoelastic connection between the 
virtual and real vehicles. 

Mobile telerobotics often uses nonholonomically- 
constrained wheeled mobile robots (WMRs) with pure- 
rolling wheels. For slave WMRs, it is often possible 
(with some low-level control as stated below) to split 
motions (i. e., velocity directions) into those requir¬ 
ing a position-velocity coupling as stated above (e.g., 
forward velocity of WMR), and others that may be con¬ 
trolled by a standard position-position coupling (e.g., 
rotation angle of WMR) as explained in Sects. 43.4.1 
and 43.4.6. 

Other mobile robots, particularly quadrotor or 
ducted-fan type aerial robots (Chap. 36) or thrust- 
propelled autonomous aquatic vehicles (AUVs, 
Chap. 25), are under-actuated with fewer control vari¬ 
ables than degrees of freedom and also defy standard 
teleoperation techniques. To address under-actuation, 
abstraction of the slave robot has been utilized [43.98, 
99], that is, human users telecontrol a fully-actuated 
virtual system, assuming that its motion is adequately 
describing that of the real slave robot, while a certain 
low-level tracking control is employed to drive the 
under-actuated slave mobile robot to tightly follow this 
virtual system. This abstraction leads to a hierarchical 
control design, composed of: 1) a high-level teleoper¬ 
ation control layer between the virtual vehicle and the 
master device; and 2) a low-level tracking control layer, 
into which the issue of the slave’s under-actuation is 
confined. 

For the teleoperation layer, we may then use 
the conventional teleoperation techniques explained in 
Sect. 43.4 with the (r, uncoupling as explained above. 
We may also use the recently proposed control tech¬ 
niques explained in Sect. 43.4.6, which promise sharper 
haptic feedback with guaranteed stability against lossy 
communication. Presenting both the slave robot’s pro¬ 
prioceptive information and the presence of surround¬ 
ing objects for obstacle avoidance via the same hap¬ 
tic feedback channel, however, typically results in 
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a perceptual ambiguity [43.100]. A further perceptual 
modality (e.g., vision) is usually necessary to resolve 
this ambiguity. In some applications, the slave mobile 
robots are required to move around in but to not directly 
contact any physical environments. In these cases, vir¬ 
tual forces can be generated for any obstacles and the 
slave robot is interacting only with this precisely known 
virtual force field. This suggests it would be sufficient 
to enforce slave stability versus slave passivity ([43.99], 
|<s>M3HSEB and l<sf'IPH'If ), as a less conservative 
controller will likely give better system performance. 

43.5.2 Multilateral Telerobotics 

Many practical telerobotic tasks require dexterous, 
complicated, and large degree-of-freedom motions, 
e.g., in surgical training, rehabilitation, or exploration. 
For such tasks, we may utilize a team of multiple co¬ 
operative slave robots or a single slave robot possessing 
many degrees of freedom. The complexity involved in 
both cases may require multiple human operators to ad¬ 
equately control and coordinate all degrees of freedom. 
Following [43.102], we thus consider the following 
scenarios: 

1. Single-master multiple-slave (SMMS) systems 

2. Multiple-master multiple-slave (MMMS) systems 

3. Multiple-master single-slave (MMSS) systems 

4. Single-master single-slave (SMSS) systems, which 
constitute the conventional telerobotic setup. 


Here we introduce some recent results applicable to 
SMMS and MMSS systems. 

It is common for single-master multiple-slave 
(SMMS) systems to autonomously control simple sub¬ 
tasks among the slaves, e.g., maintaining a grasp, main¬ 
taining connectivity, or avoiding collisions (Fig. 43.19). 
In particular, in [43.101, 104, 105], passive decomposi¬ 
tion [43.106] is utilized to decompose the dynamics of 
multiple slave robots into their shape system, describ¬ 
ing the inter-slave formation aspects, and the locked 
system, abstracting their collective motion and centroid 
behavior. The locked system can be telecontrolled by 
a single human user, while an autonomous controller 
regulates the shape system to maintain the cooperative 
grasp of an object between the slaves. 

Another interesting development in SMMS teler¬ 
obotics is the combination with the frameworks of mul¬ 
tiagent cooperative control (i. e., consensus, flocking, 
synchronization, and other behaviors). For instance, 
in [43.103] and |4a>MM3!EB, a single human user 
directly telecontrols a single leader agent among the 
slaves, while the behavior of the other slaves is dictated 
by a leader-follower information graph (Fig. 43.20). 
A time-varying graph topology with arbitrary split/join 
operations among the slaves allows for reconfigura¬ 
tion, e.g., for navigation in cluttered environments. And 
the concept of virtual energy tanks, along with the 
port-Hamiltonian modeling and port-based approach 
(Sect. 43.4.7), is utilized to enforce passivity and sta¬ 
bility of the total system. 
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Fig. 43.19 SMMS telerobotic control of multiple slave robots (after [43.101]) 
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Fig. 43.20 SMMS telerobotic control architecture with possibly time-varying leader-follower information topology (af¬ 
ter [43.103]) 



Fig. 43.21 SMMS control architecture for multiple un¬ 
manned aerial vehicles (UAVs; after [43.99]), where a sin¬ 
gle user telecontrols some of the virtual systems (via 
u\,u\) while telesensing the state of some of real UAVs 
(i. e., yi, V3), which are low-level controlled to follow their 
respective virtual systems 

A distributed SMMS approach was also proposed 
in [43.99], in l-sstMUEHilMI and l<gf'tnwf to en¬ 
able a single human user to telecontrol some of the 
slave robots. The inter-slave behaviors are encoded via 
a distributed artificial potential constructed on a time- 
invariant undirected information graph (Fig. 43.21). By 
using kinematic virtual systems to abstract the slave 
robots, the work achieves the combination of master- 
passivity and slave-stability. It guarantees no collisions 
among the slave robots or with obstacles and no sepa¬ 
rations among the slave robots. 

A control approach for MMSS telerobotic systems 
was proposed in [43.107] and in where 

two human users telecontrol distinct frames on a single 
large-DOF slave robot. The velocity space of the slave 
is decomposed according to the two command motions, 
resolving conflicts between and constraints imposed on 
them. Priority is given to the primary user’s commands 


Redundant Slave 



Fig. 43.22 Trilateral teleoperation (after [43.107]), where 
the primary master controls the end-effector frame P, while 
the secondary master controls task-space frame A, B or 
C: if A is chosen, primary and secondary tasks are non- 
conflicting/nonconstrained; if B is chosen, nonconflicting, 
yet, the secondary task is constrained; if C is chosen, 
conflicting, thus, prioritized nullspace control is necessary 
between primary and secondary tasks 

(Fig. 43.22). The kinematics-level prioritized velocity 
commands are realized by dynamics-level adaptive con¬ 
trol for all master and slave robots. 

A different shared trilateral MMSS teleoperation 
framework was developedin [43.108], where two hu¬ 
man users teleoperate the same point of a single slave 
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robot. Their control authority is adjusted with a domi¬ 
nance factor a e [0,1] set according to the task objec¬ 
tive (e.g., a = 1 for training or a = 0 for evaluation). 
The MMSS system is optimized for a measure of trans¬ 
parency and Llewellyn’s criteria is applied to the equiv¬ 
alent two-port system (with environment impedance Z e 
embedded) to establish unconditional stability. 

These multilateral and the previous mobile teler- 
obotic approaches are promising to significantly expand 
the utility and application horizons of conventional 
telerobotics. Significant technical challenges and re¬ 
search questions remain, for example: 


1. How to assign and determine the roles for multiple 
coordinating human users? 

2. How to systematically split control tasks into tele¬ 
operative and autonomous control? 

3. What are the most suitable performance measures, 
which are likely different from the conventional 
metrics? 

4. What is the best form of human interface (or hap¬ 
tic feedback) [43.100] for the mobile and mul¬ 
tilateral telerobotics and how to complement the 
interface with other modality (e.g., visual feed¬ 
back). 


43.6 Conclusions and Further Reading 


Despite its age, telerobotics remains an exciting and vi¬ 
brant area of robotics. In many ways, it forms a platform 
which can utilize the advances in robotic technologies 
while simultaneously leveraging the proven skills and 
capabilities of human users. Compare this, for example, 
with the development of the automobile and its relation 
to the driver. As cars are gradually becoming more so¬ 
phisticated with added electronic stability control and 
navigation systems, they are becoming safer and more 
useful to their operators, not replacing them. Similarly 
telerobotics serves as a pathway for gradual progress 
and, as such, is perhaps best suited to fulfill robotics 
long-held promise of improving human life. It is seeing 
use in the challenging area of search and rescue. And 
with the recent developments and commercializations 
in telerobotic surgery systems, it is indeed impacting 
on the lives of tens of thousands of patients in a pro- 

Video-References 


found fashion and extending the reach of robotics into 
our world. 

For further reading in the area of supervisory con¬ 
trol, we refer to Sheridan [43.39], Though published 
in 1992, it remains the most complete discussion on 
the topic. Unfortunately few other books are devoted 
to or even fully discuss telerobotics. In [43.109] many 
recent advances, including methods, experiments, ap¬ 
plications, and developments, are collected. Beyond 
this, in the areas of bilateral and shared control, as 
well as to understand the various applications, we can 
only refer to the citations provided. Finally, in addition 
to the standard robotics journals, we note in particu¬ 
lar Presence: Teleoperators and Virtual Environments, 
published by the MIT Press. Combined with virtual- 
reality applications, it focuses on technologies with 
a human operator. 
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Semi-autonomous teleoperation of multiple UAVs: Passing a narrow gap 
available from http://handbool<ofrobotics.org/view-chapter/43/videodetails/71 
Semi-autonomous teleoperation of multiple UAVs: Tumbing over obstacle 
available from http://handbookofrobotics.org/view-chapter/43/videodetails/72 
Bilateral teleoperation of multiple quadrotors with time-varying topology 
available from http://handbookofrobotics.org/view-chapter/43/videodetails/73 
Passive teleoperation of nonlinear telerobot with tool-dynamics rendering 
available from http://handbookofrobotics.org/view-chapter/43/videodetails/74 
Asymmetric teleoperation of dual-arm mobile manipulator 
available from http://handbookofrobotics.org/view-chapter/43/videodetails/75 
Tele-existence master-slave system for remote manipulation 
available from http://handbookofrobotics.org/view-chapter/43/videodetails/297 
JPL dual-arm telerobot system 

available from http://handbookofrobotics.org/view-chapter/43/videodetails/298 
Single and dual arm supervisory and shared control 

available from http://handbookofrobotics.org/view-chapter/43/videodetails/299 
Teleoperated humanoid robot - HRP 

available from http://handbookofrobotics.org/view-chapter/43/videodetails/318 
Teleoperated humanoid robot - HRP: Tele-driving of lifting vehicle 
available from http://handbookofrobotics.org/view-chapter/43/videodetails/319 
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Idam'ilrM'gHB Multi-modal multi-user telepresence and teleaction system 

available from http://handbookofrobotics.org/view-chapter/43/videodetails/321 
|4S®JB2EEI Laparoscopic telesurgery workstation 

available from http://handbookofrobotics.org/view-chapter/43/videodetails/322 
Passivity of IPC strategy at 30 Hz sample rate 

available from http://handbookofrobotics.org/view-chapter/43/videodetails/724 
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As of 2013, almost all robots have access to com¬ 
puter networks that offer extensive computing, 
memory, and other resources that can dramatically 
improve performance. The underlying enabling 
framework is the focus of this chapter: networked 
robots. Networked robots trace their origin to teler¬ 
obots or remotely controlled robots. Telerobots are 
widely used to explore undersea terrains and outer 
space, to defuse bombs and to clean up hazardous 
waste. Until 1994, telerobots were accessible only 
to trained and trusted experts through dedicated 
communication channels. This chapter will de¬ 
scribe relevant network technology, the history of 
networked robots as it evolves from teleoperation 
to cloud robotics, properties of networked robots, 
how to build a networked robot, example sys¬ 
tems. Later in the chapter, we focus on the recent 
progress on cloud robotics, and topics for future 
research. 
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44.1 Overview and Background 

As illustrated in Fig. 44.1, the field of networked 
robots locates at the intersection between two exciting 
fields: robotics and networking. Similarly, teleopera¬ 
tion (Chap. 43) and multiple mobile robot systems 
(Chap. 51) also find their overlaps in the intersection. 
The primary concerns of the teleoperation are stability 
and time delay. Multiple mobile robot systems con¬ 
cerns coordination and planning of autonomous robots 
and sensors communicating over local networks. The 
subfield of Networked Robots focuses on the robot sys¬ 
tem architectures, interfaces, hardware, software, and 
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applications that use networks (primarily the Inter¬ 
net/Cloud). 

By 2012, several hundred networked robots have 
been developed and put online for public use. Many 
papers have been published describing these systems 
and a book on this subject by Goldberg and Siegwart 
is available [44.1]. Updated information about new re¬ 
search and an archive/survey of networked robots is 
available on the website of the IEEE technical commit¬ 
tee on networked robots, which fosters research in this 
area [44.2], 
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Fig. 44.1 Relationship between the subjects of networked 
robots (Chap. 44, this chapter), teleoperation (Chap. 43), 
and multiple mobile robot systems (Chap. 53) ► 

The rest of the chapter is organized as follows: we 
first review the history and related work in Sect. 44.2. 
In Sect. 44.3, we review network and communica¬ 
tion technology to provide necessary background for 
the following two main Sects. 44.4 and 44.5. Sec¬ 
tion 44.4 focuses on traditional networked robots while 
Sect. 44.5 summarize the new development in cloud 

44.2 A Brief History 

Networked robots have their root in teleoperation sys¬ 
tems, which started as remotely controlled devices. 
However, thanks to the recent evolution of the Inter¬ 
net and wireless networks, networked robots quickly 
expand their scope from the traditional master-slave 
teleopereation relationship to an integration of robots, 
human, agents, off-board sensors, databases, and clouds 
over the globe. To review the history of networked 
robots, we trace back to the root: remotely controlled 
devices. 

44.2.1 Networked Teleoperation 

Like many technologies, remotely controlled devices 
were first imagined in science fiction. In 1898, Nicola 
Tesla [44.3] demonstrated a radio-controlled boat in 
New York’s Madison Square Garden. The first ma¬ 
jor experiments in teleoperation were motivated by 
the need to handle radioactive materials in the 1940s. 
Goertz and Thompson demonstrated one of the first 
bilateral simulators in the 1950s at the Argonne Na¬ 
tional Laboratory [44.4]. Remotely operated mech¬ 
anisms have been designed for use in inhospitable 
environments such as undersea [44.5] and space explo¬ 
ration [44.6]. At General Electric, Mosher [44.7] devel¬ 
oped a two-arm teleoperator with video cameras. Pros¬ 
thetic hands were also applied to teleoperation [44.8]. 
More recently, teleoperation is being considered for 
medical diagnosis [44.9], manufacturing [44.10] and 
micromanipulation [44.11]. See Chap. 43 and the book 
from Sheridan [44.12] for excellent reviews on teleop¬ 
eration and telerobotics research. 

The concept of hypertext (linked references) was 
proposed by Vannevar Bush in 1945 and was made pos¬ 
sible by the subsequent developments in computing and 
networking. In the early 1990s, Berners-Lee introduced 
the hypertext transmission protocol (HTTP). A group 
of students led by Marc Andreessen developed an open 



robotics. Section 44.6, we conclude the chapter with re¬ 
cent applications and future directions. 


source version of the first graphical user interface, the 
Mosaic browser, and put it online in 1993. The first 
networked camera, the predecessor of today’s webcam , 
went online in November 1993 [44.13] 

Approximately nine months later, the first net¬ 
worked telerobot went online. The Mercury Project 
combined an IBM industrial robot arm with a digi¬ 
tal camera and used the robot’s air nozzle to allow 
remote users to excavate for buried artifacts in a sand¬ 
box [44.14, 15]. Working independently, a team led 
by Taylor and Trevelyan at the University of Western 
Australia demonstrated a remotely controlled six-axis 
telerobot in September 1994 [44.16,17]. These early 
projects pioneered a new field of networked telerobots. 
See [44.18-26] for other examples. 

Networked telerobots are a special case of supervi¬ 
sory control telerobots, as proposed by Sheridan and his 
colleagues [44.12]. Under supervisory control, a local 
computer plays an active role in closing the feedback 
loop. Most networked robotics are type (c) supervisory 
control systems (Fig. 44.2). 

Although a majority of networked telerobotic sys¬ 
tems consist of a single human operator and a single 
robot [44.27-34], Chong et al. [44.35] propose a use¬ 
ful taxonomy: single operator single robot (SOSR), sin¬ 
gle operator multiple robot (SOMR) [44.36, 37], multi¬ 
ple operator single robot (MOSR), and multiple opera¬ 
tor multiple robot (MOMR) [44.38, 39] 

These frameworks greatly extend sys¬ 
tem architecture of networked robots. In fact, human op¬ 
erators can often be replaced with autonomous agents, 
off-board sensors, expert systems, and programmed 
logics, as demonstrated by Xu and Song [44.40] and 
Sanders et al. [44.41], The extended networked con¬ 
nectivity also allows us to employ techniques such as 
crowd sourcing and collaborative control for demand¬ 
ing applications such as nature observation and environ¬ 
ment monitoring [44.42,43]. Hence, networked teler- 
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Fig. 44.2 A spectrum of teleoperation control modes adapted from Sheridan’s text (after [44.12]). We label them (a-e), 
in order of increasing robot autonomy. At the far left would be a mechanical linkage where the human directly operates 
the robot from another room through sliding mechanical bars, and on the far right is the system where the human role is 
limited to observation/monitoring. In (c-e), the dashed lines indicated that communication may be intermittent 


obots fully evolute into networked robots: an integration 
of robots, humans [44.44], computing power, off-board 
sensing, and databases over the Internet. 

The last 18 years (1994—2012) witnessed the exten¬ 
sive development in networked robots. New systems, 
new experiments, and new applications go well beyond 
traditional fields such as defense, space, and nuclear 
material handing [44. 12] that motivated teleoperation in 
early 1950s. As the Internet introduces universal access 
to every corner of life, the impact of networked robots 
becomes broader and deeper in modern society. Recent 
applications range from education, industry, commer¬ 
cial, health care, geology, environmental monitoring, to 
entertainment, and arts. 

Networked robots provide a new medium for people 
to interact with remote environment. A networked robot 
can provide more interactivity beyond what a normal 
videoconferencing system. The physical robot not only 
represents the remote person but also transmits multi¬ 
modal feedback to the person, which is often referred as 
telepresence in the literature [44.30]. Paulos et al.’s Per¬ 
sonal ROving Presence (PRoP) robot [44.45], Jouppi 
and Thomas' Surrogate robot [44.30], Takayama et al.’s 
Texai [44.46], and Lazewatsky and Smart’s inexpensive 
platform [44.47] are representative work. 


Networked robots have great potential for educa¬ 
tion and training. In fact, one of the earliest networked 
telerobot systems [44.48] originates from the idea of 
a remote laboratory. Networked telerobots provide uni¬ 
versal access to the general public, who may have 
little to no knowledge of robots, with opportunities 
to understand, learn, and operate robots, which were 
expensive scientific equipment limited to universities 
and large corporate laboratories before. Built on net¬ 
worked telerobots, online remote laboratories [44.49, 
50] greatly improves distance learning by providing an 
interactive experience. For example, teleoperated tele¬ 
scopes help students to understand astronomy [44.51]. 
Teleoperated microscope [44.52] helps student to ob¬ 
serve micro-organisms. The Tele-Actor project [44.53] 
allows a group of students to remotely control a human 
tele-actor to visit environments that are normally not ac¬ 
cessible to them such as clean-room environments for 
semiconductor manufactory facility and deoxyribonu¬ 
cleic acid (DNA) analysis laboratories. 

44.2.2 Cloud Robotics and Automation 

Recent development of cloud computing provide 
new means and platform for networked robots. In 
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2010, Kuffner at Google introduced the term Cloud 
robotics [44.54] to describe a new approach to robotics 
that takes advantage of the Internet as a resource for 
massively parallel computation and real-time sharing 
of vast data resources. The Google autonomous driv¬ 
ing project exemplifies this approach: the system in¬ 
dexes maps and images that are collected and updated 
by satellite, Streetview, and crowdsourcing from the 
network to facilitate accurate localization. Another ex¬ 
ample is Kiva Systems new approach to warehouse 
automation and logistics using large numbers of mo¬ 
bile platforms to move pallets using a local network to 
coordinate planforms and update tracking data. These 
are just two new projects that build on resources from 
the Cloud. Steve Cousins of Willow Garage aptly sum¬ 
marized the idea: No robot is an island. Cloud robotics 
recognizes the wide availability of networking, incorpo¬ 
rates elements of open-source, open-access, and crowd¬ 
sourcing to greatly extend earlier concepts of Online 
Robots [44.1] and Networked Robots [44.55, 56]. 


The Cloud has been used as a metaphor for 
the Internet since the inception of the World Wide 
Web in the early 1990s. As of 2012, researchers are 
pursuing a number of cloud robotics and automa¬ 
tion projects [44.57,58]. New resources range from 
software architectures [44.59-62] to computing re¬ 
sources [44.63]. The RoboEarth project [44.64] aims to 
develop [44.65]: 

a World Wide Web for robots: a giant network 
and database repository where robots can share in¬ 
formation and learn from each other about their 
behavior and their environment. 

Cloud robotics and automation is related to concepts of 
the Internet of Things [44.66] and the Industrial Inter¬ 
net, which envision how radio-frequency identification 
(RFID) and inexpensive processors can be incorporated 
into a vast array of objects from inventory items to 
household appliances to allow them to communicate 
and share information. 


44.3 Communications and Networking 


Below is a short review of relevant terminologies and 
technologies on networking. For details, see the texts 
by [44.67], 

A communication network includes three elements: 
links, routers/switchers, and hosts. Finks refer to the 
physical medium that carry bits from one place to an¬ 
other. Examples of links include copper or fiber-optic 
cables and wireless (radio frequency or infrared) chan¬ 
nels. Switches and routers are hubs that direct digital 
information between links. Hosts are communication 
end points such as browsers, computers, and robots. 

Networks can be based in one physical area (local- 
area network, or FAN), or distributed over wide dis¬ 
tances (wide-area network, or WAN). Access control 
is a fundamental problem in networking. Among a va¬ 
riety of methods, the ethernet protocol is the most 
popular. Ethernet provides a broadcast-capable multi¬ 
access FAN. It adopts a carrier-sense multiple-access 
(CSMA) strategy to address the multiple-access prob¬ 
lem. Defined in the IEEE 802.x standard, CSMA allows 
each host to send information over the link at any 
time. Therefore, collisions may happen between two 
or more simultaneous transmission requests. Collisions 
can be detected either by directly sensing the voltage 
in the case of wired networks, which is referred to 
as collision detection (CSMA/CD), or by checking the 
time-out of an anticipated acknowledgement in wireless 
networks, which is referred to as collision avoidance 
(CSMA/CA). If a collision is detected, both/all senders 
randomly back off a short period of time before retrans¬ 


mitting. CSMA has a number of important properties: 
(1) it is a completely decentralized approach, (2) it does 
not need clock synchronization over the entire network, 
and (3) it is very easy to implement. However, the disad¬ 
vantages of CSMA are: (1) the efficiency of the network 
is not very high and (2) the transmission delay can 
change drastically. 

As mentioned previously, LANs are interconnected 
with each other via routers/switchers. The information 
transmitted is in packet format. A packet is a string 
of bits and usually contains the source address, the 
destination address, content bits, and a checksum. 
Routers/switchers distribute packets according to their 
routing table. Routers/switchers have no memory of 
packets, which ensures scalability of the network. Pack¬ 
ets are usually routed according to a first-in first-out 
(FIFO) rule, which is independent of the application. 
The packet formats and addresses are independent of 
the host technology, which ensures extensibility. This 
routing mechanism is referred to as packet switching 
in the networking literature. It is quite different from 
a traditional telephone network, which is referred to as 
circuit switching. A telephone network is designed to 
guarantee a dedicated circuit between a sender and a re¬ 
ceiver once a phone call is established. The dedicated 
circuitry ensures communication quality. However, it 
requires a large number of circuits to ensure the qual¬ 
ity of service (QOS), which leads to poor utilization 
of the overall network. A packet-switching network 
cannot guarantee dedicated bandwidth for each indi- 
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vidual pair of transmissions, but it improves overall 
resource utilization. The Internet, which is the most 
popular communication media and the infrastructure of 
networked telerobots, is a packet-switching network. 

44.3.1 The Internet 

The creation of the Internet can be traced back to US 
Department of Defense’s (DOD) APRA NET network 
in the 1960s. There are two features of the APRA 
NET network that enabled the successful evolution of 
the Internet. One feature is the ability for information 
(packets) to be rerouted around failures. Originally, this 
was designed to ensure communication in the event of 
a nuclear war. Interestingly, this dynamic routing capa¬ 
bility also allows the topology of the Internet to grow 
easily. The second important feature is the ability for 
heterogeneous networks to interconnect with one an¬ 
other. Heterogeneous networks, such as X.25, G.701, 
ethernet, can all connect to the Internet as long as they 
can implement the Internet protocol (IP). The IP is me¬ 
dia, operating system (OS), and data rate independent. 
This flexible design allows a variety of applications and 
hosts to connect to the Internet as long as they can gen¬ 
erate and understand IP. 

Figure 44.3 illustrates a four-layer model of the pro¬ 
tocols used in the Internet. On the top of the IP, we have 
two primary transport layer protocols: the transmis¬ 
sion control protocol (TCP) and the user data protocol 
(UDP). TCP is an end-to-end transmission control pro¬ 
tocol. It manages packet ordering, error control, rate 
control, and flow control based on packet round-trip 
time. TCP guarantees the arrival of each packet. How¬ 
ever, excessive retransmission of TCP in a congested 
network may introduce undesirable time delays in a net¬ 
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Fig. 44.3 A four-layer model of Internet protocols (af¬ 
ter [44.67]) 


worked telerobotic system. UDP behaves differently; it 
is a broadcast-capable protocol and does not have a re¬ 
transmission mechanism. Users must take care of error 
control and rate control themselves. UDP has a lot less 
overhead compared to TCP. UDP packets are transmit¬ 
ted at the sender’s preset rate and the rate is changed 
based on the congestion of a network. UDP has great 
potential, but it is often blocked by firewalls because 
of a lack of a rate control mechanism. It is also worth 
mentioning that the widely accepted term TCP/IP refers 
to the family of protocols that build on IP, TCP, and 
UDP. 

In the application layer of the Internet protocols, the 
HTTP is one of the most important protocols. HTTP is 
the protocol for the World Wide Web (WWW). It allows 
the sharing of multimedia information among hetero¬ 
geneous hosts and OSs including text, image, audio, 
and video. The protocol has significantly contributed 
to the boom of the Internet. It also changes the tradi¬ 
tional client/server (C/S) communication architecture to 
a browser/server (B/S) architecture. A typical configu¬ 
ration of the B/S architecture consists of a web server 
and clients with web browsers. The web server projects 
the contents in hypertext markup language (HTML) 
format or its variants, which is transmitted over the 
Internet using HTTP. User inputs can be acquired 
using the common gateway interface (CGI) or other 
variants. The B/S architecture is the most accessible 
because no specialized software is needed at the client 
end. 

44.3.2 Wired Communication Links 

Even during peak usage, the network backbones of 
the Internet often run at less than 30% of their over¬ 
all capacity. The average backbone utilization is around 
15—20%. The primary speed limitation for the Internet 


Table 44.1 Last-mile Internet speed by wired connection 
type (if not specified, the downstream transmission and the 
upstream transmission share the same bandwidth) 


Types 

Bits per second 

Dialup modem (V.92) 

Up to 56 K 

Integrated services digital 
network (ISDN) 

64-160 K for BRI, up to 

2048 K for PRI 

High data rate digital sub¬ 
scriber line (HDSL) 

Up to 2.3 M duplex on two 
twisted-pair lines 

Asymmetric digital sub¬ 
scriber line (ADSL) 

1.544—24.0 M downstream, 
0.5—3.3 M upstream 

Cable modem 

2—400 M downstream, 

0.4—108 M upstream 

Fiber to the home (FTTH) 

0.005—1 G downstream, 
0.002—1 G upstream 

Direct Internet II node 

1.0—10.0G 
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Table 44.2 Survey of wireless technologies in terms of bit rate and range 


Types 

Bit rate (bps) 

Band (Hz) 

Range(m) 

Zigbee (802.15.4) 

20-250 K 

868-915 M/2.4 G 

50 

Bluetooth 

732 K-3.0M 

2.4 G 

100 

3G HSPA 

400 K-14.0M 

< 3.5 G 

N/A 

HSPA+ 

5.76 M—44.0 M 

< 3.5 G 

N/A 

LTE 

10M-300M 

< 3.5 G 

N/A 

WiFi (802.1 la,b,g,n) 

11 —600 M 

2.4G/5G 

100 


is the last mile, the link between clients and their local 
Internet service providers (ISP). 

Table 44.1 lists typical bit rates for different con¬ 
nection types. It is interesting to note the asymmetric 
speeds in many cases, where upstream bit rate (from the 
client to the Internet), are far slower than downstream 
bit rates (from the Internet to the client). These asym¬ 
metries introduce complexity into the network model 
for teleoperation. Since the speed difference between 
the slowest modem link and the fastest Internet II node 
is over 10000, designers of a networked telerobotic 
system should anticipate a large variance of communi¬ 
cation speeds. 

44.3.3 Wireless Links 

Table 44.2 compares the speed, band, and range of 
wireless standards as of 2012. Increasing bit rate and 
communication range requires increasing power. The 
amount of radio frequency (RF) transmission power re¬ 
quired over a distance d is proportional to d k , where 
2 < k < 4 depending on the antenna type. In Table 44.2, 
Bluetooth and Zigbee are typical low-power transmis¬ 
sion standards that are good for short distances. HSPA+ 
and LTE are commercially marketed as the 4G cell¬ 
phone network. 

By providing high-speed connectivity at low cost, 
WiFi is the most popular wireless standard in 2012. Its 
range is approximate 100 m line of sight and the WiFi 
wireless network usually consists of small-scale inter¬ 
connected access points. The coverage range usually 
limits these networks to an office building, home, and 
other indoor environments. WiFi is a good option for 
indoor mobile robots and human operators. If the robot 
needs to navigate in the outdoor environment, the 3G 
or 4G cellphone network can provide the best cover¬ 
age available. Although obvious overlap exists among 
wireless standards in coverage and bandwidth, there are 
two import issues that have not been covered by Ta¬ 
ble 44.2. One is mobility. We know that, if an RF source 
or receiver is moving, the corresponding Doppler effect 
causes a frequency shift, which could cause problems in 
communication. WiFi is not designed for fast-moving 
hosts. 3G HSPA cellphone allows the host to move at 
a vehicle speed under 120 km/h. However, LTE allows 


the host to move at a speed of 350 or 500km/h, which 
even works for high-speed trains. 

Long range wireless links often suffer from latency 
problem, which may drastically decreases system per¬ 
former as discussed in Chap. 43. One may notice that 
we did not list satellite wireless in Table 44.2 because 
the long latency (0.5—1.7 s) and high price makes it dif¬ 
ficult to be useful for robots. The large antenna size 
and high power consumption rate also limits its usage 
in mobile robots. In fact, the best option for long range 
wireless is LTE. LTE is designed with a transmission 
latency of less that 4 ms whereas 3G HSPA cellphone 
networks have a variable latency of 10—500 ms. 

44.3.4 Video and Audio Transmission 
Standards 

In networked robots systems, the representation of the 
remote environment is often needed to be delivered 
to online users in video and audio format. To deliver 
video and audio over the Internet, raw video and audio 
data from camera optical sensor and microphone must 
be compressed according to different video and au¬ 
dio compression standards to fit in the limited network 
bandwidth. Due to lack of bandwidth and computing 
power to encode streaming video, most early systems 
only transmit periodic snapshots of the remote scene in 
JPEG format at limited frame rate, i. e., 1—2 frames per 
second or less. Audio was rarely considered in the early 
system design. The rudimentary video delivery meth¬ 
ods in the early system were mostly implemented using 
HTML and JavaScript to reload the JPEG periodically. 

Today, the expansion of HTML standards allow web 
browsers to employ plug-ins as the client end of stream¬ 
ing video. HTML5 even natively supports video decod¬ 
ing. Therefore, the server end of recent systems often 
employs streaming server software, such as Adobe 
Flash Media Encoder, Apple Quick Time Streaming 
Server, Oracle Java Media Framework, Helix Media 
Delivery Platform, Microsoft DirectX, SkypeKit, etc., 
to encode and deliver video. These streaming video 
sever packages often provide easy-to-use software de¬ 
velopment kit (SDK) to facilitate system integration. 

It is worth noting that these different software pack¬ 
ages are just different implementations of video/audio 
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Table 44.3 A comparison of existing videostreaming stan¬ 
dards for the same resolution under the same fixed band¬ 
width (FMBT represents buffering time settings that would 
not significantly decrease compression ratio or video qual¬ 
ity.) 


Standards 

Feasible minimum 
buffering time 
(FMBT) 

Framerate 

MJPEG 

Zero (< 10 ms) 

Low 

MPEG2 

Variable (i. e., 50 ms - 
video length), 2—10 s 

are common 

Moderate 

H.263+ 

< 300 ms 

High 

H.264/ MPEG4-AVC 

Zero (< 10 ms) 

Highest 


streaming protocols. Not every protocol is suitable for 
networked robots. Some protocols are designed to de¬ 
liver video on demand while others are designed for live 
streaming for videoconferencing purposes. Networked 
robots use real-time video as feedback information, 
which imposes strict requirements in latency and band¬ 
width similar to those of videoconferencing. One way 
latency of more than 150 ms can significantly degrade 
telepresence and hence the performance of the human 
operator. 

Latency is often caused by bandwidth and video 
encoding/decoding time. Since audio data amount is 
negligible when comparing to that video data. We will 


focus the discussion on video compression standards. 
There is always a tradeoff between frame rate and reso¬ 
lution for a given bandwidth. There is also a tradeoff 
between compression ratio and computation time for 
a given central processing unit (CPU). The computation 
time includes both CPU time and data-buffering time at 
both client and server ends. Video encoding is a very 
computationally intensive task. A long computation pe¬ 
riod introduces latency and significantly impair the sys¬ 
tem performance. It is possible to use hardware to cut 
down the computation time but not the data-buffering 
time, which are controlled by the video encoder. 

There are many standards and protocols avail¬ 
able but most of them are just variations of MJPEG, 
MPEG2, H.263, and MPEG4/AVC/H.264. We compare 
those standards in Table 44.3. Note that the compari¬ 
son is qualitative and may not be the most accurate due 
to the fact that each video encoding standard has many 
parameters that affect the overall buffering time. 

From networked robot point of view, the buffer¬ 
ing time determines the latency and the frame rate 
determines the responsiveness of the system. An ideal 
videostream should have both high frame rate and low 
buffering time. But if both cannot be achieved at the 
same time, low latency is preferred. From Table 44.3, 
H.264/MPEG4-AVC clearly outperforms other com¬ 
petitors and is the most popular video compression 
method. 


44.4 Properties of Networked Robots 

Networked robots have the following properties: 

• The physical world is affected by a device that is 
locally controlled by a network server, which con¬ 
nects to the Internet to communicate with remote 
human users, databases, agents, and off-board sen¬ 
sors, which are referred to as clients of the system. 

• Human decision making capability is often an inte¬ 
gral part of the system. If so, humans often access 
the robot via web browsers, such as Internet Ex¬ 
plorer or Firefox, or apps in mobile device. As of 
2012 , the standard protocol for network browsers is 
the hypertext transfer protocol (HTTP), a stateless 
transmission protocol. 

• Most networked robots are continuously accessible 
(online), 24 h a day, 7 days a week. 

• Networks may be unreliable or have different speed 
for clients with different connections. 

• Since hundreds of millions of people now have 
access to the Internet, mechanisms are needed to 
handle client authentication and contention. System 


security and privacy of users are important in the 
networked robots. 

• Input and output for human users for networked 
robots are usually achieved with the standard com¬ 
puter screen, mouse, and keyboard. 

• Clients may be inexperienced or malicious, so on¬ 
line tutorials and safeguards are generally required. 

• Additional sensing, databases, and computing re¬ 
sources may be available over the network. 

44.4.1 Overall Structure 

As defined by Mason, Peshkin, and others [44.68, 69], 
in quasistatic robot systems, accelerations, and inertial 
forces are negligible compared to dissipative forces. In 
quasistatic robot systems, motions are often modeled as 
transitions between discrete atomic configurations. 

We adopt a similar terminology for networked teler¬ 
obots. In quasistatic telerobotics (QT), robot dynamics 
and stability are handled locally. After each atomic mo¬ 
tion, a new state report is presented to the remote user. 
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who sends back an atomic command. The atomic state 
describes the status of the robot and its corresponding 
environment. Atomic commands refer to human direc¬ 
tives, which are desired robotic actions. 

Several issues arise: 


A reader can follow the below example to build his/her 
owner networked robot system as well as understand 
challenges in the issues. 

44.4.2 Building a Networked Robot System 


• State-command presentation'. How should state and 
available commands be presented to remote human 
operators using the two-dimensional (2-D) screen 
display? 

• Command execution/state generation: How should 
commands be executed locally to ensure that the de¬ 
sired state is achieved and maintained by the robot? 

• Command coordination : How should commands be 
resolved when there are multiple human operators 
and/or agents? How to synchronize and aggregate 
commands issued by users/agents with different 
network connectivity, background, responsiveness, 
error rate, etc., to achieve the best possible system 
performance? 

• Virtual Fixture: Error prevention and state correc¬ 
tion'. How should the system prevent the wrong 
commands that may lead the robot to collision or 
other undesirable states? 

Before we detail these issues, let us walk through 

how to build a minimum networked robot system. 



Users 

Fig. 44.4 Typical system architecture for a networked telerobot 



This minimal system is a networked telerobotic sys¬ 
tem which allows a group of users to access a robot 
via web browsers. As illustrated in Fig. 44.4, a typical 
or minimal networked telerobotic system typically in¬ 
cludes three components: 

• Users: Anyone with an Internet connection and 
a web browser or equivalent apps that understand 
HTTP. 

• Web server: A computer running a web server soft¬ 
ware. 

• Robot: A robot manipulator, a mobile robot, or any 
device that can modify or affect its environment. 

Users access the system via their web browsers. 
Any web browser that is compatible with W3C’s 
HTML standard can access a web server. In 2012, the 
most popular web browsers are Microsoft Internet Ex¬ 
plorer, Mozilla Firefox, Google Chrome, Apple Safari, 
and Opera. New browsers and updated versions with 
new features are introduced periodically. All of these 
popular browsers issue the corresponding mobile apps 
to support mobile devices such as Apple iPads, Apple 
iPhones, and Google Andriod-based Tablets and smart 
phones. 

A web server is a computer that responds to HTTP 
requests over the Internet. Depending upon the operat¬ 
ing system of the web server, popular server software 
packages include Apache and Microsoft Internet In¬ 
formation Services (IIS). Most servers can be freely 
downloaded from the Internet. 

To develop a networked telerobot, one needs a basic 
knowledge of developing, configuring, and maintaining 
web servers. As illustrated in Fig. 44.5, the development 
requires knowledge of HTML and at least one local pro¬ 
gramming languages such as C, C#, CGI, Javascript, 
Perl, PHP, .Net, or Java. 

It is important to consider compatibility with the va¬ 
riety of browsers. Although HTML is designed to be 
compatible with all browsers, there are exceptions. For 
example. Javascript, which is the embedded scripting 
language of web browsers, is not completely compat¬ 
ible between Internet Explorer and Firefox. One also 
needs to master the common HTML components such 
as forms that are used to accept user inputs, frames that 
are used to divide the interface into different functional 
regions, etc. An introduction to HTML can be found 
in [44.70]. 


Fig. 44.5 A sample software architecture of a networked telerobot 
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User commands are usually processed by the web 
server using CGI, the common gateway interface. Most 
sophisticated methods such as PHP, Java Server Pages 
(JSP), and socket-based system programming can also 
be used. CGI is invoked by the HTTP server when the 
CGI script is referred in the uniform resource locator 
(URL). The CGI program then interprets the inputs, 
which is often the next robot motion command, and 
sends commands to the robot via a local communica¬ 
tion channel. CGI scripts can be written in almost any 
programming language. The most popular ones are Perl 
and C. 

A simple networked telerobotic system can be con¬ 
structed using only HTML and CGI. However, if the 
robot requires a sophisticated control interface, ad¬ 
vanced plug-ins such as Java Applet, Silver Light, or 
Flash, are recommended. These plug-ins run inside 
the web browser on the client’s computer. Information 
about these plug-ins can be found at home pages of Or¬ 
acle, Microsoft, and Adobe, respectively. Java applet is 
highly recommended because it is the most widely sup¬ 
ported by different browsers. Recently, the fast adoption 
of HTML5 also provide a new long term solution to 
solve the compatibility issue. 

Most telerobotic systems also collect user data and 
robot data. Therefore, database design and data pro¬ 
cessing program are also needed. The most common 
used databases include MySQL and PostgresSQL. Both 
are open-source databases and support a variety of 
platforms and operation systems. Since a networked 
telerobotic system is online 24 h a day, reliability is also 
an important consideration in system design. Website 
security is critical. Other common auxiliary develop¬ 
ments include online documentation, online manual, 
and user feedback collection. 

It is not difficult to expand this minimal networked 
telerobotic system into a full-fledged networked robot 
system. For example, some users can be replaced by 
agents that runs 24 h a day and 7 days a week to 
monitor system states and co-perform tasks with hu¬ 
mans or take over the system when nobody is online. 
These agents can be implemented using cloud comput¬ 
ing. Such extensions are usually based on the need of 
the task. 

44.4.3 State-Command Presentation 

To generate a correct and high-quality command de¬ 
pends on how effectively the human operator under¬ 
stands the state feedback. The state-command presenta¬ 
tion contains three subproblems: the 2-D representation 
of the true robot state (state display), the assistance 
provided by the interface to generate new commands 
(spatial reasoning), and the input mechanism. 


State Displays 

Unlike traditional point-to-point teleoperation, where 
specialized training and equipment tire available to op¬ 
erators, networked telerobots offer wide access to the 
general public. Designers cannot assume that operators 



Fig. 44.6 Browser’s view of the first networked telerobot 
interface (after [44.55]). The schematic at lower right gives 
an overhead view of position of the four-axis robot arm 
(with the camera at the end marked with X), and the image 
at the lower left indicates the current view of the cam¬ 
era. The small button marked with a dot at the left directs 
a 1 s burst of compressed air into the sand below the cam¬ 
era. The Mercury Project was online from August 1994 to 
March 1995 



Fig. 44.7 Browser interface to the Australian networked telerobot 
which was a six-axis arm that could pick up and move blocks (af¬ 
ter [44.17]) 
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have any prior experience with robots. As illustrated 
in Fig.44.6, networked telerobotic systems must display 
the robot state on a 2-D screen display. 



Fig. 44.8 Use of a multicamera system for multiviewpoint state 
feedback (after [44.71]) 



Fig. 44.9 Camera control and mobile robot control in Patrick 
Saucy and Francesco Mondada’s Khep on the web project 


The states of the teleoperated robot are often char¬ 
acterized in either world coordinates or robot joint con¬ 
figuration, which are either displayed in numerical for¬ 
mat or through a graphical representation. Figure 44.6 
lists robot XYZ coordinates on the interface and draws 
a simple 2-D projection to indicate joint configura¬ 
tions. Figure 44.7 illustrates another example of tele¬ 
operation interface that was developed by Taylor and 
Trevelyan [44.48]. In this interface, XYZ coordinates are 
presented in a sliding bar near the video window. 

The state of the robot is usually displayed in a 2-D 
view as shown in Figs. 44.6 and 44.7. In some systems, 
multiple cameras can help the human operator to un¬ 
derstand the spatial relationship between the robot and 
the objects in the surrounding environment. Figure 44.8 
shows an example with four distinct camera views for 
a six-degree-of-freedom industrial robot. 

Figure 44.9 demonstrate an interface with a pan- 
tilt-zoom robotic camera. The interface in Fig. 44.9 is 
designed for a mobile robot. 

More sophisticated spatial reasoning can eliminate 
the need for humans to provide low-level control by au¬ 
tomatically generating a sequence of commands after 
it receives task-level commands from the human op¬ 
erator. This is particularly important when the robotic 
system is highly dynamic and requires a very fast re¬ 
sponse. In this case, it is impossible to ask the human to 
generate intermediate steps in the robot control; for ex¬ 
ample, Belousov et al. adopt a shared autonomy model 
to direct a robot to capture a moving rod [44.28] as 
shown in Fig. 44.10. Fong and Thorpe [44.72] sum¬ 
marize vehicle teleoperation systems that utilize these 
supervisory control techniques. Su and Luo developed 
an incremental algorithm for better translation of the in¬ 
tention and motion of operators into remote robot action 
commands [44.33]. 

The fast development of sensing and display tech¬ 
nology makes it possible to visualize robot and envi¬ 
ronment states in three-dimensional (3-D) displays or 
generate synthetic eco-centric views (a.k.a. third person 
views) (l<gf 11‘KtliM (. To achieve that, it often requires 
the robot to be equipped with multiple cameras and 
laser range finders to quickly reconstruct the remote 
environment [44.73,74]. Sometimes, the reconstructed 
sensory information can be superimposed on priorly 
known 3-D information to form an augmented reality. 
This kind of display can drastically increase telepres¬ 
ence and performance. 

Human Operator Input 

Most networked telerobotic systems only rely on 
mouses and keyboards for input. The design problem 
is what to click on in the interface. Given the fact that 
user commands can be quite different, we need to adopt 
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Fig.44.10a,b A web-based teleoperation system that al¬ 
lows a robot to capture a fast-moving rod. (a) User inter¬ 
face, and (b) system setup (after [44.28]) 

an appropriate interface for inputs; for example, inputs 
could be Cartesian XYZ coordinates in world coordi¬ 
nate system or robot configurations in angular joint 
configurations. 

For angular inputs, it is often suggested to use 
a round dial as a control interface, as illustrated in 
bottom left of Fig. 44.7 and the right-hand side of 
Fig. 44.9. For linear motion in Cartesian coordinate, ar¬ 
rows operated by either mouse clicks or the keyboard 
are often suggested. Position and speed controls are of¬ 
ten needed, as illustrated in Fig. 44.9. Speed control is 
usually controlled by mouse clicks on a linear progress 
bar for translation and a dial for rotation. 

The most common control type is the position con¬ 
trol. The most straightforward way is to click on the 
video image directly. To implement the function, the 
software needs to translate the 2-D click inputs into 
three-dimensional (3-D) world coordinates. To simplify 
the problem, the system designer usually assumes that 
the clicked position is on a fixed plane; for example, 
a mouse click on the interface of Fig. 44.6 assumes 
the robot moves on the X—Y plane. The combination of 
a mouse click on the image can also allow abstract task- 
level command. The example in Fig. 44.12 uses mouse 


clicks to place votes on an image to generate a command 
that directs a robot to pick up a test agent at the task level. 

44.4.4 Command Execution/State 
Generation 

When a robot receives a command, it executes the 
command and a new state is generated and transmit¬ 
ted back to the human operator. Flowever, commands 
may not arrive in time or may get lost in transmis¬ 
sion. Also, because users are often inexperienced, their 
commands may contain errors. Over the limited com¬ 
munication channel, it is impossible to ask the human to 
control the manipulator directly. Computer vision, laser 
range finder, local intelligence, and augmented-reality- 
based displays [44.74] are required to assist the human 
operator. 

Belousov et al. demonstrated a system that allowed 
a web user to capture a fast rod that is thrown at a robot 
manipulator [44.28]. The rod is on bifilar suspension, 
performing complicated oscillations. Belousov et al. 
designed a shared-autonomy control to implement the 
capture. First, an operator chooses the desired point for 
capture on the rod and the capture instant using a 3-D 
online virtual model of the robot and the rod. Then, the 
capturing operation is performed automatically using 
a motion prediction algorithm that is based on the rod’s 
motion model and two orthogonal camera inputs, which 
perceive the rod’s position locally in real time. 

This shared autonomy approach is often required 
when the task execution requires much faster response 
than the Internet can allow. Human commands have to 
remain at the task level instead of directing the move¬ 
ments of every actuators. The root of this approach can 
be traced back to the Tele-Autonomous concept pro¬ 
posed by Conway et al. [44.75] in 1990. In this paper, 
two important notions including time clutch and po¬ 
sition clutches are introduced to illustrate the shared 
autonomy approach. The time clutch disengages the 
time synchronization between the human operator and 
the robot. The human operator verifies his/her com¬ 
mands on a predictive display before sending a set of 
verified commands to remote robots. The robot can 
then optimize the intermediate trajectory proposed by 
the human operator and disengage the position corre¬ 
spondence, which is referred to as the position clutch. 
Recent work [44.76] uses the similar idea to guide load- 
haul-dump vehicles in the underground mines by com¬ 
bining human inputs with tunnel following behavior. 

44.4.5 Virtual Fixtures 

Due to time delay, lack of background, and possible ma¬ 
licious behavior, human errors are inevitably introduced 
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to system from time to time. Erroneous states may be 
generated from the incorrect commands. If unchecked, 
robots or objects in the environment may be damaged. 
Sometimes, users may have good intention but are not 
able to generate accurate commands to control the robot 
remotely. For example, it is hard to generate a set of 
commands to direct a mobile robot to move along the 
wall and maintain a distance of 1 m to the wall at the 
same time. 

Virtual fixtures are designed to cope with these 
challenges in teleoperation tasks. Proposed by Rosen¬ 
berg [44.77], virtual fixtures are defined as an overlay 
of abstract sensory information on a robot workspace in 
order to improve the telepresence in a telemanipulation 
task. To further explain the definition, Rosenberg uses 
a ruler as an example. It is very difficult for a human 
to draw a straight line using bare hands. However, if 
a ruler, which is a physical fixture, is provided, then the 
task becomes easy. Similar to a physical fixture, a vir¬ 
tual fixture is designed to guide robot motion through 
some fictitious boundaries or force fields, such as vir¬ 
tual tubes or surface, generated according to sensory 
data. The virtual fixtures are often implemented us¬ 
ing control laws [44.78,79] based on a virtual contact 
model. 

Virtual fixtures serve for two main purposes: avoid¬ 
ing operation mistakes and guide robots along the 
designable trajectories. This is also a type of shared 
autonomy that is similar to that in Sect. 44.4.4 where 
both the robot and the human share control in the sys¬ 
tem. Chapter 43 details the shared control scheme. It is 
worth noting that virtual fixtures should be visualized in 
the display to help operators understand the robot state 
to maintain situation awareness. This actually turns the 
display to augmented reality [44.80]. 

44.4.6 Collaborative Control 
and Crowd Sourcing 

When more than one human is sharing control of the 
device, command coordination is needed. According 
to [44.81], multiple human operators can reduce the 
chance of errors, cope with malicious inputs, utilize 
operators’ different expertise, and train new operators. 
In [44.82,83], a collaboratively controlled networked 
robot is defined as a telerobot simultaneously controlled 
by many participants, where input from each participant 
is combined to generate a single control stream. 

When group inputs are in the form of direc¬ 
tion vectors, averaging can be used as an aggrega¬ 
tion mechanism [44.84]. When decisions are distinct 
choices or at the abstract task level, voting is a better 
choice [44.53]. As illustrated in Fig. 44.11, Goldberg 
and Song develop the Tele-Actor sys- 



Fig. 44.11 Spatial dynamic voting interface for the Tele- 
Actor system: the spatial dynamic voting (SDV) interface 
as viewed by each user. In the remote environment, the 
Tele-Actor takes images with a digital camera, which are 
transmitted over the network and displayed to all partici¬ 
pants with a relevant question. With a mouse click, each 
user places a color-coded marker (a votel or voting ele¬ 
ment) on the image. Users view the position of all votels 
and can change their votel positions based on the group’s 
response. Votel positions are then processed to identify 
a consensus region in the voting image that is sent back 
to the Tele-Actor. In this manner, the group collaborates to 
guide the actions of the Tele-Actor (after [44.53]) 

tern using spatial dynamic voting. The Tele-Actor is 
a human equipped with an audio/video device and con¬ 
trolled by a group of online users. Users indicate their 
intensions by positioning their votes on a 320 x 320 
pixel voting image during the voting interval. Votes are 
collected at the server and used to determine the Tele- 
Actor’s next action based on the most requested region 
on the voting image [44.85]. 

Another approach to collaboratively control a net¬ 
worked robot is the employ a optimization frame¬ 
work. Song et al. [44.86, 87] developed a collabora¬ 
tively controlled camera that allowed many clients to 
share control of its camera parameters, as illustrated 
in Fig. 44.12. Users indicate the area they want to view 
by drawing rectangles on a panoramic image. The algo¬ 
rithm computes an optimal camera frame with respect 
to the user satisfaction function, which is defined as the 
frame selection problem [44.88, 89]. 

Recent work by Xu et al. [44.40, 90] further the 
optimization framework to p-frames that allows multi¬ 
ple cameras to be controlled and coordinated whereas 
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Fig.44.12a ,b Frame selection inter¬ 
face (after [44.86]). The user interface 
includes two image windows. The 
lower window (b) displays a fixed 
panoramic image based on the cam¬ 
era’s full workspace (reachable field 
of view). Each user requests a cam¬ 
era frame by positioning a dashed 
rectangle in (b). Based on these 
requests, the algorithm computes an 
optimal camera frame (shown with 
a solid rectangle), moves the camera 
accordingly, and displays the resulting 
live streaming video image in the 
upper window (a) 


human inputs can also be replaced by autonomous 
agents and other sensory inputs. These developments 
have been applied to a recent project, the Collabo¬ 
rative Observatory for Nature Environments (CONE) 
project [44.91], which aims to design a networked 
robotic camera system to collect data from the wilder¬ 
ness for natural scientists. 

One important issue in collaborative control is the 
disconnection between individual commands and the 
robot action, which may lead to the loss of situation 
awareness, less participation, and eventual system fail¬ 
ure. Inspired by engaging power in scoring systems in 
computer games, Goldberg et al. [44.92] design scoring 


44.5 Cloud Robotics 

As noted earlier, the term Cloud Robotics is increas¬ 
ingly common based on advances in what is now called 
Cloud Computing. Cloud Robotics extends what were 
previously called Online Robots [44.1] and Networked 
Robots [44.55,56]. Cloud computing provides robots 
with vast resources in computation, memory, program¬ 
ming. 


mechanism for the collaborative control architecture by 
evaluating the individual leadership level. The early 
results show great improvement in the group perfor¬ 
mance. Furthermore, the recent development of social 
media, such as Blog and Twitter, can also be employed 
in the collaborative control to facilitate user interaction 
in real time, which can make the system more engag¬ 
ing and effective. The resulting new architecture can be 
viewed as a crowd sourcing [44.42,93] type approach 
to networked robots that combines human recognition 
and decision making capabilities to robot execution at 
a different scale and depth than a regular teleoperation 
system. 


Here we review five ways that cloud robotics and 
automation can potentially improve robots and automa¬ 
tion performance: 

1. Providing access to global libraries of images, 
maps, and object data, eventually annotated with ge¬ 
ometry and mechanical properties. 


Part D | 44.5 




















Part D | 44.5 


1122 Part D 


Manipulation and Interfaces 


2. Massively parallel computation on demand for de¬ 
manding tasks like optimal motion planning and 
sample-based statistical modeling 

3. Robot sharing of outcomes, trajectories, and dy¬ 
namic control policies. 

4. Human sharing of open-source code, data, and de¬ 
signs for programming, experimentation, and hard¬ 
ware construction. 

5. On-demand human guidance (call centers) for ex¬ 
ception handling and error recovery. Updated infor¬ 
mation and links are available at [44.94]. 

44.5.1 Big Data 

The term Big Data describes data sets that are beyond 
the capabilities of standard relational database systems, 
which describes the growing library of images, maps, 
and many other forms of data relevant to robotics and 
automation on the Internet. One example is grasping, 
where online datasets can be consulted to determine ap¬ 
propriate grasps. The Columbia Grasp dataset [44.95] 
and the MIT KIT object dataset [44.96] are available 
online and have been widely used to evaluate grasping 
algorithms [44.97-100], 

Related work explores how computer vision can 
be used with Cloud resources to incrementally learn 
grasp strategies [44.102, 103] by matching sensor data 
against 3-D computer-aided drafting (CAD) models in 
an online database. Examples of sensor data include 

2- D image features [44. 104], 3-D features [44. 105], and 

3- D point clouds [44.106]. Google Goggles [44.107], 
a free network-based image recognition service for 
mobile devices, has been incorporated into a sys¬ 


tem for robot grasping [44.101] as illustrated in 
Fig. 44.13. 

Dalibard et al. attach manuals of manipulation 
tasks to objects [44.108]. The RoboEarch project stores 
data related to objects maps, and tasks, for applications 
ranging from object recognition to mobile navigation to 
grasping and manipulation (Fig. 44.14) [44.64]. 

As noted below, online datasets are effectively used 
to facilitate learning in computer vision. By leveraging 
Google’s 3-D warehouse, [44.109] reduced the need for 
manually labeled training data. Using community photo 
collections, [44.110] created an augmented reality ap¬ 
plication with processing in the cloud. 

44.5.2 Cloud Computing 

As of 2012, Cloud Computing services like Ama¬ 
zon’s EC2 elastic computing engine provide massively 
parallel computation on demand [44. 111]. Examples in¬ 
clude Amazon Web Services [44.112] Elastic Compute 
Cloud, known as EC2 [44.113], Google Compute En¬ 
gine [44. 114], Microsoft Azure [44. 115]. These provide 
a large pool of computing resources that can be rented 
by the public for short-term computing tasks. These ser¬ 
vices were originally used primarily by web application 
developers, but have increasingly been used in scien¬ 
tific and technical high-performance computing (HPC) 
applications [44.1 16-119]. 

Cloud computing is challenging when there are 
real-time constraints [44.120]; this is an active area of 
research. However, there are many robotics applications 
that are not time sensitive such as decluttering a room 
or precomputing grasp strategies. 



Fig. 44.13 System architecture for cloud-based object recognition for grasping. The robot captures an image of an object 
and sends via the network to the Google object recognition server. The server processes the image and returns data for 
a set of candidate objects, each with precomputed grasping options. The robot compares the returned CAD models with 
the detected point cloud to refine identification and to perform pose estimation, and selects an appropriate grasp. After 
the grasp is executed, data on the outcome is used to update models in the cloud for future reference (after [44.101]) 
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Fig. 44.14 RoboEarth architecture (after [44.64]) 


There are many sources of uncertainty in robotics 
and automation [44. 121]. Cloud computing allows mas¬ 
sive sampling over error distributions and Monte Carlo 
sampling is embarrassingly parallel, recent research 
in fields as varied as medicine [44.122] and particle 
physics [44.123] have taken advantage of the cloud. 
Real-time video and image analysis can be performed 
in the Cloud [44.109,124,125]. Image processing in 
the cloud has been used for assistive technology for 
the visually impaired [44.126] and for senior citi¬ 
zens [44.127]. Cloud computing is ideal for sample- 
based statistical motion planning under uncertainty, 
where it can be used to explore many possible per¬ 
turbations in object and environment pose, shape, and 
robot response to sensors and commands [44.128]. 
Cloud-based sampling is also being investigated for 
grasping objects with shape uncertainty [44.129,130] 
(Fig. 44.15). A grasp planning algorithm accepts as 
input a nominal polygonal outline with Gaussian un¬ 
certainty around each vertex and the center of mass to 
compute a grasp quality metric based on a lower bound 
on the probability of achieving force closure. 

44.5.3 Collective Robot Learning 

The Cloud allows robots and automation systems to 
share data from physical trials in a variety of envi¬ 


ronments, for example, initial and desired conditions, 
associated control policies and trajectories, and impor¬ 
tantly: data on performance and outcomes. Such data 
are a rich source for robot learning. 

One example is for path planning, where previ¬ 
ously generated paths are adapted to similar environ¬ 
ments [44.131] and grasp stability of finger contacts can 
be learned from previous grasps on an object [44.98]. 

The MyRobots project [44.132] from RobotShop 
proposes a social network for robots [44.133]: 

In the same way humans benefit from socializing , 
collaborating and sharing, robots can benefit from 
those interactions too by sharing their sensor infor¬ 
mation giving insight on their perspective of their 
current state. 

44.5.4 Open Source and Open Access 

The Cloud facilitates sharing by humans of designs 
for hardware, data, and code. The success of open- 
source software [44.134—136] is now widely accepted 
in the robotics and automation community. A primary 
example is ROS, the Robot Operating System, which 
provides libraries and tools to help software developers 
create robot applications [44.137, 138], ROS has also 
been ported to Android devices [44.139]. ROS has be- 



Fig. 44.15 A cloud-based approach to geometric shape uncertainty for grasping (after [44.129,130]) 
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Fig. 44.16 Suckerbot, designed by Tom Tilley of Thai¬ 
land, a winner of the $10 Robot Design Challenge (af¬ 
ter [44.140]) 


come a standard akin to Linux and is now used by 
almost all robot developers in research and many in 
industry. 

Additionally, many simulation libraries for robotics 
are now open-source, which allows students and re¬ 
searchers to rapidly set up and adapt new systems and 
share the resulting software. Open-source simulation 
libraries include Bullet [44.141], a physics simulator 
originally used for video games, OpenRAVE [44.142] 
and Gazebo [44.143], simulation environments geared 
specifically toward robotics, OOPSMR a motion¬ 
planning library [44.144], and Grasplt!, a grasping 
simulator [44.145]. 


Another exciting trend is in open-source hard¬ 
ware, where CAD models and the technical details 
of construction of devices are made freely avail¬ 
able [44.147,148]. The Arduino project [44.149] is 
a widely used open-source microcontroller platform, 
and has been used in many robotics projects. The 
Raven [44.150] is an open-source laparoscopic surgery 
robot developed as a research platform an order of 
magnitude less expensive than commercial surgical 
robots [44.151]. 

The Cloud can also be used to facilitate open 
challenges and design competitions. For example, the 
African Robotics Network with support from IEEE 
Robotics and Automation Society hosted the $10 Robot 
Design Challenge in the summer of 2012. This open 
competition attracted 28 designs from around the world 
including a winning entry from Thailand (Fig. 44.16) 
that modified a surplus Sony game controller, adapt¬ 
ing its embedded vibration motors to drive wheels and 
adding lollipops to the thumb switches as inertial coun¬ 
terweights for contact sensing, which can be built from 
surplus parts for US $8.96 [44.140]. 

44.5.5 Crowdsourcing and Call Centers 

In contrast to automated telephone reservation and 
technical support systems, consider a future scenario 
where errors and exceptions are detected by robots 
and automation systems, which then access human 
guidance on-demand at remote call centers. Human 
skill, experience, and intution is being tapped to solve 
a number of problems such as image labeling for 
computer vision [44.54,62,102,152], Amazon’s Me¬ 
chanical Turk is pioneering on-demand crowdsourcing 
that can draw on human computation or social com¬ 
puting systems. Research projects are exploring how 


Autonomous image acquisition Sparse 3-D reconstruction TURK: Outline opbjects TURK: Group by object 
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Fig. 44.17 A cloud robot system that incorporates Amazon’s Mechanical Turk to crowdsource object identification to 
facilitate robot grasping (after [44.146]) 





Networked Robots I 44.6 Conclusion and Future Directions 


this can be used for path planning [44.153], to de¬ 
termine depth layers, image normals, and symmetry 
from images [44.154], and to refine image segmenta¬ 


tion [44.155]. Researchers are working to understand 
pricing models [44.156] and apply crowdsourcing to 
grasping [44.146] (Fig. 44.17). 


44.6 Conclusion and Future Directions 


As this technology matures, networked robots will 
gradually go beyond university laboratories and find ap¬ 
plication in the real world. 

As mentioned earlier in Sects. 44.2.2 and 44.5, 
the new efforts in cloud robotics lead by Google and 
RoboEarth naturally bridge research and applications. 
The open source nature and ready-to-use APIs can 
quickly spread and deploy research results. Japan’s 
Advanced Telecommunications Research Institute In¬ 
ternational (ATR) Intelligent Robotics and Communi¬ 
cation Laboratory has also announced its networked 
robot project led by Norihiro Hagita (ATR). Its mis¬ 
sion is to develop network-based intelligent robots 
for applications such as service, medical, and safety. 
Hideyuki Tokuda (Keio University) chaired the Net¬ 
worked Robot Forum in Spring 2005, which promotes 
research and development (R&D) and standardiza¬ 
tion on network robots through activities to support 
awareness campaigns and verification experiments in 
collaboration among wide-ranging parties, which in¬ 
cludes over 100 industry and academic members. Ko¬ 
rea’s Ministry of Information and Communication has 
also announced the Ubiquitous Robotic Companion 
(URC) project to develop network-based intelligent 
robots. 

Networked robots have allowed tens of thousands of 
nonspecialists around the world to interact with robots. 
The design of networked robots presents a number of 
engineering challenges to build reliable systems that 
can be operated by nonspecialists 24 h a day, 7 days 
a week, and remain online for years. Many new research 
challenges remain: 

• New interfaces'. As portable devices such as cell¬ 
phones and tablet computers increase in computa¬ 
tion power, networked robotics should be able to 
adopt them as new interfaces. As computers be¬ 
comes increasingly powerful, they become capable 
of visualizing more sophisticated sensor inputs. De¬ 
signers of new interfaces should also keep track 
of new developments in hardware such as haptic 
interfaces and voice recognition systems. New soft¬ 
ware standards such as flash, extensible markup 
language (XML), extensible hyper text markup lan¬ 
guage (XHTML), virtual reality modeling language 


(VRML), and wireless markup language (WML) 
will also change the way we design interface. 

New interface technology arises as human com¬ 
puter interaction technology, mobile computing, 
and computer graphics areas progress. Recent 
progresses on brain-machine interaction explore 
the possibility of using brain wave, such as elec¬ 
troencephalography (EEG) signals, to control 
robot movements for ground robots [44.157] 
and unmanned aerial vehicles (UAV)s [44.158]. 
Gesture [44.159] and multitouch [44.160] are also 
used to generate control commands. Unlike the 
traditional mouse and keyboard interfaces, the 
new interfaces facilitate more natural interaction 
but suffers from precision issues, because these 
methods have large noise and require more research 
efforts in improving robustness and accuracy. 

• New algorithms'. Algorithms determine perfor¬ 
mance. Scalable algorithms that are capable of 
handing large amounts of data such as video/sensor 
network inputs and utilize fast-evolving hardware 
capability such as distributed and parallel compu¬ 
tation will become increasingly important in the 
networked robotics, especially in cloud robotics. 

• New protocols'. Although we have listed some 
pioneering work in changing the network en¬ 
vironment to improve teleoperation, there are 
still a large number of open problems such as 
new protocols, appropriate bandwidth alloca¬ 
tion [44.161], QoS [44.162], security, routing 
mechanisms [44.29, 163], and many more. Network 
communication is a very fast-evolving field. The 
incorporation/modification of network commu¬ 
nication ideas into networked telerobotic system 
design will continue to be an active research area. 
The common object request broker architecture 
(CORBA) or real-time CORBA [44.20, 21, 39,164, 
165] have great potential for networked robots. 

• New performance metrics'. As more and more 
robots enter service, it is important to develop 
metrics to quantify the performance of the robot- 
human team. As we are more familiar with metrics 
developed to assess robot performance or task 
performance [44.162], recent progresses on using 
the robot to assess human performance [44.166, 
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167] shed light on new metrics. Standardizing these 
metrics will also be an important direction. 

• Video for robotics: Another interesting observation 
is that all of existing video compression and trans¬ 
mission standards try to rebuild a true and complete 
representation of camera field of view. However, 
it might not be necessary or infeasible due to 
bandwidth limit for a networked robot [44.168]. 
Sometimes, a high level abstraction is sufficient. 
For example, when a mobile robot is avoiding 
an moving obstacle, all the robot needs to know 
is the speed and bounding box of the moving 
object instead of knowledge that whether this 
object is human or other robots. We might want to 


control the level of details in video perception and 
transmission. This actually imposes an interesting 
problem: we need a new streaming standard that 
serves for networked robots. 

• Applications : Recent successful applications 
include environment monitoring [44.43, 169], 
manufacturing [44.170,171], and infrastructure 
inspection and maintenance [44.172,173]. The 
fast development of networked robot systems is 
worldwide. Many new applications are emerging 
in areas such as security, inspection, education, and 
entertainment. Application requirements such as 
reliability, security, and modularity will continue to 
pose new challenges for system design. 
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A heterogeneous multiple-operator-multiple-robot system 

available from http://handbookofrobotics.org/view-chapter/44/videodetails/81 

Teleoperation of a mini-excavator 

available from http://handbookofrobotics.org/view-chapter/44/videodetails/82 
Tele-Actor 

available from http://handbookofrobotics.org/view-chapter/44/videodetails/83 
A multi-operator-multi-robot teleoperation system 

available from http://handbookofrobotics.org/view-chapter/44/videodetails/84 
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Until the mid 1960s, robots were only able to move in 
a predetermined workspace, the one they could reach 
from their firmly fixed base. Part E is about their con¬ 
quest of the whole space. Mobile Robotics started as 
a research domain in its own right in the late 1960s 
with the Shakey project at SRI. 2015 marks the 50th 
anniversary of this seminal project that had a lasting 
legacy such as A*. The seminal paper by N.J. Nils¬ 
son A Mobile Automaton: An Application of Artificial 
Intelligence Techniques at the International Joint Con¬ 
ference on Artificial Intelligence (IJCAI) 1969, already 
addressed perception, mapping, motion planning, and 
the notion of control architecture. Those issues would 
indeed be at the core of mobile robotics research for 
the following decades. The 1980s boomed with mo¬ 
bile robot projects, and as soon as it was necessary 
to cope with the reality of the real physical world, 
problems appeared that fostered novel research direc¬ 
tions, actually moving away from the original concept 
in which the robot was just an application of artificial 
intelligence (AI) techniques. This part addresses all the 
issues that, put together, are necessary to build and con¬ 
trol a mobile robot, except for the mechanical design 
itself. 

Navigation is the capacity of moving from one 
location to another arbitrarily distant one. To move 
efficiently, a robot needs to use appropriate represen¬ 
tations of its environment in order to plan and control 
its motions according to the presence of obstacles and 
terrain difficulties, and also to use environment fea¬ 
tures as landmarks for its localization. This is the topic 
of Chapter 45 which addresses the different repre¬ 
sentations used for indoors or outdoors environments, 
including topological maps and semantic attributes. 

However, building environment maps is usually 
achieved incrementally, as the robot discovers its envi¬ 
ronment while navigating in it. Hence partial percep¬ 
tions built from different positions, need to be fused 
together - taking into account sensing errors and un¬ 
certainties (see Chapter 5, Part A) to construct a con¬ 
sistent global map. This requires the robot to know 
these positions, and, because of motion inaccuracies, 


the transforms between them are uncertain. This re¬ 
quires to reference the positions to environment features 
which are only defined in the environment map itself. 
As a result, localization needs the map and mapping 
needs localization. Hence, localization and mapping are 
two interwound problems that must be solved simul¬ 
taneously. The solution to this problem is the topic of 
Chapter 46. 

Once environment maps are available, or during 
their construction, the robot has to plan its path as op¬ 
timally as possible to reach its targets while avoiding 
obstacles. Chapter 7 in Part A overviews the tech¬ 
niques for solving this central problem, which requires 
geometrical reasoning in the configuration space (CS) 
whose construction is complex. To avoid an explicit 
construction of the CS, probabilistic techniques proved 
the most efficient way to compute paths - at the price 
of optimality. However, the kinematic constraints of 
the robot’s locomotion system such as non-holonomy 
have to be taken into account. We see in Chapter 47 
in this part how control problems cannot be separated 
from geometry. This chapter addresses motion planning 
from the viewpoint of mobile robotics and introduces 
the tools from control theory and differential geometry 
that enable to tackle kinematics constraints. In addition, 
it provides an overview of local sensor-based methods 
that are used when the robot encounters unknown or 
mobile obstacles while moving. 

The rest of this part essentially focusses on various 
means of locomotion used to move in the environment: 
legged locomotion (Chapter 48), robots with wheels 
(Chapter 49), with tracks (Chapter 50), and underwa¬ 
ter and aerial robots (Chapters 51 and 52 respectively) 
which face the specific problems of three-dimensional 
(3-D) motion with environment perturbations. In these 
two last cases the close links between Robotics and 
Control are of course central. 

Finally, after explaining how one single robot 
moves in different environments. Part E concludes 
with Chapter 53 on the interaction and coordination of 
multiple mobile robot systems. 
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45. World Modeling 


Wolfram Burgard, Martial Hebert, Maren Bennewitz 


In this chapter we describe popular ways to rep¬ 
resent the environment of a mobile robot. For 
indoor environments, which are often stored us¬ 
ing two-dimensional representations, we discuss 
occupancy grids, line maps, topological maps, and 
landmark-based representations. Each of these 
techniques has its own advantages and disadvan¬ 
tages. Whilst occupancy grid maps allow for quick 
access and can efficiently be updated, line maps 
are more compact. Also landmark-based maps can 
efficiently be updated and maintained, however, 
they do not readily support navigation tasks such 
as path planning like topological representations 
do. 

Additionally, we discuss approaches suited 
for outdoor terrain modeling. In outdoor envi¬ 
ronments, the flat-surface assumption underling 
many mapping techniques for indoor environ¬ 
ments is no longer valid. A very popular approach 
in this context are elevation and variants maps, 
which store the surface of the terrain over a reg¬ 
ularly spaced grid. Alternatives to such maps are 
point clouds, meshes, or three-dimensional grids, 


The construction of models of the environment is cru¬ 
cial to the development of several applications of mobile 
robot systems. It is through these environment mod¬ 
els that the robot can adapt its decisions to the current 
state of the world. The models are constructed from sen¬ 
sor data as the robot discovers its environment. There 
are three challenges in constructing environment mod¬ 
els from sensor data. First, the models must be compact 
so that they can be used efficiently by other compo¬ 
nents of the system, such as path planners. Second, the 
models must be adapted to the task and to the type of 
environment. For example, modeling the environment 
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which provide a greater flexibility but have higher 
storage demands. 


as a set of planes is not relevant to a robot operating in 
natural terrain. In particular, this implies that a univer¬ 
sal representation for mobile robots in not possible and 
that we must instead choose from an array of different 
approaches. Third, the representation must accommo¬ 
date the uncertainty inherent to both sensor data and to 
the robot’s state estimation system. The latter point is 
particular important since environment models typically 
accumulate sensor readings over substantial distances 
into a common reference frame. Drift in position esti¬ 
mates are unavoidable and must be accounted for in the 
model representation and in its construction. 


Part E | 45 






















Part E | 45.1 


1136 Part E I Moving in the Environment 


45.1 Historical Overview 

Historically, work in this area concentrated first on 
robots operating in indoor environments. In that case, 
the models take advantage of the fact that the world 
can be represented as vertical structures on reference 
ground planes. This simplification can be used for rep¬ 
resenting the world as a two-dimensional (2-D) grid. 
Uncertainty in the measurements and in the robot’s pose 
can then be modeled by using probabilities of occu¬ 
pancy in the grid rather than binary occupied/empty 
flags. Another feature of indoor environments is that 
they are highly structured in that they contain primarily 
linear structures such as lines and planes. This obser¬ 
vations led to a second class of representations based 
on collections of points, lines, and planes to represent 
the environment. Here again, a lot of attention was paid 
to representing the uncertainty on the relative poses 
of these geometric elements, including a considerable 
amount of work in the 1980s on using Kalman filters 
and other probabilistic techniques. 

With continued progress on sensing (e.g., three- 
dimensional laser range scanners and stereo vision), and 
on mechanical and controls aspects of mobile robots 
systems, it became possible to develop mobile robot 
systems for operations in unstructured, natural terrain, 
motivated in part by planetary exploration and military 
applications. In these scenarios, it is no longer appropri¬ 
ate to project the data in a 2-D grid, and environments 
cannot be described adequately by a small vocabulary 
of geometric elements. Since, in most cases, it is still 
appropriate to assume (at least locally) that there is 
a reference ground plane, a natural representation is 
a two-and-a half-dimensional 2.5-D grid, in which each 
cell contains the elevation (and possibly other features) 
of the terrain at that location. Although they have been 
used extensively, the main challenges with such eleva¬ 
tion maps are that they are not compact representations 
and that it is difficult to incorporate uncertainty in the 
representation. Accordingly, much of the research has 
focused on designing efficient data structures and al¬ 
gorithms for elevation maps, for example, hierarchical 
representations. Recently, the uncertainty challenge has 
been tackled by using probabilistic variants of elevation 
maps. 

While elevation maps provide a natural represen¬ 
tation for many types of natural terrains, they cannot 
represent environments with vertical or overhanging 
structures. This limitation has become more evident in 


recent years as applications of mobile robots are in¬ 
creasingly demanding operation in urban environment 
(in which building walls and other structures cannot be 
represented by elevation maps) and the use of aerial 
data, which involves overhanging structures such as tree 
canopies. This has led to the development of represen¬ 
tations that are truly three-dimensional (3-D), such as 
point clouds, 3-D grids, and meshes. The two chal¬ 
lenges described above apply here as well, except that 
the situation is more complicated because of the in¬ 
creased complexity introduced by the third dimension. 
Current research includes efficient computation over 
3-D structures and probabilistic representations of 3-D 
data. 

Because of the large volume of data involved in all 
of these representations, it is important to be able to 
group the data into larger chunks corresponding to se¬ 
mantically meaningful parts of the environment. This 
can be done at different levels, depending on the ap¬ 
plication the environment. At the lowest level, work 
in this area involves classifying the points into classes 
that are relevant to navigation tasks (e.g., distinguish¬ 
ing between vegetation and ground, extracting walls, 
tree surfaces, etc.). At an intermediate level of repre¬ 
sentation, this part of the work involves extracting parts 
of the environments that are considered landmarks of 
interest for the navigation task (e.g., roads). Finally, 
the highest representation level involves extracting and 
representing objects in the environments (e.g., natural 
obstacles, or specific objects such as cars for operation 
in urban environments). 

All of these representations assume a static envi¬ 
ronment. In fact, many of the current applications of 
mobile robots call for operation in mixed environments 
in which the robot shares its environments with other 
moving agents, including other robots, people, and ve¬ 
hicles. Assuming that perception algorithms are able to 
detect and track individual moving objects in the envi¬ 
ronment, the challenge here is to insert this information 
into a representation that can be used by a planner. In 
this case, any one of the previous representations can 
be used as a foundation, but it needs to be extended to 
handle the temporal dimension by storing a history of 
the locations and states of the detected objects. Such 
representations include the trajectories of the detected 
objects and, in some cases, information on the future 
predicted trajectories of the objects. 
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45.2 Models for Indoors and Structured Environments 


45.2.1 Occupancy Grids 

Occupancy grid maps, which were introduced in the 
1980s by Moravec and Elfes [45.1], are a popular, prob¬ 
abilistic approach to represent the environment. They 
are an approximative technique in which we calculate 
for each cell of a discrete grid the posterior probability 
that the corresponding area in the environment is occu¬ 
pied by an obstacle. The advantage of occupancy grid 
maps lies in the fact that they do not rely on any prede¬ 
fined features. Additionally, they offer a constant-time 
access to grid cells and provide the ability to represent 
unknown (unobserved) areas, which can be important, 
for example, in exploration tasks. Their disadvantage 
lies in potential discretization errors and the high mem¬ 
ory requirements. 

Throughout this section we assume that the map m 
consists of a discrete, two-dimensional grid of L cells 
denoted as m\ ,..., m L . Given the sensory input zi-. t ob¬ 
tained by the robot at the corresponding positions x\- t , 
the occupancy grid mapping approach calculates a pos¬ 
terior probability p{m \ X\- t ,zi-.i). 

To keep the calculations tractable, the overall ap¬ 
proach assumes that the individual cells of the grid 
are independent, i. e., that the following equation 
holds 


p(m, | x 1:t ,zi:t) 

= "j | (l-p(m/ \x„z,)) p(mi) 

p(mi\x,,z t ) (1 -p(mi)) 

1 -p(mi | Zi:,-!)! -1 

X- 

p(m\xi :t -i,zv.t-i) _ 


(45.2) 


In practice, one often assumes that the prior p{mi) is 0.5 
so that the second factor in the product becomes 1 and 
therefore vanishes from the equation. 

Additionally, if we define 


Odds(.r) = P ^ X ] , (45.3) 

1 -p(x) 


the incremental updates can be computed using the fol¬ 
lowing equation 


Odds(m, | X\-_t, Zut) 

= Odds (mi | x,,z t ) ■ Odds(/w/) _1 

xOdds(m/ I Xi:t-\,Z\-.t-\) ■ (45.4) 


To recover the occupancy probability from the Odds 
representation given in (45.4), one can use the follow¬ 
ing formula, which can easily be derived using (45.3) 


L 

p(m | xi :t9 Z\ :t ) = ]~] p(mi \ x\ :u zv.t) . (45.1) 

/=! 


Note that this assumption is rather strong. It basically 
states that any information about the occupancy of one 
cell does not tell us anything about its neighboring cells. 
In practice, we often find objects that are larger than 
the individual grid cells, like doors, cabinets, chairs etc. 
Thus, if we know that one cell is occupied, for each 
of its neighboring cells the probability raises that it is 
occupied. Despite this fact, occupancy grid maps have 
been successfully applied in numerous installations of 
mobile robots and have been proven to be a powerful 
tool that supports various navigation tasks such as lo¬ 
calization and path planning. 

Because of the independence assumption expressed 
by (45.1) we can concentrate on the estimation of the 
occupancy probability of the individual cells m/ in m. 
Under additional independence assumptions one can fi¬ 
nally arrive at the following formula for calculating the 
occupancy probability p(m/ \ X\ :t , zi-.i) that cell w; is oc¬ 
cupied given the prior probability p(m /1 x\- t —\,ziu—\) 
and the new observation z t observed at position x, 


Odds(.y) 

^ 1 + Odds(jc) 


(45.5) 


It remains to describe how to compute the oc¬ 
cupancy probability p(m /1 x t , z t ) of a grid cell given 
a single observation z, and the corresponding pose x t of 
the robot. This quantity strongly depends on the sensor 
of the robot and has to be defined individually for each 
type of sensor. Additionally, the parameters of these 
models have to be adapted according to the properties of 
each sensor. Let us assume that the function dist(x r , mi) 
refers to the distance between the sensor at pose x t and 
the center of the cell ni[. Let us first assume that we only 
need to consider the optical axis of the sensor cone as is, 
for example, the case for laser beams sent out by a laser 
range finder. Then, p(mi \ x t , zi) can be formulated as 


p(m | x,,z t ) = 


' 

pprior, Zt is a maximum range 
reading 

' Pprior, m l is not covered by z t 
Pace, \z t - dist(jq, w ; )| < r/2 

Pfree > Zt > distfc, m/) 


(45.6) 
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Fig.45.1a,b Occupancy probability introduced by a single ultrasound measurement of (a) z = 2.0m and (b) z = 2.5 m 
(after [45.2]) 


where r is the resolution of the grid map. Obviously, it 
must hold 0 < p free < Pprior < Pace < 1 • 

If a sonar sensor is used, the sensor model is slightly 
more complicated, since the sensor is not a beam sen¬ 
sor and the observations are more noisy than the ones 
of a laser range finder. To deal with these properties, 
one can use a mixture of three functions to express the 
model. First, the influence of an observation (which is 
represented by the difference between p P n 0t and p occ as 
well as between p pT j or and Pf ree ) decreases with the mea¬ 
sured distance. Second, the proximity information of 
a sonar is substantially affected by noise. This can be 
considered by employing a piecewise linear function to 
model a smooth transition from pf Ke to p occ . Finally, the 
sonar sensor should not be modeled as a beam sensor, 
since it sends out a conic signal. The accuracy of an ob¬ 
servation decreases with the angular distance between 
the cell under consideration and the optical axis of the 
observation. This is expressed by the derivation from 
the prior and is typically modeled using a Gaussian with 
zero mean. Therefore, it is maximal along the optical 
axis and decreases the bigger the angular distance form 
the optical axis is [45.2]. 

Two examples of resulting models are depicted in 
Fig. 45.1, which shows two three-dimensional plots of 
the resulting occupancy probabilities for a measurement 
of 2 m (Fig. 45.1a) and 2.5 m (Fig. 45.1b). In this figure, 
the optical axis of the sensor cone was identical with 
the X-axis and the sensor was placed in the origin of the 
coordinate frame. As can be seen, the occupancy prob¬ 
ability is high for cells whose distance to x t is close to 
z t . It decreases for cells with shorter distance than z t as 
well as with increasing values of the angular distance. 

Figure 45.2 depicts the mapping process for a se¬ 
quence of observations recorded with an iRobot B21r 


robot. The first row shows how a map was built from 
a sequence of previous ultrasound scans. Afterwards the 
robot perceived a series of 18 ultrasound scans, each 
consisting of 24 measurements. The occupancy proba¬ 
bilities for these 18 scans are depicted in rows 2—7. The 
occupancy probability grid obtained by integrating the 
individual observations into the map is shown in the last 
row of this figure. As can be seen, the belief converges 
to a representation of the corridor structure in which the 
scans where recorded. A typical resulting map obtained 
for an indoor environment is depicted in Fig. 45.3. 

45.2.2 Line Maps 


The representation of the environment by line models is 
a popular alternative to the grid-based approximations 
described above. Line models have several advantages 
over these nonparametric representations. They require 
substantially less memory than grids and therefore scale 
better with the size of the environment. They further¬ 
more are more accurate since they do not suffer from 
discretization problems. In this section, we consider the 
problem of calculating lines that provide the approxi¬ 
mation of a set of range points by a line. If the data 
points are given by n pairs (x;,y,) of Cartesian coordi¬ 
nates, the line that minimizes the squared distances to 
all points can be calculated in closed form according to 


tan 2 </> 


~ 2 E/(*-■*/)(>- yd 

£i[(y—yO 2 —(*—*>) 2 ] ’ 


r = x cos 0 + y sin <p , 


(45.7) 

(45.8) 


where x = - and y = ^ E, V;. In these equations, 
r is the normal distance of the line from the origin and <p 
is angle of the normal. 
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Unfortunately, there is no closed-form solution for 
situations in which the data points are generated by 
multiple linear structures. In such a situation two prob¬ 
lems arise. First one has to answer the question how 
many lines there are, and second one has to solve a data 
association problem, which is to find the assignment of 
the data points to the individual lines. Once the num¬ 
ber of lines and the associations are known, we can 
apply (45.7) and (45.8) to calculate the individual line 
parameters. 

There is a popular approach to solve this problem 
for range scans with multiple linear structures. This ap¬ 
proach, which goes back to the work of Douglas and 
Peucker [45.3], is also known as the split-and-merge 
algorithm. Its key idea is to recursively subdivide the 
point set into subsets that can be more accurately ap¬ 
proximated by a line. The approach starts with the line 
calculated from all data points and determines the data 
point with maximum distance from this line. If this 
distance is below a given threshold, the algorithm termi¬ 
nates and provides the fitted line as output. Otherwise, 
it calculates the point with the largest distance from the 
line that connects the first and the last point. This point 
is the so-called splitting point. It then divides the point 
set into two subsets, one containing all points from the 
first up to the splitting point and one containing all 
points after the splitting point. The algorithm is then re¬ 
cursively applied to the two subsets. Figure 45.4 shows 
a visualization of the procedure. 

Whereas the split-and-merge algorithm is quite ef¬ 
fective and efficient, it is not guaranteed that the result¬ 
ing model is actually optimal, i. e., is the model that 
minimizes the squared distances of all data points. One 
approach that is able to find such a model is based on 
the expectation maximization (EM) algorithm, which in 
the application described here can be regarded as a vari¬ 
ant of the fuzzy k-means clustering algorithm. Let us 
suppose that the number m of lines in the model 8 is 
known. Let us furthermore suppose that the likelihood 
of a data point z = (x , y ) given the model 9 consisting 
of the line set {9 \,..., 8 m } is defined as 


P(z\8) = 


\flna 


exp 


1 d(z, 9 k ) 2 

2 a 2 


(45.9) 


where a is the standard deviation of the measurement 
noise and 9\ is the line for which Euclidean distance 
d(z, 9k) to z is minimal. 

The goal of the EM algorithm is to generate an it¬ 
erated sequence of models of increased likelihood. To 
achieve this, one introduces so-called correspondence 
variables Cy e {0,1} that specify to which linear compo¬ 
nent of the model each point belongs. Since the correct 
values of these assignment variables are unknown, one 



Fig. 45.2 Incremental mapping in a corridor environment. The up¬ 
per left image shows the initial map and the lower one shows the 
resulting map. The maps in between are the local maps built from 
the individual ultrasound scans perceived by the robot (after [45.2]) 



Fig. 45.3 Occupancy grid map obtained from ultrasound 
data (after [45.2]) 


estimates a posterior about their values. Let Oj be a com¬ 
ponent of the model and z, be a measurement. Then the 
expectation about c,y, i. e., that measurement i belongs 
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Fig. 45.4 Split-and-merge algorithm applied to a point set 


to line j is computed in the E-step as 

E[dj | Zi\ = p{Cij | 6j, Zi ) , (45.10) 

= ap(zi | dj, 6j)p(cij | Qj) , (45.11) 

= tt'p{zj | 0j) . (45.12) 

In the M-step, the algorithm then computes the param¬ 
eters of the model by taking into account the expecta¬ 
tions computed in the E-step 


= arg min ^ ^ E[c tJ \ 6j , Zi]d 2 (zi, O') . 
9 J i i 


(45.13) 


Given a fixed variance a for all data points in z we can 
determine the most likely model in closed form accord¬ 
ing to the following equations, which are probabilistic 
variants of (45.7) and (45.8) that take into account the 
data association uncertainty given by the expectations 
calculated in the E-step 

2 . _ -2Ef £ Mfyzf](*-*i)(y-yf) 

an " 1 'Li E [ c ij\Oj’Zi\[(y-yd 2 -(x-x i ) 2 ]' 

(45.14) 

rj = x cos (pj + y sin cpj . (45.15) 

Here x and y are computed as 


- = H, E [cij\9j,Zi]xi _ = Y.iE[cij\9},Zi\yi 
Y,i E [cij\0j’Zi] ' y Hi E [cn\9j,Zi] 

(45.16) 

Figure 45.5 depicts a line map extracted from 311 823 
data points using the EM-based approach. In this ex¬ 
ample, the model consists of 94 lines. One approach to 



Fig. 45.5 Line map consisting of 94 lines generated for 
311 823 range points generated with the EM-based ap¬ 
proach (after [45.4]) 


determine the optimal number of lines is to utilize the 
Bayesian information criterion [45.4]. 

45.2.3 Topological Maps 

In contrast to the previously discussed representations, 
which mainly focus on the geometric structure of the 
environment, topological representations have also re¬ 
ceived significant attraction. One of the pioneering ap¬ 
proaches to topological mapping, which has been pre¬ 
sented in 1988, is the work by Kuipers and Byun [45.7]. 
In this approach, the environment is represented by 
a graph-like structure, in which the nodes are locally 
distinguishable places and the nodes are travel edges 
along which the robot can move between the places 
h I Here, distinctive places are identified 
according to the distance to nearby objects. The dis¬ 
tinctive places were been defined by Choset and Na- 
gatani [45.8] as the meet-points in the generalized 
Voronoi diagrams, i. e., the points with an out-degree 
of three or more. A generalized Voronoi diagram is 
the set of points equidistant from the closest two or 
more obstacle boundaries. Generalized Voronoi dia¬ 
grams are a very popular representation as they can be 
considered as road-maps with a high correspondence 
to the topological structure of the environment. They 



Fig. 45.6 Example of a generalized Voronoi graph (af¬ 
ter [45.5]) 
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Fig. 45.7 (a) Mobile robot mapping 
a set of stone rocks. The image 
(b) depicts the path and the estimated 
landmark positions. The manually 
determined positions of the rocks are 
marked by circles (after [45.6]) 


have been used extensively for path planning. To plan 
a path from a starting position to a goal point in the 
environment, all the robot has to do is to first plan 
a path to the generalized Voronoi graph, than along 
the generalized Voronoi graph, and then from the gen¬ 
eralized Voronoi graph to the goal point [45.8]. Fig¬ 
ure 45.6 shows an example generalized Voronoi graph 
for an indoor environment. Note that in this figure 
only those parts of the graph that can be traversed 
by the robot without colliding with objects are dis¬ 
played. 

45.2.4 Landmark-Based Maps 

For environments with locally distinguishable features, 
landmark-based maps have been extensively used. If we 
assume that the position of the robot is always known, it 
simply remains to maintain an estimate about the posi¬ 
tions of the individual landmarks over time. In a planar 
environment, m is represented by K two-dimensional 
Gaussians with mean /z* and covariance one for 


each landmark. If a linearized version of the perception 
problem is given, the individual Gaussians can be up¬ 
dated using the equations for the extended Kalman filter. 
Note that, compared to the use of the extended Kalman 
filter EKF for landmark-based simultaneous localiza¬ 
tion and mapping (SLAM), where we needed a 2 K + 3- 
dimensional state vector (3 dimensions for the position 
of the robot and 2 K dimensions for the positions of the 
individual landmarks), we only need K two-dimensional 
Gaussians to represent the entire map since the robot 
pose is known. This property, for example, has been uti¬ 
lized in the FastSLAM algorithm [45.6], Over the last 
few years, the graph-based SLAM paradigm has be¬ 
come a popular approach for estimating landmark maps 
due to their ability of relinearizing the motion and the 
measurement functions (for more details see Chap. 46). 

Figure 45.7a shows a robot equipped with a SICK 
laser range finder that maps the position of rocks. The 
right image depicts the path of the vehicle as well as 
the manually determined and automatically estimated 
landmark positions. 


45.3 World and Terrain Models for Natural Environments 


From a survey of environment modeling [45.9] with an 
emphasis on probabilistic techniques for indoor envi¬ 
ronments, a taxonomy can be created along several axis: 
metric versus topological versus semantic, robot-centric 
versus world-centric, or application-based. We choose 
to first review purely geometric models (elevation grid, 
3-D grid, meshes), then geometric models with low- 
level attributes (cost maps), and then models with richer 
semantic attributes, to finish with heterogeneous and hi¬ 
erarchical models. 

45.3.1 Elevation Grids 

Assuming that the terrain can be represented as a func¬ 
tion h =f(x,y), where x and y are the coordinates of 


a reference place and h is the corresponding eleva¬ 
tion. A natural representation is a digital elevation map 
which stores the value of h at discrete locations (jc;,y,). 
Because they are simple data structures and can be gen¬ 
erated from sensor data in a relatively straightforward 
manner, elevation maps have been used extensively for 
mobile robots operating in natural environments with 
no vertical surfaces or overhangs (e.g., for planetary ex¬ 
ploration scenarios [45.10]). Several issues need to be 
addressed when using elevation maps for mobile robots. 

A regularly sampled grid is appropriate when the 
sensor data is roughly uniformly distributed on the ref¬ 
erence plane. This is the case, for example, with aerial 
data. For ground robots, however, the distribution of the 
data on the reference ground plane varies dramatically 


Part E | 45.3 





Part E | 45.3 


1142 Part E I Moving in the Environment 



Fig. 45.8 Example elevation map built by accumulating 3-D data 
from a range sensor 

because of the small incidence angles with the refer¬ 
ence plane. This can be addressed by using variable-size 
cells instead of regularly sampled cells. In that case, 
the distribution of the cells on the reference plane is 
designed to approximate the distribution of points that 
would be measured by the sensor on this reference 
plane. Such nonuniform representations are useful pri¬ 
marily in cases in which the map is referenced to the 
current position of the robot [45.11] (Fig. 45.9). Fre¬ 
quent resampling is needed if the map is expressed in 
a global reference frame. 

Irrespective of the sampling scheme used to con¬ 
struct the reference grid, the density of data in the grid 
varies due to local self-occlusions of the terrain surface 
and to mismatches between the resolution of the pro¬ 
jected sensor data and the resolution of the grid. This 
can be problematic because, from the point of view of 
a planner, the elevation map would appear as a scat¬ 
tering of cells with elevation data and cells with no 
evidence (Fig. 45.8). Various schemes have been pro¬ 
posed to remedy this problem. The basic idea is to 
estimate the elevation values of the empty cells by in¬ 
terpolating across the cells with know elevation values. 
This has to be done with great care to avoid filling 
up regions that correspond to parts of the environment 



Fig. 45.9 Example of a robot-centric map with variable 
size (after [45.11]) 


occluded by the terrain (range shadows ) in which ele¬ 
vation data should not be inferred. Such mistakes can be 
catastrophic since a planner may generate paths through 
areas that are entirely unknown. One general approach 
to this problem is to use surface interpolation tech¬ 
niques including a term that allows for discontinuities in 
the resulting surfaces [45.12]. An alternative approach 
is to use the visibility constraints induced by the known 
geometry of the sensor in order to estimate plausible 
values of elevation at each cell [45.13, 14]. 

A difficulty in interpolating techniques is to explic¬ 
itly take into account the sensor uncertainty. In partic¬ 
ular, it is difficult to account for the varying resolution 
of the sensor as a function of range in a fixed-resolution 
grid. An alternative is to use multiple grids at differ¬ 
ent resolutions. The appropriate resolution to be used 
at a particular point ( x , y) can be decided based on the 
range from ( x,y ) to the closest sensor location to re¬ 
trieve the value at ( x , y) from the appropriate map for 
that resolution. Generally, coarser map resolutions are 
used for longer ranges. This approach eliminates the 
gaps in the map due to undersampling at long range 
from the sensor by using the optimal resolution based 
on sensor geometry [45.15]. 

Uncertainty in the sensor measurement is expressed 
most naturally with respect to the sensor frame, for ex¬ 
ample, by expressing the uncertainty in the direction 
of the measurement. As a result, converting such un¬ 
certainty models to elevation maps is difficult because 
a distribution in the direction of the measurement maps 
to a distribution in the reference plane, not a distribu¬ 
tion on the elevation value at a particular grid point. 
When updating a cell based on sensory input, one has 
to take into account that the uncertainty in a measure¬ 
ment increases with the distance measured due to errors 
in the tilting angle. A popular approach to estimate the 
height h at location (x, y) is to use a Kalman filter. If 
we assume that a is the standard deviation of the cur¬ 
rent measurement h in the vertical direction in (x,y) 
and oy_i is the standard deviation of the current esti¬ 
mate or h t — 1 , we can apply the following equations to 



Fig. 45.10 The standard deviation of a height measure¬ 
ments can be modeled to depend linearly on the distance 
of the grid cell to the sensor 
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a) b) c) d) 



Fig.45.11a-d Different versions of digital elevation maps (after [45.16]): (a) scan (point set) of a bridge, (b) a standard elevation 
map computed from this data set, (c) an extended elevation map, which correctly represents the underpass under the bridge, and 
(d) a multilevel surface map that correctly represents the height of the vertical objects 


obtain the new estimate h, with standard deviation a t . 


h, = 


a 2 h,—i + o 2 _\h 

°f—1 + ° 2 


o? = 


of_l+CT 2 


(45.17) 

(45.18) 


One possible solution to deal with varying uncertain¬ 
ties is to use a model in which the standard deviation 
of the height of a measurement increases linearly with 
the length of the range measurement, as indicated in 
Fig. 45.10. 

Vegetation cover is another source of error in the 
recovery of terrain elevation by totally or partially oc¬ 
cluding the ground surface from the vehicle sensors. 
Online learning techniques can address this issue by 
predicting ground elevation ahead of the vehicle based 
on past appearance observations of similar terrain and 
vehicle state over the terrain [45.17], 

Intermediate representations between 2-D elevation 
maps and full 3-D representations, addressed below, are 
extended elevation maps [45.18] and so-called multi¬ 
level surface maps [45.16]. Such approaches are specif¬ 
ically useful in structured terrain with vertical objects 
or overhangs such as bridges (Fig. 45.1 1). 


45.3.2 3-D Grids and Point Sets 


Elevation maps as described above assume a reference 
direction. In many cases, that assumption is violated. 
An alternative is to represent the data directly in 3-D 
without projecting it on a reference 2-D plane. The ad¬ 
vantage of doing this is that all the sensor data can 
be preserved in its original distribution and that there 
is no restriction on the geometry of the environment. 
Figure. 45.12 shows an example representation using 
3-D points. The drawbacks of SLAM systems oper¬ 
ating on such point clouds [45.19-21] is that neither 
free space nor unknown areas can be modeled and that 
sensor noise and dynamic objects cannot be dealt with 
directly. Another problem is that very large sets of 3-D 
points are difficult to handle efficiently. The point cloud 
library [45.22] addresses this problem by providing an 


octree-based implementation to store and address large 
point clouds in an efficient manner. 

Data structures based on dynamic 3-D grids can 
also be used for this purpose [45.26]. An interesting 
characteristic of these representations is that they en¬ 
able the computation of cost evaluated from the true 
local 3-D distribution of the data (as opposed to costs 



Fig.45.12a,b Example of mapping and classification of 
3-D point sets: (a) 3-D data and classification result (green 
= vegetation, red = surfaces, blue = lines; the saturation 
of the color is proportional to the confidence in the classi¬ 
fication result); (b) map accumulated over a large number 
of scans 
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Fig.45.13a,b Examples of volumetric maps for a natural environment. Whereas (a) shows a map of an underwater cave 
(after [45.23]), (b) depicts a terrain map obtained from a radar (after [45.24]) 



Fig. 45.14 Resulting octree-based representation of an outdoor environment at 0.2 m resolution (size of the scene: 
292 m x 167 m x 28 m) (after [45.25]). For clarity, only occupied volumes are shown with height visualized by a color 
(gray scale) coding 


computed from a local surface z =f(x,y)). This is im¬ 
portant, for example, in environments with vegetation 
scatter, which cannot be modeled as a surface. 

Extensions of occupancy grids to dense 3-D oc¬ 
cupancy grids have been used successfully to model 
natural environments using a radar [45 .24] . For more ef¬ 
ficiently maintaining 3-D representations of even large- 
scale environments, also tree-based representations can 
been employed. In particular, octree-based 3-D repre¬ 
sentations have been employed, for example, to map 
an underwater cave from several sonars mounted on 
an underwater vehicle [45.23]. Example maps for these 
methods are presented in Fig. 45.13. 

The OctoMap approach [45.27] uses octrees and 
supports a compact memory representation, multireso¬ 


lution queries, and probabilistic occupancy estimates. 
The volumetric representation of space allows for ex¬ 
plicitly representing free space and unknown areas. 
To exploit hierarchical dependencies in the environ¬ 
ment it can be extended to maintain a collection of 
submaps in a tree-structure, where each node represents 
a subspace of the environment [45.28]. Figure. 45.14 
and l<sz>M4l'RiTr»l show an example outdoor map ac¬ 
quired from laser data and built by OctoMap. The 
OctoMap framework has been used successfully for 
various tasks including autonomous navigation with air 
vehicles [45.29], humanoid navigation in multilevel in¬ 
door environments [45.30], and mobile manipulation in 
cluttered environments [45.31, 32] where volumetric in¬ 
formation is necessary to plan paths. 
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Fig.45.15a-c Two views (b-c) of the mesh representation of a map (a) obtained by scanning an environment.The meshes 
are reduced by a factor of 10 from the initial data 


In the approaches mentioned above, the data sets 
are accumulated, sometimes probabilistically, into dis¬ 
crete environment volumes. A different approach is to 
represent the environment with discrete sensor samples 
integrated probabilistically into a metric map [45.33]. 
Such an approach is different from occupancy grids in 
several aspects: the storage requirement and resolution 
is adaptive instead of fixed and the potential for exten¬ 
sibility and scalability is higher. 

45.3.3 Meshes 

As described above, elevation maps are compact and 
easy to implement but they are restricted to a particular 
class of terrain; at the other extreme, using 3-D repre¬ 
sentations directly is more general but it is also more 
expensive computationally, and it does not represent ex¬ 
plicitly surface continuity. A compromise is to represent 
the map by a mesh. This approach is attractive because 
it can, in principle, represent any combination of sur¬ 
faces. It is also a compact representation in that, even 
though the size of the mesh may be initially very large, 
efficient mesh simplification algorithms exist in the lit¬ 
erature [45.34] and can be used to reduce the map to 
a small number of vertices. 

The key issue with meshes is that extraction of 
the correct surfaces from raw data can be difficult in 
complex environments. In practice, the data may be 
corrupted by sensor noise and by random clutter from 
other sources, such as vegetation, which cannot be rep¬ 
resented as a continuous surface. In addition, it is nec¬ 
essary to precisely detect discontinuities in the data so 
that disconnected pieces of surface do not become acci¬ 
dentally linked in the mesh formation process [45.18]. 
Figure 45.15 shows typical mesh representations of an 
urban environment. 

In some applications, the objective is to produce 
terrain models not to support autonomous navigation 
but to produce a model to be visualized to a human. 


City planning is a typical application. Examples include 
producing 3-D texture models of urban environments 
using videos from a camera mounted on a moving vehi¬ 
cle equipped with an inertial navigation system [45.35] 
f|4S£>Miljlitiil) s using two laser scanners in push-broom 
mode, one for mapping and the other for localiza¬ 
tion [45.36], with a camera coregistered with the map¬ 
ping laser, and collecting high-resolution laser data and 
imagery to produce geometric and photometric correct 
3-D models of buildings [45.37]. Figure 45.16 depicts 
examples of textured meshes representing urban terrain. 

45.3.4 Cost Maps 

The most direct use of elevation maps is to compute 
traversability costs at each cell of the grid. The costs 
are computed by comparing the local terrain shape with 
a kinematic model of the robot. Several approaches 
have been proposed for computing the costs depending 
on the exact vehicle model; for example, a cost can be 
computed by considering the local slope and 3-D tex¬ 
ture of the terrain [45.38]. More involved models take 
into account a detailed model of the robot dynamics. In 
that case, different costs are generated for different pos¬ 
sible robot speeds and path curvatures. The cost grid is 
used in a minimum-cost planner (I<S3>JEM!EIM). Since 
the terrain map is updated continuously as the robot tra¬ 
verses the environment, it is also necessary to update the 
costs continuously as new data arrives. Consequently, it 
is important that the planner is able to support changing 
costs without needing to process the entire grid every 
time new data is inserted. 

An example of such a combination of grid repre¬ 
sentation and dynamic planner is the D* system, which 
uses a version of A* that fully supports dynamic grid 
updates [45.39 — 41]. Whenever a grid cell (or a group of 
grid cells) is updated, the planner makes minimal up¬ 
dates to its internal representation so that the optimal 
path can be updated quickly. 
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Fig.45.16a-c Examples of urban terrain modeled as a textured mesh (a) (after [45.35]), (b) (after [45.37]), and (c) 
(after [45.36]) 


Defining the exact relation between the costs and 
the elevation values stored in the grid can be quite 
difficult. In fact, beyond the limiting cases of terrain 
with large elevation gradient or large slope, there is lit¬ 
tle guidance as to why a particular part of the terrain 
may be more traversable than another as it depends 
critically on the exact configuration of the robot. For 
this reason, recent work has focused on deriving cost 
maps directly from observations, rather than by using 
handcrafted algorithms. One approach involves learn¬ 
ing the best weights to combine a set of predefined 



Fig. 45.17 Example of online learning of cost (af¬ 
ter [45.42]). The vehicle trajectory is represented in red. 
As the vehicle progresses, note the change in cost over all 
the map. The darker the appearance, the lower the cost 


costs. Another approach uses costs computed from 
other sources for inferring how to determine the costs 
on the terrain currently seen by the robot; for exam¬ 
ple, data from overhead imagery can be used to predict 
the traversability costs that should be used for a ground 
robot [45.42] (Fig. 45.17) or for learning to map local 
elevation distributions to costs values by analyzing ac¬ 
tual robot trajectories through the terrain. Similar online 
learning approaches are used to predict terrain rough¬ 
ness [45.43], terrain slippage [45.44], or traversability 
in general [45.45, 46] to be used in a cost map. 

45.3.5 Semantic Attributes 

The representations described above are concerned only 
with storing the data in a way that is compact and that 
can be used to estimate drivability costs. Often it is 
necessary to reason with high-level knowledge about 
the environment, e.g., the location and types of ob¬ 
jects around the robot or the type of terrain (vegetation, 
mud, wall) in the environment. For convenience we 
will refer to this type of information as semantic at¬ 
tributes attached to different parts of the environment. 
There are several different ways to approach the prob¬ 
lem of generating and representing semantic attributes. 
One possible direction is to approach the problem as 
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one of extracting landmarks from the environment. As 
for their indoor counterparts, outdoor environments can 
also be modeled using landmarks. Landmarks are de¬ 
fined broadly as scene elements easy to detect, salient 
in their surroundings, and easy to recognize. They can 
be specific objects, such as rocks [45.47], tree trunks in 
forest [45.48], or locations with a distinct appearance 
signature in 2-D or 3-D, for urban terrain [45.49, 50] or 
natural environments [45.51]. Landmarks can be used 
to produce topological representations of the environ¬ 
ment or used in conjunction with metric maps. Recent 
efforts involve learning a compact representation of en¬ 
vironments without defining a priori what a landmark 
should be [45.52, 53] or using statistical learning tech¬ 
niques [45.54] to extract new features from images. 
Nonlinear dimension-reduction techniques are used to 
obtain the representation and the results were demon¬ 
strated onboard ground and aerial vehicles (Fig. 45.18). 

Another view of the problem is as a classification 
and grouping problem. Indeed, the map representations 
described in the earlier sections are low-level represen¬ 
tations in that they do not attempt to group the data 
points into larger structures that are coherent with re¬ 
spect to geometry, terrain type, or semantic content. In 
practice, one would like to abstract the data into larger 
units that can be used by a planner or transmitted to 
another robot or an operator; for example, in an urban 
environment, one would like to group the parts of the 
data that belong to planes corresponding to pieces of 
walls. One such example is presented in Fig. 45.19. 

One approach is to group 3-D points into com¬ 
ponents based on low-level classification and feature 
detection [45.58]. The shape of these component is an¬ 
alyzed further to discriminate between various natural 
object components (tree trunks, branches) and man¬ 
made obstacle (wires). Geometric primitives are then 
fitted to the components (mesh for the ground surface, 
ball tree for the vegetation, and cylinder to branches) to 
produce a compact high-level geometric description of 
the terrain. 

Local terrain classification can be achieved by com¬ 
puting local features from the map and classifying each 
element of the map in different classes of terrain. This 
can be done from an elevation map, in which case the 
elements are the cells of the elevation map, or from 
a point cloud representation in which case the elements 
are 3-D locations. Given a location in the map x, a fea¬ 
ture vector is V(x) is computed in a neighborhood N(x) 
of x and a classifier f(V) returns the type of terrain at x. 
Features that have been used in the past are statistics of 
the distribution of the map data around each element, 
such as the slope and distribution of elevation [45.59], 
and the second-order moment of the distribution of 
the 3-D points in a neighborhood [45.58]. The ter- 
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Fig. 45.18 Sample image and low-dimensional embedding of ran¬ 
domly sampled high-dimensional image patches (after [45.53]) 



Fig. 45.19 Terrain classification and extraction of geomet¬ 
ric features.Whereas the 3-D data is shown on the left , the 
feature map with extracted planes and vegetation regions 
is shown on the right (after [45.55]) 

rain classes depend on the application. The most direct 
classification approach uses a binary classifier that sep¬ 
arates the terrain into obstacle regions and traversable 
regions. More involved classifiers segment the map into 
more classes such as vegetation, solid surfaces, and 
linear structures [45.13,58,60]; an example is shown 
in Fig. 45.12. In some cases, it is possible to store, 
with each data element x in the map, the direction in 
which a measurement was taken for this element. In this 
case, it is possible to refine the classification by rea- 
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Fig. 45.20 (a) Structure learning for 
terrain classification (after [45.56]) 
and (b) building features extraction 
(after [45.57]) 


soning about the intersection of the measurement ray 
d{x) with the rest of map; for example, this type of geo¬ 
metric reasoning [45.61] has been exploited to recover 
the load-bearing surface obscured by vegetation [45.5 1, 
62, 63] and for extracting negative obstacles (such as 
ditches) [45.64]. In all of these cases, the classifica¬ 
tion information cannot be recovered directly from local 
statistics and must be inferred from longer-range geo¬ 
metric reasoning. 

This class of approaches to local feature classifica¬ 
tion is similar to approaches for extracting features from 
images and it suffers from similar limitations. Specifi¬ 
cally, these representations are sensitive to the choice 
of neighborhood used for computing the features. If 
it is too large, information over a large area is aver¬ 
aged out, leading to poor classification performance. 
If it is too small, there is not enough information in 
the neighborhood for reliable classification. The situ¬ 
ation is complicated by the fact that the resolution of 
the map, or more precisely the density of data points 
in the map, may vary drastically as the distance from 
the sensor changes. This problem is addressed by using 
different neighborhood sizes at different locations in the 
map, or by tuning the classifier differently depending on 
the map location, typically based on distance from the 
sensor [45.15, 65]. A second problem is to generate the 
classifier/. This can be done by using a physical model 
that predicts the statistics of the local data distribution in 
the map assuming different terrain types [45.66]. This 
is generally difficult and a preferred approach is to train 
the classifier on training data. 

This level of classification provides information 
about the local type of terrain, which can be used 
in planning, but it does not extract the extended 
geometric structures that may exist in the environ¬ 
ment (Fig. 45.19). Geometric structures such as planar 
patches can in principle be extracted by using tech¬ 


niques similar to the ones described in the context 
of indoor environments, such as EM, with the added 
difficulty that there is a larger amount of clutter that 
complicates the extraction of the patches. Robust tech¬ 
niques that can handle this level of clutter are typically 
used for extracting the planes [45.67, 68], 

Classification based on local attributes can be im¬ 
proved significantly by taking into account contex¬ 
tual information; for example, hidden Markov models 
have been successfully used for laser data analysis 
to determine terrain traversability [45.69]. Another 
structure learning approach based on Markov random 
fields in conjunction with margin-maximization crite¬ 
rion has been demonstrated for a wide class of appli¬ 
cations, including 3-D terrain classification and object 
segmentation [45.56] or building structures extrac¬ 
tion [45.57] (Fig. 45.20). Finally there are approaches to 
detect, select, model, and recognize natural landmarks 
automatically for robot localization and environment 
mapping [45.47]. 

45.3.6 Heterogeneous 

and Hierarchical Models 

For long-range navigation, an autonomous vehicle must 
perform numerous tasks, including absolute and rela¬ 
tive localization, path planning, and reactive obstacle 
avoidance. In addition, it must perform some tasks to 
fulfill the mission requirements, detecting and model¬ 
ing objects, for example. To achieve this, a hierarchical 
framework has been presented that accounts for the 
different scales and granularities of the representation 
needed [45.59]. Furthermore, models built from het¬ 
erogeneous imagery sources (overhead, descent, and 
ground) have been used to produce 3-D multiresolution 
terrain models that support Mars exploration [45.70]. 
The hybrid metric map [45.71], enhances feature maps 
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Fig. 45.21 (a) A mobile robot acquiring a three-dimensional scan of an urban scene, (b) The people in the scene cause 
spurious data points in the resulting meshes, (c) The same scene after filtering people 


with metric maps to provide a dense but compact envi¬ 
ronment representation. Another representation extracts 


landmarks based on laser/imagery appearance as well 
as a metric map [45.72]. 


45.4 Dynamic Environments 

The majority of techniques for mapping have been de¬ 
veloped for static environments. Certain approaches 
like occupancy grids or elevation maps can in prin¬ 
ciple deal with dynamic environments in which ob¬ 
jects move. Their drawback lies in the fact that the 
time to unlearn that a cell is free or that the eleva¬ 
tion has changed can require as many observations 
as the robot received with the same area being oc¬ 
cupied or with a different elevation. To resolve this 
problem, several alternative approaches have been pro¬ 
posed in the past. One very popular technique is to 
track moving objects using feature-based tracking al¬ 
gorithms [45.73,74]. Such approaches are especially 
useful when the type of the dynamic objects is known 
in advance. They have been successfully applied in 
the context of learning three-dimensional city maps 
from range data. Figures 45.21 and 45.22 show appli¬ 
cations in which dynamic objects were removed from 
the 3-D data acquired with mobile robots in urban 
scenes. Alternative approaches to such tracking tech¬ 
niques include those that learn maps on different time 
scales [45.75], that explicitly learn different states of 


dynamic environments [45.76,77], or that only map the 
static aspects [45.78]. 



Fig. 45.22 Complex three-dimensional scene acquired 
with a mobile robot after filtering dynamic objects (af¬ 
ter [45.74]) 


45.5 Summary and Further Reading 

Further reading about typical representations and how 80]. A comprehensive survey of the fundamentals of 
they can be utilized for mobile robot navigation can be spatial data structures and their applications is available 
found in recent textbooks on mobile robotics [45.2, 79, in Samet [45.81]. 
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Video-References 
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OctoMap visualization 

available from http://handbool<ofrobotics.org/view-chapter/45/videodetails/79 
3-D textured model of urban environments 

available from http://handbookofrobotics.org/view-chapter/45/videodetails/269 
Service robot navigation in urban environments 

available from http://handbookofrobotics.org/view-chapter/45/videodetails/270 
Learning navigation cost grids 

available from http://handbookofrobotics.org/view-chapter/45/videodetails/271 


References 


45.1 H.P. Moravec, A.E. Elfes: High resolution maps from 
wide angle sonar, Proc. IEEE Int. Conf. Robotics Au- 
tom. (ICRA) (1985) 

45.2 H. Choset, K. Lynch, S. Hutchinson, G. Kantor, 
W. Burgard, L. Kavraki, S. Thrun: Principles of Robot 
Motion: Theory, Algorithms and Implementation 
(MIT Press, Cambridge 2005) 

45.3 D.H. Douglas, T.K. Peucker: Algorithms for the re¬ 
duction of the number of points required to rep¬ 
resent a line or its caricature, Cdn. Cartogr. 10(2), 
112-122 (1973) 

45.4 D. Sack, W. Burgard: A comparison of methods for 
line extraction from range data, Proc. IFAC Symp. 
Intell. Auton. Veh. (IAV) (2004) 

45.5 P. Beeson, N.K. Jong, B. Kuipers: Towards au¬ 
tonomous topological place detection using the 
extended Voronoi graph, Proc. IEEE Int. Conf. 
Robotics Autom. (ICRA) (2005) 

45.6 M. Montemerlo, S. Thrun, D. Koller, B. Wegbreit: 
FastSLAM: A factored solution to the simultaneous 
localization and mapping problem, Proc. Nat. Conf. 
Artif. Intell. (AAAI) (2002) 

45.7 B.J. Kuipers, Y.-T. Byun: A robust qualitative 
method for spatial learning in unknown environ¬ 
ments, Proc. Nat. Conf. Artif. Intell. (AAAI) (1988) 

45.8 H. Choset, K. Nagatani: Topological simultaneous 
localization and mapping (SLAM): Toward exact lo¬ 
calization without explicit localization, IEEE Trans. 
Robotics Autom. 17(2), 125-137 (2001) 

45.9 S. Thrun: Robotic mapping: A survey. In: Explor¬ 
ing Artificial Intelligence in the New Millenium, ed. 
by G. Lakemeyer, B. Nebel (Morgan Kaufmann, San 
Diego 2003) 

45.10 M. Maimone, P. Leger, J. Biesiadecki: Overview of 
the Mars exploration rovers' autonomous mobil¬ 
ity and vision capabilities, Proc. IEEE Int. Conf. 
Robotics Autom. (ICRA) (2007) 

45.11 S. Lacroix, A. Mallet, D. Bonnafous, G. Bauzil, 
S. Fleury, M. Herrb, R. Chatila: Autonomous rover 
navigation on unknown terrains: functions and in¬ 
tegration, Int. J. Robotics Res. 21(10-11), 917-942 
( 2002 ) 

45.12 R. Olea: Geostatistics for Engineers and Earth Sci¬ 
entists (Kluwer, Boston 1999) 

45.13 A. Kelly, A. Stentz, 0. Amidi, M. Bode, D. Bradley, 
A. Diaz-Calderon, M. Happold, H. Herman, R. Man- 
delbaum, T. Pilarki, P. Rander, S. Thayer, N. Vallidi, 
R. Warner: Toward reliable off road autonomous 


vehicles operating in challenging environments, 
Int. J. Robotics Res. 25(5-6), 449-483 (2006) 

45.14 I.S. Kweon, T. Kanade: High-resolution terrain map 
from multiple sensor data, IEEE Trans. Pattern Anal. 
Mach. Intell. 14(2), 278-292 (1992) 

45.15 M. Montemerlo, S. Thrun: A multi-resolution pyra¬ 
mid for outdoor robot terrain perception, Proc. AAAI 
Nat. Conf. Artif. Intel., San Jose (2004) 

45.16 R. Triebel, P. Pfaff, W. Burgard: Multi-level surface 
maps for outdoor terrain mapping and loop clos¬ 
ing, Proc. IEEE/RSJ Int. Conf. Intell. Robotics Syst. 
(IROS) (2006) 

45.17 C. Wellington, A. Courville, A. Stentz: A genera¬ 
tive model of terrain for autonomous navigation 
in vegetation, Int. J. Robotics Res. 25(12), 1287-1304 
(2006) 

45.18 P. Pfaff, R. Triebel, W. Burgard: An efficient exten¬ 
sion to elevation maps for outdoor terrain mapping 
and loop closing, Int. J. Robotics Res. 26(2), 217-230 
(2007) 

45.19 D.M. Cole, P.M. Newman: Using laser range data for 
3D SLAM in outdoor environments, Proc. IEEE Int. 
Conf. Robotics Autom. (ICRA) (2006) 

45.20 A. Nuchter, K. Lingemann, J. Hertzberg, H. Sur- 
mann: 6D SLAM - 3D mapping outdoor environ¬ 
ments: Research articles, J. Field Robotics 24(8-9), 
699-722 (2007) 

45.21 J. Elseberg, D. Borrmann, A. Nuchter: Efficient pro¬ 
cessing of large 3D point clouds, Proc. 23rd Int. 
Symp. Infor. Commun. Autom. Technol. (ICAT) (2011) 

45.22 R.B. Rusu, S. Cousins: 3D is here: Point cloud library 
(PCL), Proc. IEEE Int. Conf. Robotics Autom. (ICRA) 
( 2011 ) 

45.23 N. Fairfield, G. Kantor, D. Wettergreen: Real-time 
SLAM with octree evidence grids for exploration in 
underwater tunnels, J. Field Robotics 24(1), 3-21 
(2007) 

45.24 A. Foessel: Scene Modelingfrom Motion-Free Radar 
Sensing, Ph.D. Thesis (Carnegie Mellon Univ., Pitts¬ 
burgh 2002) 

45.25 K.M. Wurm, A. Hornung, M. Bennewitz, C. Stach- 
niss, W. Burgard: OctoMap: A probabilistic, flexible, 
and compact 3D map representation for robotic 
systems, Proc. ICRA Workshop Best Pract. 3D Per¬ 
cept. Model. Mob. Manip. (2010) 

45.26 J.-F. Lalonde, N. Vandapel, M. Hebert: Data struc¬ 
ture for efficient processing in 3-D, Proc. Robotics 
Sci. Syst., Vol. I (2005) p. 48 











World Modeling I References 1151 


45.27 A. Hornung, K.M. Wurm, M. Bennewitz, C. Stach- 
niss, W. Burgard: OctoMap: An efficient probabilis¬ 
tic 3D mappingframework based on octrees, Auton. 
Robots 34(3), 189-206 (2013) 

45.28 K.M. Wurm, D. Hennes, D. Holz, R.B. Rusu, C. Stach- 
niss, K. Konolige, W. Burgard: Hierarchies of octrees 
for efficient 3D mapping, Proc. IEEE/RSJ Int Conf. In- 
tell. Robots Syst. (IROS) (2011) 

45.29 L. Heng, L. Meier, P. Tanskanen, F. Fraundorfer, 
M. Pollefeys: Autonomous obstacle avoidance and 
maneuvering on a vision-guided MAV using on¬ 
board processing, Proc. IEEE Int. Conf. Robotics 
Autom. (ICRA) (2011) 

45.30 S. Oswald, A. Hornung, M. Bennewitz: Improved 
proposals for highly accurate localization using 
range and vision data, Proc. IEEE/RSJ Int. Conf. In- 
tell. Robots Syst. (IROS) (2012) 

45.31 M. Ciocarlie, K. Hsiao, E.G. Jones, S. Chitta, 

R. B. Rusu, I.A. Sucan: Towards reliable grasping 
and manipulation in household environments, Int. 
Symp. Exp. Robotics (ISER) (2010) 

45.32 A. Hornung, M. Phillips, E.G. Jones, M. Bennewitz, 
M. Likhachev, S. Chitta: Navigation in three- 
dimensional cluttered environments for mobile 
manipulation, Proc. IEEE Int. Conf. Robotics Autom. 
(ICRA) (2012) 

45.33 J. Leal: Stochastic Environment Representation, 
Ph.D. Thesis (Univ. of Sydney, Sydney 2003) 

45.34 P. Heckbert, M. Garland: Optimal triangula¬ 
tion and quadric-based surface simplification, 
J. Comput. Geom. Theory Appl. 14(1-3), 49-65 
(1999) 

45.35 A. Akbarzadeh: Towards urban 3d reconstruction 
from video, Proc. Int. Symp. 3D Data Vis. Transm. 
(2006) 

45.36 C. Frueh, S. Jain, A. Zakhor: Data processing algo¬ 
rithms for generating textured 3d building facade 
meshes from laser scans and camera images, Int. 
J. Comput. Vis. 61(2), 159-184 (2005) 

45.37 I. Stamos, P. Allen: Geometry and texture recovery 
of scenes of large scales, Comput. Vis. Image Un- 
derst. 88, 94-118 (2002) 

45.38 D. Gennery: Traversability analysis and path plan¬ 
ning for a planetary rover, Auton. Robotics 6,131— 
146 (1999) 

45.39 D. Ferguson, A. Stentz: The delayed D* algorithm 
for efficient path replanning, Proc. IEEE Int. Conf. 
Robotics Autom. (ICRA) (2005) 

45.40 D. Ferguson, A. Stentz: Field D*: An interpolation- 
based path planner and replanner, Proc. Int. Symp. 
Robotics Res. (ISRR) (2005) 

45.41 M. Likhachev, D. Ferguson, G. Gordon, A. Stentz, 

S. Thrun: Anytime dynamic A*: An anytime, replan¬ 
ning algorithm, Proc. Int. Conf. Autom. Plan. Sched. 
(ICAPS) (2005) 

45.42 B. Sofman, E. Lin, J. Bagnell, J. Cole, N. Vandapel, 
A. Stentz: Improving robot navigation through self- 
supervised online learning, J. Field Robotics 23(12), 
1059-1075 (2006) 

45.43 D. Stavens, S. Thrun: A self-supervised terrain 
roughness estimator for off-road autonomous 
driving, Uncertainty Artif. Intell., Boston (2006) 


.44 A. Angelova, L. Matthies, D. Helmick, P. Perona: Slip 
prediction using visual information, Robotics Sci. 
Syst., Philadelphia (2006) 

.45 D. Kim, J. Sun, S. Oh, J. Rehg, A. Bobiclc 
Traversability classification using unsupervised on¬ 
line visual learning for outdoor robot naviga¬ 
tion, Proc. IEEE Int. Conf. Robotics Autom. (ICRA) 
(2006) 

.46 S. Thrun, M. Montemerlo, A. Aron: Probabilis¬ 
tic terrain analysis for high-speed desert driving, 
Robotics Sci. Syst. (2005) 

.47 R. Murrieta-Cid, C. Parra, M. Devy: Visual naviga¬ 
tion in natural environments: From range and color 
data to a landmark-based model, Auton. Robotics 
13(2), 143-168 (2002) 

.48 D. Asmar, J. Zelek, S. Abdallah: Tree trunks as land¬ 
marks for outdoor vision SLAM, Proc. Conf. Comp. 
Vis. Pattern Recogn. Workshop (2006) 

.49 I. Posner, D. Schroeter, P. Newman: Using scene 
similarity for place labelling, Int. Symp. Exp. 
Robotics (2006) 

.50 A. Torralba, K.P. Murphy, W.T. Freeman, M.A. Rubin: 
Context-based vision system for place and object 
recognition, Proc. IEEE Int. Conf. Comput. Vis. (ICCV) 
(2003) 

.51 D. Bradley, S. Thayer, A. Stentz, P. Rander: Vegeta¬ 
tion Detection for Mobile Robot Navigation, Tech. 
Rep. CMU-RI-TR-OU-12 (Carnegie Mellon Univ., 
Pittsburgh 2004) 

.52 S. Kumar, J. Guivant, H. Durrant-Whyte: Infor¬ 
mative representations of unstructured environ¬ 
ments, Proc. IEEE Int. Conf. Robotics Autom. (ICRA) 
(2004) 

.53 S. Kumar, F. Ramos, B. Douillard, M. Ridley, 
H. Durrant-Whyte: A novel visual perception 
framework, Proc. 9th Int. Conf. Control Autom. 
Robotics Vis. (ICARCV) (2006) 

.54 F. Ramos, S. Kumar, B. Upcroft, H. Durrant-Whyte: 
Representing natural objects in unstructured en¬ 
vironments, NIPS Workshop Mach. Learn. Robotics 
(2005) 

.55 C. Pantofaru, R. Unnikrishnan, M. Hebert: Toward 
generating labeled maps from color and range data 
for robot navigation, Proc. IEEE/RSJ Int. Conf. Intell. 
Robotics Syst. (2003) 

.56 D. Anguelov, B. Taskar, V. Chatalbashev, D. KoHer, 
D. Gupta, G. Heitz, A. Ng: Discriminative learning of 
Markov random fields for segmentation of 3D scan 
data, Proc. Conf. Comp. Vis. Pattern Recogn. (CVPR) 
(2005) 

.57 R. Triebel, K. Kersting, W. Burgard: Robust 3D scan 
point classification using associative Markov net¬ 
works, Proc. IEEE Int. Conf. Robotics Autom. (ICRA) 
(2006) 

.58 J.F. Lalonde, N. Vandapel, D. Huber, M. Hebert: Nat¬ 
ural terrain classification using three-dimensional 
ladar data for ground robot mobility, J. Field 
Robotics 23(10), 839-861 (2006) 

.59 M. Devy, R. Chatila, P. Fillatreau, S. Lacroix, 
F. Nashashibi: On autonomous navigation in a nat¬ 
ural environment, Robotics Auton. Syst. 16(1), 5-16 
(1995) 


45 

45 

45 

45 

45 

45 

45 

45 

45 

45 

45 

45 

45 

45 

45 

45 


Part E | 45 



Part E | 45 


1152 Part E I Moving in the Environment 


45.60 R. Manduchi, A. Castano, A. Talukder, L. Matthies: 
Obstacle detection and terrain classification for 
autonomous off-road navigation, Auton. Robotics 
18(1), 81-102 (2005) 

45.61 D. Huber, M. Hebert: 3D modeling using a statisti¬ 
cal sensor model and stochastic search, Proc. IEEE 
Conf. Comput. Vision Pattern Recogn. (CVPR) (2003) 
pp. 858-865 

45.62 S. Balakirsky, A. Lacaze: World modeling and be¬ 
havior generation for autonomous ground vehi¬ 
cles, Proc. IEEE Int. Conf. Robotics Autom. (ICRA) 
( 2000 ) 

45.63 A. Lacaze, K. Murphy, M. Delgiorno: Autonomous 
mobility for the demo III experimental unmanned 
vehicles, Proc. AUVSI Int. Conf. Unmanned Veh. 
( 2002 ) 

45.64 P. Bellutta, R. Manduchi, L. Matthies, l(. Owens, 
A. Rankin: Terrain perception for demo III, Proc. In- 
tell. Veh. Symp. (2000) 

45.65 J.F. Lalonde, R. Unnikrishnan, N. Vandapel, 
M. Hebert: Scale selection for classifica¬ 
tion of point-sampled 3D surfaces, Proc. 5th 
Int. Conf. 3-D Digital Imaging Model. (3DIM) 
(2005) 

45.66 J. Macedo, R. Manduchi, L. Matthies: Ladar-based 
discrimination of grass from obstacles for au¬ 
tonomous navigation, Proc. 7th Int. Symp. Exp. 
Robotics (ISER) (2000) 

45.67 H. Chen, P. Meer, D. Tyler: Robust regression 
for data with multiple structures, Proc. IEEE 
Int. Conf. Comput. Vis. Pattern Recogn. (CVPR) 
( 2001 ) 

45.68 R. Unnikrishnan, M. Hebert: Robust extraction of 
multiple structures from non-uniformly sampled 
data, Proc. IEEE/RSJ Int. Conf. Intell. Robotics Syst. 
(IROS) (2003) 

45.69 D. Wolf, G. Sukhatme, D. Fox, W. Burgard: Au¬ 
tonomous terrain mapping and classification us¬ 
ing hidden Markov models, Proc. IEEE Int. Conf. 
Robotics Autom. (ICRA) (2005) 


45.70 C. Olson, L. Matthies, J. Wright, R. Li, K. Di: Visual 
terrain mapping for Mars exploration, Comput. Vis. 
Underst. 105, 73-85 (2007) 

45.71 J. Nieto, J. Guivant, E. Nebot: The hybrid met¬ 
ric maps (hymms): A novel map representation for 
denseSLAM, Proc. IEEE Int. Conf. Robotics Autom. 
(ICRA) (2004) 

45.72 F. Ramos, J. Nieto, H. Durrant-Whyte: Recognising 
and modelling landmarks to close loops in outdoor 
SLAM, Proc. IEEE Int. Conf. Robotics Autom. (ICRA) 
(2007) 

45.73 D. Hahnel, D. Schulz, W. Burgard: Mobile robot 
mapping in populated environments, Adv. 
Robotics 17(7), 579-598 (2003) 

45.74 C.-C. Wang, C. Thorpe, S. Thrun: Online simultane¬ 
ous localization and mapping with detection and 
tracking of moving objects: Theory and results from 
a ground vehicle in crowded urban areas, Proc. IEEE 
Int. Conf. Robotics Autom. (ICRA) (2003) 

45.75 P. Biber, T. Duckett: Dynamic maps for long-term 
operation of mobile service robots, Proc. Robotics 
Sci. Syst. (2005) 

45.76 D. Meyer-Delius, J. Hess, G. Grisetti, W. Burgard: 
Temporary maps for robust localization in semi¬ 
static environments, Proc. IEEE/RSJ Int. Conf. Intell. 
Robots Syst. (IROS), Taipei (2010) 

45.77 C. Stachniss, W. Burgard: Mobile robot mapping 
and localization in non-static environments, Proc. 
Nat. Conf. Artif. Intell. (AAAI), Pittsburgh (2005) 

45.78 D. Hahnel, R. Triebel, W. Burgard, S. Thrun: Map 
building with mobile robots in dynamic environ¬ 
ments, Proc. IEEE Int. Conf. Robotics Autom. (ICRA) 
(2003) 

45.79 R. Siegwart, I. Nourbakhsh: Introduction to Au¬ 
tonomous Mobile Robots (MIT Press, Cambridge 
2001) 

45.80 S. Thrun, W. Burgard, D. Fox: Probabilistic Robotics 
(MIT Press, Cambridge 2005) 

45.81 H. Samet: Foundations of Multidimensional and 
Metric Data Structures (Elsevier, Amsterdam 2006) 



1153 


Multimedia Contents 



46. Simultaneous Localization 

and Mapping 


Cyrill Stachniss, John J. Leonard, Sebastian Thrun 


This chapter provides a comprehensive intro¬ 
duction in to the simultaneous localization and 
mapping problem, better known in its abbreviated 
form as SLAM. SLAM addresses the main percep¬ 
tion problem of a robot navigating an unknown 
environment. While navigating the environment, 
the robot seeks to acquire a map thereof, and 
at the same time it wishes to localize itself us¬ 
ing its map. The use of SLAM problems can be 
motivated in two different ways: one might be in¬ 
terested in detailed environment models, or one 
might seekto maintain an accurate sense of a mo¬ 
bile robot's location. SLAM serves both of these 
purposes. 

We review the three major paradigms from 
which many published methods for SLAM are de¬ 
rived: (l) the extended Kalman filter (EKF); (2) 
particle filtering; and (3) graph optimization. We 
also review recent work in three-dimensional (3-D) 
SLAM using visual and red green blue dis- 
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tance-sensors (RGB-D), and close with a discussion 
of open research problems in robotic mapping. 


This chapter provides a comprehensive introduction 
into one of the key enabling technologies of mobile 
robot navigation: simultaneous localization and map¬ 
ping , or in short SLAM. SLAM addresses the problem 
of acquiring a spatial map of an environment while si¬ 
multaneously localizing the robot relative to this model. 
The SLAM problem is generally regarded as one of the 
most important problems in the pursuit of building truly 
autonomous mobile robots. It is of great practical im¬ 
portance; if a robust, general-purpose solution to SLAM 
can be found, then many new applications of mobile 
robotics will become possible. 

While the problem is deceptively easy to state, it 
presents many challenges, despite significant progress 
made in this area. At present, we have robust methods 
for mapping environments that are mainly static, struc¬ 


tured, and of limited size. Robustly mapping unstruc¬ 
tured, dynamic, and large-scale environments in an on¬ 
line fashion remains largely an open research problem. 

The historical roots of methods that can be applied 
to address the SLAM problem can be traced back to 
Gauss [46.1], who is largely credited for inventing the 
least squares method. In the Twentieth Century, a num¬ 
ber of fields outside robotics have studied the making 
of environment models from a moving sensor platform, 
most notably in photogrammetry [46.2-4] and com¬ 
puter vision [46.5]. Strongly related problems in these 
fields are bundle adjustment and structure from mo¬ 
tion. SLAM builds on this work, often extending the 
basic paradigms into more scalable algorithms. Mod¬ 
em SLAM systems often view the estimation problem 
as solving a sparse graph of constraints and applying 
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nonlinear optimization to compute the map and the tra¬ 
jectory of the robot. As we strive to enable long-lived 
autonomous robots, an emerging challenge is to handle 
massive sensor data streams. 

This chapter begins with a definition of the SLAM 
problem, which shall include a brief taxonomy of dif¬ 
ferent versions of the problem. The centerpiece of this 
chapter is a layman introduction into the three major 
paradigms in this field, and the various extensions that 
exist. As the reader will quickly recognize, there is no 
single best solution to the SLAM method. The method 


46.1 SLAM: Problem Definition 

The SLAM problem is defined as follows: A mobile 
robot roams an unknown environment, starting at an 
initial location xq. Its motion is uncertain, making it 
gradually more difficult to determine its current pose in 
global coordinates. As it roams, the robot can sense its 
environment with a noisy sensor. The SLAM problem 
is the problem of building a map of the environment 
while simultaneously determining the robot’s position 
relative to this map given noisy data. 

46.1.1 Mathematical Basis 

Formally, SLAM is best described in probabilistic ter¬ 
minology. Let us denote time by t, and the robot 
location by x,. For mobile robots on a flat ground, x, is 
usually a three-dimensional vector, comprising its two- 
dimensional (2-D) coordinate in the plane plus a single 
rotational value for its orientation. The sequence of lo¬ 
cations, or path, is then given as 

Xt = {x 0 ,xi,x 2 . x T ) . (46.1) 

Here T is some terminal time (T might be 00 ). The 
initial location jco often serves as a point of reference 
for the estimation algorithm; other positions cannot be 
sensed. 

Odometry provides relative information between 
two consecutive locations. Let u, denote the odometry 
that characterized the motion between time t— 1 and 
time t ; such data might be obtained from the robot’s 
wheel encoders or from the controls given to those mo¬ 
tors. Then the sequence 

Uj = {u\, ut, M 3 ..., uj} (46.2) 

characterizes the relative motion of the robot. For noise- 
free motion, Uj would be sufficient to recover the poses 
from the initial location xq. However, odometry mea¬ 


chosen by the practitioner will depend on a number of 
factors, such as the desired map resolution, the update 
time, and the nature of the features in the map, and so 
on. Nevertheless, the three methods discussed in this 
chapter cover the major paradigms in this field. 

For more a detailed treatment of SLAM, we refer 
the reader to Durrant-Whyte and Bailey [46.6,7], who 
provide an in-depth tutorial for SLAM, Grisetti et al. 
for a tutorial on graph-based SLAM [46.8], and Thrun 
et al., which dedicates a number of chapters to the topic 
of SLAM [46.9]. 


surements are noisy, and path integration techniques 
inevitably diverge from the truth. 

Finally, the robot senses objects in the environment. 
Let m denote the true map of the environment. The 
environment may be comprised of landmarks, objects, 
surfaces, etc., and m describes their locations. The en¬ 
vironment map m is often assumed to be time-invariant, 
i. e., static. 

The robot measurements establish information be¬ 
tween features in m and the robot location x t . If we, 
without loss of generality, assume that the robot takes 
exactly one measurement at each point in time, the se¬ 
quence of measurements is given as 

Z T = {Z1,Z2,Z3, ■ ■ ■ ,Zt) ■ (46.3) 

Figure 46.1 illustrates the variables involved in the 
SLAM problem. It shows the sequence of locations and 
sensor measurements, and the causal relationships be¬ 
tween these variables. This diagram represents a graph- 



fig. 46.1 Graphical model of the SLAM problem. Arcs in¬ 
dicate causal relationships, and shaded nodes are directly 
observable to the robot. In SLAM, the robot seeks to re¬ 
cover the unobservable variables 
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ical model for SLAM. It is useful in understanding the 
dependencies in the problem at hand. 

The SLAM problem is now the problem of recov¬ 
ering a model of the world m and the sequence of 
robot locations X T from the odometry and measurement 
data. The literature distinguishes two main forms of the 
SLAM problem, which are both of equal practical im¬ 
portance. One is known as the full SLAM problem: it 
involves estimating the posterior over the entire robot 
path together with the map 

p(Xj, m | Zj, Uj ) . (46.4) 

Written in this way, the full SLAM problem is the 
problem of calculating the joint posterior probability 
over X T and m from the available data. Notice that the 
variables right of the conditioning bar are all directly 
observable to the robot, whereas those on the left are 
the ones that we want. As we shall see, algorithms for 
the full SLAM problem are often batch, that is, they 
process all data at the same time. 

The second, equally important SLAM problem is 
the online SLAM problem. This problem is defined via 

p(x t , m | Z t , U t ) . (46.5) 

Online SLAM seeks to recover the present robot loca¬ 
tion, instead of the entire path. Algorithms that address 
the online problem are usually incremental and can pro¬ 
cess one data item at a time. In the literature, such 
algorithms are typically called filters. 

To solve the SLAM problem, the robot needs to 
be endowed with two more models: a mathematical 
model that relates odometry measurements u, to robot 
locations x,—\ and x t ; and a model that relates measure¬ 
ments z t to the environment m and the robot location x t . 
These models correspond to the arcs in Fig. 46.1. 

In SLAM, it is common to think of those mathemat¬ 
ical models as probability distributions: p{x t \ x t —\, u t ) 
characterizes the probability distribution of the location 
x, assuming that a robot started at a known location 
x,—\ and measured the odometry data u t . And likewise, 
p(z t | x t , m ) is the probability for measuring z t if this 
measurement is taken at a known location x t in a known 
environment m. Of course, in the SLAM problem we do 
not know the robot location, and neither do we know the 
environment. As we shall see, Bayes rule takes care of 
this, by transforming these mathematical relationships 
into a form where we can recover probability distribu¬ 
tions over those latent variables from the measured data. 

46.1.2 Example: SLAM in Landmark Worlds 

One common setting of SLAM involves an assumption 
that the environment is populated by point-landmarks. 


When building 2-D maps, point-landmarks may cor¬ 
respond to door posts and corners of rooms, which, 
when projected into a 2-D map are characterized by 
a point coordinate. In a 2-D world, each point-landmark 
is characterized by two coordinate values. Hence the 
world is a vector of size 2 N, where N is the number 
of point-landmarks in the world. In a commonly stud¬ 
ied setting, the robot can sense three things: the relative 
range to nearby landmarks, their relative bearing, and 
the identity of these landmarks. The range and bearing 
may be noisy, but in the most simple case the identity 
of the sensed landmarks is known perfectly. Determin¬ 
ing the identity of the sensed landmarks is also known 
as the data association problem. In practice, it is one of 
the most difficult problems in SLAM. 

To model the above described setup, one begins 
with defining the exact, noise-free measurement func¬ 
tion. The measurement function h describes the work¬ 
ings of the sensors: it accepts as input a description of 
the environment m and a robot location x t , and it com¬ 
putes the measurement 

h(x,,m) . (46.6) 

Computing h is straightforward in our simplified land¬ 
mark setting; it is a simple exercise in trigonometry. The 
probabilistic measurement model can be derived from 
this measurement function by adding a noise term. It 
is a probability distribution that peaks at the noise-free 
value h(x t , m) but allows for measurement noise, for ex¬ 
ample, 

p(z t | x t , m) = N(h(x t , m), Q,) . (46.7) 

Here .'N denotes the 2-D normal distribution, which is 
centered at h(x t ,m). The 2-by-2 matrix Q, is the noise 
covariance, indexed by time. 

The motion model is derived from a kinematic 
model of robot motion. Given the location vector x t —\ 
and the motion u u textbook kinematics tells us how to 
calculate x t . Let this function be denoted by g 

g(x t -i,u r ). (46.8) 

The motion model may then be defined by a normal dis¬ 
tribution centered at g(x t — 1 , u,) but subject to Gaussian 
noise 

p{x, \x,-i,u t ) = :Af(g(x,_i,w,).R,) . (46.9) 

Here R, is a covariance. It is of size 3-by-3, since the 
location is a three-dimensional 3-D vector. 

With these definitions, we have all we need to 
develop a SLAM algorithm. While in the literature. 
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point-landmark problems with range-bearing sensing 
are by far the most studied, SLAM algorithms are not 
confined to landmark worlds. But no matter what the 
map representation and the sensor modality, any SLAM 
algorithm needs a similarly crisp definition of the fea¬ 
tures in m, the measurement model p(z r \ x r , m), and the 
motion model p(x, \ x t — 1 , u t ). Note that none of those 
distributions has to be restricted to Gaussian noise as 
done in the example above. 

46.1.3 Taxonomy of the SLAM Problem 

SLAM problems are distinguished along a number of 
different dimensions. Most important research papers 
identify the type of problems addressed by making the 
underlying assumptions explicit. We already encoun¬ 
tered one such distinction: full versus online. Other 
common distinctions are as follows: 

Volumetric Versus Feature-Based 
In volumetric SLAM, the map is sampled at a resolution 
high enough to allow for photo-realistic reconstruction 
of the environment. The map m in volumetric SLAM 
is usually quite high-dimensional, with the result that 
the computation can be quite involved. Feature-based 
SLAM extracts sparse features from the sensor stream. 
The map is then only comprised of features. Our 
point-landmark example is an instance of feature-based 
SLAM. Feature-based SLAM techniques tend to be 
more efficient, but their results may be inferior to volu¬ 
metric SLAM due to the fact that the extraction of fea¬ 
tures discards information in the sensor measurements. 

Topological Versus Metric 

Some mapping techniques recover only a qualitative de¬ 
scription of the environment, which characterizes the 
relation of basic locations. Such methods are known as 
topological. A topological map might be defined over 
a set of distinct places and a set of qualitative rela¬ 
tions between these places (e.g., place A is adjacent to 
place B). Metric SLAM methods provide metric infor¬ 
mation between the relation of such places. In recent 
years, topological methods have fallen out of fashion, 
despite ample evidence that humans often use topolog¬ 
ical information for navigation. 

Known Versus Unknown Correspondence 
The correspondence problem is the problem of relat¬ 
ing the identity of sensed things to other sensed things. 
In the landmark example above, we assumed that the 
identity of landmarks is known. Some SLAM algo¬ 
rithms make such an assumption, others do not. The 
ones that do not provide special mechanisms for es¬ 
timating the correspondence of measured features to 


previously observed landmarks in the map. The prob¬ 
lem of estimating the correspondence is known as data 
association problem. It is one of the most difficult prob¬ 
lems in SLAM. 

Static Versus Dynamic 

Static SLAM algorithms assume that the environment 
does not change over time. Dynamic methods allow 
for changes in the environment. The vast literature on 
SLAM assumes static environments. Dynamic effects 
are often treated just as measurement outliers. Meth¬ 
ods that reason about motion in the environment are 
more involved, but they tend to be more robust in most 
applications. 

Small Versus Large Uncertainty 
SLAM problems are distinguished by the degree of 
location uncertainty that they can handle. The most sim¬ 
ple SLAM algorithms allow only for small errors in 
the location estimate. They are good for situations in 
which a robot goes down a path that does not intersect 
itself, and then returns along the same path. In many 
environments it is possible to reach the same location 
from multiple directions. Here the robot may accrue 
a large amount of uncertainty. This problem is known 
as the loop closing problem. When closing a loop, the 
uncertainty may be large. The ability to close loops is 
a key characteristic of modern-day SLAM algorithms. 
The uncertainty can be reduced if the robot can sense 
information about its position in some absolute coor¬ 
dinate frame, e.g., through the use of a satellite-based 
global positioning system (GPS) receiver. 

Active Versus Passive 

In passive SLAM algorithms, some other entity controls 
the robot, and the SLAM algorithm is purely observing. 
The vast majority of algorithms are of this type; they 
give the robot designer the freedom to implement ar¬ 
bitrary motion controllers, and pursue arbitrary motion 
objectives. In active SLAM, the robot actively explores 
its environment in the pursuit of an accurate map. Ac¬ 
tive SLAM methods tend to yield more accurate maps 
in less time, but they constrain the robot motion. There 
exist hybrid techniques in which the SLAM algorithm 
controls only the pointing direction of the robot’s sen¬ 
sors, but not the motion direction. 

Single-Robot Versus Multi-Robot 
Most SLAM problems are defined for a single robot 
platform, although recently the problem of multi¬ 
robot exploration has gained in popularity. Multi-robot 
SLAM problems come in many flavors. In some, robots 
get to observe each other, in others, robots are told their 
relative initial locations. Multirobot SLAM problems 
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are also distinguished by the type of communication al¬ 
lowed between the different robots. In some, the robots 
can communicate with no latency and infinite band¬ 
width. More realistic are setups in which only nearby 
robots can communicate, and the communication is 
subject to latency and bandwidth limitations. 

Any-Time and Any-Space 
Robots that do all computations onboard have limited 
resources in memory and computation power. Any¬ 
time and any-space SLAM systems are an alternative 

46.2 The Three Main SLAM Paradigms 

This section reviews three basic SLAM paradigms, 
from which most others are derived. The first, known as 
EKF SLAM, is in robotics historically the earliest but 
has become less popular due to its limiting computa¬ 
tional properties and issues resulting from performing 
single linearizations only. The second approach uses 
nonparametric statistical filtering techniques known as 
particle filters. It is a popular method for online SLAM 
and provides a perspective on addressing the data asso¬ 
ciation problem in SLAM. The third paradigm is based 
on graphical representations and successfully applies 
sparse nonlinear optimization methods to the SLAM 
problem. It is the main paradigm for solving the full 
SLAM problem and recently also incremental tech¬ 
niques are available. 

46.2.1 Extended Kalman Filters 

Historically, the EKF formulation of SLAM is the earli¬ 
est, and perhaps the most influential, SLAM algorithm. 
EKF SLAM was introduced in [46.10, 11] and [46.12, 
13], which were the first papers to propose the use of 
a single state vector to estimate the locations of the 
robot and a set of features in the environment, with an 
associated error covariance matrix representing the un¬ 
certainty in these estimates, including the correlations 
between the vehicle and feature state estimates. As the 
robot moves through its environment taking measure¬ 
ments, the system state vector and covariance matrix 
are updated using the extended Kalman filter [46.14, 
15]. As new features are observed, new states are added 
to the system state vector; the size of the system covari¬ 
ance matrix grows quadratically. 

This approach assumes a metrical, feature-based 
environmental representation, in which objects can be 
effectively represented as points in an appropriate pa¬ 
rameter space. The position of the robot and the loca¬ 
tions of features form a network of uncertain spatial 
relationships. The development of appropriate repre- 


to traditional methods. They enable the robot to com¬ 
pute a solution given the resource constraints of the 
system. The more resources available, the better the 
solution. 

As this taxonomy suggests, there exists a flurry of 
SLAM algorithms. Most modern-day conferences ded¬ 
icate multiple sessions to SLAM. This chapter focuses 
on the very basic SLAM setup. In particular it assumes 
a static environment with a single robot. Extensions are 
discussed towards the end of this chapter, in which the 
relevant literature is discussed. 


sentations is a critical issue in SLAM, and intimately 
related to the topics of sensing and world modeling dis¬ 
cussed in Chap. 36 and in Part C. 

The EKF algorithm represents the robot estimate by 
a multivariate Gaussian 

p(x,,m | Z,, U,) = £,) . (46.10) 

The high-dimensional vector fi, contains the robot’s 
best estimate of its own current location x, and the 
location of the features in the environment. In our 
point-landmark example, the dimension of fi t would be 
3 + 2 N, since we need three variables to represent the 
robot location and 2 N variables for the N landmarks in 
the map. 

The matrix E, is the covariance of the robot’s as¬ 
sessment of its expected error in the guess ji,. The 
matrix E , is of size (3 + 2 N) x (3 + 2 N) and it is pos¬ 
itive semi-definite. In SLAM, this matrix is usually 
dense. The off-diagonal elements capture the correla¬ 
tions in the estimates of different variables. Nonzero 
correlations come along because the robot’s location is 
uncertain, and as a result the locations of the landmarks 
in the maps are uncertain. 

The EKF SLAM algorithm is easily derived for our 
point-landmark example. Suppose, for a moment, the 
motion function g and the measurement function h were 
linear in their arguments. Then, the vanilla Kalman fil¬ 
ter, as described in any textbook on Kalman filtering, 
would be applicable. EKF SLAM linearizes the func¬ 
tions g and h using Taylor series expansion. In its most 
basic form and in the absence of any data association 
problems, EKF SLAM is basically the application of 
the EKF to the online SLAM problem. 

Figure 46.2 illustrates the EKF SLAM algorithm for 
an artificial example. The robot navigates from a start 
pose that serves as the origin of its coordinate system. 
As it moves, its own pose uncertainty increases, as in¬ 
dicated by uncertainty ellipses of growing diameter. It 
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Fig.46.2a-d EKF applied to the 
online SLAM problem. The robot’s 
path is a dotted line, and its estimates 
of its own position are shaded ellipses. 
Eight distinguishable landmarks of 
unknown location are shown as small 
dots, and their location estimates are 
shown as white ellipses. In (a-c) 
the robot’s positional uncertainty is 
increasing, as is its uncertainty about 
the landmarks it encounters. In (d) the 
robot senses the first landmark again, 
and the uncertainty of all landmarks 
decreases, as does the uncertainty 
of its current pose (image courtesy 
of Michael Montemerlo, Stanford 
University) 


also senses nearby landmarks and maps them with an 
uncertainty that combines the fixed measurement uncer¬ 
tainty with the increasing pose uncertainty. As a result, 
the uncertainty in the landmark locations grows over 
time. The interesting transition happens in Fig. 46. 2d: 
Here the robot observes the landmark it saw in the very 
beginning of mapping, and whose location is relatively 
well known. Through this observation, the robot’s pose 
error is reduced, as indicated in Fig. 46. 2d - notice 
the very small error ellipse for the final robot pose. 
This observation also reduces the uncertainty for other 
landmarks in the map. This phenomenon arises from 
a correlation that is expressed in the covariance matrix 
of the Gaussian posterior. Since most of the uncertainty 
in earlier landmark estimates is caused by the robot 
pose, and since this very uncertainty persists over time, 
the location estimates of those landmarks are correlated. 
When gaining information on the robot’s pose, this in¬ 
formation spreads to previously observed landmarks. 
This effect is probably the most important characteristic 
of the SLAM posterior [46.16]. Information that helps 
localize the robot is propagated through the map, and as 
a result improves the localization of other landmarks in 
the map. 

With a few adaptations, EKF SLAM can also be ap¬ 
plied in the presence of uncertain data association. If 
the identity of observed features is unknown, the basic 


EKF idea becomes inapplicable. The solution here is 
to reason about the most likely data association when 
a landmark is observed. This is usually done based 
on proximity: which of the landmarks in the map cor¬ 
responds most likely to the landmark just observed? 
The proximity calculation considers the measurement 
noise and the actual uncertainty in the poster estimate, 
and the metric used in this calculation is known as 
a Mahalanobis distance, which is a weighted quadratic 
distance. To minimize the chances of false data asso¬ 
ciations, many implementations use visible features to 
distinguish individual landmarks and associate groups 
of landmarks observed simultaneously [46.17, 18], al¬ 
though distinct features can also be computed from 
laser data [46.19,20]. Typical implementations also 
maintain a provisional landmark list and only add 
landmarks to the internal map when they have been 
observed sufficiently frequently [46.16,21]. With an 
appropriate landmark definition and careful implemen¬ 
tation of the data association step, EKF SLAM has been 
applied successfully in a wide range of environments, 
using airborne, underwater, indoor, and various other 
platforms. 

The basic formulation of EKF SLAM assumes that 
the location of features in the map is fully observable 
from a single position of the robot. The method has 
been extended to situations with partial observability. 
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with range-only [46.22] or angle-only [46.23, 24] mea¬ 
surements. The technique has also been utilized using 
a feature-less representation, in which the state consists 
of current and past robot poses, and measurements take 
the form of constraints between the poses (derived for 
example from laser scan matching or from camera mea¬ 
surements) [46.25,26], 

A key concern of the EKF approach to SLAM 
lies in the quadratic nature of the covariance matrix. 
A number of researchers have proposed extensions to 
the EKF SLAM algorithms that achieve scalability, for 
example through submap decomposition [46.27-30]. 
A related family of approaches [46.31-34] employs the 
Extended Information Filter, which operates on the in¬ 
verse of the covariance matrix. A key insight is that 
whereas the EKF covariance is densely populated, the 
information matrix is sparse when the full robot tra¬ 
jectory is maintained, leading to the development of 
efficient algorithms and providing a conceptual link 
to the pose graph optimization methods described in 
Sect. 46.2.3. 

The issues of consistency and convergence in 
EKF SLAM have been investigated in [46.35,36]. 
Observability-based rules for designing consistent EKF 
SLAM estimators are presented in [46.37]. 

46.2.2 Particle Methods 

The second principal SLAM paradigm is based on par¬ 
ticle filters. Particle filters can be traced back to [46.38], 
but they have become popular only in the last two 
decades. Particle filters represent a posterior through 
a set of particles. For the novice in SLAM, each par¬ 
ticle is best thought as a concrete guess as to what 
the true value of the state may be. By collecting many 
such guesses into a set of guesses, or set of particles, 
the particle filter approximates the posterior distribu¬ 
tion. Under mild conditions, the particle filter has been 
shown to approach the true posterior as the particle set 
size goes to infinity. It is also a nonparametric repre¬ 
sentation that represents multimodal distributions with 
ease. 

The key problem with the particle filter in the con¬ 
text of SLAM is that the space of maps and robot paths 
is huge. Suppose we have a map with 100 features. How 
many particles would it take to populate that space? 
In fact, particle filters scale exponentially with the di¬ 
mension of the underlying state space. Three or four 
dimensions are thus acceptable, but 100 dimensions are 
generally not. 

The trick to make particle filters amenable to the 
SLAM problem goes back to [46.39, 40] and is known 
as Rao-Blackwellization. It has been introduced into 
the SLAM literature in [46.41], followed by [46.42], 


who coined the name fastSLAM (fast simultaneous lo¬ 
calization and mapping). Let us first explain the basic 
FastSLAM algorithm on the simplified point-landmark 
example, and then discuss the justification for this ap¬ 
proach. 

At any point in time, FastSLAM maintains K parti¬ 
cles of the type 


X 


M 

V't, t> 


M T W 

■ P't.N’ ^ t, 1 ’ 


• M 


(46.11) 


Here [k\ is the index of the sample. This expression 
states that a particle contains: 


• A sample path x \ k ^, and 

• A set of N 2-D Gaussians with means p,\ k }, and 
variances £®), one for each landmark in the en¬ 
vironment. 


Here n is the index of the landmark (with 1 < n < 
N). From that it follows that K particles possess K path 
samples. It also possesses KN Gaussians, each of which 
models exactly one landmark for one of the particles. 

Initializing FastSLAM is simple: just set each parti¬ 
cle’s robot location to the starting coordinates, typically 
(0,0, 0) T , and zero the map. The particle update then 
proceeds as follows: 

• When an odometry reading is received, new loca¬ 
tion variables are generated stochastically, one for 
each of the particles. The distribution for generat¬ 
ing those location particles is based on the motion 
model 

x f w » p(x t | tj^a,) . (46.12) 

Here is the previous location, which is part 
of the particle. This probabilistic sampling step is 
easily implemented for any robot whose kinematics 
can be computed. 

• When a measurement z t is received, two things hap¬ 
pen: first, FastSLAM computes for each particle the 
probability of the new measurement z. t . Let the in¬ 
dex of the sensed landmark be n. Then the desired 
probability is defined as follows 

w} k] = M{z, | . (46.13) 

The factor w\ k ^ is called the importance weight, 
since it measures how important the particle is in 
the light of the new sensor measurement. As before, 
JsT denotes the normal distribution, but this time it 
is calculated for a specific value, z f . The importance 
weights of all particles are then normalized so that 
they sum to 1. 

Next, FastSLAM draws with replacement from the 
set of existing particles a set of new particles. The 
probability of drawing a particle is its normalized 
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importance weight. This step is called resampling. 
The intuition behind resampling is that particles 
for which the measurement is more plausible have 
a higher chance of surviving the resampling pro¬ 
cess. 

Finally, FastSLAM updates for the new particle set 
the mean p \ k l and covariance based on the 
measurement z t . This update follows the standard 
EKF update rules - note that the extended Kalman 
filters maintained in FastSLAM are, in contrast to 
EKF SLAM, all low-dimensional (typically 2-D). 

This all may sound complex, but FastSLAM is 
quite easy to implement. Sampling from the motion 
model usually involves simple kinematic calculations. 
Computing the importance of a measurement is often 
straightforward too, especially for Gaussian measure¬ 
ment noise. And updating a low-dimensional particle 
filter is also not complicated. 

FastSLAM has been shown to approximate the full 
SLAM posterior. The derivation of FastSLAM exploits 
three techniques: Rao-Blackwellization, conditional in¬ 
dependence, and resampling. Rao-Blackwellization is 
the following concept. Suppose we would like to com¬ 
pute a probability distribution p(a, b), where a and b 
are arbitrary random variables. The vanilla particle filter 
would draw particles from the joint distributions, that 
is, each particle would have a value for a and one for b. 
Flowever, if the conditional p(b \ a) can be described in 
closed form, it is equally legitimate to just draw parti¬ 
cles from p(a), and attach to each particle a closed-form 
description of p(b \ a). This trick is known as Rao- 



Fig. 46.3 The SLAM problem depicted as Bayes network graph. 
The robot moves from location x,-i to location driven by 

a sequence of controls. At each location x t it observes a nearby 
feature in the map m = { 1 nt.m 2 .m 3 }. This graphical network illus¬ 
trates that the location variables separate the individual features in 
the map from each other. If the locations are known, there remains 
no other path involving variables whose value is not known, be¬ 
tween any two features in the map. This lack of a path renders the 
posterior of any two features in the map conditionally independent 
(given the locations) 


Blackwellization, and it yields better results than sam¬ 
pling from the joint. FastSLAM applies this technique, 
in that it samples from the path posterior p{x\ k ^ \ U,. Z,) 
and represents the map p(m | , U t .Z t ) in Gaussian 

form. 

FastSLAM also breaks down the posterior over 
maps (conditioned on paths) into sequences of low¬ 
dimensional Gaussians. The justification for this de¬ 
composition is subtle. It arises from a specific condi¬ 
tional independence assumption that is native to SLAM. 
Fig. 46.3 illustrates the concept graphically. In SLAM, 
knowledge of the robot path renders all landmark esti¬ 
mates independent. This is easily shown for the graph¬ 
ical network in Fig. 46.3: we find that if we remove 
the path variables from Fig. 46.3, then the landmark 
variables are all disconnected [46.43]. Thus, in SLAM 
any dependence between multiple landmark estimates 
is mediated through the robot path. This subtle but 
important observation implies that even if we used 
a large, monolithic Gaussian for the entire map (one 
per particle, of course), the off-diagonal element be¬ 
tween different landmarks would simply remain zero. 
It is therefore legitimate to implement the map more 
efficiently, using N small Gaussians, one for each land¬ 
mark. This explains the efficient map representation in 
FastSLAM. 

Figure 46.4 shows results for a point-feature prob¬ 
lem; here the point features are the centers of tree 
trunks as observed by an outdoor robot. The dataset 
used here is known as the Victoria Park dataset [46.44]. 
Fig. 46.4a shows the path of the vehicle obtained by 
integrating the vehicle controls, without perception. As 
can be seen, controls are a poor predictor of location for 
this vehicle; after 30 min of driving, the estimated po¬ 
sition of the vehicle is well over 100 m away from its 
GPS position. 

The FastSLAM algorithm has a number of inter¬ 
esting properties. First, it solves both full and online 
SLAM problems. Each particle has a sample of an en¬ 
tire path but the actual update equation only uses the 
most recent pose. This makes FastSLAM a filter. Sec¬ 
ond, FastSLAM can maintain multiple data association 
hypotheses. It is straightforward to make data associa¬ 
tion decisions on a per-particle basis, instead of having 
to adopt the same hypothesis for the entire filter. While 
we will not give any mathematical justification, we 
note that the resulting FastSLAM algorithm can even 
deal with unknown data association - something that 
the extended Kalman filter cannot claim. And third, 
FastSLAM can be implemented very efficiently using 
advanced tree methods to represent the map estimates, 
the update can be performed in time logarithmic in the 
size of the map N, and linear in the number of parti¬ 
cles M. 
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Fig. 46.4 (a) Vehicle path predicted by the odometry; (b) True path (dashed line) and FastSLAM 1.0 path (solid line); 
(c) Victoria Park results overlaid on aerial imagery with the GPS path in blue (dashed), average FastSLAM 1.0 path in 
yellow (solid), and estimated features as yellow dots (data and aerial image courtesy of Jose Guivant and Eduardo Nebot, 
Australian Centre for Field Robotics) 



Fig. 46.5 Occupancy grid map generated from laser range data and based on pure odometry (image courtesy of Dirk 
Hahnel, University of Freiburg) 


FastSLAM has been extended in several ways. One 
set of variants are grid-based versions of FastSLAM, 
in which the Gaussians used to model point landmarks 
are replaced by an occupancy grid map [46.45-47]. The 
variant of [46.46] is illustrated in Fig. 46.5. 

Figure 46.6 illustrates a simplified situation with 
three particles just before closing a large loop. The three 
different particles each stand for different paths, and 
they also posses their own local maps. When the loop 
is closed importance resampling selects those particles 


whose maps are most consistent with the measure¬ 
ment. A resulting large-scale map is shown in Fig. 46.5. 
Further extensions can be found in [46.48,49], whose 
methods are called DP-SLAM and operate on ances¬ 
try trees to provide efficient tree update methods for 
grid-based maps. Related to that, approximations to 
FastSLAM in which particles share their maps have 
been proposed [46.50]. 

The works in [46.45,47,51] provide ways to in¬ 
corporate new observations into the location sampling 
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Fig. 46.6 Application of the grid-based variant of the FastSLAM algorithm. Each particle carries its own map and the 
importance weights of the particles are computed based on the likelihood of the measurements given the particle’s own 
map 


process for landmarks and grid maps, based on prior 
work in [46.52]. This leads to an improved sampling 
process 




pizt | m^l v x ,) p(x t | x, L ^j, u,) 


W 


P(z t | 


(46.14) 


which incorporates the odometry and the observation 
at the same time. Using an improved proposal dis¬ 
tribution leads to more accurately sampled locations. 
This in turn leads to more accurate maps and requires 
a smaller number of particles compared to approaches 
using the sampling process given in (46.12). This ex¬ 
tension makes FastSLAM and especially its grid-based 
variants robust tools for addressing the SLAM problem. 

Finally, there are approaches that aim to overcome 
the assumption that the observations show Gaussian 
characteristics. As shown in [46.47], there are sev¬ 
eral situations in which the model is nonGaussian and 
also multimodal. A sum of Gaussians model on a per- 
particle bases, however, can be efficiently considered 
in the particle filter and it eliminates this problem in 
practice without introducing additional computational 
demands. 

The so-far developed particle filters-based SLAM 
systems suffer from two problems. First, the number of 
samples that are required to compute consistent maps 
is often set manually by making an educated guess. 
The larger the uncertainty that the filter needs to rep¬ 
resent during mapping, the more critical becomes this 


parameter. Second, nested loops combined with exten¬ 
sive re-visits of previously mapped areas can lead to 
particle depletion, which in turn may prevent the system 
from estimating a consistent map. Adaptive resampling 
strategies [46.45], particles sharing maps [46.50], or fil¬ 
ter backup approaches [46.53] improve the situation but 
cannot eliminate this problem in general. 

46.2.3 Graph-Based 

Optimization Techniques 

A third family of algorithms solves the SLAM problem 
through nonlinear sparse optimization. They draw their 
intuition from a graphical representation of the SLAM 
problem and the first working solution in robotics was 
proposed in [46.54]. The graph-based representation 
used here is closely related to a series of papers [46.55- 
64]. We note that most of the earlier techniques are 
offline and address the full SLAM problem. In more 
recent years, new incremental versions that effectively 
re-use the previously computed solution have been pro¬ 
posed such as [46.65-67]. 

The basic intuition of graph-based SLAM is a fol¬ 
lows. Landmarks and robot locations can be thought of 
as nodes in a graph. Every consecutive pair of locations 
x t —\,x, is tied together by an edge that represents the 
information conveyed by the odometry reading u t . Fur¬ 
ther edges exist between the nodes that correspond to 
locations x t and landmarks m,-, assuming that at time t 
the robot sensed landmark i. Edges in this graph are 
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soft constraints. Relaxing these constraints yields the 
robot’s best estimate for the map and the full path. 

The construction of the graph is illustrated in 
Fig. 46.7. Suppose at time t= 1, the robot senses 
landmark m\. This adds an arc in the (yet highly in¬ 
complete) graph between x\ and m \. When caching 
the edges in a matrix format (which happens to cor¬ 
respond to a quadratic equation defining the resulting 
constraints), a value is added to the elements be¬ 
tween X\ and mi, as shown on the right hand side of 
Fig. 46.7a. 

Now suppose the robot moves. The odometry read¬ 
ing U 2 leads to an arc between nodes x\ and xi, as 
shown in Fig. 46.7b. Consecutive application of these 
two basic steps leads to an graph of increasing size, 
as illustrated in Fig. 46.7c. Nevertheless this graph is 
sparse, in that each node is only connected to a small 
number of other nodes (assuming a sensor with limited 
sensing range). The number of constraints in the graph 
is (at worst) linear in the time elapsed and in the num¬ 
ber of nodes in the graph. 

a) 


Xi m 1 



Fig.46.7a-c Illustration of the graph construction. The (a) 
diagram shows the graph, the (b) the constraints in matrix 
form. (a) Observation Is landmark m\. (b) Robot motion 
from.ri to X 2 . (c) Several steps later 


If we think of the graph as a spring-mass mod¬ 
el [46.60], computing the SLAM solution is equivalent 
to computing the state of minimal energy of this model. 
To see, we note that the graph corresponds to the log- 
posterior of the full SLAM problem (46.4) 

logp(X T , m | Zj, Ut) ■ (46.15) 

Without derivation, we state that this logarithm is of the 
form 

log p{Xj, m \Zj,Uj) 

= const + E log p(x, I X ,— 1 , u t ) 

t 

+ y>gKz> I X,,m) , (46.16) 

t 

assuming independence between the individual obser¬ 
vations and odometry readings. Each constraint of the 
form log p(x, | x t —i, u t ) is the result of exactly one robot 
motion event, and it corresponds to an edge in the graph. 
Likewise, each constraint of the form log p(z t \ x t , m) is 
the result of one sensor measurement, to which we can 
also find a corresponding edge in the graph. The SLAM 
problem is then simply to find the mode of this equa¬ 
tion, i. e., 

Xj ,m* = argmaxlog p(X T ,m | Z T , Uj) ■ (46.17) 

Xj ,m 

Without derivation, we note that under the Gaussian 
noise assumptions, which was made in the point- 
landmark example, this expression resolves to the fol¬ 
lowing quadratic form 

log p(X T , m | Zj, Ut) = const 
+ Y. \ x t - g( x t- 1, «r)] T R 7 1 \ x t ~ g( x t -1 ■ “/)] 

t ' V ' 

odometry reading 

+ E t Zf ~ h ( Xl ' '”)] T [zt “ h(xt, m)]. 

t ' ^ 

feature observation 

(46.18) 

This quadratic form yields a sparse system of equations 
and a number of efficient optimization techniques can 
be applied. Common choices include direct methods 
such as sparse Cholesky and QR decomposition, or iter¬ 
ative ones such as gradient descent, conjugate gradient, 
and others. Most SLAM implementations rely on iter¬ 
atively linearizing the functions g and h, in which case 
the objective in (46.18) becomes quadratic in all of its 
variables. 

Extensions to support an effective correction of 
large-scale graphs are hierarchical methods. One of 
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the first is the ATLAS framework [46.25], which con¬ 
structs a two-level hierarchy combining a Kalman filter 
that operates in the lower level and a global optimiza¬ 
tion at the higher level. Similar to that, Hierarchical 
SLAM [46.68] is a technique for using independent 
local maps, which are merged in case of re-visiting 
a place. A fully hierarchical approach has been pre¬ 
sented in [46.65]. It builds a multilevel pose-graph and 
employs an incremental, lazy optimization scheme that 
allows for optimizing large graphs and at the same time 
can be executed at each step during mapping. An al¬ 
ternative hierarchical approach is [46.69], which recur¬ 
sively partitions the graph into multiple-level submaps 
using the nested dissection algorithm. 

When it comes to computing highly accurate envi¬ 
ronment reconstructions, approaches that do not only 
optimize the poses of the robot but also each indi¬ 
vidual measurement of a dense sensor often provide 
better results. In the spirit of bundle adjustment [46.4], 
approaches for laser scanners [46.70] and Kinect cam¬ 
eras [46.71] have been proposed. 

The graphical paradigm can be extended to han¬ 
dle the data association problems as we can integrate 
additional knowledge on data association into (46.18). 
Suppose some oracle informed us that landmarks m,- 
and nij in the map corresponded to one and the same 
physical landmark in the world. Then, we can ei¬ 
ther remove ny from the graph and attach all adjacent 
edges to m,-, or we can add a soft correspondence con¬ 
straint [46.72] of the form 

(nij — mi) 1 T (mj — nij). (46.19) 

Here T is 2-by-2 diagonal matrix whose coefficients de¬ 
termine the penalty for not assigning identical locations 
to two landmarks (hence we want T to be large). Since 
graphical methods are usually used for the full SLAM 
problem, the optimization can be interleaved with the 
search for the optimal data association. 

Data association errors typically have a strong im¬ 
pact in the resulting map estimate. Even a small number 
of wrong data associations is likely to result in incon¬ 
sistent map estimates. Recently, novel approaches have 
been proposed that are robust under a certain number of 
false associations. For example, [46.73, 74] propose an 
iterative procedure that allows for disabling constraints, 
an action that is associated with a cost. A generalization 
of this method introduced in [46.75] formulates [46.74] 
as a robust cost function also reducing the compu¬ 
tational requirements. Such approaches can deal with 
a significant number of false associations and still pro¬ 
vide high-quality maps. Consistency checks for loop 
closure hypotheses can be found in other approaches 
as well, both in the front-end [46.76] and in the opti¬ 


mizer [46.77]. There has also been an extension that can 
deal with multimodal constraints [46.78], proposing 
a max-mixture representation for maintaining efficiency 
of the log likelihood optimizing in (46.16). As a result 
of that, the multimodal extension has only little im¬ 
pact on the runtime and can easily be incorporated in 
most optimizers. Also robust cost function are used for 
SLAM, for example pseudo Huber and several alterna¬ 
tives [46.75,79-81]. 

Graphical SLAM methods have the advantage that 
they scale to much higher-dimensional maps than EKF 
SLAM, exploiting the sparsity of the graph. The key 
limiting factor in EKF SLAM is the covariance matrix, 
which takes space (and update time) quadratic in the 
size of the map. No such constraint exists in graphical 
methods. The update time of the graph is constant, and 
the amount of memory required is linear (under some 
mild assumptions). A further advantage of graph-based 
methods over the EKF is their ability to constantly re¬ 
linearize the error function which often leads to better 
results. Performing the optimization can be expensive, 
however. Technically, finding the optimal data associa¬ 
tion is suspected to be an NP-hard problem, although in 
practice the number of plausible assignments is usually 
small. The continuous optimization of the log likeli¬ 
hood function in (46.18) depends among other things 
on the number and size of loops in the map. Also the 
initialization can have a strong impact on the result and 
a good initial guess can simplify the optimization sub¬ 
stantially [46.8, 82, 83]. 

We note that the graph-based paradigm is very 
closely linked to information theory, in that the soft con¬ 
straints constitute the information the robot has on the 
world (in an information-theoretic sense [46.92]). Most 
methods in the field are offline and they optimize for 
the entire robot path. If the robot path is long, the opti¬ 
mization may become cumbersome. Over the last five 
years, however, incremental optimization techniques 
have been proposed that aim at providing a sufficient 
but not necessarily perfect model of the environment 
at every point in time. This allows a robot to make 
decisions based on the current model, for example, to 
determine exploration goals. In this context, incremen¬ 
tal variants [46.93, 94] of stochastic gradient descent 
techniques [46.8,91] have been proposed that estimate 
which part of the graph requires re-optimization given 
new sensor data. Incremental methods [46.66, 79, 95] in 
the smoothing and mapping framework can be execute 
at each step of the mapping process and achieve the 
performance by variable ordering and selective relin¬ 
earization. As also evaluated in [46.96] for the SLAM 
problem, variable ordering impacts the performance of 
the optimization. Others use hierarchical data struc¬ 
tures [46.89] and pose-graphs [46.97] combined with 
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Table 46.1 Recent open-source graph-based SLAM implementations 


Name 

Comment 

Dynamic covariance 
scaling (DCS) [46.75] 

Optimization with a robust cost function for dealing with outliers 

Integrated into g 2 o 

g 2 o [46.80] 

Flexible and easily extendable optimization framework for SLAM 

Comes with different optimization approaches and error functions 

Supports external plugins 

GTSAM2.1 [46.79] 

Flexible optimization framework for SLAM and SFMstructure from motion 

Implements direct and iterative optimization techniques 

Implements smoothing and mapping (SAM), iSAM, and iSAM2 

Implements bundle adjustment for Visual SLAM and SFM 

HOG-Man [46.65] 

Incremental optimization approach via hierarchical pose graphs and lazy optimization 

Requires pose-graphs with full rank constraints 

iSAM2 [46.66] 

General incremental nonlinear optimization with variable elimination 

Variable re-ordering to retain sparsity 

On-demand re-linearization of selected variables 

KinFu (KinectFusion 
reimplemented) 

Open source reimplementation of KinectFusion [46.84] within the point cloud library (PCL) 

Dense and highly accurate reconstruction using a Kinect camera 

Currently limited to medium sized rooms 

MaxMixture [46.78] 

Optimization for multimodal constraints and outliers 

Robust to outliers 

Plugin for g 2 o 

Parallel tracking and 
mapping (PTAM) [46.85] 

System for tracking a hand-held monocular camera and observed features 

Operates on comparably small workspaces 

RGBD-SLAM [46.86] 

Kinect-frontend for HOG-Man and g 2 o 

Fairly standard combination of SURF matching and RANSAC 

ScaViSLAM [46.87] 

SLAM system for stereo and Kinect-style cameras 

Combines local bundle adjustment with sparse global optimization for on-the-fly processing 

SLAM6-D [46.88] 

SLAM system that operates on point clouds from 3-D laser data 

Applies iterative closest point algorithm (ICP) and global relaxation 

Sparse surface adjustment 
(SSA) [46.70,71] 

Optimizes robot poses and proximity sensor data jointly 

Provides smooth surface estimates 

Assumes a range sensor (e.g., laser scanner, Kinect, or similar) 

TreeMap [46.89] 

Incremental optimization approach 

Update in O (log N ) time 

Provides only a mean estimate 

TORO [46.90] 

Optimization approach that extends stochastic gradient descent (SGD) [46.91] 

Robust under bad initial guesses 

Recovers quickly from large errors but slow convergence at minimum 

Assumes that constraints have roughly spherical covariance matrices 

Provides only a mean estimate 

Vertigo [46.74] 

Switchable constraints for robust optimization 

Plugin for g 2 o 


a lazy optimization for on-the-fly mapping [46.65]. As 
an alternative to global methods, relative optimization 
approaches [46.98] aim at computing locally consistent 
geometric maps but only topological maps on the global 
scale. Hybrid approaches [46.87] seek to combine the 
best of both worlds. 

There also exists a number of cross-overs that ma¬ 
nipulate the graph online so as to factor out past 
robot location variables. The resulting algorithms are 
filters [46.25,33,99,100], and they tend to be inti¬ 
mately related to information filter methods. Many of 
the original attempts to decompose EKF SLAM repre¬ 
sentations into smaller submaps to scale up are based 


on motivations that are not dissimilar to the graphical 
approach [46.27,28, 101]. 

Recently, researchers addressed the problem of 
long-term operation and frequent revisits of already 
mapped terrain. To avoid densely connected pose- 
graphs that lead to slow convergence behavior, the robot 
can switch between SLAM and localization, can merge 
nodes to avoid a growth of the graph [46.90, 102], or 
can discard nodes or edges [46.32, 103-105], 

Graphical and optimization-based SLAM algorithm 
are still subject of intense research and the paradigm 
scales to maps large numbers of nodes [46.25, 55,57, 
59,63-65, 89,90,106,107]. Arguably, the graph-based 
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paradigm has generated some the largest SLAM maps 
ever built. Furthermore, the SLAM community started 
to release flexible optimization frameworks and SLAM 
implementations under open source licenses to sup¬ 
port further developments and to allow for efficient 
comparisons, (Table 46.1). Especially the optimization 
frameworks [46.66,79,80] are flexible and powerful 
state of the art tools for developing graph-based SLAM 
systems. They can be either used as a black box or can 
be easily extended though plugins. 

46.2.4 Relation of Paradigms 

The three paradigms just discussed cover the vast ma¬ 
jority of work in the field of SLAM. As discussed, 
EKF SLAM comes with a computational hurdle that 
poses serious scaling limitations and the linearization 
may lead to inconsistent maps. The most promising ex¬ 
tensions of EKF SLAM are based on building local 
submaps; however, in many ways the resulting algo¬ 
rithms resemble the graph-based approach. 

Particle filter methods sidestep some of the is¬ 
sues arising from the natural inter-feature correlations 
in the map - which hindered the EKF. By sampling 
from robot poses, the individual landmarks in the map 

46.3 Visual and RGB-D SLAM 

A popular and important topic in recent years has been 
Visual SLAM - the challenge of building maps and 
tracking the robot pose in full 6-DOF using data from 
cameras [46.108] or RGB-D (Kinect) sensors [46.86, 
109]. Visual sensors offer a wealth of information that 
enables the construction of rich 3-D models of the 
world. They also enable difficult issues such as loop¬ 
closing to be addressed in novel ways using appearance 
information [46.110]. Visual SLAM is anticipated to 
be a critical area for future research in perception for 
robotics, as we seek to develop low-cost systems that 
are capably of intelligent physical interaction with the 
world. 

Attempting SLAM with monocular, stereo, om¬ 
nidirectional, or RGB-D cameras raises the level-of- 
difficulty of many of the SLAM components, such 
as data association and computational efficiency, de¬ 
scribed above. A key challenge is robustness. Many 
visual SLAM applications of interest, such as aug¬ 
mented reality [46.85], entail handheld camera motions, 
which present greater difficulties for state estimation, 
in comparison to the motion of a wheeled robot across 
a flat floor. 

Visual navigation and mapping was a key early goal 
in the mobile robotics community [46.111,112], but 


become independent, and hence are decorrelated. As 
a result, FastSLAM can represent the posterior by 
a sampled robot pose, and many local, independent 
Gaussians for its landmarks. The particle representation 
offers advantages for SLAM as it allows for compu¬ 
tationally efficient updates and for sampling over data 
associations. On the negative side, the number of neces¬ 
sary particles can grow very large, especially for robots 
seeking to map multiple nested loops. 

Graph-based methods address the full SLAM prob¬ 
lem, hence are in the standard formulation not online. 
They draw their intuition form the fact that SLAM 
can be modeled by a sparse graph of soft constraints, 
where each constraint either corresponds to a motion or 
a measurement event. Due to the availability of highly 
efficient optimization methods for sparse nonlinear op¬ 
timization problems, graph-based SLAM has become 
the method of choice for building large-scale maps. 
Recent developments have brought up several graph- 
based methods for incremental map building that can 
be executed at every time step during navigation. Data 
association search can be incorporated into the basic 
mathematical framework and different approaches that 
are even robust under wrong data associations are avail¬ 
able today. 


early approaches were hampered by the lack of suffi¬ 
cient computational resources to handle massive video 
data streams. Early approaches were typically based 
on extended Kalman filters [46.113-116], but did not 
compute the full covariance for the feature poses and 
camera trajectory, resulting in a loss of consistency. 
Visual SLAM is closely related to the structure from 
motion (SFM) problem in computer vision [46.4,5]. 
Historically, SFM was primarily concerned with off¬ 
line batch processing, whereas SLAM seeks to achieve 
a solution for online operation, suitable for closed-loop 
interaction of a robot or user with its environment. In 
comparison to laser scanners, cameras provide a fire hy¬ 
drant of information, making online processing nearly 
impossible until recent increases in computation have 
become available. 

Davison was an early pioneer in developing com¬ 
plete visual SLAM systems, initially using a real-time 
active stereo head [46.121] that tracked distinctive vi¬ 
sual features with a full covariance EKF approach. Sub¬ 
sequent work developed the first real-time SLAM sys¬ 
tem that operated with a single freely moving camera 
as the only data source [46.23, 122]. This system could 
build sparse, room-size maps of indoor scenes at 30 Hz 
frame-rate in real-time, a notable historical achievement 
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y (m) y (m) 

Fig. 46.8 (a) A 2-km path and 50000 frames estimated for the New College Dataset (after [46.117]) using relative 
bundle adjustment (after [46.98]). (b) the relative bundle adjustment solution is easily improved by taking FAB-MAP 
(after [46. 118]) loop-closures into account - this is achieved without global optimization (after [46. 1 19]). Sibley et al. 
advocate that relative metric accuracy and topological consistency are the requirements for autonomous navigation, 
and these are better achieved using a relative manifold representation instead of using a conventional single Euclidean 
representation [46.120] 



Fig. 46.9 (a) 3-D Model and (b) close-up view of a corridor environment in the Paul G. Allen building at University of 
Washington built from Kinect data (after [46. 109]; image courtesy of Peter Henry, University of Washington) 


in visual SLAM research. A difficulty encountered with 
initial monocular SLAM [46.23] was coping with the 
initialization of points that were far away from the cam¬ 
era, due to nonGaussian distributions of such feature lo¬ 
cations resulting from poor depth information. This lim¬ 
itation was overcome in [46.24], introducing an inverse 
depth parameterization for monocular SLAM, a key 
development for enabling a unified treatment of initial¬ 
ization and tracking of visual features in real-time. 


A milestone in creating robust visual SLAM sys¬ 
tems was the introduction of keyframes in parallel 
tracking and mapping (PTAM) [46.85], which sep¬ 
arated the tasks of keyframe mapping and localiza¬ 
tion into parallel threads, improving robustness and 
performance for online processing. Keyframes are 
now a mainstream concept for complexity reduc¬ 
tion in visual SLAM systems. Related approaches 
using keyframes include [46.87, 123-126]. The work 
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in [46.108] analyzes the tradeoffs between filtering and 
keyframe-based bundle adjustment in visual SLAM, 
and concluded that keyframe bundle adjustment outper¬ 
forms filtering, as it provides the most accuracy per unit 
of computing time. 

As pointed out by Davison and other researchers, 
an appealing aspect of visual SLAM is that camera 
measurements can provide odometry information, and 
indeed visual odometry is a key component of modern 
SLAM systems [46.127]. Here, [46.128] and [46.129] 
provide an extensive tutorial of techniques for visual 
odometry, including feature detection, feature match¬ 
ing, outlier rejection, and constraint estimation, and 
trajectory optimization. Finally, a publicly available vi¬ 
sual odometry library [46.130] that is optimized for 
efficient operation on small unmanned aerial vehicles 
is available today. 

Visual information offers a tremendous source of in¬ 
formation for loop closing, not present in the canonical 
2-D laser SLAM systems developed in the early 2000s. 
The work in [46.110] was one of the first to employ 
techniques for visual object recognition [46.131] to lo¬ 
cation recognition. More recently FAB-MAP [46.118, 
132] has demonstrated appearance-only place recogni¬ 
tion at large scale, mapping trajectories with a length 
of 1000 km. Combining a bag-of-features approach 
with a probabilistic place model and Chow-Liu tree 
inference leads to place recognition that is robust 
against perceptual aliasing while remaining compu¬ 
tationally efficient. Other work on place recognition 
includes [46.133], which combines bag-of-words loop 
closing with tests of geometrical consistency based on 
conditional random fields (Fig. 46.8). 

The techniques described above have formed the 
basis for a number of notable large-scale SLAM sys¬ 
tems developed in recent years. A 2008 special is¬ 
sue of the IEEE Transactions on Robotics provides 
a good snapshot of recent state-of-the-art SLAM tech¬ 
niques [46.135], Other notable recent examples in¬ 
clude [46.98, 119,126, 136-138]. The idea of employ¬ 
ing relative bundle adjustment [46.98] to compute a full 
maximum likelihood solution in an online fashion, 
even for loop closures, by employing a manifold rep¬ 
resentation that does not attempt to enforce Euclidean 
constraints results in maps that can be computed at 
high frame rate (see also Fig. 46.8). Finally, view-based 
mapping systems [46.126, 136, 137] aim at large-scale 
and/or life-long visual mapping based on the pose 
graph optimization techniques described above in Sec¬ 
tion 46.2.3. 

Several compelling 3-D mapping and localization 
have been created in recent years with RGB-D (Kinect) 
sensors. The combination of direct range measurements 
with dense visual imagery can enable dramatic im¬ 


provements in mapping and navigation systems for in¬ 
door environments. State-of-the-art RGB-D SLAM sys¬ 
tems include [46.109] and [46.86]. Figure 46.9 shows 
examples of the output of these systems. 

Other researchers aim at exploiting the surface 
properties of scanned environments to correct for sen- 



Fig.46.10a-c Results obtained with KinectFusion (af¬ 
ter [46.134]). (a) A local scene as a normal map and (b) as 
a Phong-shaded rendering. The (c) image depicts a larger 
scene (image courtesy of Richard Newcombe, Imperial 
College London) 
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Fig. 46.11 Spatially ex¬ 
tended KinectFusion output 
produced in real-time with 
Kintinuous (after [46.139]) 


sor noise of range sensors such as the Kinect [46.71]. 
They jointly optimize the poses of the sensor and 
the positions of the surface points measured and it¬ 
eratively refine the structure of the error function 
by recomputing the data associations after each opti¬ 
mization, resulting in accurate smooth models of the 
environment. 

An emerging area for future research is the de¬ 
velopment of fully dense processing methods that 
exploit recent advances in commodity graphical pro¬ 
cessing unit (GPU) technology. Kinect-based dense 
tracking and mapping, a fully-dense method for small- 
scale visual tracking and reconstruction is described 
in [46.140]. Dense modeling and tracking are achieved 


via highly parallelized operations on commodity GPU 
hardware to yield a system that outperforms previ¬ 
ous methods such as PTAM for challenging camera 
trajectories. Dense methods offer an interesting per¬ 
spective from which to address long-standing problems, 
such as visual odometry, from a fresh perspective, 
without requiring explicit feature detection and match¬ 
ing [46.141]. KinectFusion [46.84, 134] is a dense mod¬ 
eling system that tracks the 3-D pose of a handheld 
Kinect while concurrently reconstructing high-quality 
scene 3-D models in real-time. See Fig. 46.10 for an 
example. KinectFusion has been applied to spatially ex¬ 
tended environments in [46.139, 142], An example is 
shown in Fig. 46.11. 


46.4 Conclusion and Future Challenges 


This chapter has provided an introduction into SLAM, 
which is defined as the problem faced by a mobile 
platform roaming an unknown environment, and seek¬ 
ing to localize itself while concurrently building a map 
of the environment. The chapter discussed three main 
paradigms in SLAM, which are based on the ex¬ 
tended Kalman filter, particle filters, and graph-based 
sparse optimization techniques, and then described re¬ 
cent progress in Visual/Kinect SLAM. 


The following references provide an in-depth tuto¬ 
rial on SLAM and much greater depth of coverage on 
the details of popular SLAM algorithms. Furthermore, 
several implementations of popular SLAM systems, in¬ 
cluding most of the approaches listed in Table 46.1, 
can be found in online resources such as http://www. 
openslam.org or in the references [46.6, 9,21, 62]. 

The considerable progress in SLAM in the past 
decade is beyond doubt. The core state estimation at 
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the heart of SLAM is now quite well understood, 
and a number of impressive implementations have 
been developed, including several widely used open 
source software implementations and some commer¬ 
cial projects. None-the-less, a number of open research 
challenges remain for the general problem of robotic 
mapping in complex and dynamic environments over 
extended periods of time, including robots sharing, ex¬ 
tending, and revising previously built models, efficient 
failure recovery, zero user intervention, and operation 
on resource-constrained systems. Another exciting area 
for the future is the further development of fully dense 
visual mapping systems exploiting the latest advances 
in GPU hardware development. 


An ultimate goal is to realize the challenge of per¬ 
sistent navigation and mapping - the capability for 
a robot to perform SLAM robustly for days, weeks, 
or months at a time with minimal human supervi¬ 
sion, in complex and dynamic environments. Taking 
the limit as t —> 00 poses difficult challenges to most 
current algorithms; in fact, most robot mapping and 
navigation algorithms are doomed to fail with the 
passage of time, as errors inevitably accrue. Despite 
recent encouraging solutions [46.102,105], more re¬ 
search is needed for techniques that can recover from 
mistakes and enable robots to deal with changes in 
the environment and enabling a long-term autonomous 
existence. 
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Deformation-based loop closure for Dense RGB-D SLAM 

available from http://handbookofrobotics.org/view-chapter/46/videodetails/439 

Large-scale SLAM using the Atlas framework 

available from http://handbookofrobotics.org/view-chapter/46/videodetails/440 
Graph-based SLAM 

available from http://handbookofrobotics.org/view-chapter/46/videodetails/441 
Graph-based SLAM 

available from http://handbookofrobotics.org/view-chapter/46/videodetails/442 
Graph-based SLAM 

available from http://handbookofrobotics.org/view-chapter/46/videodetails/443 
Graph-based SLAM 

available from http://handbookofrobotics.org/view-chapter/46/videodetails/444 
Graph-based SLAM 

available from http://handbookofrobotics.org/view-chapter/46/videodetails/445 
Graph-based SLAM using TORO 

available from http://handbookofrobotics.org/view-chapter/46/videodetails/446 
Sparse pose adjustment 

available from http://handbookofrobotics.org/view-chapter/46/videodetails/447 
Pose graph compression for laser-based SLAM 

available from http://handbookofrobotics.org/view-chapter/46/videodetails/449 
Pose graph compression for laser-based SLAM 

available from http://handbookofrobotics.org/view-chapter/46/videodetails/450 
Pose graph compression for laser-based SLAM 

available from http://handbookofrobotics.org/view-chapter/46/videodetails/451 
DTAM: Dense tracking and mapping in real-time 

available from http://handbookofrobotics.org/view-chapter/46/videodetails/452 
MonoSLAM: Real-time single camera SLAM 

available from http://handbookofrobotics.org/view-chapter/46/videodetails/453 
SLAM++: Simultaneous localisation and mapping at the level of objects 
available from http://handbookofrobotics.org/view-chapter/46/videodetails/454 
Extended Kalman filter SLAM 

available from http://handbookofrobotics.org/view-chapter/46/videodetails/455 
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47. Motion Planning 
and Obstacle Avoidance 


Javier Minguez, Florant Lamiraux, Jean-Paul Laumond 


This chapter describes motion planning and ob¬ 
stacle avoidance for mobile robots. We will see 
how the two areas do not share the same mod¬ 
eling background. From the very beginning of 
motion planning, research has been dominated 
by computer sciences. Researchers aim at devising 
well-grounded algorithms with well-understood 
completeness and exactness properties. 

The challenge of this chapter is to present both 
nonholonomic motion planning (Sects. 47.1-47.6) 
and obstacle avoidance (Sects. 47.7-47.10) issues. 
Section 47.11 reviews recent successful approaches 
that tend to embrace the whole problem of motion 
planning and motion control. These approaches 
benefit from both nonholonomic motion planning 
and obstacle avoidance methods. 
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The introduction of nonholonomic constraints has en¬ 
tailed revisiting these algorithms via the introduction 
of differential geometry materials. Such a combination 
has been made possible for certain classes of systems, 
the so-called small-time controllable ones. It remains 
that the underlying hypothesis of motion planning al¬ 
gorithms is the knowledge of a global and accurate 
map of the environment. More than that the consid¬ 
ered system is a formal system of equations that does 
not account for the entire physical system: uncertain¬ 
ties in the world or system modeling are not considered. 
Such hypotheses are too strong in practice. This is why 
other complementary research has been done in par¬ 
allel in a more pragmatic but realistic manner. Such 
research deals with obstacle avoidance. The problem 
here is not to deal with complicated systems like a car 
with multiple trailers. The considered systems are much 
simpler with respect to their geometric shape. The prob¬ 
lem considers sensor-based motions to face the physical 
issues of a real system navigating in a real world better 
than motion planning algorithms. How can we navigate 
toward a goal in a cluttered environment when the ob¬ 
stacles to avoid have just been discovered in real time? 
This is the question that obstacle avoidance addresses. 
The appearance of mobile robots in the late 1960s early 
1970s initiated a new research domain: autonomous 
navigation. It is interesting to note that the first nav¬ 
igation systems were published at the very first In¬ 
ternational Joint Conferences on Artificial Intelligence 
(IJCAI 1969). These systems were based on seminal 
ideas, which have been very fruitful in the development 
of robot motion planning algorithms. For instance, in 
1969, the mobile robot Shakey used a grid-based ap¬ 
proach to model and explore the environment [47.1], in 
1977 Jason used a visibility graph built from the corners 
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of the obstacles [47.2], and in 1979 Hilare decomposed 
the environment into collision-free convex cells [47.3]. 

In the late 1970s, studies of robot manipulators pop¬ 
ularized the notion of the configuration space of a me¬ 
chanical system [47.4]. In the configuration space the 
piano becomes a point. The motion planning problem 
for a mechanical system was thus reduced to finding 
a path for a point in the configuration space. The way 
was open to extending the seminal ideas and to develop¬ 
ing new and well-grounded algorithms (see Latombe’s 
book [47.5]). 

One decade later, the notion of nonholonomic sys¬ 
tems (also borrowed from mechanics) appeared in the 
literature [47.6] on robot motion planning through the 
problem of car parking. This problem had not been 
solved by the pioneering works on mobile robot nav¬ 
igation. Nonholonomic motion planning then became 
an attractive research field [47.7]. 

Besides this research effort in path planning, work 
was initiated in order to make robots move out of their 
initially artificial environments where the world was 
cylindrical and composed of wooden vertical boards. 
Robots started to move in laboratory buildings, with 
people walking around. Inaccurate localization, uncer¬ 
tain and incomplete maps of the world, and unexpected 
moving or static obstacles made roboticists aware of the 
gap between planning a path and executing a motion. 
Since then the domain of obstacle avoidance has been 
very active. 

In the 2000s a lot of effort has been made toward 
the integration of motion planning and obstacle avoid¬ 
ance. This effort was stimulated by the DARPA Urban 
challenge competition. The most successful integration 
was done by the Carnegie Mellon University team, who 
won the competition [47.8]. 


47.1 Nonholonomic Mobile Robots: Where Motion Planning Meets 
Control Theory 


Nonholonomic constraints are nonintegrable linear con¬ 
straints over the velocity space of a system. For 
instance, the rolling without slipping constraint of 
a differentially-driven mobile robot (Fig. 47.1) is lin¬ 
ear with respect to the velocity vector (vector of linear 
and angular velocities) of the differentially-driven robot 


and is non integrable (it cannot be integrated into 
a constraint over the configuration variables). As a con¬ 
sequence, a differentially-driven mobile robot can go 
anywhere but not following any trajectory. Other types 
of nonholonomic constraints arise when considering 
second-order differential equations such as the conser- 
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vation of inertial momentum. A number of papers in¬ 
vestigate the famous falling cat problem or free-floating 
robots in space [47.7]. This chapter is devoted to non- 
holonomic constraints for mobile robots with wheels. 

While the constraints due to obstacles are expressed 
directly in the configuration space, that is a manifold, 
nonholonomic constraints are expressed in the tangent 
space. In the presence of a linear kinematic constraint, 
the first question that naturally arises is: does this con¬ 
straint reduce the space reachable by the system? This 
question can be answered by studying the structure of 
the distribution spanned by the Lie algebra of the con¬ 
trol system. 

Even in the absence of obstacles, planning admissi¬ 
ble motions (i. e., that satisfy the kinematic constraints) 
for a nonholonomic system between two configurations 
is not an easy task. Exact solutions have been proposed 


only for some classes of systems but a lot of systems 
remain without exact solution. In the general case, how¬ 
ever, approximate solutions can be used. 

The motion planning problem for a nonholonomic 
system can be stated as follows: given a map of the 
environment with obstacles in the workspace, a robot 
subject to nonholonomic constraints, an initial config¬ 
uration and a goal configuration, find an admissible 
collision-free path between the initial and goal con¬ 
figurations. Solving this problem requires taking into 
account both the configuration space constraints due 
to obstacles and the nonholonomic constraints. The 
tools developed to address this issue thus combine 
motion planning and control theory techniques. Such 
a combination is possible for the class of so-called 
small-time controllable systems due to topological ar¬ 
guments (Theorem 47.2 in Sect. 47.3). 


47.2 Kinematic Constraints and Controllability 


In this section, we give the main definition of controlla¬ 
bility, using Suss man's terminology [47.9]. 

47.2.1 Definitions 

Let us denote by CS the configuration space of dimen¬ 
sion n of a given mobile robot and by q the configura¬ 
tion of this robot. If the robot is mounted on wheels, it 
is subject to kinematic constraints, linear in the velocity 
vector 

a>i(q)q = 0, i e {1,- k}. 

We assume that these constraints are linearly indepen¬ 
dent for any q. Equivalently, for each q. there exists 
m = n — k linearly independent vectors fi(q), ... ,f„,(q ) 
such that the above constraints are equivalent to 

m 

3(mi-- Um) G R"' , u ifi(s) ■ (47.1) 

i= 1 

Let us note that the choice of vectors f(q) is not 
unique. Fortunately, all the following developments are 
valid whatever choice we make. Moreover, if the linear 
constraints are smooth, vector fields / 1 ,... ,/,„ can be 
chosen smooth with respect to q. We assume this con¬ 
dition from now on. 

Let us define by 'll a compact subset of R" ! . We 
denote by E the control system defined by (47.1), with 
(u \, .. . , U m ) E 'll. 


Definition m.l 

A local and small-time controllability: 


1. E is locally controllable about configuration q 
iff the set of configurations reachable from q by 
an admissible trajectory contains a neighborhood 
of q. 

2. E is small-time controllable about configuration q 
iff the set of configurations reachable from q by an 
admissible trajectory in time less than T contains 
a neighborhood of q for any T. 


fi,... ,f m are called control vector fields of E. A sys¬ 
tem’s small-time controllable about each configuration 
is said to be small-time controllable. 

47.2.2 Controllability 

Checking the controllability properties of a system re¬ 
quires the analysis of the control Lie algebra associated 
with the system. Let us illustrate in an informal way 
what the Lie bracket of two vector fields is. Consider 
two basic motions: go along a straight line and turn on 
the spot, supported by vector fields denoted by / and g, 
respectively. Now consider the following combination: 
go forward during a time t, turn clockwise during the 
same time t, go backward during the same time t and 
then turn counterclockwise during the time t. The sys¬ 
tem reaches a configuration that is not the starting one. 
Of course when t tends to 0 the goal configuration is 
very close to the starting one. The direction indicated 
by such goal configurations when t tends to 0 corre¬ 
spond to a new vector field, which is the Lie bracket of 
/ and g. In a more mathematical formulation, the Lie 
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bracket [f, g] of two vector fields / and g is defined as 
being the vector field 3 f.g — dg.f. The k -th coordinate 
of [A g] is 

[AgP] = X] g^./P] • 

The following theorem [47.10] gives a powerful result 
for symmetric systems (a system is said to be symmetric 
when 'll is symmetric with respect to the origin). 


Theorem 47.7 

A symmetric system is small-time controllable about 
configuration q iff the rank of the vector space spanned 
by the family of vector fields f together with all their 
brackets is n at q. 


Checking the Lie algebra rank condition (LARC) on 
a control system consists in trying to build a basis of the 
tangent space from a basis (e.g., a R Hall family) of the 
free Lie algebra spanned by the control vector fields. An 
algorithm is proposed in [47. 11, 12]. 

47.2.3 Example: The Differentially-Driven 
Mobile Robot 

To illustrate the notions developed in this section, 
we consider the differentially-driven mobile robot dis¬ 
played in Fig. 47.1. The configuration space of this 
robot is R 2 xS 1 , and a configuration can be repre¬ 
sented by q = (x,y,9) where (x, y) is the position 
in the horizontal plane of the center of the wheel 
axis of the robot and 9 the orientation with respect 
to the v-axis. The rolling without slipping kinematic 


x 



Fig. 47.1 A differentially-driven mobile robot is subject to 
one linear kinematic constraint, due to the rolling without 
slipping constraint of the wheel axis 

constraint 

— x sin 9 + y cos 9 = 0 

is linear with respect to the velocity vector (Jc, y, 9 ). 
Therefore, the subspace of admissible velocities is 
spanned by two vector fields, for instance, 

/cos 9\ /0\ 

/i (q) = I sin <9 I and / 2 ( ? )=(oj. (47.2) 

The Lie bracket of these two vector fields is 
/ sin 9 \ 

h(q) = I - cos# j . 

This implies that about any configuration q, the rank of 
the vector space spanned by f\(q),f 2 (q),h(q) is 3 and, 
therefore, that the differentially-driven mobile robot is 
small-time controllable. 


47.3 Motion Planning and Small-Time Controllability 


Motion planning raises two problems: the first one ad¬ 
dresses the existence of a collision-free admissible path; 
this is the decision problem. The second one addresses 
the computation of such a path; this is the complete 
problem. 

47.3.1 The Decision Problem 

From now on, we assume that the set of collision-free 
configurations is an open subset. This implies that con¬ 
tact configurations are assumed in collision. 


Theorem 47.2 

The existence of a collision-free admissible path for 
a symmetric small-time controllable mobile robot be¬ 
tween two configurations is equivalent to the existence 


of a collision-free (not necessarily admissible) path be¬ 
tween these configurations. 


Proof: Let us consider a not necessarily admissi¬ 
ble collision-free path between two configurations q\ 
and <72 as a continuous mapping r from interval [0,1] 
into the configuration space CS such that: 

1. r(0)=<7i,r(i) = <7 2 , 

2. for any t e [0, 1], r(t) is collision free. 

Point 2 implies that for any t there exists a neighbor¬ 
hood U(t ) of r{t) included in the collision-free subset 
of the configuration space. 

Let us denote by e(t) the bigger lower bound of 
the time to collision of all the trajectories starting from 
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r(t). As the control vector (u 1 ,..., u m ) remains in the 
compact set 'Ll, e(t) > 0. 

As the system is small-time controllable about /"(f), 
the set reachable from /"(f) in time less than e(f) is 
a neighborhood of /"(f), which we denote by V{t). 

The collection {V(t),te [0,1]} is an open cover¬ 
ing of the compact set {/"(f), t e [0,1]}. Therefore, we 
can extract a finite covering, {V(t \),..., V(?i)}, where 
t\ = 0 < t 2 < ... < ti—i <ti = 1 such that for any i be¬ 
tween 1 and l— 1, V(ti)rV(t i+l ) ^ 0. For each i 
between 1 and l— 1 , we choose one configuration r, 
in V(tj) fl V(f,-+ 1 ). As the system is symmetric, there 
exists an admissible collision-free path between 0 ( 4 ) 
and r , and between r, and q(t,-\-i ). The concatenation of 
these paths is a collision-free admissible path between 
q\ and < 72 . I 

47.3.2 The Complete Problem 

In the former section, we have established that the de¬ 
cision problem, i. e., determining whether there exists 
a collision-free admissible path between two configu¬ 
rations, is equivalent to determining whether the con¬ 
figurations lie in the same connected component of 
the collision-free configuration space. In this section 
we present the tools necessary to solve the complete 
problem. These tools blend ideas from the classical 
motion planning problem addressed in Chap. 7 and 
from open loop control theory, but require specific de¬ 
velopments that we are going to present in the next 
section. Two main approaches have been devised in 
order to plan admissible collision-free motions for non- 
holonomic systems. The first one proposed by [47.13] 
exploits the idea of the proof of Theorem 47.2 by re¬ 
cursively approximating a not necessarily admissible 
collision-free path by a sequence of feasible paths. The 
second approach replaces the local method of proba¬ 
bilistic roadmap method (PRM) algorithms (Chap. 7) 
by a local steering method that connects configuration 
pairs by admissible paths 

Both approaches use a steering method. Before 
briefly describing them, we give the definition of a local 
steering method. 


Definition U7.2 

A local steering method for system E is a mapping 


■Sloe: CS x CS —Cp W ([0,1], CS) 

(q\.qi) i-> Sloe ( 01 . 92 ) 

5 where Si oc ( 0 i, 02 ) is a piecewise continuously 
differentiable curve in CS satisfying the following 
properties: 

1 • Sioc ( 0 i • 02 ) satisfies the kinematic constraints asso¬ 
ciated to E, 

2. Sioc( 01 , 02 ) connects q t to q 2 : S ioc (quq 2 )(0) = q lt 
Sioc(0i.02)(l) = 02- 


Approximation of a Not Necessarily 
Admissible Path 

A not necessarily admissible collision-free path 
r(t), t G [0,1] connecting two configurations and a lo¬ 
cal steering method Si oc being given, the approxima¬ 
tion algorithm proceeds recursively, by calling function 
approximation defined by Algorithm 47.1 with in¬ 
put /", 0 and 1 . 


Algorithm U7.1 

approximation function: inputs are a path /" and 
two abscissas fl and t2 along this path 
if Sio C (T’(tl), /"(t2)) collision free then 
return Si oc (f"(tl), /"(t 2 )) 
else 

return concat( approximation]/",tl,(tl+t 2 )/ 2 ), 
approximation!/" ,(tl+t 2 )/ 2 ,t 2 )) 

end if 


Sampling-Based Roadmap Methods 
Most sampling-based roadmap methods as described 
in Chap. 6 can be adapted to nonholonomic systems 
by replacing the connection method between pairs of 
configurations by a local steering method. This strat¬ 
egy is rather efficient for PRM algorithms. For the 
rapidly-exploring random tree (RRT) method, the effi¬ 
ciency strongly depends on the metric used to choose 
the nearest neighbor. The distance function between 
two configurations needs to account for the length of 
the path returned by the local steering method to con¬ 
nect these configurations [47.14]. 


47.4 Local Steering Methods and Small-Time Controllability 


The approximation algorithm described in the former 
section is recursive and raises the completeness ques¬ 
tion: does the algorithm finish in finite time or may it 
fail to find a solution? 


A sufficient condition for the approximation algo¬ 
rithm to find a solution in a finite number of iterations 
is that the local steering method accounts for small-time 
controllability. 
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Definition k7.3 

A local steering method 5i oc accounts for the small-time 
controllability of system E iff 

For any q e CS, for any neighborhood U of q, there 
exists a neighborhood V of q such that for any r e 
V, S\ oc (q, f) ([0, 1]) C U. 

In other words, a local steering method accounts for 
the small-time controllability of a system if it produces 
paths getting closer to the configurations it connects 
when these configurations get closer to each other. 

This property is also sufficient for probabilistic 
completeness of roadmap sampling-based methods. 

47.4.1 Local Steering Methods Accounting 
for Small-Time Controllability 

Constructing a local steering method that accounts 
for small-time controllability is a difficult task that 
has been achieved only for a few classes of systems. 
Most mobile robots studied in the domain of motion 
planning are wheeled mobile robots towing trailers or 
not. 



Fig. 47.2 (a) Two perspective views of an RS (Reeds and Shepp) 
ball: set of configurations reachable by a path of length less than 
a given distance for the Reeds and Shepp car. (b) Perspective 
views of two DD (differentially driven) balls: set of configurations 
reachable by a path of a length less than a given distance for the 
differentially-driven robot. Orientation 6 is represented on z-axis 


Steering Using Optimal Control 
The simplest system, namely the differentially-driven 
robot presented in Sect. 47.2.3 with bounded veloc¬ 
ities or the so-called Reeds and Shepp [47.15] car 
with bounded curvature have the same control vector 
fields (47.2). The difference lies in the domain of the 
control variables: 

• — 1 < Mi < 1, \u 2 \ < |«i| forRS car, 

• \u.\ \ + b\u 2 \ < \ for the differentially-driven robot, 

where b is half of the distance between the right and 
left wheels. For these systems, a local steering method 
accounting for small-time controllability can be con¬ 
structed using optimal control theory. For any admis¬ 
sible path defined over an interval I of one of these 
systems, we define a length as follows 

J |«i | for the RS car , 

/ 

J \u\ | + b\ui\ for the differentially-driven robot. 

/ 

The length of the shortest path between two configura¬ 
tions in both cases defines a metric over the configura¬ 
tion space. The synthesis of the shortest paths, i. e., the 
determination of the shortest path between any pair of 
configurations was achieved by [47.16] for the RS car 
and later by [47.17] for the differentially-driven robot. 
Figure 47.2 shows a representation of the balls corre¬ 
sponding to these metrics. 

Optimal control naturally defines a local steering 
method that associates a shortest path between these 
configurations to any pair of configurations. Let us note 
that the shortest path is unique between most pairs of 
configurations. A general result states that the collec¬ 
tion of balls of radius r > 0 centered about q induced 
by nonholonomic metrics constitutes an increasing col¬ 
lection of neighborhoods of q , the intersection of which 
is {q}. This property directly implies that local steering 
methods based on shortest paths account for small-time 
controllability. 

The main advantage of optimal control is that it 
provides both a local steering method and a distance 
metric consistent with the steering method. This makes 
the steering methods well suited for path planning al¬ 
gorithms designed for holonomic systems and using 
a distance function, such as RRT (Chap. 7), for instance. 

Unfortunately, the synthesis of the shortest paths 
has been realized only for the two simple systems de¬ 
scribed in this section. For more complex systems, the 
problem remains open. 

The main drawback of the shortest path-based steer¬ 
ing methods described in this section is that input 
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functions are not continuous. This requires an ad¬ 
ditional step to compute a time-parameterization of 
the paths before motion execution. Along this time- 
parameterization, input discontinuities force the robot 
to stop. For instance, to follow two successive arcs of 
circles of opposite curvature a mobile robot needs to 
stop between the arcs of circle in order to ensure conti¬ 
nuity of the linear and angular velocities u\ and «2- 

Steering Chained-Form Systems 
Some classes of systems can be put into the form called 
the chciined-form system by a change of variable 

Zl=Wl, (47.3) 

7.2 = 112 , (47.4) 

7s = Z.2U\ , (47.5) 

Zn=Zn-lU\- (47.6) 

Let us consider the following inputs [47.18] 

u 1 (t) = ao + fli sinwt, 

< uz(t) = bo + b\ cos cot (47.7) 

+ ... + b n — 2 cos(n — 2 )wt. 

Let Z start e M" be a starting configuration. Each 7 , (1) 
can be computed from the coordinates of Z starl and 
parameters (ao, a\, bo, b\,..., b n — 2 ). For a given 
a 1 ^ 0 and a given configuration Z starl , the mapping 
from (ao, bo, b\,b 2 ,bi) to Z( 1) is a C^-diffeomorphism 
at the origin; the system is then invertible. For n smaller 
or equal to 5, parameters (ao, bo, b\, ..., b„— 2) can 
be analytically computed from the coordinates of the 
two configurations Z start and Z goal . The corresponding 
sinusoidal inputs steer the system from Z start to Z goal . 
The shape of the path only depends on parameter eq. 
Each value of a\ thus defines a local steering method 
denoted by S^. None of these steering methods 
account for small-time controllability since for any 
Z el", S“‘ n (Z, Z)([0,1]) is not reduced to {Z}. To 
construct a local steering method accounting for 
small-time controllability from the collection of S“' n , 
we need to make a\ depend on the configurations Z\ 
and Z 2 that we want to connect 

lim a 1 (Z 1 ,Z 2 )=0, 

Z2-S-Z 1 

lim a 0 \Z 1 ,Z 2 ,a I (Z 1 ,Z 2 )]=0, 

z 2 ->-z 1 L J 

lim ^ bi [Z 1 ,Z 2 ,ai (Z 1 , Z 2 )] = 0. 

Such a construction is achieved in [47.19]. 

Steering Feedback-Linearizable Systems 
The concept of feedback linearizability (or differential 
flatness) was introduced by Fliess et al. [47.20, 21]. 


A system is said to be feedback linearizable if there 
exists an output (i. e., function of the state, input and in¬ 
put derivatives) called the linearizing output, such that 
the state and the input of the system is a function of the 
linearizing output and its derivatives. The dimension of 
the linearizing output is the same as the dimension of 
the input. 

Let us illustrate this notion with a simple example. 
We consider a differentially-driven mobile robot towing 
a trailer hitched on top of the wheel axis of the robot, 
as displayed in Fig. 47.3. The tangent to the curve fol¬ 
lowed by the center of the wheel axis of the trailer gives 
the orientation of the trailer. From the orientation of the 
trailer along the curve, we can deduce the curve fol¬ 
lowed by the center of the robot. The tangent to the 
curve followed by the center of the robot gives the ori¬ 
entation of the robot. Thus the linearizing output of this 
system is the center of the wheel axis of the trailer. By 
differentiating the linearizing output twice, we can re¬ 
construct the configuration of the system. 

Feedback linearizability is very interesting for steer¬ 
ing purposes. Indeed, the linearizing output is not 
subject to any kinematic constraints. Therefore, if we 
know the relation between the state and the linearizing 
output, planning an admissible path between two con¬ 
figurations simply consists in building a curve in R m , 



Fig. 47.3 A differentially-driven robot towing a trailer. 
Hitched on top of the wheel axis of the robot is a feedback- 
linearizable system. The linearizing output is the center of 
the wheel axis of the trailer. The configuration of the sys¬ 
tem can be reconstructed by differentiating the curve y(s), 
where 5 is an arc-length parameterization, followed by the 
linearizing output. The orientation of the trailer is given 
by r = arctan(y 2 /yi). The angle between the robot and the 
trailer is given by <p = —l arctan(dr/ds). I is the length of 
the trailer connection 
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where m is the dimension of the input with differen¬ 
tial constraints at both ends. This problem can be easily 
solved using, for instance, polynomials. 

For two input driftless systems like E, the lineariz¬ 
ing output only depends on state q. The state q depends 
on the linearizing output through the parameterization 
invariant values, namely, the linearizing output y, the 
orientation r of the vector tangent to the curve followed 
by y and the successive derivatives of r with respect to 
the curvilinear abscissa s. 

Thus, the configuration of a two input feedback- 
linearizable driftless system of dimension n can be 
represented by a vector (y, r, r 1 ,..., x "~ 3 ) represent¬ 
ing the geometric properties of the curve followed by 
the linearizing output along an admissible path passing 
through the configuration. 

Therefore, designing a local steering method for 
such a system is equivalent to associating to any pair 
of vectors (jq, n, r/,..., r" -3 ), (y 2 , T 2 , *2 > ■ ■ ■, f 2 ~ 3 ) 
a curve in the plane starting from jq and ending 
at y 2 with orientation of the tangent vector and 
successive derivatives with respect to s and equal to 
ti, x\ ,..., r" 3 at the beginning and to r 2 , x \,..., r 2 ~ 3 
at the end. This exercise is relatively easy using poly¬ 
nomials and transforming the boundary conditions 
into linear equations over the coefficients of the 
polynomials. However, taking into account small¬ 
time controllability is a little bit more tricky. Refer¬ 
ence [47.22] proposes a flatness-based steering method 
built on convex combinations of canonical curves. 

47.4.2 Equivalence Between 
Chained-Formed and 
Feedback-Linearizable Systems 

In the previous section, we proposed methods to 
steer feedback-linearizable control systems or sys- 


47.5 Robots and Trailers 

Robotics systems studied in motion planning are mainly 
those composed of a mobile robot alone or towing one 
or several trailers. The input of these systems is two di¬ 
mensional. 

47.5.1 Differentially-Driven 
Mobile Robots 

The simplest mobile robot, namely the differentially- 
driven mobile robot displayed in Fig. 47.1, is obviously 
feedback-linearizable. the trajectory of the center of the 
wheel axis (the linearizing output) of the robot gives 


terns that can be put into chained-form. We now 
give a necessary and sufficient condition for feedback- 
linearizability. 

Feed back-Li nea riza bi I ity: 

A Necessary and Sufficient 

Condition 

In [47.23], Rouchon gives conditions to check 
whether or not a system is feedback-linearizable. 
For two-input driftless systems a necessary and suf¬ 
ficient condition is the following: let us define 
as A k , k > 0 the collection of distributions (i. e., 
set of vector fields) iteratively defined by: Aq = 
span{/i,/ 2 }, Ai = span{/i,/ 2 , [/i,/ 2 ]} and A i+ 1 = A 0 + 
[Ai, Ai] with [Ai, Ai] = span{[/, g ], / e A„ g e At}. 
A system with two-dimensional input is feedback- 
linearizable iff rank(A,) = 2 + i. 

Example: The Chained-Form System 
Let us consider the chained-form system defined 
by (47.3)-(47.6). The control vector fields of this sys¬ 
tem are 

fi = (1,0, z 2 ,... ,z„-i) , 

/ 2 = ( 0 , 1 ,..., 0 ), 

rankAo = 2. If we compute /3 = \f\ , / 2 ] = (0, 0,1,0, 
..., 0), we note that rankAi = 3. By computing / = 
[f \ , /i —1 ] for i up to n, we find a sequence /■ = 
(0,..., 0, 1,0,..., 0), where 1 is at position i. There¬ 
fore, rankA,- = 2 + i for i up to n — 2 and the chained- 
form system is feedback linearizable. This conclu¬ 
sion could have been drawn in a more straight¬ 
forward way by noticing that the state can be re¬ 
constructed from (zi,z 2 ) and its derivatives. (zi,z 2 ) 
is thus the linearizing output of the chained-form 
system. 


the orientation of the robot. It can thus be steered using 
a flatness-based local steering method. 

47.5.2 Differentially-Driven Mobile Robots 
Towing One Trailer 

The differentially-driven mobile robot towing a trailer 
hitched on top of the wheel axis of the robot displayed 
in Fig. 47.3 is feedback linearizable and the linearizing 
output is the center of the wheel axis of the trailer. The 
differentially-driven mobile robot towing a trailer with 
hitched kingpin (Fig. 47.4) is also feedback lineariz- 
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x = (x u x 2 ) 


Fig. 47.4 A differentially-driven mobile robot towing 
a trailer with hitched kingpin is feedback linearizable 


able [47.24] but the relation between linearizing output 
and configuration variables is more intricate 

b sin 6 + a sin(0 + <p) 


yi = x\ — b cos 8 + L((p) 
y >2 = X 2 — b sin 6 — L(<p) 


y/a 2 + b 2 + lab cos <p 
a cos((9 + <p) + b cos 6 


y/a 2 + b 2 + lab cos (p 
where L is defined by the following elliptic integral 


/ cos a 

— - dir , 

Ja 2 + b 2 + lab cos(ct) 

0 v 

which is the linearizing output of the system. These re¬ 
lations together with the following ones 

a sin($ + (p) + b sin 8 

tan r = ---—-- , 

b cos 8 + a cos (8 + <p) 

sin(</>) 

k = - , -, 

cos (f> y/ a 2 + b 2 + lab cos <p + sin((/>) 

make it possible to reconstruct the configuration vari¬ 
ables from the linearizing output and its two first deriva¬ 
tives. 


47.5.3 Car-Like Mobile Robots 


A car-like mobile robot is constituted of a fixed rear 
wheel axis and of two steerable front wheels, the axes 
of which intersect at the center of curvature (Fig. 47.5). 
A car-like mobile robot is kinematically equivalent to 
a differentially-driven mobile robot towing a trailer 
hitched on top of the wheel axis of the robot (Fig. 47.3): 
the virtual front wheel corresponds to the differentially- 
driven robot, while the body of the car corresponds to 
the trailer. 



Fig. 47.5 Car-like mobile robot. The axes of the front 
wheels intersect at the center of curvature. The steering 
angle <p is the angle between the longitudinal axis of the 
car and a virtual front wheel in the middle of the two front 
wheels 



Fig. 47.6 Bi-steerable robot. The front and rear wheels are 
steerable and there is a relation between the rear steering 
angle and the front steering angle: a =f(cp) 

47.5.4 Bi-Steerable Mobile Robots 

The bi-steerable mobile robot (Fig. 47.6) is a car with 
front and rear steerable wheels and with a relation be¬ 
tween the front and rear steering angles. This system 
has been proved to be feedback linearizable in [47.25]. 
As for the mobile robot towing a trailer with hitched 
kingpin, the linearizing output is a moving point in the 
robot reference frame. 

47.5.5 Differentially-Driven Mobile Robots 
Towing Trailers 

Let us consider the differentially-driven mobile robot 
towing a trailer connected on top of the wheel axis of 
the robot, and let us add an arbitrary number of trailers, 
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Fig. 47.7 A truck towing a towbar trailer with hitched 
kingpin and a differentially-driven mobile robot towing 
two trailers with hitched kingpin: two open problems for 
exact path planning for nonholonomic systems 


the whole system by differentiating a sufficient number 
of times. The system is therefore feedback lineariz- 
able and the linearizing output is the center of the last 
trailer. 

Combining the above reasoning with the sys¬ 
tems reviewed in this section, we can build hy¬ 
brid feedback-linearizable trailer systems. For instance, 
a differentially-driven mobile robot towing an arbitrary 
number n of trailers each one connected on top of the 
wheel axis of the previous trailer, except the last trailer 
with hitched kingpin is feedback linearizable. Simply 
consider the two last trailers as a feedback-linearizable 
system composed of a mobile robot with one trailer as 
in Fig. 47.4. The linearizing output of this system en¬ 
ables us to reconstruct the trajectory of the two last 
trailers. The center of trailer n — 1 is a linearizing output 
for the mobile robot towing the n— 1 last trailers. 

47.5.6 Open Problems 


each one connected on top of the wheel axis of the pre¬ 
vious one. By differentiating once the curve followed 
by the center of the last trailer, we get the orientation 
of the last trailer (this orientation coincides with the 
orientation of the tangent to the curve). If we know 
the orientation and the position of the center of the 
last trailer along the path, we can reconstruct the curve 
followed by the center of the previous trailer. Repeat¬ 
ing this reasoning, we can reconstruct the trajectory of 


Finally, all the systems for which we are able to plan 
exact motions between arbitrary pairs of configurations 
are included in the large class of feedback-linearizable 
systems. We have indeed seen that chained-form sys¬ 
tems are also part of this class. For other systems, no 
exact solution has been proposed up to now. For in¬ 
stance, neither of the systems displayed in Fig. 47.7 is 
feedback linearizable. They do not satisfy the necessary 
condition of Sect. 47.4.2. 


47.6 Approximate Methods 

To deal with nonholonomic systems not belonging to 
any class of systems for which exact solutions exist, 
numerical approximate solutions have been developed. 
We review some of these methods in this section. 

47.6.1 Forward Dynamic Programming 

In [47.26] Barraquand and Latombe propose a dy¬ 
namic programming approach to nonholonomic path 
planning. Admissible paths are generated by a sequence 
of constant input values, applied over a fixed interval of 
time 8 1 . Starting from the initial configuration the search 
generates a tree: the children of a given configuration q 
are obtained by setting the input to a constant value and 
integrating the differential system over 8 1 . The configu¬ 
ration space is discretized into an array of cells of equal 
size (i. e., hyper-parallelepipeds). A child q' of a config¬ 
uration q is inserted in the search tree if and only if the 
computed path from q to q' is collision free and q' does 
not belong to a cell containing an already generated 


configuration. The algorithm stops when it generates 
a configuration belonging to the same cell as the goal 
(i. e., it does not necessarily reach the goal exactly). 

The algorithm has been proved to be asymptotically 
complete with respect to both 8? and the size of the cells. 
As a brute force method, it remains quite time consum¬ 
ing in practice. Its main interest is that the search is 
based on Dijkstra’s algorithm, which allows taking into 
account optimality criteria such as the path length or the 
number of reversals. Asymptotical optimality to gener¬ 
ate the minimum of reversals is proved for the car-like 
robot alone. 

47.6.2 Discretization of the Input Space 

In [47.27] Divelbiss and Wen propose a method to 
produce an admissible collision-free path for a non¬ 
holonomic mobile robot in the presence of obstacles. 
They restrict the set of input functions over the subspace 
spanned by Fourier basis over interval [0, 1]. An in- 
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put function is thus represented by a finite-dimensional 
vector A. Reaching a goal configuration thus becomes 
a nonlinear system of equations, the unknowns of which 
are the coordinates A,’s of A. The authors use the 
Newton-Raphson method to find a solution. Obstacles 
are defined by inequality constraints over the configura¬ 
tion space. The path is discretized into N samples. The 
noncollision constraint expressed at these sample points 
yields the inequality constraint over vector A. These in¬ 
equality constraints are turned into equality constraints 
through function g defined as 



Reaching a goal configuration while avoiding obstacles 
thus becomes a nonlinear system of equations over vec¬ 
tor A again solved using the Newton-Raphson method. 
The method is rather efficient for short maneuvers. The 
main difficulty is to tune the order of the Fourier expan¬ 


sion. Long motions in cluttered environments require 
a higher order, while motion in empty space can be 
solved with a low order. The authors do not mention 
the problem of numerical instability in the integration 
of the dynamic system. 

47.6.3 Input-Based Rapidly 

Exploring Random Trees 

The RRT algorithms described in Chap. 7 can be used 
without a local steering method to plan paths for non- 
holonomic systems. New nodes can be generated from 
existing nodes by applying random input functions over 
an interval of time [47.28]. The main difficulty consists 
in finding a distance function that really accounts for 
the distance the system needs to travel to go from one 
configuration to another. Moreover, the goal is never ex¬ 
actly reached. This latter drawback can be overcome by 
post-processing the path returned by RRT using a path 
deformation method, as described in [47.29]. 


47.7 From Motion Planning to Obstacle Avoidance 


Up to now we have described motion planning tech¬ 
niques. Their objective is to compute a collision-free 
trajectory to the target configuration that complies with 
the vehicle constraints. They assume a perfect model 
of the robot and scenario. The advantage of these 
techniques is that they provide complete and global 
solutions of the problem. Nevertheless, when the sur¬ 
roundings are unknown and unpredictable, these tech¬ 
niques fail. 

A complementary way to face the motion problem 
is obstacle avoidance. The objective is to move a ve¬ 
hicle towards a target location free of collisions with 
the obstacles collected by the sensors during motion 
execution. The advantage of reactive obstacle avoid¬ 
ance is to compute motion by introducing the sensor 
information within the control loop, used to adapt the 


motion to any contingency incompatible with initial 
plans. 

The main cost of considering the reality of the world 
during execution is locality. In this instance, if global 
reasoning is required, a trap situation could occur. De¬ 
spite this limitation, obstacle avoidance techniques are 
mandatory to deal with mobility problems in unknown 
and evolving surroundings. 

Notice that methods have been developed to com¬ 
bine both the global point of view of motion planning 
and the local point of view of obstacle avoidance. How 
can we consider robot perception at the planning level? 
This is so-called sensor-based motion planning. Several 
variants exists, such as the Bug algorithms initially in¬ 
troduced in [47.30]. However, non of them consider the 
practical context of nonholonomic mobile robots. 


47.8 Definition of Obstacle Avoidance 


Let JA be the robot (a rigid object) moving in the 
workspace TV, whose configuration space is CS. Let q 
be a configuration, q, this configuration in time t, 
■A{q,) G TV the space occupied by the robot in this con¬ 
figuration. In the vehicle there is a sensor, which in q, 
measures a portion of the space S(q r ) C TV identifying 
a set of obstacles 0(q,) C TV. 

Let u be a constant control vector and u(q,) this 
control vector applied in q, during time 8 1 . Given u(q t ), 
the vehicle describes a trajectory q,+& r =f(u,q t ,$t). 


with 8 1 > 0. Let <7, r be the set of configurations of 
the trajectory followed from q, with 8? G [0, T], a given 
time interval. T > 0 is called the sampling period. Let 
F : C5 x C5 —s- be a function that evaluates the 

progress of one configuration to another. 

47.8.1 Obstacle Avoidance Problem 

Let * 7 target be a target configuration. Then , in time t[ 
the robot LA is in q ti , where a sensor measurement 
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a ) Unknown space 

Target 

Obstacles (O) 



Fig. 47.8 (a) Obstacle avoidance problem consists of com¬ 
puting motion control that avoids collisions with the ob¬ 
stacles gathered by the sensors whilst driving the robot 
towards the target location, (b) The result of applying this 
technique at each time is a sequence of motions that drive 
the vehicle free of collisions to the target 

is obtained and thus an obstacle description 

0(q„). 


The objective is to compute a motion control 
such that ( i ) the trajectory generated is free of col¬ 
lisions with the obstacles JA(Q ti< T) fl 0((?/,) = 0; 
and (ii) it makes the vehicle progress to the target 
location F{qt^q target) Fiq^-y/, ^target)* 

The result of solving this problem at each sample 
time (Fig. 47.8a) is a sequence of motion controls {«i 
... «„} computed in execution time that avoids the ob¬ 
stacles gathered by the sensors, while making the vehi¬ 
cle progress towards the target location in each configu¬ 
ration {q tl ... t/target} (Fig. 47.8b). Notice that the motion 
problem is a global problem. The obstacle avoidance 
methods are local and iterative techniques to address 
this problem. The disadvantage of locality (trap situa¬ 
tions) is counteracted with the advantage of introducing 
the sensory information within the control cycle (to take 
into account the reality of the world in execution). 

There are at least three aspects that affect the devel¬ 
opment of an obstacle avoidance method: the avoidance 
technique, the type of robot-sensor and the type of 
scenario. These subjects correspond to the next three 
sections. First, we describe the obstacle avoidance tech¬ 
niques (Sect. 47.9). Second, we discuss the techniques 
to adapt a given obstacle avoidance method to work 
on a vehicle taking into account the shape, kinematics, 
and dynamics (Sect. 47.10). The sensory processing is 
detailed in Chaps. 5 and 25 of this book. Finally, the us¬ 
age of an obstacle avoidance technique on a vehicle in 
a given scenario is highly dependent on the scenario na¬ 
ture (static or dynamic, unknown or known, structured 
or not, or its size, for example). Usually, this problem is 
associated with the integration of planning and obstacle 
avoidance (Sect. 47.11). 


47.9 Obstacle Avoidance Techniques 

We describe here a taxonomy of obstacle avoidance 
techniques and some representative methods. First there 
are two groups, methods that compute the motion in one 
step and methods that do it in more than one. Methods 
of one step directly reduce the sensor information to 
motion control. There are two types: 

• The heuristic methods were the first techniques 
used to generate motion based on sensors. The ma¬ 
jority of this work was derived in classic planning 
methods and will not be described here. See [47.1, 
30-33] for some representative work. 

• The methods of physical analogies assimilate the 
obstacle avoidance to a known physical problem. 
We discuss here the potential field methods [47.34, 
35]. Other works are variants adapted to uncertain 


models [47.36] or that use other analogies [47.37— 
39]. 

Methods with more than one step compute some 
intermediate information, which is processed next to 
obtain the motion: 

• The methods of subset of controls compute an in¬ 
termediate set of motion controls, and next they 
choose one of them as a solution. There are two 
types: (i) methods that compute a subset of motion 
directions. We describe here the vector field his¬ 
togram (VFH) [47.40] and the obstacle restriction 
method [47.41] (see [47.42] for three-dimensional 
(3-D) workspaces). Another method is the steer 
angle field approach [47.43]. (ii) Methods that com- 
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pute a subset of velocity controls. We describe here 
the dynamic window approach (DWA) [47.44] and 
velocity obstacles [47.45] (see [47.46] for moving 
obstacles). Another method based on similar prin¬ 
ciples but developed independently is the curvature 
velocity method [47.47]. 

• Finally, there are methods that compute some high- 
level information such as intermediate information, 
which is translated next in motion. We describe 
the basic versions of the nearness diagram naviga¬ 
tion [47.48,49]. The reader is directed to [47.50- 
52] for further improvements. 

All the methods outlined here have advantages and 
disadvantages, depending on the navigation context, 
like uncertain worlds, motion at high speeds, motion 
in confined or troublesome spaces, etc. Unfortunately 
there is no metric available to quantitatively measure 
the performance of the methods. However, for an exper¬ 
imental comparison in terms of their intrinsic problems 
see [47.48]. 

47.9.1 Potential Field Methods 


The potential field method (PFM) uses an analogy in 
which the robot is a particle that moves in the configu¬ 
ration space under the influence of a force field. While 
the target location exerts a force that attracts the par¬ 
ticle F att , the obstacles exert repulsive forces F rep . At 
each time t„ the motion is computed to follow the di¬ 
rection of the artificial force induced by the sum of 
both potentials F tol (q ti ) = F M (q ti ) +F lep (q, i ) (the most 
promising motion direction), Fig. 47.9. 

Example 


F att (#*/) — -^att W 4target 
^rep (<7 ti ) — 

*rep Y, f -[ 

0 


(47.8) 

-y) n Pi if d(q ti ,pj) < d 0 , 
otherwise 

(47.9) 
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Fig. 47.9 (a) Computation of the motion direction with 
a potential field method. The target attracts the particle F at t 
while the obstacle exerts a repulsive force F rep . The re¬ 
sulting force F to t is the most promising motion direction, 
(b) Motion directions computed in each point of the space 
with the classic method 

Example 


F rC p (*7/,) — 


^ ,£ p J2 ( j 


aq ti 


-'\[2ad(q li ,p j )-q 2 ti ]l PJ 

if q,. > 0 , 

0 otherwise 

(47.10) 


where F att and F rep are the constants of the forces, do 
is the influence distance of the obstacles pj, q tj is the 
current vehicle configuration and n 9target and n pj are the 
unitary vectors that point from q tj to the target and each 
obstacle p r From F tot (q ti ), the control u, can be ob¬ 
tained with a position or force control [47.53]. 

This is the classic version, where the potentials only 
depend on the current vehicle configuration. Comple- 
mentarily the generalized potentials depend also on the 
instantaneous robot velocity and vehicle accelerations. 


where q t is the current robot velocity, n q the unitary 
vector pointing in the direction of the robot velocity, 
and a is the maximum vehicle acceleration. This ex¬ 
pression arises when the repulsive potential is defined 
as the inverse of the difference between the estimated 
time until a collision takes place and the time needed to 
stop the robot until full backward acceleration. Notice 
how the repulsive only affect in the direction of motion 
of the vehicle, as opposed to the classic potential. For 
a comparison of the classic and generalized versions, 
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and the relation with the way to compute the motion 
controls next, see [47.53]. This method is widely used 
because it is easy to understand and due to its clear 
mathematic formalism. 

47.9.2 Vector Field Histogram 

The vector field histogram (VFH) solves the problem 
in two steps by computing a set of candidate motion 
directions and then selecting one of them. 

Candidate Set of Directions 
Firstly the space is divided into sectors from the robot 
location. The method uses a polar histogram H con¬ 
structed around the robot, where each component rep¬ 
resents the obstacle polar density in the corresponding 
sector. The function that maps the obstacle distribution 
in sector k on the corresponding component of the his¬ 
togram h k (q ti ) is 

h k (q ti ) = f P(p) n . (1 - } dp . (47.11) 

J \ ^-max / 

Qt 

The dominion of integration is = {p e 'WSp e 
kAd(q tj ,p) <do}. The density h k (q t[ ) is proportional 
to the probability P(r ) that a point was occupied by an 
obstacle, and to a factor that increases when the distance 
to the point decreases. 

Typically the resulting histogram has peaks (di¬ 
rections with high density of obstacles) and valleys 
(directions with low density). The set of candidate 
directions is the set of adjacent components with den¬ 
sity lower than a given threshold, and closest to the 
component that contains the target direction. This 
set of components (sectors) is called the selected 
valley, and represents a set of candidate directions 
(Fig. 47.10). 

Motion Computation 

The objective of the next step is to choose a direction 
of this set. The strategy is to apply three heuristics that 
depend on the component that contains the target or on 
the size of the selected valley. The cases are checked in 
sequence: 

1. Case 1: goal sector in the selected valley. Solution: 
k so i = ^target, where ^target is the sector that contains 
the goal location. 

2. Case 2: goal sector not in the selected valley and 
number of sectors of the valley greater than m. So¬ 
lution: k so 1 = kj ± m/2, where m is a fixed number 
of sectors and k, the sector of the valley closer to the 

^target ■ 


a) e sol 



90 ° 


Selected set of 
directions 

270 ° 

b) 



Fig.47.10a ,b Computation of the motion direction 6 so 1 
with VFH. (a) Robot and obstacle occupancy distribution, 
(b) The candidate valley is the set of adjacent components 
with values lower than the threshold. The navigation case is 
Case 3, since the sector of the target ^target is not in the val¬ 
ley and the number of sectors is lower than a fixed quantity 
m (m = 8, i. e., 45°). Thus the solution is k so i = (kj + kj)/ 2, 
whose bisector is 0 so i in (a). The bisectors of kj and kj are 
9i and 6), respectively 

3. Case 3: goal sector not in the selected valley and 
number of sectors of the valley lower or equal to m. 
Solution: k so i = (kj + kj)/ 2, where kj and kj are the 
extreme sectors of the valley. 

The result is a component or sector k so i, whose bi¬ 
sector is the direction solution 9 so \. The velocity v so \ is 
inversely proportional to the distance to the closest ob¬ 
stacle. The control is h, = (w so i, 9 S 0 i). 

VFH is a method formulated to work with probabil¬ 
ity obstacle distributions, and thus, it is well adapted to 
work with uncertain sensors as ultrasonic sonars. 
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47.9.3 The Obstacle Restriction Method 

The obstacle restriction method (ORM) solves the prob¬ 
lem in three steps, where the result of the two first steps 
is a set of candidate motion directions. The first step is 
to compute an instantaneous subgoal if necessary. The 
second one associates a motion constraint to each obsta¬ 
cle and joins them next to compute the set of desirable 
directions. The last step is an strategy to compute the 
motion given this set. 

Instantaneous Target Selection 
This step computes a subgoal when is better to direct 
the motion towards a given zone of the space (which 
ameliorates the situation to reach the goal latter), rather 
than directly towards the goal itself. The subgoals are 
located in between obstacles or in the edge of an ob¬ 
stacle (Fig. 47.11a). Next, the process is to check with 
a local algorithm whether the goal can be reached from 
the robot location. If it is not, then the closest reachable 
subgoal to the goal is selected. To check if a point can 
be reached, there is a local algorithm that computes the 
existence of a local path that joins two locations [47.48]. 

Let x a and xi, be two locations of the space, R the 
robot radius, and L a list of obstacle points, wherep, is an 
obstacle of the list. Let J4. and 2> be the two semiplanes 
divided by the line that joins x„ and jcj ,. Then, if for all 
the points of L, cKpj.pk) > 2 R (with Pj e ./I and/?/ e ®), 
there is a collision free path that joins both locations. If 
this condition is not satisfied, then there is no local path 
(although a global one could exist). The interesting re¬ 
sult is when the result is positive, since the guarantee that 
one point can be reached from the other exists. 

The result of the process is the goal or an instan¬ 
taneous subgoal (from now this location is called the 
target location). Note that this process is general and 
can be used as a preprocess step by other methods to 
instantaneously validate the goal location or to compute 
an instantaneous subgoal to drive the vehicle. 

Candidate Set of Directions 
For each obstacle i a set of not desirable directions for 
motion S ( ' |D is computed (motion constraint). This set 
is the union of two subsets Sj and S 2 . Sj represents 
the side of the obstacle not suitable to do the avoid¬ 
ance and S 2 is an exclusion area around the obstacle 
(Fig. 47.11b). The motion constraint for the obstacle 
is the union of both sets Sj lD = Sj USj. The set of de¬ 
sired directions of motion is the complementary So = 
{[-jr, n] \ S nD }, where S nD = U ; S,', D . 

Motion Computation 

The final step is to select a direction of motion. There 
are three cases that depend on the set of desirable direc- 


a) Goal 



Fig. 47.11 (a) Distribution of the subgoals Xj. The selected 
instantaneous target location is xn. The set of candidate di¬ 
rections is S„d and the solution is 6 so 1 (the second case), 
(b) The two sets of not desirable directions Si and S2 for 
a given obstacle 

tions Sd and on the target direction target ■ The cases are 
checked in sequence: 

1. Case 1: So ^ 0 and (target e Sd- Solution: 0 so i = 

(target- 

2. Case 2: So i=- 0 and (target ^ Sd- Solution: 0 so j = 
@ii m , where 9n m is the direction of So closest to 

(target- 

3. Case 3: S D = 0. Solution: 6ta,i = + 0D/2, 

where <p[ im and c/>[ im are the directions of So, and Sot 
closer to (target (So r and So, are the set of desirable 
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directions of obstacles on the left and right-hand 
sides of the target, respectively). 

The result is a direction of motion solution f9 S0 |. The 
velocity v so \ is inversely proportional to the distance 
to the closest obstacle. The control is m, = (v so i, 0 so i). 
This is a geometric-based method based on cases. The 
advantage is that it has been demonstrated to address 
effective motion in confined spaces. 

47.9.4 Dynamic Window Approach 

The dynamic window approach (DWA) 
is a method that solves the problem in two steps, by 
computing as intermediate information a subset of the 
control space U. For simplicity, we consider a mo¬ 
tion control such as translational and rotational velocity 
(v, w)\ 'll is defined by 

'LL = {(v, w) e R 2 \ 

€= [ tt max , t^nax] A 

W G [ tn max , tn max ] } . (47.12) 

Set of Candidate Controls 

The candidate set of controls Ur contains the controls: 

(i) within the maximum velocities of the vehicle U, 

(ii) that generate safe trajectories Ua , and (iii) that 
can be reached within a short period of time given 
the vehicle accelerations Uq. The set Ua contains the 
admissible controls. These controls can be canceled 
before collision by applying the maximum decelera- 

tlOn (tty , tty) ) 

U A = {(v,w) eU\v < yj2 d obs a v A w < V2 6 ohs a w } , 

(47.13) 

where d 0 b st and 0 o b S t are the distance to the obstacle and 
the orientation of the tangent to the trajectory in the ob¬ 
stacle. The set Ud contains the controls reachable in 
a short period 

U D = { (v, w) G U \ 

v G [v 0 — a v T, v 0 + a v T] A 
w e[w 0 -a w T,w 0 +a w T]} , (47.14) 

where q tj = ( v D , w a ) is the current velocity. 

The resulting subset of controls is (Fig. 47.12) 

U R = U n U A n U D ■ (47.15) 

Motion Computation 

The next step is to select one control h, g Ur. The 
problem is set out as the maximization of an objective 




Wmax 
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(fo,w 0 ) 
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U 

, -Wmax 




Fig. 47.12 Subset of controls Ur = U (~l Ua fl Ud, where 
U contains the controls within the maximum velocities, 
Ua the admissible controls, and Ur, the controls reachable 
in a short period of time 

function 

G(u) = a\ ■ Goal(w) + a 2 ■ Clearance(«) 

+ (23 ■ Velocity(w) . (47.16) 

This function is a compromise among Goal(u) 
that favors velocities that offer progress to the goal, 
Clerance(M) that favors velocities far from the ob¬ 
stacles, and Velocity(w) that favors high speeds. The 
solution is the control h, that maximizes this function. 

DWA solves the problem in the control space using 
information of the vehicle dynamics, thus the method 
is well adapted to work on vehicles with slow dynamic 
capabilities or that work at high speeds. 

47.9.5 Velocity Obstacles 

The method of velocity obstacles (VO) solves the prob¬ 
lem in two steps, by computing as intermediate infor¬ 
mation a subset of the U. The framework is equal to 
that of DWA. The difference is that the computation of 
the set of safe trajectories Ua takes into account the 
velocity of the obstacles, which is described next. 

Let Vj be the velocity of obstacle i (which became 
enlarged with the vehicle radius occupying an area B,) 
and 11 a given vehicle control. The set of colliding rela¬ 
tive velocities is called collision cone 

CC, = {«,|A,p|B,/0j , (47.17) 

where A, is the direction of the unitary vector h, = 
Uj — v,. The velocity obstacle is this set in a common 
absolute system of reference 

VOj = CCi ® Vi , (47.18) 

where ® is the Minkowski vector sum. The set of un¬ 
safe trajectories is the union of the velocity obstacles for 
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Fig. 47.13 Subset of not safe controls Ha = VO 1 U VO 2 . 
A control vector out of this velocity generates a collision- 
free motion with the moving obstacles 

each moving obstacle Ha = U {VOi (Fig. 47.13). The 
advantage of this method is that takes into account the 
velocity of the obstacles; thus it is well suited in dy¬ 
namic scenarios. 

47.9.6 Nearness Diagram Navigation 

This method is more a methodology to design obstacle 
avoidance methods rather than a method in itself. Near¬ 


ness diagram navigation (ND) is an obstacle avoidance 
method obtained with a geometric implementation fol¬ 
lowing this methodology. The idea behind is to employ 
a divide and conquer strategy based on situations to 
simplify the obstacle avoidance problem following the 
situated-activity paradigm [47.54]. First, there is a set 
of situations that represent all the cases among robot 
locations, obstacles, and target locations. Also, for each 
case there is a motion law associated with it. During the 
execution phase, at time t, one situation is identified and 
the corresponding law is used to compute the motion. 

Situations 

The situations are represented in a binary decision tree. 
The selection of a situation depends on the obstacles 
0(q tj ), on the robot location q tj and target q Vdrgel . The 
criteria are based on high-level entities, like a security 
distance around the robot bounds and a motion area 
(that identifies suitable areas of motion). For example, 
one criterion is if there are obstacles within the security 
zone. Another is if the motion area is large or narrow. 
The result is only one situation, since by definition and 
representation (binary decision tree) the set of situations 
is complete and exclusive (Fig. 47.14). 

Actions 

Associated to any situation there is an action that com¬ 
putes the motion to adapt the behavior to the case 


a) 


Situations Actions 


Decision tree 


Goal location 



HSNR-► 


.flsT] —- 

.furl—- 


Robot location 


Motion 
commands 
(v , W’) 



b) 


Target 0disc 



Fig. 47.14 (a) Design diagram of the method. Given the obstacle information and the target location, one situation is identified 
given some criteria. Next the associated action is executed computing the motion, (b) Example of computation of the solution 
of the ND (geometric implementation). The first step is to identify the situation. There are no obstacles closer than the security 
distance D s . Next q talget is not within the motion area. Third, the motion area is wide. With these three criteria we identify the cur¬ 
rent situation, HSWR. In this situation, the associated action computes the control as h, = (u so i, 9 so \), where d so i is the maximum 
velocity and # so i is computed as a deviation a from the limit direction (9disc of the motion area closer to the target direction 
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represented by each situation. At a high level the ac¬ 
tions describe the behavior desired in each situation. 
For example, one situation is when there are no obsta¬ 
cles within a security area and the goal is within the 
motion area (HSGR). The solution is to move towards 
the goal. Another situation is when there are no obsta¬ 
cles within a security area, and the goal is not within 
the motion area but is wide (HSWR). The solution is to 
move towards the limit of the motion area but clearing 
the security area of obstacles. 


The interesting aspect of this method is that it 
employs a divide and conquer strategy to solve the 
navigation. As a consequence, it simplifies the diffi¬ 
culty of the problem. The first advantage is that the 
methodology is described at a symbolic level and, thus, 
there are many ways to implement the method. Sec¬ 
ond, a geometric implementation of this methodology 
(ND method) has been demonstrated to address diffi¬ 
cult navigation cases, which translate in to achieve safe 
navigation in dense, complex, and difficult scenarios. 


47.10 Robot Shape, Kinematics, and Dynamics in Obstacle Avoidance 


There are three aspects of the vehicle that have to be 
taken into account during the obstacle avoidance pro¬ 
cess: shape, kinematics, and dynamics. The shape with 
the kinematics is a geometric problem that involves the 
representation of the vehicle configurations in collision, 
given the admissible trajectories Q t<00 . The dynamics 
involves the accelerations and temporal considerations, 
and is derived in two aspects: (i) choosing a control 
reachable in a short period of time T given the cur¬ 
rent velocity q tj and the maximum accelerations, (ii) 
Taking the braking distance into account, so that after 
a control execution, the vehicle can always stop before 
collision by applying the maximum deceleration (im¬ 
proving safety). 

The problem of shape, kinematics, and dynamics in 
obstacle avoidance has been taken into account from 
three different points of view: 

1. Designing a way to incorporate the constraints 
within the method (Sect. 47.9.4) 

2. Developing techniques that abstract the vehicle as¬ 
pects from the application of the method [47.55-58] 

3. By techniques that break down the problem into 
subproblems and incorporate the aspects in se¬ 
quence [47.59-61] after method usage. 

47.10.1 Techniques that Abstract Vehicle 
Aspects 


Abstraction Construction 

For these vehicles, the configuration space CS is three- 
dimensional. The idea is to construct, centered in the 
robot at each time 4 , the manifold of the configuration 
space ARM(q tj ) = ARM defined by elementary circular 
paths. The function that defines the manifold is 

9=f(x,y) = 

if v > 0 , 
otherwise . 

(47.19) 

It is easy to see that function / is differentiable in 
R 2 \(0,0). Thus (x,y,f(x,y)) defines a two-dimen¬ 
sional manifold in R 2 x S 1 when ( x , y) e R 2 \(0,0). This 
manifold ARM contains all the configurations that can 
be reached at each step of the obstacle avoidance. 

Next, in the ARM one computes the exact region of 
the configurations in collision COarm given any shape 
of the robot (i. e., obstacle representation in the man¬ 
ifold). Given an obstacle point (x p ,y p ) and a point of 
the robot bounds (x r ,y r ), a point (* 5 , 34 ) of the COarm 
boundary is obtained by 

x s = (xf + Xi) ■ a , 

y s = (yf-yd-a , ( 47 . 20 ) 


arctan 2 I x. 


2 2 

x-y 


2v 


— arctan 2 I x, — 


2 2 
xr -y 


2y 


These techniques are based on constructing an abstrac¬ 
tion layer between the aspects of the vehicle and the 
obstacle avoidance method, in such a way that when the 
method is applied its solutions already take into account 
these aspects [47.55-58]. We consider here vehicles 
whose elementary paths obtained under execution of 
constant controls can be approximated by circular arcs 
(for example, a differential-drive robot, syncro-drive, or 
a tri-cycle). To simplify, a control is translational and 
rotational velocity u = (v,w). The set of reachable con¬ 
trols "Ua given the current velocity q t = (vo, Wq) and the 
maximum accelerations (a v , a w ) is obtained by (47. 14). 


with 

a = {[{yf -yf) + (Xf-xj)] [Of -y ,) 2 + {Xf -jc,) 2 ]} 
x [Of “ - v ') 4 + 2 $ +x*)(y'f~ yd 2 

This result is used to map the robot bounds for all ob¬ 
stacles in the manifold, computing the exact shape of 
the COarm ■ Next, one compute the nonadmissible con¬ 
figurations CNAarm in the manifold ARM, which cor¬ 
respond to configurations that once reached at a given 
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Fig. 47.15 (a) Robot location, obstacle information and 
goal location, (b) The abstraction layer in ARM P . In this 
representation the obstacles are the nonadmissible region 
cna ar M in ARM and that the motion is omnidirectional 
(applicability conditions of many obstacle avoidance meth¬ 
ods). The method is applied to obtain the most promising 
direction /3 so i, used to obtain the configuration solution 
in the set of reachable configurations RC arm . Finally, the 
solution is the control m so i that reaches this configuration 
at time T. This control complies with the kinematics and 
dynamics and takes into account the exact vehicle shape 


omnidirectional point in a bidimensional space free of 
any constraint. 

Method Application 

The final step is to apply the avoidance method on the 
ARM P to avoid the CNA p arm regions. The method so¬ 
lution soi is the most promising direction in ARM 1 ’. 
This direction is used to select and admissible location 
<j^ ol CNA arm in the set of dynamic reachable config¬ 
urations RC arm . Finally, the control w so 1 that leads to 
this location at time T is selected. By construction, this 
control is kinematically and dynamically admissible, 
avoids collisions with the exact vehicle shape, and takes 
into account the braking distance (Fig. 47.15). 

Note how with these techniques, the three- 
tridimensional obstacle avoidance problem is converted 
into the simple problem of moving a point in a two- 
dimensional space without kinematic and dynamic re¬ 
strictions (applicability conditions of many existing 
methods). Thus, many existing methods can be applied 
in this abstraction, and as a consequence, the solutions 
take the vehicle constraints into account. 

47.10.2 Techniques of Decomposition 
in Subproblems 

These techniques address the obstacle avoidance prob¬ 
lem taking into account the vehicle constraints by 
breaking down this problem into subproblems: 

1. Obstacle avoidance 

2. Kinematics and dynamics 

3. The shape. 

Each subproblem is dealt with in sequence 
(Fig. 47.16). 

Obstacle Avoidance 

First the obstacle avoidance method is used by assum¬ 
ing a circular and omnidirectional vehicle. The solution 
is the most promising motion direction and the velocity 
t<i = (tti, 0i) to direct the vehicle towards the target. 


velocity in time T, the vehicle cannot be stopped by 
applying the maximum deceleration without collision 
(i. e., there is not enough braking distance). These re¬ 
gions CNAarm are the COarm enlarged by a magnitude 
that depends on the maximum vehicle accelerations. 
The set of configurations RCarm reachable by controls 
in a short period of time Ti A is also computed in the 
ARM. Finally, a change of coordinates is applied to 
ARM leading to ARM p . Its effect is that the elementary 
circular paths become straight segments in the mani¬ 
fold. As a consequence, the problem now is to move an 


Kinematics and Dynamics 
Second, this control is converted into a control that 
complies with the kinematics and dynamics, which 
tends to align the vehicle with the instantaneous mo¬ 
tion direction 9 1 of«i. For example, [47.60,61] modify 
the output of the obstacle avoidance method by a feed¬ 
back action that aligns the vehicle with the direction 
solution in a least squares fashion. Moreover, [47.59] 
use a dynamic controller of the vehicle. This con¬ 
troller models the behavior of the vehicle as it would 
be pulled by a virtual force, and computes the mo- 
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Steps 
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1 
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Fig. 47.16 Decomposition of the obstacle avoidance problem into 
subproblems that are addressed in steps, successively incorporating 
the vehicle aspects 

tion that would result after applying this force during 
a short period of time. The motion generated is the 


new control. In order to use the controller, the previ¬ 
ous control u 1 is converted into an instantaneous force 
F = (9i, F mlix (vi/ v mm )) input of the controller, which 
computes a control ui that complies with the kinematic 
and dynamics. 

Shape 

The final step is to assure that the control ui obtained in 
the previous step avoids collisions with the exact shape 
of the vehicle. To do this, a shape corrector is used to 
check collisions by dynamic simulation of the control. 
If there is collision, there is a search in the set of reach¬ 
able controls (47.14) until one of them is collision free. 
The result of this process is a motion control h, that 
guarantees obstacle avoidance and complies with the 
kinematics and dynamics of the vehicle. 


47.11 Integration Planning - Reaction 


In this section we show how the obstacle avoidance 
methods are integrated in real systems. On the one hand, 
the obstacle avoidance methods are local techniques to 
address the motion problem. Thus, they are doomed to 
fall in local minima that translate in trap situations or 
cyclic motions. This reveals the necessity of a more 
global reasoning. On the other hand, motion planning 
techniques compute a geometric path free of collisions, 
which guarantees global convergence. However, when 
the scenarios are unknown and evolve, these techniques 
fail, since the precomputed paths will almost surely col¬ 
lide with obstacles. It seems clear that one key aspect 
to build a motion system is to combine the best of both 
worlds: the global knowledge given by motion planning 
and the reactivity of the obstacle avoidance methods. 

The most extended ways to specify the interaction 
between the deliberation and reaction are: (i) to pre¬ 
compute a path to the target deformed in execution as 
a function of the changes in the scenario obtained from 
the sensor information (systems of path deformation), 
for example, [47.62-67]. (ii) To use a planner at a high 
frequency with a tactical role, leaving the degree of ex¬ 
ecution to the reactor [47.68-73]. 

47.11.1 Systems of Path Deformation 

Elastic bands is a method that initially assumes the 
existence of a geometric path to the target location 
(computed by a planner). The path is assimilated with 
a band, subjected to two types of forces: an inner con¬ 
traction force and an external force. The inner force 
simulates the tension of a strip and maintains the stress. 
The external force is exerted by the obstacles and moves 
the band far from them. During the execution, the new 


obstacles produce forces that remove the band far from 
them, guaranteeing their avoidance. These methods are 
described in Chap. 37. 

An extension of the method of path deformation 
was proposed in [47.64] for nonholonomic systems 
(|<£CHJMi!lldM). Even though the objective is the same 
as for mobile robots without kinematic constraints, 
avoiding obstacle while following a trajectory, the con¬ 
cepts are completely different. The trajectory r of 
a nonholonomic system is completely defined by an ini¬ 
tial configuration F(0) and the value of the input func¬ 
tion u G C 1 (/, R'"), where I is an interval. The trajectory 
deformation method for nonholonomic systems is thus 
based on the perturbation of the input function of the 
current trajectory in order to achieve three objectives: 

1. Keeping the nonholonomic constraints satisfied 

2. Getting away from obstacles detected online by on¬ 
board sensors 

3. Keeping unchanged the initial and last configura¬ 
tions of the trajectory after deformation. 

Perturbing the input function of F, by a vec¬ 
tor-valued input perturbation v G C 1 (/, R m ), yields 
a trajectory deformation G C 1 (/, R") 

u <— u + rv => r <— F + rrj , 

where r is a positive, asymptotically small real number. 
As an approximation of order 1, the relation between 
u and r] is given by the linearized system, which we do 
not express here. 

Potential Field of Obstacles 
Obstacles are taken into account by defining a potential 
field over the configuration space, which increases when 
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the robot gets closer to obstacles. This potential field is 
lifted into the space of trajectories by integration along 
the trajectory of the configuration potential value. 

Discretization of the Input Space 
The space C 1 (/, R m ) of input perturbations is an 
infinite-dimensional vector space. The choice of an in¬ 
put perturbation is restricted to a finite-dimensional 
subspace spanned by p arbitrary test functions, 
e\,... ,e p where p is a positive integer. An input per¬ 
turbation: u = ^i e i i s thus defined by a vector 
Acl''. The variation of the trajectory potential is linear 
with respect to A. 

Boundary Conditions 

The boundary conditions consisting in applying a tra¬ 
jectory perturbation equal to zero at both ends of inter¬ 
val / is linear with respect to A. It is, therefore, easy to 
find a vector A making the potential decrease and satis¬ 
fying the boundary conditions. 

Nonholonomic Constraint Deviation 
The approximation of order 1 in the relation between 
the input perturbation and trajectory deformation in¬ 
duces a side effect: after a few iterations, the non¬ 
holonomic constraints are no longer satisfied. This side 
effect is corrected by considering an augmented sys¬ 
tem with n control vector fields / 1 ,... ,/„ spanning R" 
for each configuration and by keeping the input com¬ 
ponents u m -pi to u n along the additional vector fields 
as close as possible to 0. Figure 47.17 shows an exam¬ 
ple of the trajectory deformation algorithm applied to 
a nonholonomic system. 

47.11.2 Systems of Tactical Planning 

The systems of tactical planning recompute at a high 
frequency a path to the target location, and use the 
main course to advise the obstacle avoidance mod¬ 
ule. The design of these motion systems involves at 
least the synthesis of three functionalities. The con¬ 
struction of a model, the deliberative planning, and the 
obstacle avoidance. The modeler constructs a represen¬ 
tation base for deliberation and memory for the reactive 
behavior. The planner generates global plans used tac¬ 
tically to guide the obstacle avoidance module, which 
generates the local motion. Next we give a perspective 
of the three functionalities and three possible tools to 
implement them [47.73]. 

Model Builder Module 

Construction of a model of the environment (to increase 
the spatial domain of the planning and used as local 



Fig. 47.17 A differentially-driven mobile robot towing a trailer ap¬ 
plying the trajectory deformation algorithm to avoid obstacles 
detected online 

memory for obstacle avoidance). One possibility is to 
use a binary occupancy grid that is updated whenever 
a new sensory measurement is available, and to employ 
a scan matching technique [47.74,75] to improve the 
vehicle odometry before integrating any new measure 
in the grid. 

Planner Module 

Extraction of the connectivity of the free space (used 
to avoid the cyclical motions and trap situations). 

A good choice is a dynamic navigation function such 
as the D* [47.76,77]. The idea behind the planner is to 
focus the search locally in the areas where the changes 
in the scenario structure have occurred and affect the 
computation of the path. The planner avoids the local 
minima and is computationally very efficient for real¬ 
time implementations. 

Obstacle Avoidance Module 
Computation of the collision-free motion. Any of the 
methods described in this chapter could be used. 

One possibility is the ND method (Sect. 47.9.6), 
since it has been demonstrated to be very effi- 



Robot 


Fig. 47.18 Overview of the system of motion 
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Fig. 47.19 (a) Snapshot of an experiment carried out with 
the motion system described working on a wheelchair 
robot equipped with a planar range laser sensor, (b) In¬ 
formation of the motion system at a given time: current 
accumulated map of the scenario, path computed with the 
planner and tactical direction of motion, and solution of the 
reactive obstacle avoidance method ◄ 

cient and robust in environments with little space to 
manoeuvre. 

Globally the system works as follows (Fig. 47.18): 
given a laser scan and the odometry of the vehicle, the 
model builder incorporates this information into the ex¬ 
isting model. Next, the information of the changes in 
obstacle and free space in the model is used by the plan¬ 
ner module to compute the course to follow to reach 
the goal. Finally, the avoidance module uses the in¬ 
formation of the obstacles contained in the grid and 
information of this tactical planner to generate the mo¬ 
tion (to drive the vehicle free of collisions towards the 
goal). The motion is executed by the vehicle controller 
and the process restarts with a new sensorial measure¬ 
ment. It is important to stress that the three modules 
should work synchronously within the perception - ac¬ 
tion cycle for consistency reasons. The advantage of 
these systems is that the synergy of the modules allows 
to avoid trap situations and the cyclic motions (limi¬ 
tations associated to the local nature of the avoidance 
methods). Figure 47.19 shows a snapshot of an experi¬ 
ment carried out with this motion system. 


47.12 Conclusions, Future Directions, and Further Reading 


The algorithmic tools presented in this chapter show 
that motion planning and obstacle avoidance research 
techniques have reached a level of maturity that al¬ 
low their transfer onto real platforms (I^EtMliiliiiliM, 
Today, several indoor mobile robots use 
obstacle avoidance techniques on a daily basis to guide 
visitors in museums. Outdoor applications still require 
some developments in perception and modeling. For 
these applications, 3-D sensing capabilities are nec¬ 
essary to model the environment but also to detect 
obstacles. As an example, several car manufacturer are 
working on parallel parking assistance. The difficult 
point of this application is to build a model of a parking 
spot from 3-D data in a great variety of environments. 

The main challenge of autonomous motion in mo¬ 
bile robots currently consists in integrating techniques 
from different robotics research domains in order to 


plan and execute motions for very complex systems, 
like humanoid robots, for instance. Integration in the 
sense that different software components need to work 
together on a machine but also in a scientific mean¬ 
ing: the classical motion planning formulation with 
configuration variables that locate the robot with re¬ 
spect to a global reference frame does not fit partially 
known environments with imprecise maps. Robotic 
tasks like motion need to be specified with respect 
to landmarks of the environment. For instance, grasp¬ 
ing an object is by definition a motion specified with 
respect to the position of the object. Very little has 
been done to design a general framework in this 
direction. 

For useful complementary reading on motion 
planning and obstacle avoidance for mobile robots 
see [47.5,78-80]. 
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Sensor-based trajectory deformation and docking for nonholonomic mobile robots 
available from http://handbookofrobotics.org/view-chapter/47/videodetails/80 
Autonomous robotic smart wheelchair navigation in an urban environment 
available from http://handbookofrobotics.org/view-chapter/47/videodetails/707 
Sena wheelchair: Autonomous navigation at University of Malaga (2007) 
available from http://handbookofrobotics.org/view-chapter/47/videodetails/708 
Robotic wheelchair: Autonomous navigation with Google Glass 
available from http://handbookofrobotics.org/view-chapter/47/videodetails/709 
A ride in the google self driving car 

available from http://handbookofrobotics.org/view-chapter/47/videodetails/710 
Mobile robot navigation system in outdoor pedestrian environment 
available from http://handbookofrobotics.org/view-chapter/47/videodetails/711 
Mobile robot autonomous navigation in Gracia district, Barcelona 
available from http://handbookofrobotics.org/view-chapter/47/videodetails/712 
Autonomous navigation of a mobile vehicle 

available from http://handbookofrobotics.org/view-chapter/47/videodetails/713 
Autonomous robot cars drive DARPA Urban challenge 

available from http://handbookofrobotics.org/view-chapter/47/videodetails/714 
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48. Modeling and Control 
of Legged Robots 


Pierre-Brice Wieber, Russ Tedrake, Scott Kuindersma 


The promise of legged robots over wheeled robots 
is to provide improved mobility over rough ter¬ 
rain. Unfortunately, this promise comes at the cost 
of a significant increase in complexity. We now 
have a good understanding of howto make legged 
robots walk and run dynamically, but further re¬ 
search is still necessary to make them walk and 
run efficiently in terms of energy, speed, reactiv¬ 
ity, versatility, and robustness. In this chapter, we 
will discuss how legged robots are usually mod¬ 
eled, how their stability analysis is approached, 
how dynamic motions are generated and con¬ 
trolled, and finally summarize the current trends 
in trying to improve their performance. The main 
problem is avoiding to fall. This can prove diffi¬ 
cult since legged robots have to rely entirely on 
available contact forces to do so. The temporal¬ 
ity of leg motions appears to be a key aspect in 
this respect, as current control solutions include 
continuous anticipation of future motion (using 
some form of model predictive control), or focus¬ 
ing more specifically on limit cycles and orbital 
stability. 


48.1 A Brief History of Legged Robots . 1204 

48.2 The Dynamics 

of Legged Locomotion . 1204 

48.2.1 Lagrangian Dynamics . 1205 

48.2.2 Newton and Euler Equations 

of Motion . 1205 

48.2.3 Contact Models . 1207 


The promise of legged robots over standard wheeled 
robots is to provide improved mobility over rough ter¬ 
rain. This promise builds on the decoupling between 
the environment and the main body of the robot that 
the presence of articulated legs allows, with two conse¬ 
quences. First, the motion of the main body of the robot 
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can be made largely independent from the roughness of 
the terrain, within the kinematic limits of the legs: legs 
provide an active suspension system. Indeed, one of the 
most advanced hexapod robots of the 1980s was aptly 
called the Adaptive Suspension Vehicle [48.1]. Second, 
this decoupling allows legs to temporarily leave their 
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contact with the ground: isolated footholds on a dis¬ 
continuous terrain can be overcome, allowing to visit 
places absolutely out of reach otherwise. Note that hav¬ 
ing feet firmly planted on the ground is not mandatory 
here: skating is an equally interesting option, although 
rarely approached so far in robotics. 

Unfortunately, this promise comes at the cost of a hin¬ 
dering increase in complexity. It is only with the un¬ 
veiling of the Honda P2 humanoid robot in 1996 [48.2], 


and later of the Boston Dynamics BigDog quadruped 
robot in 2005 that legged robots finally began to deliver 
real-life capabilities that are just beginning to match 
the long sought animal-like mobility over rough ter¬ 
rain. Not matching yet the capabilities of humans and 
animals, legged robots do contribute however already 
to understanding their locomotion, as evidenced by 
the many fruitful collaborations between robotics and 
biomechanics researchers. 


48.1 A Brief History of Legged Robots 

Before the advent of digital computers, legged ma¬ 
chines could be approached only by electromechanical 
means, lacking in advanced feedback control. This pre¬ 
robotics period culminated with the General Electric 
Walking Truck developed by Ralph Mosher, which in¬ 
spired awe in the mid 1960s. The limb motions of this 
elephant-size quadruped machine were directly reflect¬ 
ing the limb motions of the onboard operator, who was 
responsible for all motion control and synchronization. 
Unfortunately, the strenuous concentration that this re¬ 
quired limited operation to less than 15 minutes. 

Digitally controlled legged robots started to ap¬ 
pear in the late 1960s. Among early pioneers, Robert 
McGhee initiated a series of quadruped and hexapod 
robots first at University of South California, then at 
Ohio State University, culminating in the mid 1980s 
with the Adaptive Suspension Vehicle, a human carry¬ 
ing hexapod vehicle walking on natural and irregular 
outdoor terrain [48. 1], while Ichiro Kato initiated a long 
series of biped and humanoid robots in the Waseda Uni¬ 
versity, a series still continuing nearly half a century 
later [48.3]. But by the end of the 1970s, all legged 
robots were still limited to quasi-static gaits, i. e., slow 
walking motions with the center of mass of the robot 
always kept above its feet. 

The transition to dynamic legged locomotion oc¬ 
curred in the beginning of the 1980s, with the first 
dynamically walking bipedal robot demonstrated at the 
Tokyo University [48.4], and the famous series of hop¬ 
ping and running monopedal, bipedal and quadrupedal 
robots developed at the MIT (Massachusetts Institute of 
Technology) LegLab under the direction of Marc Raib- 
ert [48.5]. Key theoretical breakthroughs came in the 
end of the 1980s, when Tad McGeer demonstrated that 
stable dynamic walking motions could be obtained by 


pure mechanical means, giving rise to a whole new field 
of research, passive dynamic walking, introducing new, 
key concepts such as orbital stability using Poincare 
maps [48.6], with one simple conclusion: you need not 
have complete (or any) control to be able to walk dy¬ 
namically and efficiently. 

Legged robots were still mostly research labora¬ 
tory curiosities working in limited situations when 
Honda unveiled the P2 humanoid robot in 1996 [48.2], 
a decade long secret project demonstrating unprece¬ 
dented versatility and robustness, followed in 2000 by 
the Asimo humanoid robot. The world of humanoid 
and legged robots was ripe for companies to begin in¬ 
vesting. A handful of other Japanese companies such 
as Toyota or Kawada were quick to follow with their 
own humanoid robots, while Sony began selling more 
than 150000 of its Aibo home companion robot dogs. 
Boston Dynamics, a company Marc Raibert founded 
after leaving the MIT LegLab, finally unveiled its Big- 
Dog quadruped robot in 2005 [48.7], which was the first 
to demonstrate true animal-like locomotion capabilities 
on rough terrain. 

The progress over the last decades has been remark¬ 
able. Profound questions have finally been answered: 
we now understand how to make legged robots walk 
and run dynamically. But other profound questions still 
have to be answered, such as how best to make them 
walk and run efficiently. The performance of legged 
robots needs to be improved in many ways: energy, 
speed, reactivity, versatility, robustness, etc. We will 
therefore discuss in this chapter how legged robots 
are usually modeled in Sect. 48.2 and how dynamic 
motions are currently generated and controlled in Sec¬ 
tions 48.4 and 48.5, before discussing in Sect. 48.6 the 
current trends in improving their efficiency. 


48.2 The Dynamics of Legged Locomotion 

One of the major difficulties in making a legged robot robot place its feet, how should it move its body in or- 
walk or run is keeping its balance: where should the der to move safely in a given direction, even in case 
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of strong perturbations? This difficulty comes from the 
fact that contact forces with the environment are an ab¬ 
solute necessity to generate and control locomotion, but 
they are limited by the mechanical laws of unilateral 
contact. 

This essential role of the contact forces is partic¬ 
ularly clear in the derivatives of the total linear and 
angular momenta of the robot, the former involving the 
motion of its center of mass. Because of the importance 
of contact forces for legged locomotion, we briefly dis¬ 
cuss here their different models. 


48.2.1 Lagrangian Dynamics 

Structure of the Configuration Space 
As for every robot moving in their 3-D environment 
(in space or underwater for example), the configuration 
space of legged robots combines the configuration q e 
R w of their N joints with a global position *0 £ R 3 and 
orientation 0o £ R 3 (representing an element of 50(3)) 

f«) 

q = *o I • (A8.1) 

w 

The position xq and orientation 0q are typically those of 
a central body (pelvis or trunk) or of an extremity (foot 
or hand). 


Structure of the Lagrangian Dynamics 
The specific structure of the configuration space out¬ 
lined above is naturally reflected in the Lagrangian 
dynamics 


M(<?) 



+ n(q,q) 



+ £c,-( ? )7t 


(48.2) 


of the system, where M(<jr) e R^+^xts+s) j s the gen¬ 
eralized inertia matrix of the robot, — g e R 3 is the 
constant gravity acceleration vector, n(q,q ) £ R^ 6 is 
the vector of Coriolis and centrifugal effects, u e R w 
is the vector of joint torques, and for all i , /, e R 3 
is a force exerted by the environment on the robot 
and C i(q) £ R (Ar+6)x3 is the associated Jacobian ma¬ 
trix [48.8], 

Since the vector u of joint torques has the same size 
as the vector q of joint positions, the whole dynamics 
including the global position xq and orientation 0 0 ap¬ 
pears to be underactuated if no external forces /, are 
exerted. 


48.2.2 Newton and Euler Equations 
of Motion 

Center of Mass and Angular Momentum 
A consequence of the structure (48.2) of the Lagrangian 
dynamics is that the part of this dynamics which is not 
directly actuated involves the Newton and Euler equa¬ 
tions of motion of the robot taken as a whole ([48.8] 
for detailed derivations). The Newton equation can be 
written in the following way 

m(c + g) = Yjj , (48.3) 


with m the total mass of the robot and c the position 
of its center of mass (COM). The Euler equation can 
be expressed with respect to the COM in the following 
way 

L = V(/7, -c) xfi, (48.4) 


with pi the points of applications of the forces /, and 
L = — c) x m k x k + 1 k<0k (48.5) 

k 

the angular momentum of the whole robot with respect 
to its COM, with Xk and <Ok the translation and rotation 
velocities of the different parts k of the robot, m-k and I* 
their masses and inertia tensor matrices (expressed in 
global coordinates). 

The Newton equation makes it obvious that the 
robot needs external forces/, in order to move its COM 
in a direction other than that of gravity. The Euler equa¬ 
tion is more subtle, as we will see during flight phases. 

Flight Phases 

During flight phases, when a legged robot is not in con¬ 
tact with its environment, not experiencing any contact 
forces/,, the Newton equation (48.3) simplifies to 

c =-g . (48.6) 

In this case, the COM invariably accelerates along the 
gravity vector — g with constant horizontal speed, fol¬ 
lowing a standard falling motion: there is absolutely no 
possibility to control the COM to move in any different 
way. The Euler equation (48.4) simplifies in the same 
way to 

L = 0 , (48.7) 

imposing a conservation of the angular momentum L. 
In this case, however, the robot is still able to gener¬ 
ate and control both joint motions and global rotations, 
this is how cats are able to fall back on their feet when 
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Fig. 48.1 Even though the angular momentum is constant 
during flight phases, the robot is still able to generate and 
control rotations of the whole body with the help of leg or 
arm motions {light brown), as a result of the nonholonomy 
of the angular momentum. This is how cats fall back on 
their feet when dropped from any initial orientation 


Since p z = 0, the x and y coordinates of this equation 
can be simplified in the following way 


c z + g 

x ;.///v 

EJi 


-(?*+?*) + 


1 


m(c z + g z ) 


-SI/ 


(48.10) 


with a simple rotation matrix 



On the right hand side of (48.10) appears the definition 
of the center of pressure (COP) z of the contact forces, 
/,. These contact forces are usually unilateral (the robot 
can push on the ground, not pull) 


dropped from any initial orientation (Fig. 48.1). This 
is a result of the nonholonomy of the angular momen¬ 
tum (48.5) which is not the derivative of any function of 
the configuration of the robot [48.9]. As a result, even 
though the angular momentum L is kept constant dur¬ 
ing the whole flight phase, the joint configuration q, and 
the global orientation 6q of the robot can be driven to 
any desired value at the end of the flight phase. We will 
see in Sect. 48.5 how this impacts the control of legged 
robots. Note that the dynamics of legged robots during 
flight phases is similar to the dynamics of free-floating 
space robots discussed in Chap. 55. Further discussion 
and developments can be found there. 

In Contact with a Flat Ground: 

The Center of Pressure 

In case the forces applied by the environment on the 
robot are due to contacts with a flat ground (while stand¬ 
ing still, walking or running), let us consider a reference 
frame oriented along the ground, with the z axis orthog¬ 
onal to it (therefore tilted if the ground is tilted, see 
Fig. 48.5). Without loss of generality, let us suppose 
that the points of contact, with the ground are all 
such that p] = 0. 

Let us consider then the sum of the Euler (48.4) and 
the cross product of the COM c with the Newton (48.3) 

wcx(c + g)+L = y ^Pi xfi , (48.8) 


and let us divide the result by the z coordinate of the 
Newton equation to obtain 


f?> 0 , 


(48.11) 


which implies that the CoP is bound to lie in the convex 
hull of the contact points (Fig. 48.2) 


g cony {p*‘ y } . 


(48.12) 


Combining this inclusion with the dynamic (48.10) re¬ 
veals an ordinary differential inclusion (ODI) 


c z + g : 


(c x -y+g-'-y) + 


1 


m(c z + g z ) 


-SI/ 


= z x,y e conv 


{PE} • 


(48.13) 



me x (c+g) + L _ Y,iPi x fi 
m(c z + g z ) 


Fig. 48.2 The CoP z is bound to lie in the convex hull of 
(48.9) . . 

contact points p 
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which bounds the motion of the COM c, of the robot 
and the variations of its angular momentum L with re¬ 
spect to the position p x,y of the contact points. 

This ODI can be reorganized in the following way 


O' T g : 


(f-y+gt-y) = (c x -y~z x - y )+ 


1 


m(c z + g z ) 


SI/ 


(48.14) 


in order to expose its simple geometric meaning in 
Fig. 48.3. We can see especially that aside from the ef¬ 
fects of gravity g x ' y and variations L x,y of the angular 
momentum, the horizontal acceleration c x,y of the COM 
is the result of a force pushing the COM c x,y away from 
the CoP z x,y , which is bound to lie in the convex hull of 
the contact points. We have here an intrinsically unsta¬ 
ble dynamics. 

Note finally that the definition (48.12) of the CoP 
can be reorganized to show that the horizontal momenta 
of the contact forces /, with respect to the CoP z are 
equal to zero 


= Y J (p? y -z ( ' y )f? = 0. 

i 

(48.15) 

Hence the CoP is also referred to as the zero moment 
point (ZMP) [48.10,11]. 

In Contact with Multiple Surfaces 
If the contact points p t are not all on the same plane, we 
can introduce a CoP for each contact surface, but we 
cannot introduce a unique CoP for all contact forces as 
we did previously. Approximations, and generalizations 


J2 (p ‘ ~ z ) x /' 



Fig. 48.3 The horizontal acceleration c x,y of the COM of 
the robot is the sum of a force pushing the COM c x,y away 
from the CoP z? ,y the effect of gravity —g i,y and variations 
L, x,y of the angular momentum 


to limited multiple contact situations have been pro¬ 
posed but haven’t been widely adopted [48.11-14]. In 
the general case, the Newton and Euler equations (48.3) 
and (48.4) have to be considered explicitly together 
with the unilaterality condition (48. 1 1) in order to check 
which motion is feasible or not [48.15-17]. The prob¬ 
lem we have to solve then relates very closely to the 
force closure problem discussed in Chap. 38 on grasp¬ 
ing. Different ways to solve it as quickly as possible 
have been proposed [48.15, 18], and used mostly so far 
to measure offline the stability robustness of a given 
motion [48.19,20], and only recently for motion plan¬ 
ning, once again offline [48.21]. Refinements to curved 
contact surfaces have also been proposed [48.22]. 

48.2.3 Contact Models 

The structure of the Lagrangian dynamics makes it 
clear that contact forces are central to the modeling 
and control of legged robots. But note that the only 
characteristics of these forces that we have introduced 
so far is their unilaterality (48.11). As a result, the 
previous analysis applies to various contact situations: 
walking and running motions, but also sliding situations 
such as when skiing or skating, or even rolling situa¬ 
tions [48.23, 24]. 

Let us briefly discuss now standard contact models 
(more details can be found in Chap. 37 on contact mod¬ 
eling and manipulation). Concerning motion and forces 
tangential to the contact surfaces, a simple Coulomb 
friction model is usually considered. Concerning mo¬ 
tion and forces orthogonal to the contact surfaces, two 
options are generally considered: a compliant or a rigid 
model, introducing impacts and other nonsmooth be¬ 
haviors. 

Coulomb Friction 

When a contact point 7 ?, is sliding on its contact surface, 
the corresponding tangential contact force f x,y is pro¬ 
portional to the normal force/? in a direction opposite 
to the sliding motion 

• x,y 

fi’ y = if P? y / 0 > ( 48 - 16 > 

with pa > 0 the friction coefficient. When the contact 
point is sticking and not sliding, the norm of the tan¬ 
gential force is simply bounded, with the same friction 
coefficient 

¥T y \\ < Paff if Pl' y = 0 ■ (48.17) 

This is typically referred to as the friction cone. Note 
that this friction model directly implies the unilaterality 
condition (48.11). 
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Compliant Contact Models 
Compliant contact models take into account the visco¬ 
elastic properties of the materials in contact in the 
direction orthogonal to the contact surfaces 

ft = -Kfp) - Aip] if p\< 0 , (48.18) 

where K , and /l, are stiffness and damping coefficients, 
respectively. In this case, the normal force/f appears to 
be the result of a penetration of the contact point below 
the contact surface, i. e., when p] < 0. Note that, as it 
is here, this model doesn’t satisfy the unilaterality con¬ 
dition (48. 1 1) in all situations and needs therefore to be 
saturated to enforce this property. Of course, when there 
is no contact, there is no contact force 

ft = 0 if P; > 0 . (48.19) 

A similar spring-damper model can also be used to de¬ 
termine the frictional contact force/) 1 ’', subject to the 
bounds of the friction cone (48.17). 

Rigid Contact Models 

Rigid contact models are somewhat simpler to intro¬ 
duce: either there is contact and the normal force can 
take any nonnegative value, 

ft > 0 if p] = 0, (48.20) 

or there is no contact and no contact force 

ft = 0 if p]>Q . (48.21) 

In this case, no penetration of the contact points be¬ 
low the contact surfaces are considered. Note also that 
the unilaterality condition (48.11) is satisfied here by 
definition. As a matter of fact, this rigid model can be 
summarized in the following way 

ft > 0, p) > 0, ftp- = 0 , (48.22) 

where it appears that both the normal force ft and the 
position of the contact point with respect to the contact 
surface p] must be nonnegative, but at least one of them 
must be equal to 0. This is called a complementarity 
condition. Analogous to (48.20) and (48.21), it defines 
the normal force to be either zero, or the force necessary 
to maintain p] = 0. Note that the sticking-or-sliding 
behavior of Coulomb friction can also be represented 
with such a complementarity condition. Note also that 
this set of implicit equations may not have a solution 
or may have multiple ones in certain pathologic situa¬ 
tions [48.25]. 

Rigid and compliant contact models are compared 
in Fig. 48.4 in a static situation (when p] = 0), showing 



Fig. 48.4 Comparison of a rigid (blue) and compliant 
(red) contact model in a static situation. Unlike the rigid 
model, the compliant model considers a penetration of the 
contact point below the contact surface, when pj < 0. The 
graph of the rigid model, with the specific shape of a right 
angle, is called a Signorini graph 

clearly how the rigid model corresponds to an infinitely 
stiff compliant model. Compliant contact models can 
model more accurately the deformation of the contact¬ 
ing bodies, but accurate models often require very stiff 
springs, causing the resulting differential equations to 
be numerically stiff, what can slow down or compli¬ 
cate numerical analysis. Rigid models are compara¬ 
tively more straightforward to integrate and analyze, 
with purely rigid bodies and contact surfaces. They are 
therefore preferred when theoretically or numerically 
studying the stability of legged robots, when optimiz¬ 
ing walking or running trajectories, etc. 

Impacts 

Since rigid contact models do not allow penetration 
of the contact points below the contact surfaces, when 
a point pi reaches a contact surface p\ = 0 with a veloc¬ 
ity p] < 0, this velocity has to change instantaneously 
in order to satisfy the no-penetration assumption, p] > 
0 [48.26]. What happens in this situation is an im¬ 
pact, a discontinuity of the velocity, where we need 
to distinguish the velocity of the robot before impact 
q ~, and its velocity after impact, q+. A straightfor¬ 
ward integration of the Lagrangian dynamics (48.2) 
over a time singleton [48.27] gives us a relation be¬ 
tween pre- and post-impact velocities and impulsive 
forces Fj 

M(q) («/+ -q-'j = ^ C, ,(q?F, . (48.23) 

i 

But in order to completely define the impact law and 
compute the post-impact velocity, we also need a model 
of the impulsive forces. Unfortunately, this is a complex 
problem still open to research [48.28,29], especially 
in situations of multiple contacts which are the stan¬ 
dard situation for legged robots. The usual approach for 
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legged robots is therefore to assume the post-impact 
behavior, usually considering that the contact points 
eventually stick to the contact surfaces 

p+=C i q+=0. (48.24) 

This is, however, a strong assumption, and reality can 
be much more complex, as demonstrated in [48.30]. 
Moreover, a recent study shows that post-impact stick¬ 
ing may not always be desirable [48.31]. 

Impacts are central in most stability analyses of pas¬ 
sive dynamic walking machines (Sect. 48.6), but they 
can be potentially destructive for the mechanical parts 
of a robot. As a result, they are often carefully avoided 
and therefore generally disregarded outside of passive 
dynamic walkers. A recent study shows however that 
this is probably a mistake [48.32]. 

Hybrid and Nonsmooth Dynamics 
With rigid contacts, the dynamics of legged robots 
appears to switch, depending on the contact situa¬ 
tions (48.20) or (48.21). Discontinuities of the state 
of the robot also occur at impacts. A classical way 
to combine these different aspects is with a hybrid 
dynamical system. This approach has however some 
limitations [48.33, 34], the most obvious one being its 
incapacity to handle properly Zeno behaviors , infinite 
accumulations of impacts in finite time [48.35]. An ex¬ 
ample of such a situation is when a legged robot is 


48.3 Stability Analysis - 

The robot model described above represents a complex, 
constrained, nonlinear dynamical system. Our goal for 
this section is to understand the long-term behavior of 
this dynamical system, to answer questions like. Will 
the robot fall down? As we will see in the following 
sections, this long-term behavior can be changed by 
open-loop or closed-loop feedback control, with the ex¬ 
plicit goal of minimizing the likelihood that the robot 
will fall down. 

For controlled nonlinear dynamical systems, there 
are a number of useful concepts relating to their safe¬ 
ness and stability, including: 

• Fixed points: Stable fixed points represent the static 
postures in which the robot can safely stand still. 

• Limit cycles: Limit cycles provide a natural exten¬ 
sion of fixed-point analysis to periodic walking or 
running motions. 

• Viability: Viability is a concept of controlled invari¬ 
ance, which analyzes the set of states from which 


rigidly standing on one foot, and gently rocking from 
one contact edge to the opposite contact edge, with 
a continuously decreasing period (at least according to 
the perfectly rigid model). 

Impacts with multiple contacts and Zeno behaviors 
are not the only difficulties with rigid contact models: 
there is also the Painleve paradox, tangential impacts, 
impacts without collisions, etc. The nonsmooth dynam¬ 
ics approach [48.25, 27,33, 34], where the Lagrangian 
dynamics is turned into a measure differential equation, 
accelerations being abstract measures, velocities being 
functions with locally bounded variations, appears to be 
a much more adequate way to deal with all these intri¬ 
cacies. But it has not been widely adopted for legged 
robots because of a significant increase in mathemati¬ 
cal complexity. 

Many of the complexities of rigid body contact 
dynamics can be avoided by discretizing the system 
dynamics in time and reasoning about the integral of 
contact forces acting over a time step. In particular, 
no distinction is made between impulses and finite 
contact forces over a time step. By conservatively ap¬ 
proximating the friction cone as a polyhedron, forward 
dynamics can be cast as a linear complementarity prob¬ 
lem (LCP) [48.36], for which efficient solvers exist. 
This has become a popular formulation for simula¬ 
tion of rigid bodies in frictional contact and it has 
recently seen applications to control design for legged 
systems [48.37], 


the robot is able to avoid to fall. Unfortunately, this 
property can be intractable to compute. 

• Controllability: Controllability provides a slightly 
restricted notion of viability, analyzing the set of 
states from which the robot is capable of returning 
to a particular fixed point (or limit cycle). This can 
be more tractable to compute than viability, espe¬ 
cially for simple models. 

In addition, if there are unknown errors in the robot 
model, if there is uncertainty about the environment 
(e.g., the location of the ground), or if there are unmod¬ 
eled disturbances, then additional tools are available 
from robustness analysis and stochastic stability: 

• Robust stability: Robust stability (or viability) ex¬ 
amines the properties of the system considering 
worst-case (bounded) disturbances. For instance, 
a robust controller may be able to guarantee that 
a fixed point is stable even if the estimate of the 
mass of the trunk is wrong by ±10%. 


Not Falling Down 
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• Stochastic stability: Stochastic analysis provides 
tools to investigate the probability of falling down. 
For many robot disturbance models, the system will 
always fall eventually (with probability one), but 
analysis can reveal long-living metastable distribu¬ 
tions. 

• Input-output stability: This analysis treats a partic¬ 
ular disturbance as an input, a performance criteria 
as output, and attempts to compute a relative gain 
or sensitivity of the robot performance due to this 
input. 

• Stability margins: Robustness analysis can be diffi¬ 
cult. In practice control designers often settle for the 
system staying comfortably away from the bound¬ 
aries of deterministic stability. 

We give a very brief overview of these tools in the 
remainder of this section, with an attempt to empha¬ 
size features of the analysis that are particular to legged 
robots. 

48.3.1 Fixed Points 

Given a first-order ordinary differential equation gov¬ 
erned by jc = a(x), & fixed point of the system is a state 
jc*, for which a(x*) = 0. The governing equations for 
a legged robot, given in Sect. 48.2, are second-order, 
take a control input, and potentially require a differ¬ 
ential inclusion to describe the contact forces, but the 
notion of a fixed point is still meaningful. Here a fixed 
point is a configuration q* , for which there exists feasi¬ 
ble control inputs u* , and feasible contact forces/*, so 
that q* = q* = 0 is a solution to (48.2). A fixed point 
of this system is a posture in which the legged robot is 
able to stand still. 

In static situations, when c = L = 0, the ODI 
(48.13) gives the following necessary condition 

c x,y - g x,y = Z*’ y S conv {pf y } (48.25) 

g z 

for a static equilibrium on a flat ground. This necessary 
condition states that the COM c must project on the 
ground along the gravity vector —g, inside the convex 
hull of the contact points p t , also known as the sup¬ 
port polygon (Fig. 48.5). This is a very well known and 
widely used necessary condition. Note, however, that 
it is valid only when considering contacts with a flat 
ground [48.15]. 

Once a fixed point has been found, one would of¬ 
ten like to examine its stability. For smooth nonlinear 
differential equations, local stability can often be estab¬ 
lished via linearizing the dynamics at the fixed point 
and then applying well-known tools from linear sys¬ 
tems theory. Indeed, these tools can also be applied 



Fig. 48.5 A necessary static equilibrium condition on 
a flat ground is that the COM c must project on the ground 
along the gravity vector — g inside the convex hull of the 
contact points Pj also known as the support polygon (red) 

here, under the strong assumption that the contact con¬ 
ditions do not change (e.g., either= 0 and p] > 0, or 
ff > 0 and p] = 0). Each active contact adds an equality 
constraint to the linearized system and the eigenvalues 
of this constrained linear system are only meaningful 
in the minimal coordinates; e.g., for a robot standing 
on horizontal ground the assumption precludes the abil¬ 
ity to examine a perturbation of the entire robot in the 
vertical direction. Note that in a hybrid model of the 
legged robot, this assumption is equivalent to lineariz¬ 
ing within a single hybrid mode, and it is common 
practice to describe each hybrid mode in its minimal 
coordinates [48.38]. Once local stability is established, 
it may also be possible to understand the region of at¬ 
traction of the fixed point [48.39,40]. Evaluating the 
stability of these models through changes in contact 
configurations is a new and important area of research. 
For recent work, see for instance [48.41], 

48.3.2 Limit Cycles 

A natural extension of fixed-point analysis to walk¬ 
ing and running motions is to examine the existence 
and stability of periodic orbits, or limit cycles, of 
the dynamical system. Here a periodic orbit is a so¬ 
lution {(q(t), u(t),f(t)) 1 1 e [0, oo)} of the dynamical 
system with a finite period T > 0 such that q(t + T) = 
q(t),u(t+ T) = u(t), and f(t+ T) = /(f). Almost al¬ 
ways, periodic orbits are discovered numerically using 
techniques from motion planning like those described 
in Sect. 48.4. For passive, or very minimally actuated 
robots, the very existence of a periodic solution can be 
exciting [48.42]; robots with more actuators typically 
have an abundance of periodic solutions and planning 
techniques can be used to find solutions that, for in¬ 
stance, minimize the a quantity of interest such as the 
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cost of transport [48.43] or a measure of open-loop sta¬ 
bility [48.44]. 

A periodic solution (<jfo(‘)- H oO'/oQ) with period 
T, is (asymptotically) orbitally stable [48.45] or limit- 
cycle stable if, given initial conditions q(0) we have 


lim 

t—>00 


min lltfM-tfolOll 

0<t'<T 


= 0 . 


(48.26) 


The key feature here that there is no requirement that 
system trajectories converge in time; only the distance 
to the closest point on the orbit must go to zero. This 
stability can be defined for initial conditions restricted 
to a local neighborhood of qo(-), for a region containing 
qo(-), or globally (though no interesting legged robot 
is globally stable) and for inputs determined by a va¬ 
riety of open- or closed-loop control policies, u(t) = 
ji(t, q(t),q(t),f(t)). Under mild conditions, a sufficient 
criteria for establishing orbital stability is establish¬ 
ing fixed-point stability of a Poincare map [48.46], as 
illustrated in Fig. 48.6. Due to the limitation that find¬ 
ing periodic solutions is typically done numerically, 
Poincare map analysis is also typically done numeri¬ 
cally, e.g., using finite differences to evaluate a local 
linearization of the map. This does not necessarily 
preclude rigorous analysis, and techniques exist to com¬ 
pute regions of attraction to hybrid limit cycles despite 
potential numerical inaccuracies [48.47]. Finally we 
note that limit cycle analysis can be applied (carefully) 
to solutions that are not periodic in all states - for in¬ 
stance, the x, y location of the floating base should be 
left out if the robot needs to make forward progress! 

The primary advantage of limit cycle analysis is 
that, using the Poincare map and related methods, many 
of the tools from fixed point analysis and linear systems 
theory can be easily extended to evaluate (local) stabil¬ 
ity and even to design a (locally) stabilizing controller. 
However, this is a very limited definition of locomotion; 
useful locomotion through nontrivial environments will 
likely require aperiodic motions. 





Fig. 48.6 Illustration of a Poincare surface T defined on 
a periodic orbit {red) and a trajectory converging to the 
orbit (black) 


Extensions to Aperiodic Stability 
The notion of stability requires convergence to some 
nominal solution as time goes to infinity, thus the 
nominal solution must be defined over an infinite 
interval. A common approach involves stitching to¬ 
gether periodic solutions with provably bounded finite¬ 
time transition maneuvers which can switch between 
them [48.48]. Orbital stability of an (infinite) aperi¬ 
odic trajectory can be established using a transverse 
linearization or moving Poincare section [48.49]. For 
real-world situations where the entire desired trajectory 
cannot be known apriori, these guarantees can also be 
provided using online motion planning and receding- 
horizon control [48.50]. 

48.3.3 Viability 


If the objective of a legged robot is simply to avoid 
falling down, it can be possible then to directly define 
(at least theoretically) the set of postures J C R w + 6 , 
where the robot has fallen down and that the robot 
should simply avoid. This leads to introducing the set 
of states from which the robot is able to avoid enter¬ 
ing y - the set of states from which the robot is able to 
avoid falling down. These states will be called viable, 
following the viability theory developed in [48.51] for 
general ODIs and introduced in the analysis of walking 
robots in [48.52]. In principle, the set of all viable states 
represents a safe operating envelope for the robot. It is 
typically intractable to compute explicitly but it can be 
approached indirectly in the case of walking robots in 
the following way. 


Center of Mass Dynamics 
During Horizontal Walking 
In case the robot is walking on a horizontal ground, 
the z axis is aligned with gravity, so g* ,y = 0 and the 
ODI (48.13) becomes 


c z + g z 


+ 


1 


m(c z + g z ) 


S L z ‘ y e conv {p.’ y }. 

(48.27) 


We can observe that it is linear with respect to the hor¬ 
izontal motion c x,y , c x,y of the COM and variations of 
the angular momentum LA 3 ’. The vertical motion c z , c z 
of the COM can usually be bounded, but in order to 
simplify the following derivations, let us suppose that 
the COM moves strictly horizontally above the ground, 
i. e., c z is constant and c z = 0, so the ODI becomes 

r z CT x,y 

c x,y — -c x,y -\ -G conv {p*’ y } . (48.28) 

g z mg z 

Variations of the angular momentum L can also be 
bounded in the x and y directions, but to make things 
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Fig. 48.7 A situation where, for some reason, no steps can 
be undertaken beyond a certain line (red), with a a vector 
orthogonal to this line and pointing beyond it 

as simple as possible, let us consider these variations to 
be equal to 0, so the ODI takes a very simple second 
order linear form 

- —c x,y = z?' y e conv \p- v \ . (48.29) 

8 Z 

An Example of Inevitable Fall 
Let us consider furthermore a situation where, for some 
reason (e.g., a cliff), no contact with the ground can 
be realized beyond a certain line (Fig. 48.7), with a 
a vector orthogonal to this line and pointing beyond 
it. If the COM of the robot reaches this line at a time 
to with a speed pointing beyond it (a J c x ’ y (to) > 0), this 
linear ODI can be integrated analytically very simply 
and lead to the following inequality [48.53], valid at all 
time t > to 

T a Y c x,y (to) 

a r [c x ' y (t) - c A ’ v (to)] > -sinh [co(t - 1 0 )], 

co 

(48.30) 

with 



The right hand side is increasing exponentially with 
time. As a result, the position c x,y of the COM is di¬ 
verging exponentially in the direction of the vector a, 
leading inexorably to a fall. 

A Sufficient Condition for Viability 
In the simplified linear case developed above, the in¬ 
equality (48.30) establishes that in the case of a fall, the 
motion of the COM diverges exponentially, so the in¬ 
tegral of the norm of any n th derivative of its position, 

OO 

/| e « W |dr, (48.32) 

io 


would be infinite. Therefore, if we can find a finite value 
for such an integral from a given initial state of the 
robot, we can conclude that this state is viable. 

48.3.4 Controllability 

For any dynamical system, an initial condition xq is 
controllable to a final state Xf, if there exists a feasible 
input trajectory /<(•), which takes the system from jco to 
Xf. An initial state xo is denoted finite-time controllable 
to Xf if the minimum time required is finite. Control¬ 
lability analysis is a well understood topic for linear 
dynamical systems, where a simple rank condition on 
the controllability matrix provides necessary and suffi¬ 
cient conditions to establish whether or not any initial 
condition can be driven to any final state [48.54], For 
nonlinear systems, the property is dependent on both 
the initial and final states. 

In the case of legged robots, instead of consider¬ 
ing the set of all viable states introduced earlier, we 
could consider instead the more limited set of states 
that are controllable to stable fixed-points - states that 
can come to a stable stop after a given number of steps. 
This is a sufficient condition for viability. Such states 
have been called capturable [48.55], and arguably en¬ 
compass most of the states of interest for legged robots. 
A variant would be to consider states from which the 
robot is able to reach a stable limit cycle, or any other 
state which is known to be viable in the first hand. 

The Capture Point 

In the simple linear case developed above, the cap¬ 
turable states can be identified analytically with the help 
of the compound variable 

1 

§=c+-c, (48.33) 

00 

introduced independently as the extrapolated center of 
mass (XCOM) [48.56], the capture point [48.57] or the 
divergent component of the dynamics [48.58]. These 
three denominations correspond to three key properties 
of this variable. First of all, a trivial reformulation gives 

c = tu(| —c) , (48.34) 

revealing that this point £ is the point where the COM is 
converging to, hence the extrapolated COM. Following 
the linear dynamics (48.29), the horizontal motion of 
this point satisfies 

k x ‘ y = ao{$ x - y -7?' y ) , (48.35) 

where we can see that this point | diverges away from 
the CoP z. But if this point is above the support polygon. 
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Fig. 48.8 The speed of the COM c points towards the cap¬ 
ture point £, while the speed of this capture point points 
away from the CoP, z . The combination of these two first- 
order dynamics gives the acceleration c, pushing the COM 
away from the CoP 

if we have 

% x,y e conv {p*’ y } , (48.36) 

we can have z x,y = % x ‘ y so the point | can stay mo¬ 
tionless, and the COM will converge to it and come 
to a stop, hence it can be seen as a capture point 
(CP). Finally, we can observe that the second-order 
linear dynamics (48.29) has been decomposed here as 
two first-order linear dynamics (48.34) and (48.35), the 
first one being stable, with the COM converging to the 
XCOM, the second one being unstable, with the XCOM 
diverging away from the CoP, hence the divergent part 
of the dynamics (48.29). Here appears an interesting 
structure relating the motion of the points z, c and §, 
shown in Fig. 48.8 [48.59, 60]. 

An important observation here is that if the state 
of the robot satisfies the condition (48.36), then the 
robot can stop without making any step and it is cap- 
turable and viable. Further analysis, considering any 
given number of steps and a nonzero angular momen¬ 
tum L can be found in [48.60]. 

48.3.5 Robust or Stochastic Stability 

For the deterministic robot dynamics models described 
in Sect. 48.2, the concepts of fixed-points, limit cy¬ 
cles, viability, and controllability form the core issues 
of analysis. However, in the presence of disturbances, 
such as model inaccuracies or unpredicted variation in 
terrain, tools from robust verification must be called 
upon to characterize the stability of a system. Broadly 
speaking, robust stability analysis exists in two forms. 
In worst-case analysis, an upper bound is given for 
a disturbance and the goal is to demonstrate that for any 


disturbance within this bound, the system will not reach 
an unstable state. Alternatively, stochastic analysis can 
be performed where a disturbance probability distri¬ 
bution is specified with the goal being to demonstrate 
that the system avoids unstable states with high prob¬ 
ability. Worst-case analysis has received surprisingly 
little attention to date in the legged robotics commu¬ 
nity, however analysis approaches based on Lyapunov 
functions [48.38,41,47] can be easily extended using 
the notion of a common Lyapunov function [48.61]. One 
challenge here is that the nominal solutions, e.g., the ex¬ 
act shape of the nominal limit cycle, are often parameter 
dependent: the system might still be stable with differ¬ 
ent parameters, but to a slightly different limit cycle. 

Stochastic stability has received relatively more 
attention in the legged robotics community, perhaps be¬ 
cause many interesting models of possible disturbances 
do result in robots that walk for a long time, but do 
eventually fall over. Such systems cannot be classified 
as stable, but it seems incomplete to simply call them 
unstable. Such systems are said to be metastable, a con¬ 
cept first established in the physics community [48.62] 
and later used to analyze walking systems [48.63]. 
From this point of view, the stability of a system can 
be characterized by its mean time to failure. 

Recall the idea of Poincare maps described above, 
but now consider the discrete time stochastic return map 
dynamics for a periodic walking system, 

*k+i = r(x k ,h k ) , (48.37) 

where h * ~ P(h) is a random variable that, e.g., captures 
the terrain height at time step k. Discretizing the state 
space X C R 0v+ 6 > x into d states allows us to 

define the Markov process based on the above return 
map 

p k +i=Tpk, (48.38) 

where p^ e is the state probably distribution vec¬ 
tor and T is a stochastic matrix with entries Ty = 
Pr(jcJ._|_ | IxJj). If we assume that all failure states are ab¬ 
sorbing and can be grouped together into a single state, 
this corresponds to a column in T containing a sin¬ 
gle 1 and d — 1 zeros. The largest eigenvalue of T must 
then be Ai = 1 with a corresponding eigenvector it 1 , 
describing the stationary distribution of the absorbing 
state. Interestingly, the second largest eigenvalue has 
been found to be close to unity At ss 1 and much larger 
than the remaining d —2 eigenvalues, for several simple 
walking systems [48.63]. This suggests the existence 
of a metastable neighborhood in state space captured 
by the distribution V 2 with an associated time constant 
r = —1/ log(A 2 ). Other approaches to computing these 
stochastic stability bounds are also possible [48.64]. 
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48.3.6 Input-Output Stability 

Input-output methods define sensitivity measures to 
summarize the system’s response to disturbances. For 
example, for a walking system, one could define a vec¬ 
tor l G R'" composed of m gait variables that directly 
relate to failure modes (such as foot-ground clearance) 
and measure the effect that a set of input disturbances 
w , have on their values. The gait sensitivity norm does 
exactly this [48.65]; here the dynamic system response 
|| 9I/3 k; || 2 is used as a measure of disturbance rejection. 
How is this response actually computed? In practice, 
computing the system response can be done experi¬ 
mentally by generating a variety of perturbations and 
measuring the effect in the gait variables. Although la¬ 
borious, this approach is probably the most appropriate 
for physical systems. Alternatively, for simulated limit 
cycle walkers, linearization of the Poincare return map 
and perturbation analysis yields a complete characteri¬ 
zation of the dynamic system response. 

More general tools from nonlinear control theory 
can also be applied. The Li gain is a sensitivity mea¬ 
sure that has seen widespread application in nonlinear 
control theory. For complex dynamical systems such as 
walking robots, an upper bound on the Lt gain can be 
approximated using storage functions, which can often 
be computed using convex optimization [48.66, 67]. 

48.3.7 Stability Margins 

In practice, computing performance measures like the 
ones mentioned above for complex walking systems 
can be very difficult. As a result, researchers often em¬ 
ploy heuristic stability margins to keep the robot away 
from the boundary of stability. Here the definition of 
stability can vary widely. The simplest stability margins 
rely only on the static configuration of the robot, where 
if the COM ground projection leaves the support poly¬ 
gon, this corresponds to an uncompensated moment on 
the foot resulting in a rotation along its edge. Popular 
static metrics include the distance between the ground 
projection of the COM and the edges of the support 
polygon or the minimum potential energy required to 


tip the robot over sideways [48.68]. Because these sta¬ 
bility criteria are based only on statics, they are only 
practically useful for slow (or quasi-static) motions. 

The most commonly used dynamic stability mar¬ 
gin requires that the CoP (or ZMP) remains within the 
boundary of the support polygon. When this condition 
is satisfied, the foot cannot rotate around the boundary 
of the support polygon. Note that this is certainly very 
conservative - many bipedal robots walk dynamically 
even with point feet (resulting in a CoP that is always 
on the boundary of the support polygon) - but this met¬ 
ric has proven extremely useful for flat-foot dynamic 
walking. Goswami introduced the related concept of the 
foot rotation indicator (FRI) [48.69] that is defined as 
the point on the ground where the net ground reaction 
force would have to act to keep the foot from rotat¬ 
ing; this point need not stay inside the support polygon 
and can therefore be applied for a wider range of gaits. 
However, the stability claims based on these criteria 
are valid only on flat terrain and they do not constitute 
necessary or sufficient conditions for stable walking in 
general [48.15,55]. 

When the COM projection exits the support poly¬ 
gon, there are two ways to make it return: by accelerat¬ 
ing the COM in the direction of the support polygon 
through changes in angular momentum or by chang¬ 
ing the support polygon by taking a step. This basic 
insight led to stability margins based on capture re¬ 
gions [48.55,60]. A capture region is simply the set 
of all capture points (defined in Sect. 48.3.4). The cap¬ 
ture margin is a metric that determines the capturability 
of a walking system. If the capture region and support 
polygon overlap, it is defined as the maximum distance 
between the points in the capture region and the clos¬ 
est edge of the support polygon, otherwise it is the 
negative distance between the capture region and sup¬ 
port polygon. Note that large values under this metric 
are related to the total capture region area, which is in 
turn related to the ability of the walking system to sta¬ 
bilize itself, e.g., with upper body motions that affect 
angular momentum [48.60]. Extensions to n-step cap¬ 
ture margins have also been developed and described in 
detail [48.55]. 


48.4 Generation of Dynamic Walking and Running Motions 


As mentioned in the brief historical introduction to this 
chapter, early legged robots were maintaining quasi¬ 
static equilibrium postures at all time. The transition 
to more dynamic motions took place at the beginning 
of the 1980s, associated with biped [48.4] and mono¬ 


pod [48.5] robots. As a result, this Section on dynamic 
legged locomotion will focus on results obtained mostly 
with biped robots, although most of the methods dis¬ 
cussed here could be applied as well to other legged 
robots. 
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48.4.1 Early Offline Motion Generation 
Schemes 

The very limited computing power of early robots 
allowed very limited online decisions. As a result, tra¬ 
jectories were usually computed offline, especially in 
complex situations such as for legged robots. Online 
motion generation has finally become possible only by 
the year 2000, thanks to the exponential increase in 
computing power and the continuous progress of nu¬ 
merical methods during the last decades. 

Trajectory Optimization 

One of the earliest approaches to computing walking 
and running motions was through numerical optimiza¬ 
tion [48.70]. The idea is to leverage the capacity of 
trajectory optimization methods to take into account 
nonlinear dynamics and objectives such as minimizing 
energy consumption, and compute the corresponding 
optimal motion. Unfortunately, the dynamics of legged 
robots are so complex that this approach was still 
stuck with simple planar models by the end of the 
1990s [48.71-75]. ft is only by the year 2000 that the 
first optimal walking motions for complete 3-D models 
could be computed [48.76, 77]. 

Fortunately, numerical methods have continued to 
improve (as discussed in Sect. 48.6.1), allowing re¬ 
searchers to solve complex problems such as optimizing 
trajectories and the mass distribution of the robot si¬ 
multaneously and maximizing the open-loop stability 
of trajectories [48.78]. 

Unfortunately, this approach still requires time con¬ 
suming computations that often cannot be realized 
online. Not being able to compute walking or running 
motions online limits the robots to a predefined set of 
precomputed actions, potentially ruining their versatil¬ 
ity and reactivity. One way to alleviate this serious lim¬ 
itation is to generate a database of trajectories [48.79] 
that can be queried online, possibly conditioned on or¬ 
ders given to the robot [48.80] or the current state of the 
robot in order to improve its stability [48.40, 81, 82]. 

Artificial Synergy Synthesis 

and the ZMP Approach 

Partitioning the problem is another early approach, that 
aimed at palliating the complexity of the dynamics 
of legged robots that hindered trajectory optimization. 
The idea is to assign some degrees of freedom of the 
robot to take care of dynamic constraints such as the 
OD1 (48.13), allowing the rest of the robot to be oper¬ 
ated more or less independently. This general approach 
has been called artificial synergy synthesis [48.10]. 

The original proposition in [48.10] was to use trunk 
rotations to ensure dynamic feasibility while the legs of 


the robot executed a given prerecorded motion. More 
precisely, leg motions and contact points p, being pre¬ 
defined, the trajectory of the CoP z could be predefined 
accordingly, so that it was just a matter of solving the 
ordinary differential equation (ODE) (48.13) for this 
predefined z to obtain the required rotations of the trunk. 
This original proposition was demonstrated experimen¬ 
tally 20 years later on the Waseda University WL-12RV 
biped walking robot [48.83]. This method of predefin¬ 
ing the trajectory of the CoP has eventually been called 
the ZMP approach to walking motion generation (re¬ 
member the ZMP is just another name given to the 
CoP). This ZMP approach has been associated later 
with having feet always flat on the ground, exclud¬ 
ing heel and toe rotation phases. Note, however, that 
such phases were considered in the original proposi¬ 
tion [48.10]. 

Some aspects of this proposition can be questioned, 
ft has been argued first of all that predefining the 
evolution of the CoP is not necessary nor even desir¬ 
able [48.84,85]. Then, the OD1 (48.13) clearly shows 
that dynamic feasibility depends on both variations of 
the angular momentum L, and motion of the COM c 
with respect to contact points,/?,. While trunk rotations 
mostly involve variations of the angular momentum, 
a recent analysis showed that this has only a weak influ¬ 
ence on the balance of legged robots [48.86]. Dynamic 
feasibility can be handled much more efficiently by 
adapting the motion of the COM, which has been the 
prevailing approach. 

The core idea of partitioning the motion of the robot 
builds on a profound and far-reaching observation: if 
the robot has a sufficient number of degrees of freedom 
and sufficient control authority, then every part of the 
motion that is not involved in the dynamic constraints - 
everything but angular momentum and motion of the 
COM with respect to contact points - can be operated 
more or less independently, and appears therefore to be 
relatively peripheral to the problem of legged locomo¬ 
tion. It is the same key observation that implicitly drives 
the templates and anchors approach and the long history 
of simple biomechanical models of legged locomotion 
that focus on a few meaningful degrees of freedom, 
mostly the motion of the COM with respect to contact 
points, and abstract all the rest [48.87-89]. This idea has 
been tremendously successful in the legged locomotion 
research community. 

48.4.2 Online Motion Generation: 

A Model Predictive Control 
Point of View 

If the algorithms for motion generation could work 
sufficiently fast to be applied online, then the robot 
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could achieve reactivity and robustness by continuously 
adapting the motions to the current state of the robot and 
its environment. But therein lies a problem: how can we 
make sure that the continuous re-evaluation of online 
decisions maintains long term viability? One solution 
comes from the model predictive control (MPC) theory. 

The online motion generation schemes that allow 
most of the great humanoid robots of today and yes¬ 
terday to walk and run can all be related to MPC 
theory and can all be seen as variants of the same MPC 
scheme, although they have rarely been introduced in 
that way. Most adopt the artificial synergy synthesis 
approach described in the previous Section, focusing 
almost exclusively on the motion of the COM with re¬ 
spect to contact points and assuming that the rest of the 
robot can be operated more or less independently. 

Viability, Optimal Control and MPC 
When trying to identify viable states in Sect. 48.3.3, the 
first option was to check if the integral (48.32) could 
have a finite value. A classical result from optimal con¬ 
trol theory is that an optimal control law minimizing 
this integral would make it decrease with time. As a re¬ 
sult, if this integral was finite in the beginning, it would 
stay finite: if the state of the robot was viable in the 
beginning, it would stay viable. The problem is, the 
dynamics of legged robots is too complex to be able 
to compute such an optimal control law in the general 
case. 

MPC is one way to obtain a computable approx¬ 
imation to this optimal control law. The fundamental 
idea is to introduce a terminal constraint [48.90], im¬ 
posing that the integral (48.32) is equal to zero after 
a finite length of time. Introducing such an artificial 
constraint naturally yields a suboptimal control law, but 
this allows considering the integral (48.32) over only 
a finite length of time while preserving viability. Note 
that enforcing viability in finite time means in this case 
enforcing capturability: here, the terminal constraint is 
a capturability constraint. 

Many MPC variants have been proposed [48.91], 
but two extremes are going to be significant in the fol¬ 
lowing. One extreme is to observe that minimizing an 
integral of the form (48.32) is not necessary to obtain 
a viable behavior: the terminal, capturability constraint 
can be sufficient in this respect [48.92]. In this case, 
however, it is not possible to continuously re-evaluate 
and adapt the motion to the state of the system: this 
must be done at carefully chosen moments, what lim¬ 
ited the reactivity of the robots. The other extreme is 
to observe that the terminal constraint is not absolutely 
necessary [48.93]: simply minimizing a truncated ver¬ 
sion of the integral (48.32) over a sufficiently long but 
finite length of time can still lead to an integral de¬ 


creasing with time, ensuring viability. In this case, it is 
possible to continuously re-evaluate and adapt the mo¬ 
tion to the state of the system without any problem. 

Let us see now how all this theory is applied to the 
online generation of walking and running motions. In 
all the following examples, the robot is supposed to 
walk on a flat horizontal ground. 


Predefined Footsteps 
and Capturability Constraint 
Let us have a look first at motion generation schemes 
implementing the original ZMP approach, with prede¬ 
fined footsteps and a predefined CoP, and considering 
a capturability constraint to ensure viability. Let us 
begin with the walking motion generation scheme im¬ 
plemented in the long series of Waseda University 
humanoid robots [48.94], which considers a four point 
mass model with predefined motion except for the hor¬ 
izontal motion of the waist and trunk masses which is 
assigned to follow a reference trajectory for the CoP. 
An iterative procedure based on fast fourier transforms 
is used then to solve the dynamics 
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In order to execute this scheme online, a capturabil¬ 
ity constraint is introduced, imposing that the robot is 
always able to stop within two steps [48.95]. Unfortu¬ 
nately, details about the choice of the reference CoP and 
the exact terminal constraint are not disclosed. 

The walking motion generation scheme imple¬ 
mented in the Munich University Johnny robot [48.96] 
considers only a three point mass model with predefined 
motion, except for the horizontal motion of the main 
mass in the trunk which is assigned to follow a piece- 
wise linear reference trajectory for the CoP. One degree 
of freedom is left in this reference trajectory in order 
to impose a terminal constraint on the position of the 
COM at the end of the next two steps 
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This constraint appears to be incomplete in imposing 
capturability, what would require to consider also the 
velocity of the COM (Sect. 48.3.3). 

The walking motion generation scheme imple¬ 
mented in the Honda Asimo robot [48.58] is very 
similar, three point masses and a piecewise linear ref¬ 
erence for the CoP with one degree of freedom left to 
satisfy the terminal constraint. The difference lies in the 
terminal constraint which is a true capturability con¬ 
straint, imposing cyclicity of the motion through the 
capture point/XCOM/divergent component of the dy- 
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namics at the end of the next step 


= £ ref • (48.41) 

The walking motion generation scheme imple¬ 
mented in the Tokyo University H7 robot [48.97] con¬ 
siders the whole dynamics (48.13) of the robot. The 
whole motion of the robot is predefined, except the hor¬ 
izontal motion of the COM which is assigned to follow 
a reference trajectory for the CoP, set in the middle of 
the contact points. An iterative procedure is used then 
to solve the dynamics 
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The terminal constraint (48.40) on the position of the 
COM (incomplete with respect to capturability) is also 
considered at the end of the next two steps. 

The walking and running motion generation scheme 
implemented in the Toyota Partner robot [48.98] is ex¬ 
actly the same, except for the terminal constraint which 
is a true capturability constraint imposing cyclicity of 
the motion through both the position and velocity of the 
COM. 

The walking and running motion generation scheme 
implemented in the Sony QRIO robot [48.99] follows 
a similar design, but considers only a single point mass 
at a constant height, with its horizontal motion assigned 
to minimizing the deviation of the CoP from the middle 
of the contact points 


min 
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while imposing a capturability constraint on both the 
position and velocity of the COM. 

Another variant tested on the Kawada HRP-2 
robot [48.100] also considers a single point mass at 
a constant height, but the CoP follows a piecewise poly¬ 
nomial trajectory 
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with some degrees of freedom left to satisfy the same 
capturability constraint as before, through both the 
position and velocity of the COM. An important charac¬ 
teristic of this scheme is that the piecewise polynomial 
trajectory of the CoP may fluctuate strongly, threatening 
to violate the ODI (48.13). An automatic adjustment of 
the step timings is proposed therefore in order to mini¬ 
mize this risk. 

All of these walking and running motion generation 
schemes try to impose capturability through terminal 


constraints, but some of them appear to fail properly 
doing so by only constraining the position of the COM. 
None of them consider an integral of the form (48.32): 
the integral (48.43) does not match. We have seen ear¬ 
lier that in this case, it is possible to re-evaluate and 
adapt the motion at specific instants, what has been 
done when having to realize new steps, or when chang¬ 
ing the walking speed or direction. But to adapt the 
motion to a perturbation, a specific observer would 
be required to trigger the adaptation at the correct in¬ 
stant [48.92], and it appears that this option hasn’t been 
investigated: no state feedback has been experimented 
with these motion generation schemes. 


Predefined Footsteps, 

Without Capturability Constraint 
In the standard walking motion generation scheme 
implemented in the Kawada HRP-2 humanoid ro¬ 
bot [48.101], the whole motion of the robot is prede¬ 
fined as before, except the horizontal motion of the 
COM which is assigned this time to minimize the 
weighted integral 
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of the norm of the third derivative of the motion of the 
COM and the deviation of the CoP from a reference in 
the middle of the contact points. This is clearly an inte¬ 
gral of the form (48.32), and we have seen that viability 
can be ensured in this case if we consider a truncated 
version of this integral over a sufficiently long but fi¬ 
nite length of time, typically the next two steps, without 
the need to impose any terminal constraint. We have 
also seen that it is possible in this case to continuously 
re-evaluate and adapt the motion to the state of the sys¬ 
tem, a clear improvement over the previous approaches 
based solely on terminal constraints. This has been val¬ 
idated experimentally in various situations, effectively 
adapting the walking motion to perturbations [48.102]. 

An interesting variant [48.103] introduces varia¬ 
tions of the angular momentum L as an additional 
variable to minimize the combined integral 
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showing an improvement in the tracking of the gener¬ 
ated motion (and the same closed loop robustness to 
small perturbations). 
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A problem however with these approaches is that 
simply minimizing the deviation of the CoP from the 
middle of the contact points doesn’t preclude it from 
fluctuating, so the ODI (48.13) could be violated, es¬ 
pecially in case of perturbations. This is monitored 
in [48.102] to trigger a change of footstep if neces¬ 
sary, but without any clear guarantee that this change 
is appropriate. For this reason, it has been proposed 
in [48.85] to impose the ODI (48.13) as a strict con¬ 
straint, considering a single point mass model with 
a constant height 


constraint), so the robot might very well be falling after¬ 
wards. However, in the presented simulations, it appears 
that maintaining this ability to make two more steps in 
the future is a sufficient condition for generating stable 
walking and running motions. The conclusion is that 
terminal constraints or integrals of the form (48.32) are 
in fact not mandatory and could be relaxed altogether. 

In all of these approaches, the footsteps were pre¬ 
defined and kept fixed, which is obviously a strong 
limitation on the robot’s capacity to adapt to a chang¬ 
ing environment or to strong perturbations. 
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and simply minimize the integral 

min J ||c* J '|| 2 dr 


(48.47) 


(48.48) 


over a sufficiently long but finite length of time to 
ensure viability. This is the walking motion gen¬ 
eration scheme implemented in the Aldebaran Nao 
robot [48.104]. 

A variant tested successfully on the DLR (Deut¬ 
sches Zentrum fur Luft- und Raumfahrt) biped ro¬ 
bot [48.105] considers minimizing the deviation from 
a reference trajectory for the XCOM together with the 
derivative of the CoP 


min 
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An additional terminal constraint was considered but 
not tested. 

A last proposition, tested only in simple simula¬ 
tions, not with a real robot [48.106], considers a single 
point mass which is assigned to minimize the deviation 
of the leg length / from a given reference, and the devi¬ 
ation of the CoP from the middle of the contact points 
over the next two steps 
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What makes this proposition particularly stimulating 
is that this is not an integral of the form (48.32), that 
can ensure viability, and no terminal constraint is there 
to impose capturability, so there is no direct relation 
with the viability/capturability analysis developed in 
Sect. 48.3.3, on which all the previous schemes were 
based. What this minimization only imposes is the abil¬ 
ity to realize two more steps in the future, with legs 
approximately at a nominal length: the situation in the 
end of these two steps is not controlled (no terminal 


Adaptive Footsteps 

In fact, adapting foot placement is straightforward, 
and has already been validated on a Kawada HRP-2 
robot [48.107]: the only required change is to consider 
foot placement as a decision variable, used in addition 
to the horizontal motion of the COM, in order to both 
satisfy the ODI (48.47) and minimize the integral 



over a sufficiently long length of time to ensure viabil¬ 
ity (since this is clearly an integral of the form (48.32)), 
and have the COM follow on top of that the refer¬ 
ence velocity . Now, since the footstep placement 
is decided online, geometric feasibility needs to be 
checked online as well. A simple but effective option 
is to consider a polygonal approximation of the reach¬ 
able volume of the COM with respect to each foot on 
the ground [48.108]. 

Yet, the timing of the steps was still predefined, 
what has a huge influence on the reactivity of legged 
robots [48.60]. In order to adapt it as well, it has been 
proposed in [48. 109] to minimize the combined integral 

min J \\c*-y-C x J\\ 2 + \r’y\\ 2 dt, (48.52) 

with f x,y the horizontal acceleration of the feet. But this 
approach has been tested only in simple simulations so 
far. Moreover, the underlying optimization problem be¬ 
comes nonlinear, what is significantly more involved to 
solve. 

Another approach, quite unique, starts with a sin¬ 
gular linear quadratic regulator (LQR) design [48.110], 
considering the classical single point mass at a constant 
height, assigned to minimize the deviation of the CoP 
from a combination of a reference CoP and the XCOM 
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for some a > 0. An interesting feature of this singular 
LQR design is that it can be solved analytically. And an 
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interesting property of this singular objective function 
is that it can be reduced to zero, in which case we have 

= ( 1 + ot)£ x,y -az x J , (48.54) 

8 Z 

what can be combined with the dynamics (48.35) of the 
XCOM to obtain a stable first order dynamics 

(48.55) 

according to which the XCOM £ is going to converge 
to z%, and the CoP z as well according to (48.54). 
But in a very unusual twist, this LQR design is not 
used as is, and is inverted analytically to find un¬ 
der which condition does a piecewise constant trajec¬ 
tory of the CoP generate a nondiverging motion of 
the COM. Unsurprisingly, this nondiverging condition 
ends up being a terminal constraint on the capture 
point/XCOM/diverging part of the dynamics, with tra¬ 
jectories corresponding in the end exactly to those 
found in [48.59]. But here, this terminal constraint is 
used in the end to decide online the footstep placement 
that will ensure viability, what is eventually validated 
experimentally with very strong perturbations. 

All Variants of a Common Design 
The viability analysis of Sect. 48.3.3 attested the crucial 
importance of anticipation to avoid falling. We observe 
now that the walking and running motion generation 
schemes that power most of the greatest humanoid 
robots of today and yesterday, the Honda Asimo, the 
Toyota Partner, the Sony Qrio, the Aldebaran Nao, the 
Kawada HRP-2, the Tokyo University H7, the Munich 
University Johnny, the DLR biped, the long series of 
Waseda University humanoids and many others, are all 
structured around this necessity to anticipate, at least 
a couple steps ahead of time. Viability is secured then in 
different ways, always related to MPC theory, through 
capturability or through minimizing an integral of the 
form (48.32). In all cases, adopting the artificial synergy 
synthesis approach, focusing on the motion of the COM 
of the robot with respect to contact points, considering 
that the rest of the motion can be handled more or less 
independently, is the key to enable efficient computa¬ 
tions. All these motion generation schemes appear then 
to share the same general design: MPC of the COM of 
the robot with respect to contact points. The rest is de¬ 
tails. 

The compelling consequence of identifying this 
common design, is revealing the possibility to cross¬ 
breed all these approaches, retaining the best features 
of each: more precise multiple mass models, for gener¬ 
ating both walking and running motions, with adaptive 
footstep placement and adaptive timing, sensor feed¬ 


back, all in order to obtain in the end the ultimate robust 
and versatile online motion generation scheme. These 
are the next obvious steps. 

Missing in this gallery of great legged robots, the 
Boston Dynamics biped and quadruped robots. They 
are without a doubt the legged robots exhibiting the 
most impressive dynamic motions today, robust and 
versatile. Exact details about their control algorithms 
are scarce, but various clues suggest that they might 
share the same design. 

48.4.3 Motion in Constrained Environments 

We have seen so far how to generate dynamic walk¬ 
ing and running motions by considering the dynamic 
feasibility constraint (48.13) independently from any 
other concern. But on truly rough terrain or in clut¬ 
tered spaces, kinematic constraints on the motion of 
the robot cannot be disregarded and can further compli¬ 
cate the problem. More precisely, kinematic constraints 
generally make the reachable space of the robot non- 
convex, so that deciding actions and motions with only 
a local view on the situation can quickly get the robot 
stuck in local loops or dead-ends. One way to decide 
actions and motions with a global view on the situation 
is with planning techniques, as described in Chap. 7 
on motion planning and Chap. 47 on motion planning 
and obstacle avoidance. In the case of legged robots, 
three classes of problems of increasing complexity have 
been considered: dynamic manipulation without loco¬ 
motion, locomotion on a flat ground with obstacles, and 
locomotion on rough terrain with complex contact tran¬ 
sitions. 

Dynamic Manipulation Without Locomotion 
In the case of manipulation tasks without locomo¬ 
tion, the only difference of legged robots with respect 
to more traditional manipulator robots is the dynamic 
feasibility constraint (48.13). Therefore, a mere appli¬ 
cation of the standard planning techniques described 
in Chaps. 7 and 47 including this feasibility constraint 
has proved to be fully sufficient. But these techniques 
are strongly biased towards quasi-static motions. As 
a result, a sequence of statically stable postures is com¬ 
puted first, and the motion is smoothed and accelerated 
afterwards, finally taking into account dynamic feasi¬ 
bility [48.111]. Much more involved are the problems 
involving locomotion, which attracted therefore much 
more attention. 

Legged Locomotion on a Flat Ground 
with Obstacles 

If we consider that the problem of generating dynamic 
legged locomotion is properly solved thanks to the dif- 
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ferent methods described earlier in this section, we can 
try to abstract its details and plan the navigation of 
the upper body of a legged robot between obstacles 
as for any other mobile robot, with the same planning 
techniques, regardless of the exact locomotion method. 
It has been shown indeed that legged robots present 
a form of small-space controllability, which means that 
their upper-body can follow any given path with any 
given accuracy [48.112], although this can require un¬ 
practically quick steps. A different option to follow 
exactly any given path with the upper-body is to cross 
legs, as if walking on a balance beam, and some robots 
have been specifically designed to be able to realize 
such feats [48.113], but this can require unpractically 
slow steps. Ruling out impractically slow or imprac- 
tically quick steps, the legged locomotion may finally 
differ from the planned path and fail to avoid obstacles. 
In this case, iterative replanning techniques have been 
proposed [48.114], 

In this approach, however, it is assumed that finally 
finding proper footholds to follow the path planned 
for the upper body is not a problem. Yet, the appeal 
of legged robots is to provide improved mobility on 
difficult terrain, situations where finding footholds is 
actually a problem. A solution then is to check the avail¬ 
ability of proper footholds when planning the upper 
body motion [48.115], an approach recently refined to 
show how to obtain in the general case an exact equiva¬ 
lence between planning a continuous path and planning 
a sequence of footholds on a flat ground [48.116, 117]. 

A different approach is to plan directly the footholds 
with respect to obstacles on the ground, and gen¬ 
erate afterwards the corresponding dynamic legged 
locomotion [48.118]. This has been applied success¬ 
fully to long distance navigation with a tiered strat¬ 
egy [48.119], with computations fast enough to run 
online in changing environments [48.120], taking into 
account deterministically moving obstacles [48.121], 
on mildly rough terrain [48.122-124], using on-board 
sensing [48.125]. Obstacles above the ground can be 
considered then with a hybrid bounding box or more re¬ 
fined swept volumes [48.126, 127], or one can approxi¬ 
mate the free-space with convex segmentation [48. 128]. 

Legged Locomotion 
with Complex Contact Transitions 
On truly rough terrain or in cluttered environments, 
legged locomotion may require much more complex 
contact situations and contact transitions than in stan¬ 
dard walking or running, involving not only contacts 
between the feet and the ground, but all potential con¬ 
tact surfaces on the robot and on the environment. The 
variability of these situations is beyond comparison 
with the previous case on flat ground, and chances of 


precomputations or simplifications are strongly lacking 
in respect. Notably, the kinematic and dynamic con¬ 
straints cannot be tackled independently, and having to 
consider them simultaneously results in an overwhelm¬ 
ing computational problem. As a result, a major effort 
has been to find a proper ordering of the different sub¬ 
problems in order to obtain a tractable problem in the 
end, finally opting for planning contacts before mo¬ 
tion [48.129-131]. Complex real-life use-cases have 
been solved this way [48.132-134], but the computa¬ 
tional requirements still limit our ability to achieve the 
versatility of locomotion possible on flat ground. On¬ 
line adaptation to changing environments has yet to be 
convincingly demonstrated and remains an active area 
of research. 

As noted before, the underlying planning tech¬ 
niques manifest a strong bias towards quasi-static 
motions: some post-processing is necessary to obtain 
a truly dynamic motion in the end. The same two op¬ 
tions as in Sect. 48.4.1 have been proposed, standard 
trajectory optimization [48.135], or following the arti¬ 
ficial synergy synthesis approach and computing first 
the motion of the COM of the robot with respect to the 
different contact points, independently from the rest of 
the motion of the robot, which is computed only after¬ 
wards [48.21]. But the strong interdependency between 
kinematic and dynamic constraints defeats the core as¬ 
sumption of the artificial synergy synthesis approach in 
this case: dynamic feasibility can’t always be handled 
independently from the rest of the motion of the robot, 
so there is a significant risk that this method fails find¬ 
ing in the end a feasible motion even if one exists. 

Further discussion of all these problems in the 
specific case of humanoid robots can be found in 
Chap. 67.5 on whole-body activities of humanoid 
robots, or in [48.136]. There is also a rich literature 
on gait selection for many-legged robots, starting as 
early as [48.137] but this is still an active area of re¬ 
search [48.138]. 

48.4.4 Motion Generation 

with Limited Computing Resources 

The walking and running motion generation schemes 
presented so far in this chapter all require significant 
computation, even when they consider only simplified 
dynamical models. Early legged robots, which had very 
limited computing resources, had to rely on much less 
demanding approaches. One option is to combine sim¬ 
ple explicit rules, each one focusing on a different part 
of the overall motion. Another option is to try to mimic 
biological hypotheses on how animals generate and 
control their motion, what ended up in fact with some¬ 
how similar propositions. 
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Combining Simple Rules 

Thanks to simplifying symmetries in biped or quadruped 
running gaits such as trot, pace and bound, the whole 
family of robots hopping in the MIT LegLab through¬ 
out the 1980s on one, two or four legs, in 2-D (two-di¬ 
mensional) or 3-D (three-dimensional), could rely on the 
same simple control design [48.5], The general idea is to 
apply simple control laws independently to the different 
parts of the overall locomotion task. The vertical oscilla¬ 
tions were controlled by open-loop vertical thrusts, sim¬ 
ply stabilized by mechanical energy losses. Since the an¬ 
gular momentum is conserved during flight phases, the 
body attitude was controlled only during stance phases, 
with standard proportional-derivative (PD) control. The 
key to maintaining long-term balance, and controlling 
the global motion was foot placement. An elementary 
analysis of the motion of the COM during the stance 
phase revealed the role of the so-called neutral foot posi¬ 
tion, which would make the stance phase perfectly sym¬ 
metric. It is then just a matter of linear corrections of 
the foot position with respect to this neutral position to 
make the stance phase asymmetric in order to control 
the robot’s speed and balance. The direct combination of 
these utterly simple control laws resulted in impressively 
robust and versatile locomotion. 

The first biped robot that walked dynamically re¬ 
lied on an equally simple approach [48.4]. This robot 
emulated walking on stilts, with contact surfaces be¬ 
tween the feet and the ground reduced to single points, 
so z = p. With a single, fixed contact point during sin¬ 
gle support phases (the double support phases were 
supposed negligible), the ODI (48.29) can be solved an¬ 
alytically: for t>t 0 , 

fc x ’ y (t)-p x ’ y (t 0 )) _ fc x - y (t 0 ) -p x ’ y (t 0 )\ 

V i x - y (t) ) c^(to) J 

(48.56) 

with 

/ coshcu(f — to) cu _1 sinhft)(t — fo) 
y® sinhft)(t —to) cosh®(t —fo) 



(48.57) 

If the next single support phase starts at time t\ with 
a foot position p x,y {t\), we end up having 


(c x ’ y {t l )-p x - y {h)\ = 

V C x ' y (ti) ) 

Air L ( cX ’ y ^)-p X ’ y {t 0 )\ . (P x ’ y {t 0 )-P x ’ y {h)\ 

Mh) { c-(fo) j + l 0 )' 

(48.58) 


which describes the motion of the COM c x,y with re¬ 
spect to the contact points p x y , from one step, starting 
at time 4 , to the next, starting at time 44 - 1 . This dis¬ 
crete time dynamical system can be controlled then 
with the foot placement p x ' y (tk)- Since it is linear, stan¬ 
dard pole placement can be applied, yielding a standard 
PD control of foot placement. This simple approach 
has been successfully applied on more complex biped 
robots [48.139, 140]. 

The drawback of combining independent control 
laws is to lack coordination yielding potentially harsh, 
unrefined motion, but this approach demonstrates that 
stable locomotion is possible even with very limited 
control resources. Of primary importance here are foot 
placement and upper body attitude; the rest of the mo¬ 
tion appears to be secondary, at least with respect to 
balance control. Pushing this observation further, it has 
been proposed, and validated in simulation, that a large 
variety of walking patterns could be used, and stabilized 
only with upper body attitude and foot placement con¬ 
trol, both with simple PD control laws [48.141], It is 
not surprising then that hand-designed motion patterns 
can be used successfully on robots as complex as the 
Korea Advanced Institute of Science and Technology 
(KAIST) Hubo humanoid robot [48.142], although this 
may require delicate fine tuning. 

Reducing even further the control requirements, the 
mechanics of the robots could be tuned so that passive 
motions would automatically land the feet on the appro¬ 
priate locations with respect to stability, ending up with 
perfectly passive stable dynamic locomotion. This idea 
will be discussed more thoroughly in Sect. 48.6. 

Biomimetic Motion Generation 
Current theories on locomotion control in animals in¬ 
clude central pattern generators (CPGs) and cascades 
of reflex motions, which interact and combine to gen¬ 
erate the final motion. CPGs are tunable oscillators, 
generating synchronized quasi-cyclic motion patterns 
in response to simple control signals such as loco¬ 
motion speed or turning angle [48.143]. In order to 
introduce such CPGs in legged robots, standard oscil¬ 
lators such as the Van der Pol equation or the Hopf 
oscillator have been proposed [48.144,145], or more 
biologically inspired neural oscillators [48.146] (iron¬ 
ically, central pattern generators often happen to be 
decentralized, composed of a collection of synchro¬ 
nized oscillators [48.143, 145, 147]). 

But CPGs are intrinsically open-loop motion gen¬ 
erators, so to stabilize and adapt the motion of the 
robot to their environment, limited feedback loops have 
been introduced, similar to the simple control laws dis¬ 
cussed earlier in this Section, focusing on upper body 
attitude and foot placement [48.145, 147, 148]. Contin- 
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uing with the biological analogy, these simple feed¬ 
back loops have been called reflexes. Following ideas 
discussed more thoroughly in Chap. 13 on behavior- 
based systems, the subsumption architecture has been 
proposed to build much more complex networks of 
reactive behaviors or reflexes [48.149], and has al¬ 
lowed a hexapod robot to walk successfully on mildly 
rough terrain without any careful planning of the mo¬ 
tion [48.150]. 

Fundamentally 

the Same Decoupling Approach 
Being inspired by simple mechanical analyses or by 
animals, these approaches based on combinations of 

48.5 Motion and Force Control 

Properly executing the walking and running motions 
computed in the previous section requires precise con¬ 
trol of the motion of the robot and of the contact 
forces with the environment, applying techniques from 
Chapters 8 and 9 on motion and force control. But 
since legged robots are often complex mechanical sys¬ 
tems with many degrees of freedom and some form of 
redundancy, techniques from Chap. 10 on redundant 
manipulators also need to be applied frequently. And 
in case of contact with multiple surfaces, problems of 
contact force distribution appear, requiring techniques 
similar to those found in Chap. 38 on grasping. And in 
the opposite case, during flight phases, when the robot 
is not at all in contact with its environment, its dynamics 
appear to share important properties with free-floating 
space robots, requiring control techniques similar to 
those found in Chap. 55 on space robotics. All these 
techniques (and many more) are necessary to make 
legged robots walk and run efficiently. The purpose of 
this Section is to see how they connect and are imple¬ 
mented in the case of legged robots. 

Whole-Body Motion 

Humanoid and other legged robots are often complex 
mechanical systems with a large number of degrees of 
freedom in order to accomplish the various kinematic 
and dynamic tasks necessary for simultaneously achiev¬ 
ing locomotion, perception and manipulation of the 
environment, interaction with humans, etc. Making use 
of all available degrees of freedom, and not only those 
in the legs, to achieve multiple simultaneous objectives, 
and not only locomotion, is usually called whole-body 
motion control (Chap. 67.5). We have seen previously 
that locomotion mostly involves the motion of the COM 
c with respect to points p, of contact with the environ¬ 
ment. Other objectives involve end-effector motion for 


simple rules demonstrate that stable walking can be 
realized without much computation, at least in simple 
situations. More complex situations may require more 
refined motions and therefore more refined motion gen¬ 
eration schemes, such as the ones discussed earlier in 
this chapter. But it is striking to observe again the same 
focus here on the motion of the COM of the robot 
with respect to the contact points, when controlling ei¬ 
ther the upper body attitude or the foot placement (or 
both), regardless of the rest of the motion, which can 
be generated by CPGs, inverse kinematics or any other 
approach, simple or complex. This is fundamentally the 
same decoupling as proposed earlier with the artificial 
synergy synthesis approach. 


manipulation, gaze control for perception, motion of 
specific parts of the robot for obstacle avoidance, etc. 
In all of these cases, what needs to be controlled is 
Cartesian motion of different parts of the robot, rather 
than joint motion. Several control schemes have been 
proposed to do this, standard inverse kinematics fol¬ 
lowed by joint space control [48.139], virtual model 
control [48.151], the task function approach [48.152, 
153], operational space control [48.154]. 

Interestingly, the last two options allow specifying 
different priorities to the different objectives assigned 
to the robot, so in case these objectives are not feasible 
jointly, the robot tries first to achieve the objectives with 
higher priority. In the case of legged robots, maintain¬ 
ing balance and avoiding falls appear to be of primary 
importance for the safety of both the robot and its 
environment, so being able to consider this objective 
with a higher priority can be crucial in this regard. 
These control schemes allow moreover some form of 
decoupling between the different objectives, further 
contributing to the artificial synergy synthesis approach 
introduced earlier in the previous Section. We are going 
therefore to continue focusing here exclusively on the 
locomotion objective, since the other ones can be con¬ 
sidered more or less independently and are not specific 
to legged robots. 

Flight Phases 

We have seen earlier in this chapter that during flight 
phases, when a legged robot is not in contact with its 
environment, the motion of the COM invariably fol¬ 
lows a standard falling motion: there is no possibility 
to make it move in any other way. But we have seen 
that even though the angular momentum L is constant 
during flight phases, the joint configuration q and the 
global orientation 0q of the robot can still be driven 
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concurrently together to any desired value thanks to 
the nonholonomy of the angular momentum (48.5). 
However, a famous theorem due to Brockett [48.155] 
implies that continuous time-invariant feedback control 
laws such as those proposed in [48.156, 157] are not 
able to do so. The solution is to use discontinuous or 
time-varying control laws [48.158], an approach well 
established in space robotics [48.159,160]. 

In the case of running motions, it has been pro¬ 
posed to specifically adapt leg trajectories during flight 
phases in order to control both the joint configura¬ 
tion and the global orientation at the end of the flight 
phases [48.161]. It has been measured indeed that 
a simple difference in the height of a single step can 
result in 1 or 2 degrees of difference in the global ori¬ 
entation of the robot [48.8]. Albeit small, this effect 
is far from negligible. More thorough discussions of 
nonholonomic constraints and the impact of Brackett’s 
theorem can be found in Chap. 49 on the modeling and 
control of wheeled mobile robots or in [48.158]. 

Angular Momentum 

Either considering the ODI (48.13) when walking on 
a flat ground, or the Newton and Euler equations (48.3) 
and (48.4) in the general case, the dynamic feasibility 
of legged motions appears to be directly and entirely 
related to the motion of the COM c, and the angular 
momentum, L. It has been naturally proposed there¬ 
fore to control specifically those two quantities con¬ 
currently [48.162-164]. Of course, since COM motion 
and angular momentum are directly related to contact 
forces /,- (or equivalently to the CoP, z), controlling 
the former is absolutely equivalent to controlling the 
latter. The question however is whether the angular mo¬ 
mentum should be specifically controlled to some given 
value and what that value should be [48.163, 164]. 

Of course, since the angular momentum is directly 
related to contact forces, it should comply with the uni¬ 
laterality and Coulomb friction limitations on contact 
forces (48.11) and (48.17). As a matter of fact, the 
regulation of angular momentum has been discussed 
only in quasi-static situations [48.162-164], where the 
reference value is clearly zero, which generally com¬ 
plies well with these limitations. And in the only case 
where walking was considered, the horizontal angular 
momentum L x,y , that appears in the ODI (48.13) when 
walking on a flat horizontal ground, was explicitly not 
controlled [48. 162]. It appears, in fact, that during walk¬ 
ing motions, the question of which value the angular 
momentum should have is very intricate, because of its 
nonholonomy, but it is definitely not zero [48.8, 103]. 

Even in quasi-static situations, both joint configu¬ 
ration and global orientation can be controlled inde¬ 
pendently from the angular momentum; the connection 


between these different values is very subtle. As a re¬ 
sult, controlling the angular momentum does not induce 
a control of either joint configuration or global ori¬ 
entation (due to its nonholonomy). The opposite is 
true however: controlling joint configuration and global 
orientation does induce a control of the angular momen¬ 
tum as a derived quantity. It has been argued further¬ 
more that common whole body motion objectives such 
as manipulation or interaction with humans are more 
directly related to joint configuration and global orien¬ 
tation than to angular momentum [48.165]. As a matter 
of fact, disregarding the exact variations of the angu¬ 
lar momentum has been the most frequent option so 
far [48.139, 151-154, 165]. In the end, it is not clear if 
specifically controlling the angular momentum can help 
controlling legged robots: this is still an open question. 

COM Motion Control 

Putting the angular momentum aside, and considering 
only the simple linear dynamics (48.29) when walking 
on a flat ground, and more precisely its decomposition 
(48.34) and (48.35), we have seen that every state satis¬ 
fying the condition (48.36), such that the capture point 
§ is in the support polygon, is capturable and can there¬ 
fore be stabilized. If the capture point is strictly inside 
the support polygon, the COM can even be stabilized to 
any reference position c re f above the support polygon, 
with a simple linear feedback of the capture point 

Z x ’ y =C X J + k(S;*' y -C X J), (48.59) 

where the feedback gain k > 1 must be tuned in order 
to keep the CoP z within the support polygon, ensur¬ 
ing dynamic feasibility of the motion. Indeed, using 
this feedback with the dynamics (48.35) yields a stable 
closed loop dynamics 

t y = M (k-l)(c x J-r’ y ), (48.60) 

according to which the capture point | will converge to 
the reference position c re f, and the COM c will converge 
to it as a result of the stable dynamics (48.34). It has 
been shown that among all proportional-derivative (PD) 
feedback of the motion of the COM 

z* 01 = c x J + k(c x ’ y + A c x ’ y -c x J) , (48.61) 

the feedback (48.59) of the capture point, with a choice 
of A = <y , is the only one that will successfully 
stabilize all capturable states [48.166]. It has been suc¬ 
cessfully applied to walking motions on the DLR biped 
robot [48.59]. 

Now, if there is a time-varying perturbation or er¬ 
ror e in our dynamics, such that 

c x ’ y - -c x ’ y = z x ’ y + e , (48.62) 

g z 
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the PD feedback (48.61) can be shown to be robustly 
stable (input-to-state stable, to be precise) with respect 
to e, but with A < a>~ 1 and k > jj -2 [48.167]. As dis¬ 
cussed above, with this choice of A, not all capturable 
states will be stabilized successfully, even if there is no 
perturbation. This is unfortunately a usual trade-off in 
robust control. 


Force Feedback 

Motion control of legged robots without contact force 
sensing and feedback has been shown to be possible 
for simple quasi-static balancing tasks, using passiv¬ 
ity based control [48.168]. However, force sensing and 
feedback is commonly used when realizing more dy¬ 
namic motions such as walking or running. One of the 
first problems to overcome in this case is the compli¬ 
ance of both the robot and its environment, which easily 
generates destabilizing oscillations [48.169-172]. Con¬ 
tact force control schemes of various complexity have 
been proposed, but always with the goal of damping 
these undesired oscillations. The exact implementation 
of these force control schemes generally relies on a stiff 
lower-level joint motion control in order to enhance 
disturbance rejection, leading to admittance control im¬ 
plementations [48.139, 169-173]. More details on such 
control schemes can be found in Chap. 9 on force 
control. 

Now, these damping control schemes inevitably in¬ 
troduce some delay in the realization of the desired 
contact forces. One option is to show that the COM 
motion control scheme described earlier, which entirely 
relies on contact forces, is robust to such delay [48.59], 
Another option is to take this delay into account with 
a simple first-order dynamics of the CoP 

Z = QJ z (Zd-Z) , (48.63) 


and adapt the previous COM motion control scheme ac¬ 
cordingly [48.174, 175] 

z/ = C X J + - <£?) + - c£) . (48.64) 


This introduces some force feedback directly in the 
COM motion control through the measurement of the 
CoP z, what can help improve the reactivity of the con¬ 
troller to perturbations. 


precisely quadratic programming (QP) with linear con¬ 
straints, using a faceted, linear approximation of the 
Coulomb friction model (48.17) 

Afi <b . (48.65) 

It has been proposed then to minimize when possible 
the norm of the joint torques u in order to reduce energy 
consumption [48.152], 

min IImII 2 , (48.66) 

fi 

or to balance the contact forces between the different 
contact points, either by minimizing the sum of their 
norms [48.165], 

min ^ \[fj\\ 2 , (48.67) 

1 i 

or the norm of their difference [48.176], 

min |[/'ieft—/rightll 2 , (48.68) 

or minimizing the tangential forces in order to reduce 
the risk of slipping [48.177], 

min ^ \lf ' ,y || 2 . (48.69) 

‘ i 

More general QP formulations have been pro¬ 
posed [48.178, 179], including hierarchies of kinematic 
and dynamic tasks and constraints [48.153]. 

But contact forces obtained in these ways are not 
explicitly kept away from the edges of the friction cone 
and unilaterality condition, so that the slightest pertur¬ 
bation may quickly lead to sliding or tipping, what can 
lead to serious issues. The simplest way to steadily im¬ 
prove the robustness of the robot to such perturbations 
is to introduce a security margin w 


Aft <b — wl, (48.70) 

where 1 is a vector of ones, but there is no obvious way 
to decide which value would be appropriate. One option 
then is to try to maximize it with a linear program (LP) 


Contact Forces Distribution 
In case of contact with multiple surfaces, computing 
contact forces/,, that allow realizing the desired motion 
control scheme through the Newton and Euler equa¬ 
tions (48.3) and (48.4), and yet comply with both the 
unilaterality condition (48.11) and the Coulomb fric¬ 
tion model (48.17), can be slightly more involved. The 
standard approach is through optimization, and more 


max W , (48.71) 

wji 

improving as much as possible the robustness of the 
robot to perturbations by selecting contact forces /, the 
farthest possible from the edges of the (faceted) friction 
cone and unilaterality condition. Note that the maximal 
value w* obtained in this way is the residual radius dis¬ 
cussed in [48.19]. 
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48.6 Towards More Efficient Walking 


Examples of remarkably efficient locomotion are ubiq¬ 
uitous in nature. From sprinting cheetahs to leaping 
sifakas, the variety and beauty of animal motion has in¬ 
spired roboticists for decades. Yet some of the most im¬ 
pressive examples of humanoid walking, e.g., Honda’s 
Asimo or Boston Dynamics’ Atlas, have employed 
high-impedance joint control to achieve stable locomo¬ 
tion. As a result, the fluidity of their motions is far from 
that of their biological counterpart. Indeed, these robots 
expend an order of magnitude more energy than a typi¬ 
cal human while walking [48.180], 

In contrast, the passive dynamic walking machines 
of McGeer and others [48.6, 181] have been able to 
achieve surprisingly human-like gaits without any actu¬ 
ation, relying solely upon their carefully designed body 
dynamics. These machines moved down sloped terrain, 
relying upon the conversion of potential to kinetic en¬ 
ergy to balance the losses introduced by impacts and 
friction. Subsequent examples of minimally-actuated 
bipeds capable of walking under their own power on 
flat terrain made the first steps toward realizing efficient 
walking machines [48.180, 182]. 

The existence of these robots leads to several inter¬ 
esting questions. For example, how can we find limit 
cycles of underactuated walkers and reason about their 
stability? Simple models of walking systems have been 
used to try to answer these questions for over two 
decades. Such analyses typically rely upon linearization 
of the model dynamics to integrate the equations of mo¬ 
tion or numerical simulation of the nonlinear dynamics 


and analysis via Poincare maps (Sect. 48.3.2). For ex¬ 
ample, the compass gait model is one of the simplest 
walking models, consisting of three point masses - one 
at the hip and one on each leg - and the legs connected 
to the hip with a pin joint (Fig. 48.10). Although even 
this very simple model cannot be analyzed analytically, 
a variety of stable gaits have been identified using nu¬ 
merical techniques [48.186]. 

Today there exists a community of researchers 
working to bridge the gap between stable and versa¬ 
tile machines like Asimo and efficient dynamic walkers. 
Topics of particular interest include navigating ro¬ 
bustly over rough terrain, generating and stabilizing 
nonperiodic gaits, and scaling up to more complex 
robot models. New computational tools for generating 
and stabilizing trajectories for underactuated dynamical 
systems have been an important result of this work. In 
this section we highlight the fundamental insights pro¬ 
vided by work on passive dynamic walking machines 
and describe ways in which these ideas have been ex¬ 
tended to more general systems. 

48.6.1 Gait Generation 

for Dynamic Walking 

The most direct approach to producing efficient walking 
gaits in underactuated robots is to design controllers that 
produce actuated trajectories that mimic passive ones. 
Fundamental to these methods is the idea of restora¬ 
tion of mechanical energy. Since mechanical energy is 



Fig.48.9a-f Passive-dynamic walking machines, (a) The Wilson Walkie (after [48.183]). (b) A slightly improved ver¬ 
sion (after [48.184]). Both (c) and (d) walk down a slight ramp with the comical, awkward, waddling gait of the penguin 
(after [48.183]). (e) Cornell copy (after [48.185]) of McGeer’s capstone design. This four-legged biped has two pairs 
of legs, an inner and outer pair, to prevent falling sideways, (f) The Cornell passive biped with arms (photo by Hank 
Morgan) (after [48.181]). This walker has knees and arms and is perhaps the most human-like passive-dynamic walker 
to date (photo reproduced with permissions from [48.180]) 
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Fig. 48.10 The compass gait 

lost at each heel strike, passive dynamic walkers require 
gravitational force to restore energy to the system before 
the next step. By choosing control inputs that regulate 
mechanical energy, for example by simulating gravity, 
walking on level terrain can be achieved [48.187, 188]. 
However, such approaches are still largely limited to 
modest terrains and periodic motions. 

Trajectory optimization methods offer the potential 
to move beyond limit cycle gaits and to more com¬ 
plicated systems without passive stability properties. 
However, the fact that walking systems must make and 
break contact with the world complicates the trajectory 
optimization problem. Simulating rigid-body contact 
forces continuously using spring and damper models 
often results in stiff differential equations that can lead 
to numerical and computational issues. An alternative 
approach is to capture discontinuities in velocity by 
modeling collisions as impulsive events using a hybrid 
system representation (Sect. 48.2.3). For a walking hy¬ 
brid system, a set of autonomous guard functions are 
defined such that cpi(q) = 0 implies that body i is in con¬ 
tact and <j>i(q) > 0 implies the opposite. When a body 
comes into contact, the generalized velocity undergoes 
an instantaneous change, q 1 =Ja (<7, q), where <f~ 
and q 1 are the velocities pre and post-impact, respec¬ 
tively. The mode of the system is defined by the set 
of active contacts, C = {i\cpj(q) = 0}, which in turn de¬ 
fines the set of Jacobian matrices in (48.2). 

The hybrid system representation creates several 
challenges for trajectory optimization methods. The 
discontinuities due to mode transitions are typically 
accounted for explicitly using either direct or multiple¬ 
shooting methods [48.189]. Since each mode implies 
a different dynamics and mode changes are neces¬ 
sary for walking, this leads to the question of how 
to formulate the dynamics constraints. One solution 
is to predefine the sequence of mode transitions. For 
simple models with point feet, this approach can 
work quite well. However, for more complex mod¬ 
els with many contacts, the number of modes grows 


exponentially, making mode sequence prespecification 
impractical. 

The complementarity-based methods for simulating 
rigid body systems in contact described in Sect. 48.2.3 
suggest another approach. By discretizing the dynam¬ 
ics in time and considering only the integral of contact 
forces over a time interval, the forward dynamics of 
a walking system can be greatly simplified. Since many 
trajectory optimization methods already discretize by 
evaluating a finite set of points along a trajectory, in¬ 
corporation of contact forces as optimization variables 
leads to trajectory optimization as a nonlinear optimiza¬ 
tion problem with complementarity constraints [48.37]. 

Many impressive examples of trajectory optimiza¬ 
tion for walking and running exist in the literature. For 
example, Mombaur [48.78] simultaneously optimized 
motion and physical parameters for simulated bipeds 
to achieve open-loop stable running and Remy [48.190] 
applied trajectory optimization to generate efficient run¬ 
ning for a quadruped. There are many more examples 
and it remains an active area of research. The problem of 
stabilizing trajectories optimized offline remains a chal¬ 
lenge, particularly when the execution deviates from the 
planned mode sequence. Computational requirements 
still prevent nonlinear trajectory optimization methods 
from being applied online as an MPC algorithm, but im¬ 
provements in numerical solvers and computing hard¬ 
ware will likely continue to reduce this gap. 

48.6.2 Orbital Trajectory 

Stabilization and Control 

Once an open-loop gait is generated, it must typically 
be stabilized in order to be executed on a robot. Clas¬ 
sical techniques for trajectory stabilization from linear 
control can work, but a strong theme in stabilization 
for legged robots has been the idea that one should 
not enforce the precise timing of the trajectory. Sec¬ 
tion 48.3.2 discussed the notion of orbital stability 
and introduced Poincare maps as a tool for stability 
analysis. Orbital stabilization, rather than traditional 
trajectory stabilization, sacrifices little and appears to 
be more compatible with underactuated robots. These 
ideas have also played an important role in developing 
stable dynamic walking and running controllers using 
virtual constraints [48.38]. Virtual constraints are holo- 
nomic constraints on monotonic functions of the robot’s 
configuration variables. For example, for a forward pe¬ 
riodic gait, it might be reasonable to assume that the 
angle of the stance leg with respect to the ground 9 st is 
monotonically increasing in time throughout the stance 
phase. The remaining configuration variables are writ¬ 
ten as functions of 6 st , effectively reparametrizing time, 
and constraints are imposed that, e.g., enforce sym- 
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metry in stance and swing leg angles (0 SW = — 9 st ) or 
coordinate arm and leg motion. This leads to outputs 


y = 


h\(q) 

h d (q) 


(48.72) 


where each h,(q) encodes a virtual constraint. For ex¬ 
ample, h(q) = 9 SW + 6 si encodes the virtual constraint 
for swing-stance leg symmetry. By driving the output 
dynamics asymptotically to 0 using high-gain control, 
the system can be seen to evolve according to the vir¬ 
tual constraints, yielding a lower dimensional problem 
for control design and stability analysis. This approach 
has been used to produce remarkable dynamic walking 
and running examples in real robots [48.191]. 

Poincare maps are defined with respect to a trans¬ 
verse surface at a single point on a periodic orbit, which 
limits their usefulness for designing stabilizing con¬ 
trollers. The transverse dynamics, on the other hand, is 
defined with respect to a continuous family of surfaces 
S(t), transverse to the orbit [48.45]. In the neighborhood 
of these coordinates, the dynamics can be written as 

r = 1 +fi(r,x±) (48.73) 

*j_ = A(t)x_l +f 2 (r,x±) , (48.74) 

where r e [ 0 , ff] represents the orbit phase, x± are the 
2N — 1 coordinates orthogonal to the flow of the sys¬ 
tem, and/i(-),/ 2 (-) are functions containing nonlinear 
terms. Note that unlike the discrete Poincare map, these 
dynamics do not require the trajectory of interest to be 
a periodic orbit. 

By extracting the linear part of the transversal dy¬ 
namics and incorporating control inputs, we obtain the 
transverse linearization 


z(t) = A(t)z(t) + B(t)u(t) , 



(48.75) 


for t e [0, ff]. Stabilization of this system, and therefore 
the underlying trajectory, can be approached using stan- 


Fig. 48.11 A simple illustration of transverse linearization along 
a nominal trajectory (red) for systems with impacts 

dard techniques from linear control theory. For walking 
systems with impulses, the dynamics can be extended 
to include a linearized impact map, z(tf~) = C iZ(t~), 
where 4 , i e C are times at which impacts occur. Fig¬ 
ure 48.11 illustrates the idea of transversal surfaces 
about a trajectory for a system with impacts. Manch¬ 
ester et al. [48.50] showed that under some mild as¬ 
sumptions, local exponential orbital stability of planned 
motions on a hybrid nonlinear system can be achieved 
using a receding horizon control with the transverse lin¬ 
ear dynamics. 

Minor variations in impact timing can be handled 
in linearization-based approaches to stabilizing walk¬ 
ing trajectories, but perturbations that lead to changes 
in mode ordering often cannot. To address this problem, 
we need algorithms for adjusting walking trajectories 
online. As described in the previous section, the hybrid 
nature of walking systems makes this challenging. One 
way to avoid this issue is to employ smooth approxi¬ 
mations of the dynamics that permit the use of efficient 
local trajectory optimization techniques at the expense 
of violating complementarity condition (48.22) by, e.g., 
allowing interpenetration of rigid bodies. This approach 
has led to impressive examples of online trajectory op¬ 
timization for simulated systems [48.192], but it is yet 
unknown how well these approximations will transfer 
to physical systems. 


48.7 Different Contact Behaviors 

Contact forces have appeared throughout this chapter 
to be central for the modeling and control of legged 
robots. As a result, a different contact behavior can re¬ 
sult in a significantly different dynamical behavior of 
the whole robot, and this can be extremely useful in sit¬ 
uations where standard walking or running is inefficient 
or downright impossible. 

48.7.1 Wall Climbing 

Wall climbing legged robots rely on various devices 
such as suction cups, magnets, adhesive materials, 


miniature spine arrays, that can generate adhesive 
forces in order to stick to various vertical surfaces such 
as glass, steel, concrete, brick, stone [48.193-195]. In 
this case, the unilaterality condition (48.1 1), which was 
central throughout this chapter, is no more relevant. As 
a result, the whole locomotion modeling and control is 
deeply transformed. 

48.7.2 Tethered Walking 

On steep slopes, rappelling is an interesting option, teth¬ 
ering the robot to anchors in order to avoid tumbling 
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down. This has been experienced successfully on con¬ 
struction sites [48.196], or in a volcano in the case 
of the impressive Carnegie Mellon University (CMU) 
Dante II octopod robot [48.197]. In this case, the loco¬ 
motion problem is similar to the general case described 
throughout this chapter, but the tether introduces an ad¬ 
ditional contact between the robot and the environment, 
on which the robot can pull but not push, so a unilat¬ 
eral contact exactly opposite to the standard unilateral 
contact between the feet and the ground, what can be 
treated however in exactly the same way. 

48.7.3 Legs with Wheels 

Adapting wheeled vehicles to rough terrain has led 
in some cases to implant wheels on legs, combining 
the flexibility of articulated legs on rough terrain with 
the efficiency of wheels on flat terrain. Different op¬ 
tions are possible, passive or active wheels, passive 


48.8 Conclusion 

Legged systems are at the heart of some of the most 
exciting work in modem robotics. They offer the oppor¬ 
tunity to travel to places beyond the reach of wheeled 
systems and gain fundamental insights into the con¬ 
ditions under which stable and efficient locomotion is 
possible. At the same time, their complex dynamics 
pose significant challenges for our computational ap¬ 
proaches to control and stability analysis. Indeed, the 
simple fact that every walking system must engage in 
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49. Modeling and Control of Wheeled 

Mobile Robots 


Claude Samson, Pascal Morin, Roland Lenain 


This chapter may be seen as a follow up to Chap. 24, 
devoted to the classification and modeling of ba¬ 
sic wheeled mobile robot (WMR) structures, and 
a natural complement to Chap. 47, which surveys 
motion planning methods for WMRs. A typical out¬ 
put of these methods is a feasible (or admissible) 
reference state trajectory for a given mobile robot, 
and a question which then arises is how to make 
the physical mobile robot track this reference tra¬ 
jectory via the control of the actuators with which 
the vehicle is equipped. The object of the present 
chapter is to bring elements of the answer to this 
question based on simple and effective control 
strategies. 

The chapter is organized as follows. Sec¬ 
tion 49.2 is devoted to the choice of control models 
and the determination of modeling equations as¬ 
sociated with the path-following control problem. 
In Sect. 49.3, the path following and trajectory sta¬ 
bilization problems are addressed in the simplest 
case when no requirement is made on the robot 
orientation (i.e., position control). In Sect. 49.4 
the same problems are revisited for the control 
of both position and orientation. The previously 
mentionned sections consider an ideal robot sat¬ 
isfying the rolling-without-sliding assumption. In 
Sect. 49.5, we relax this assumption in order to 
take into account nonideal wheel-ground contact. 
This is especially important for field-robotics ap¬ 
plications and the proposed results are validated 
through full scale experiments on natural terrain. 
Finally, a few complementary issues on the feed¬ 
back control of mobile robots are briefly discussed 
in the concluding Sect. 49.6, with a list of com¬ 
mented references for further reading on WMRs 
motion control. 
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49.1 Background 

Wheeled mobile robotics remains a very active 
research topic, as witnessed by the Darpa Grand Chal¬ 
lenges [49.1]. The first Grand Challenge, in 2005, 
took place in a clear environment and was won by 
Stanley (Fig. 49.1a). The second Grand Challenge, in 
2007, took place in urban environment and demon¬ 
strated autonomous driving capabilities, especially in 
complex environments with strong safety and relia¬ 
bility issues. This made possible the development of 
autonomous cars, such as google car (Fig. 49.1b). 
Whatever the sensors used for localization and de¬ 
tection, the use in every day life of cybercars, such 
as VipaLab (Fig. 49.1c), is becoming a reality. Nu¬ 
merous tests are under progress all around the world 
showing the benefits of autonomous transportation sys¬ 
tems. On-road transportation probably constitutes the 
main and most visible application domain of emerg¬ 
ing autonomous wheeled robots. Driving assistance for 
human activities in off-road conditions constitutes an¬ 
other application domain. Indeed, WMRs can also be 
used in hazardous environments, or in environments 
that are not reachable by humans. Planetary exploration 
(Chap. 55), with the example of Curiosity (Fig. 49. Id), 
is a popular example of such applications. Other devel¬ 
opments, concerning people well-being and safety, are 
also investigated in the field of environment monitoring, 
surveillance, agriculture, civil protection or defence. 
Figure 49.1e,f illustrate these developments with the 
first one representing the Claas etrion robot (for au¬ 
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and Other Dynamical Issues . 1262 
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tonomous farming, see also Chap. 56) and the second 
one representing the SUGV robot manufactured by 
iRobot. This robot, dedicated to civil protection, was 
the first one to enter the damaged Fukushima power 
plant. All these examples highlight the possible bene¬ 
fits and different applications of WMRs. The variety of 
robot designs (in term of scale, locomotion mode, or 
related actuation strategy see Chap. 24), is in line with 
the diversity of application needs. Whatever the system 
and application, however, autonomy relies on the de¬ 
sign of efficient feedback laws that can ensure precise 
autonomous motion of the vehicle despite all possible 
modeling errors and perturbations. This chapter is ded¬ 
icated to this issue and aims at providing the basics of 
feedback control design for nonholonomic WMRs. Im¬ 
plementation of these control laws supposes that one 
is able to measure the variables involved in the control 
loop (typically the position and orientation of the mo¬ 
bile robot with respect to either a fixed frame or a path 
that the vehicle should follow). Throughout this chapter 
we will assume that these measurements are available 
continuously in time and that they are not corrupted by 
noise. In a general manner, robustness considerations 
will not be discussed in detail, one reason being that, 
beyond imposed space limitations, a large part of the 
presented approaches are based on linear control the¬ 
ory. The feedback control laws then inherit the strong 
robustness properties associated with stable linear sys¬ 
tems. Results can also be subsequently refined by using 



Fig.49.1a-f Some example of mobile 
robots for different applications: (a) 
Stanley (Stanford Univ.), off-road 
motion; (b) Google car, autonomous 
car; (c) Vipa (Institut Pascal/Ligier), 
urban transportation; (d) Curiosity 
(NASA) planetary exploration; (e) 
Etrion (Claas), agriculture robot; (f) 
PackBot (iRobot), public safety 
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complementary, eventually more elaborate, automatic 
control techniques. For the sake of simplicity, the con¬ 
trol methods are developed mainly for unicycle-type 
and car-like mobile robots, which correspond respec¬ 
tively to the types (2,0) and (1,1) in the classification 
proposed in Chap. 24. Most of the results can in fact 
be easily extended/adapted to other mobile robots, in 
particular to systems with trailers. We will mention the 
cases where such extensions are straightforward. All 
reported simulation results, illustrating various control 
problems and solutions, are carried out for a car-like ve¬ 
hicle, whose kinematics is slightly more complex than 
that of unicycle-type vehicles. 

Recall (Fig. 49.2) that: 

1. A unicycle-type mobile robot is schematically com¬ 
posed of two independent actuated wheels on 
a common axle whose direction is rigidly linked 
to the robot chassis, and one or several passively 
orientable - or caster - wheels, which are not con¬ 
trolled and serve for sustentation purposes. 

2. A (rear-drive) car-like mobile robot is composed of 
a motorized wheeled axle at the rear of the chas¬ 
sis, and one (or a pair of) orientable front steering 
wheel(s). 

Note also, as illustrated by the diagram below 
(Fig. 49.3), that a car-like mobile robot can be viewed 
(at least kinematically) as a unicycle-type mobile robot 
to which a trailer is attached. 

Despite the variety of application, three generic 
control problems can mainly be considered in mobile 
robotics and are detailed in this chapter. 

49.1.1 Path Following 

Given a curve C on the plane, a (nonzero) longitudinal 
velocity vq for the robot chassis, and a point P attached 
to the chassis, the goal is to have the point P follow 
the curve C when the robot moves with the velocity 
Vq- The variable that one has to stabilize at zero is thus 
the distance between the point P and the curve (i. e., 
the distance between P and the closest point M on C). 
This type of problem typically corresponds to driving 
on a road while trying to maintain the distance between 



Fig.49.2a,b Unicycle-type (a) and car-like (b) mobile 
robots 


the vehicle chassis and the side of the road constant. Au¬ 
tomatic wall following is another possible application. 

49.1.2 Stabilization of Trajectories 

This problem differs from the previous one in that the 
vehicle’s longitudinal velocity is no longer predeter¬ 
mined because one also aims to monitor the distance 
gone along the curve C. This objective supposes that 
the geometric curve C is complemented with a time 
schedule, i. e., that it is parameterized with the time 
variable t. This boils down to defining a trajectory 
1 1 —> {x r (t),y r (t)) with respect to a reference frame J'o- 
Then the goal is to stabilize the position error vector 
(x{t) — x t {t),y(t) — y r (t)) at zero, with (x(t),y(t)) denot¬ 
ing the coordinates of point P in J'o at time t. The 
problem may also be formulated as one of control¬ 
ling the vehicle in order to track a reference vehicle 
whose trajectory is given by ri—> (x r (t),y r (f)). Note 
that perfect tracking is achievable only if the reference 
trajectory is feasible for the physical vehicle, and that 
a trajectory which is feasible for a unicycle-type vehicle 
is not necessarily feasible for a car-like vehicle. Also, 
in addition to monitoring the position (x(t),y(t)) of the 
robot, one may be willing to control the chassis orien¬ 
tation 9(t ) at a desired reference value 9 r (t) associated 
with the orientation of the reference vehicle. For a non- 
holonomic unicycle-type robot, a reference trajectory 
(x r (t),y r (t), 9 t (t )) is feasible if it is produced by a refer¬ 
ence vehicle which has the same kinematic limitations 
as the physical robot. For instance, most trajectories 
produced by an omnidirectional vehicle (omnibile vehi¬ 
cle in the terminology of Chap. 24) are not feasible for 
a nonholonomic mobile robot. However, nonfeasibility 
does not imply that the reference trajectory cannot be 
tracked in an approximate manner, i. e., with small (al¬ 
though nonzero) tracking errors. 

49.1.3 Stabilization of Fixed Postures 

Let jFi denote a frame attached to the robot chassis. In 
this chapter, we call a robot posture (or situation) the 
association of the position of a point P located on the 
robot chassis with the orientation 9(t) of Jj with re¬ 
spect to a fixed frame Jq in the plane of motion. For 



Fig. 49.3 Analogy car/unicycle with trailer 
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this last problem, the objective is to stabilize at zero the 
posture vector £(t) = ( x(t),y(t ), 9{t)), with (x(t),y(t)) 
denoting the position of P expressed in J'q. Although 

49.2 Control Models 

49.2.1 Kinematics Versus Dynamics 

In order to proceed with the control of mobile robots, 
a model describing its motion has to be derived. Several 
levels of motion equations may be derived, pending on 
the targeted application (expected velocity, terrain con¬ 
figuration, etc.). Chapter 24 provides a general config¬ 
uration dynamic model for WMRs. Its particularization 
to the case of unicycle-type and car-like mobile robots 
gives 

H(<jr)ii + F(q, u)u = T(</>)t , (49.1) 

with q denoting a robot’s configuration vector, u a vec¬ 
tor of independent velocity variables associated with the 
robot’s degrees of freedom, H(^) a reduced inertia ma¬ 
trix (which is invertible for any q), F(q, u)u a vector 
of forces combining the contribution of Coriolis and 
wheel-ground contact forces, <p the orientation angle 
of the car’s steering wheel, T an invertible control ma¬ 
trix (which is constant in the case of a unicycle-type 
vehicle), and r a vector of independent motor torques 
(whose dimension is equal to the number of degrees 
of freedom in the case of full actuation, i. e., equal to 
two for the vehicles considered herein). In the case of 
a unicycle-type vehicle, a configuration vector is com¬ 
posed of the components of the chassis posture vector £ 
and the orientation angles of the castor wheels (with re¬ 
spect to the chassis). In the case of a car-like vehicle, 
a configuration vector is composed of the components 
of £ and the steering wheel angle </>. To be complete, 
this dynamic model must be complemented with kine¬ 
matic equations in the form 

q = S(q)u, (49.2) 

from which one can extract a reduced kinematic model 

Z = B(z)u, (49.3) 

with z = §, in the case of a unicycle-type vehicle, 
and z = (|, tp) in the case of a car-like vehicle. In the 
automatic control terminology, the complete dynamic 


a fixed desired (or reference) posture is obviously a par¬ 
ticular case of a feasible trajectory, this problem cannot 
be solved by classical control methods. 


model (49.1) and (49.2) forms a control system which 
can be written as 

X=f(X, t) with X = (<jf, u) 

denoting the state vector of this system, and r the vec¬ 
tor of control inputs. The kinematic models (49.2) and 
(49.3) are also control systems with respective state 
vectors q and z and control vector u. Any of these mod¬ 
els can be used for control design and analysis purposes. 
In the remainder of this chapter, we have chosen to work 
with the kinematic model (49.3). By analogy with the 
motion control of manipulator arms, this boils down 
to using a model with velocity control inputs, rather 
than a model with torque control inputs. This is jus¬ 
tified when dynamic effects are not preponderant and 
low-level velocity control loops on the wheels motor are 
powerful enough to ensure good velocity tracking. In 
Sect. 49.5, however, when nonideal wheel-ground con¬ 
tact is addressed, dynamics is considered. 

49.2.2 Modeling in an Absolute Frame 

For the unicycle-type mobile robot, the kinematic 
model (49.3) used from now on is 

x = Ml cos 9 , 

< y — u\ sin 9 , (49.4) 

9 =U2 , 

where (x, y) represents the coordinates of the point P m 
located at mid-distance of the actuated wheels, and the 
angle 9 characterizes the robot’s chassis orientation 
(Fig. 49.4). In this equation, u\ represents the intensity 
of the vehicle’s longitudinal velocity, and «2 is the chas¬ 
sis instantaneous velocity of rotation. The variables u\ 
and «2 are themselves related to the angular velocity of 
the actuated wheels via the one-to-one relations 
r ■ 

«1 = + TO , 

r 

U2 = 'A/ ) . 

with r the wheels’ radius, R the distance between the 
two actuated wheels, and 1 jr, (resp. \j/i) the angular ve¬ 
locity of the right (resp. left) rear wheel. 




Modeling and Control of Wheeled Mobile Robots I 49.2 Control Models 1239 



Fig. 49.4 Configuration variables 



Fig. 49.5 Representation in a Frenet frame 


For the car-like mobile robot, the kinematic model 
(49.3) used from now on is 

x = u.\ cos 9 , 
v = u\ sin (9 , 

< (49.5) 

9 = j; tan <p , 

tp = U 2 , 

where (p represents the vehicle’s steering wheel angle, 
and L is the distance between the rear and front wheels’ 
axles. In all forthcoming simulations, L is set equal to 
1.2 m. 

49.2.3 Modeling in a Frenet Frame 

The object of this subsection is to generalize the pre¬ 
vious kinematic equations when the reference frame is 
a Frenet frame. This generalization will be used later on 
when addressing the path following problem. 

Let us consider a curve C in the plane of motion, as 
illustrated on Fig. 49.5, and let us define three frames 
Jo, J m , and J s , as follows. J 0 = {0,ij} is a fixed 
frame, J m = {P m , i m j m } is a frame attached to the mo¬ 
bile robot with its origin - the point P m - located on the 
rear wheels axle, at the mid-distance of the wheels, and 
Js = {Ps, is Js}, which is indexed by the curve’s curvi¬ 
linear abscissa s, is such that the unit vector i s tangents 
C. Consider now a point P attached to the robot chassis, 
and let (4, I 2 ) denote the coordinates of P expressed in 
the basis of J m . To determine the equations of motion 
of P with respect to the curve C let us introduce three 
variables s, d, and 9 e , defined as follows: 

• s is the curvilinear abscissa at the point P s obtained 
by projecting P orthogonally on C. This point exists 
and is unique if the point P is close enough to the 
curve. More precisely, it suffices that the distance 
between P and the curve be smaller than the lower 
bound of the curve radii. We will assume that this 
condition is satisfied. 

• d is the ordinate of P in the frame J s ; its absolute 
value is also the distance between P and the curve. 

• 9 e = 9 — 9 S is the angle characterizing the orien¬ 
tation of the robot chassis with respect to the 
frame J s . 

We also define the curvature c(s) of C at P s , i.e., 
c(j) = d9 s /ds. 


Using these notations one easily deduces from 

(49.4) the following system of equations ([49.2] for de¬ 
tails), which generalizes (49.4) 

s = i_j c (s) [(“1 - hrn) cos 9 t - 1\U 2 sin 0 e ] , 

■ d = (u 1 — I 2 U 2 ) sin 9 t + J\U 2 cos 9 t , 

9 e = u 2 -sc(s). 

(49.6) 

For car-like vehicles, one easily verifies, by using 

(49.5) , that the system (49.6) becomes 

"s = j_%. (i0 [cos 9 e - ^-{l 2 cos 6» e + h sin 0 e )] , 
d = u\ [sin$ e + cos 9 t — l 2 sin (9 e )] , 

9 e = ^ tan <p — sc(s) , 

<p = u 2 . 


(49.7) 
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49.3 Adaptation of Control Methods for Holonomic Systems 


We address in this section the problems of trajectory 
stabilization and path following. When we defined these 
problems in the introduction, we considered a reference 
point P attached to the robot chassis. It turns out that 
the choice of this point is important. Indeed, consider 
for instance the equations (49.6) for a unicycle point P 
when C is the axis (O,i). Then, s = xp, d = yp, and 
9 e = 9 represent the robot’s posture with respect to the 
fixed reference frame J'o- There are two possible cases 
depending on whether P is, or is not, located on the ac¬ 
tuated wheels axle. Let us consider the first case, for 
which l\ = 0. From the first two equations of (49.6), 
one has 

xp = (u 1 — I2U2) cos 8 , yp = (u\ — I2U2) sin 9 . 

These relations indicate that P can move only in the 
direction of the vector (cos 9, sin 9). This is a direct 
consequence of the nonholonomy constraint to which 
the vehicle is subjected. Now, if P is not located on the 
wheels axle, then 

( xp\ _ /cos 8 — /isin 0 \/l —l 2 \(ui\ 

Vp J ( sin 8 l\ cos 9 J yO I J (m 2 / 

(49.8) 

The fact that the two square matrices in the right-hand 
side of this equality are invertible indicates that xp 
and yp can take any values, and thus that the motion of 
P is not constrained. By analogy with holonomic ma¬ 
nipulator arms, this means that P may be seen as the 
extremity of a two-degree-of-freedom (2-DOF) manip¬ 
ulator, and thus that it can be controlled by applying 
the same control laws as those used for manipulators. 
In this section, we assume that the point P, used to char¬ 
acterize the robot’s position, is chosen away from the 
rear wheels axle. In this case we will see that the prob¬ 
lems of trajectory stabilization and path following can 
be solved very simply. However, as shown in the subse¬ 
quent section, choosing P on the wheels axle may also 
be of interest in order to better control the vehicle’s ori¬ 
entation. 

49.3.1 Stabilization of Trajectories 
for a Nonconstrained Point 

Unicycle 

Consider a differentiable reference trajectory 1 1 —> 
(x r (f),y r (f)) in the plane. Let e = (xp — x r ,yp — y r ) de¬ 
note the tracking error in position. The control objective 
is to asymptotically stabilize this error at zero. In view 


of (49.8), the error equations are 

( cos 8 —l\ sin 9\ / u\ — / 2 n 2 \ (xA 

sin# l\cos9 J\ 112 ) \y'r) 

(49.9) 

Introducing new control variables ( v \, V 2 ) defined by 

/.A/cos e —l\ sin ^A /At — Z 2 mA 

\ V 2 J \sm(j l\ cost? J \ U 2 J 
the equations (49.9) become simply 

*-©-(*)• 

The classical techniques of stabilization for linear sys¬ 
tems can then be used. For instance, one may consider 
a proportional feedback control with precompensation 
such as 

V\ = x r — k\ e\ =x t — k\ (xp — x r ) , (k\ > 0) , 

V 2 =y r ~ k 2 e 2 =>>- k 2 (yp ~ y r ), (k 2 > 0), 

which yields the closed-loop equation e = —Ke. Of 
course, this control can be rewritten for the initial con¬ 
trol variables u, since the mapping (u\,U 2 ) i—> (vi,vi) 
is bijective. 

Car 

This technique extends to car-like vehicles (and also to 
trailer systems) by choosing a point P attached to the 
steering wheel frame and not located on the steering 
wheel axle. 

49.3.2 Path Following 

with no Orientation Control 

Unicycle 

Let us adopt the notation of Fig. 49.5 to address the 
problem of following a path associated with a curve C 
in the plane. The control objective is to stabilize the dis¬ 
tance d at zero. From (49.6), one has 

d = u i sin0 e + U 2 (—l 2 sin0 e + l\ cos 0 e ) . (49.11) 

Recall that in this case the vehicle’s longitudinal veloc¬ 
ity Mi is either imposed or prespecified. We will assume 
that the product l\U\ is positive, i. e., the position of the 
point P with respect to the actuated wheels axle is cho¬ 
sen in relation to the sign of u\. This assumption will 
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be removed in Sect. 49.4. To simplify, we will also as¬ 
sume that l 2 = 0, i. e., the point P is located on the axis 
(P m , ( m ). Let us then consider the following feedback 
control law 


u 2 = 


u\ tan@ e 

h 


u 1 

cos 6. 


k(d, e t )d , 


(49.12) 


B. A reference point P of coordinates (l \, 0) in the ve¬ 
hicle’s chassis frame, with l\u\ > 0. 

Let k denote a continuous function, strictly posi¬ 
tive onlx (—7r/2, jt/2), and such that k(d, ±n/2) = 0 
for every d (for instance, k(d, 6 e ) = kg cos 9 e ). Then, for 
any initial conditions (s(0), d(0 ), (9 e (0)) such that 


with k a continuous, strictly positive, function on 
R x (—7r/2, jr/2) such that k(d, ±jr/2) = 0. Since l 2 = 
0, applying the control (49.12) to (49.11) gives 


d = — l\u\k(d , 6 t )d , 


which suggests that the control law (49.12) ensures con¬ 
vergence of d to zero under adequate conditions on 
l \, ni, and k. This is made precise in the following result 
([49.2] for details). 


6U0)e (-tt/ 2, jr/2) , 


i\ Lnax 

1 - |c/(0)|c max 


< 1 , 


with c max = max s |c(j)|, the feedback control (49.12) 
makes the distance \d\ between P and the curve non¬ 
increasing, and makes it converge to zero if 


/ M ,)|d ,—»+00 when?—s- +00 . 
0 


Proposition k9.1 

Consider the path following problem for a unicycle- 
type mobile robot with 

A. A strictly positive, or strictly negative, longitudinal 
velocity u\. 


Car 

This control technique also applies to car (or trailer sys¬ 
tems) by considering a point P attached to the steering 
wheel frame, with u \ positive. 


49.4 Methods Specific to Nonholonomic Systems 


The control methods recalled in the previous section are 
simple and sufficient for many applications. Their main 
limitation comes from the fact that the robot’s orienta¬ 
tion is not actively controlled. This becomes a problem 
when the application requires manoeuvers (i. e., when 
the vehicle’s velocity u\ has to become negative). This 
issue is especially critical for trailer systems (including 
car-like vehicles). Indeed, when the longitudinal veloc¬ 
ity is positive, the leader vehicle has a pulling action 
which tends to align the followers along the curve. In 
the other case, the leader has a pushing action which 
tends to misalign them, thus leading to collisions be- 



Fig. 49.6 Path-following instability with reverse longitu¬ 
dinal velocity 


tween the vehicles’ components (the jackknife effect, 
see Fig. 49.6 for illustration). In order to remove this 
constraint on the sign of the longitudinal velocity, the 
control has to be designed so that all orientation an¬ 
gles are actively stabilized. An indirect way to do this 
consists in choosing the point P on the actuated wheels 
axle, at the mid-distance of the wheels, for instance. In 
this case, the nonholonomy constraints intervene much 
more explicitly, and the control can no longer be ob¬ 
tained by applying the techniques used for holonomic 
manipulators. 

This section is organized as follows. First, the mod¬ 
eling equations with respect to a Frenet frame are recast 
into a canonical form called the chained form. From 
there, a solution to the path-following problem with ac¬ 
tive stabilization of the vehicle’s orientation is worked 
out. The problem of (feasible) trajectory stabilization 
is also revisited with the complementary objective of 
controlling the vehicle’s orientation. The asymptotic 
stabilization of fixed postures is then addressed. Finally, 
some comments on the limitations of the proposed con¬ 
trol strategies, in relation to the objective of asymptotic 
stabilization, serve to motivate and introduce a new con¬ 
trol approach developed in the subsequent section. 
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49.4.1 Transformation of Kinematic Models 
into the Chained Form 


We then set zi= d and Z3 = [1—t/cfs - )] tan 9 e , so that the 
above equation becomes z .2 = 14 Z 3 . Finally, we define 


The chapter dedicated to path planning shows how the 
kinematic equations of the mobile robots here con¬ 
sidered (unicycle-type, car-like, with trailers) can be 
transformed into the chain form via a change of state 
and control variables. In particular, the equations of 
a unicycle (49.4), and those of a car (49.5), can be trans¬ 
formed into a three-dimensional and a four-dimensional 
chained system, respectively. Those of a unicycle-type 
vehicle with N trailers yield a chained system of dimen¬ 
sion N + 3 when the trailers are hooked to each other in 
a specific way. As shown below, this transformation can 
be generalized to the kinematic models derived with re¬ 
spect to a Frenet frame. The result will be given only for 
the unicycle and car cases (49.6) and (49.7), but it also 
holds for trailer systems. The reference point P is now 
chosen at the mid-distance of the vehicle’s rear wheels 
(or at the mid-distance of the wheels of the last trailer, 
when trailers are involved). 

Let us start with the unicycle case. Under the as¬ 
sumption that P corresponds to the origin of y m , one 
has Zi = I 2 = 0 so that the system (49.6) simplifies to 


V2 = Z 3 


9c. 

— dc(s ) — d—s 
as 


tan f 


+ [1 — dc(i)](l + tan 2 $ e )0 e . 


The equations (49.14) are satisfied with the variables z; 
and Vi so defined. 

From this construction it is simple to verify that the 
mapping (s, d, 9 e ) 1—> z is a local change of coordinates 
defined on K 2 x (— tz/ 2, n/2) (to be more rigorous, 
one should also take the constraint \d\ < 1 /c{s) into 
account). Let us finally remark that the change of con¬ 
trol variables involves the derivative ( dc/ds ) of the 
path’s curvature (whose knowledge is thus needed for 
the calculations). In summary, we have shown that 
the change of coordinates and of control variables 
(s,d, 0 e , mi, M 2) 1— > (zt.Z2.Z3, th, V 2 ) defined by 

(zt.Z2.z3) = (L d, [1 — dc(j)] tan 9 e ) 

{Vi, V2) = (zi,z 3 ) 


Ml 


1 — dc(s) 
d = mi sin( 


■ cos Of. 


(49.13) 


0 e = m 2 - .sc(,s) . 

Let us determine a change of coordinates and con¬ 
trol variables (s, d, 0 e , u\, M2) 1 —> (zt.Z2.Z3,14, ih) al¬ 
lowing to (locally) transform (49.13) into the three- 
dimensional chained system 


zi = Vi , 

‘ Z 2 = V 1 Z 3 , (49.14) 

Z3 = V2. 


By first setting 


transforms the model (49.13) of a unicycle-type ve¬ 
hicle into a three-dimensional (3-D) chained system. 
One can similarly transform the car’s equations into 
a four-dimensional (4-D) chained system, although the 
calculations are slightly more cumbersome. More pre¬ 
cisely, the change of coordinates and control variables 
( 5 , d, 9 e .<p, Mi, m 2 ) 1 —> (z 1 .Z 2 .Z 3 .Z 4 , Vi, V 2 ) defined by 


(z1.Z2.Z3.Z4) = \s,d. [1 - rfc(s)] tan 0 e 


— c(i')[l — dc(s)] (l + 2tan 2 0 e ) 

dc 

— d — tan@ e 

ds 

+ [1 -t/c(j)] 2 


tarup 1 + tan 2 0 e 
L cos Of. 


(vu V 2 ) = (Zl,Z 4 ) 


Zl = s , V\ = 's = 
we already obtain zi = 


Ml 


■ cos 6 e 


1 — dc(s) 

Vi. This implies that 


transforms the model (49.7) of a car-like vehicle (with 
U = h = 0) into a four-dimensional chained system. 

49.4.2 Tracking of a Reference Vehicle 
with the Same Kinematics 


. , u 1 

d = Mi sin 0 e = -cos @ e [l — dc(s)] tan 9 e 

1 — dc{s) 

= 14 [1 — tZc(s)] tan 9 e . 


Let us first consider the problem of tracking, in both po¬ 
sition and orientation, a reference vehicle (Fig. 49.7). 
For simplicity, we choose P as the origin P m of the 
robot’s chassis frame J m . 
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Although the terminology is rather loose, the track¬ 
ing problem is usually associated, in the control liter¬ 
ature, with the problem of asymptotically stabilizing 
the reference trajectory. In this case, a necessary con¬ 
dition for the existence of a control solution is that 
the reference is feasible. Feasible trajectories 1 1 —> 
(x T (t),y T (t), 9 r (t)) are smooth time functions which 
are solution to the robot’s kinematic model for some 
specific control input 1 1 —>- u r (t) = («i, r (f), «2,r(X)) T > 
called the reference control. For a unicycle-type robot 
for example, this means in view of (49.4) that 


By denoting 6 t = 6 — 0 r , the orientation error between 
the frames jFm and jp r , we obtain 

X t = t<2, r ye + u l COS 9 e — Mi, r , 

< y e = —U 2 ,rX e + ui sin$ e , (49.17) 

9 e = II 2 ~ «2,r • 

To determine a control (u\ , M 2 ) which asymptotically 
stabilizes the error (x e ,y e , 9 C ) at zero, let us consider the 
following change of coordinates and control variables 


x t = U\ T cos 9 r , 

> y r = u\j sinO r , (49.15) 

Or = «2,r ■ 

In other words, feasible reference trajectories corre¬ 
spond to the motion of a reference frame J t = {Pr. h,jr} 
rigidly attached to a reference unicycle-type robot, 
with P r (alike P = P m ) located at the mid-distance of the 
actuated wheels (Fig. 49.7). From there, the problem is 
to determine a feedback control which asymptotically 
stabilizes the tracking error (x—x r ,y—y r , 9 — 9 r ) at zero, 
with (x r ,_Vr) being the coordinates of P r in Jq, and 9 r 
the oriented angles between i and t r . One can proceed 
as in the path-following case, first by establishing the 
error equations with respect to the frame jp r , then by 
transforming these equations into the chain form via 
a change of variables like the one used to transform the 
kinematic equations of a mobile robot into a chained 
system, and finally by designing stabilizing control laws 
for the transformed system. 

Expressing the tracking error in position {x — x T , 
y — y r ) with respect to the frame jp r gives the vector 
(Fig. 49.7) 


( x e ,y e , 9 e ,u l ,u 1 ) 1 —> (zi,Z2,Z3> wi, w 2 ), 


defined by 

zt = x e , 


zi=y e . 

Z 3 = tan 9 t , 

W\ = U 1 COS 9 e — £<1 s r , 
Ul ~ W2, r 

W2 ~ cos 2 9 e ' 


(49.18) 


Note that, around zero, this mapping is only defined 
when 9 e e {—n/2, jr/2). In other words, the orientation 
error between the physical robot and the reference robot 
has to be smaller than n/2. 

It is immediate to verify that, in the new variables, 
the system (49.17) can be written as 

Zl = UljZl + Wi , 

■ Zl = -M2,rZl + t'l,rZ3 + WJ 1 Z 3 . (49.19) 

Z3 = W 2 . 


fx e \ _ ( cos 9, sin 9 r \ kx — x T \ 
\y t ) \ — sin 9 r cos0 T J \y-y r J 


(49.16) 


Calculating the time derivative of this vector yields 


We remark that the last term in each of the above three 
equations corresponds to the one of a chained system. 
From here, it can be shown that the feedback control 
law 


fx e \ f M2, r Ve + Ul COS (9 - Or) - Ml,A 

\y e J \ -U 2 , r x e + Ul sin(@ - 0 r ) ) ' 



Fig. 49.7 Reference vehicle and error coordinates 


| w\ = A|tti,ij(zi T Z 2 Z 3 ) (h > 0) , 

I W 2 = -At2t<l,rZ2 - A|«l,r|Z3 , (kl,k 3 > 0) , 

(49.20) 

makes the origin of system (49.19) globally asymp¬ 
totically stable provided essentially that u i, r does not 
tend to zero as t tends to infinity. A precise stability 
statement and associated proof are provided in [49.2], 
together with complementary details about the gain tun¬ 
ing. Finally, the velocity control inputs u\ and ui can be 
deduced from w\ and wi by using the two last lines of 
(49.18). 
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Generalization to a Car-Like Vehicle 
The previous method extends to the car case. We pro¬ 
vide below the main steps of this extension. Consider 
the car’s kinematic model (49.5) and a reference ve¬ 
hicle satisfying the same kinematic equations with all 
variables indexed by the subscript r. We assume that 
there exists S G (0,7r/2) such that the reference steer¬ 
ing angle 0 r belongs to the interval [—5, <5]. By defining 
x e , y e , and 9 e as in the unicycle case, and by setting 
0 e = 0 — cp r , one obtains the following system 

/Mi r \ 

x e = ' -j- tan0 r J y e + u\ cos 9 e — u\_ t , 

/ Ml r \ 

>’ e = - ( —j~ tan 0 r J x e + u 1 sin 9 e , 

• Mj r 

9 e = — - tan 0- — tan 0 r , 

t Pe = U 2 — U2,r • 

(49.21) 

Let us introduce the following new state variables 

zi=x e . 


Z 2 = y e , 

Z 3 = tan 9 e , 


Z4 


tan 0 — cos 9 e tan 0 r 

Lcos 3 $ e 


+ k 2 y e , (k 2 > 0 ). 


We note that for any 0 r G (—n/2,n/2), the mapping 
(jc e ,y e , 0 e ,(p) 1—> z defines a diffeomorphism between 
R 2 x (—7r/2, tz/2) 2 and R 4 . Introduce now the new 
control variables 


W\ = Ml COS (7 e — Mi. r , 

/ 3 tan 0 

W 2 = z 4 = k 2 y e + - - - 2 tan 0, 

\ cos 9 e 


) sin 9 e 
Lcos 3 0 e 


“2,1 


■ + 


u 2 


Lcos 2 0 r cos 2 0 e Lcos 2 0cos 3 0 e 


(49.22) 


One shows that (ui, u 2 ) 1—> (w\, w 2 ) defines a change 
of variables for 9 e , 0, and 0 r , inside the interval 
(—7r/2, ;r/2). These changes of state and control vari¬ 
ables transform the system (49.21) into 


From here, one can deduce that the control law w\ 
and w 2 defined as 


wi = —*i |«i, r | 


W2 = -k 3 Ui tI Z3 -fct|Hl,r|Z4 , 

(49.24) 

with 2,3,4 denoting positive numbers, makes the ori¬ 
gin of system (49.23) globally asymptotically stable 
provided essentially that Mi, r does not tend to zero as t 
tends to infinity. 

Simulation Results 

The simulation shown on Fig. 49.8 and l<£tHUltll 
illustrates this control scheme. The gain parameters k t 
have been chosen as (Ay,£2,£3,£4) = (1,1,3,3). The 
initial configuration of the reference vehicle (i. e., at 
t = 0), which is represented in Fig. 49.8a by dashed 
lines, is (jc r ,y r , 0 r )(O) = (0,0, 0). The reference con¬ 
trol M r is defined by (49.25). The initial configuration 
of the controlled robot, represented in the figure by 
plain lines, is (x,y, 9)(0) = (0, —1.5, 0). The config¬ 
urations at time t = 10, 20, and 30, are also rep¬ 
resented on the figure. Due to the fast convergence 
of the tracking error to zero (see the time evolution 
of the components x e ,y e ,9 e of the tracking error in 
Fig. 49.8b), one can basically consider that the con¬ 
figurations of both vehicles coincide after time t = 
10 . 


x! = 1 + l 


Z4 + (1 + Z3) 


2 tan0 r 


(1,0) T , iff g [0 ,10], 


u,(t) = 


{— 1,0.5 cos[2?r(f— 10)/5 ]} t , 
iff g [10 ,20] , 


(1,0) T , iff g [20, 30], 


(49.25) 


49.4.3 Path Following 

with Orientation Control 


. / U\ r \ 

Z 1 = —j~ tan 0 r j z 2 + w\ , 

/Mir \ 

Z 2 = - y-j- tan0 r J Z\ + Mi, r Z 3 + W 1 Z 3 ■ 
Zj = -k 2 U\sZ2 + Mi, r Z4 


+ W\ 


Z4-k 2 Z2+ {I+Z 3 ) 


t tan 0 r 


Z4 = W2 ■ 


(49.23) 


We reconsider the path-following problem with the ref¬ 
erence point P now located on the actuated wheels axle, 
at the mid-distance of the wheels. The objective is to 
synthesize a control law which allows the vehicle to fol¬ 
low the path in a stable manner, independently of the 
sign of the longitudinal velocity. 

Unicycle Case 

We have seen in Sect. 49.4.1 how to transform kine¬ 
matic equations with respect to a Frenet frame into the 
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three-dimensional chained system 


the controlled system provided that 


zi = th, 

< Z 2 = W1Z3 , 


( 49 . 26 ) 


4(0) + 


ki-t 


4(0) < 1 - • 


Z3 = V2. 

Recall that (zi, Z 2 ,Z 3 ) = (s,d, (1 — dc(s)) tan 9 e ) and 
that v\ = iii/(l — dc{s)) cos 6 e . The objective is to de¬ 
termine a control law which asymptotically stabilizes 
(d = 0 ,6 e = 0) and also ensures that the constraint on 
the distance d to the path (i. e., |rfc(j)| < 1) is satisfied 
along the trajectories of the controlled system. For the 
control law, a first possibility consists in considering 
a proportional feedback like 

v 2 = -v\k 2 Z 2 -\v\\k 3 Z3 , (* 2 ,* 3 > 0 ). ( 49 . 27 ) 


Generalization 

One of the assets of this type of approach, besides 
the simplicity of the control law and little demanding 
conditions of stability associated with it, is that it can 
be generalized in a straightforward manner to car-like 
vehicles and unicycle-type vehicles with trailers. The 
result is summarized hereafter by considering an n- 
dimensional chained system 

Zl = Vi, 

Z2 = VlZ3 , 


It is straightforward to verify that the origin of the 
closed-loop subsystem 


Z 2 = V 1 Z 3 , 

Z 3 = -Vik 2 Z2~\v\\k3Z3 


( 49 . 28 ) 


is asymptotically stable when V\ is constant, either pos¬ 
itive or negative. Since u\ (not 14 ) is the intensity of 
the vehicle’s longitudinal velocity, one would rather 
establish stability conditions which depend on u\. As 
detailed in [49.2], the control law (49.27) in fact makes 
the origin of system (49.28) asymptotically stable un¬ 
der the main condition that u\ does not tend to zero a t 
tends to infinity. Note that the sign of u.\ now does not 
need to be constant. As for the constraint |dc(j)| < 1, it 
is satisfied along any solution to the controlled system 
provided that 


4(0) + 7 ^ 3 ( 0 ) < 

h 


1 


c 


2 

max 


with c max = max s |c(s)|. 

From a practical point of view it can be useful to 
complement the control action with an integral term. 
More precisely, let us define a variable zo by zo = 14 Z2 
with zo(0) = 0. The control (49.27) can be modified as 
follows 


V 2 = -\vi\k 0 Zo ~ Vik 2 Z 2 -\v\\k3Z3 


I 

■/ 


= -| v l |Ao / V\Z 2 — Vik 2 Z 2 - | Vi \k 3 Z3 


(* 0 , *2. *3 > 0, *0 < * 2 * 3 ) 


( 49 . 29 ) 


( 49 . 30 ) 


Zb—1 = Vl Zn , 


Zn = V2, 


a) y 



-3 


■ -| 

-2 0 2 4 6 8 10 12 


b) (x e , y e , 6 e ) 



leading to similar stability results as before. The con- Fig.49.8a,b Tracking of a reference vehicle, (a) Cartesian motion; 
straint |<7c(.j)| < 1 is now satisfied along any solution to (b) error coordinates versus time 
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with n > 3. Its proof is a direct extension of the one in 
the three-dimensional case. The dimension n = 4 corre¬ 
sponds to the car case (Sect. 49.4.1). As for a unicycle- 
type vehicle with N trailers, one has n = N + 3. Recall 
also that, in all cases, zz represents the distance d 
between the path and the point P located at the mid¬ 
distance of the rear wheels of the last vehicle. The 
control objective is then to ensure the convergence of 
Z 2 . z n to zero. A solution to this problem, which ex¬ 

tends the solution (49.27) proposed for the unicycle, is 
given by 


V 2 = -v\ sign(^i) n+1 l kiZi . (49.31) 

1=2 

where the control gains kz ,..., k n are chosen such that 
the polynomial 

s n 1 + k n s" 2 + k n —\s n 2 + ... + k^s + kz 

is Hurwitz stable. Satisfaction of the constraint 
|r/c(s)| < 1 along any solution to the controlled sys¬ 
tem can also be guaranteed, upon a condition on 
( 22 ( 0 ), • • • ,z„(0)) specified in [49.2]. The possibility of 
adding an integral correction term in (49.31) is also ad¬ 
dressed in [49.2], 

Simulation Results 

The simulation results reported in Fig. 49.9 and 
laaiM'il'lt'lliB illustrate how this control scheme per¬ 
forms for a car-like vehicle. The reference curve is the 
circle of radius equal to four, centered at the origin. The 
robot’s longitudinal velocity u\ is defined by u\ = 1 
for t e [0, 5], and u\ — — 1 for t > 5. The control gains 
have been chosen as (& 2 ,£ 3 ,A 4 ) = (1,3,3). The mo¬ 
tion of the car-like robot in the plane is represented in 
Fig. 49.9a, and its configuration at times t = 0, 5, and 25 
are also depicted in the figure. The time evolution of the 
variables zz,Zz,Z 4 (defined at the end of Sect. 49.4.1) 
is represented in Fig. 49.9b. One can observe that the 
(discontinuous) change of the longitudinal velocity u\ 
at t = 5 does not affect the convergence of these vari¬ 
ables to zero. 

49.4.4 Asymptotic Stabilization 
of Fixed Postures 

We now consider the problem of asymptotic stabiliza¬ 
tion of a fixed desired (reference) posture (i. e., position 
and orientation) for the robot chassis. This problem 
may be seen as a limit case of the trajectory track¬ 
ing problem. Flowever, none of the feedback controllers 
proposed previously in this chapter provides a solu¬ 
tion to this problem. From the automatic control point 


of view, the asymptotic stabilization of fixed postures 
is very different from the problems of path following 
and trajectory tracking with nonzero longitudinal ve¬ 
locity, much in the same way as a human driver knows, 
from experience, that parking a car at a precise loca¬ 
tion involves techniques and skills different from those 
exercised when cruising on a road. In particular, it can¬ 
not be solved by any classical control method for linear 
systems (or based on linearization). Technically, the 
underlying general problem is the one of asymptotic 
stabilization of equilibria of controllable driftless sys¬ 
tems with less control inputs than state variables. This 
problem has motivated numerous studies during the last 
decade of the last century, from many authors and with 
various angles of attack, and it has remained a sub¬ 
ject of active research at the beginning of this century. 



Fig.49.9a,b Path following along a circle, (a) Cartesian 
motion; (b) coordinates Z2.3.4 versus time 
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The variety of candidate solutions proposed until now, 
the mathematical technicalities associated with several 
of them, together with unsolved difficulties and limita¬ 
tions, particularly (but not only) in terms of robustness 
(an issue on which we will return), prevent us from 
attempting to cover the subject exhaustively. Instead, 
we have opted for a somewhat informal exposition of 
approaches which have been considered, with the illus¬ 
tration of a few control solutions, without going into 
technical and mathematical details. 

A central aspect of the problem, which triggered 
much of the subsequent research on the control of non¬ 
holonomic systems, is that asymptotic stabilization of 
equilibria (or fixed points) cannot be achieved by using 
continuous feedbacks which depend on the state only 
(i. e., continuous pure-state feedbacks). This is a conse¬ 
quence of an important result due to Brockett in 1983. 
The original result by Brockett concerned differentiable 
feedbacks; it has later been extended to the larger set of 
feedbacks which are only continuous. 

Theorem 49.7 (Brockett [U9.3]) 

Consider a control system 

x=f(x,u ) 

with / a differentiable function and (x,u) = ( 0 , 0 ) an 
equilibrium of this system. A necessary condition for 
the existence of a continuous feedback control u(x) 
which renders the origin of the closed-loop system 

x = f(x, u{x)) 

asymptotically stable is the local surjectivity of the ap¬ 
plication (x, u) 1 — >f(x,u). More precisely, the image 
by/ of any neighborhood 12 of ( 0 , 0 ) in H' ! + m must be 
a neighborhood of 0 in K". 


This result implies that the equilibria of many con¬ 
trollable (nonlinear) systems are not asymptotically 
stabilizable by continuous pure-state feedbacks. All 
nonholonomic WMRs belong to this category of sys¬ 
tems. This will be shown in the case of a unicycle-type 
vehicle; the proof for the other mobile robots is simi¬ 
lar. Let us thus consider a unicycle-type vehicle, whose 
kinematic equations (49.4) can be written asir = f(x, u) 
with x = (x\,X 2 ,Xt,) t , ii = (ui,U 2 ) t , and f(x,u) = 
(u\ COSX 3 , «i sinjf 3 , M 2 ) T , and let us show that/ is not 
locally onto in the neighborhood of (x,u) = (0,0). To 
this purpose, take a vector in R 3 of the form (0, 8 , 0) T . It 
is obvious that the equation f(x, u) = ( 0 ,8, 0 ) T does not 
have a solution in the neighborhood of (x,u) = ( 0 , 0 ) 
since the first equation, namely mi cos *3 = 0 , implies 
that mi = 0 , so that the second equation cannot have 
a solution if 8 is different from zero. 


It is also obvious that the linear approximation 
about the equilibrium ( x , u) = ( 0 , 0 ) of the unicycle 
kinematic equations is not controllable. If it were, it 
would be possible to (locally) asymptotically stabilize 
this equilibrium with a linear (thus continuous) state 
feedback. 

Therefore, by application of the above theorem, 
a unicycle-type mobile robot (like other nonholonomic 
robots) cannot be asymptotically stabilized at a desired 
posture (position/orientation) by using a continuous 
pure-state feedback. This impossibility has motivated 
the development of other control strategies in order to 
solve the problem. Three major types of controls have 
been considered: 

1. Continuous time-varying feedbacks, which, besides 
from depending on the state x, depend also on the 
exogenous time variable (i. e., u(x. t ) instead of u(x) 
for classical feedbacks). 

2. Discontinuous feedbacks, in the classical form u(x), 
except that the function u is not continuous at the 
equilibrium that one wishes to stabilize. 

3. Hybrid discrete/continuous feedbacks. Although 
this class of feedbacks is not defined as precisely as 
the other two sets of controls, it is mostly composed 
of time-varying feedbacks, either continuous or dis¬ 
continuous, such that the part of the control which 
depends upon the state is only updated periodically, 
e.g., u(t) = u[x(kT),t] for any t e [kT, (k+ 1)7], 
with T denoting a constant period, and k e N. 

We focus hereafter on continuous time-varying feed¬ 
backs. Examples of hybrid discrete/continuous feed¬ 
backs for unicycle and car-like vehicles are provided in 
[49.2, Sect 34.4.4], together with associated simulation 
results. As for discontinuous feedbacks, they involve 
difficult questions (existence of solutions, mathemati¬ 
cal meaning of these solutions, etc.) which complicate 
their analysis and for which complete answers are not 
available. Moreover, for most of the discontinuous con¬ 
trol strategies described in the literature, the property of 
stability in the sense of Lyapunov is either not granted 
or remains an open issue. 

The use of time-varying feedbacks for the asymp¬ 
totic stabilization of a fixed desired equilibrium, for 
a nonholonomic WMR, in order to circumvent the 
obstruction pointed out by Brockett’s Theorem, was 
first proposed in [49.4], Since then, very general re¬ 
sults about the stabilization of nonlinear systems by 
means of time-varying feedbacks have been obtained. 
For instance, it has been proved that any controllable 
driftless system can have any of its equilibria asymp¬ 
totically stabilized with a control of this type [49.5]. 
This includes the kinematic models of the nonholo- 
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nomic mobile robots here considered. We will illustrate 
this approach in the case of unicycle-type and car-like 
mobile robots modeled by three- and four-dimensional 
chained systems, respectively. In order to consider the 
three-dimensional case, let us come back on the results 
obtained in Sect. 49.4.3 for path following. The control 
law (49.27), i. e., v 2 = —Vik 2 z 2 — \ v\ \k 3 Z 3 applied to the 
system 

zt = Vi , 

* Z 2 = V 1 Z 3 , 

Z3 = V 2 , 

renders the function 

v (z):=l(zl+l4) 

nonincreasing along any trajectory of the controlled 
system, i. e., 

and ensures the convergence of Z 2 and Z 3 to zero if, for 
instance, 14 does not tend to zero as t tends to infinity. 
For example, if v\ ( t ) = sin t, Z 2 and Z 3 tend to zero, and 

t t 

zi (t) = z.\ ( 0 ) + / Vi (j)d.s = z\ ( 0 ) + J sin id.s 
0 0 

= zi( 0 ) + 1 — cost, 

so that zi ( t ) oscillates around the mean value zi ( 0 ) + 
1. To reduce these oscillations, one can multiply v\ by 
a factor which depends on the current state. Take, for 
example, 14 (z, t) = || (z 2 » Z3) || sinr, that we complement 
with a stabilizing term like —k\z.\ with £4 > 0 , i. e., 

Vi (z, f) = -hzi + life, £3) II sinf. 

The feedback control so obtained is time-varying and 
asymptotically stabilizing. 

Proposition U9.2 

The continuous time-varying feedback 

(wife?) = -kizi T or|| fe, z 3 )|| sint, ^ 

l v 2 fc t) = -V 1 (z, t)k 2 z 2 - | Vi (z, t)\k 3 Z3 , 

with a , £ 4 , 2,3 > 0, renders the origin of the 3-D chained 
system globally asymptotically stable [49.6]. 


The above proposition can be extended to chained sys¬ 
tems of arbitrary dimension [49.6]. For the case n = 4, 
which corresponds to the car-like robot, one has the fol¬ 
lowing result. 


Proposition k9.3 

The continuous time-varying feedback 

Vi (z, t) = —k\zi + ff || fe, z 3 , z 4 ) || sin t , 

‘ V 2 (Z, t) = -1 Vi (z, t) | k 2 Z 2 ~Vi (z, t)k3Z3 

\vi (z, t)\k 4 z4 , 

(49.33) 

with a, £4,2,3,4 > 0 chosen such that the polynomial 
.s 3 + k 4 s 2 + k 2 s + k 2 is Hurwitz stable, renders the ori¬ 
gin of the 4-D chained system globally asymptotically 
stable [49.6]. 


Figure 49.10 below illustrates the previous re¬ 
sult. For this simulation, the parameters o;,£ 4 . 2 , 3,4 in 
the feedback law (49.33) have been chosen as a = 3 
andki 2 , 3,4 = (1-2, 10,18,17). Figure 49. 10a shows the 
motion of the car-like robot in the plane. The initial 
configuration, at time t = 0 , is depicted in plain lines, 
whereas the desired configuration is shown in dashed 
lines. The time evolution of the variables x, y, and 6 
(i. e., the position and orientation variables correspond¬ 
ing to the model (49.5)) is shown in Fig. 49.10b. 

A shortcoming of this type of control, very clear 
from this simulation, is that the system’s state con¬ 
verges to zero quite slowly. One can show that the 
rate of convergence is only polynomial, i. e., it is com¬ 
mensurable with t~ 01 (for some a e ( 0 , 1 )) for most 
of the trajectories of the controlled system. This slow 
rate of convergence is related to the fact that the con¬ 
trol function is Lipschitz-continuous with respect to x. 
It is a characteristics of systems the linear approxi¬ 
mation of which is not stabilizable. Indeed, it can be 
shown that exponential stability (in the classical sense 
of linear systems) of an equilibrium point of a nonlinear 
system cannot be obtained with a Lipschitz-continuous 
feedback when the system’s linear approximation at 
that point is not stabilizable ([49.2, Prop. 34.11] for 
a precise statement and details). The need of better per¬ 
formance and efficiency, has triggered the development 
of stabilizing time-varying feedbacks which are contin¬ 
uous, but not Lipschitz-continuous. Examples of such 
feedbacks, yielding uniform exponential convergence, 
are given in the following propositions for chained sys¬ 
tems of dimension three and four, respectively. 


Proposition 49.4 

Let a , £4,2,3 > 0 denote scalars such that the polynomial 
p(s ) = s 2 + £' 3 $ + £2 is Hurwitz stable. For any integers 
p, q G N*, let Pp_ q denote the function defined on R 2 by 

Vz 2 = fe,Z3) G R 2 , Pp.qfa) 

I 

= (|Z2| FFT + \Z3\~ q y ■ 
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Then, there exists qo > 1 such that, for any q>qo and 
p > q + 2, the continuous state feedback 

v\ (z, t) = —ki(zi sinf— |zi |) sinf 


the ratios 


22 

Pp.q^l) 


and 


23 

2 ) 


, +<XPp,q(z 2 ) sinf 
v 2 (z, t ) = -Vi(.z , t)k 2 

f$,q(Zl) 


I *4 (2,01*3 


23 

Pp,q(.Z 2 ) 


(49.34) 


renders the origin of the three-dimensional chained 
system globally asymptotically stable, with a uniform 
exponential rate of convergence [49.7]. 


The parenthood of the controls (49.32) and (49.34) is 
noticeable. One can also verify that the control (49.34) 
is well defined (by continuity) at z 2 = 0. More precisely. 



x 


b) (x,y,e) 



Fig.49.10a,b Asymptotic stabilization with a Lipschitz- 
continuous controller, (a) Cartesian motion; (b) error co¬ 
ordinates versus time 


which are obviously well defined when z 2 0, tend to 
zero when z 2 tends to zero. This guarantees the conti¬ 
nuity of the control law. 

The property of exponential convergence pointed 
out in the above result calls for some remarks. Indeed, 
this property does not exactly correspond to the clas¬ 
sical exponential convergence property associated with 
stable linear systems. In this latter case, exponential 
convergence means that for some constant K, y, and 
along any solution to the controlled system, one has 

\\z(t)\\<K\\zme-^-^ 

This corresponds to the common notion of exponential 
stability. In the present case, this inequality becomes 

p[z(0] < ^p[z(fo)]e _y<f_ ' 0> 

for some function p, defined for example by 

P(z) = |zi| + Pp,q(Z2,Zi) , 

with as specified in Proposition 49.4. Although the 
function p shares common features with the Euclidean 
norm of the state vector (it is definite positive and it 
tends to infinity when ||z|| tends to infinity), it is not 
equivalent to this norm. Of course, this does not change 
the fact that each component z; of z converges to zero 
exponentially. However, the transient behavior is differ¬ 
ent because one only has 

\zm<K\\z{to)re- v(t -^ , 

with a < 1, instead of 

M 0 l<K||zMk- y(r - ,o) . 

In the case of the four-dimensional chained system, 
one can establish the following result, which is similar 
to Proposition 49.4. 


Proposition U9.5 

Let a,k\,k 2 ,k 2 ,k 4 > 0 be chosen such that the polyno¬ 
mial p(s) = s 3 + & 4 J 2 + k 2 s + k 2 is Hurwitz stable. For 
any integers p, q e N*, let Pp q denote the function de¬ 
fined on R 3 by 

1 

Pp.qiz 2 ) = + \Z4\<y 

with z 2 = ( 22 . 23 . 24 ) e R 3 - Then, there exists <70 > 1 
such that, for any q>qo and p > q + 2, the continuous 
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state feedback 


Vi (. z , t) = —k\ (zi sin t - |zi |) sin t 
+oip P ,q(z2) sin t, 

' V2(z,t) = -\vi{zJ)\k 2 —^—- -Vi(z,t)h 

Pp, q (z 2) 


£3 

Pp.qih) 


-\vi(z, t)\k 4 


Z4 

Pp,q(Z.2) 


( 49 . 35 ) 


renders the origin of the 4-D chained system globally 
asymptotically stable, with a uniform exponential rate 
of convergence [49.7]. 


The performance of the control law (49.35) is illustrated 
by the simulation results shown in Fig. 49.11. The con¬ 
trol parameters have been chosen as follows: a = 0.6, 
fci. 2 , 3,4 = (1.6,10. 18,17), q = 2, p = 5. The compar¬ 
ison with the simulation results of Fig. 49.10 shows 
a clear gain in performance. 

49.4.5 Limitations Inherent to the Control 
of Nonholonomic Systems 


Let us first mention some problems associated with 
the nonlinear time-varying feedbacks just presented. An 
ever important issue, when studying feedback control, 
is robustness. Indeed, if it were not for the sake of ro¬ 
bustness, feedback control would lose much of its value 
and interest with respect to open-loop control solutions. 
There are various robustness issues. One of them con¬ 
cerns the sensitivity to modeling errors. For instance, in 
the case of a unicycle-type robot whose kinematic equa¬ 
tions are in the formi = ii\b\(x) + W 2 & 2 M, one would 
like to know whether a feedback law which stabilizes an 
equilibrium of this system also stabilizes this equilib¬ 
rium for the neighbor system x = u\ [b\ (jc) + eg\ (x)] + 
U 2 [b 2 (x) + eg 2 (x)\, with g\ and g 2 denoting continu¬ 
ous applications, and e a parameter which quantifies 
the modeling error. This type of error can account, for 
example, for a small uncertainty concerning the orien¬ 
tation of the actuated wheels axle with respect to the 
chassis, which results in a bias in the measurement of 
this orientation. One can show that time-varying control 
laws like (49.34) are not robust with respect to this type 
of error in the sense that, for certain functions g 1 and g 2 , 
and for e arbitrarily small, the system’s solutions end up 
oscillating in the neighborhood of the origin, instead of 
converging to the origin. In other words, both the prop¬ 
erties of stability of the origin and of convergence to 
this point can be jeopardized by arbitrarily small mod¬ 
eling errors, even in the absence of measurement noise. 
In view of these problems, one is brought to question 


the existence of fast (exponential) stabilizers endowed 
with robustness properties similar to those of stabiliz¬ 
ing linear feedbacks for linear systems. The answer is 
that, to our knowledge, no such control solution (ei¬ 
ther continuous or discontinuous) has ever been found. 
More than likely such a solution does not exist for non¬ 
holonomic systems. Robustness of the stability property 
against modeling errors, and control discretization and 
delays, has been proved in some cases, but this could 
only be achieved with Lipschitz-continuous feedbacks 
which, as we have seen, yield slow convergence. The 
classical compromise between robustness and perfor¬ 
mance thus seems much more acute than in the case of 
stabilizable linear systems (or nonlinear systems whose 
linear approximation is stabilizable). 

A second issue is the proven nonexistence of a uni¬ 
versal feedback controller capable of stabilizing any 
feasible reference state trajectory asymptotically [49.8]. 



x 



t 


Fig.49.11a,b Asymptotic stabilization with a continuous 
(non-Lipschitz) time-varying feedback, (a) Cartesian mo¬ 
tion; (b) error coordinates versus time 
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This is another notable difference with the linear case. 
Indeed, given a controllable linear system x = Ax + 
Bh, the feedback controller u = u r + K(x — x r ), with K 
a gain matrix such that A + BK is Hurwitz stable, expo¬ 
nentially stabilizes any feasible reference trajectory x r 
(solution to the system) associated with the control in¬ 
put u T . The nonexistence of such a controller, in the 
case of nonholonomic mobile robots, is related to the 
conditions upon the longitudinal velocity stated in pre¬ 
vious propositions concerning trajectory stabilization. 
This basically indicates that such conditions cannot be 
removed entirely: whatever the chosen feedback con¬ 
troller, there always exists a feasible reference trajec¬ 
tory that this feedback cannot asymptotically stabilize. 
Note that this limitation persists when considering non¬ 
standard feedbacks (such as, e.g., time-varying periodic 
feedbacks capable of asymptotically stabilizing refer¬ 
ence trajectories which are reduced to a single point). 
Moreover, it has clear practical consequences because 
there are applications (automatic tracking of a human- 
driven car, for instance) for which the reference trajec¬ 
tory, and thus its properties, are not known in advance 
(is the leading car going to keep moving or stop?) so 
that one cannot easily decide on which controller to 
use. Switching between various controllers is a possi¬ 
ble strategy, which has been studied by some authors 
and may give satisfactory results in many situations. 
However, since implementing a predefined switching 
strategy between two controllers sums up to designing 
a third controller, this does not solve the core of the 
problem nor grant any certitude of success. 

A third issue, which is not specific to nonholonomic 
systems, but has seldom been addressed in the nonlin¬ 
ear control literature, concerns the problem of tracking 
nonfeasible trajectories (i. e., trajectories which are not 
solutions to the system’s equations). Since exact track¬ 
ing is not possible, by the definition of a nonfeasible 
trajectory, the control objective is then to ensure that 
the tracking errors shrink to, and ever after never ex¬ 
ceed, certain nonzero thresholds. The fact that these 
thresholds can theoretically be arbitrarily small in the 
case of nonholonomic systems, if the amplitude of the 
velocity control inputs is not limited, makes this prob¬ 
lem particularly relevant for these systems. This can 
be termed as a practical stabilization objective which, 
although slightly less ambitious than the objective of 
asymptotic stabilization considered in previous sec¬ 
tions, opens up both the control design problem and 
the range of applications significantly. For instance, it 
allows one to address the problem of tracking an om¬ 
nidirectional vehicle with a unicycle-type, or a car-like, 
vehicle. In the context of planning a trajectory with ob¬ 
stacle avoidance, transforming a nonfeasible trajectory 
into a feasible approximation for a certain mobile robot 


can be performed by applying a practical stabilizer 
to a model of this robot and by numerical integra¬ 
tion of the system’s closed-loop equations. Also, if one 
reformulates the former question about the existence 
of a universal stabilizer, with the objective of asymp¬ 
totic stabilization now replaced by the one of practical 
stabilization, then the answer becomes positive: such 
a stabilizer exists and, moreover, the reference trajec¬ 
tories do not even have to be feasible. 


49.4.6 Practical Stabilization 
of Arbitrary Trajectories 


A possible approach for the design of practical stabi¬ 
lizers in the case of controllable driftless systems is 
described in [49.9]. Some of its basic principles, here 
adapted to the specific examples of unicycle-type and 
car-like mobile robots, are recalled next. 

Let us introduce some matrix notation that will be 
used in this section. 

»'”/)• 

RW = (7 »). 

Unicycle Case 
With the above notation, the kinematic model (49.4) can 
be written as 

g = R(6)Cu , (49.36) 

with g = (x,y, 0)' and 



Let us now consider a smooth function 


(fx(a)\ 

f : a 1 — >f{a) = f y (a) 
\fd(u)J 


with aeS 1 = M/27rZ (i. e., tx is an angle variable), and 
define 


g-= 


|.vj :=g-R(0-fy(a))f(a) 



9-fg{u) 


(49.37) 
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Note that g can be viewed as the situation of approach [49.9, 10]. In the present case, a family of 
a frame J m (u) the origin of which has components transverse functions is given by 


- RW » ( “)) (*w) 

in the frame f m . In term of differential geometry, g 
is the product of g by the inverse /(a ) -1 of /(a), in 
the sense of the Lie group operations in SE(2). Hence, 
J m (a) is all the closer to as the components off (a) 
are small. For any smooth time function t 1 —> a(t), and 
along any solution to system (49.36), the time derivative 
of g is given by 


g = R(9)u , 


(49.38) 


with 


u = A (a) (r (f e (a)) - ^(a)) j ■ 


(49.39) 


m 


/ e sin a \ 
-f sin 2a 


\arctan(£ r] cos a)/ 


with e, r) > 0 . 

(49.43) 


Indeed, with this function one can verify that, for 
any a e S 1 , 


detH(a) 



cos(arctan(£?j cos a)) < 0 . 


Note that the components of / uniformly tend to zero 
as £ tends to zero, so that the associated omnidirectional 
frame can be made arbitrarily close to by 

choosing e small (but different from zero). 

Now, let 


and 


1 1—> 81(f) = MO.wM. 9 r (t)] T 


A (or) 



denote a smooth, but otherwise arbitrary, reference 
(49 40 ) trajectory. It is not difficult to derive from (49.38) 
a feedback law u which asymptotically stabilizes g at g r . 
A possible choice is given by 


From (49.38) and (49.39), one can view a as a comple¬ 
mentary control input that can be used to monitor the 
motion of the frame J m (a). More precisely, J' m (a) can 
be viewed as an omnidirectional frame provided that u 
can be rendered equal to any vector of R 3 , i. e., provided 
that the mapping ( u , a) 1 —> u is onto. Let us determine 
when this condition is satisfied. Equation (49.39) can 
also be written as 


U = A(Q!)H(Q!) 



(49.41) 


with 


U = R(—0)|£ r -fcfe-g r )] , (49.44) 

which implies that 

(g-gr) = -k(g-gi) 

and therefore that g — g t = 0 is an exponentially stable 
equilibrium of the above equation for any k > 0. Then, 
it follows from (49.37) that 

lim {g(r) - gr(t) - R[0 r (f)]fla(f)]} = 0 . 

t —» + oo 

(49.45) 


H(a) 


^cos fe(a) 0 
sin fe(a) 0 

l 0 1 


-&(«A 

- 1 («) 

-!(«)/ 


(49.42) 


Since A(a) is invertible, f m (a) is omnidirectional if 
and only if the matrix H(a) is also invertible. A func¬ 
tion / which satisfies this property for any a & S l is 
called a transverse function [49.10]. The issue of the 
existence of such functions has been treated in the 
much more general context of the transverse function 


The norm of the tracking error ||g — g r || is thus ulti¬ 
mately bounded by the norm of f{a) which, in view 
of (49.43), can be made arbitrarily small via the choice 
of £. It is in this sense that practical stabilization is 
achieved. The control u for the unicycle-like robot is 
then calculated by inverting the relation (49.41) and us¬ 
ing the expression (49.44) of u. 

While it can be tempting to use very small val¬ 
ues of £ for the transverse function / in order to 
obtain a good tracking precision, one must be aware 
of the limits of this strategy. Indeed, when e tends to 
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zero, the matrix H(a) defined by (49.42) becomes ill- 
conditioned, and its determinant tends to zero. This 
implies, by (49.41), that the robot’s velocities u\ and 
«2 may become very large. In particular, when the ref¬ 
erence trajectory g r is not feasible, many manoeuvres 
are likely to occur. Note that this difficulty is intrinsic 
to the robot’s nonholonomy and that it cannot be cir¬ 
cumvented (think about the problem of parking a car 
in a very narrow parking place). For this reason, trying 
to impose very accurate tracking of nonfeasible tra¬ 
jectories is not necessarily a good option in practice. 
On the other hand, when the trajectory is feasible, ma¬ 
noeuvres are not needed to achieve accurate tracking, 
so that smaller values of e can be used in this case. 
This clearly leads to a dilemma when the reference tra¬ 
jectory is not known in advance and its properties in 
term of feasibility can vary with time. A control strat¬ 
egy which addresses this issue, based on the use of 
transverse functions whose magnitude can be adapted 
online, is proposed in [49.11], Experimental validations 
of the present approach on a unicycle-like robot can also 
be found in [49.12]. 

Car Case 

The control approach presented above can be extended 
to car-like vehicles (and also to the trailer case). Again, 
the idea is to associate with the robot’s frame J ' m an 
omnidirectional companion frame which can be 

maintained arbitrarily close to via the choice of 
some design parameters. Let us show how this can be 
done for a car-like vehicle. To simplify the forthcoming 
equations, let us rewrite system (49.5) as 


with a G S 1 x S 1 (i. e., a = (ctq, o^)), and define (com¬ 
pare with (49.37)) 


g = 


|) j = g- R(9 -fg(a))f s (a) 



O-fd(u) 


(49.47) 


which, as in the unicycle case, can be viewed as the 
situation of some companion frame By differ¬ 

entiating g along any smooth time function 1 1 —> a{t ) 
and any solution to system (49.46), one can verify that 
(49.38) is still satisfied, except that u is now given by 


:A(«)(rIM a)]-3^(00-^(co) 


/C© Ml \ 
oq 
Oil 


(49.48) 


rather than by (49.39) (with A(a) still defined by 
(49.40)). Using the fact that 


C©m = eft (<*))«! + {C© - C[f f («)]}«! 
/ 1 \ / 0 \ 


0 


U\ + 


0 

U-/?(“)/ 


U 1 


x = u\ cos 9 , 


(49.48) can also be written as 


y = mi sin 6 , 

9 = mi£ , 
f = Mf , 

with § = (tan <p)/L and mj = ui(\ + tan 2 0)/L. This 
system can also be written as (compare with (49.36)) 


( li '\ 

u = A(a)H(a) I a q I 

w 


+ A(a) 


( 0 ) 
-/f(«)]y 


(49.49) 


with 


U = R(0)C©m 1; 

(? = «£ . 

with g = (x,y, 9 ) T and C© 
consider a smooth function 




(49.46) 


''cos fe(a) 






H(a) = 

sin/ e (a) 




(1.0© 1 

. Let us now 


K fei 01 ) 


-fe(«v 

(49.50) 

(fx(a)\ 

fy(0i) 


By setting 





fe(oi) 

Vf(“)/ 


«t =/e(« )-fc(?-/e(«) • 


(49.51) 
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with k > 0, it follows from (49.46) that £ — /g (a) expo¬ 
nentially converges to zero. Hence, after some transient 
phase whose duration is commensurable with 1 /k, f — 
/j(a) pb 0, and (49.49) reduces to 


u = A(Q')H(a) I di I 


(49.52) 


Provided that the function / is such that H(«) is 
always invertible, this latter relation means that the 
frame j7 m (Qt) associated withg is omnidirectional. Any 
function / for which this property is satisfied is called 
a transverse function. Once it has been determined, one 
can proceed as in the unicycle case to asymptotically 
stabilize an arbitrary reference trajectory g r for g, for 
example, by defining u as in (49.44). The control ii\ for 
the car is then obtained by inverting relation (49.52). 


a) y 




*0 10 20 30 40 50 60 70 80 

t 

Fig.49.12a ,b Practical stabilization of an arbitrary trajec¬ 
tory by the transverse function approach, (a) Cartesian 
motion; (b) error coordinates versus time 


The following lemma specifies a family of transverse 
functions for the car case. 


Lemma 49.7 

For any e > 0 and any 41 , 42 , 43 such that 41 , r] 2 , 43 > 0 
and 6)7243 > 843 + 4142 , the function/ defined by 


/(«) 


( f\ (a) \ 

/t(a) 

arctan(/ 3 (a)) 
\f 2 (oi)cos 3 f 3 {a) J 


with 

/: S 1 x S 1 
given by 

7 («) = 

( 


ejsinai + 42 sin(*2) 

erji cos oq 
/ 41 sin 2 a 1 


4 


■ — 43 cos Cl 2 


m- 


sin 2 a 1 cos a 1 42 43 sin 2«2 


- — 43 sinai cosof2 


satisfies the transversality condition detH(a) ^OVa, 
with H(q!) defined by (49.50) [49.13]. 


The simulation results reported in Fig. 49.12 il¬ 
lustrate the application of this control approach for 
a car-like robot. The reference trajectory is defined by 
the initial condition g r ( 0 ) = 0 and its time derivative 


( 0 , 0 , 0 ) T 

if t g 

[0, 30] 

( 1 , 0 , 0 ) T 

if t e 

[30,38] 

(0,0.3,0) T 

if t e 

[38,53] 

(— 1 , 0 , 0 ) T 

if te 

[53,61] 

( 0 , 0 , 0 . 2 ) T 

if t e 

[61,80] 


This corresponds to a fixed situation when t e [0, 30], 
three sequences of pure translational motion when t e 
[30, 61], and a pure rotational motion when t e [61,80]. 
Let us remark that this trajectory is not feasible for the 
car-like robot when t e [38, 53], since it corresponds to 
a lateral translation in the direction of the unit vec¬ 
tor j T of the frame y r associated with g T , nor when 
t e [61,80], since a rear-drive car cannot perform pure 
rotational motion. The initial configuration of the car¬ 
like robot, at t = 0, is g(0) = (0,1.5, 0), and the initial 
steering wheel angle is </>( 0 ) = 0 . 
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In Fig. 49.12a, the robot is drawn with a solid line 
at several time instants, whereas the chassis of the ref¬ 
erence vehicle is shown as a dashed line at the same 
time instants. The figure also shows the trajectory of 
the point located at the mid-distance of the robot’s 
rear wheels. Figure 49.12b shows the time evolution 
of the tracking error expressed in the reference frame 
(i. e., (jc e ,y e ) as defined by (49.16), and 9 t = 8 — 8 r ). It 
follows from (49.45) that, after the transient phase asso¬ 
ciated with the exponential convergence of g to zero, the 
ultimate bound for |jc e |, |y e |, and |0 e | is upper-bounded 
by the maximum amplitude of the functions and 
fg , respectively. For this simulation, the control param¬ 


eters of the transverse function / of Lemma 49.1 have 
been chosen as follows: £ = 0.17, > 71 , 2,3 = (12,2,20). 
With these values, one can verify that f x |, [/, |, and [fg \ 
are bounded by 0.51, 0.11, and 0.6, respectively. This is 
consistent with the time evolution of the tracking error 
observed in the figure. As pointed out for the unicycle 
case, tracking errors could be diminished by decreasing 
the value of s, but this would involve larger values of the 
control inputs and also more frequent manoeuvres, es¬ 
pecially on the time intervals [38, 53] and [61,80] when 
the reference trajectory is not feasible. Additional sim¬ 
ulation and experimental results on this approach are 
illustrated by I<e pM 7I'1 and l-g &M'il'M'KtM 


49.5 Path Following in the Case of Nonideal Wheel-Ground Contact 


The kinematic models used in the previous sections 
are derived under the classical rolling-without-sliding 
assumption for the vehicle’s wheels. This assump¬ 
tion is satisfied with a good degree of accuracy for 
many applications both indoor and outdoor (e.g., on¬ 
road). In some cases, however, sliding can be sig¬ 
nificant. This happens for example when a vehicle 
operates on natural terrain with poor grip conditions 
(grass, earth) and when the vehicle’s speed is sig¬ 
nificant and/or the terrain is not perfectly horizon¬ 
tal. In these cases, the control laws presented in the 
previous sections may not give full satisfaction. We 
show in this section that these control laws can still 
be used successfully, provided that sliding is taken 
into account at the modelling level and estimated on¬ 
line via a dedicated observer. For simplicity, only the 
path following problem is addressed, but the tech¬ 
niques here presented can be extended to other control 
problems. 



49.5.1 Extended Control Models 
in the Presence of Sliding 

Extended Kinematic Model 
Consider the two-wheels schematic representation of 
a car-like vehicle on Fig. 49.13. The angles /) R and /p 
are introduced in order to represent the sliding of the 
rear and front wheels respectively. More precisely, de¬ 
noting the centres of the rear and front wheels as P R 
and Pp respectively, /J R is the angle between the vec¬ 
tor P R Pp and the velocity vector u R of P R , whereas /3p 
represents the angle between the steering direction and 
the velocity vector Vp ofPp. The kinematic modeling of 
Sect. 49.2 is easily extended to the present case [49.14], 
resulting in the following model 

x = u\ cos (9 + /1 R ) 
y = «i sin(0 + /1 R ) 

’a ,0 , tan(/ + yS F ) - tan(/ R ) 

8 = u { cos(p R )--- 

<P = m 

(49.53) 

with it 1 the (signed) intensity of the vector tt R . Note that 
these equations reduce to (49.5) when /f = /J R = 0. 

Kinematic Model in a Frenet Frame 
In the context of path following, a kinematic model with 
reference to a Frenet frame moving along the desired 
path is obtained via a straightforward adaptation of the 
pure-rolling case discussed in Sect. 49.2.3. Defining the 
distance between the vehicle and the desired path C 
as the distance d between the point P R and this path 
(Fig. 49.5), the equations of this model are readily de- 


Fig. 49.13 Car-like vehicle in the presence of sliding 
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duced from (49.53) (compare with (49.13)) 
cos($ e + Pr) 

i = iii - 

1 — d c(s ) 

< d = Mi sin( 0 e + /Sr) , (49.54) 

6>e = Ml [cOS(/6r)Ai -A 2 ] 

</> = 112 

with 9 e the angle between the vehicle’s body 
axis PrPf and the tangent to the desired path 
evaluated at the projection of the point Pr on 
this path, c(s) the path curvature at the pro¬ 
jected point, Ai = (tan(0 + Pp) — tan(/lR ))/L and A 2 = 
(t-(.v) cos(6> e + jSr))/(1 - d c(s». 


can be neglected in the calculus of the lateral 
dynamics. 

• Lateral tire/ground contact forces, that are orthogo¬ 
nal to the wheels planes, are denoted as Fr and Fp 
respectively. 

• The lateral component of the gravity force is de¬ 
noted as Fq. This force applies at G and its mag¬ 
nitude is mg sin a with a denoting the terrain slope 
angle in the lateral direction. 

Application of Newton’s law yields [49.15], after 
projection in the direction orthogonal to Vr, 

/Sr = — \ (Fr + mg sin a) cos Pr 
mu 1 l 

+Fp cos(/Sr — <p) — hiLrO cos /Sr — 9 


Dynamic Model of Sliding Angles 
The kinematic model (49.53) can be used for control 
design once it is completed by a model of the dynamics 
of the sliding angles /Sr and ftp. Such a model can be 
obtained from Newton’s law and a model of tire/ground 
interactions. A few notation (Fig. 49.14 for details) and 
assumptions are introduced for this purpose: 

• The vehicle’s mass is denoted as m and its moment 
of inertia with respect to the (body-fixed) vertical 
axis is denoted as F. The vehicle’s center of mass G 
is located on the segment joining Pr to Pp, at a dis¬ 
tance Lr from Pr and Lp from Pp. 

• The longitudinal dynamics is neglected. More pre¬ 
cisely, it is assumed that the traction force applied 
to the vehicle, in relation to the monitoring of lon¬ 
gitudinal tire/ground contact forces, is controlled 
independently of the vehicle’s lateral dynamics and 
that, as a result of this control, the longitudinal ve¬ 
locity of the vehicle expressed in body frame, i. e., 
mi cos P r , varies slowly so that its time-derivative 



with Fp and Fr the (signed) intensity of the lateral 
forces Fp and Fr and 

9 = — (FpFpCOS0 — F r Fr) (49.55) 

Iz 

From (49.53) 

L9 

tan (0 + Pp) = tan p R -— . 

Ml COS Pr 

Differentiating this equation with respect to time and 
using the assumption that u\ cos Pr is constant yields 

/ip = cos 2 (</> + Pp) ( —L - d — ) -<p. 

\COS z Pr MiCOSPr/ 

The dynamics of the sliding angles is thus given by 

Pr = —— I" (Fr + mg sin a) cos /Jr 
mu\ L 

+Fp COS(/Sr — <P) — mL R 9 COS Pr I — 9 , 

Pp = cos 2 (</> + Pp) ( , —L -^-) -0. 

\ COS- Pr Ml COS Pr / 

(49.56) 

Replacing the vehicle’s angular acceleration 9 by its 
expression (49.55), and 9,<p by (49.53), one obtains ex¬ 
pressions of the sliding angles dynamics in terms of <p, 
a, mi, M 2 , F r , and Fp. 


Tire/Ground Interaction Models 
for Lateral Forces Computation 
In view of (49.56), knowledge of the lateral ground 
forces F r and Fp is needed to calculate the evolu¬ 
tion of the sliding angles and the vehicle’s motion. In 


Fig. 49.14 Lateral forces applying to a car-like vehicle 
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this respect, a model of tire/ground interactions is use¬ 
ful, in particular for simulation purposes. The Coulomb 
model, although very popular, does not allow one to 
describe the complexity of tire/ground interactions in 
a large operating domain. For this reason, other con¬ 
tact models have been proposed in the literature. For 
instance, the Dahl [49.16] and LuGre [49.17] models, 
eventhough they are not dedicated to tire/ground con¬ 
tact description, give a relationship between contact 
force and sliding velocity with a small number of pa¬ 
rameters. They are used to describe a vehicle’s dynam¬ 
ics in [49.18]. Of particular interest is the celebrated 
tire/ground contact model of Pacejka et al. [49.19], of 
which several versions depending on the application 
have been derived. A convenient model for mobile robot 
simulation and analysis is the so-called magic formula 
proposed in [49.20]. Alike other contact models, this 
formula expresses the lateral ground force in term of 
the sliding angle. It is defined as follows, 

E 

carctan(5[l — F])/l* 4— arctan(B/l*) 

B 

D = ai (F*) 2 + a 2 F* 

E = a 6 (F*) 2 + a 7 F*+a & 

«3 sin («4 arctan(fl 5 F*)) 

D — * 


(49.57) 

with * G {F, R\ used to denote either the front (F) or 
Rear ( R ) wheel, and F* the corresponding tire load 
here defined as -fm. The parameters a,-, i e [1,.... 8] 
and c are representative of the grip conditions and tire 
properties (pressure, contact patches). Table 49.1 shows 
typical values of these parameters for a vehicle moving 
on wet grass at a velocity of 2 m/s (Sect. 49.5.4 fur¬ 
ther on). Fig. 49.15 depicts the corresponding relation 
between the front lateral force Fp and the front sliding 
angle fp. This relationship is symmetric with respect 


Table 49.1 Parameters used for dynamic modelling 


Coefficients 

Values 

a\ 

-25 

ai 

500 

as 

1000 

a 4 

2 

as 

1 

ae 

0 

an 

-0.35 

as 

5 

c 

1.6 

L = Lp + Lr 

1.3 = 0.6+ 0.7 

m 

380 

h 

300 


F* = D sin 


to the origin, is quasi linear for small angles, has an ex¬ 
tremum near ± —10°, and involves a saturation for large 
sliding angles. The linear part of the function is often 
sufficient to account for slow to moderately fast motion 
in the case of good grip conditions, whereas its non¬ 
linear part becomes important when the vehicle moves 
on natural ground or at high speed with possibly large 
sliding angles. 

Tire/ground contact models are necessary to de¬ 
scribe the sliding angles’ dynamics with some accuracy 
and are useful to evaluate the performance of control 
laws in simulation. The main drawback of these mod¬ 
els is the large number of involved parameters and 
the difficulty to evaluate appropriate values in prac¬ 
tical applications, since such values heavily depend 
on imprecisely known ground characteristics which, 
furthermore, may rapidly change along the vehicle’s 
trajectory. For these reasons, previously evoked mod¬ 
els may not be best suited for control design. In many 
cases, rather than trying to exploit a complete model in¬ 
volving unreliable and rapidly varying parameters it is 
preferable to use cruder models associated with an on¬ 
line estimation procedure. The control design proposed 
further on follows this latter avenue. 

49.5.2 On-Line Estimation of Sliding Angles 

As already mentionned, some knowledge of the sliding 
angles fp and f R is useful to precisely control the lat¬ 
eral distance d between the vehicle and the desired path. 
Since direct measurement of these angles via the use of 
dedicated sensors is quite difficult in practice, an alter¬ 
native consists in designing an observer that produces 
on-line estimates of the sliding angles based on the 
measurement of the relative vehicle/path position and 
orientation. The underlying implicit (crude) model used 


Lateral force (N) 



Fig. 49.15 Relationship between lateral force and sliding angle ob¬ 
tained using Pacejka Model 
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for the design of this observer is that the sliding angles 
do not change rapidly, as can be anticipated in common 
situations like driving along a road with a slowly vary¬ 
ing curvature or following a straight line across a field 
with a constant slope. A solution of this type, exploiting 
also the vehicle’s kinematic model (49.54), is proposed 
next. 

Define £ = ( d , $ e ) T , / = (/If. Pr) 1 , and consider 
the function/ defined by 


iii) The function u\ is persistently exciting in the sense 
that there exist two constants T, S > 0 such that 

t+T 

W, J |mi(s)| ds > <5. 

t 

Then, the origin of system (49.60) is locally expo¬ 
nentially stable. 


( sin(^2 + Pi) 

tan (/ + Pi)- tan (/ 2 ) c(/) cos(f 2 + Pi) 


l cos (P 2 )- 
\ 

From (49.54), 

£ = uif(%,p,<p,c(s)) . 

Define an observer of the form 

(f = uif(%,p,<p,c(s)) + oii= , 
(P =a lB . 


l-£i c(s) 


(49.58) 


(49.59) 


The estimation error (£, P) T = (£ — £,/ — /3) T satisfies 
tf = mi lf((;,P,<t>,c(s))-f(t;,p,</>,c(s))]-ai£ , 

Ip = -ap+p ■ 

(49.60) 


The objective is to define the terms aj and ap so as 
to ensure the asymptotic stability of the origin of the 
above estimation error system when P is constant, i. e., 
when P = 0, and thus the asymptotic convergence of 
the estimate P to / in this case. Furthermore, if P varies 
slowly the estimation error should remain small. 


Sketch of Proof: From (49.61), the estimation error dy¬ 
namics (49.60) can be written as 


? = Ml 


t^{%,P,<P,c(s))P + 0 2 (P) 

L d/> 


13 — — K 2 U 1 


-P,4>,c(s )) 
L op 


ImiIA'iI , 


(49.62) 


where 0 2 (P) denotes a second-order term in P in 
a neighborhood of P = 0. Note that this term also de¬ 
pends on £, (p , c(i) and 1 —dc(s). The linearized system 
associated with system (49.62) is thus given by 


I = Mi P< </>■ c(s))P - \ui |^if , 
dp 


P — —Kju\ 


^(£,/,</>,c(s)) 

L dp 


?• 


(49.63) 


Consider the candidate Lyapunov function V defined by 


v(l,p)=K 2 \l\ 2 + \p\ 2 . 


One verifies that the time-derivative of V along the so¬ 
lutions of system (49.63) satisfies 

V = -\ui\K 2 i T Kil. 


Proposition k9.6 

Let 


Off = |wi|Aj£ 


r 3/ - 1 

Olfi = K2U1 

^r(£,/j, 0 ,C(S)) 

L d P J 


(49.61) 


with K\ a 2 x 2 positive definite matrix and Ki a positive 
scalar. Assume that P is constant and that 

i) The variables u\, t;, <j), c(s), 1 — dc(s) are bounded, 
differentiable, and their time-derivatives are 
bounded. 

ii) There exists r > 0 such that 


\4> + Pl\< \Pl\< I £2 + Pl\ < 7t/2— X . 


Since kj is positive definite and Ki is positive, V 
is decreasing. This ensures the stability of the origin 
(i;,P) = 0. Using assumptions i) and ii) and the fact 
that the matrix df/dp (£, /, cf>, c(s)) is invertible one can 
show, that the origin of system (49.63) is asymptotically 
stable. Assumption iii) implies that the origin is in fact 
exponentially stable. Using assumptions i) and ii) again, 
one shows that the origin of the original (nonlinear) sys¬ 
tem (49.62) is locally exponentially stable. ■ 

Proposition 49.6 calls for a few remarks. Although 
convergence of the estimation errors is proved in the 
case of constant sliding angles only, the observer can 
still provide a good estimate of P if P is small. For 
fast variations of /), estimation based on a dynamic 
model and IMU measurements (inertial measurement 
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unit) will usually provide better results [49.21]. Satis¬ 
faction of assumptions i) and ii) is strongly related to the 
choice of the controller used to determine the steering 
wheel angular velocity uo in (49.54). In this respect, let 
us recall that the separation principle, which allows one 
to design independently a feedback controller and an 
observer for a linear system with a guarantee of stabil¬ 
ity of the coupled system, is not systematically satisfied 
for nonlinear systems. 

49.5.3 Feedback Laws for Path Following 

The kinematic model (49.54) can be transformed into 
a chained system. This is a direct extension of the 
results of Sect. 49.4.1 . More precisely, the change of co¬ 
ordinates and control variables (s, r/, 0 e , 0, «i, M 2 ) 1 —> 
(zi,Z2,Z3,Z4, Vi, V 2 ) defined by 

(Zl,Z2,Z3,Z4) = 

s, d, [1 — cfcOs)] tan(0 e + /Jr), 

— c(s)[l -<*•(.?)] [l + 2tan 2 (0 e + /J R )] 
dc 

-d—tan(9 e + M 

as 

+ [1 - c/c(i)l 2 tan ^ + ftp) ~ tan fa 1 +tan 2 (0 e + /J R ) 1 

L COS (0 e + £ R ) (’ 

(z»l, Ih) = (zi,z 4 ) 


with 

62 = 9 e + /Jr 
' k = 1 — c(s) d 

A = —knd — &3 k sign(wi) tan 62 + c(s)k tan 2 66 • 

(49.65) 

49.5.4 Path Following Illustrations 
in Low Grip Conditions 

Simulation Results 

The simulation results presented next have been ob¬ 
tained by using the kinematic and dynamic models 
(49.54) and (49.56) respectively, with the set of pa¬ 
rameters for the tire/ground interaction specified in 
Table 49.1. The reference path, depicted on Fig. 49.16 
in black plain line, is composed of two straight lines 
connected by a circular path with a radius of 13 m. 
The simulated grip conditions generate sliding along 
the circular part of the path. The transitions between 
this circular part and the straight lines yield transient 
errors due to the curvature discontinuity. The vehicle’s 
velocity is 3 m/s. 

The steering velocity U 2 is computed according to 
the simple proportional feedback law 

M2 = kffa (0 0ref) 


transforms the model (49.54) of a car-like vehicle into 
a 4-D chained system. 

Based on this transformation, Sect. 49.4 provides 
a feedback control solution V 2 to the path following 
problem. Since the relation between V 2 and the origi¬ 
nal control variable ui involves the sliding angles /Jf 
and /Jr, 112 can be computed from the outputs /Jp, /Jr of 
the observer proposed in the previous section. 

In practice, preexisting low-level control loops may 
use the steering angle 0 itself, rather than its time- 
derivative, as a control input. In this case the expression 
for the desired steering angle can be obtained by iden¬ 
tifying the expression of 9 t in (49.54) with the one in 
(49.13), where U 2 is the feedback law deduced from the 
(chained-form) control law (49.27) of a unicycle-like 
robot. Further assuming that c(,v) is constant, one ob¬ 
tains the following expression 


0 re f = arctan 


tan(/J R ) 


L 

H-^— 

COs(jSr) 


c(s) COS 62 



- /J F 


(49.64) 


with 0 re f defined by (49.64) and the control gain = 
12 (settling time of 250 ms). The other control gains 
in (49.64) are defined as (^ 2 .^ 3 ) = (0.09,0.6), yield¬ 
ing a settling distance of 12m without overshoot. First, 

Y coordinates (m) 



Fig. 49.16 Reference path and trajectories in Cartesian 
plane 
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the control law is applied with (ftp, /Jr) = (0,0) (i. e., 
without considering sliding). The results are shown 
in blue on Figs. 49.16 and 49.17. Then, the control 
law is applied with j3p and /3 r obtained via the ob¬ 
server (49.59)-(49.61), using for Ki a diagonal matrix 



0 5 10 15 20 25 30 35 

Time (s) 


Fig. 49.18 Comparison of estimated and actual sliding 
angles 

with elements equal to 10 and 20 and the gain Ki = 10. 
The results are shown in red. Fig. 49.17 clearly illus¬ 
trates the reduction of lateral error resulting from the 
sliding angles estimation. 


Lateral inclination (°) 
20 
15 



Y coordinates (m) 0 0 

c) Tracking error (m) 



Fig.49.19a-c Path following with sliding effects for an autonomous off-road robot: (a) Reference trajectory in 3-D 
frame (b) Robot during autonomous tracking (c) Tracking errors ( dashed: without sliding compensation, dash-dotted: 
with sliding estimation and compensation) 
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Performance of the proposed observer is illustrated 
on Fig. 49.18 with the true sliding angles (i. e., obtained 
from the simulated dynamics) in plain lines and the es¬ 
timated angles in dashed lines. One can verify that the 
sliding angles are correctly estimated. In practice, using 
an observer allows one to also compensate for vari¬ 
ous unmodeled dynamics resulting, for instance, from 
imprecise knowledge of the vehicle’s geometry (the dis¬ 
tance L, for example) or from an unknown offset on the 
steering angle. 

Experimental Results 

Full scale experiments on natural terrain with the car¬ 
like vehicle of Fig. 49.19 are reported next and in 
l^yil'Jf'JfJ. This mobile robot is equipped with a real 
time kinematics global positioning system (RTK-GPS) 
producing a ±2 cm accurate absolute position mea¬ 
surement. The GPS antenna is located straight above 
the control point P R . The reference path, consisting of 
a straight line perpendicular to the slope direction, has 
been previously recorded during a manual driving. The 
top left of Fig. 49.19 shows the recorded coordinates in 
x-y plane, and the z axis represents the absolute value 
of the lateral slope angle a. This angle progressively 
reaches the value of 10° at the curvilinear abscissa 27 m, 
and it subsequently rapidly increases up to about 18°. 


As for the simulation results reported previ¬ 
ously, the feedback control (49.64) was first applied 
with 

Pf = Pr = 0 

(i. e., without taking sliding into account), then with 
Pf,Pr given by the observer. In both cases, control 
gains equal to these used in simulation have been used, 
i.e., (£ 2 . £ 3 ) = (0.09, 0.6). The vehicle’s longitudinal 
velocity during experiments was 3 m/s. The bottom 
part of Fig. 49.19 shows the tracking error along the 
path with the blue dashed line corresponding to using 
(Pf- Pr) = (0,0) in the control law, and the red dashed- 
dotted line corresponding to the use of sliding angles 
estimated values. 

These experimental results confirm the simulation 
ones reported before. The improvement in tracking 
accuracy resulting from the use of estimated sliding 
angles in the control law is clearly illustrated: the lat¬ 
eral error remains small (smaller than 0.25 m) all the 
time and it becomes negligible as soon as the ter¬ 
rain’s slope is alsmost constant (beyond the curvilinear 
abcissa 30 m). This is consistent with the observer’s sta¬ 
bility and convergence analysis when the sliding angles 
are assumed constant. 
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49.6.1 General Trailer Systems 

Most of the control design approaches here presented 
and illustrated for unicycle-like and car-like vehicles 
can be extended to the case of trains of vehicles com¬ 
posed of trailers hitched to a leading vehicle. In par¬ 
ticular, the methods of Sect. 49.4 which are specific to 
nonholonomic systems can be extended to this case, 
provided that the kinematic equations of motion of 
the system can be transformed (at least semiglobally) 
into a chained system [49.6]. This basically requires 
that the hitch point of each trailer is located on the 
rear-wheel axle of the preceding vehicle [49.22], For 
instance, the transformation to the chained form is not 
possible when there are two (or more) successive trail¬ 
ers with off-axle hitch points [49.23]. So-called general 
trailer systems (with off-axle hitch points) raise diffi¬ 
cult control design issues, and the literature devoted 
to them is sparse. For this reason, and also because 
these systems are not met in applications as frequently 
as simpler vehicles, control methods specifically devel¬ 
oped for them are not reported here. Nonetheless, a few 


related references are given next. The path-following 
problem has been considered in, e.g., [49.24] for a sys¬ 
tem with two trailers and, more generally, in [49.25, 
Chap. 3] and [49.26] for an arbitrary number of trailers. 
To our knowledge, the problem of stabilizing nonsta¬ 
tionary reference trajectories has not been addressed 
for these systems (except in the single trailer case for 
which the system can be transformed into the chained 
form [49.23,27]). In fact, the explicit calculation of 
feasible trajectories joining a given configuration to an¬ 
other is already a very difficult problem, even in the 
absence of obstacles. As for the asymptotic stabilization 
of fixed configurations, the problem can (in theory) be 
solved by using existing general methods developed for 
the larger class of controllable driftless systems. FIow- 
ever, the calculations associated with these methods 
quickly become intractable when the number of trail¬ 
ers increases. More specific and simpler ones have been 
proposed in [49.28], for an arbitrary number of trail¬ 
ers and the asymptotic stabilization of a reduced set of 
configurations, and in [49.29], in the case of two trailers 
and arbitrary fixed configurations. 
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49.6.2 Sensor-Based Motion Control 

The control laws described in the present chapter, 
and their calculation, involve the online measurement, 
eventually complemented by the online estimation, of 
variables depending on the position of the vehicles in 
their environment. Measures can be acquired via the use 
of various sensors (odometry, GPS, proximetry, vision, 
etc.). Usually, various treatments are applied to raw 
sensory data prior to computing the control variables 
themselves. For instance, noise filtering and state esti¬ 
mation are such basic operations, well documented in 
the automatic-control literature. Among all sensors, vi¬ 
sion sensors play a particularly important role in robotic 
applications, due to the richness and versatility of the 
information which they provide. The combination of vi¬ 
sual data with feedback control is often referred to as 
visual sending. In Chap. 34, a certain number of visual 
servoing tasks are addressed, mostly in the context of 
manipulation and/or under the assumption that realiz¬ 
ing the robot task is equivalent to controlling the pose of 
a camera mounted on an omnidirectional manipulator. 
In a certain number of cases, the concepts and methods 
described in this chapter can be adapted, without much 
effort, to the context of mobile robots. These cases ba¬ 
sically correspond to the control methods adapted from 
robotic manipulation which are described in Sect. 49.3 
of the present chapter. For instance, automatic driving 
via the control of the visually estimated lateral distance 
between a robotic vehicle and the side of a road, or 
car-platooning by controlling the frontal and lateral dis¬ 
tances to a leading vehicle, can be addressed with the 
control techniques reported in Chap. 34. The reason is 
that it is possible to simply recast these techniques in 
the form of the control laws proposed in Sect. 49.3. 
However, there are also vision-based applications for 
nonholonomic mobile robots which cannot be solved 
by applying classical visual-servoing techniques. This 
is the case, for instance, of the task objectives addressed 
in Sect. 49.4, an example of which is the stabilization 
of the complete posture (i. e., position and orientation) 
of a nonholonomic vehicle at a desired one. Vision- 
based control problems of this type have been addressed 
in [49.11,30]. 

49.6.3 Sliding Effects 

and Other Dynamical Issues 

Results of Sect. 49.5 can be viewed as a first step to 
addressing dynamical issues associated with sliding ef¬ 
fects. The kinematic control models (49.53)-(49.54) 
can be extended to other wheeled vehicles ([49.31] 
for the mobile robot classification proposed by [49.32] 
and [49.33] for a trailer system). On-line estimation of 


sliding angles via an observer can also be achieved by 
exploiting the dynamics of these angles and making use 
of complementary measurements (from an IMU for ex¬ 
ample). A result in this direction is proposed in [49.21] 
where a model of sliding angle dynamics is used at the 
observer level, thus yielding a much more reactive esti¬ 
mation of these angles. Note that the kinematic control 
law of Sect. 49.5 may still be used in this case. When 
prior knowledge of the reference path is available, a pre¬ 
dictive control strategy may also be adopted to reduce 
transient error effects associated with slow steering an¬ 
gle dynamics. 

Various risks related to the vehicle’s integrity when 
operating at high speed, like e.g., swing around or roll¬ 
over, were not addressed in this chapter. In [49.34] 
these issues are treated at the path planning level. 
Knowledge of the robot’s dynamics can also be used 
to relate control inputs to a metrics measuring the im¬ 
portance of such risks in order to introduce constraints 
at the control level [49.35]. A multimodels approach 
(kinematic and dynamic) can be used to translate sta¬ 
bility aspects into control constraints, for instance by 
defining a maximal admissible longitudinal velocity. 
A predictive control strategy may also help to limit 
the risk of roll-over, of steering saturation, or of swing 
around [49.36]. Finally, it is possible to take advantage 
of additional actuators, as in the case of four indepen¬ 
dent actuated wheels [49.37], or by using inclination 
actuators [49.38]. 

49.6.4 Bibliographical Guide 

A few former surveys on the control of WMRs have 
been published. Let us mention [49.39-41], which con¬ 
tain chapters on the modeling and control issues. A de¬ 
tailed classification of kinematic and dynamic models 
for the different types of WMR structures, on which 
Chap. 24 is based, is provided in [49.32]. The use of 
the chain form to represent WMR equations has been 
proposed in [49.42], then generalized in [49.22]. 

Path following may have been the first mobile robot 
control problem addressed by researchers in robotics. 
Among the pioneering works, let us cite [49.43,44]. 
Several results presented in the present chapter are 
based on [49.6,45]. 

The problem of tracking admissible trajectories for 
unicycle-type and car-like vehicles is treated in the 
books [49.39-41], and also in numerous conference 
and journal papers. Several authors have addressed this 
problem by applying dynamic feedback linearization 
techniques. In this respect, one can consult [49.46-48], 
and [49.39, Chap. 8], for instance. 

Numerous papers on the asymptotic stabilization 
of fixed configurations have been published. Among 
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them, [49.49] provides an early overview of feed¬ 
back control techniques elaborated for this purpose, 
and also a list of references. The first result present¬ 
ing a time-varying feedback solution to this problem, 
in the case of a unicycle-type vehicle, is in [49.4]. 
The conference paper [49.50] provides a survey on 
time-varying feedback stabilization, in the more gen¬ 
eral context of nonlinear control systems. More spe¬ 
cific results, like Propositions 49.2 and 49.4, are given 
in [49.6,7]. Other early results on the design of 
smooth time-varying feedbacks can be found in [49.51, 
52], for example. Concerning continuous (but not 
Lipschitz-continuous) time-varying feedbacks yield¬ 
ing exponential convergence, one can consult [49.53]. 
Designs of hybrid discrete/continuous fixed-point sta¬ 


bilizers can be found in [49.54-57], for instance. 
Discontinuous control design techniques are not ad¬ 
dressed in the present chapter, but the interested 
reader will find examples of such feedbacks in [49.58, 
59], 

To our knowledge, the control approach presented 
in Sect. 49.4.6, which is based on the concept of 
transverse functions [49.9, 10], is the first attempt to 
address the problem of tracking arbitrary trajectories 
(i. e., not necessarily feasible for the controlled robot). 
Implementation issues and experimental results for this 
approach can be found in [49.11, 12]. An overview of 
trajectory tracking problems for wheeled mobile robots, 
with a detailed case study of car-like systems, is pre¬ 
sented in [49.13]. 


Video-References 
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Tracking of an admissible trajectory with a car-like vehicle 

available from http://handbookofrobotics.org/view-chapter/49/videodetails/181 

Tracking of arbitrary trajectories with a truck-like vehicle 

available from http://handbookofrobotics.org/view-chapter/49/videodetails/182 

Tracking of an omnidirectional frame with a unicycle-like robot 

available from http://handbookofrobotics.org/view-chapter/49/videodetails/243 

Mobile robot control in off-road condition and under high dynamics 

available from http://handbookofrobotics.org/view-chapter/49/videodetails/435 
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50. Modeling and Control of Robots 

on Rough Terrain 


Keiji Nagatani, Genya Ishigami, Yoshito Okada 


In this chapter, we introduce modeling and control 
for wheeled mobile robots and tracked vehicles. 
The target environment is rough terrains, which 
includes both deformable soil and heaps of rubble. 
Therefore, the topics are roughly divided into two 
categories, wheeled robots on deformable soil and 
tracked vehicles on heaps of rubble. 

After providing an overview of this area in 
Sect. 50.1, a modeling method of wheeled robots 
on a deformable terrain is introduced in Sect. 50.2. 
It is based on terramechanics, which is the study 
focusing on the mechanical properties of natural 
rough terrain and its response to off-road vehicle, 
specifically the interaction between wheel/track 
and soil. In Sect. 50.5, the control of wheeled 
robots is introduced. A wheeled robot often expe¬ 
riences wheel slippage as well as its sideslip while 
traversing rough terrain. Therefore, the basic ap¬ 
proach in this section is to compensate the slip 
via steering and driving maneuvers. In the case 
of navigation on heaps of rubble, tracked vehicles 
have much advantage. To improve traversability 
in such challenging environments, some tracked 
vehicles are equipped with subtracks, and one 
kinematical modeling method of tracked vehi¬ 
cle on rough terrain is introduced in Sect. 50.4. 
In addition, stability analysis of such vehicles is 
introduced in Sect. 50.5. Based on such kinemat¬ 
ical model and stability analysis, a sensor-based 
control of tracked vehicle on rough terrain is in¬ 
troduced in Sect. 50.6. Sect. 50.7 summarizes this 
chapter. 
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50.1 Overview 

50.1.1 Modeling Deformable Terrain 
for Robots 

The category of rough terrain includes a variety of 
ground conditions. In the case of weak and fine gravel, 
the motion of a wheeled mobile robot becomes rela¬ 
tively complicated because the interaction mechanics 
of the wheel in a rough terrain differs from that of an 
indoor robot on a flat surface; a wheel on gravel has 
multiple point contact, and a wheel on loose soil has 
a surface contact that deforms the soil around the wheel. 

There has been a large amount of research on robot 
mobility analysis in the military community [50.1,2]. 
These studies have primarily focused on empirical anal¬ 
yses of large vehicles (i. e., gross vehicle weights of 
several tons). The mobility analyses of small mobile 
robots that consider the interaction mechanics of a slip¬ 
ping wheel on rough terrain have also been investigated. 
Additionally, a multibody system simulation for the 
longitudinal slip of tires with respect to the tire-soil in¬ 
teraction has been demonstrated [50.3]. A dynamic in¬ 
teraction model, denoted as soil contact model (SCM), 
has been developed to provide the dynamics of a plasti¬ 
cally deformable surface using an elevation grid, as well 
as to consider the multipass effect of wheels, shown in 
Fig. 50.1 [50.4]. Recently, multibody dynamics toolkit, 
such as Vortex (I<£»JKM1!1MI), can be used to simulate 
terrain reaction forces for mobility analyses of mobile 
robots [50.5]. 

Recently, terramechanics [50.6] has been widely 
used in applications such as off-road UGVs (unmanned 
ground vehicles) [50.7, 8] and planetary rovers on Mars. 
A fundamental requirement for such off-road mobile 
robots is to maintain a traversing capability in rough 


Following wheels 


Leading wheels 


Fig. 50.1 Dynamics of a plastically deformable surface 
in consideration with the multipass effect of wheels (af¬ 
ter [50.4]) 


terrain composed of loose sand, mud, pebbles, etc. One 
typical issue in such an environment is wheel slippage, 
which is generated because of the soil deformation at 
the wheel-terrain contact patch. Wheel slippage often 
degrades the mobility performance of the robot because 
some amount of tractive power from a driving actua¬ 
tor is consumed by the soil deformation. Furthermore, 
the wheel slippage induces vehicle sideslip, which is 
an additional disturbance for the motion control of 
a wheeled robot. Figure 50.2 shows one typical ex¬ 
ample of lateral- and side-slip situation of a wheeled 
mobile robot in rough terrain. One typical example of 
terramechanics-based modeling of rough terrain is de¬ 
scribed in Sect. 50.2. Recently, a study of terradynamics 
of legged locomotion was proposed for traversal in 
granular media in [50.9] and is shown in l<s>Mfl3l!lE3l. 

50.1.2 Controlling Robots 

in Deformable Terrain 

A wheeled robot often experiences wheel slippage as 
well as vehicle sideslip while traversing rough terrain. 
This degrades the motion control performance of the 
robot, i. e., a robot could deviate from the path to be fol¬ 
lowed or a wheel may get trapped in loose soil. There¬ 
fore, a control scheme that can derive the appropriate 
steering/driving maneuvers is necessary for a wheeled 
robot to achieve traversability on a rough terrain and to 
manage any slippage problems that may arise. 

A large number of studies related to the path¬ 
following control of mobile robots have been published. 
General information on path-following issues can be 
obtained from [50.10-12], Rezaei et al. investigated an 
online path-following strategy combined with a simul¬ 
taneous localization and mapping (SLAM) algorithm 
for a car-like robot in outdoor environments [50.13]. 



Fig. 50.2 Lateral- and side-slip situation of wheeled mo¬ 
bile robot 
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Coelho and Nunes et al. proposed a Kalman-based 
active observer controller for the path-following of 
wheeled mobile robots [50.14]. Helmick et al. devel¬ 
oped a path-following algorithm that included slip 
compensation by using visual odometry and a Kalman 
filter [50.15, 16]. The method was applied to the rocker- 
bogie configuration with six steerable wheels robot, 
shown in Fig. 50.3. In Sect. 50.3, a path-following 
control scheme for a four-wheeled mobile robot is in¬ 
troduced. The experimental validation of the control 
described in the section was reported in [50.17]. 

50.1.3 Modeling and Controlling Robots 
in Heaps of Rubble 

In environments that involve heaps of rubble, such as 
an urban search and rescue situation, the approach for 
modeling and controlling robots is different from the 
afore mentioned methodology. To tackle such chal¬ 
lenging environments, tracks are used to enhance the 
mobility of a robot in such environments because they 
have more contact with the terrain than wheels. Some 
tracked vehicles are equipped with active subtracks, 
and multitracks configuration enables the vehicle to 
get over relatively large steps. A typical geometrical 
model of tracked vehicles with subtracks is introduced 
in Sect. 50.4. 

In case that the target rubble is stable, kinematic ap¬ 
proaches are useful for navigation and control of the 
robot. In this case, stability analysis is very important. 
In classical stability analysis, the energy stability mar¬ 
gin (ESM) proposed by Messuri and Klein was defined 
in terms of the potential energy of the robot [50.18]. 
Hirose assumed that increasing the robot’s weight did 
not always contribute to its stability because it also in¬ 
creased the dynamic disturbance around its center of 
gravity. Therefore, he proposed the normalized ESM 



Fig. 50.3 Rocky 8: rocker-bogie configuration with six 
steerable wheels robot 


(NESM), in which the robot’s weight was normal¬ 
ized [50.19]. For this criterion, the NESM is used to 
evaluate the stability of a robot on the basis of the verti¬ 
cal distance between the initial and largest heights of the 
center of gravity during a rollover. Recently, Tubouchi s 
group proposed a method to control tracked vehicles 
exposed to random steps [50.20] (Fig. 50.4). An artifi¬ 
cial uneven environment is random steps, arranged with 
different lengths of squared timbers. It also appears in 
This environment is used to evaluate the 
performance for urban search and rescue robots [50.21]. 
The basic objective is to maintain the maximal stability 
of the robot at every step of its path in the random step 
field. The earlier-given topic relating to stability anal¬ 
ysis is introduced in Sect. 50.5. Another approach for 
stability analysis of tracked vehicles-based NESM was 
reported in [50.22]. 

To evaluate the stability of vehicles, terrain de¬ 
tection is an important technology. Therefore, many 
robotics researchers have focused on the terrain map¬ 
ping, particularly based on the SLAM, described 
in Chap. 46, and machine learning. LIDAR (light detec¬ 
tion and ranging) and stereo cameras are widely used 
for the terrain mapping (Sect. 22.3). Vandapel et al. 
proposed a classification method of three-dimensional 
(3-D) point cloud data on the basis of its geometri¬ 
cal features [50.23], and Lacaze et al. demonstrated 
a method of finding a path in heavy grasses [50.25]. To 
improve the LIDAR-based mapping, Ohno et al. pro¬ 
posed a method to obtain thin objects such as nets, 
poles, and wires [50.26]. Figure 50.5 shows a mapping 
result at an outdoor rough terrain. 

If the rubble is unstable, modeling the machine- 
rubble interaction is very complicated. One challenging 



Fig.50.4a-c Teleoperated system of tracked vehicle. Itconsists of 
(a) a remote operation station, (b) a mobile robot platform, and (c) 
path planner that maintain the maximal stability of the robot (af¬ 
ter [50.20]) 
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Fig.50.5a,b Classification of outdoor 
rubble data, (a) Target field: Hyogo 
Prefectural Emergency Management 
and Training Center, and (b) classified 
result for traversal (after [50.23]) 
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Fig. 50.6 Approaches for gareki data acquisition (af¬ 
ter [50.24]) 


approach to modeling rubble is called gareki (rub¬ 
ble in Japanese) engineering [50.27,28]. The model 
includes not only the structure data, but also the in¬ 
ternal force of each gareki element. To obtain the 
data, they proposed to use information about the fol¬ 
lowing, as shown in Fig. 50.6: (1) a real-collapsed 
building, (2) artificial full- and miniature-scale col¬ 


lapsed buildings, and (3) a simulated virtual digital 
building. Their research aims to eventually apply the 
gareki model for the control of mobile robots and res¬ 
cue missions. 

The sensor-based method is another approach for 
controlling robots in unstable rubbles. Such robots often 
have multiple tracks connected by joints to maneuver 
over natural/artilicial steps, bumps, and stairs. Some 
are equipped with passive joints [50.29], but many are 
equipped with active joints [50.30]. Multitrack config¬ 
urations enable the vehicle to maneuver over a step that 
is higher than the radius of the tracks. To enable such 
motion, a sensor-based method is proposed to control 
the multi tracks. The robot moves and detects the sur¬ 
face shape of the ground continuously, and it changes 
the configuration of the subtracks based on the sur¬ 
face information. Once the surface of the ground is 
deformed by the traversal of the robot, the subtracks 
adapt quickly. A representative example for the con¬ 
trol of robots in unstable rubble is shown in Sect. 50.6. 
Another sensor-based approach is to use force-feedback 
from contact points. Inoue et al., embedded force sen¬ 
sors in their subtracks to obtain the contact force, and 
changed the configuration of the subtracks [50.31]. 


50.2 Modeling of Wheeled Robot in Rough Terrain 


This section introduces a mechanics of wheeled mobile 
robot in rough terrain, particularly focusing on a mod¬ 
eling of wheel-terrain interaction. 

50.2.1 Dynamics of Mobile Robots 

Figure 50.7 depicts a schematic illustration of a dy¬ 
namic model for a wheeled mobile robot. The robot is 
modeled as an articulated multibody system. The equa¬ 
tion of motion for the robot is numerically given by 

H(^ b )+C + G= (^ b )+J T J e , (50.1) 


where the symbols have the following meanings: H: in¬ 
ertia matrices composed of multibody of the robot, 
C: velocity depending term, G: gravity term, "V j,: trans¬ 
lational and angular velocities of the vehicle, q: joint 
angles (such as wheel rotation and steering angles), 
forces and moments at the centroid of the vehicle 
body, r: torques acting at each joint (driving/steering 
torques), J: Jacobian matrix, external forces and 
moments acting at the centroid of each wheel, namely 

fj(i = {r, l}J={r,m,f }) . 

The dynamic motion of the robot with given trav¬ 
eling and steering maneuvers are numerically obtained 
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by successively solving (50.1). Here, a key approach 
for the dynamic model is to incorporate a well-defined 
contact model into the equation of motion in order to 
calculate the external forces and torques at the con¬ 
tact patch between the wheel and terrain. Some recent 
works have reported dynamic simulation tools com¬ 
bined with the terramechanics wheel model: for exam¬ 
ple, [50.32, 33] for the NASA’s rovers, and [50.4, 34] 
for the ExoMars. 

50.2.2 Wheel-Terrain Interaction Mechanics 

Terramechanics is a study focusing on the mechan¬ 
ical properties of natural rough terrain and its re¬ 
sponse to off-road vehicle, specifically the interaction 
between wheel/track and soil. In 1960s, Bekker de¬ 
veloped a classical terramechanics with many of the 
key concepts related to vehicle-terrain response, such 
as a well-known pressure-sinkage equation and shear 
stress model [50.35,36]. Wong developed a compre¬ 
hensive procedure for predicting the performance of 
both driven and towed wheels [50.37-39]. The proce¬ 
dure calculates the wheel mechanics with applying the 
stress distribution model beneath the wheel. 

Terramechanics-based approach for vehicle-terrain 
mechanics includes mainly three methodologies [50.40, 
41]: (1) Analytical approach, (2) empirical approach, 
and (3) numerical approach. 

The analytical approach is based on a theoreti¬ 
cal model for vehicle-terrain interactions along with 
experimental model validation. The empirical method 
is to utilize a practical measurement of soil strength 



Fig. 50.7 Dynamic model of wheeled mobile robot 

by a specialized apparatus in order to estimate vehi¬ 
cle traversability on weak/rough terrain. The numerical 
method includes the finite element method (FEM) and 
discrete element method (DEM) that models soils as 
vast numbers of multiple particles and simulates each 
particle’s behavior while accounting vehicle-terrain in¬ 
teraction [50.42-44]. 

This subsection focuses on the analytical method 
and introduces a typical interaction model of rigid 
wheel on deformable terrain (Fig. 50.8). It should be 
noted that Chap. 55.3.13 also addresses the wheel trac¬ 
tion mechanics in detail, and therefore, the following 
subsection recalls some key equations in order to dis- 



Fig. 50.8 Wheel-terrain contact 
model 
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Fig.50.9a-d Single wheel test 
beds for experimental validation of 
terramechanics models, (a) Single 
wheel test bed at MIT (after [50.45]), 
(b) Single track test bed at JAXA 
(after [50.46]), (c) Single wheel test 
bed at DLR (after [50.47]), (d) Single 
wheel test bed at Tohoku University, 
(after [50.48]) 


cuss a wheel traction characteristics on rough terrain, 
soil parameter identification, and model uncertainty. 

Wheel Slippage 

Wheel slippage is generally observed when a robot 
travels on loose soil. Also, the wheel will slip in the lat¬ 
eral direction while the robot steering or traversing on 
sloped terrain. The slip ratio (i. e., slip in the longitudi¬ 
nal direction of wheel travel) is defined as a function of 
the longitudinal traveling velocity of the wheel v x and 
the circumferential velocity of the wheel rtw, where r is 
the wheel radius and a> represents the angular velocity 
of the wheel [50.37] 

( (rco — v x )/rco ( \rco\ > \v x \ : driving) , 
((rco-v x )/v x (|m>| < |v,| : braking). 

(50.2) 

The slip ratio assumes a value in the range from — 1 to 1. 

The slip in the lateral direction of the wheel is 
expressed by the slip angle f), which is defined as 
a function of v x and the lateral traveling velocity ty, 
as 


and shear stresses from the entry angle 8 f and the exit 
angle 9 r [50.38,48] 

Of 

F x = rb J{z x (9) cos 9—a(9) sin 9} d9 , (50.4) 

0, 

Of 

F y = I [rbz y {9) + R b {r — z(9) cos 9}] d 9, (50.5) 

Or 

Of 

F z = rb J{r x (9) sin 9 + a(9) cos 9} d9 , (50.6) 

Or 

Of 

T x = r 2 b J z x {9) d9 . (50.7) 

Or 

where 0 ( 8 ) is the normal stress, t x (9) and z y (9) are 
the shear stresses in the longitudinal or lateral direction. 
z{9) is the wheel sinkage at an angle of 9, b is the wheel 
width, and 74 is the bulldozing resistance generated on 
a side wall of the wheel. 


P 


tan 1 {v y /v x ) {v x ^Q), 
sgn(Uy) ■ tr/2 (v x = 0 ). 


(50.3) 


Wheel Traction Forces 

The wheel-terrain contact forces, including the drawbar 
pull F x , side force F y , vertical force F z , and resistance 
torque T, are calculated as the integral of the normal 


Experimental Validation 

An experimental validation of the afore mentioned 
wheel-terrain interaction model is performed by the 
use of a single wheel test bed (Fig. 50.9), with 
varied state parameters such as soil or wheel slip 
conditions. A carriage velocity controlled relative to 
a wheel velocity realizes a wheel slip (or traction 
load) while measuring wheel traction forces, wheel 
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sinkage, and others. Experimental data obtained from 
the test bed are then compared with the values ob¬ 
tained from numerical simulation of the wheel traction 
model. 

50.2.3 Wheel Traction Performance 

The wheel-terrain model can be exploited for an 
evaluation of the mobility performance of a wheeled 
robot (i. e., traversability on sloped or deformable ter¬ 
rain [50.49,50]). Such mobility evaluation technique is 
further valuable for the mobility system design in or¬ 
der to find a feasible wheel/track design that maximizes 
the traction performance for off-road locomotion under 
some specific constraints [50.46,47,51]. 

Ishigami et al. proposed a wheel traction diagram 
called a thrust-cornering characteristic diagram that 
determines the slope traversal criteria with respect to 
wheel slip conditions [50.17]). Figure 50.10 illustrates 
the relationship between the thrust and cornering forces 
of wheel on rough terrain, with varied slip ratios/slip 
angles. The thrust and cornering forces F c and Ft 
are obtained by calculating the drawbar pull and side 
force 

F c = F x sin/l + F y cos /3 , (50.8) 

Fp = F x cos P — F y sin ji . (50.9) 

Considering the slope traversal scenario as shown in 
Fig. 50.11, the thrust-cornering characteristic diagram 
can draw the criteria with a minimum cornering force 
and thrust force required for a slope traversal: Case 1 as 
a traversal and uphill traveling, Case 2 as a straight-line 
traversal, and Case 3 as a traversal and downhill trav¬ 
eling. In addition, a robot decelerates or cannot move 
forward due to its negative thrust force. This diagram, 
for example, indicates that the slip ratio must exceed 
0.6 in order to generate positive thrust force, and also, 
the cornering force varies primarily with the slip an¬ 
gle. Therefore, the diagram can determine the slope 
traversability on the basis of the corresponding slip¬ 
page, indicated by several characteristic curves on the 
diagram. 

50.2.4 Model Uncertainty and Soil 
Parameter Identification 

It should be noted that the classical terramechanics 
model has primarily developed for an application for 
large-heavy vehicles (hundreds/thousands kg weight). 
Therefore, several assumptions that may cause an in¬ 
accurate calculation of wheel traction performance of 


the classical model would be omitted while exploiting 
the classical model to an analysis of small lightweight 
robot, resulting in an inaccurate calculation of wheel 
traction performance. The error due to the inadequate 
assumptions for the small-lightweight robot can be ad¬ 
dressed by the following approaches: model improve¬ 
ment and uncertainty analysis of model parameters. 
It should be noted that the Bekker’s pressure-sinkage 
model is assumed that the contact patch of wheel on 
deformable soil (circumferential section) is discretized 
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Fig. 50.10 Thrust-cornering characteristic diagram 



Fig. 50.11 Force balance on slope traversal 
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as a series of consecutive flat plate; however, Bekker 
noted [50.36]: 

Predictions for wheels smaller than 20 inches in 
diameter become less accurate as wheel diameter 
decreases, because the sharp curvature of the load¬ 
ing area was neither considered in its entirety nor is 
it reflected in bevameter tests. 

Also another research reported that the assumptions 
provide inaccurate prediction for vehicles with wheels 
diameter less than about 50 cm (approx.) and normal 
loading less than 45 N (approx.) [50.36]. 

Some researchers have tried to update/improve the 
classical terramechanics model that can be successfully 
applied to relatively light weight vehicles: For example, 
Nagatani et al. developed a direct measurement de¬ 
vice for the normal stress distribution [50.52], A wheel 
diameter dependent pressure-sinkage model has been 
proposed by [50.53]. Senatore and Iagnemma proposed 


an improved approach for the calculation of shear de¬ 
formation modulus [50.54]. 

Model parameters in the wheel-terrain interaction 
related to the soil properties are subject to uncer¬ 
tainties since their values would stochastically vary 
with location. Several researches have addressed the 
soil parameter identification: an on-line terrain pa¬ 
rameter estimator with simplifying classical terrame¬ 
chanics equations was proposed by [50.55]; a mod¬ 
ified nonlinear wheel-terrain interaction model for 
an identification of pressure-sinkage coefficient, in¬ 
ternal friction angle, and shear deformation modu¬ 
lus [50.56]. Some recent works have also attempted 
to predict rover mobility even in uncertain condi¬ 
tions: a learning-based approach for slip-prediction 
and traversability analysis of a wheeled robot [50.57]; 
and a statistical method for mobility prediction of 
a robot with taking into account of terrain uncer¬ 
tainty [50.58]. 


50.3 Control of Wheeled Robot in Rough Terrain 


This section introduces a control scheme for the 
wheeled robot that compensates wheel-and-vehicle 
slippages while following a given path is described. 

A wheeled robot often experiences wheel slip¬ 
pages as well as vehicle sideslip while traversing in 
rough terrain, which degrades a performance of a mo¬ 
tion control of the robot: i. e., a robot will deviate 
from a path to be followed, or a wheel might get 
stuck into loose soil. Therefore, a control scheme 
that can derive appropriate steering/driving maneu¬ 
vers is necessary for a wheeled robot to achieve 
a rough terrain traverse and to manage any slippage 
problems. 

A significant number of studies related to the path 
following control of mobile robot have been published. 
A general note for path following issues can be seen 
in [50.10-12]. Rezaei et al. investigated an online path 
following strategy combined with a simultaneous local¬ 
ization and mapping (SLAM) algorithm for a car-like 
robot in outdoor environments [50.13]. Coelho and 


Nunes et al. proposed a Kalman-based active observer 
controller for the path following of wheeled mobile 
robots [50.14], 

As noted before, a key issue of the path following 
on rough terrain is to compensate wheel-and-vehicle 
slippage. This subsection briefly introduces a slip- 
compensated path follower along with a derivation 
for steering and driving maneuvers for path follow¬ 
ing [50.15-17] which is shown in 

50.3.1 Slip-Compensated Path Follower 

A two-dimensional path-following problem of 
a mobile robot is shown in Fig. 50.12. Here, a path 
following control generally performs to let the vehi¬ 
cle velocity i>o coincide with the tangent of a given 
path such that the distance and orientation errors de¬ 
crease to zero. However, once the vehicle experiences 
a sideslip with a certain amount of slip angle fo, the 
vehicle has an additional orientation error of fio even 


a) No-slippage condition 



b) Slippage condition 



A> = o 




Fig.50.12a,b Path following 
problem without (a) and with 
(b) slip angle of vehicle 
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when the vector of vehicle velocity completely follows 
the path (Fig. 50.12b). 

The path follower basically derives a commanded 
robot velocity vector (Tcmd..Vcmd> $cmd) while the robot 
position (xo,yo, 6 q) and a reference path vector (jc p , _v p ) 
are given. The slip-compensated path follower de¬ 
scribed here consists of two algorithms to achieve a path 
following task through high-slip environments. 

First, the carrot heading algorithm determines 
the heading error of the robot 0 e , which is an an¬ 
gle calculated from the intersection of a circle cen¬ 
tered on the robot coordinate with the desired path 
(Fig. 50.13): 

9 e = e d - e 0 (50.10) 

Flere, 6 d is the desired heading angle which is a tan¬ 
gent angle on the intersection point and 9q is the robot 
heading angle. A large radius of the circle will generate 
a smooth motion of the robot while neglect a small fea¬ 
ture of the path. A small radius will reduce the total path 
following error, but requires frequent heading changes 
for small path errors. 

Once the heading error is determined, the slip- 
compensated path following algorithm executes the 
heading controller that calculates the commanded yaw 
rate of the robot 0 cm( \ from the heading error 9 e and the 
yaw slip rate Po 

(A| ■ 6> e + K 2 • Po) 

6>cmd = A ---- , (50.11) 

1 s 

where K\ and Ki are the tuned controller gains, and T s 
is the sampling period. 

Note that the command for the linear velocity i cm( ] 
and v cm d are closely related to the maximum speed 
of the driving motors and the desired heading angle 
9 C md- Practically, these commands are accomplished by 
actuators of the robotic mobility system, namely steer¬ 
ing and driving mechanisms. The following subsection 
describes the derivation of those steering and driving 
maneuvers that satisfies the heading controller com¬ 
mand 6 cml 1 as defined in (50.11). 

50.3.2 Steering and Driving Maneuvers 

Here, a nonholonomic kinematic model of a four- 
wheeled mobile robot with each wheel being inde¬ 
pendently steerable is considered. Figure 50.14 illus¬ 
trates a two-dimensional kinematic model of the mo¬ 
bile robot having the slip angle of the vehicle Po 
and lateral wheel slippage /l, (the subscript i denotes 
the wheel identification number, i = 1.4 in this 


case). The dimension of the rover is defined by If, 
l T , £/r, and d l- The position and orientation of the 
vehicle are defined as (xq, yo, #o)> and the position 
of each wheel is defined as (x/, yi). In this model, 
the following assumptions are considered: the dis¬ 
tance between the wheels is constant, and the steering 
axle of each wheel is perpendicular to the terrain 
surface. 

The nonholonomic constraints of the mobile robot 
are defined by the following equations, with taking 
the lateral slips of wheels and that of vehicle into 
account 

To sin <t>o — yo cos (/>o = 0 , 

Xi sin cpj — V; cos (pi = 0 , (50.12) 

where 0o = $0 + Po and 0; = #0 + <5; + Pi. The geomet¬ 
ric constraints between each wheel and the centroid of 



Fig. 50.13 Path following with carrot heading calculation 




Fig. 50.14 Kinematic model of four-wheeled rover with 
wheel/vehicle slips 
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the vehicle are written as 


the wheel linear velocity Vi is given as 


x\ = xo + If cos 6q — cIl sin 0o ' 
X2 = xq — l x cos 0o — fit sin 9q 
X 3 = Xq — l T cos 9q + r/R sin 9q 
X 4 = Xq + /f cos 0 O + d R sin 0 O 


Xi — XQ + » 


(50.13) 


Vi = ran/ cos Pi - 


Vf is also expressed by x, or y t 
Vi = x t / cos <pi = y t / sin (pi . 


(50.17) 


(50.18) 


y\ = yo + k sin 0 O + d L cos 0 O 
yi=yo~ It sin 00 + d L cos 0 O 
y 3 = — It sin 0o — d R cos 9q 

y4=yo + If sin 0 O - d R cos 0 O 


->■ 3 u =+ y, ■ 

(50.14) 


Given the desired heading angle 0o = 0d and de¬ 
sired linear velocity v&, the desired steering angle 8<u 
that satisfies the path following control as well as the 
slip compensation is obtained as follows: first, trans¬ 
form (50.12) 

<5 d f = tan -1 (y,/x,) - 0 d - Pi . (50.15) 

Subsequently, substituting (50.13) and (50.14) into 
(50.15), the desired steering angle is determined as fol¬ 
lows 


8di = tan 


Vi sin0 d — F;(0 d ) 

, 7 ! d COS 0o Xj (0cmd) 

— 0d — Pi ■ 


(50.16) 


The desired yaw rate 0 cm d is given from (50.11). 

The driving maneuver is practically executed by the 
wheel angular velocity of the robot. From Fig. 50.14, 


Substituting (50.1 3) or (50.14) into the above equations, 
the desired wheel angular velocity con can be obtained 


m = 


(Vi cos 0 d + X;(0 cm d)) cos Pi/r cos (pi 
(0 d < jt/4) , 

(v d sin 0 d + Fi(0 C md)J cos Pi/r sin (p, 
(9 a > 7T/4) . 


(50.19) 


Position and orientation errors relative to a desired 
path should be accurately determined for deriving the 
control input as well as calculating the correspond¬ 
ing steering and driving maneuvers. A visual odometry 
method is of promising technique for such accurate 
measurement of position, orientation, velocity, and slip¬ 
page, and also it is sufficiently robust for loose sandy 
terrain. 

The visual odometry estimates the traveling velocity 
of the vehicle based on the optical flow vectors obtained 
from the time-consecutive images taken by an onboard 
camera(s). Integrating the velocity estimates with iner¬ 
tial measurement unit (IMU) readouts or stereo images 
for pose estimation provides an accurate estimation of 
the six degrees of freedom of the rover motion. 


50.4 Modeling of Tracked Vehicle on Rough Terrain 


Tracks are one of typical mechanism that can enhance 
the mobility of a robot on rough terrain because they 
can have contacts between the terrain more widely 
than wheels. Thus, many types of robots which are 
required to maneuver rough terrain such as military 
and search-and-rescue robots have been equipped with 
tracks. 

Some tracked vehicles often have multiple tracks 
connected by joints to maneuver natural/artificial steps, 
bumps and stairs. Some are equipped with passive 
joints, but many are equipped with active joints. Mul¬ 
titracks configuration enables the vehicle to get over 
a step which is higher than the radius of the tracks. 

It should be noted that many tracked vehicles are 
intended to maneuver the unknown rough terrain con¬ 


sisted of rubbles, rocks, concrete, and woods. Because 
such types of terrain are nonuniform unlike fine soil dis¬ 
cussed in the Sect. 50.2, to dynamicaly model them is 
obviously difficult. In other words, we can easily sense 
the shape of the terrain, but dynamic parameters as well 
as center of gravity (COG), weight or stiffness of terrain 
cluster. 

Thus, to geometrically consider the interaction be¬ 
tween tracks and terrain is a reasonable approach. In 
this section, a geometrical model of a contact between 
a track and the terrain surface is introduced. The analy¬ 
sis includes (1) a parameterization of a generic shaped 
track, (2) contacts between the track and a single point 
in the terrain surface, and (3) contacts between the track 
and the terrain surface represented by a point cloud. 
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50.4.1 Parameterization of a Track 

The geometry of a generic-shaped track is the key issue 
that should be discussed for modeling a tracked vehicle 
on rough terrain. In this section, we introduce a param- 
etarization of the shape of a generic track. 

As shown in Fig. 50.15, a generic track can be pa¬ 
rameterized by the radius of the first and the second 
pulleys, R and r respectively, and the distance between 
the centers of the pulleys L. 

In addition, let us embed the coordinate frame; its 
origin is at the center of the adjacent track and the x 
axis is along the center line of the adjacent track. The 
relative pose of the track respect to the adjacent track 
can be parameterized by the distance between the origin 
and the center of the first pulley, and the angle of the 
joint that connects the tracks 9. 

50.4.2 Contact with a Single Point 
in the Terrain Surface 

To discuss contacts between a track and a single point 
in the terrain surface is a good entrance to understand 
it between a track and the whole terrain surface. In the 
following sections, we geometrically describe the for¬ 
mer contacts with the parameters of the track defined in 
the previous section. 

As shown in Fig. 50.16, a track generally has four 
sections that can contact with the terrain surface; since 
it is tracked, it comprises a round and straight sections. 
Moreover, it can be in two states, namely, folded and 
spread states. When the track is in the spaded state, 
side-S supports the robot; on the other hand, when it 
is in the folded state, side-F is in contact with the ter¬ 
rain. We call the case where side-S is in contact with 
the ground as the spreading mode and the case where 
side-F is in contact with the ground as the folding 
mode. 



Fig. 50.15 Example of geometrical analysis contact be¬ 
tween track and single point on the terrain surface 


In addition, we have assumed the pose of the track 
to be 0[°] when its straight section on side-S is in con¬ 
tact with the flat surface; we have also assumed the 
direction in which the track lifts its second pulley from 
0[°] to be positive. 

The geometrical relationship between the contact 
point and pose of the track is summarized in Table 50.1 . 
For example, in the case of the spreading mode and the 
contact on the straight section (Fig. 50.15), we can de¬ 
rive the following equation according to the geometry 
of the track contacting with the surface 

^contact — 9 ] T O 2 

_1 7 

= tan - 

X -^support 

+ sin -1 r . (50.20) 

\J (x -^support T 7r 

50.4.3 Contact with the Terrain Surface 
Represented by a Point Cloud 


Now we are well prepared to geometrically understand 
contacts between the track and the terrain surface. In 
this section, we describe those contacts by iterating 
a point cloud that represents the terrain surface. 

Let the shape of the terrain surface be represented 
by a set of points {u\,U 2 ,... ,u n }. Then, the contact an¬ 
gle of the track with the surface is determined using the 
following equation 

111ifl(^contact.! . • ■ • » ^contact, n)- 

in spreading mode , 

F 6 (50.21) 

max(0 conta ct,l. ■ • • 1 ^contact./i)• 

in folding mode. 

To calculate a contact angle of the track with a single 
point in the terrain surface, we need to determine which 
section and which side of the track makes the contact. 




Fig. 50.16 Generic-shaped track consisted of staraight and 
round sections 
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Table 50.1 Geometric-based contact angle of subtrack 



The type of the section can be determined by checking 
the distance between the center of the first pulley and 
the point; if d < Vi 2 + R 2 the straight section makes 
the contact, else the round section does. On the other 
hand, the side will be determined by the choice of the 
application. 


Now we understand a geometric model which de¬ 
scribes contacts between the tracked vehicle consisted 
of multiple tracks and the free-formed terrain surface 
whose shape is represented as a point cloud. In the fol¬ 
lowing subsection, we introduce an application example 
utilizing the contact model. 


50.5 Stability Analysis of Tracked Vehicles 


As shown in the previous sections, it is hardly possible 
to build an indention model of a rough terrain consisted 
of rubbles. Thus, in this research domain, stability cri¬ 
teria based only on geometrical parameters are often 
used to evaluate pose of the tracked vehicle on the rough 
terrain. 

This section introduces two typical stability crite¬ 
ria which have been adopted for tracked vehicles; (1) 


support polygon and (2) normalized energy stability 
margin. 

The support polygon is a simple and reasonable cri¬ 
terion to determine whether an object on a surface is 
statically stable. The support polygon for the tracked 
vehicle is equal to the horizontal projection of the con¬ 
vex hull of all the contact points between the tracks and 
the terrain. With this support polygon, we can consider 



Label 

Balance quality 

Technical details 

Red 

Turn upside down or 

Pitch > ji /4 or roll 


get stuck 

> Jt /6 or at least 
one of conditions A 
is not satisfied 

Orange 

Lose balance on 

Two optional postures 


purpose 

exist 

Magenta 

Climb up or slide down 

Oscillations in 
posture estimation 
algorithm 

Cyan 

Jump down 

Jump of CM between 
two stable postures 
< 50 mm 

Yellow 

Fair 

NESM parameter < 1 

Green 

Good 

NESM parameter > 1 


Fig.50.17a,b Example of stability analysis of tracked vehicle on a rough terrain based on support polygon and NESM 
(after [50.20]); (a) Stable posture corresponding to green label in table, (b) Unstable posture corresponding to orange 
label in table 
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that the vehicle is statically stable when it contains the 
projection of the vehicle’s center of gravity. 

Another stability criterion is the normalized energy 
stability margin, or NESM, proposed by Hirose. The 
NESM is a criterion that is used to evaluate the stability 
of a robot on the basis of the vertical distance between 
the initial position of its center of gravity and its high¬ 
est position during tumbling. Although it is mainly used 
for walking robots, its evaluation only requires the posi¬ 
tions of contact points with the ground and the center of 
gravity of the robot. In other words, there is no concep¬ 


tual difference if this criterion is applied to the stability 
of tracked vehicles. 

One typical application of stability criteria for the 
tracked vehicle is to rank possible path in a naviga¬ 
tion. For example, Magid proposed a rating method for 
possible paths based on stability analysis utilizing the 
support polygon and NESM. This method evaluates the 
stability of each adjacent grid of the current position of 
the vehicle; paths to grids on which the support polygon 
condition is violated are denied. Then survived paths 
are ranked utilizing the NESM (Fig. 50.17). 


50.6 Control of Tracked Vehicle on Rough Terrain 


In this section, an approach for the autonomous control 
of active joints in a tracked vehicle with multiple tracks 
is introduced. As shown in the previous subsections, the 
geometric model of the tracked vehicle gives the pose of 
each track, which makes contact with the rough terrain 
on the basis of the shape of the terrain surface near the 
vehicle. The autonomous control utilizes the geomet¬ 
ric model to determine the desired angles of the active 
joints between the tracks to perform a smooth maneuver 
on an unknown rough terrain. 

Tracked vehicles with multiple tracks and joints 
typically have better locomotion than single-tracked 
vehicles. This is because one can change the pose 
of a track by actuating an adequate joint to negoti¬ 
ate bumps or steps in a rough terrain. Some multi- 
tracked vehicles a are equipped with a single-degree-of- 
freedom (1-DOF) tracked arms, often called subtracks, 
and the others are snake shaped. 

However, multitracked vehicles are difficult to con¬ 
trol manually because of their multiple degrees of 



Fig. 50.18 Tracked vehicle Kenaf equipped with four sub¬ 
tracks enhansing mobility and three LIDAR sensors sens¬ 
ing shape of terrain 


freedom. For military and search-and-rescue robots 
in particular, which are likely to be teleoperated on 
an unknown rough terrain with limited camera views, 
it is quite difficult to control all of their DOFs 
manually. 

In the rest of this section, an approach for the au¬ 
tonomous control of active subtracks on the basis of 
the geometric model of a multitracked vehicle is intro¬ 
duced. In the control algorithm, the geometric model 
automatically determines the desired poses of active 
joints, which support subtracks on the basis of the ter¬ 
rain shape obtained in real time. 

The multitracked vehicle Kenaf, which has au¬ 
tonomous subtrack control, is shown in Fig. 50.18. 
Kenaf has two main tracks covering its main body 
and four subtracks actuated by 1-DOF joints on each 
comer of the main body. It has 6-DOFs in total: 4- 
DOFs corresponding to joints supporting the subtracks, 
1-DOF corresponding to the one main track and two 
subtrack belts on the right, and 1-DOF corresponding 
to the one main track and two subtrack belts on the 
left. 

In addition, Kenaf is equipped with three LIDAR 
sensors, one on the front, one on the right side, and 
one on the left side of the main body, to obtain data 
on the shape of the terrain online. Scans from LIDARs 
are integrated to estimate the three-dimensional shape 
of the terrain near the vehicle. The aforementioned ap¬ 
proach for autonomous control of active subtracks are 
described in [50.59]. 

Figure 50.19 shows the flowchart of the algorithm 
for the autonomous subtrack control, and its actual mo¬ 
tions can be seen in and l<a>M'ii>mit« 

The control algorithm is divided into six steps and sum¬ 
marized as follows: 

1. Slices of the shape of the terrain around the robot 

are first obtained from the LIDAR sensors attached 
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to the robot body and the three-dimensional terrain 
shape near the robot is estimated. 

2. The desired posture (roll and pitch angles) of the 
body is then calculated on the basis of the estimated 
terrain shape. 

3. The desired positions of the subtracks that realize 
the desired posture of the robot body are also deter¬ 
mined. 

4. Next, the stability of the desired posture and sub¬ 
track positions is evaluated. 

5. If the desired pose (posture of the robot body and 
subtrack positions) are unstable, then the desired 
pose is redefined and steps (3)-(5) are repeated. 

6. When the desired subtrack positions that realize 
a stable posture are generated, position control of 
the subtracks is finally performed. 

In the following sections, we explain each step of 

the algorithm. 

50.6.1 Ground Detection and Trimming 
of the Scanned Data 


50.6.2 Determination of Desired Posture 

We then determine the desired posture on the basis 
of the least-squares plane for the target terrain (/target- 
This is intended to enable smooth locomotion on rough 
terrain. Parameters a, b, and c of the least-squares 
plane z = ax + by + c are determined using the follow¬ 
ing equations (an over-bar indicates the average of in 
scanned points in (/target) 

, O'v.y &x,y&y,z 

a = -—- , 

(j'l . l Ct'v. y Oi x ^yCC Xt y 

I _ 0ty t z0C XtX &x,yCt z x 

(X K ., Q'v.v Qfr ,y&x,y 

c = z^ — x^a — y^b , 

U.T.y =X u -y u -X^-%, 

=y,t-z u -%-z [,, 

&z,x — Zh ' X u Zu * X u , 

&x,x — X u ■ Xit X u • Xk , 

Uy.y =y u -y u -%-% ■ 


(50.23) 

(50.24) 

(50.25) 

(50.26) 

(50.27) 

(50.28) 

(50.29) 

(50.30) 


In this step, we obtain the scanned points (/target near the 
robot using the LIDAR sensors attached to the robot; 
ground detection is performed according to the ap¬ 
proach described in the paper [50.59]. We first obtain 
two-dimensional terrain slices from three LIDAR sen¬ 
sors and integrate them according to each tagged and 
estimated position of the robot to generate the three- 
dimensional information of the terrain shape. At the end 
of this step, we filter distant points from the integrated 
terrain shape. 

/^target — {ttl, t<2, • • • , U m } (50.22) 


Then, a translation to the coordinate system, if the 
robot body is parallel to the least-squares plane of the 
ground surface and makes contact with the ground, 
is described by the following equation in quaternion 
algebra: 
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Fig. 50.19 Algorithm for autonomous control of subtracks 
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50.6.3 Determination 

of Desired Subtrack Positions 


(50.31) 


(50.32) 


(50.33) 


In this step, using the desired body posture and the inte¬ 
grated terrain slices, we determine the desired subtrack 
positions on the basis of the geometric model of con¬ 
tacts between the vehicle and the terrain surface. 
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Our application allows the operator to manually 
switch between the spreading and the folding control 
modes. Each control mode employs a corresponding ge¬ 
ometric calculation method described in Sect. 50.4 In 
other words, the autonomous controller uses the corre¬ 
sponding calculation to determine the desired subtrack 
positions according to the control mode specified by the 
operator. 

In both modes, we determine the desired subtrack 
positions to make contact with the ground through the 
desired robot posture. In particular, we calculate the an¬ 
gular position of each subtrack that makes contact with 
each point scanned by the left and the right LIDAR 
sensors. The desired position of each subtrack is deter¬ 
mined for the maximum or minimum angular position 
of the subtrack in the spreading or folding mode, re¬ 
spectively. 

50.6.4 Stability Evaluation of Desired Pose 

In the proposed controller, we have adopted the nor¬ 
malized energy stability margin (NESM) [50.19] as the 
stability criterion for the desired pose. The stability of 
a desired pose is evaluated by NESM, and the pose is 
redefined if the stability is insufficient. 

In the case of a tracked vehicle with four subtracks, 
four contact points (front-right, front-left, rear-right, 
and rear-left) can be determined by the step described 
in Sect. 50.6.3. In addition, the four axes of tumbling 
that pass through the front, rear, right, and left contact 
points can be assumed. Hence, the stability of a tracked 


50.7 Summary 

In this chapter, an overview of the modeling and con¬ 
trol of robots on the rough terrain motion has been 
provided. The target environment was rough terrains, 
which includes a variety of ground conditions. In the 
case of weak and fine gravel, the motion of a wheeled 
mobile robot becomes relatively complicated because 
the interaction mechanics of the wheel in rough terrains 
differs greatly from that of an indoor robot on a flat sur¬ 
face. In this chapter, we introduced a terramechanics- 


vehicle with four subtracks can be determined from the 
minimum value of the NESM about these four axes. 

50.6.5 Stabilization of Desired Pose 

When the NESM of the robot is less than a predeter¬ 
mined threshold, we repeat the following steps until 
a desired, stable pose is realized. This step is intended 
to realize the strategy 4: 

la. If the NESM about the front or rear is adopted, re¬ 
duce the pitch angle of the desired posture to zero. 

lb. When the NESM about the right or left is adopted, 
reduce the roll angle of the desired posture to zero. 

2. Redefine the desired subtrack positions by recalcu¬ 
lating them to realize the redefined desired posture. 

3. Evaluate the NESM about the redefined posture and 
positions of subtracks. 

50.6.6 Position Control of Subtracks 

Finally, we perform position control of the subtracks to 
realize the desired subtrack position produced through 
the above-mentioned steps on the basis of the strategy 
described in the previous subsection. 

All subtracks of Kenaf and Quince are controlled by 
microprocessors present on the built-in motor drivers. 
Each desired subtrack position is sent to the correspond¬ 
ing microprocessor as a reference for position control 
using the conventional proportional-integral-derivative 
(PID) controller. 


based model for the interactions between wheels and 
deformable terrains. Furthermore, we explained the 
steering control of wheeled mobile robots to enable 
path-following in such environments. In the case of 
an environment with heaps of rubble, the approach for 
controlling robots is different from the terramechanics 
approach. Therefore, we introduced an initial attempt 
at a sensor-based approach that involved measuring the 
terrain shape. 


Video-References 


I^MjnESEI Mobility prediction of rovers on soft terrain 

available from http://handbookofrobotics.org/view-chapter/ 50 /videodetails /184 
Experiments of wheeled rovers in a sandbox covered with loose soil 
available from http://handbookofrobotics.org/view-chapter/ 50 /videodetails /185 
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Terradynamics of legged locomotion for traversal in granular media 
available from http://handbookofrobotics.org/view-chapter/50/videodetails/186 
Interaction human-robot supervision, long range science rover for Mars exploration 
available from http://handbookofrobotics.org/view-chapter/50/videodetails/187 
A path-following control scheme for a four-wheeled mobile robot 
available from http://handbookofrobotics.org/view-chapter/50/videodetails/188 
Evaluation test of tracked vehicles on random step fields in the Disaster City 
available from http://handbookofrobotics.org/view-chapter/50/videodetails/189 
Autonomous sub-tracks control 

available from http://handbookofrobotics.org/view-chapter/50/videodetails/190 
Autonomous sub-tracks control 

available from http://handbookofrobotics.org/view-chapter/50/videodetails/191 
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51. Modeling and Control 
of Underwater Robots 


Gianluca Antonelli, Thor I. Fossen, Dana R. Yoerger 


This chapter deals with modeling and control 
of underwater robots. First, a brief introduction 
showing the constantly expanding role of marine 
robotics in oceanic engineering is given; this sec¬ 
tion also contains some historical backgrounds. 
Most of the following sections strongly overlap 
with the corresponding chapters presented in this 
handbook; hence, to avoid useless repetitions, 
only those aspects peculiar to the underwater en- 
vironmentare discussed, assumingthatthe reader 
is already familiar with concepts such as fault de¬ 
tection systems when discussing the corresponding 
underwater implementation. The modeling section 
is presented by focusing on a coefficient-based 
approach capturing the most relevant underwater 
dynamic effects. Two sections dealing with the de¬ 
scription of the sensor and the actuating systems 
are then given. Autonomous underwater vehicles 
require the implementation of mission control sys¬ 
tem as well as guidance and control algorithms. 
Underwater localization is also discussed. Under¬ 
water manipulation is then briefly approached. 
Fault detection and fault tolerance, together with 
the coordination control of multiple underwater 
vehicles, conclude the theoretical part of the 
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chapter. Two final sections, reporting some 
successful applications and discussing future per¬ 
spectives, conclude the chapter. The reader is 
referred to Chap. 25 for the design issues. 


The world’s oceans cover 2/3 of the Earth’s surface and 
have been critical to human welfare throughout history. 
As in ancient times, they enable the transport of goods 
between nations. Presently, the seas represent critical 
sources of food and other resources such as oil and gas. 


In the near term, we may soon see the emergence of 
offshore mining for metals as well as the exploitation 
of gas hydrates. Conversely, the ocean can also threaten 
human safety and damage infrastructure through natural 
phenomena such as hurricanes and tsunamis. 



51.1 The Expanding Role of Marine Robotics in Oceanic Engineering 

Our scientific understanding of the deep sea is expand- The first scientific explorations were conducted pri- 
ing rapidly through the use of a variety of technologies, marily through the use of diving and human-occupied 
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submersibles, complemented by a variety of other tech¬ 
nologies such as towed or lowered instruments, trawls, 
dredges, autonomous seafloor instruments, and deep- 
sea drilling. More recently remotely operated and au¬ 
tonomous vehicles have begun to revolutionize seafloor 
exploration, often returning superior data at reduced 
costs. In the near future, seafloor observatories linked 
by fiber-optic cables and satellites will return massive 
amounts of data from coastal and deep-sea sites. These 
observations will complement those from conventional 
expeditionary investigations, and will require teleoper- 
ated or robotic intervention during installation and for 
service. An example of a remotely operated vehicle de¬ 
veloped for the scientific study of the seafloor is the 
Jason 2 vehicle developed at the Woods Hole Oceano¬ 
graphic Institution (shown in Fig. 51.1), and a list of 
remotely operated vehicles for scientific exploration ap¬ 
pears in Table 51.1 (the last vehicle in the table, Kaiko, 
was lost several years ago). 

Offshore oil and gas installations are presently ser¬ 
viced almost exclusively by remotely operated vehicles 
(ROVs), physically connected via a tether to receive 
power and data, with human divers used only for the 
shallowest installations. Subsea systems require ex¬ 
tensive work capability during installation, and need 
frequent inspection and intervention to support drilling 
operations, actuate valves, repair or replace subsea 
components, and to accomplish a variety of tasks re¬ 
quired to maintain production rates and product quality. 
The trend toward robotic and teleoperated subsea inter¬ 
vention is certain to continue as offshore oil and gas 
production moves into deeper waters, and economic 
considerations push key production steps from surface 
platforms to the seafloor. Remotely operated manipu¬ 
lators enable these systems to perform complex tasks 
such as debris removal, cleaning using abrasive tools, 



Fig. 51.1 The ROV Jason 2 (courtesy of Woods Hole 
Oceanographic Institution) 


Table 51.1 ROVs for scientific use 


Vehicle 

Depth (m) 

Institution 

Manufacturer 

Hyperdolphin 

3000 

JAMSTEC 2 

ISE 

Dolphin 3K 

3000 

JAMSTEC 

JAMSTEC 

Quest 

4000 

MARUM b 

Shilling 

Tiburon 

4000 

MBARI C 

MBARI 

ROPOS 

5000 

CSSF 11 

ISE 

Victor 

6000 

IFREMER e 

IFREMER 

Jason 

6500 

WHOI f 

WHOI 

ISIS 

6500 

NOC 8 

WHOI 

UROV 7K 

7000 

JAMSTEC 

JAMSTEC 

Kaiko 

11000 

JAMSTEC 

JAMSTEC 

a Japan Marine Science and Technology Center; b Zentrum fur 
Marine Umweltwissenschaften; c Monterey Bay Aquarium Re¬ 
search Institute; d Canadian Scientific Submersile Facility; e 
Institut frangais de recherche pour 1’exploitation de la mer; f 
Woods Hole Oceanographic Institution; g National Oceanogra- 

phy Centre 





and to operate a variety of nondestructive testing tools. 
The effectiveness of using ROVs decreases with depth 
mainly due to the cost increase and the difficulties of 
handling the long tether. 

Autonomous underwater vehicles (AUVs) are free- 
swimming unoccupied underwater vehicles that can 
overcome the limitations imposed by ROV tethers for 
some tasks. Such vehicles carry their own energy sup¬ 
plies (presently batteries, perhaps fuel cells in the 
future) and communicate only through acoustics and 
perhaps optical links in the near future. Limited com¬ 
munications require these vehicles to operate indepen¬ 
dently of continuous human control, in many cases 
the vehicles operate completely autonomously. AUVs 
are currently used for scientific survey tasks, oceano¬ 
graphic sampling, underwater archeology and under-ice 
survey. Military applications, such as mine detection 
and landing site survey, are presently operational, and 
more ambitious applications such as long-term un¬ 
dersea surveillance are in engineering development. 
Presently, AUVs are incapable of sampling or manip¬ 
ulations tasks like those done routinely by ROVs, as 
typical work environments tend to be complex and chal¬ 
lenging even to skilled human pilots. 

Today, approximately 1000 AUVs are operational, 
many of them experimental. However, they are ma¬ 
turing rapidly. Recently several companies now offer 
commercial services with AUVs. As an example, for 
the oil and gas industry the cost reduction of a survey 
performed with an AUVs instead of a towed vehi¬ 
cle is up to 30% and the data quality is generally 
higher. Likewise, commercial manufacturers in several 
countries now offer turnkey AUV systems for specific, 
well-defined tasks. Currently, remotely operated manip- 
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ulators are standard equipment for most ROVs, while on 
the contrary autonomous manipulation is still a research 
challenge; the projects SAUVIM [51.1], ALIVE [51.2] 
and TRIDENT [51.3] were devoted to studying this 
control problem. shows a graphical ren¬ 

dering of a mission with the AUV Nereus, 
shows the perspective view from the Remus AUV, 
l-gsM'll'IhlTB a 10-minutes long documentary of an 
underice mission and finally a strong in¬ 

teraction with a shark. 

51.1.1 Historical Background 

Boats have been used by humans since the start of 
recorded history, but vehicles able to go under wa¬ 
ter are more recent. Perhaps the first recorded idea of 
an underwater machine came from Aristotle; accord¬ 
ing to legend he built the: skaphe andros (boat-man) 
that allowed Alexander the Great (Alexander III of 
Macedonia, 356—323 BC) to stay submerged for at 
least half a day during the war of Tiro in 325 BC. 
This is probably unrealistic; if true it would precede 
Archimedes’ law, which was first articulated in approx¬ 
imately 250 BC. Leonardo Da Vinci may have been 
the first to design an underwater vehicle. His efforts 

51.2 Underwater Robotics 

51.2.1 Modeling 

A rigid body is completely described by its position and 
orientation with respect to a reference frame V,. O, — 
xyz that is supposed to be Earth-fixed and inertial. Let 
us define rp eK 3 as 

q 1 = C* y £) t , 

the vector of the body position coordinates in an Earth- 
fixed reference frame. The vector i] \ is the correspond¬ 
ing time derivative (expressed in the Earth-fixed frame). 
If one defines 

V \ = (u v w) T 

as the linear velocity of the origin of the body-fixed 
frame Xj>, Ob — *b Vb 7b with respect to the origin of 
the Earth-fixed frame expressed in the body-fixed frame 
(from now on: body-fixed linear velocity) the following 
relation between the defined linear velocities holds 

v 1 = Rf 1 , (51.1) 

where R[ ! is the rotation matrix expressing the transfor¬ 
mation from the inertial frame to the body-fixed frame. 


were recorded in the Codice Atlantico (Codex Atlanti- 
cus), written between 1480 and 1518. Legends say that 
Leonardo worked on the idea of an underwater military 
machine but he destroyed the results as he judged them 
to be too dangerous. The first use of feedback theory for 
marine control was probably the Northseeking device, 
patented in 1908, that used gyroscopic principals to de¬ 
velop the first autopilot [51.4]. From that point on, the 
use of feedback theory in marine control grew contin¬ 
uously; it is interesting to notice that the proportional- 
integral-derivative (PID) control commonly used today 
in numerous industrial applications was first formally 
analyzed in 1929 by Minorsky [51 .5]. The first remotely 
operated underwater vehicle, POODLE, was built in 
1953, and the ROV evolved through the 1960s and 
1970s, mostly for military purposes. In the 1980s ROVs 
became established for use in the commercial offshore 
industry and began to emerge for scientific applications. 
The first tetherless, autonomous vehicles were built for 
experimental purposes in the 1970s. Currently, AUVs 
are becoming increasingly commonplace for scientific, 
military, and commercial applications. Turnkey AUV 
systems for a range of tasks are available from commer¬ 
cial vendors, and AUV services can be acquired from 
a number of companies [51.6]. 


Let us define 1/2 e SO(3) as 

)/2 = (4> 9 f) T 

the vector of body Euler angle coordinates in a Earth- 
fixed reference frame. In the nautical field those are 
commonly named roll, pitch, and yaw. Yaw is defined 
as rotation around the z axis of the fixed frame; pitch is 
defined as rotation around the y axis resulting after the 
yaw movement; and roll is defined as rotation around 
the x axis resulting after both yaw and pitch movements. 
The vector ))2 is the corresponding time derivative (ex¬ 
pressed in the inertial frame). Let us define 

vi = (p q r) T 

as the angular velocity of the body-fixed frame with 
respect to the Earth-fixed frame expressed in the body- 
fixed frame (from now on: body-fixed angular velocity). 
The vector r )2 does not have a physical interpretation 
and it is related to the body-fixed angular velocity by 
a proper Jacobian matrix 

V2 =Jk,o(V2)V2 ■ 


(51.2) 
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The matrix J^ j0 e R 3 * 3 can be expressed in terms of the 
Euler angles as 


3k, 0 (^/ 2 ) 


/I 0 -s e \ 

0 cgs# 

S(ji cgc&J 


(51.3) 


where c a and s a are abbreviations for cos (a) and 
sin(a), respectively. The matrix Ja-, 0 (J72) is not invert¬ 
ible for every value of >/2 - In detail, it is 


Jfc»(*2) 


/I 

0 


C(t>CQ 

^0 


CQS(f) 
c <t> ) 


(51.4) 


that it is singular for 6 = (21+ 1)^ rad, with leN.i.e., 
for a pitch angle of ± y rad. 

The rotation matrix Rf, needed in (51.1) to trans¬ 
form the linear velocities, is expressed in terms of the 
Euler angles by 

R?(V2) = 

^ CxlfCQ SxJ/CQ SQ ^ 

+C+S9 + SiJ/SQS(fr SfiCQ 

\ T C^fSQCfjj Cil/Sfp T S^/fSgCf/) CfiCQ J 

(51.5) 


Table 51.2 shows the common notation used for ma¬ 
rine vehicles according to the Society of Naval Archi¬ 
tects and Marine Engineers (SNAMEs) notation [51.7]; 
a sketch is shown in Fig. 51.2. 

As for any representation of a rigid body’s orienta¬ 
tion several possibilities arise, among them, the use of 
a four-parameter description given by unit quaternions. 
The term quaternion was introduced by Hamilton in 


Table 51.2 Common notation for the motion of a marine 
vehicle 



Forces 

and moments 

»1, »2 

VuVi 

Motion in the 

^-direction 

Surge 

X 

u 

X 

Motion in the 
y-direction 

Sway 

Y 

V 

y 

Motion in the 
z-direction 

Heave 

Z 

w 

Z 

Rotation about 

the x-axis 

Roll 

K 

p 

<t> 

Rotation about 
the y-axis 

Pitch 

M 

q 

e 

Rotation about 
the z-axis 

Yaw 

N 

r 




z 

Fig. 51.2 Motion variables for an underwater vehicle 


x 


1840, 70 years after the introduction of a four-parameter 
rigid-body attitude representation by Euler. An intro¬ 
duction to alternative orientation representations can be 
found in Chap. 2 and concerning the marine environ¬ 
ment in [51.8]. 

It is useful to collect the kinematic equations in six¬ 
dimensional matrix forms. Let us define the vector 1 / e 
R 6 as 



and the vector v e K 6 as 


(51.6) 



(51.7) 


and, defining the matrix J e (Rg) e 
O 3 X 3 

(03x3 Jk,o 


Je(R^) = 


Rf 


(51.8) 


where the rotation matrix Rf is given in (51.5) and 3k,o 
is given in (51.3), 

v = J e (Rf))?. (51.9) 


The inverse mapping, given the block-diagonal struc¬ 
ture of J e , is given by 


>) = J e -'(R T B )i; 


( r b 0 3x3 \ 

V°3X3 J Vo) 


where J k 3 is given in (51.4). 
Defining as 


(51.10) 



the vector of generalized forces, where 
n = (X Y Z) T , 


(51.11) 
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the resultant forces acting on the rigid body expressed 
in a body-fixed frame, and 

t 2 = (K M N) t , (51.12) 

the corresponding resultant moment to the pole Ob, it is 
possible to rewrite the Newton-Euler equations of mo¬ 
tion of a rigid body moving in the space as 

M rb v + C RB (v)v = t v . (51.13) 


The derivation of (51.13) can be found in Chap. 3. 

The matrix M RB is constant, symmetric, and pos¬ 
itive definite, i. e., M RB = 0 and M RB = > 0. Its 

unique parametrization is of the form 


M rb 


/ ml 3 — /;zS(rjj,)\ 

V«S(r^ : ) I 0b j ' 


(51.14) 


where is the 3 x 1 distance vector to the center of 
gravity (CG) expressed in the body-fixed frame, I 3 is 
the 3x3 identity matrix, and I 0b is the inertia tensor 
expressed in the body-fixed frame S(x) is the matrix 
operator performing the cross product between two (3 x 
1 ) vectors 


S(x) 


< 0 

-x 3 

x 2 ^ 

x 3 

0 

-Xl 


Xi 

0 ) 


On the other hand, there does not exist a unique 
parametrization of the matrix Crb , which represents the 
Coriolis and centripetal terms. It can be demonstrated 
that the matrix Crb can always be parameterized such 
that it is skew symmetric, i. e., 

C RB (v) = —C£b(v) VveR 6 ; (51.15) 


explicit expressions for Crb can be found in [51.8]. 

Notice that (51.13) can be greatly simplified if the 
origin of the body-fixed frame is chosen to be coinci¬ 
dent with the central frame, i. e., /•/ = 0. 


Hydrodynamic Generalized Forces 
Equation (51.13) represents the motion of a rigid body 
in an empty space, while dealing with ships or underwa¬ 
ter vehicles requires the consideration of the presence of 
the hydrodynamics generalized forces, i. e., the forces 
and moments caused by the presence of the fluid. In 
hydrodynamics it is common to assume that the gen¬ 
eralized hydrodynamics forces on a rigid body can be 
linearly superimposed [51.9]; in particular, those are 
separated into radiation-induced forces, environmental 
disturbances, and restoring forces due to gravity and 
buoyancy. 


Radiation-induced forces are defined as the: 

forces on the body when the body is forced to os¬ 
cillate with the wave excitation frequency and there 

are no incident waves; 

these can be identified as the sum of the added mass due 
to the inertia of the surrounding fluid and the radiation- 
induced potential damping due to the energy dissipated 
by generated surface waves. 

Environmental disturbances can be identified as the 
generalized forces caused by the wind, the waves, and 
the ocean current. 

The overall equations of motions can therefore be 
written in matrix form as [51.8, 10,1 1] 

M v v + C v (v)v + D v (v)v TgvfRg) = r v , (51.16) 

where M v = M RB + Ma and C v = Crb + Ca also in¬ 
clude the added mass terms. 

In the following sections these generalized forces, 
specific to the marine environment, will be briefly dis¬ 
cussed. 

Added Mass and Inertia 

When a rigid body is moving in a fluid, the addi¬ 
tional inertia of the fluid surrounding the body that is 
accelerated by the movement of the body has to be 
considered. This effect can be neglected in industrial 
robotics since the density of the air is much lower than 
the density of a moving mechanical system. In under¬ 
water applications, however, the density of the water, 
p fa 1000 kg/m 3 , is comparable with the density of the 
vehicles. In particular, at 0 °C, the density of freshwater 
is 1002.68kg/m 3 ; for sea water with 3.5% salinity it is 
p = 1028.48 kg/m 3 . 

The fluid surrounding the body is accelerated with 
the body, so a force is necessary to achieve this accel¬ 
eration, while the fluid exerts a reaction force which is 
equal in magnitude and opposite in direction. This re¬ 
action force is the added mass contribution. The added 
mass is not a quantity of fluid to add to the system such 
that it has an increased mass. Different properties hold 
with respect to the 6 x 6 inertia matrix of a rigid body 
due to the fact that the added mass is a function of the 
body’s surface geometry. 

The hydrodynamic force along Xb due to the linear 
acceleration in the Xb-direction is defined as 

dX 

X A := -X;,u , where X- u := X;, := — , 

ou 

where the symbol 3 denotes the partial derivative. In the 
same way it is possible to define all the remaining 35 
elements that relate the six force/moment components 
(X Y Z K M N) r to the six linear/angular accelera¬ 
tions (ii v w p q r) T . These elements can be grouped 
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into the added mass matrix Ma e R 6x6 . Usually, all the 
elements of the matrix are nonzero. 

In general, added mass and potential damping will 
be frequency dependent and depend on the forward 
speed. This is also the case for certain viscous damp¬ 
ing terms (skin friction, roll damping, etc.). This gives 
a pseudo differential equation describing the frequency 
response of the vehicle. Since some of the coefficients 
depend on the frequency this is not an ordinary differen¬ 
tial equation (ODE). The frequency equation, however, 
can be transformed to the time domain using the con¬ 
cepts described in [51.12] and [51.13], and recently 
in [51.14]. The time-domain equations will in general 
include fluid memory effects [51.4]. However, under¬ 
water robots are usually modelled using maneuvering 
theory where the potential coefficients are evaluated at 
the zero frequency. The resulting equation is an ODE 
where the added inertia matrix M A is constant, speed 
independent, and positive definite 

M a =MX>0, M a = 0. (51.17) 

This result is well known from ship hydrodynam¬ 
ics; [51.15] for instance. The matrix M A can be 
computed using numerical programs such as WAMIT 
or Matlab, based on the US Air Force Digital Dat- 
com [51.16]; in this case, the zero-frequency result 
should be used, that is, Ma = A(0) where A(tu) is the 
frequency-dependent added mass matrix. The potential 
damping matrix will be small compared to the vis¬ 
cous effects and drag/lift terms. Hence, this term can be 
set to zero for underwater vehicles. If the added mass 
is computed experimentally, it is common practice to 
symmetrize the results such that 

Ma = — i^Aexp + A exp ^ , 

where A exp denotes the experimentally obtained added 
mass terms. 

If the body is completely submerged in the wa¬ 
ter and is designed with port/starboard symmetry (xz- 
plane) as is common for underwater vehicles in six 
degrees-of-freedom (DOF), the following structure of 
the matrices M A can be considered 

0 X- q 0 \ 

Yp 0 7 ;. 

0 Z- q 0 
K p 0 Ky 
0 M q 0 
Np 0 Ny , 

(51.18) 


The added mass coefficients can theoretically be de¬ 
rived by exploiting the geometry of the rigid body or 
numerically by strip theory [51.17]. 

In [51.18] the coefficients for the experimental 
AUV Phoenix of the Naval Postgraduate School (NPS) 
are reported. These coefficients have been derived ex¬ 
perimentally, and the geometry gives a nondiagonal Ma 
matrix. To provide an order of magnitude for the added 
mass terms, for the vehicle mass of about 5000 kg, the 
Xy is approximately —500kg. 

The added mass also makes an added Coriolis and 
centripetal contribution. It can be demonstrated that the 
matrix expression can always be parameterized such 
that 

C A (v) = —Ca(v) , Vvel 6 , 
whose symbolic expressions can be found in [51.4]. 

Hydrodynamic Damping 

The hydrodynamic damping for marine vehicles is 
mainly caused by: 

• Potential damping 

• Skin friction 

• Wave drift damping 

• Vortex shedding damping 

• Viscous damping. 

The radiation-induced potential damping due to 
forced body oscillations is commonly known as po¬ 
tential damping; its dynamic contribution is usually 
negligible with respect to, e.g., the viscous friction for 
underwater vehicles while it may be significant for sur¬ 
face vessels. 

Linear skin friction is due to laminar boundary lay¬ 
ers and can affect the low-frequency motion of the ve¬ 
hicle. Together with this effect, at high frequencies it is 
possible to observe a quadratic, or nonlinear, skin fric¬ 
tion phenomenon caused by turbulent boundary layers. 

Wave drift damping is the dominant dynamic damp¬ 
ing effect in surge motion of surface vessels in high sea. 
It can be considered as an added resistance for boats ad¬ 
vancing in waves; its drift is proportional to the square 
of the significant wave height. In the sway and yaw di¬ 
rections, however, its dynamic contribution is negligible 
with respect to the effect of vortex shedding. 

A body moving in a fluid causes a separation of 
the flow; this can still be considered as laminar in the 
upstream while two antisymmetric vortices can be ob¬ 
served in the downstream. In case that the body is 
a cylinder moving in a direction normal to its axis, the 
result is a periodic force normal to both the velocity 
and the axis. This effect may cause the oscillation of 
cables and other underwater structures. However, con¬ 
cerning underwater vehicles, this effect is negligible for 


(Xu 

0 

Xu, 

0 

Y v 

0 

Zk 

0 

Zw 

0 

Ky, 

0 

Mu 

0 

Mu, 


Ni 

0 
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ROVs and may be counteracted by designing proper 
small control surfaces for torpedo-like AUVs. 

Vortex shedding is an unsteady flow that takes place 
at special flow velocities (according to the size and 
shape of the cylindrical body). In this flow vortices are 
created at the back of the body, periodically from each 
side. 

The viscosity of the fluid also causes dissipative 
forces. These are composed of drag and lift forces, the 
former being parallel to the relative velocity of the vehi¬ 
cle with respect to the water while the latter are normal 
to it. For a sphere moving in a fluid, the drag force can 
be modeled as [51 .9] 

^drag = ^pU 2 SC d (Rn) , (51.19) 

where p is the fluid density, U is the velocity of the 
sphere, S is the frontal area of the sphere, C d is the 
nondimensional drag coefficient, and!?,, is the Reynolds 
number. For a generic body, S is the projection of the 
frontal area along the flow direction. The drag force 
can be considered as the sum of two physical effects: 
the frictional contribution of the surface whose normal 
is perpendicular to the flow velocity, and the pressure 
contribution of the surface whose normal is parallel to 
the flow velocity. For a hydrofoil moving in a fluid, the 
lift force can be modeled as [51 .9] 

F m = \pU 2 SQ{R n , a), (51.20) 

where S is now the area, Q is the nondimensional lift 
coefficient, and a is the angle of attack, i. e., the an¬ 
gle between the relative velocity and the tangent to the 
surface. For small angles of attack, i. e., |q;| < 10°, the 
lift coefficient is approximatively proportional to a and 
rapidly decays to zero as a increases [51.19], 

The drag and lift coefficients are therefore de¬ 
pendent on the Reynolds number, i. e., on the lami¬ 
nar/turbulent fluid motion 

p _ P\U\D 

*M1 - i 

V 

where D is the characteristic dimension of the body 
perpendicular to the direction of U and p is the dy¬ 
namic viscosity of the fluid. Table 51.3 reports the drag 
coefficients as a function of the Reynolds number for 
a cylinder [51.20]. 


A common simplification considers only linear and 
quadratic damping terms and group these into a ma¬ 
trix D v as in (5 1.16) such that 

D v (v) > 0 . Vvel 6 . 


Gravity and Buoyancy 

When a rigid body is completely or partially submerged 
in a fluid under the effect of the gravity two more forces 
have to be considered: the gravitational force and buoy¬ 
ancy. The latter is the only hydrostatic effect, i. e., it 
is not a function of the relative movement between the 
body and fluid. 

Let us define as 

g' = (0 0 9.81) T m/5 2 


the acceleration of gravity. This effect is not constant 
but varies with depth, longitude, and latitude; however, 
this value is usually accurate enough for most applica¬ 
tions except for inertial navigation systems. 

For a completely submerged body the computation 
of these dynamic effects is straightforward. The sub¬ 
merged weight of the body is defined as W = Hit'll, 
while its buoyancy B = p V | |g* 11, where V is the vol¬ 
ume of the body and m is its mass. The gravity force, 
which acts at the center of mass r[(, is represented in the 
body-fixed frame by 


/ g (R?) = R? 


(°\ 

0 


\WJ 


while the buoyancy force, acting at the center of buoy¬ 
ancy /g, is represented in the body-fixed frame by 


/b(R?) = -R? 


fo\ 

0 


W 


The 6x1 vector of force/moment due to gravity and 
buoyancy in the body-fixed frame, included in the left- 
hand side of the equations of motion, is represented by 


*v(R?) 


( /g(R?)+/b(R?) \ 

\ r G x / G (Rf) + fg x/ B (Rf )J 


Table 51.3 Lift and drag coefficient for a cylinder 


Reynolds number 

Regime motion 

Cd 

c, 

R n < 2 x 10 5 

Subcritical flow 

1 

3-0.6 

2x 10 5 <R n <5x 10 5 

Critical flow 

T 

o 

4^ 

0.6 

5 x 10 5 < R n < 3 x 10 5 

Transcritical flow 

0.4 

0.6 
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In the following, the symbol = (x G Vg Cg) T (with 
r G = r c) w iU t> e use d f° r t ^ le cen,;er °f gravity. The ex¬ 
pression for g v in terms of the Euler angles is 

gv(Vl) = 

f (W-B)sg \ 

-(VT-S)cej^ 

-(W-B)coc < t > 

~(ycW - ynB)c e c t/> + (z G W - z^ces# 

(zgW-ZbB)S0 + {X G W-XbB)C0C < i > 

y -(x G W-XBB)c0S < p-(y G W-yBB)sg y 

(51.21) 

Current 

Ocean currents are mainly caused by tidal movement, 
the atmospheric wind system over the sea surface, heat 
exchange at the sea surface, salinity changes, the Cori¬ 
olis force due to the Earth’s rotation, nonlinear waves, 
the major ocean circulations such as the Gulf Stream, 
the effect of setup phenomena or storm surges, and 
strong density gradients in the upper ocean. Currents 
can be very different due to local climatic and/or ge¬ 
ographic characteristics; as an example, in fjords, the 
tidal effect can cause currents of up to 3 m/s, moreover, 
specific mathematical models exist for the various com¬ 
ponents [51.8]. 

Let us assume that the ocean current, expressed in 
the inertial frame, id, is constant and irrotational, i. e., 

»c = (Vc,* Vc.y V c , z 0 0 0) T , 

and id = 0 ; its effects can be added to the dynamics of 
a rigid body moving in a fluid simply by considering 
the relative velocity in the body-fixed frame 

v, = v — Rf vj. (51.22) 

in the derivation of the added Coriolis and centripetal 
and the damping terms. 

M v v + Crb(v)v + C A (v r )v r + D v (v r )v r +g v (R{,) 
= t v . (51.23) 

Notice that the term C A (v r )v r includes the important 
destabilizing effect known as the Munk moment [51 .9]. 

If D v (v r ) is unknown, quadratic surge resistance 
and the cross-flow drag principle can be used to de¬ 
scribe the dissipative forces and moments in surge, 
sway, and yaw [51.9]. Moreover 

C A (Vr)v r + D v (v r )v r « {X c Y c 0 0 0 A C ) T 

(51.24) 

for large relative current angles |/3 C — i/r|, where /l c is 
the current direction, the cross-flow principles models 


the sway force 7 C and yaw moment N c as 

T c = j H(x)C n (x)v r ( (x) | vf(x)\dx (51.25) 

L 

N c = I xH{x)C D {x)v*(x)\v*{x)\dx-X M r\r\ , 

L 

(51.26) 

where L is the vehicle length, H(x) is the vehicle 
height, Cb>(x) is the two-dimensional drag coefficient, 
and vf(x) = v r + rx is the relative cross-flow velocity 
at x. In practice Cd(x) can be chosen as a constant be¬ 
tween 0 and 1. The proper value can be determined by 
curve fitting of experimental data. Along the surge di¬ 
rection, however, the quadratic damping contribution X c 
is still well represented by a term proportional to the 
square of the relative velocity, whose symbolic expres¬ 
sion can be written as 

X c = 3f u j u ] M r |n r | , (51.27) 

where —X u \ u \ > 0 is the quadratic surge damping coef¬ 
ficient, which can be found by curve fitting of exper¬ 
imental data or relating it to the drag coefficient Cd 
as in (51.19). The approximation embedded in (51.24) 
does not represent sufficiently enough the dynamics at 
low velocities, when the quadratic terms are negligible. 
Hence, it is common to add a linear optional damper in 
surge, sway and yaw; this can be physically explained 
as skin friction which shows a linear behavior. 

Alternatively, the computation of the quadratic 
surge resistance, nonlinear roll damping, and the cross- 
flow drag effect can be made by resorting to the Datcom 
database for aircraft, as shown in [51.21]. 

Model Properties 

For completely submerged bodies in an ideal fluid mov¬ 
ing at low velocity where there are no currents or 
waves, (51.16) satisfies the following properties: 

• The inertia matrix is symmetric and positive defi¬ 
nite, i. e., 

M v = > 0 . 

• The damping matrix is positive definite, i. e., 

D v (v) > 0 . 

• The matrix C v (v) is skew symmetric, i. e., 

C v (v) = —Cy (v), Vrel 6 . 

Hydrodynamic Modeling 

The mathematical model of an underwater robot as ex¬ 
pressed in (51.16) is of great importance; even when 
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Table 51.4 Dorado payload for scientific sampling (courtesy of Monterey Bay Aquarium Research Institute [51.22]) 


Sensor model 

Description 

WHN300 

300 kHz Acoustic Doppler Current Profiler/Doppler Velocity Log, manufactured by Teledyne/RD Instruments 

8CB4000I 

Paroscientific Digiquartz pressure sensor, 4000 m full scale 

Gulpers 

10x2 liter Water Samplers 

HS2 

Hobilabs 2 channel backscatter/fluorometer, 420 nm/700 nm excitation 

LIS ST-100 

Sequoia Scientific particle size spectrum instrument 

ISUS 

In-situ ultraviolet spectrometer for nitrate measurement 

2xSBE3/SBE4 

Seabird temperature/conductivity instrument, 2 pairs of instruments carried 

SBE43 

Seabird dissolved oxygen cell 

2xSBE5 

Pumps for C/T/DO/ISUS and C/T/LISST flow paths 

UBAT 

Wetlabs bioluminescence assessment tool (bathyphtometer for measuring plankton bio-luminsescence) 

LOPC 

Laser Optical Plankton Counter 

ECO-CDOM 

Fluorometer for measuring CDOM (colored dissolved organic matter) 


simplified it captures the most important part of the 
dynamics. Moreover, it is in a form appropriate for 
control design. A wide literature exists on AUV/ROV 
controllers whose stability relies on the properties re¬ 
ported above. On the other hand, there are working 
conditions in which these assumptions are no longer 
valid, i. e., when the AUV is traveling at high speed, 
or close to the surface, or when its shape does not al¬ 
low geometric simplifications. The latter is the case 
of, e.g., several ROVs. In addition, it is still com¬ 
mon to design the controllers for AUVs based on 
linearized models and to control ROVs with simple PID 
controllers. 

These considerations justify a modeling effort to 
calculate the hydrodynamic terms more accurately with 
the aim of prediction, simulation, and performance 
analysis rather than control design. This can be done 
by switching from a coefficient-based approach, such 
as that presented above, to a component modeling 
method, the latter being based on computational fluid 
dynamics theory. In detail, each vehicle geometry, 
with its specific angle of attack and sideslip, is taken 
into consideration when computing the hydrodynamic 
forces/moments. This increased computational effort 
makes it possible to capture some dynamic effects, such 
as the vortex-induced roll moment, not justifiable with 
the coefficient-based approach. 

The control plant model is usually a simplified 
model that captures the most important parts of the dy¬ 
namics. The most accurate model of the vehicle should 
be used for prediction and motion simulation. 

51.2.2 Sensor Systems 

Underwater vehicles are equipped with a sensor system 
devoted to enabling motion control as well as accom¬ 
plishing the specific mission it has been commanded 
to complete. In the latter case, sensors developed for 
chemical/biological measurements or mapping may be 


installed, which is beyond the scope of this chapter. For 
seek of example, Table 51.4 reports the payload for the 
AUV Dorado, shown in Fig. 51.3. 

AUVs need to operate underwater most of the time; 
one of the major problems with underwater robotics is 
in the localization task due to the absence of a single, 
proprioceptive sensor to measure the vehicle position. 
The global positioning system (GPS) cannot be used 
underwater. Redundant multisensor systems are com¬ 
monly combined using state estimation or sensor fusion 
techniques to provide fault detection and tolerance ca¬ 
pability to the vehicle. Table 51.5 lists the types of 
sensors and the corresponding variable measured com¬ 
monly available for unmanned underwater vehicles 
(UUVs). 

The sensors that can be found on an underwater ve¬ 
hicle are: 

• Compass: A gyrocompass can provide an estimate 
of geodetic north accurate to a fraction of a de¬ 
gree. Magnetic compasses can provide estimates of 



Fig. 51.3 The AUV Dorado (courtesy of Monterey Bay 
Aquarium Research Institute [51.22]) 
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Table 51.5 UUVs: possible instrumentation 


Sensor 

Measured variable 

Inertial system 

Linear acceleration and angular 
velocity 

Pressure meter 

Vehicle depth 

Frontal sonar 

Distance from obstacles 

Vertical sonar 

Distance from the bottom 

Doppler velocity log 

Relative velocity vehicle/bottom 

Current meter 

Relative velocity vehicle/current 

Global positioning 
system 

Absolute position at the surface 

Compass 

Orientation 

Acoustic positioning 

Absolute position in known area 

Vision systems 

Relative position/velocity 

Acoustic Doppler 
current profiler 

Relative water velocity 


magnetic north with an accuracy of less than 1° 
if carefully calibrated to compensate for magnetic 
disturbances from the vehicle itself. Tables or mod¬ 
els can be used to convert from magnetic north to 
geodetic north. 

• Gyroscope: The term gyroscope, or gyro, is the 
name given generically to any instrument measuring 
inertial angular rotation. Those are based on inertial 
properties of a vibrating mass or of light. Very accu¬ 
rate gyros may output signal that, when integrated, 
give heading information with a small enough drift 
for practical purposes. 

• Inertial measurement unit ( IMU ): An IMU provides 
information about the vehicle’s linear acceleration 
and angular velocity. These measurements are com¬ 
bined to form estimates of the vehicle’s attitude in¬ 
cluding an estimate of geodetic (true) north from the 
most complex units. In most cases, for slow-moving 
underwater vehicles, an independent measurement 
of the vehicle’s velocity is also required to produce 
accurate estimates of the translational velocity or 
relative displacement. 

• Depth sensor: Measuring the water pressure gives 
the vehicle’s depth. At depths beyond a few hundred 
meters, the equation of state of seawater must be in¬ 
voked to produce an accurate depth estimate based 
on the ambient pressure [51 .23] . With a high-quality 
sensor, these estimates are reliable and accurate, 
giving a small error of order 0.01%. 


• Altitude and forward-looking sonar: These are used 
to detect the presence of obstacles and distance from 
the seafloor. 

• Doppler velocity log ( DVL ): By processing reflected 
acoustic energy from the seafloor and the water col¬ 
umn from three or more beams, estimates of vehicle 
velocity relative to the seafloor and relative water 
motion can be obtained. Bottom-tracking velocity 
estimates can be accurate to 1 mm/s. 

• Global navigation satellite system ( GLS ): This is 
used to localize the vehicle while on the surface 
to initialize or reduce drift of estimates from an 
IMU/DVL combination. GNSS, such as GPS or 
Galileo, only works at the surface. 

• Acoustic positioning: A variety of schemes exist for 
determining vehicle position using acoustics. Long- 
baseline navigation can determine the position of 
the vehicle relative to a set of acoustic beacons an¬ 
chored to the seafloor or on the surface through 
range estimates obtained from acoustic travel times. 
Ultrashort-baseline navigation uses phase informa¬ 
tion to determine direction from a cluster of hy¬ 
drophones; this is most often used to determine 
the direction of the vehicle (in two dimensions) 
from a surface support vessel, which is then com¬ 
bined with an acoustic travel-time measurement to 
produce an estimate of relative vehicle position in 



Fig. 51.4 The fully actuated AUV ODIN (courtesy 
of Autonomous Systems Laboratory, University of 
Hawaii [51.24]) 


Table 51.6 JHUROV instrumentation 


Measured variable 

Sensor 

Precision 

Update rate 

3 -DOF vehicle position 

SHARP acoustic transponder 

0.5 cm 

10 Hz 

Depth 

Foxboro/ICT model n. 15 

2.5 cm 

20 Hz 

Heading 

Litton LN200 IMU Gyro 

0.01° 

20 Hz 

Roll and pitch 

KVH ADGC 

0.1° 

10 Hz 

Heading 

KVH ADGC 

1° 

10 Hz 
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Table 51.7 ODIN III sensors summary 


Measured variable 

Sensor 

Update rate 

xy vehicle position 

8 sonars 

3 Hz 

Depth 

Pressure sensor 

30 Hz 

Roll, pitch, and yaw 

IMU 

30 Hz 


spherical coordinates. These techniques will be dis¬ 
cussed later in the localization section. 

• Vision systems'. Cameras can be used to obtain 
estimates of relative, and in some cases abso¬ 
lute, motion using a type of simultaneous localiza¬ 
tion and mapping (SLAM) algorithm [51.25] and 
used to perform tasks such as visual tracking of 
pipelines, station keeping, visual servoing or image 
mosaicking. 

As an example, Table 51.6 reports some data from 
the instrumentation of the ROV developed at the John 
Hopkins University [51.26], and Table 51.7 some data 
from the AUV ODIN III [51.27] shown in Fig. 51.4. 
Majumder et al. [51.28] show some data fusion results 
with a redundant sensorial system mounted on the AUV 
Oberon, while [51.29] reviews advances in navigation 
technology. 

51.2.3 Actuating Systems 

Marine vehicles are generally propelled by means of 
thrusters or hydrojets. In the case of ROVs with struc¬ 
tural pitch-roll stability, there are usually four thrusters 
that provide holonomic mobility to the four remained 
DOFs, in particular, the depth is often decoupled and 
the vehicle is controlled on a plane in the surge, sway, 
and yaw DOFs. Those vehicles, being underactuated, 
cannot easily be used for interactive control by means of 
a manipulator due to the impossibility of counteracting 
the generalized forces exchanged with the manipu¬ 
lator’s base; in such case, six or more thrusters are 
required. AUVs generally have a torpedo-like shape and 
are used for mapping/exploration. They are propelled 
using one or two thrusters parallel to the fore-aft direc¬ 
tion and a fin and a rudder; this kind of propulsion is 
obviously nonholonomic and experiences a loss of mo¬ 
bility at low velocities. Hydrojets, also known as pump 
jets or water jets, are systems that create a jet of wa¬ 
ter for propulsion; they have certain advantages over 
thrusters such as a higher power density and usability 
in shallow water, but can provide thrust in one direction 
only. 

Several efforts have been made to accurately 
and efficiently describe the mathematical model of 
a thruster; [51.30] reports a one-state model where the 
state is n, the propeller shaft speed. In [51.31] a two- 


state model is proposed to take into account the ex¬ 
perimentally observed overshoot in the thrust; together 
with n, the additional state variable is u p , the axial 
flow velocity in the propeller disc. In [51.32] a thruster 
model incorporating the effects of rotational fluid ve¬ 
locity and inertia on thruster responses is presented 
together with a method for experimentally determining 
nonsinusoidal lift/drag curves. A three-state model is 
described in [51.33] 

J m h + K„n = x — Q, 
infiip + 4o« P + 4 K|(m p — M a ) = T , 
(m-Xu)u-X u u-X uM u\u\ = (1 -t)T, 

where J m is the moment of inertia for the dc- 
motor/propeller, K„ is the linear motor damping coef¬ 
ficient, r is the motor control input, Q is the propeller 
torque, mf is the mass of water in the propeller control 
volume, ii p is the axial flow velocity in the propeller 
disc, dfo and df are the linear and quadratic damping 
coefficients for control volume, respectively, « a is the 
ambient water velocity, T is the propeller thrust, and t 
is the thrust deduction number (Fig. 51.5). In the case 
of steady-state motion, i. e., u = 0, the ambient water 
velocity w a is related to the surge by the wake fraction 
number w as 

M a = (1 — w)u . (51.28) 

Notice also that the unmeasured variable u p can be esti¬ 
mated using a nonlinear observer [51.33]. 

The outputs of the nonlinear three-state dynamic 
systems are the thrust T and the torque Q, which 
are functions of several variables; in the following, 
unsteady flow effects such as air suction, cavitation, 
the in-and-out-of-water (Wagner), boundary layer, and 



Fig. 51.5 Ambient water and axial flow velocities affect¬ 
ing thruster behavior 
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gust (Kuessner) effects will be neglected. This leads to 
a quasi-steady representation of the model 

T = pD 4 K T (J 0 )n\n\ , (51.29) 

Q = pD s K Q (J 0 )n\n\ , (51.30) 


this case, the problem of allocation of the desired 
force/moment acting on the vehicle among the thrusters 
must also be solved. Reference [51.35] reports a survey 
of control allocation methods of ships and underwater 
vehicles. 


where D is the propeller diameter and Kj(Jq) and 
Kq(Jq) are the thrust and torque coefficients, respec¬ 
tively. The latter are function of the advance ratio Jo 

Ua 

Jo = • (51.31) 

nD 

The open-water propeller efficiency in undisturbed wa¬ 
ter is given as the ratio of the work done by the propeller 
in producing a thrust force divided by the work required 
to overcome the shaft torque 


uj _ Jq Kf 
2 nnQ 2n Kq 


(51.32) 


Figure 51.6 shows the values of Kj, Kq, and r] 0 as func¬ 
tions of the advance ratio for the Wageningen B4-70 
propeller [51.34]. 

Controlling a marine vehicle usually requires that 
desired forces/moments act on the vehicle’s body; these 
generalized forces are mapped into desired thrusts to be 
provided by the propellers. There is, thus, a nontrivial 
control problem in that the motors are required to pro¬ 
vide the appropriate propeller shaft speed n that satisfies 
the nonlinear relationship with the thrust T presented 
above. 

To enable robustness with respect to possible fail¬ 
ures, the actuating system is often redundant. In 


K t , 10 Kq, }];, 



Fig. 51.6 Values of Kj (solid), IOI^q (doited), and rjo (dash-dotted) 
as a function of Jo (after [51.34]) 


51.2.4 Communication Systems 

Autonomous underwater vehicles may require to com¬ 
municate with a remote operator or with a base sta¬ 
tion for monitoring purposes. In case of coordinated 
mission among several AUVs a network of vehi¬ 
cles/modes communicating among them need to be 
developed. 

Due to the physical characteristics of the water, 
the more diffused communication technology is based 
on acoustic propagation. However, the performance of 
acoustic modems are not as efficient as their aerial 
conuterparts. Three main factors affect the acoustic 
propagation: a low speed of sound 1500 m/s), the 
presence of time-varying multiple paths, an attenuation 
increasing with the frequency. Finally, the channel ca¬ 
pacity is function of the distance and is limited [51.36], 
For such a challenging communication medium, ef¬ 
ficient communication protocols need to be properly 
designed [51.37]. 

51.2.5 Mission Control System 

The mission control system (MCS) can be considered 
as the highest-level process running during an AUV’s 
mission; it is responsible for achieving several control 
objectives. At the highest level it works as an inter¬ 
face between the operator, accepting his instructions in 
a higher-level language and decomposing those instruc¬ 
tions into mission tasks according to the implemented 
software architecture. The mission tasks are generally 
concurrent and their handling depends on the vehicle 
state and environmental conditions; it is therefore the 
MCS that handles the tasks, eventually suppressing, se¬ 
quencing, modifying, and prioritizing them. An MCS 
is also usually equipped with a graphical user interface 
(GUI) to report the mission state to the operator, see, 
e.g., the Neptus Command and Control Infrastructure 
in 

As for most advanced robotics applications, an ef¬ 
ficient MCS should allow the use of complex robotic 
systems by users that do not necessarily know all of 
their technical details. An overview relevant to un¬ 
derwater mission control is given in [51.38], which 
includes an interesting classification of the MCSs in use 
in several laboratories according to which four major 
AUV control architectures were identified: the hierar¬ 
chical, heterarchical, subsumption, and hybrid. 
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From a mathematical point of view, the MCS gener¬ 
ally needs to be designed in order to be able to address 
hybrid dynamical systems, i. e., handling both event- 
driven and time-driven processes. In [51.39], e.g., the 
MCS developed at the Portuguese Instituto Superior 
Tecnico (1ST), named CORAL, is implemented by re¬ 
sorting to a Petri-net-based architecture that properly 
handles all the necessary tasks in order to manage 
navigation, guidance and control, sensing, communica¬ 
tions, etc. 

The motion-oriented operating system (MOOS), de¬ 
signed at the Massachusetts Institute of Technology, is 
a software tool capable of executing and coordinating 
a multitude of subsea operations. The MCS developed 
at the Naval Postgraduate School is in the framework of 
the behavioral control organized in three layers [51.40]; 
it is based on PROLOG, an artificial intelligence lan¬ 
guage for predicate logic. 

51.2.6 Guidance and Control 

The terms guidance and control can be defined 
as [51.8]: 

• Guidance is the action of determining the course, 
attitude, and speed of the vehicle, relative to some 
reference frame (usually the Earth), to be followed 
by the vehicle. 

• Control is the development and application to a ve¬ 
hicle of appropriate forces and moments for operat¬ 
ing point control, tracking, and stabilization. This 
involves designing the feedforward and feedback 
control laws. 

Figure 51.7 shows the corresponding block dia¬ 
gram, in which the navigation component is also out¬ 
lined. 


Guidance of Underwater Vehicles 
Guidance algorithms may benefit from a wide range of 
inputs, overall mission information, real-time operator 
input, environmental measured data such as the ocean 
current, environmental topological information such as 
a bathymetric map, exteroceptive sensors for obstacle 
avoidance, and obviously the vehicle state as output 
from the navigation system. 

The vehicle may be required to follow a path, i. e., 
a curve geometrically represented in two or three di¬ 
mensions, or a trajectory, i. e., a path with a specific 
time law assigned. Moreover, when the desired position 
is constant, the problem is called set-point regulation or 
maneuvering. The guidance problem is commonly de¬ 
composed into simple subtasks of lower dimension: an 
attitude control problem and a path control. Moreover, 
attitude is usually considered as a simple depth set-point 
with null roll and pitch and the path is usually a line in 
the horizontal plane. 

One of the most common guidance approaches is 
based on the generations of way-points. Those are usu¬ 
ally stored in a database and are used to generate the 
vehicle path/trajectory; a passing velocity, in fact, may 
be defined together with the Cartesian coordinates of 
the points. The simplest way to connect the way-points 
is to use the segments connecting two successive way- 
points. Efficient way-point-based guidance approaches 
need to take into account the presence of the current and 
the eventual nonholonomicity of the vehicle [51.41]. 
A technique for adaptively tracking bathymetric con¬ 
tours by proper generation of way-points is presented 
in [51.42]; environment information is acquired by 
mean of a single vertical sonar. An alternative method 
is based on line-of-sight guidance [51.43-45]. In this 
case, the heading control is computed by considering as 
input the angle formed by the vector from the vehicle to 
the next way-point rather than requiring the vehicle to 



Fig. 51.7 Guidance, navigation, and control for an autonomous marine vehicle 


Part E | 51.2 





































Part E | 51.2 


1298 Part E I Moving in the Environment 


exactly follow the line segment between the current and 
the following way-point. For docking maneuvers, algo¬ 
rithms must enable control that meets the precision and 
alignment constraints of the docking fixture [51.46]. 

By combining vision-based guidance with a neuro¬ 
controller trained by reinforcement learning, in [51 .47], 
an algorithm aimed at a hold station on a reef or 
swimming along a pipe has been presented. In [51 .48] 
guidance for AUVs specifically involved in a pre¬ 
deployment survey of the sea bottom and visual inspec¬ 
tion of pipelines is given. Hyland and Taylor [51.49] 
report a specific guidance system aimed at mine avoid¬ 
ance for AUVs. Based on a three-dimensional dis¬ 
cretization of the environment, the path-planning tech¬ 
nique consists of computing a safe path avoiding the 
unsafe cells of the map. Due to the poor manoeuvrabil¬ 
ity at low speed under some conditions, the vehicle has 
to make a 360° turn to avoid stopping and to map the 
environment close to it before generating a safe path. 

A deep discussion on guidance for surface and un¬ 
derwater vehicles can be found in [51.4, 8]. 

Control of Underwater Vehicles 
Control of underwater vehicles needs to consider the 
different operating conditions and actuating configu¬ 
rations in which a submerged vehicle is required to 
operate. In particular, there are three main control prob¬ 
lems: 

• An AUV traveling at high speed (> 1 m/s) generally 
equipped with at least one thruster aligned in the 
fore-aft direction and at least two control surfaces 
(stem and rudder). 

• An underactuated ROV, with high metacentric sta¬ 
bility, i. e., structurally stable in roll and pitch, and 
equipped with at least four thrusters. 

• A fully actuated AUV equipped with at least six 
trusters. 

AUVs equipped with control surfaces are under¬ 
actuated vehicles mainly used for survey/exploration 
missions. Inheriting the common practice of submarine 
control, they are not allowed to perform arbitrary mo¬ 
tions in six DOFs but are rather designed to perform 
specific movements such as: cruising along a given di¬ 
rection at constant depth, steering at constant depth, 
or diving. Marine experience and mathematical insight, 
in fact, demonstrate that these movements are lightly 
coupled in dynamic terms. For these vehicles, more¬ 
over, specific manoeuvres such as homing or docking 
require special capabilities [51.46]. This requires the 
design of vehicles that are structurally stable in the roll 
DOF. Cruising requires control of the surge velocity 
u(t); steering requires control of the sway velocity v(t) 
and the yaw DOF r{t), //(/). diving requires control of 


the heave DOF w{t), z(t ) and the pitch DOF q(t), 9(t). 
The simplest configuration of actuators that can con¬ 
trol an AUV through these movements is composed of 
one thruster aligned along the fore-aft direction, one 
stem, and one rudder; the control variables, thus, are the 
propeller speed and the deflection of the fins. Several 
approaches can then be considered to solve this con¬ 
trol problem, among them, in [51.50] the sliding mode 
control is proposed, while [51.51] presents an adaptive 
sliding mode control for the diving manoeuvre. Healey 
and Lienard [51.18] reports a successful implementa¬ 
tion of multivariable sliding mode control on the NPS 
AUV II, later also implemented on the NPS ARIES 
AUV [51.52]. As the model of an AUV traveling at high 
speed is nonlinear and coupled, the tuning of the param¬ 
eters is mainly based on a linearized model around the 
working conditions. 

From a descriptive point of view, an ROV is mainly 
a box-shaped underwater vehicle equipped with tools 
such as a video camera or robot manipulator, while its 
payload is often variable depending on the task. It is 
remotely operated and physically connected to another 
vehicle, either an underwater or a surface vessel. It is 
mainly designed to travel at low speed and it is struc¬ 
turally stable in roll and pitch, while its depth, surge, 
sway, and yaw are independently controllable. Due to 
the absence of a specific shape, the varying payload, and 
the relatively low required performances, it is common 
to control a ROV by means of single-input single-output 
(SISO) controllers. Moreover, the PID approach is of¬ 
ten used due to its simplicity. A two-layer guidance 
and control architecture for the ROV Romeo is given 
in [51.53]. 

Control of a fully actuated AUV in six DOFs is 
needed in the case of, e.g., an interaction task performed 
by a manipulator mounted on a vehicle; the latter, in 
fact, needs to provide all the force/moment compo¬ 
nents in order to counteract the presence of the ma¬ 
nipulator dynamically. This problem is kinematically 
similar to the problem of controlling a satellite in six 
DOFs; the underwater environment, however, makes 
it significatively different from the dynamic point of 
view. In kinematic terms the main issue is in imple¬ 
menting a suitable policy for orientation control; any 
three-parameter representation of orientation, in fact, 
experiences representation singularities (Chap. 2). This 
problem may be overcome by resorting to a redundant 
representation of the orientation such as the quater¬ 
nion. Most of the six-DOF controllers proposed in the 
literature are based on (51.16), which model the simpli¬ 
fied effect of the hydrodynamic terms and which have 
very similar properties as the equations of motion of 
an industrial manipulator. Based on this, it is obviously 
possible to find a collection of approaches inherited 
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from classical robotics, [51 .4, 8] for some examples. 
In [51.54], some specific considerations for the under¬ 
water environment lead to a quaternion-based, adaptive 
controller; it is worth noticing that adaptive control 
requires a suitable, and simplified, expression for the 
hydrodynamic terms. In [51.55] a comparison among 
several 6-DOF controllers is made. I<gf'iMfm and 
l<E£>MZH2E2J show two controllers validation imple¬ 
mented in a pool. 

51.2.7 Localization 

Localization in the underwater environment can be 
a complex task, mainly due to the absence of a single 
external sensor that gives the vehicle position such as, 
e.g., the GPS for outdoor ground vehicles; moreover, 
the environment is often poorly structured. 

One of the most reliable methods is based on 
the use of acoustic systems such as the baseline 
systems: the long-baseline system (LBL), the short- 
baseline system (SBL), and the ultrashort-baseline sys¬ 
tem (USBL). These systems are based on the presence 
of a transceiver mounted on the vehicle and a variable 
number of transponders located in known positions. 
The transceiver’s distance from each transponder can 
be measured via the measurement of an echo delay; 
from this information the position of the vehicle can 
be calculated by basic triangulation operations [51.56], 
The USBL can be used with a single transponder, 
which is usually mounted on a surface ship whose posi¬ 
tion is measured by GPS. Recent works are aimed at 
developing localization algorithms based on a single 
beacon, i. e., range-only measurement among vehicles 
and buoys [51.57]. 

Another localization system is called terrain-aided 
navigation and is based on the use of terrain elevation 
maps; bathymetric maps are available, especially in the 
case of well-known locations such as harbors where 
they usually have a resolution of 1 m. In this case, the 
vehicle position is obtained by filtering the information 
coming from a downward-looking sonar. In [51.58], 
a particle filter approach was used to localize an AUV 
in Sydney harbor. 

Moving vehicles may be equipped with an IMU or 
DVL in order to measure its velocity and/or accelera¬ 
tion. This data can then be integrated to estimate the 
vehicle position. This kind of information is subject to 
the drift phenomenon and may not be reliable for long- 
duration runs or may become cost ineffective if accurate 
IMU devices are needed. 

Relative localization can be obtained by resorting to 
any device that provides information about the relative 
position of the vehicle with respect to the environ¬ 
ment, even in the absence of a map. In this case, by 


filtering the distance measurements taken along the mo¬ 
tion, the vehicle’s position can be measured. This is the 
case, e.g., of sonar or vision-based localization tech¬ 
niques [51.59]. 

Often, the techniques presented above are used to¬ 
gether in a redundant system and the effective position 
is obtained by resorting to sensor fusion techniques 
such as the Kalman filtering approach. 

Simultaneous Localization and Mapping 
Simultaneous localization and mapping (SLAM), also 
known as concurrent mapping and localization (CML), 
is a wide topic in mobile robotics. The problem can 
be formulated as the requirement for a mobile robot 
to be placed in an unknown environment and progres¬ 
sively build a map while locating itself inside the map. 
Chapter 46 discusses this topic in detail. For the ma¬ 
rine environment an additional issue arises due by the 
large-scale map that needs to be used for long-duration 
missions; [51.60] implements a decoupled stochastic 
mapping to handle this computational problem in an 
extended Kalman filter. Terrain-aided navigation with 
the use of a scanning sonar is implemented in [51.61]. 
Newman and Leonard [51 .62] use long-baseline range 
measurements as the input for a nonlinear least-squares 
approach solved by the Gauss-Newton method; both 
the initially unknown position of the transponders and 
the vehicle position are estimated. An interesting sur¬ 
vey on navigation and SLAM for underwater vehicles 
is given in [51.29], 

51.2.8 Underwater Manipulation 

A manipulator may be mounted on an AUV or a ROV 
in order to accomplish interaction operations. In this 
case, the vehicle needs to be fully actuated to counteract 
the forces and moments generated by the manipulator’s 
base. By considering a manipulator with n links, thus 
six DOFs, the underwater vehicle manipulator system 
(UVMS) is a (6 + n)-DOF robotic system whose veloc¬ 
ity vector is 

< = (vj vj q T ) T , (51.33) 

where q e M" is the vector collecting the manipulator 
joints positions. 

Repeating the same considerations as for an under¬ 
water vehicle, it is possible to write the equations of 
motions of an UVMS in matrix form as 

M(f){ + C(q, M + D( ? , M +g(‘L Rb) = T , 

(51.34) 

where M e j^(6+«)x(6+n) j s t jj e i ner ti a matrix, includ¬ 
ing added mass terms, C (q, £)£ e M 6 "*"" is the vector of 
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the Coriolis and centripetal terms, D(<jr, £)£ e M 6 +" is 
the vector of dissipative effects, andg(<jr, Rj 5 ) e R 6 +" is 
the vector of gravity and buoyancy effects. The relation¬ 
ship between the generalized forces r and the control 
input is given by 



where u e R Pv +' ! is the vector of the control input. No¬ 
tice that, while for the vehicle a generic number p v > 6 
of control inputs is assumed, for the manipulator it is 
supposed that n joint motors are available. 

Under this hypothesis, which can be considered as 
reasonable at low velocity, it holds that: 

• The inertia matrix M of the system is symmetric and 
positive definite. 

• For a suitable choice of the parametrization of C and 
if all the single bodies of the system are symmetric, 
M — 2C is skew symmetric. 

• The matrix D is positive definite. 

In [51.20] the mathematical model written with 
respect to the Earth-fixed-frame vehicle position and 
the manipulator end-effector can be found. However, 
it must be noted that, in this case, a six-dimensional 
manipulator is considered in order to have a square Ja¬ 
cobian to work with; moreover, kinematic singularities 
need to be avoided. 

The equations of motion of UVMSs in matrix form 
presented in (51.34) are formally similar to the equa¬ 
tions of motion of ground fixed manipulators (Chap. 3) 
for which a wide control literature exists. This has sug¬ 
gested a suitable translation/implementation of existing 
control algorithms. However, some differences, crucial 
from the control aspect, need to be underlined. UVMSs 
are complex systems characterized by several strong 
constraints: 

• Uncertainty in the model knowledge, mainly due to 
the poor knowledge about the hydrodynamic effects 

• The complexity of the mathematical model 

• The kinematic redundancy of the system 

• The difficulty in controlling the vehicle in hovering, 
mainly due to poor thruster performance 

• The dynamic coupling between the vehicle and the 
manipulator 

• The low bandwidth of the sensor readings. 

In 1996 McLain et al. [51.63] presented a control 
law for UVMSs with some interesting experimental 
results conducted at the Monterey Bay Aquarium Re¬ 
search Institute (MBARI). A one-link manipulator was 



Fig. 51.8 An underwater vehicle-manipulator system 
(courtesy of FP7-TRIDENT Project Consortium) 


mounted on the OTTER vehicle controlled in all six 
DOFs by means of eight thrusters. A coordinated con¬ 
trol system was then implemented to improve the track¬ 
ing errors of the end effector. 

The monograph [51 .64] is focused on the modeling 
and control issues of such systems, and can be con¬ 
sidered as a reference for further reading. Moreover, 
interaction with the environment is also discussed. 

Currently, remotely operated manipulator are stan¬ 
dard equipment for several underwater ROVs. However, 
autonomous manipulation is still a research challenge. 
The SAUVIM vehicle, developed at the Autonomous 
Systems Laboratory, University of Hawaii, one of the 
first semiautonomous underwater vehicle manipulator 
systems. Similar research projects, ALIVE and TRI¬ 
DENT were funded by the Framework Program of the 
European Community [51.2], Figure 51.8 reports the 
UVMS developed under the TRIDENT project [51.3], 
shows a sampling performed with the 

Nereus arm. 

51.2.9 Fault Detection/Tolerance 

Generally, AUVs must operate over long periods of 
time in unstructured environments in which an unde¬ 
tected failure could cause the loss of the vehicle. Failure 
detection and a fault-tolerant strategy are required to de¬ 
termine whether a mission must be terminated in the 
safest manner possible or if the vehicle can continue in 
a diminished capacity. An example is the case of the 
arctic mission of Theseus [51.65]. 

In the case of the use of ROVs, a skilled human op¬ 
erator is in charge of commanding the vehicle; a failure 
detection strategy is then of help in the human decision¬ 
making process. Based on the information detected, the 
operator can decide on vehicle rescue or to terminate 
the mission by, e.g., turning off a thruster. 
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Fault detection is the process of monitoring a sys¬ 
tem in order to recognize the presence of a failure; 
fault isolation or diagnosis is the capability to deter¬ 
mine which specific subsystem is subject to failure. 
Often in the literature there is a certain overlap in the 
use of these terms. Fault tolerance is the capability to 
complete the mission in the case of the failure of one 
or more subsystems; it is also known as fault control, 
fault accommodation, or control reconfiguration. In the 
following the terms fault detection/tolerance will be 
used. 

The characteristics of a fault detection scheme are 
the capability to isolate the detected failure, sensitivity 
in terms of the magnitude of the failure that can be de¬ 
tected, and robustness in the sense of the capability to 
continue working properly in non-nominal conditions. 
The requirements of a fault-tolerant scheme are reliabil¬ 
ity, maintainability, and survival. The common concept 
is that, to overcome the loss of capability due to a fail¬ 
ure, a kind of redundancy is required in the system. 

In this section, a survey of existing fault-detection 
and fault-tolerant schemes for underwater vehicles is 
presented. For these specific systems, if proper strate¬ 
gies are applied, a hardware/software (HW/SW) sensor 
or thruster failure can be successfully handled. In some 
conditions, the fault-detection scheme must also be 
able to diagnose some external abnormal working con¬ 
ditions such as a multipath phenomena affecting the 
echo-sounder system. It is worth noticing that, for au¬ 
tonomous systems such as AUVs, space systems or 
aircraft, a fault-tolerant strategy is necessary to safely 
recover the damaged vehicle and, obviously, there is no 
panic button in the sense that the choice of turning off 
the power or activating some kind of brake is not avail¬ 
able. 

Most fault-detection schemes are model based 
[51 .66, 67] and consider the dynamic relationship be¬ 
tween the actuators and vehicle behavior or the spe¬ 
cific input-output thruster dynamics. In general, fault- 
detection/tolerance theory has been applied to the spe¬ 
cific case of the underwater environment even if only 
a few papers report experimental results; see [51 .68] for 
a survey on this topic. 

Most fault-tolerant schemes consider a thruster- 
redundant vehicle that, after a fault has occurred in 
one of its thrusters, is still actuated in six DOFs. 
Based on this assumption a reallocation of the desired 
forces on the vehicle over the working thrusters is per¬ 
formed [51 .69] . Of interest is also the study of reconfig¬ 
uration strategies if the vehicle becomes underactuated. 

Possible Failures 

Underwater vehicles are currently equipped with sev¬ 
eral sensors in order to provide information about their 


localization and velocity. The problem is not easy. No 
single, reliable sensor is available that gives the re¬ 
quired position/velocity measurement, or information 
about the environment such as the presence of obsta¬ 
cles. For this reason the use of sensor fusion by, e.g., 
a Kalman filtering approach, is a common technique 
to provide the controller with the required variables. 
This structural redundancy can be used to provide fault- 
detection capabilities to the system. 

For each of the sensors listed in Sect. 51.2.2 failure 
can consist of an output of zero if, e.g., there is an elec¬ 
trical trouble, or a loss of meaning. It can be considered 
as sensor failure also an external disturbance such as 
a multipath reading of the sonar that can be interpreted 
as a sensor fault and correspondingly detected. 

Thruster blocking occurs when a solid body is 
present between the propeller blades. It can be checked 
by monitoring the current required by the thruster. This 
was observed, e.g., during the Antarctic mission of 
Romeo [51.70], in that case caused by a block of ice. 
During the same mission a thruster also flooded with 
water. The consequence was an electrical dispersion, 
causing an increasing blade rotation velocity and thus 
a thruster force higher then desired. 

A possible consequence of different failures of 
the thrusters is the zeroing of the blade rotation. The 
thruster in question thus simply stops working. This 
has been intentionally experienced during experiments 
with, e.g., ODIN [51.67,69], Roby 2 [51.66], and 
Romeo [51.70]. 

Other failures include a hardware/software crash or 
the occurrence of fin sticking or loss. A very common 
type of failure involves the loss of electrical isolation 
due to seawater intrusion into underwater electrical ca¬ 
bles or connectors. Such a condition can be detected 
through a technique called ground-fault monitoring. 
Should this occur, electrical power must be removed 
from the affected device. 

51.2.10 Multiple Underwater Vehicles 

A growing research effort has recently been devoted 
to developing strategies to design coordinated con¬ 
trol for underwater vehicles. The use of multiple 
AUVs, in fact, might improve overall mission perfor¬ 
mance as well as provide greater tolerance to failures. 
Specific applications of this method in the underwa¬ 
ter environment might include the naval mine coun¬ 
termeasure problem, harbor monitoring, and inspec¬ 
tion, exploration, and mapping of large areas. AUVs 
might be coordinated with one or more surface ves¬ 
sels or connected to ground or aerial vehicles to form 
a coordinated network of heterogeneous autonomous 
robots. 
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Beside several institutions that have developed sim¬ 
ulation packages for multiple-AUV operations, the use 
of real multiple AUVs is being considered for the adap¬ 
tive sampling and forecasting plan of the Autonomous 
Ocean Sampling Network, formed by several research 
institutions such as (for the robotic components) Cal¬ 
tech, MBARI, Princeton, and WHOI [51.71]. Adap¬ 
tive sampling is also investigated within the project 
Adaptive Sampling and Prediction (ASAP) [51.72], 
The Australian National University is currently work¬ 


51.3 Applications 

Underwater robots currently play prominent roles in 
a number of scientific, commercial, and military tasks. 
Remotely teleoperated vehicles are very well estab¬ 
lished in all these areas, and are becoming increasingly 
automated to relieve the burden on human operators 
and to improve performance. Increasingly, autonomous 
underwater vehicles are finding application in these 
areas as well. Presently, AUVs are used almost ex¬ 
clusively for survey work, but sampling and other 
intervention tasks are becoming more feasible. Addi¬ 
tionally, the line between ROVs and AUVs continues 
to blur, as systems that have the best properties of both 
evolve. 

The offshore oil and gas industry relies heavily 
on ROVs for installation, inspection, and servicing of 
platforms, pipelines, and subsea production facilities. 
As the search for oil and gas goes deeper, this trend 
can only continue. The Marine Technology Society 
estimates that there are over 435 work-class ROVs oper¬ 
ating in the commercial offshore industry today. AUVs 
are now beginning to appear in the commercial off¬ 
shore industry for survey tasks, and concepts for hybrid 
systems that can perform intervention tasks are now ap¬ 
pearing. The goal is not only for these robotic vehicles 
to replace human divers or human-occupied vehicles, 
but to enable an entire new generation of subsea equip¬ 
ment that is serviced without intervention by drill ships 
or other heavy-lifting vessels. This holds the prospect 
of greatly reduced cost. 

Scientific demand for ROVs and AUVs is also in¬ 
creasing dramatically. Scientific applications for ROVs 
include survey, inspection, and sampling tasks previ¬ 
ously performed by human-occupied submersibles or 
towed vehicles. While ROVs operating for science are 
not nearly as numerous as those in the offshore oil 
and gas industry, they are becoming commonplace. 
Most nations involved in global seafloor studies have 
several vehicles. Like the vehicles for the commer¬ 


ing on a shoal of small, autonomous robots, named 
Serafina [51.73]. At Instituto Superior Tecnico (1ST), 
work is ongoing on the coordination between an AUV 
and a catamaran [51.74], i. e., a multirobot system 
constituted by heterogeneous autonomous vehicles. 
In [51.75] the cooperation between AUVs and a sensor 
network is experimentally investigated. 
and l<s>M3H5EBI show two examples of multi-vehicle 
bathymetry mission and multi-vehicle patrolling, re¬ 
spectively. 


cial offshore sector, these vehicles are becoming in¬ 
creasingly automated. High-quality electronic imaging, 
including high-definition television, is becoming in¬ 
creasingly common. Scientific ROVs are now equipped 
with sophisticated sampling devices for sampling ani¬ 
mals, microbes, caustic hydrothermal vent fluids, and 
a variety of rock samples. Moreover, ROVs are also 
used to deploy and operate seafloor experiments, which 
can involve difficult tasks such as drilling and delicate 
placement of instruments. 

ROVs have also emerged as powerful tools for in¬ 
vestigating underwater shipwrecks and other cultural 
sites. Applications include forensic investigations of 
modem shipwrecks to determine the cause of sinking, 
archaeology, and salvage. For archaeology, the goals are 
the same as for excavation on land: detailed mapping 
followed by careful excavation. Beyond diver depths, 
ROVs are the preferred method for these investigations. 
Great progress has been made in the detailed map¬ 
ping phase, and capabilities for excavation are evolving. 
Unfortunately, the same technology also opens the pos¬ 
sibility for shipwrecks to be looted for financial gain, 
which usually results in the loss of the most valuable 
historical information. 

After a long period of skepticism, AUVs are now 
accepted for scientific tasks. Presently, AUVs most of¬ 
ten perform mapping tasks while tended by a vessel. 
Specific mapping tasks include seafloor bathymetry, 
sidescan sonar imaging, magnetic field mapping, hy¬ 
drothermal vent localization, and photo surveys. AUVs 
have been shown to improve productivity and data qual¬ 
ity compared to towed and tethered systems. They have 
also operated in environments where no other means of 
gathering data is possible, such as under ice shelves. 
Likewise, the increasing availability of sophisticated 
in situ chemical sensors, biological sensors, and mass 
spectrometers now allows AUVs to build spatial and 
temporal maps of environmental features that could pre- 
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viously only be studied by bringing samples back to the 
laboratory. AUV systems that can dock to subsea nodes 
to recharge batteries, offload data and receive new in¬ 
structions are presently being tested. 

The military has always been a leader in the devel¬ 
opment of underwater robotic capabilities. They pio¬ 
neered ROVs for tasks such as recovering test weapons 
and deep-sea salvage, and present-day commercial and 
scientific ROVs have descended directly from these 
early systems. Likewise, military interests are presently 
pushing AUV technology very hard. Many different 
countries operate AUVs for military surveys, gather¬ 
ing environmental data as well as searching for hazards 
such as mines. An operational success was achieved 


in surveying for mines in the Persian Gulf harbor of 
Umm Qasr using REMUS vehicles. AUVs in devel¬ 
opment will not only be able to detect mines, but to 
disable them. Bolder, more innovative concepts are also 
in development. These include networks of AUVs that 
can act as extensions of conventional surface vessels 
and submarines, enabling surveillance over wide ar¬ 
eas for extended periods of time at costs far less than 
could be achieved with conventional surface vessels, 
submarines, and aircraft. These developments will rely 
on improvements in acoustic communications, energy 
systems, sensors, and onboard intelligence that will 
likely find their way into commercial and scientific 
practice. 


51.4 Conclusions and Further Reading 


The underwater environment is extremely hostile for 
human engineering activities. In addition to high pres¬ 
sures and hydrodynamic forces that are both nonlinear 
and unpredictable, water is not an appropriate media for 
electromagnetic communication except at short ranges. 
This pushes underwater technology to rely on acous¬ 
tic communication and positioning systems that are 
characterized by low bandwidth. On the other hand, 
the ocean is extremely important for numerous human 
activities from the commercial, cultural, and environ¬ 
mental points of view. 

Research on underwater robotic applications is ac¬ 
tive both from the technological and methodological 
aspects. The power endurance of commercially AUVs 
is currently up to 50 h; this will increase as energy- 
storage devices improve. Improved energy and power 
capability will enable longer missions, higher speeds, 
or better/additional sensors such as, e.g., more pow¬ 
erful lighting for underwater video/photography. The 
current trend for the price of AUVs prices is down¬ 
ward, with more and smaller research institutions 
building or buying AUVs to enrich their research re¬ 
sults; moreover the setup of multiple-AUV systems 
is becoming cost effective. The goal is to develop 
fully autonomous, reliable, robust, decision-making 
AUVs. 

There are a number of technology issues that are 
needed in order to improve AUV capabilities: to in¬ 
crease the underwater bandwidth of current acoustic 
modems, to increase onboard power to handle larger 
tools and interact more strongly with the environment, 
to create AUVs with significant hovering capability to 
allow better interaction, and to enable easier launch and 
recovery. 


In the near future, the ROV/AUV dichotomy will 
likely become less prominent, with a variety of systems 
appearing that have attributes of both systems: 

• For offshore oil and gas intervention tasks, a vehicle 
could transit to the work site as a self-powered, fully 
autonomous vehicle, then dock to the work site. Uti¬ 
lizing energy and communications infrastructure at 
the work site, the vehicle could then be operated 
much like a conventional ROV. 

• Battery-operated ROVs can communicate with the 
surface by very lightweight fiber-optic links, en¬ 
abling the mobility of an AUV but with a high- 
bandwidth connection to skilled human operators 
for complex intervention or scientific sampling 
tasks. 

• Acoustic and optical data links can provide moder¬ 
ate to high communication bandwidths over short 
ranges, enabling human supervision without any 
tether restrictions. At longer ranges, more modest 
acoustic bandwidths are available. 

These developments make marine robotics a chal¬ 
lenging engineering problem with strong connections to 
several engineering domains. Sending an autonomous 
vehicle into an unknown and unstructured environment 
with limited online communication requires some on¬ 
board intelligence and the ability for the vehicle to react 
in a reliable way to unexpected situations. 

A major challenge concerning underwater robotics 
is the interaction with the environment by means of one 
or more manipulators. Autonomous UVMSs are still 
the object of research; the current trend is in devel¬ 
oping the first semiautonomous robotic devices, which 
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might be acoustically operated; moreover, if physically 
possible, the capability to dock to the structure where 
the intervention is needed might significatively sim¬ 
plify the control. The final aim might be to develop 
a completely autonomous UVMS, able to localize the 
intervention site, recognize the task to be performed, 
and act on it without docking to the station and with¬ 
out human intervention. This might make it possible to 
perform missions that are currently impossible such as 
autonomous archaeological intervention at deep sites. 
This would also enable the oil and gas industry to sig¬ 
nificatively decrease costs and risks to humans. 

The Marine Systems Simulator [51.76] is a Mat- 
lab/Simulink library and simulator for marine systems. 


It includes models for ships, underwater vehicles, and 
floating structures. The library also contains guidance, 
navigation, and control (GNC) blocks for real-time 
simulation. A numerical simulator including also the 
presence of a manipulator has been developed under the 
TRIDENT project [51.77], 

For further reading on the topic of underwater 
systems, the reader is referred to several survey arti¬ 
cles, including [51.6,29,35,38,68]. Additionally, sev¬ 
eral journals cover oceanic engineering topics, in¬ 
cluding robotics aspects. A variety of symposia and 
workshops have been held on a regular basis. Some 
books/monographs treating marine robotics are [51 .4, 
8,9,17,64], 
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Dive with REMUS 

available from http://handbookofrobotics.org/view-chapter/51/videodetails/87 
Underwater vehicle Nereus 

available from http://handbookofrobotics.org/view-chapter/51/videodetails/88 
Mariana Trench: HROV Nereus samples the Challenger Deep seafloor 
available from http://handbookofrobotics.org/view-chapter/51/videodetails/89 
REMUS SharkCam: The hunter and the hunted 

available from http://handbookofrobotics.org/view-chapter/51/videodetails/90 
The Icebot 

available from http://handbookofrobotics.org/view-chapter/51/videodetails/92 
Two underwater Folaga vehicles patrolling a 3-D area 

available from http://handbookofrobotics.org/view-chapter/51/videodetails/94 
Adaptive LI depth control of a ROV 

available from http://handbookofrobotics.org/view-chapter/51/videodetails/267 
Saturation based nonlinear depth and yaw control of an underwater vehicle 
available from http://handbookofrobotics.org/view-chapter/51/videodetails/268 
Multi-vehicle bathymetry mission 

available from http://handbookofrobotics.org/view-chapter/51/videodetails/323 
Neptus command and control infrastructure 

available from http://handbookofrobotics.org/view-chapter/51/videodetails/324 
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52 . Modeling and Control 
of Aerial Robots 


Aerial robotic vehicles are becoming a core field 
in mobile robotics. This chapter considers some of 
the fundamental modelling and control architec¬ 
tures in the most common aerial robotic platforms; 
small-scale rotor vehicles such as the quadrotor, 
hexacopter, or helicopter, and fixed wing vehi¬ 
cles. In order to control such vehicles one must 
begin with a good but sufficiently simple dynamic 
model. Based on such models, physically moti¬ 
vated control architectures can be developed. Such 
algorithms require realisable target trajectories 
along with real-time estimates of the system state 
obtained from on-board sensor suite. This chap¬ 
ter provides a first introduction across all these 
subjects for the quadrotor and fixed wing aerial 
robotic vehicles. 
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52.1 Overview 

The term aerial robotics is often attributed to Robert 
Michelson [52.1], as a term to capture a new class 
of autonomous and intelligent small flying machines. 
The quest for autonomy of aerial vehicles goes back 
almost to the origins of powered flight. The first au¬ 
topilot for a fixed wing vehicle (the Sperry autopilot) 
was demonstrated in 1912, only a decade after the first 
powered flight by the Wright brothers. As early as 
World War I, the potential for an autonomously stabi¬ 
lized and remotely controlled airplane to be an effective 
weapon was understood and the Curtis-Sperry flying 
bomb first demonstrated autonomous unmanned flight 
in 1918 [52.2]. Autopilot technology was perfected dur¬ 
ing the 1930s as improving aircraft technology lead 


to longer flight times and the need to relieve the pilot 
from constant attention to flight stability of the vehi¬ 
cle. Autonomous and radio controlled target aircraft 
were developed in the late 1930s and were extensively 
used during World War II for training. The German V-l 
cruise missile, commonly known as the buzz bomb or 
doodlebug, was a highly successful autonomous vehi¬ 
cle; its relatively low impact in World War II being due 
to more British intelligence efforts than to the vehicles 
capabilities. 

Following the World War II, the relative matu¬ 
rity of autopilot technology and airplane construction 
meant that the construction of unmanned fixed-wing 
vehicles was straightforward. Remote-controlled heli- 
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copters, such as the Gyrodyne QH-50 DASH, were 
developed as early as the 1960s. The Yamaha R-50 
and subsequent Yamaha R-Max (Fig. 52.1), was devel¬ 
oped during the 1980s and provides a remote controlled 
commercial aerial platform, with significant onboard 
autonomy, used primarily for agricultural applications. 

Extensive development of fixed-wing autonomous 
vehicles was held back in the mid-20th century by the 
difficulty of localizing a vehicle when far from the base 
station. Indeed, the requirements of navigation systems 
for cruise missiles developed during the cold war was 
one of the main drivers for the development of satel¬ 
lite global positioning system (GPS) technology. Full 
operational capability of the United States GPS sys¬ 
tem with 24 satellites was declared in April 1995 and 
it was mandated as a dual-use system (both commer¬ 
cial and military) in 1996. Smaller scale commercial 
unmanned aerial vehicles were also constrained by the 
lack of robust and small-scale avionics systems. The 
availability of small low power computers, the devel¬ 
opment of microelectromechanical systems (MEMS) 
that provide affordable and robust Inertial Measurement 
Units (IMUs), and access to reliable GPS, lead to the 
beginning of the modem era of nonmilitary UAV and 
aerial robotic systems in the mid-1990s. 

The largest class of commercial and military robotic 
aerial vehicles fly predominantly in obstacle free 
airspace. For such vehicles, once the aircraft has taken 
off, there is no need to employ obstacle avoidance 
techniques or interact with cluttered three-dimensional 
environments. 


Definition 52.1 

An unmanned aerial vehicle (UAV) is a system capa¬ 
ble of sustained flight with no direct human control and 
able to perform a specific task. 



Fig. 52.1 Autonomous aerial robotic vehicle based on the 
Yamaha RMAX platform developed in the University of 
New South Wales, Australia (after [52.3,4]) 


Navigation and control of an unmanned aerial vehi¬ 
cle is typically based on stabilizing a reference heading 
and altitude derived from the error between preset 
GPS way-points and the present vehicle position us¬ 
ing a classical autopilot control system. The sensor 
suite used for control and navigation is usually a GPS 
and IMU along with barometric pressure. These sen¬ 
sor suites can be thought of as proprioceptive sensors, 
since they measure the internal state of the vehicle with¬ 
out reference to the external world. The kinds of tasks 
that UAVs are ideal for include high level surveillance 
and sensing tasks where the payload sensing suite is 
separate from the vehicle systems, with applications 
in agriculture, environmental monitoring, geophysical 
surveys, search and rescue, and security surveillance, as 
well as deployment of materials, where once again the 
payload is separate from the vehicle system, with appli¬ 
cations in agriculture, search and rescue, and of course 
military missions. Such systems are a natural develop¬ 
ment of traditional flight technology and much of the 
research and development that goes into these vehicles 
is undertaken in aerospace departments in universities 
and aerospace companies. 


Definition 52.2 

An aerial robotic vehicle is an aerial vehicle capable 
of autonomously interacting with a complex dynamic 
three-dimensional environment and achieving complex 
environment-dependent goals. 


The nature of the interaction with the environment 
for an aerial robotic vehicle, means that the presence 
of exteroceptive sensors, those that sense the environ¬ 
ment around the vehicle, and how these sensor inputs 
are integrated into the vehicle guidance is the defining 
property of such a vehicle. Typical examples of exte¬ 
roceptive sensors for aerial robotic vehicles are vision 
systems, laser range finders, acoustic sensors, etc., and 
external sensor systems such as VICON [52.5] and Op- 
titrack [52.6] systems. The close coupling between the 
goal of the vehicle, its sensing suite, and the dynamic 
nature of the environment makes the simple way-point 
navigation control architecture of an unmanned aerial 
vehicle unsuitable for aerial robotic vehicles. The kinds 
of tasks that aerial robotic vehicles are ideal for are 
small-scale interactive tasks such as inspection of civil 
infrastructure including dam walls, girders of bridges, 
industrial pressure vessels; and surveillance tasks such 
as, inspection of damaged or burning buildings, moni¬ 
toring of crowds, etc. Future applications may involve 
aerial manipulation including repair of infrastructure, 
and material handling in the construction and agricul¬ 
tural industries. Although the flight technology remains 
important, the sensing and control tasks become cru- 
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cial in such applications and the development that goes 
into these vehicles tends to be undertaken in robotics 
departments in universities and in the growing field of 
new aerial-robotics start-up companies. The ability to 
move in three-dimensional space brings new research 
challenges to the robotics community compared to the 
wheeled mobile robot technology that has motivated 
mobile robotics research over the last couple of decades. 


The term unmanned aerial system (UAS) is useful 
to refer to the combination of the infrastructure, human 
interfaces, aerial platform, and sensing and control sub¬ 
systems. It is clear that certain vehicles may spend time 
operating in either UAV mode or aerial robotic mode 
and the terms are more useful to describe the way of 
thinking about the operation of a system rather than 
classifying a vehicle. 


52.2 Modeling Aerial Robotic Vehicles 


There is already a vast range of aerial vehicle and UAV 
designs in existence, and there are even more ideas un¬ 
der development by companies and research groups as 
we write. In one short chapter, we must limit the scope 
of our presentation to a couple of the key examples. 
There are two major categories of vehicles that are used 
as platforms for aerial robotics: small-scale rotor vehi¬ 
cles such as the quadrotor, hexacopter, or helicopter, 
and fixed wing vehicles. The word quad is derived 
from the Latin quadrangulum for a four-sided figure. 
Rotor is derived from the Latin rotationem for a ro¬ 
tating object. However, hexa is derived directly from 
Greek for six. Heli is derived from ancient Greek he- 
likos for spiral, while the term copter is derived from 
the modern term helicopter, a combination of helikos 
with the ancient Greek pteron for wing. Based on this 
etymology, we propose the terms quadrotor and hex¬ 
acopter to describe the most common modern rotary 
wing aerial robotic vehicles in preference to the ety¬ 
mological questionable terms quadcopter and hexarotor 
that are also in common usage in the literature. We will 
present modeling and control material relevant to these 
two categories, focusing on the basic structure of the 
model. We will also focus on vehicles in 500g-4kg 
weight range as this corresponds to the primary general 
robotics applications where avionics and sensor suites 
take up the majority of the payload. Other classes of 
autonomous aerial vehicles that we are unable to cover 
in this chapter include lighter-than-air vehicles such as 
blimps and balloons, flapping wing vehicles, ducted 
fans, and rockets. 

52.2.1 Rigid Body Motion of the Airframe 

Heavier-than-air aerial robotic vehicles consist of 
a rigid airframe coupled with aerodynamic mechanisms 
to generate lift and thrust. For vehicles in the weight 
range considered, the compactness and structural in¬ 
tegrity of the airframe means that there is little flexing 
of the airframe in normal operation and the rigid-body 
assumption is an effective model. In this case, the ve¬ 


hicle model can be developed based on rigid-body 
dynamics with exogenous forces and torque generated 
by an aerodynamic model. 

Let ! e 1 . 62 , <? 3 1 be the coordinate axis unit vec¬ 
tors e\ = (1,0, 0 ) T , e 2 = (0.1,0) T , and e 3 = (0,0,1) T 
without specification of a frame of reference. Let {B} 
be a (right-hand) body fixed frame for the airframe 
with unit vectors {A1.A2.A3} where these vectors are 
the axes of frame {B} with respect to frame {A} as 
shown in Fig. 52.2. Note that the convention for choice 
of axes shown in Fig. 52.2 is the convention com¬ 
mon in the aerial robotics community [52.7-14], but 
is opposite to the usual convention in the aerospace 
community where the hi and A? axes are usually re¬ 
versed in order so that A3 points down in the direction 
of gravity. In this chapter, we have chosen to follow 
the robotic community convention and this will lead to 
some nonconventional definitions for the modeling of 
fixed-wing airplanes. We will discuss this further in the 
specific sections on fixed-wing airplane modeling and 
control. 

The orientation of the rigid body is given by a rota¬ 
tion matrix A R B = R = [Ai.Aj, A 3 ] e SO(3) in the spe- 



Fig. 52.2 The body-fixed frame and the directions of rota¬ 
tion for the propellers 
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cial orthogonal group. We have b\ = Re \ , hi = Re 2 , 
b$ = Re 3 by construction. 

We will use Z-X-Y Euler angles to model this ro¬ 
tation as shown in Fig. 52.3. Note that this is not the 
normal roll-pitch-yaw convention used in aerospace. To 
get from {A} to { B }. we first rotate about e^ by the yaw 
angle, x//, and we will call this intermediary frame {D} 
with a basis {di,d 2 ,di} where d, is expressed with re¬ 
spect to frame {A}. This is followed by a rotation about 
the x-axis in the rotated frame through the roll angle, 
< p , to intermediary frame {£}, followed by a third pitch 
rotation about the new y-axis through the pitch angle 
6 which results in the body-fixed triad {Z»i, ^ 3 }- The 

associated rotation matrix is 

R = 

( C1//C9 S^S^S# C^S^f C-^SQ T CgS^SyA 
T C^s^sg S^sg C^cgs^ I . 

—C</>S 0 C^C g ) 

(52.1) 

where c and s are shorthand for cosine and sine, respec¬ 
tively. 

Let £ denote the position of the center of mass of 
the airframe in frame {A} and assume that this is also 
the origin of frame {5}. Let v G {A} denote the linear 
velocity of {5} with respect to {A} expressed in {A}. 
Let 12 G {B} denote the angular velocity of j B } with 
respect to {A}, this time expressed in {B}. Let m de¬ 
note the mass of the rigid object and I G R 3x3 denote 
the constant inertia matrix (expressed in the body fixed 
frame {B}). The rigid body equations of motion of the 
airframe are [52.15, 16] 

l = v, (52.2a) 

mil = —mgej, + RF , (52.2b) 



Fig. 52.3 The vehicle model. The position and orientation 
of the robot in the global frame are denoted by £ and R, 
respectively 


R = R£2 X , (52.2c) 

112 =-12 x 112 + r . (52.2d) 

The notation 12 x denotes the skew-symmetric matrix 
such that 12 x v = 12 x v for the vector cross-product 
x and any vector dgI 3 . The vectors F, r G {B} com¬ 
bine the principal nonconservative forces and torques 
applied to the airframe by the aerodynamics of the 
propulsion systems and lifting surfaces of the vehicle. 

52.2.2 Modeling for Quadrotors 

Quadrotors are currently the most popular research 
aerial robotics research platform because they are 
highly maneuverable and enable safe and low-cost 
experimentation in mapping, navigation, and control 
strategies in three dimensions. They are also arguably 
the simplest of the aerial robotics platforms to model 
for robotics applications. Early work in aerial robotics 
modeling and control dates back to the late 1990s and 
the field has remained active since [52.15-21]. 

In its most common form, the quadrotor vehicle is 
a very simple machine. It consists of four individual 
rotors attached to a rigid cross airframe as shown in 
Fig. 52.4. Control of a quadrotor is achieved by dif¬ 
ferential control of the thrust generated by each rotor. 
Pitch, roll, and heave (total thrust) control is straight¬ 
forward to conceptualize. As shown in Fig. 52.2, rotor 
i rotates anticlockwise (positive about the z-axis) if 
i is even and clockwise if i is odd. Yaw control is 
obtained by adjusting the average speed of the clock¬ 
wise and anticlockwise rotating rotors. The system is 
under-actuated and the remaining degrees of freedom 
corresponding to the translational velocity in the b i-Z >2 
plane, must be controlled through the system dynamics. 

The aerodynamics of rotors was extensively studied 
during the mid, 1900s with the development of manned 
helicopters and detailed models of rotor aerodynamics 



Fig. 52.4 A quadrotor made by Ascending Technologies 
at the University of Pennslvania with Vicon markers for 
state estimation 
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are available in the literature [52.22,23]. Much of the 
detail in these aerodynamic models is useful for the de¬ 
sign of rotor systems where the whole range of param¬ 
eters (rotor geometry, profile, hinge mechanism, and 
much more) are fundamental to the design problem. For 
a typical robotic quadrotor vehicle, the rotor design is 
a question of choosing one among the several available 
rotors from the hobby shop and most of the complexity 
of aerodynamic modeling is best ignored. Nevertheless, 
a basic level of aerodynamic theory is important to un¬ 
derstand the particularities of the control design. 

The steady-state thrust generated by a hovering 
rotor (i. e., a rotor that is not translating horizon¬ 
tally or vertically) in free air may be modeled [52.23, 
Sect. 2.26] as 

Ti := C T pA n rrmf , (52.3) 

where for rotor i, A n is the rotor disk area, r, is the 
radius, zzr, is the angular velocity, Cx is the thrust coef¬ 
ficient that depends on rotor geometry and profile, and 
p is the density of air. In practice, a simple lumped pa¬ 
rameter model 

Ti = Cjmf , (52.4) 

is used where cx > 0 is modeled as a constant that can 
be easily determined from static thrust tests. 

The reaction torque (due to rotor drag) acting on the 
airframe generated by a hovering rotor in free air may 
be modeled as [52.23, Sect. 2.30] 

Qi := CQVjf , (52.5) 

where the coefficient cq (which also depends on A n , r,, 
and p) can be determined by static thrust tests. 

As a first approximation, assume that each rotor 
thrust is oriented in the A 3 -axis of the vehicle, although 
we note that this assumption does not hold exactly for 
a rotor translating through the air. 

The total thrust at hover T applied to the airframe 
is the sum of the thrusts from each individual rotor 
(Fig. 52.2) 

4 

T=J2\ T i\=CT 

i= 1 

The hover thrust is the primary component of the ex¬ 
ogenous force 

F=Te 3 + A (52.7) 

in (52.2b) where A comprises secondary aerodynamic 
forces that are induced when the assumption that the 
rotor is in hover is violated. 


The net moment arising from the aerodynamics (the 
combination of the individual rotor forces) applied to 
the quadrotor vehicle in directions b\, hi, and b 3 , re¬ 
spectively, are 

ti = cjdiml - ml) , 

%2 = — cjd(m ( — m 3 ), 

4 

T3 = Cq T: (Jj-mf , (52.8) 

/= 1 

where d is the arm length of the quadrotor. A con¬ 
sequence of the structure of the thrust generation and 
the lack of aerodynamic lifting surfaces for a quadrotor 
means that one can solve for the heave and torque in 
a single equation in terms of the motor inputs 
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Inverting this relationship provides a mapping from de¬ 
sired control input for the rigid-body dynamics to rotor 
speed set-points for the motor control. 

In practice, there are additional second-order aero¬ 
dynamic effects present in the thrust generation 
for quadrotor vehicles [52.17, 19-21]. The principle 
second-order aerodynamic effects that are present at 
low speeds are inflow variation, rotor flapping, and in¬ 
duced drag. The first of these effects reduces the heave 
thrust when the quadrotor is ascending and increases 
the thrust when it is descending due to variation in 
rotor inflow velocity caused by the quadrotor motion. 
This acts as a damping term in the vertical direction 
of motion for the quadrotor. The remaining two effects 
generate forces that oppose horizontal translation of the 
quadrotor; blade flapping by inclining the rotor plane of 
flexible blade rotors and tilting the thrust vector away 
from the direction of motion, and induced drag by in¬ 
creasing the drag on the blade of a rigid rotor advancing 
in the direction of motion of the quadrotor [52.21]. In 
practice, it is difficult to separate the difference between 
the various effects and it is sufficient to model them all 
by a single damping force 

A = —Dv , (52.10) 

where D is a diagonally dominant positive definite ma¬ 
trix. For small quadrotors with relatively rigid rotor 
blades designed for hover performance, that is with 
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close to ideal chord and ideal twist, these effects are 
far more significant than is usual in classical helicopters 
and the effect must be modeled to achieve good con¬ 
trol performance. These aerodynamic effects are also 
important in providing key low-frequency excitation 
to attitude and velocity estimation in the absence of 
GPS [52.21,24-26]. 

At higher translation speeds, a quadrotor also ex¬ 
periences translational lift, translational drag, and par¬ 
asitic drag. These aerodynamic effects are associated 
with efficiency gains in the rotor operation associated 
with increased inflow velocity generated by the forward 
velocity of the vehicle. Such effects are extremely im¬ 
portant in large scale manned helicopters, which spend 
much of their time in forward flight. For many robotics 
applications, however, the vehicle is in quasi-stationary 
or near hover flight almost all the time and these sec¬ 
ondary aerodynamic effects may be ignored. We will 
not discuss them further in this chapter but refer the in¬ 
terested reader to recent work [52.18, 20] and any of the 
classical helicopter texts [52.22, 23, 27]. 

52.2.3 Modeling of Fixed-Wing Airplanes 

The conventional model for fixed-wing airplane uses 
a north-east-down frame of reference. The frame of 
reference used in this chapter, corresponding to the 
common aerial robotics convention inherited from the 
mobile-robotics community, is a north-west-up con¬ 
vention. We will still need to define the usual auxil¬ 
iary angles associated with fixed-wing modeling and 
control, in particular; the angle-of-attack, side-slip, 
aerodynamic-bank, flight-path, and course angles. We 
will use the normal conventions to define these angles. 
The north-west-up convention now means that flight- 
path angle is positive when descending and negative 
when ascending, and positive roll leads course angle to 
decrease. The careful reader can easily transform be¬ 
tween the conventions used here and the conventional 
airplane modeling approach and the authors apologize 
for any confusion that is caused by using a single con¬ 
vention for the whole chapter. 

The key difference in modeling fixed-wing air¬ 
planes to quadrotors is that they rely on lift generated 
by airflow over the wings to support them in flight. 
The angle of incidence of the wind on the wing is 
a key variable in the dynamics of the vehicle and 
must be modeled in the dynamics. Since the main lift 
force is generated by an aerodynamic process, rather 
than directly by a controlled input as was the case for 
quadrotors and blimps, the modeling and control of an 
airplane is somewhat more complicated. 

The aerodynamics of a fixed wing vehicle are de¬ 
fined relative to the local wind frame {W}. Frame {W} 


is chosen co-linear with the inertial frame {A}, but with 
linear velocity equal to the average extrinsic wind ve¬ 
locity with respect to the inertial frame. The wind frame 
is a Galilean frame (moving with constant velocity) 
with the property that it has zero extrinsic wind on av¬ 
erage. Let u a e {W} denote the aerodynamic velocity of 
the vehicle, that is the linear velocity of the vehicle in 
the wind frame {W}. Let v w e {A} denote the average 
wind velocity as measured in the inertial frame {A}. 
Since the orientation of frames {A} and {VT} are equal, 
the inertial velocity of the vehicle can be written as 

v = v a + v w e {A}. 

To model the incident wind as seen by the vehicle 
as it flies through the air we introduce the flow frame 
(also termed the air frame) denoted by \ F\ = (/) ,f 2 ,ff) 
where the orientation vectors {/,-} are expressed in the 
wind frame {F} which is co-linear with the inertial 
frame {A}. The first axis of the flow frame /1 is oriented 
in the direction of the incident wind on the vehicle, 
is chosen to lie in the plane of symmetry of the vehicle 
orthogonal to b 2 , and /2 makes up the right-hand frame. 
By construction, the velocity of the vehicle in the flow 
frame always lies in the direction/ 1 . 

We define two orientation matrices associated with 
the wind frame. The angle-of-attack (AOA) matrix 
b R f = R a 0 describes the orientation of the airflow 
frame {F} with respect to the body-fixed-frame {B} in 
terms of the angle-of-attack a, and the side-slip angle 
f [52.28] (Fig. 52.5). Using the Z-Y-X Euler angle 
convention (yaw by f, pitch by a, and there is no roll) 
we have 

/Ca c /3 ~Ca S,g 

Raj3 = I S/j Cp 

\ s a c /3 s a s /3 



Fig. 52.5 Representation of angle-of-attack, side-slip an¬ 
gle, and airspeed. Here v air is denoted by i> a in the text to 
save space 


-s«\ 

0 . (52.11) 

c« / 
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The second orientation matrix is the flight-path matrix 
W R F = R/j,,y, x that denotes the orientation of the flow 
frame {F} with respect to the wind frame {VT} in terms 
of the aerodynamic bank angle pt, the flight path angle 
y, and the course angle /. Using the Z-Y-X Euler angle 
convention (yaw by /, pitch by y, and roll by fi) we 
have 

Ru.v.x = 

CyC X 
CySy 
~Sy 


Note that the frames {VT} and {A} are co-linear so 
a R f = w R f . It follows that W R F = A R F = A R B B R F . 

Aerodynamic forces generated by lifting surfaces 
are proportional to the dynamic pressure [52.28] 

Q=\p\vA\ (52.13) 

where p is the air density and |u a | represents the air¬ 
speed or norm of the velocity v a . Consider the situation 
where the aircraft is in normal flight, that is cruising at 
subsonic velocity while avoiding stall phenomenon. Let 
S denote the surface area of the wing (or lifting surface 
of the vehicle). The lift L = F| generated by the lifting 
surface is defined as the aerodynamic force oriented in 
the /3 direction in flow frame {F} and can be modeled 
as 


s y s M c y c M s x s y c M c x ~b s a s x 
s y s /x s y + C P C X s y c M s y — S M C X 


(52.12) 


L = gSCg(a + a 0 ) + QSC^f 
»0S(C L +C£«) (52.14) 

for sufficiently small angle of attack a (typically less 
that 15°). The terms Cg(a + ao) and Cyf are linear 
approximations to the lift-coefficient curves for small a 
and ft [52.28]. The offset angle of attack ao is thought 
of as the effective angle of attack of the wing in normal 
level flight conditions. In particular, at normal cruising 
speed in level flight, QSC®a 0 = mg is the lift required 
to sustain the vehicle in level flight, so that a, f = 0 
in level flight. The term QSCyi6 is small and can be 
discarded in many applications. 

The drag D = —F% is a combination of parasitic and 
induced drag and is oriented in the —f 1 direction. 

D = QS[C'^ + K(C?) 2 (a + a 0 ) 2 ] 

«eS(C D + C», (52.15) 

where k is the Oswald coefficient [52.28] and the sec¬ 
ond equation is obtained by ignoring a 2 terms. The 


constant Co combines the parasitic drag coefficient 
Cg along with the induced drag /c(Cg) 2 a q due to ao, 
while Cd = 2Arao(C “) 2 is the linear coefficient for the 
dependence of induced drag on the angle-of-attack. 
It is also possible to model a contribution QSCq/3 
to the drag force, however, this is sufficiently small 
that it is normally ignored. There is a final lateral 
aerodynamic force component F'y that is due to side¬ 
slip 


Fy = QSCy ft . (52.16) 

The propeller or thrust mechanism of a fixed wing 
vehicle generates thrust T in direction b, in the body- 
fixed-frame. The total linear force F e { B\ applied to 
the vehicle due to aerodynamic effects and thrust effects 
is 


1^1 _ 1 ^u^a^p 1 

= I 0 I + QS I —CdS/3 
\0 / \-CnSaCp + C L c a ) 

/—Cgc aCjS -C“s a \ 

+ < 2 S 


-Cgsp 

) -Cgs^ + Cgc a 


+ QS 


^ e - CySa 


A 


~Cy ) C a C p CyC a Sp 

~^D S /3 + CyCp 
^ b Cy Sq, Sp t Cy C(x J 

t 


p 

(52.17) 


expressed in the flow frame. 

The aerodynamic moments r e {B} in the body- 
fixed frame are modeled by 



( c?p + cp^ + + c?.s. ^ 


QSl 


X 21 D a 


l 2 |t; a 


bQ v 


Cy + C-a + q-^L + C^gy + SO) 


C ^ + C ^\ +C ^ 


(52.18) 


where ( 12 v , £2 y , 12 -) are the components of the an¬ 
gular velocity 12 e {B} and represent the roll, pitch 
and yaw rates of body-fixed frame, respectively; b 
is the wing span, c is the mean cord of the wing. 
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and (S x ,S y ,S z ) are deflections of the airplane con¬ 
trol surfaces (ailerons, elevator, and rudder). The con¬ 
stants {c^,c;,cf,c v ,c“,c«,cf,cf,q,cf} are di¬ 
mensionless aerodynamic coefficients [52.28]. In nor¬ 
mal trim conditions, the trimmed elevator deflection 8 y 
is chosen to cancel the static moment C y generated by 
the wing in level flight, C y 8 y = —C y . 

Combining (52.17) and (52.18) with (52.2) yields 
a dynamic model for a fixed wing vehicle with mini¬ 
mal approximations. The resulting equations of motion 
are complex and can be difficult to work with directly. 
The most common approach to control fixed-wing ve¬ 
hicles based on the full model is to take this model 
and linearize the state equations, either along a trajec¬ 
tory, or in regions of state space. The resulting model 
is either a linear time-varying or linear parameter vary¬ 
ing system and can be controlled using classical linear 
system control techniques [52.29]. Although this ap¬ 
proach is well established in the aerospace industry it 
tends to hide the underlying structure of the dynamics 
and requires good models of the various aerodynamic 
parameters, a challenge for many aerial robots with air¬ 
frames that are always subject to being fiddled with 
and having sensor packages attached externally. For 
aerial robotic applications it is of interest to consider 
a simplified but still structurally consistent model of 
fixed wing dynamics and use robust and simple control 
strategies. 

Bank-to-Turn Flight Regime 
In this section, we propose a model that is suitable for 
a wide range of fixed-wing applications where the ve¬ 
hicle is in normal flight mode. The approach taken uses 
a flight regime where the lateral acceleration required 
to turn the vehicle is obtained by tilting the lift gen¬ 
erated by the wings, a strategy termed bank-to-turn or 
coordinated turn in the aerospace literature [52.30, 31], 
A bank-to-turn maneuver is characterized by zero side¬ 
slip, /I = 0 , and leads to considerable simplification of 
the equations of motion. This is a very common mode 
of flying for any fixed-wing vehicle and, unless the task 
requires aerobatic maneuvers or the vehicle lacks con¬ 
trol surface actuation, is the natural mode to control 
a UAV. 

Given that accurate measurements of the roll rates 
Q are available through an onboard IMU system and 
most UAV systems have large control surfaces relative 
to their size, it is possible to use high gain to dominate 
the attitude dynamics (52.2d) and (52.18). That is, the 
angular velocity 12 ss 12 * can be thought of as an in¬ 
put to the reduced order model (52.2a), (52.2b), (52.2c). 
The above discussion motivates a dynamic reduction of 
the system equations with new inputs 12 *, as well as T 
the propeller thrust. 


Based on the assumption that the vehicle is flying 
using bank-to-turn control and for small angle-of-attack 
then the approximation /3 0 holds and simplified 

dynamic equations can be derived. Using this approx¬ 
imation and canceling all second-order terms in a and 
fi allows one to rewrite (52.17) as 


j T \ t-C„\ /-Cg-C L \ 

0Ues 0 \ + i,s 0 \a. 

\l)/ \ t-L / \ K / 

(52.19) 

From the construction of {F} we have v F a = |u a [f \ 
by definition. It follows that (52.2a) can be written 
as 

k = ^ = Kl Ra,v,x e \ ■ 

Note that = (c x c y , s x c y , —s y ) T does not 

depend on the aerodynamic bank angle /r. Thus, 
(Kl- y, x) can be used as generalized coordinates for 
the velocity of the vehicle 

k = |Wa|(c y c y ,s y c y ,-s y ) T . (52.20) 

Differentiating u a = £ and rearranging yields 

0 0 \ /\vA 

k|c y S M -|v a |s M X 

-Nc y S M -KlW V Y 7 

(52.21) 

Consider the (3,2) element of (52.12) and recall 
that R^, v , x = Rd,,e,^Ra,/3 and applying yS = 0 then 
s M c y = s 0 and we have 


v* = R 


a.v.x 



for normal flight conditions where c y ^ 0 and s^ < c y . 
By simple geometry. 



and the dependence on the bank angle can be removed 
entirely from kinematics of (| u a |, y, /) to be replaced by 
the roll angle (j>. 
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Inverting (52.21) and substituting for (52.2b), 
(52.19) and for (52.22) we obtain 


7 -N = ~gsy + —c a -a — (Cg + C L ) 
at m m 

-^c D , 

m 


(52.23a) 


Back substituting from (52.23c) and rearranging yields 


a = Q y + 


S(/>S(x£“z S(/>C(x^x 

(C0S#Sqt + Cq,C0C6») 


t“t + a — (Cg — C D ) 
kU m 


mlttal L 


(52.26) 


* = - — ( C L + (Cg-C D )a|, 
c v \ mc v 


(52.23b) 


+ (C?-C D ) 

fa W 


-C L 




c y s 0 


m|f a | c y 


(52.23c) 


where the denominators are always well defined for 
normal flight conditions. The angular velocity S2 y enters 
(52.26) as a free input providing the control for AOA. 

Equations (52.23a), (52.23c) and (52.26) lead to 
a cascade nonlinear system 

/ =-s^ ^^-A 2 +, (52.27a) 

<£ = «,£, (52.27b) 


where one should recall that Q= |p|f a | 2 (52.13) de¬ 
pends on |f a |. 

Note that for a small, T provides a free input to sta¬ 
bilize the speed of the vehicle in (52.23a), although we 
have seen in Sect. 52.3.2 that a different approach is ad¬ 
vantageous in practice. In (52.23b), there is no direct 
input variable, however, the roll ip will play this role 
in the control design. Similarly, in (52.23c), the angle- 
of-attack a, which itself has dynamics, must be used to 
control y. In fact, both (52.23b) and (52.23c) depend on 
a complex mix of variables including dynamic pressure 
(and consequently | u a |) as well as cp, a and y. Neverthe¬ 
less, the identification of roll <p with control of heading 
and angle-of-attack a with control of flight path is a nat¬ 
ural approach in the development of aircraft control. 

The kinematics of tp are straightforward to compute 
by differentiating (52.1). Differentiating the (3,2) entry 
oiRc/),e,V/ (52.1) we have 


<j> = Q x c e + i2 z s e . (52.24) 

The primary control for <p is the roll angular velocity £2 X 
and this control will be used to cancel the disturbance 
created by 12, as discussed in Sect. 52.3.2. 

To model the dynamics of a, recall the relationship 
R/j,,y,x = R(/>,d,xU^ce,/3 again. This time considering the 
(3, 1 ) element of R/j,,y,x we obtain 


h = — |tt a |s y , (52.27c) 

d 

- ( ;KI = TAi- gs y +f\{y,a, |w a |) , (52.27d) 

y = a|v a | 2 A 4 — — (c 2 -s^,)2A 5 

Cy 

+/2(y>a|), (52.27e) 

d = u a +f 3 (y,a,\v a \;t) , (52.27f) 


with the following constants 

Ai = — , A 2 = ^, A 3 = (C“-C d ), 
m 2m 

k=^(Cg-C D ), a 5 = c l ^, 

2m 2m 

functions; 


/i(y,a, |w a |) = -o' —(Cg +C L )-C D , 

m m 

/ 2 (y. kl) = , 

kl 


/3(y.a.kl) 


—(Cg 
\v,\ m 


C D ) 


QScn 
m |w a | L 


« (4(/>S$Sg. T Cq[C(I)Cq) — 

S(/)Coc T" s^,SQff2 z 

T (c^s^Sq. T c^cQC(x)£2y 

— Cy y. 


and the input functions; 


M(j) —£2 x cq + £2 z sq , 

u a =iS2 v + - ; 

(c^SgSa + CaC^Ce) 


(52.25) 
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Here the height h kinematics have been added to com¬ 
plete the bank-to-turn dynamics. The course (52.27a), 
and the height (52.27c) dynamics have negative signs 
compared to conventional fixed-wing modeling devel¬ 
opments due to the nonstandard north-west-up conven¬ 
tion chosen. 

These equations are fully nonlinear equations based 
on the sliding mode simplification associated with hold¬ 
ing fi = 0. The condition on side-slip angle will can be 
enforced using the free degree of freedom in Q z as will 
be discussed in Sect. 52.3.2. We note that the dynam¬ 
ics presented in this section are closely related to the 
longitudinal dynamics considered in classical aeronau- 

52.3 Control 

Control for aerial vehicles is challenging for a number 
of reasons. First, most aerial vehicles are under actu¬ 
ated and the control design must exploit interactions 
between dynamic states to control the vehicle. Second, 
aerial vehicles use aerodynamic effects for thrust and 
lift generation, and regulation of these forces is by na¬ 
ture approximate, leading to significant modeling error. 
Third, external effects such as wind, turbulence, and 
vortex generation, lead to high levels of load distur¬ 
bance in the control loops. Finally, it is often difficult 
or impossible to measure the vehicle and aerodynamic 
state directly making it necessary to use an observer or 
design the controller from first principles to use explicit 
measurements. 

Most aerial vehicles are controlled using a hier¬ 
archical control structure with nested feedback loops 
based on three levels; planning, guidance, and control: 

• Planning : It is the outermost loop in aerial vehicle 
control and is associated with path planning, setting 
way-points, etc. 

• Guidance: The guidance level of control concerns 
tracking trajectories to achieve local goals. This 
control loop is typically designed using the attitude 
reference as a virtual input, leading to a fully actu¬ 
ated control problem. 

• Control: It is the high gain innermost loop of the 
control hierarchy and is concerned primarily with 
attitude and flight stability of the vehicle. This con¬ 
trol problem is typically fully actuated and can be 
tackled using standard control techniques. 

The more dynamic and aggressive the task consid¬ 
ered is, the more these levels of control will interact and 
an integrated control design must be considered. How¬ 
ever, in most real-world situations, splitting the control 
into three hierarchical levels leads to a simpler design 


deal texts that treat the angle of attack, glide angle, and 
velocity of the vehicle. The more classical expressions 
for longitudinal control of a fixed-wing vehicle are ob¬ 
tained from the above model by setting / = /z = cp = 0. 
An additional simplifying assumption often made for 
the course kinematics (52.27a) is that lift is compensat¬ 
ing gravity during level-flight. With these assumptions, 
(52.27a) can be replaced by 

X = —-p-rtan(0) 

kl 

based on centripetal force balance. 


problem that achieves the desired performance. For ex¬ 
ample, the recent aggressive maneuvers undertaken and 
demonstrated in quadrotor control are based on hier¬ 
archical control strategies that use trajectory planning 
processes for the outer loop that specify achievable 
trajectories that then are stabilized in real-time using at¬ 
titude as a virtual input, with high-gain feedback on the 
inner-loop attitude stability control. Most of this work 
has been done with the use of external motion capture 
systems to provide the inertial frame position and ve¬ 
locity feedback [52.7-12, 32]. However, more recently 
there have been a few attempts to do this with on¬ 
board cameras and inertial measurement units [52.13, 
14]. 

52.3.1 Quadrotor Control 

The quadrotor control and guidance problem is con¬ 
ceptually more straightforward than that of fixed- 
wing vehicles. In its simplest form, the goal is 
to design control algorithms to track smooth fea¬ 
sible trajectories (/?*(?), £*(f)) e SE(3). We will as¬ 
sume that a planner specifies the full desired tra¬ 
jectory, including the higher order derivative terms 
(f2*(t),ih(t)) and (r2*(t),f*(t)). A quadrotor vehi¬ 
cle is an under-actuated system - there are four inputs 
u = (T, r T ) T , while the trajectory lives in SE(3) and 
is six-dimensional. The hierarchical control structure 
proposed leads to an inner loop regulating the attitude 
using the moments r as the control input. The guidance 
level utilizes the attitude R and heave T to regulate the 
trajectory £(f) to track £*(/). In high performance con¬ 
trol of a quadrotor, the major limitation to performance 
comes from the limits of motor response and the control 
hierarchy must be augmented by a low level motor reg¬ 
ulation system. The proposed control architecture forms 
nested feedback loops as shown in Fig. 52.6. 
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Fig. 52.6 Typical control architecture. A base-level motor regulation, an inner attitude control loop, an intermediate 
position guidance loop, with an outer trajectory planner 


Motor Regulation 

The aerodynamics of thrust generation by rotors is 
a subject that has been studied in detail in the classical 
rotorcraft literature [52.22,23,27]. Using a full model 
of rotor aerodynamics has significant potential in the 
design of the motor control for high-performance rotor 
control for aerial robotic applications [52.33]. Despite 
this, it is sufficient in most cases to consider a static 
thrust model that holds for a rotor in hover and is a rea¬ 
sonable approximation for most robotic applications. In 
such a model, the thrust generated is proportional to the 
square of the rotor speed. Since most quadrotor vehicles 
are equipped with brushless DC motors that use back 
EMF sensing for rotor commutation, making it possible 
to measure rotor angular velocity, the control of the ro¬ 
tor thrust is usually implemented as a local control loop 
on the electronic speed controller (ESC) regulating ro¬ 
tor speed. 

Most ESC use PWM (pulse width modulation) reg¬ 
ulation of voltage bus to provide voltage control of the 
motors. A typical rise time for the uncontrolled system 
is in the order of 200 ms and it is necessary to include 
a local control loop to improve the system response. 
The aerodynamic drag of the rotor means that the sys¬ 
tem is naturally heavily damped and there is no need for 
derivative control. Similarly, an integral term is rarely 
deemed necessary at the rotor control level since the 
thrust model used is not particularly accurate anyway 
and it will always be necessary to have integral control 
at higher levels in the control hierarchy. Since an inte¬ 
gral term is inadvisable, it is important to use the best 
model of the thrust generation possible as a feedforward 
term to minimize the requirement on the pro¬ 

portional regulation. Thus, a typical ESC rotor control 
is given by saturated proportional control with feedfor¬ 
ward [52.34] 

Vj = sat [k(m* — Toy) + Vff(ttJ*)] , (52.28) 

where V, is the applied motor voltage, m* is the 
desired speed and the actual motor speed m, is mea¬ 
sured from the electronic commutation in the embedded 


speed controller. The performance of the motor con¬ 
trollers is ultimately limited by the current that can be 
supplied from the batteries [52.34] and a saturation on 
the demanded voltage is necessary. Without the satu¬ 
ration, extreme maneuvers may cause the voltage bus 
to drop excessively, destroying the rotor regulation per¬ 
formance and unless care is taken, causing the onboard 
electronics to brownout. 

Attitude Control 

The control problem considered in attitude regulation is 
to use full actuation of r in (52.2d) to control (52.2c) 
to track a desired attitude trajectory R*(t) along with 
its velocity £2*(t) such that R* (t) = R*(t)£2*(t). The 
approach described uses a global stability design based 
on minimizing the matrix error 

R=(R*) J R (52.29) 

similar to [52.7, 35]. Driving R —> I 3 ensures that R —> 
R *. The kinematics of the tracking error is given by 

R = -£2*R + R£2 X 
= [R, £2*] + R(£2 x -£2*) 

= [R,£2*]+R£2 X , (52.30) 

where [A, B] = AB — BA is the matrix commutator and 

£2:=£2-£2*. (52.31) 

We assume that derivative £2* of the target angu¬ 
lar velocity is available for the purposes of designing 
a feedforward torque for the angular velocity dynamics 

r* :=l£2* + £2*l£2 , (52.32) 

where I is the inertia matrix of the airframe (52.2d). 
Feed-forward control for the angular velocity of the 
attitude is important for high-performance maneuvers, 
especially when the motor response is pushed to near 
performance limits. The higher derivative information 
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for the angular velocity is available from path planning 
algorithms such as those discussed in Sect. 52.4.1 or us¬ 
ing predictive control algorithms [52.8-10, 36]. Choose 
the control input 

r:=r*+u 2 (52.33) 

where u 2 G R 3 is the free control that will be used to 
stabilize the error dynamics (Fig. 52.6). The error dy¬ 
namics for the angular velocity error are now 

112 = —I2 X II2 + u 2 . (52.34) 

The goal is to choose u 2 in order to stabilize the 
error dynamics (52.30) and (52.34) robustly. Define 

P (R) =^(R-R r ) 

to be the skew-symmetric projection of the error matrix. 
It can be verified that P(7?) = sin(0)a x where (a, 9) 
is the angle-axis representation of R. That is the skew- 
symmetric projection is the axis of rotation that rotates 
R to R* (equivalently rotates R to the identity) scaled 
by the sin of the angle between the two orientations. 
Choose a proportional-derivative (PD) control 

u 2 = — £pvex(P(/?)) — k D I2 , 

where vex: R 3x3 —> R 3 is the inverse of the skew- 
symmetric operator applied to skew-symmetric ma¬ 
trices, vex(I2 x ) = 12. The proportional term applies 
a torque associated with a nonlinear spring associated 
with a potential energy tr(/? T /?) while the derivative 
kp>Q provides damping. It is straightforward to demon¬ 
strate that the Lyapunov function 

£ := k P tr(R T R) + ,T2 t II2 

is decreasing 

y£:=-k D \\n\\ 2 . 

Invoking Barbalat’s lemma, with some care, it follows 
that the system is almost globally asymptotically stable. 
The zero measure exception set for the basin of attrac¬ 
tion is associated with situations where the quadrotor 
is completely flipped, 9 = n rad, and P(I?) = 0 even 
though R ^ I. 

To verify the local exponential convergence and 
provide guidance for the gain tuning, we consider the 
linearization of (52.30) and (52.34). Write R ss 7 + 
(Zfi)x where the skew-symmetric matrix (z fi ) x (for<j? £ 


R 3 ) is the linear approximation of R around the identity 
matrix I. Write 12 « zn for the linear approximation 
of Q around the origin Q = 0. The linearization of the 
error z = (zr,z&) system is given by 



or in more compact notation z = A(t)z where A(t) = 
Ai(f)+A 2 comprises the sum of two matrices in 

(52.35) . The matrix A\{t) is a time-varying matrix de¬ 

pending on the exogenous system signals 12 and 12* 
while the second matrix A 2 is a time-invariant Hurwitz 
second-order linear system matrix. It is straightforward 
to show that this system is asymptotically stable us¬ 
ing a Lyapunov function i = k P /2|z fl | 2 + 1 /2z T Q \zn, 
using an analog of the nonlinear argument made ear¬ 
lier. We have that l = —kp\z.Q | 2 - Define C = (0 73 ) £ 

R 3x6 then if the pair (A(t), C) is uniformly completely 
observable (UCO) asymptotic stability implies expo¬ 
nentially stability [52.37, pp. 626-628]. It is intuitively 
clear that the system matrices (A(t), C) associated with 

(52.35) are UCO, although providing the algebraic 
proof is beyond the scope of the this chapter. Tuning 
the gains is then a process of choosing kp and kp, to as¬ 
sign the eigenvalues of the Hurwitz system matrix, with 
an eye to dominate the oscillatory, but bounded energy, 
disturbance introduced by Ai (f). 

Trajectory Tracking Control 
The trajectory planning algorithm, discussed later in 
Sect. 52.4.1, provides a full trajectory specification as 
well as the feedforward inputs consisting of (£ p , 

R p , Q p , Q p , ii\, u%) and their derivatives. The goal of 
trajectory tracking control is to regulate the linear dy¬ 
namics of a quadrotor to track a specified goal trajectory 
f*(f). Trajectory tracking control operates at a higher 
level in the hierarchy of the quadrotor control architec¬ 
ture (Fig. 52.6) and the desired attitude R* used in the 
attitude control loop is designed as an input in the tra¬ 
jectory loop control design. In addition to specifying the 
attitude goal, R* , the trajectory tracking control speci¬ 
fies the heave (total thrust), T, input reference that is 
used along with (52.33) for the low-level motor control 
reference (52.9). 

The linear trajectory goal £* = and are 

used directly from the path planning design as the goal 
of the trajectory planner. The actuation for the lin¬ 
ear dynamics, however, depends on the attitude of the 
vehicle. Thus, it is necessary to define the goal for 
the attitude control R* as a part of the control design 
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rather than rely on the planned trajectory set-point; in 
particular R* / R p in general. Note that the feedfor¬ 
ward control in Sect. 52.3.1 requires £2 * and £2* to 
plan the feedforward torque input for the attitude dy¬ 
namics. Feed-forward in the attitude regulation loop is 
a critical component of high-performance control and 
without this input, there will not be sufficient gain in 
the attitude dynamics to track aggressive attitude tra¬ 
jectories. However, computing the angular velocity goal 
that corresponds to a feedback dependent attitude con¬ 
trol goal R* in real-time requires forward prediction 
of the control trajectory using tools such as model 
predictive control (MPC) [52.8-10, 36] 1. A simpler ap¬ 
proach for trajectory tracking is to use the planned 
angular velocity £2 * = £2 P even though it does not ex¬ 
actly match the variation of R*. As long as (f,f) is 
close to (f *, £*) = (£ p , £ p ) then R* is close to RP and 
(£2*, £2*) = ( £2 P , £2 P ) will provide a good estimate of 
the critical feedforward attitude input. 

We will discuss here the linear trajectory tracking 
algorithm for which the specified trajectory is required 
to have roll and pitch values that are small thus justi¬ 
fying a linear approximation of the dynamics. Accord¬ 
ingly, we assume that the Euler angles associated with 
the rotation R p (52.1), are given by 9 P = (p p = 0, while 
t !r p is a specified function of t. For a given arbitrary 
yaw angle, we can linearize the dynamics about the 
hover position (0 = 0 , cp = 0 , ijr = \j/ p ) and the nom¬ 
inal inputs Mi = mg, 112 = 0. Linearizing (52.2a), we 
obtain 

fi = g(A9 cos 1 lr p + A<p sin \f/ p ) , 

£2 = g(A 6 sini j/ p — A(p cos i/F’) , 

£3 = -«1 ~g, (52.36) 

m 

where A9 = 9* and A<p = <p* represent the small devi¬ 
ations from the hover position. In order to exponentially 
drive all three components of error in the position part 
of the trajectory, we want the acceleration vector £ to 
satisfy 

<a*-b+K d (t*-i)+K P G*-s)= 0 . 

Front (52.36), we can immediately write 

“I = m + f * + £rf,z(i* — £3) + kp,z(£* ~ £3)] - 

(52.37) 

to guarantee (§3 (?) — f * (f)) —> 0. We choose the appro¬ 
priate desired roll, pitch, and yaw angles for 9 *, < p*, 


and \fr * to guarantee exponential convergence: 

<p* = — (f* sini/f — cosi/r) , (52.38a) 

g 

9* = -(f,* cos i/f + & sin ip) , (52.38b) 

g 

f* = f p . (52.38c) 

Now (ip*, <p* ,9*) define the rotation matrix R* (52.1) 
provided as a set point for the exponentially convergent 
attitude controller discussed in the previous section. 
Thus, as shown in Fig. 52.6, the control problem is ad¬ 
dressed by decoupling the position control and attitude 
control subproblems and the position control loop pro¬ 
vides the attitude set points for the attitude controller. 

The position controller can also be obtained without 
linearization. This is done by projecting the position er¬ 
ror (and its derivatives) along b} and applying the input 
Mi that cancels the gravitational force and provides the 
appropriate proportional plus derivative feedback 

Mi = mbj * + K, t (p* — £) + K p (%* — £) + ge 3j . 

(52.39) 

Note that the projection operation is a nonlinear func¬ 
tion of the roll and pitch angles and thus this is a non¬ 
linear controller. In [52.7, 35], it is shown that this 
approach results in exponential stability and allows the 
robot to track trajectories in SE(3). 

52.3.2 Control of Fixed-Wing Aircraft 

An aircraft is a highly coupled nonlinear dynamical 
system and it may seem most natural to treat the con¬ 
trol problem as an integrated multi-input multi-output 
(MIMO) nonlinear design [52.29, 38^10]. However, de¬ 
spite the nonlinear coupling of aircraft states there 
are still clear dependencies between specific actuators 
and certain dynamic modes of the open-loop response. 
A disadvantage of the MIMO approach is that it tends 
to obscure insight between control design and specific 
open-loop mode response. Moreover, and as a conse¬ 
quence, it tends to be less robust to large changes in 
model parameters (such as caused by using a different 
set of wings, or bolting on a completely new sensor 
package, not an uncommon situation for a small UAV 
system) than a classical control design. The classical, 
and more intuitive, control architecture for fixed wing 
vehicles is to consider a set of separate, but intercon¬ 
nected, single-input single-output (SISO) control loops. 
Such an approach has the advantage of being highly in¬ 
tuitive and leading to a modular design methodology 
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based on successive loop closure, providing a straight¬ 
forward gain tuning regime [52.31]. The robustness and 
simplicity of the SISO architecture make it the preferred 
choice for most small UAV control systems. A modern 
implementation, however, uses the total energy con¬ 
trol system (TECS) architecture and input decoupling 
to overcome some of the limitations of classical fixed- 
wing control design. 

The main control actions for a fixed wing aircraft 
are associated with the throttle, and the three aerody¬ 
namic control surfaces; rudder, ailerons, and elevators. 
The throttle regulates power to the motor and gov¬ 
erns thrust T. The rudder, ailerons and elevators actuate 
the angular velocities I2 Z , Q x , and i2 y , respectively, 
through the angular dynamics of the vehicle. For a small 
fixed-wing UAV in normal flight, a simple high-gain 
proportional control scheme will effectively regulate 
and decouple the variables (T,£2 x ,£2 y ,£2 z ) [52.31]. 
The control problem can then be posed as one where 
(T. S2 X , 12 v . Q z ) are inputs for regulation of the re¬ 
mainder of the vehicle dynamics. Since there is weak 
physical coupling between the lateral variables (roll 
rate, bank angle, yaw rate, and course angle) and the 
longitudinal variables (airspeed, pitch rate, pitch an¬ 
gle, altitude rate, and altitude), and since the elevator 
and rudder most strongly effect the lateral variables and 
the throttle and aileron most strongly influence the lon¬ 
gitudinal variables, most control architectures separate 
functionality into a lateral control system and a longi¬ 
tudinal control system. The input to the lateral control 
system is the desired course angle, and the input to the 
longitudinal control system is the desired airspeed and 
the desired altitude. 

Lateral Control System 

There are three open-loop dynamic modes associated 
with the response of the lateral dynamics of a fixed- 
wing aircraft: the roll mode, the spiral mode, and 
the dutch roll mode. The roll mode is associated 
with the first-order response of the roll rate of the 
vehicle to aileron input. Usually the wings and air¬ 
frame are designed (e.g., with a positive dihedral angle 
Chap. 26.4.4) so that this mode is highly stable and is 
regulated using high-gain in the aileron input - indeed, 
this mode is subsumed into the control simplification 
(regulation of Q x ) proposed above. 

The spiral mode is associated with the natural turn¬ 
ing motion of the vehicle. That is, if the vehicle is 
banked into a turn, it will continue to turn until control 
action is applied to flatten the vehicle out. The spiral 
mode is a linearization of the bank-to-turn dynamics 
and is a single real pole close to the origin (Chap. 26, 
Fig. 26.22). Typically the spiral mode is slightly un¬ 
stable due to the tendency of the vehicle to side-slip 


into a turn. In manned flight, the spiral mode can be 
dangerous since there is very little external perception 
of turning. The nature of the banked turn dynamics 
ensure that the body-fixed-frame acceleration remains 
oriented through the central axis of the vehicle and rate 
of rotation is slow and nearly constant. The mode be¬ 
comes dangerous if the pilot is not aware as the vehicle 
slowly side-slips down into the turn, progressively div¬ 
ing more and more steeply and increasing airspeed in 
what is known as a spiral-dive. For autonomous flight 
control, the spiral mode is relatively benign since au¬ 
topilots never stop looking at the instruments and take 
immediate action if the vehicle is drifting off course. 

The dutch-roll mode, is associated with the oscilla¬ 
tion of the vehicle around a heading. In this motion, the 
vehicle yaws away from the heading and then rolls due 
to the dihedral effect, before the tilted lift vector and 
rudder action force the airplane back toward the inci¬ 
dent wind. If the rudder is small and does not provide 
sufficient damping, the vehicle can overshoot to yaw 
and roll in the other orientation before repeating the os¬ 
cillatory sequence. Dutch role motion is less dangerous 
for manned flight since it is usually stable (except for 
vehicles with strongly swept back wings or negative di¬ 
hedral) and produces unpleasant sideways acceleration 
with the side-slip motion that is immediately apparent 
to a pilot. Damping is increased by large tail plane sur¬ 
faces and moderate to small dihedral. A figure of the 
typical pole positions of an aerobatic RC airplane is 
shown in Chap. 26, Fig. 26.22. 

To understand the control problem for lateral dy¬ 
namics, consider first the angular yaw velocity 12. 
induced by rudder actuation. The primary effect of 12. 
will yaw the vehicle in the air and generate nonzero 
side-slip /3. In the bank-to-turn control paradigm de¬ 
scribed later, we will use 12- to regulate /3 = 0, however, 
it is instructive first to consider the effect of nonzero 
side-slip in order to build an intuitive understanding of 
the lateral dynamics. If the side-slip fi 7^ 0, the airplane 
will be sliding sideways or crabbing through the air. In 
particular, applying rudder input (nonzero I2 Z ) acts to 
twist the airplane in the air; it does not in itself cause 
the trajectory of the airplane to curve. For a typical air¬ 
plane with dihedral, this will lead the aircraft to bank 
away from the side-slip, and if the dutch-roll mode is 
stable, the airplane will settle into a banked turn. The 
causality of the system signals is given by 

-> P <t> X • 

Here the actuation from /3 —> tp relies on the inherent 
stability of the dutch roll mode of the lateral open-loop 
dynamics of the airframe. Indeed, small-scale vehicles 
with strong dihedral and large tail planes can be flown 
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without aileron surfaces. For example, the aerial robotic 
vehicle as shown in Fig. 52.7, designed for slow low- 
level flight and equipped for visual terrain tracking 
using optic flow computed from a pair of downward 
pointing web cameras [52.41], does not have ailerons 
and the lateral motion is controlled using only the 
rudder. 

This control paradigm has several negatives. It is 
rarely used for manned flight because side-slip motion 
generates sideways accelerations that do not feel natural. 
More importantly, the control response is limited by the 
low pass response of the open loop dutch roll dynamics 
of the aircraft. Although this mode is stable, there is of¬ 
ten a noticeable damped oscillatory response that causes 
the vehicle trajectory to wander around the desired head¬ 
ing and degrades course tracking performance. 

The alternative is to explicitly use the ailerons to im¬ 
pose the desired roll angle (p on the vehicle and actuate 
the heading dynamics (52.27a) without using the rud¬ 
der. The rudder then becomes a separate control input 
that can be used to regulate /3. That is 

&X ->■ <P ->■ X ■ 

Although, this may seem counter intuitive, (the rud¬ 
der is not used to steer the airplane) it is, however, the 
most effective heading control strategy. Using this con¬ 
trol strategy and regulating the side-slip = 0 ensures 
that the bank-to-turn simplifications can be applied to 
the model as was shown in Sect. 52.2.3. 

Specifically, in the bank-to-turn strategy where side¬ 
slip /3 is regulated to zero using the rudder, the lateral 
equations of motion are given by (52.24) and (52.27a). 
In level flight where the pitch angle is small and the ve¬ 
locity and angle of attack are roughly a constant, these 



Fig. 52.7 An autonomous fixed-wing aerial robot for ex¬ 
perimental work on vision-based terrain tracking at ANU 
(after [52.41]). Note the upswept wings to generate dihe¬ 
dral and the lack of ailerons 


equations simplify to the cascade structure 

<P = + d<i> , 

X = -&x<P + d x - 

where A x is a constant and d$ and d x are disturbance 
signals introduced through the modeling process. From 
classical control theory, we know that a step in the head¬ 
ing command /* can be tracked while simultaneously 
rejecting low-frequency disturbances and d x using 
a nested PI control law given by 

t 

<t>* =kp x (x* ~X) + ki x J (/*-/) dr, (52.40) 

— OO 
t 

&X = kp^icp* -<j>) + k I(t> J (</>*-0) dr, (52.41) 

— OO 

where kp x , kj x , kp (j) , and k Itj) are positive control gains. 
The proportional control 

£2 Z = -k Pe p, (52.42) 

is usually effective at regulating the side-slip angle to 
zero, where kp e is a positive control gain. 

Longitudinal Control System 
The longitudinal dynamics concern the thrust T and 
elevator (regulating f2 y ) inputs, along with the an¬ 
gle of attack a, the flight path angle y, the airspeed 
|u a |, and the altitude h. There are two primary open- 
loop dynamic modes associated with the open loop 
response of the longitudinal dynamics, the short-period 
and phugoid modes. The short-period mode is associ¬ 
ated with variation of pitch of the vehicle for constant 
velocity, constant altitude flight. It is a consequence 
of the linear dependence of lift of the primary wing 
to angle-of-attack as the vehicle pitches up and down 
around its center of the mass. The center of mass of 
an airplane is always designed to lie in front of the 
center of force of the primary wing surface to ensure 
that the short-period mode oscillation is stable, (placing 
the mass further forward increases the stability margin,) 
while damping provided by the tail plane of the vehicle 
ensures asymptotic stability (Chap. 26.4.2). 

The phugoid mode is associated with the interplay 
of aerodynamic lift, velocity, glide path, and height of 
the vehicle for constant angle-of-attack flight. As a ve¬ 
hicle starts to descend gently, the airspeed increases 
as the loss of potential energy in height is transferred 
into kinetic energy. As the vehicle accelerates, the in¬ 
creased airspeed leads to increased primary lift on the 
wings leading the vehicle to pull out of the dive and 
to start to ascend again. The vehicle will slow as the 
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stored kinetic energy is traded back into potential en¬ 
ergy. The resulting open-loop unstabilized motion is 
a repeated swooping cycle. Since aircraft are designed 
with low drag coefficients, the phugoid mode is usually 
very lightly damped, however, it is slow and easily sta¬ 
bilized. A key point to note in phugoid motion is that 
it is a constant energy oscillation; potential energy is 
transformed into kinetic energy and is then transformed 
back into potential energy. 

Both thrust and pitch rate inputs affect both velocity 
and angle-of-attack of an airplane. In turn, the glide- 
path angle can be controlled through regulation of either 
angle-of-attack or velocity of the vehicle. The coupling 
associated with these dependencies mean that simple 
thrust to velocity or pitch-rate to glide-path SISO con¬ 
trol loops cannot cover the full flight regime of a typical 
aircraft, a limitation of many existing autopilot systems 
even in commercial airliners. The industry standard ap¬ 
proach to address this issue is to have different control 
modes for different flight conditions, take-off and climb, 
level flight, and descent, that exploit different input to 
state control mappings [52.31]. The total energy control 
system (TECS) design paradigm [52.42,43] partially 
avoids these difficulties for a wide range of operating 
conditions and has the additional advantage of minimiz¬ 
ing the control activity undertaken by the throttle. 

Due to the energy preserving nature of the phugoid 
mode, small variations in velocity and height tend to be 
stable and energy conserving. Based on this insight, it 
is desirable to allow small variations of the airspeed of 
the vehicle as long as it is associated with the phugoid 
mode response. The key characteristic of the phugoid 
mode is that it preserve total energy 


E 


—m |w a | 2 + mgh 


(52.43) 


of the aircraft. The TECS control architecture uses the 
total energy as an output and uses the throttle to regulate 
this output to a desired energy set point. The set point 


E*(t) := -m\v *\ 2 + mgh* 

is chosen to correspond to the desired trajectory. The 
energy error is 

E := E*(t) - E(t) , 

and the throttle input is assigned as 

t 

8 t = kp E E(t) + k lE J E{ r)dr, (52.44) 

— OO 

where kp E and k iE are positive control gains. 

Once the throttle input is assigned, the remaining 
longitudinal dynamics must be controlled using the el¬ 
evator to servo the pitch rate E2 y . The primary effect 


of pitch rate is to rotate the wing section of the air¬ 
craft, directly effecting the angle-of-attack. Changing 
the angle-of-attack effects lift and drag generated by the 
wing, with lift most strongly effected since it depends 
linearly on a (52.14) while drag is small and only varies 
with the square of the angle-of-attack (52.15). By def¬ 
inition, the lift L is the component of airfoil force that 
lies perpendicular to the velocity. As a consequence the 
lift force L does not contribute directly to changes in 
the total energy E of the vehicle. In particular, actu¬ 
ation of the pitch dynamics of the vehicle is (almost) 
decoupled from total energy regulation loop. It is now 
possible to choose an output for the elevator input and 
have confidence that the resulting SISO loop response 
will only marginally disturb the total energy regulation 
loop. Rather than simply regulate the height or velocity, 
both variables that have been considered as outputs for 
elevator regulation loops in classical fixed-wing control 
architectures [52.31], it is of interest to regulate a bal¬ 
ance between kinetic and potential energy [52.42]. 

Consider the kinetic energy and potential energy of 
the vehicle separately 

K = -m\Va\ 2 , U := mgh . 

A given flight trajectory corresponding to desired val¬ 
ues of (\v*(t)\,h*(t)) defines desired set points for 
kinetic and potential energy K*(t) = \m\v*(t)\ 2 and 
U* = mgh*(t). Define the errors in the kinetic and po¬ 
tential energy along the trajectory as 

k = K* (t) - K(t) = ] -m [K*(t)| 2 - K«| 2 ] , 

U = U* (f) - U(t) = mg [h*(t) - h(t )] , 

(52.45) 

and define the combined energy balance error B by 

B = K-U. 

The proposed control strategy is to use the commanded 
pitch angle 6 * to regulate B —> 0. 

To understand this control architecture, note that if 
E = 0 then K = E — U = —U and hence 

B = 2K= -2 U. 

It follows that driving B to zero forces both 1141 —> \ v* \ 
and h —^ h* . The real advantage of the balanced energy 
control architecture occurs when E ^ 0. In this case, 
driving B — > 0 will not force either |u a | or h to its ref¬ 
erence set point at all costs, but rather, it will balance 
the excess or deficit energy between kinetic and po¬ 
tential energy set points around the desired trajectory. 
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Balancing the energy error in this way is a highly ro¬ 
bust control strategy that maximizes the ability of the 
throttle to maintain good control of the total energy. 

An additional insight is that the difference of the en¬ 
ergy terms B is maximally sensitive to excitation in the 
phugoid mode. In particular, if power flows between po¬ 
tential and kinetic energy levels, then the relative vari¬ 
ation in B is maximal compared to any other combined 
measure incorporating |ir a | and h subject to scaling fac¬ 
tors. Thus, the balanced energy control strongly directly 
damps the phugoid mode that is uncontrolled by the 
thrust control. 

The desired pitch angle is therefore given by 

t 

9* = kp B B(t) + ki B J B( r)dr, (52.46) 

— OO 

where kp B and kj B are positive control gains. In practice, 
the pitch angle actuates stable angle-of-attack dynam¬ 
ics that in turn lead to regulation of the flight-path 
angle and finally control the tradeoff encoded in B. 
The stability of the internal dynamics for the proposed 
SISO control loop are handled by suitable tuning of the 
PI gains, avoiding the need to explicitly model these 
dynamics. The effectiveness and robustness of this ap¬ 
proach is well established in the literature [52.31], 

Control Implementation and Gain Tuning 
The lateral control strategy for fixed wing aircraft is 
based on (52.40), (52.41), and (52.42). In using these 
commands, we are assuming high-bandwidth feedback 
loops regulating E2 X and Q z using the ailerons and rud¬ 
der, respectively. Practical implementation of the lateral 
control loops required saturating the roll command </>* 
to ±<p, where, for most airframes, a reasonable value 
is (f> = 30°. Commanding higher roll angles usually re¬ 
sults in large side-slip angles, which leads to unaccept¬ 
ably large drops in altitude during turns. The roll rate 
command Q x is also saturated to a fraction of the limits 
of the rate gyro sensor to ensure adequate convergence 
of the attitude estimator. Practical implementation of 
(52.40) also requires careful wrapping of the angles / 
and x* around ±180° to avoid large heading errors that 
result in not associating 180° with —180°. 

The gains of the lateral control loops can be tuned in 
flight by selectively enabling one loop at a time. A strat¬ 
egy that has proven success over numerous flight test is 
to tune the gains in the following order: 

1. Attitude rate loops regulating 12 v and Q.. 

2. Side slip gain kp e . 

3. The proportional gain on roll error kp ^, followed by 

the integral gain kj ^. The integral gain k tlj> is often 


set to zero, since the integrator on the heading loop 
provides suitable robustness. 

4. The proportional gain on the heading loop kp x , fol¬ 
lowed by the integral gain ki x . 


The use of (52.46) assumes a high-bandwidth pitch 
control loop using the elevator as an actuator. Practical 
implementation of (52.44) requires saturating S, above 
by <5™“ and below by <5™ n > 0. Saturating the pitch an¬ 
gle is also critical to avoid stall conditions that occur at 
large angles of attack. An effective method to do this is 
to saturate the altitude error in computing the error in 
potential energy, replacing (52.45) with 


U = mg sat \h* (t) — h(t), h\ , 
where 


sat(x, £) 


'i 

< -l 


ifx>£, 
if x<-l , 
otherwise . 


(52.47) 


Saturating the altitude error will cause a steady climb 
rate when h* — h is large. To understand why, assume 
that velocity is well regulated, and that the altitude error 
is large, then B « — mgsat(h* —h,h) = —mgh. Ignor¬ 
ing the integrator, this will cause a constant pitch angle 
command from (52.46). However, simply saturating the 
pitch angle will not result in good performance since 
driving 

B = k-U= ~;m(\ V ;(t)\ 1 2 3 -\vM 2 ) 

— mg[h* (t) — h(t)] 

to zero will cause errors in kinetic and potential energy 
to balance. Therefore, a large unsaturated altitude error 
will result in the controller increasing the airspeed er¬ 
ror. A sustained altitude error will cause the integrator 
in (52.46) to wind up. Therefore, an integrator anti¬ 
windup scheme is essential. 

Similar to the lateral control loop, the gains for the 
longitudinal controller can be tuned in flight by selec¬ 
tively enabling one loop at time. The gains should be 
tuned in the following order: 


1. The pitch attitude loops is first tuned. Pitch attitude 
is typically controlled using a PID controller. The 
derivative gain can be tuned to provide adequate 
damping on the RC controller. The proportional 
gain is then added and adjusted to provide adequate 
transient response. The integrator is added after¬ 
ward to remove steady state error in pitch. 

2. The throttle gains kp E and k/ E in (52.44) are then 
adjusted. We have found that normalizing the en¬ 
ergy error E by the reference kinetic energy K rct = 
\m\ w a ef | makes these gains particularly easy to tune. 


Part E | 52.3 




Part E | 52.4 


1324 Part E I Moving in the Environment 


and that the scaling results in similar gains working 
across different platforms. 

3. The energy balance gains k? B and kj B are tuned last. 
Again, scaling B by K' ct seems to simplify the tun¬ 
ing process for different airframes. 

An advantage of the TECS is that it reduces the 
need for different control modes for the longitudinal au¬ 
topilot. In particular, the altitude hold mode , the climb 
mode , and the descend mode described in [52.31] are 
reduced to one mode using TECS. However, we should 


note that the take-off mode described in [52.31] is still 
needed for this scheme. For small aircraft that are hand 
launched, or any aircraft whose velocity at take off is 
significantly below the commanded velocity, the TECS 
control in (52.46) will cause the vehicle to pitch down 
to gain airspeed when the airspeed error is large. This 
pitch-down behavior will cause the airframe to crash if 
it is executed immediately after take-off. Therefore, the 
best strategy is to apply full throttle immediately after 
take-off, and to set the commanded pitch angle to fixed 
value that is adequate for a climb rate that avoids stall. 


52.4 Trajectory Planning 

There are many different control tasks that aerial 
robotic vehicles will be required to undertake in the 
coming years. In this chapter, it is impossible to cover 
the whole range of potential tasks along with their asso¬ 
ciated planning problems. We will consider only basic 
trajectory or path planning problems for both quadrotor 
and fixed-wing vehicles. This problem is fundamental 
to fixed-wing navigation where the distances involved 
are such that path and trajectory planning is sufficient 
for almost all goals. For small aerial robotic vehicles 
flying in cluttered 3-D-spaces, there is a much wider 
selection of goals including things such as, physical 
interaction with the environment, obstacle avoidance, 
etc., as well as simply planning and following a tra¬ 
jectory to a given way-point. However, the trajectory 
planning provides a basic building block in achieving 
a wide range of goals, and is a necessary component of 
achieving high-performance tasks. 

52.4.1 Trajectory Planning for Quadrotors 

The quadrotor is underactuated and this makes it diffi¬ 
cult to plan trajectories in 12-dimensional state space 
(6DOF position and velocity). However, the problem 
is considerably simplified if we use the fact that the 
quadrotor dynamics are differentially flat [52.44]. To 
see this, we consider the output position £ and the yaw 
angle i/'" • We show that we can write all state variables 
and inputs as functions of the outputs (£, VO and their 
derivatives. Derivatives of £ yield the velocity v, and 
the acceleration, 

v = —uib 3 +ge 3 . 
m 

From Fig. 52.3, we see that 

d 1 = [cos i/4 sin 1/4 0] T 


and the unit vectors for the body-fixed frame, can be 
written in terms of the variables and v as follows 


*3 


v-ge 3 
i>-ge 3 


, *2 


b 2 xd\ 

11*3 Xrf 1 


b\ =b 2 xft 3 


provided b 2 xdi / 0. This defines the rotation matrix 
a R b as a function of v (the second derivative of £) and 

In this way, we write the angular velocity and the 
four inputs as functions of the position, velocity (v = 
£), acceleration (a = £), jerk (y = and snap or 

the derivative of jerk (<7 = From these equations, 

it is possible to verify that there is a diffeomorphism 
between the 18 x 1 vector 

X = (£ t , v t , a T , y T , ct t , \j / T , V^) 1 


and the state augmented with the inputs and their 
derivatives 

J2 T , Mi, ill, Hi, . 

This property of differential flatness makes it easy 
to design trajectories that respect the dynamics of the 
underactuated system. Any four-times-differentiable 
trajectory in the space of flat outputs, (lj T (f), \fr(t)) T , 
corresponds to a feasible trajectory, one that satisfies 
the equations of motion. All inequality constraints of 
states and inputs can be expressed as functions of 
the flat outputs and their derivatives. This mapping to 
the space of flat outputs can be used to generate tra¬ 
jectories that minimize a cost functional formed by 
a weighted combination of the different flat outputs and 
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their derivatives 


(mr«) = 


arg min / L(£, v, a, y, < 7 , \j/, \j/, xfr) dr, 


g(%(i),t(t))< 0. 


(52.48) 


In [52.7], minimum snap trajectories were generated by 
minimizing a cost functional derived from the snap and 
the angular yaw acceleration with 

£(£.£.£>?>fV r >V r >$) = (i-«)© 4 +«W 2 • 

By suitable parameterizing trajectories with basis func¬ 
tions in the flat space and by considering linear inequal¬ 
ities in the flat space to model constraints on states and 
inputs (e.g., u\ > 0), it is possible to turn this optimiza¬ 
tion into a quadratic program which can be solved in 
real-time for planning. 

Finally, as shown in [52.45], it is possible to com¬ 
bine this controller with attitude only controllers to fly 
through vertical windows or land on inclined perches 
with close to zero normal velocity. A trajectory con¬ 
troller is used by the robot to build up momentum while 
the attitude controller enables reorientation when coast¬ 
ing with the generated momentum. 

52.4.2 Trajectory Planning 

for Fixed-Wing Aircraft 


For fixed wing vehicles, the critical constraints that in¬ 
fluence path planning is the minimum turn radius and 
the maximum flight path angle. As shown in [52.29, 
3 1], a good approximation for the minimum turn radius 
is given by 

p N 2 7 
Rmm = -tan</> , 

g 

where g is the acceleration of gravity at sea level, and ip 
is the maximum allowable bank angle. From (52.27c), 
we see that a good approximation for the maximum 
flight path angle is 



where h max is the maximum climb/descent rate of the 
airframe. Fixed wing aircraft are typically designed 
to fly at a specific airspeed where the lift-to-drag ra¬ 
tio is maximized, thereby minimizing fuel expenditure. 


Therefore path planners for fixed wing vehicles com¬ 
monly assume a constant airspeed. In this section, we 
will assume zero wind conditions where airspeed equals 
ground speed. Therefore, the path planning problem 
for fixed wing vehicles is naturally posed as planning 
a constant velocity path between two configurations that 
satisfy the minimum turn radius and maximum flight 
path angle constraints. 

If the maneuver is executed at a constant altitude, 
then a traditional Dubins car planner can be used to 
find a constant velocity path satisfying the turn rate 
constraints [52.46]. Using the Dubins car model for 
UAV path planning is a common practice in the litera¬ 
ture [52.31,47-50]. However, the constant altitude lim¬ 
itation can be removed by extending the traditional Du¬ 
bins car model to the Dubins airplane model given by 

h = V cos / , 
w = V sin x , 

X = u\, 
h = —V sin M 2 , 

where (n, w) is the inertial North-West position, V is 
the speed of the Dubins airplane, and in the case of zero 
ambient wind is V = |v a |, and where \u\ \ < V/R m i n , 
and |m 2 | < y [52.51]. 

The configuration of a Dubins airplane is given by 
C = ( n , w, h, x), where n and w are North-West iner¬ 
tial coordinates, h is the altitude, and x is the course 
angle. The Dubins airplane planning problem is to find 
a minimum distance path between a start configuration 
C s and an end configuration C e that satisfies the con¬ 
straints |mi | < V/R m i n , and \ui\ < y. 

Recall that Dubins car paths are composed of three 
segments: a turn segment that follows a circle with the 
minimum turn radius R m i n , a straight line segment, and 
finally another turn segment that again follows a cir¬ 
cle with minimum turn radius [52.46]. The planning 
problem for the Dubins airplane is closely related to 
the planning problem for the Dubins car, with some 
variation due to the altitude component. As explained 
in [52.51,52], Dubins airplane paths can be separated 
into three categories based on the altitude gain/loss 
between the start and end configurations, i. e., Ah = 
\h e ~h s \. 

If Ah is small enough, then the Dubins airplane can 
follow the standard Dubins car path while gaining or 
losing altitude at a constant rate that satisfies the flight 
path angle constraint and that allows the altitude differ¬ 
ence to be realized. Such paths are termed low-altitude 
paths in [52.5 1]. If on the other hand. Ah is so large that 
flying the Dubins car path at the flight path angle con¬ 
straints does not result in sufficient altitude gain/loss, 
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then the Dubins car path must be suitably modified. One 
alternative is to fly at the flight path angle constraint 
while making multiple complete orbits on the minimum 
turn radius helix at the beginning of the Dubins car path 
before proceeding along the straight line segment of the 
path. Of course, a similar strategy would be to make 
multiple complete orbits on the minimum turn radius 
helix at the end of the Dubins car path. Such paths are 
termed high-altitude paths in [52.51]. The middle case 
is when the flight path angle constraint does not allow 
sufficient altitude gain/loss following the Dubins car 
path, but where one complete orbit along the start or end 
circle of the Dubins car path at the maximum flight path 
angle results in more altitude gain/loss than is needed. 
In this case, a deviation can be placed in the Dubins car 
path to extend the path length to just the right amount 
so that the altitude gain/loss can be achieved while fly¬ 
ing at the flight path angle constraint. These paths are 
termed medium-altitude paths in [52.51]. 

Following the notation defined in [52.52] let 
Tear(C s , T e , R) denote the path length of the Dubins car 
path between the projection of the start configuration 
C s onto the North-West plane, and the projection of the 
end configuration C e onto the North-West plane, using 
R as the turn radius of the vehicle. 

Low-Altitude Dubins Paths 
The altitude gain between the start and end configura¬ 
tion is said to be low altitude if 

| Ae h s \ £ L car (C s , C e , I?min) tan y , 

where the term on the right is the maximum altitude 
gain that can be obtained by flying at flight-path angle 
± y for a distance of L car (C s , C e , Rmin)- 

In the low-altitude case, the altitude gain between 
the start and end configurations can be achieved by 
flying the Dubins car path with a flight-path angle satis¬ 
fying |i/ 2 1 < Y ■ Therefore, the optimal flight-path angle 
can be computed by 

* . — 1 ( \h e — h s \ \ 

\L caI (C s , C e , Rmin ) / 

Medium-Altitude Dubins Paths 
The altitude gain between the start and the end config¬ 
uration is said to be medium altitude if 

^car 1 -^min) tcin y “C |/Zg /l§| 

— [^car (c s» » ^min ) + 27ri? min ] tan y , (52.49) 

where the addition of the term 2nR m [ n accounts for 
adding one orbit at radius R„ u n to the path length. 

In the medium-altitude case, the altitude difference 
between the start and end configurations is too large to 


obtain by flying the Dubins car path at the flight-path 
angle constraint, but small enough that adding a full 
turn on the helix at the beginning or end of the path and 
flying so that y = ±y results in more altitude gain/loss 
than is needed. As shown in [52.51], the minimum dis¬ 
tance path is achieved by setting y = sign ( h e — h s ) y 
and inserting an extra maneuver in the Dubins car path 
that extends the path length so that the altitude gain 
when y = ±y is exactly \h e — h s \. While there are nu¬ 
merous possible ways to extend the path length, the 
method proposed in [52.52] is to add an additional in¬ 
termediate arc to the start or end of the path, as shown in 
Figs. 52.8 and 52.9. If the start altitude is lower than the 
end altitude, then the intermediate arc is inserted imme¬ 
diately after the start helix. If on the other hand, the start 
altitude is higher than the end altitude, then the interme¬ 
diate arc is inserted immediately before the end helix. 

High-Altitude Dubins Paths 
The altitude gain between the start and end configura¬ 
tions is said to be high altitude if 

I h e /i s | > [T cal (C s , Cq , Rmtn) + 2jT R mln J tan y . 

In the high-altitude case, the altitude gain cannot 
be achieved by flying the Dubins car path within the 
flight-path angle constraints. As shown in [52.51], the 
minimum distance path is achieved when the flight- 
path angle is set at its limit of ±y, and the Dubins car 
path is extended to facilitate the altitude gain. While 
there are many different ways to extend the Dubins car 
path, [52.52] suggests extending the path by spiraling 
a certain number of turns at the beginning or end of the 
path, and then by increasing the turn radius by the ap¬ 
propriate amount. 

For UAV scenarios, the most judicious strategy is 
typically to spend most of the trajectory at as high an 
altitude as possible. Therefore, if the altitude at the end 
configuration is higher than the altitude at the start con¬ 
figuration, then the path will be extended by a climbing 
helix at the beginning of the path. If on the other hand, 
the altitude at the start configuration is higher than the 
end configuration, then the path will be extended by 
a descending helix at the end of the path. If multiple 
turns around the helix are required, then the turns could 
be split between the start and end helices and still re¬ 
sult in the same path length. For high-altitude Dubins 
paths, the required number of turns in the helix will be 
the smallest integer k such that 

[Tcar(f-s, f'e, A* m jn) I 27TArA IT1 j n J tan y < h c As I 
< [T ca r(C s , Ce,i?min) + 27 x{k-\- ljl^min] tan y , 

( 52 . 50 ) 
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Fig. 52.8 A top down view of low altitude, medium alti¬ 
tude, and high-altitude Durbin airplane paths. The initial 
and end configurations are identical except for the final al¬ 
titude 

or in other words 





Fig. 52.9 A three-dimensional view of low altitude, 
medium altitude, and high-altitude Durbin airplane paths. 
The initial and end configurations are identical except for 
the final altitude 


k = 


1 ~\h e -h s \ 
_2jtR min l tan y 


L car (C s ,C e ,f?„ 



where |xj is the floor function that rounds x down to the 
nearest integer. The radius of the start and end helices 
is then increased to R* so that 


[L car (C s , C e , R*) + 2nkR*] tan y = \h e -h s \ . 

( 52 . 51 ) 

Figures 52.8 and 52.9 show two perspective of three 
different Dubins airplane paths that start and end in the 
same configuration with the exception of the final al¬ 
titude, which changes from 250 m for the low-altitude 
path, to 350 m for the medium-altitude path, to 450 m 
for the high-altitude path. A top down view is shown 
in Fig. 52.8. Note that for both the low-altitude and 
medium-altitude paths, that the minimum turn radius 
constraint is active, whereas for the high-altitude path, 
the turn radius in the turns is larger than the mini¬ 
mum turn radius constraint. A three-dimensional view 


is shown in Fig. 52.9. Note that for both the medium- 
and high-altitude paths, the flight path angle constraint 
is active, but that it is not active for low-altitude paths. 

There are a variety of techniques that allow the 
control architecture described in Sect. 52.3.2 to follow 
Dubins airplane paths. A simple method is to parame¬ 
terize the path using a path parameter a. Suppose that 
the resulting parameterized path is given by p{a). At 
each sample time, the path parameter is advanced along 
the path so as to minimize the distance to the path from 
the aircraft 

ct,+ i = argmin ||^(t)-p(j)|| , 

S>G, 


where £(t) is the inertial position of the aircraft. The 
commanded airspeed \v*\, course angle /*, and alti¬ 
tude h* at time t are then given by the parameterized 
Dubins airplane path p{a t - pi). This technique is similar 
to the guidance strategy suggested in [52.53]. Alterna¬ 
tive method based on vector field methods are described 
in [52.54-56], 
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52.5 Estimating the Vehicle State 

A critical aspect in implementation of real world aerial 
robotic vehicles is providing good estimates of the state 
of the vehicle. The key state estimates required for 
control of an aerial vehicle are associated with the rigid- 
body dynamics of its airframe, height, attitude, angular 
velocity, and linear velocity. Moreover, fixed-wing ve¬ 
hicle dynamics depend on the aerodynamics states of 
angle-of-attack and side-slip angle. Of these states, the 
attitude and angular velocity are the most important as 
they are the primary variable used in attitude control 
and flight-regulation of the vehicle. Angle-of-attack and 
side-slip angles are rarely estimated explicitly and are 
dealt with as internal dynamics in the control design 
rather than explicit outputs or disturbances. 

The ubiquitous instrumentation carried by any 
aerial vehicle is an inertial measurement unit (IMU) 
often augmented by some form of height measure¬ 
ment, either acoustic, infra red, barometric, or laser 
based. Vehicles that fly in outside environments carry 
a GPS system, most of which now provide velocity 
estimation as well as position. Many fixed wing vehi¬ 
cles also include a pitot tube (for dynamic pressure) 
or an anemometer to measure forward velocity. In¬ 
door aerial robotic vehicles in research laboratories are 
often flown in flight environments equipped with mo¬ 
tion capture systems like VICON or Optitrack [52.5, 
6]. Finally, many aerial robotic systems have also been 
equipped with exteroceptive sensor systems including 
Kinect 3D-range cameras, scanning laser rangefinder, 
integrated stereo camera systems, or simple monocular 
cameras. There is an active research field in developing 
3-D-SLAM (simultaneous localization and mapping; 
Chap. 46.4) algorithms for mapping and localization of 
aerial robotic vehicles. 

The rigid-body dynamic state is crucial to the con¬ 
trol performance of an aerial robotic system and we will 
concentrate on this estimation problem in this chap¬ 
ter. A natural approach to state-estimation is to apply 
classical filter design to a coordinate representation 
of the rigid-body state. A good development of this 
approach for small-scale aerial robotic systems is pre¬ 
sented in [52.31]. An alternative approach is to exploit 
the nonlinear structure of the rigid-body kinematics to 
develop nonlinear observers. This approach has been 
shown to generate simple, robust, and highly effective 
filters for small-scale aerial robotic systems, especially 
for the key problem of attitude estimation. 

52.5.1 Estimating Attitude 

A typical IMU includes a three-axis rate gyro, three- 
axis accelerometer, and three-axis magnetometer. The 


rate gyro measures angular velocity of {5} relative to 
{A} expressed in the body-fixed-frame of reference {B} 

f^iMu = + bf2 + t] e {B }, 

where t] denotes additive measurement noise and bn 
denotes a constant (or slowly time-varying) gyro bias. 
Generally, the gyroscopes installed on quadrotor vehi¬ 
cles are lightweight MEMS devices that are reasonably 
robust to noise and quite reliable. 

The accelerometers (in a strap down IMU configu¬ 
ration) measure the instantaneous linear acceleration of 
{5} due to exogenous force 

aiMU = R T (v + ge i ) + b !l + e {B} , (52.52) 

where h a is a bias term, tj a denotes additive measure¬ 
ment noise, and v is in the inertial frame. Accelerom¬ 
eters are highly susceptible to vibration and mounted 
on a typical aerial robotic platform they require signif¬ 
icant low-pass mechanical and/or electrical filtering to 
be reliable. Most vehicles avionics will incorporate an 
analogue anti-aliasing filter on a MEMS accelerometer 
before the signal is sampled. 

The magnetometers provide measurements of the 
ambient magnetic field 

'«imu = R J A m + B m + r)b £ {B} , 

where A m is the Earth’s magnetic field vector (ex¬ 
pressed in the inertial frame), B m is a body-fixed-frame 
expression for the local magnetic disturbance and r]b 
denotes measurement noise. The noise rp, is usually 
quite low for magnetometer readings, however, the lo¬ 
cal magnetic disturbance B,„ can be very significant, 
especially if the sensor is placed near power cables for 
the motors. 

In outdoor environments, the GPS is typically used 
to provide position, inertial velocity, and course angle. 
The GPS system consists of a constellation of 24 satel¬ 
lites that continuously orbit the earth at an altitude of 
20180 km. The position of a GPS receiver is deter¬ 
mined by observing the time-of-flight of signals sent 
from the satellites and detected by the receiver. If ac¬ 
curate timing information is available at the receiver, 
then a minimum of three satellites signals are neces¬ 
sary to resolve the position of the receiver. However, for 
low-cost GPS receivers, timing information must also 
be received and this requires a minimum of four satel¬ 
lite signals. A variety of factors effect the accuracy of 
GPS estimates. The dominant sources of error include 
inaccurate satellite orbital data, inaccuracy in the satel¬ 
lite clocks, variable signal delay as it passes through 
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the ionosphere, weather conditions near the earth sur¬ 
face, and multipath reflections from nearby buildings 
and mountains. Total GPS bias error is approximately 
5—10 m. Modem GPS receivers exploit doppler shift in 
the carrier phase of the received signals to estimate the 
course angle '/ and the ground speed. 

The accelerometers and magnetometers can be used 
to provide absolute attitude information on the vehicle 
while the rate gyroscope provides complementary 
angular velocity measurements. The attitude informa¬ 
tion in the magnetometer signal is straightforward to 
understand; in the absence of noise and bias then otimu 
provides a body-fixed frame measurement of R JA m, 
and consequently constrains two degrees of freedom in 
the rotation R. The accelerometer can also be used as 
long as the component of acceleration associated with 
the inertial motion of the vehicle is compensated. The 
simplest way in which to do this is to use an absolute 
external signal such as GPS to estimate the vehicle 
acceleration 

flCTD = aiMU — ^ $gps * 

where actd is the corrected acceleration. In this case, 
rzcTD ~ R T e 3 and this provides attitude information. It 
is important to note that in the complementary fil¬ 
ter proposed below (52.53), only the low-frequency 
component of «ctd is required. As such, the potential 
noise in the second derivative of a GPS signal £gps 
is not necessarily the problem that may be initially 
feared. The key to good filter performance is to en¬ 
sure that the cross over frequency of the complimentary 
sensitivity is low enough that any phase distortion asso¬ 
ciated with low-pass filtering of £gps does not effect the 
estimate. 


Remark 52.1 

There may be issues in applying the correction R t ^gps 
if there is a delay in receiving the GPS signals. This 
can be addressed, at the cost of storing IMU data, by 
combining a short time prediction along with a time- 
lagged observer [52.57]. 


In the case where GPS or an external measure of the 
vehicles motion is not available, there are a number of 
alternative techniques available to use the accelerome¬ 
ter information for attitude estimation. For fixed-wing 
vehicles, the most important disturbance is due to cen¬ 
tripetal acceleration due to the vehicle turning. It is 
possible to model the centripetal acceleration using 
airspeed and angular velocity and compensate using 
a feedforward term [52.58,59]. In the case of rotor- 
craft in mostly hover flight conditions, it turns out 


that using the raw IMU accelerometer readings still 
works extremely well [52.60-62] due to the disturbance 
caused by the aerodynamic drag term A = —Dv (52.10) 
[52.21, 24,25, 63]. In this case the corrected accelerom¬ 
eter flcTD used in the filter (52.53) below is just the raw 
IMU accelerometer measurement. 

The attitude kinematics of the quadrotor are given 
by (52.2c). Let R denote an estimate for attitude R of 
the quadrotor vehicle. The following observer [52.61, 
62] fuses accelerometer, magnetometer, and gyroscope 
data as well as other direct attitude estimates Re (such 
as provided by a VICON or other external measurement 
system) should they be available 


R R ^I2imu — b} — 1 ■ 

(52.53a) 

II 

■<*£ 

(52.53b) 

f [(-R T e 3 ) x «ctd j 


+ [(^ TA 'W) x m TMu] j 


+ 

IP 

O 

If 

(52.53c) 


where k a , k m , k E , and k/, are arbitrary nonnegative 
observer gains and P so ( 3 )(M) = (M —M T )/2 is the 
Euclidean matrix projection onto the skew-symmetric 
matrices. If any one of the measurements in the in¬ 
novation T are not available or unreliable, then the 
corresponding gain should be set to zero in the ob¬ 
server. Note that both the attitude R and the bias 
corrected angular velocity 12 = I 2 jmu — b are estimated 
by this observer. The observer (52.53) has been exten¬ 
sively studied in the literature [52.61, 62] and shown to 
converge exponentially (both theoretically and exper¬ 
imentally) to the desired attitude estimate of attitude 
with b converging to the gyroscope bias b. The filter has 
a complementary nature, using the high-frequency part 
of the gyroscope signal and the low-frequency parts of 
the magnetometer, accelerometer, and external attitude 
measurements [52.61]. The roll off frequencies associ¬ 
ated with each of these signals is given by the gains k a , 
k m , and k E in rad/s. 

52.5.2 Estimating Velocity and Position 

Estimating velocity and position is a straightforward 
process if the vehicle is equipped with a GPS system. 
In this case, the dynamics (52.2a) and (52.2b) are linear, 
and a linear filter can be used. Let £ denote the estimate 
of position and v denote the estimate of velocity. A sim- 
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pie linear filter is given by 

f = v-k x ($-%G ps) , 
v = ^ T aimu — ge 3 — k v (| — ^gps) , 

for gains k x , k v > 0. As long as the attitude estimate 
R is accurate, this filter design is highly robust. It is 
easily possible to add bias estimates for the accelerom¬ 
eter if desired. Equally well it is straightforward to use 
Kalman filter techniques to tune the gains k x and k v in 
real-time if that is considered advisable. In practice, for 
small aerial robotic systems, the noise characteristics of 
the measurements are so poor that it is often best to use 
a constant gain filter rather than introduce the additional 
complexity and potential instability of the Riccati equa¬ 
tion associated with a Kalman filter. 

In the absence of GPS, estimating position depends 
on the availability of additional sensor systems. Since 
most aerial robots are equipped with a barometer to pro¬ 
vide an estimate of height and this can be utilized for an 
estimate of the vertical motion of the vehicle 

h = v z — k],(h — hi) , (52.54a) 

v z = e]R T a mv -g-k Vz (h-h ), (52.54b) 

where ki,,k Vz > 0 are positive gains. 

For quadrotor vehicles, there is also the possibility 
of exploiting the linear drag aerodynamic forces (52.10) 
associated with blade flapping and induced drag to es¬ 
timate horizontal velocity for a vehicle in near hover 
conditions [52.24]. Define a projector matrix 



that takes the first two components of a vector. Then the 
horizontal component of the inertial acceleration can be 
measured by 

A a\, := P/, A a = P i,Ra sa P h Ra , (52.56) 


52.6 Conclusion 

This chapter has focused on the fundamental technol¬ 
ogy associated with making an aerial robotic system 
operate effectively and efficiently. In particular, we have 
focused on the design of control and navigation algo¬ 
rithms, and along with that the associated questions of 
modeling and state estimation. Once a robust and re¬ 
liable aerial robotic platform is available, the range of 


where we assume that the estimate R is close to R. If we 
assume the vehicle is only slowly varying in height one 
has v z 0 in comparison to horizontal velocity 

v h pb P^v . 

Furthermore, the thrust T ss mg must compensate the 
weight of the vehicle. Recalling (52.10) and taking just 
the horizontal component one has 

% » -gP h Re 3 - gP h RDR r P^v h . (52.57) 

If the attitude filter estimate is good and the as¬ 
sumptions about vehicle motion hold then (52.56) and 
(52.57) can be solved for an estimate of Vh 

v h~-- (PhRDR'Pl) 1 ( A a h + gP h Re 3 ) ■ 

(52.58) 

Equation (52.58) provides a measurement of the 
horizontal velocity, however, since it directly incorpo¬ 
rates the unfiltered accelerometer readings it is gener¬ 
ally too noisy to be of much use. Its low-frequency 
content can, however, be used to drive a velocity com¬ 
plementary observer that uses the attitude estimate and 
the system model (52.2b). Let Vh be an estimate of the 
horizontal component of the inertial velocity of the ve¬ 
hicle then an observer is given by 

Vh = -gP^(Re 3 +RDR T P^v h )-k w (v h -v h ) , (52.59) 

where Vh is given by (52.58). The gain k w > 0 provides 
a tuning parameters that indicates the roll-off frequency 
for the information from 0/, that is used in the filter. It 
also uses estimated velocity 0/, to provide an approx¬ 
imation of the more correct RDR T P^Vh term in the 
feedforward velocity estimate, however, since the un¬ 
derlying dynamics associated with this term is stable, 
then the observer is stable even with this approximation. 
It is found that this observer works remarkably well in 
practice for quadrotor vehicles that are not engaged in 
acrobatic maneuvers [52.21, 24,25, 63]. 


potential applications for unmanned aerial systems is 
vast. An indication of the potential is given by a list 
of application categories that are already under con¬ 
sideration [52.64,65]: Remote sensing such as pipeline 
spotting, power-line monitoring, volcanic sampling, 
mapping, meteorology, geology, and agriculture [52.66, 
67], as well as unexploded mine detection [52.68]. 
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Disaster response such as chemical sensing, flood mon¬ 
itoring, and wildfire management. Sur\'eillance such as 
law enforcement, traffic monitoring, coastal and mar¬ 
itime patrol, and border patrols [52.69]. Search and 
rescue in low-density or hard-to-reach areas. Trans¬ 
portation including small and large cargo transport, 
and possibly passenger transport. Communications as 
permanent or ad hoc communication relays for voice 


and data transmission, as well as broadcast units for 
television or radio. Payload delivery for a wide range 
of applications including agriculture, firefighting, and 
even logistics of product delivery. Image acquisition 
for cinematography and real-time entertainment. It is 
no surprise that aerial robotics is one of the most dy¬ 
namic and exciting fields of robotics research at the 
moment. 
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Autopilot using total energy control 

available from http://handbookofrobotics.org/view-chapter/52/videodetails/436 
|43>M20HESI Dubins airplane 

available from http://handbookofrobotics.org/view-chapter/52/videodetails/437 
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53. Multiple Mobile Robot Systems 


Lynne E. Parker, Daniela Rus, Gaurav S. Sukhatme 


Within the context of multiple mobile, and net¬ 
worked robot systems, this chapter explores the 
current state of the art. After a brief introduction, 
we first examine architectures for multirobot co¬ 
operation, exploring the alternative approaches 
that have been developed. Next, we explore com¬ 
munications issues and their impact on multirobot 
teams in Sect. 53.3, followed by a discussion of 
networked mobile robots in Sect. 53.4. Following 
this we discuss swarm robot systems in Sect. 53.5 
and modular robot systems in Sect. 53.6. While 
swarm and modularsystems typically assume large 
numbers of homogeneous robots, other types of 
multirobot systems include heterogeneous robots. 
We therefore next discuss heterogeneity in coop¬ 
erative robot teams in Sect. 53.7. Once robot teams 
allow for individual heterogeneity, issues of task 
allocation become important; Sect. 53.8 therefore 
discusses common approaches to task allocation. 
Section 53.9 discusses the challenges of multirobot 
learning, and some representative approaches. We 
outline some of the typical application domains 
which serve as test beds for multirobot systems 
research in Sect. 53.10. Finally, we conclude in 
Sect. 53.11 with some summary remarks and sug¬ 
gestions for further reading. 


53.1 History . 1336 

53.2 Architectures for Multirobot Systems ... 1337 

53.2.1 The Nerd Herd . 1337 

53.2.2 The ALLIANCE Architecture . 1338 

53.2.3 The Distributed Robot 

Architecture . 1339 

53.3 Communication . 1339 


53.4 Networked Mobile Robots . 1340 

53.4.1 Overview . 1341 

53.4.2 State of the Art and Potential .. 1342 

53.4.3 Research Challenges . 1344 

53.4.4 Control . 1346 

53.4.5 Communication for Control . 1347 

53.4.6 Communication for Perception. 1347 

53.4.7 Control for Perception . 1349 

53.4.8 Control for Communication . 1350 

53.5 Swarm Robots . 1351 

53.6 Modular Robotics . 1354 

53.6.1 Chain Systems . 1354 

53.6.2 Lattice Systems . 1354 

53.6.3 Truss Systems . 1356 

53.6.4 Free-Form Systems . 1356 

53.6.5 Self-Assembling Systems . 1356 

53.7 Heterogeneity . 1357 

53.8 Task Allocation . 1359 

53.8.1 Taxonomy for Task Allocation ... 1359 

53.8.2 Representative Approaches . 1360 

53.9 Learning . 1361 

53.10 Applications . 1362 

53.10.1 Foraging and Coverage . 1362 

53.10.2 Flocking and Formations . 1362 

53.10.3 Object Transportation 

and Cooperative Manipulation . 1363 

53.10.4 Multitarget Observation . 1364 

53.10.5 Traffic Control and Multirobot 

Path Planning . 1365 

53.10.6 Soccer . 1365 

53.11 Conclusions and Further Reading . 1366 

Video-References . 1366 

References . 1367 


Part E | 53 





































Part E | 53.1 


1336 Part E I Moving in the Environment 


Researchers generally agree that multirobot sys¬ 
tems have several advantages over single-robot sys¬ 
tems [53.1,2]. The most common motivations for de¬ 
veloping multirobot system solutions are that: 

1. The task complexity is too high for a single robot to 
accomplish. 

2. The task is inherently distributed. 

3. Building several resource-bounded robots is much 
easier than having a single powerful robot. 

4. Multiple robots can solve problems faster using par¬ 
allelism. 

5. The introduction of multiple robots increases ro¬ 
bustness through redundancy. 

The issues that must be addressed in developing 
multirobot solutions are dependent upon the task re¬ 
quirements and the sensory and effector capabilities of 
the available robots. 


53.1 History 

Since the earliest work on multiple mobile robot sys¬ 
tems in the 1980s, the field has grown significantly, and 
covers a large body of research. At the most general 
level, approaches to multiple mobile robot systems fall 
into one of two broad categories: collective swarm sys¬ 
tems and intentionally cooperative systems. Collective 
swarm systems are those in which robots execute their 
own tasks with only minimal need for knowledge about 
other robot team members. These systems are typified 
by the assumption of a large number of homogeneous 
mobile robots, in which robots make use of local con¬ 
trol laws to generate globally coherent team behaviors, 
with little explicit communication among robots. On 
the other hand, robots in intentionally cooperative sys¬ 
tems have knowledge of the presence of other robots 
in the environment and act together based on the state, 
actions, or capabilities of their teammates in order to 
accomplish the same goal. Intentionally cooperative 
systems vary in the extent to which robots take into ac¬ 
count the actions or state of other robots, and can lead to 
either strongly or weakly cooperative solutions [53.3]. 
Strongly cooperative solutions require robots to act in 
concert to achieve the goal, executing tasks that are 
not trivially serializable. Typically, these approaches re¬ 
quire some type of communication and synchronization 
among the robots. Weakly cooperative solutions allow 
robots to have periods of operational independence, 
subsequent to coordinating their selection of tasks or 
roles. Intentionally cooperative multirobot systems can 


The types of robots considered in the study of mul¬ 
tiple mobile robot systems are those robots that move 
around in the environment, such as ground vehicles, 
aerial vehicles, or underwater vehicles. This chapter fo¬ 
cuses specifically on the interaction of multiple mobile 
robots, as distinguished from other types of multirobot 
interaction. For example, a special case of multiple mo¬ 
bile robot systems are the reconfigurable or modular 
robots that interconnect with each other for the purposes 
of navigation or manipulation. Algorithmic aspects of 
these systems are covered in Sect. 53.5, while hardware 
aspects are covered in Chap. 22. Networked robotics is 
very closely related to multiple mobile robot systems; 
the focus in networked robotics is on systems of robots, 
sensors, embedded computers, and human users that are 
all connected by networked communication. Another 
variant of multirobot cooperation is multiple manipula¬ 
tor arm cooperation. Chapter 39 describes these systems 
in detail. 


deal with heterogeneity in the robot team members, in 
which team members vary in their sensor and effector 
capabilities. In these teams, the coordination of robots 
can be very different from collective swarm approaches, 
since robots are no longer interchangeable. 

Most of the work specific to multiple mobile robot 
cooperation can be categorized into a set of key top¬ 
ics of study. These topics, which are the foci of this 
chapter, include architectures, communication, swarm 
robots, heterogeneity, task allocation, and learning. Ar¬ 
chitectures and communication in multirobot systems 
are relevant for all types of multirobot systems, as these 
approaches specify how the robot team members are 
organized and interact. Swarm robots is a particular 
type of multirobot system, typified by large numbers 
of homogeneous robots that interact implicitly with 
each other. Such systems are often contrasted with 
heterogeneous robots, in which team members may 
vary significantly in their capabilities. When robots 
vary in capabilities, challenges arise in determining 
which robots should perform which tasks - a chal¬ 
lenge commonly referred to as task allocation. Finally 
learning in multirobot teams is of particular inter¬ 
est in designing teams that are adaptive over time 
and can learn new behaviors. Illustrating the advances 
in each of these areas often takes place in a set 
of representative application domains; these applica¬ 
tions are the final major topic of discussion in this 
chapter. 
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53.2 Architectures for Multirobot Systems 


The design of the overall control architecture for the 
multirobot team has a significant impact on the robust¬ 
ness and scalability of the system. Robot architectures 
for multirobot teams are composed of the same fun¬ 
damental components as in single-robot systems, as 
described in Chap. 12. However, they also must address 
the interaction of robots and how the group behavior 
will be generated from the control architectures of the 
individual robots in the team. Several different philoso¬ 
phies for multirobot team architectures are possible; the 
most common are centralized, hierarchical, decentral¬ 
ized, and hybrid. 

Centralized architectures that coordinate the entire 
team from a single point of control are theoretically 
possible [53.4], although often practically unrealistic 
due to their vulnerability to a single point of failure, 
and due to the difficulty of communicating the entire 
system state back to the central location at a frequency 
suitable for real-time control. Situations in which these 
approaches are relevant are cases in which the central¬ 
ized controller has a clear vantage point from which to 
observe the robots, and can easily broadcast group mes¬ 
sages for all robots to obey [53.5]. 

Hierarchical architectures are realistic for some ap¬ 
plications. In this control approach, each robot oversees 
the actions of a relatively small group of other robots, 
each of which in turn oversees yet another group of 
robots, and so forth, down to the lowest robot, which 
simply executes its part of the task. This architecture 
scales much better than centralized approaches, and is 
reminiscent of military command and control. A point 
of weakness for the hierarchical control architecture is 
recovering from failures of robots high in the control 
tree. 

Decentralized control architectures are the most 
common approach for multirobot teams, and typically 
require robots to take actions based only on knowledge 
local to their situation. This control approach can be 
highly robust to failure, since no robot is responsible 
for the control of any other robot. However, achiev¬ 
ing global coherency in these systems can be difficult, 
because high-level goals have to be incorporated into 
the local control of each robot. If the goals change, 
it may be difficult to revise the behavior of individual 
robots. 

Hybrid control architectures combine local control 
with higher-level control approaches to achieve both ro¬ 
bustness and the ability to influence the entire team’s 
actions through global goals, plans, or control. Many 
multirobot control approaches make use of hybrid ar¬ 
chitectures. 


A plethora of multirobot control architectures have 
been developed over the years. We focus here on three 
early approaches that illustrate the spectrum of control 
architectures. The first, the Nerd Herd, is representative 
of a pure swarm robotics approach using large num¬ 
bers of homogeneous robots. The second, ALLIANCE, 
is representative of a behavior-based approach that 
enables coordination and control of possibly heteroge¬ 
neous robots without explicit coordination. The third, 
distributed robot architecture (DIRA), is a hybrid ap¬ 
proach that enables both robot autonomy and explicit 
coordination in possibly heterogeneous robot teams. 

53.2.1 The Nerd Herd 

One of the first studies of social behaviors in multi¬ 
robot teams was conducted by Mataric [53.6], with 
results being demonstrated on the Nerd Herd team of 
20 identical robots (shown in Fig. 53.1). This work is 
an example of swarm robotic systems, as described fur¬ 
ther in Sect. 53.4. The decentralized control approach 
was based on the subsumption architecture (Chap. 12), 
and assumed that all robots were homogeneous, but 
with relatively simple individual capabilities, such as 
detecting obstacles and kin (i. e., other robot team 
members). A set of basic social behaviors were de¬ 
fined and demonstrated, including obstacle avoidance, 
homing, aggregation, dispersion, following, and safe 
wandering. These basic behaviors were combined in 
various ways to yield more composite social behaviors, 
including flocking (composed of safe wandering, aggre¬ 
gation, and dispersion), surrounding (composed of safe 
wandering, following, and aggregation), herding (com¬ 
posed of safe wandering, surrounding, and flocking), 
and foraging (composed of safe wandering, dispersion, 
following, homing, and flocking). The behaviors were 



Fig. 53.1 The Nerd Herd robots 
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implemented as rules, such as the following rule for ag¬ 
gregate: 


time step, the motivation level is recalculated based 
on: 


Aggregate: 

If agent is outside aggregation 
distance 

turn toward aggregation centroid 
and go. 

Else 
stop. 

This work showed that collective behaviors could be 
generated through the combination of lower-level ba¬ 
sic behaviors. Related work on this project studied 
issues such as using bucket brigades to reduce interfer¬ 
ence [53.7], and learning [53.8]. 

53.2.2 The ALLIANCE Architecture 

Another early work in multirobot team architectures 
is the ALLIANCE architecture (Fig. 53.2), developed 
by Parker [53.9] for fault-tolerant task allocation in 
heterogeneous robot teams. This approach builds on 
the subsumption architecture by adding behavior sets 
and motivations for achieving action selection with¬ 
out explicit negotiations between robots. Behavior sets 
group low-level behaviors together for the execution of 
a particular task. The motivations consist of levels of 
impatience and acquiescence that can raise and lower 
a robot’s interest in activating a behavior set corre¬ 
sponding to a task that must be accomplished. 

In this approach, the initial motivation to perform 
a given behavior set is set to zero. Then, at each 


Alliance 



1. The previous motivation level 

2. The rate of impatience 

3. Whether the sensory feedback indicates the behav¬ 
ior set is needed 

4. Whether the robot has another behavior set already 
activated 

5. Whether another robot has recently begun work on 
this task 

6. Whether the robot is willing to give up the task, 
based on how long it has been attempting the task. 

Effectively, the motivation continues to increase at 
some positive rate unless one of four situations oc¬ 
curs: 

1. The sensory feedback indicates that the behavior set 
is no longer needed. 

2. Another behavior set in the robot activates. 

3. Some other robot has just taken over the task for the 
first time. 

4. The robot has decided to acquiesce the task. 

In any of these four situations, the motivation re¬ 
turns to zero. Otherwise, the motivation grows until it 
crosses a threshold value, at which time the behavior 
set is activated and the robot can be said to have selected 
an action. When an action is selected, cross-inhibition 
within that robot prevents other tasks from being ac¬ 
tivated within that same robot. When a behavior set is 
active in a robot, the robot broadcasts its current activity 
to other robots at a periodic rate. 

The L-ALLIANCE extension [53.10] allows a robot 
to adapt the rate of change of the impatience and ac- 



Fig. 53.2 The ALLIANCE architecture 


Fig. 53.3 Robots using the ALLIANCE architecture for 
a mock clean-up task 
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quiescence values depending on the quality with which 
that robot is expected to accomplish a given task. The 
result is that robots that have demonstrated their abil¬ 
ity to better accomplish certain tasks are more likely 
to choose those tasks in the future. Additionally, if 
problems occur during team performance, then robots 
may dynamically reallocate their tasks to compensate 
for the problems. This approach was demonstrated 
on a team of three heterogeneous robots performing 
a mock clean-up task, two robots performing a box¬ 
pushing task, and four robots performing a cooperative 
target observation problem. The approach has also been 
demonstrated in the simulation of a janitorial service 
task and a bounding overwatch task. Figure 53.3 shows 
robots using ALLIANCE to perform the mock clean-up 
task. 


53.2.3 The Distributed Robot 
Architecture 


Simmons et al. [53.11] have developed a hybrid archi¬ 
tecture called the distributed robot architecture (DIRA). 
Similar to the Nerd Herd and ALLIANCE approaches, 
the DIRA approach allows autonomy in individual 
robots. However, unlike the previous approaches, DIRA 
also facilitates explicit coordination among robots. This 
approach is based on layered architectures that are 
popular for single-robot systems (Chap. 12). In this 
approach (Fig. 53.4), each robot’s control architecture 



Fig. 53.5 Robots using the distributed robot architecture 
for assembly tasks 


consists of a planning layer that decides how to achieve 
high-level goals; an executive layer that synchronizes 
agents, sequences tasks, and monitors task execution; 
and a behavioral layer that interfaces to the robot’s sen¬ 
sors and effectors. Each of these layers interacts with 
those above and below it. Additionally, robots can in¬ 
teract with each other via direct connections at each of 
the layers. 

This architecture has been demonstrated in a team 
of three robots - a crane, a roving eye, and a mo¬ 
bile manipulator - performing a construction assembly 
task (Fig. 53.5). This task requires the robots to work 
together to connect a beam at a given location. In 
these demonstrations, a foreman agent decides which 
robot should move the beam at which times. Initially, 
the crane moves the beam to the vicinity of the em¬ 
placement based on encoder feedback. The foreman 
then sets up a behavioral loop between the roving eye 
and the crane robot to servo the beam closer to the 
point of emplacement. Once the beam is close enough, 
the foreman tasks the roving eye and the mobile ma¬ 
nipulator to servo the arm to grasp the beam. After 
contact is made, the foreman tasks the roving eye 
and the mobile manipulator to coordinate to servo the 
beam to the emplacement point, thus completing the 
task. 


53.3 Communication 

A fundamental assumption in multirobot systems re¬ 
search is that globally coherent and efficient solutions 
can be achieved through the interaction of robots lack¬ 
ing complete global information. However, achieving 
these globally coherent solutions typically requires 
robots to obtain information about their teammates’ 
states or actions. This information can be obtained in 


a number of ways; the three most common techniques 
are: 

1. The use of implicit communication through the 
world (called stigmergy), in which robots sense the 
effects of teammate’s actions through their effects 
on the world [53.6, 12-16] 
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2. Passive action recognition, in which robots use 
sensors to directly observe the actions of their team¬ 
mates [53.17] 

3. Explicit (intentional) communication, in which 
robots directly and intentionally communicate rele¬ 
vant information through some active means, such 
as radio [53.9, 18-21] - an area widely studied in 
the field of networked robot systems. 

Each of these mechanisms for exchanging infor¬ 
mation between robots has its own advantages and 
disadvantages [53.22]. Stigmergy is appealing because 
of its simplicity and its lack of dependence upon ex¬ 
plicit communications channels and protocols. How¬ 
ever, it is limited by the extent to which a robot’s 
perception of the world reflects the salient states of 
the mission the robot team must accomplish. Passive 
action recognition is appealing because it does not de¬ 
pend upon a limited-bandwidth, fallible communication 
mechanism. As with implicit cooperation, however, it 
is limited by the degree to which a robot can success¬ 
fully interpret its sensory information, as well as the 
difficulty of analyzing the actions of robot team mem¬ 
bers. Finally, the explicit communication approach is 
appealing because of its directness and the ease with 
which robots can become aware of the actions and/or 
goals of its teammates. The major uses of explicit 
communication in multirobot teams are to synchro¬ 
nize actions, exchange information, and to negotiate 
between robots. Explicit communication is a way of 
dealing with the hidden-state problem [53.23], in which 
limited sensors cannot distinguish between different 
states of the world that are important for task perfor¬ 
mance. However, explicit communication is limited in 
terms of fault tolerance and reliability, because it typ¬ 
ically depends upon a noisy, limited-bandwidth com¬ 
munications channel that may not continually connect 
all members of the robot team. Thus, approaches that 
make use of explicit communications must also provide 
mechanisms to handle communication failures and lost 
messages. 

Selecting the appropriate use of communication in 
a multirobot team is a design choice dependent upon the 
tasks to be achieved by the multirobot team. One needs 
to carefully consider the costs and benefits of alternative 

53.4 Networked Mobile Robots 

Networked robots are multiple robots operating to¬ 
gether coordinating and cooperating by networked com¬ 
munication to accomplish a specified task. Multiple 
robots enable new capabilities and the communication 
network enables new approaches and solutions that are 


communications approaches to determine the method 
that can reliably achieve the required level of system 
performance. Researchers generally agree that commu¬ 
nication can have a strong positive impact on the per¬ 
formance of the team. One of the earliest illustrations 
of this impact was given in the work of MacLennan and 
Burghardt [53.24], which investigates the evolution of 
communication in simulated worlds and concludes that 
the communication of local robot information can result 
in significant performance improvements. Interestingly, 
for many representative applications, researchers have 
found a nonlinear relationship between the amount of 
information communicated and its impact on the per¬ 
formance of the team. Typically, even a small amount of 
information can have a significant impact on the team, 
as found in the study of Balch and Arkin [53.25]. How¬ 
ever, more information does not necessarily continue 
to improve performance, as it can quickly overload 
the communications bandwidth without providing an 
application benefit. The challenge in multirobot sys¬ 
tems is to discover the optimal pieces of information 
to exchange that yield these performance improvements 
without saturating the communications bandwidth. Cur¬ 
rently, no general approaches to identifying this critical 
information are available; thus, the decision of what to 
communicate is an application-specific question to be 
answered by the system designer. Dudek et al.’s tax¬ 
onomy of multirobot systems [53.26] includes axes 
related to communication, including communication 
range, communication topology, and communication 
bandwidth. These characteristics can be used to com¬ 
pare and contrast multirobot systems. 

Several related issues of active research in commu¬ 
nications for multirobot teams deal with dynamic net¬ 
work connectivity and topologies; for example, robot 
teams must either be able to maintain communications 
connectivity as they move, or employ recovery strate¬ 
gies that allow the robot team to recover when the com¬ 
munications connectivity is broken. These concerns 
may require robots to adapt their actions in response to 
the anticipated effects on the communications network, 
or in response to knowledge of the anticipated prop¬ 
agation behavior of information through the dynamic 
network. These and related issues are discussed next in 
the context of networked robot systems. 


difficult with just perception and control. Communica¬ 
tion enables new control and perception capabilities in 
the system (e.g., access to information outside the per¬ 
ception range of the robot system). Conversely, control 
enables solutions for problems that are difficult with- 
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out mobility (e.g., localization). Section 53.4.1 defines 
the field, examines the benefits of networking in robot 
coordination, and discusses applications. Section 53.4.2 
highlights a few projects focused on networked robotics 
and discusses the application potential of the field. Sec¬ 
tion 53.4.3 discusses the research challenges at the 
intersection of control, communication, and perception. 
Section 53.4.4 defines a model for the control of a net¬ 
worked system which is used in Sects. 53.4.5-53.4.8 
to examine specific research issues and opportunities 
facilitated by the interplay between communication, 
control, and perception. 

53.4.1 Overview 

The term networked robots refers to multiple robots 
operating together coordinating and cooperating by 
networked communication to accomplish a specified 
task. Communication between entities is fundamental 
to cooperation (and coordination), hence there is a cen¬ 
tral role for the communication network in networked 
robots. Networked robots may also involve coordina¬ 
tion and cooperation with stationary sensors, embedded 
computers, and human users. The central feature of net¬ 
worked robots is the ability of the system to perform 
tasks that are well beyond the abilities of a single robot 
or multiple uncoordinated robots. 

The IEEE (Institute of Electrical and Electronics 
Engineers) Technical Committee on Networked Robots 
has adopted the following definition of a networked 
robot: 

A networked robot is a robotic device connected 
to a communications network such as the Internet or 
local-area network (LAN). The network could be wired 
or wireless, and based on any of of a variety of proto¬ 
cols such as the transmission control protocol (TCP), 
the user datagram protocol (UDP), or 802.11. Many 
new applications are now being developed ranging from 
automation to exploration. There are two subclasses of 
networked robots: 

1. Teleoperated, where human supervisors send com¬ 
mands and receive feedback via the network. Such 
systems support research, education, and public 
awareness by making valuable resources accessible 
to broad audiences. 

2. Autonomous, where robots and sensors exchange 
data via the network. In such systems, the sensor 
network extends the effective sensing range of the 
robots, allowing them to communicate with each 
other over long distances to coordinate their ac¬ 
tivity. Sensing, actuation, and computation need 
no longer be collocated. A broad challenge is to 
develop a science base that couples communica¬ 


tion, perception, and control to enable such new 
capabilities. 

This definition of autonomous networked robots 
also includes a third class of distributed systems, mobile 
sensor networks, which is a natural evolution of sen¬ 
sor networks. Robot networks allow robots to measure 
spatially and temporally distributed phenomena more 
efficiently. The robots in turn can deploy, repair, and 
maintain the sensor network to increase its longevity, 
and utility. The focus of this chapter is autonomous net¬ 
worked robots. 

Embedded computers and sensors are becoming 
ubiquitous in homes and factories, and increasingly 
wireless ad hoc networks or plug-and-play wired net¬ 
works are becoming commonplace. Human users inter¬ 
act with embedded computers and sensors to perform 
tasks ranging from monitoring (e.g., supervising the op¬ 
eration of a factor and surveillance in a building) to 
control (e.g., running an assembly line consisting of 
sensors, actuators, and material-handling equipment). 
In most of these cases, the human users, embedded 
computers, and sensors are not collocated and the co¬ 
ordination and communication happens through a net¬ 
work. Networked robots extends this vision to multiple 
robots functioning in a wide range of environments per¬ 
forming tasks that require them to coordinate with other 
robots, cooperate with humans, and act on information 
derived from multiple sensors. 

Figure 53.6 shows prototype concepts derived from 
academic laboratories and industry. In all these exam- 



Fig. 53 . 6 a-d Small modules (after [53.27]) can automat¬ 
ically connect and communicate infonnation to perform 
locomotion tasks (a); robot arms (after [53.28]) on mo¬ 
bile bases can cooperate to perform household chores (b); 
swarms of robots (after [53.29]) can be used to explore an 
unknown environment (c); and industrial robots can coop¬ 
erate in welding operations (d) 
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Fig. 53 . 7 a-c Robotic modules (after [53.30]) can be re¬ 
configured to morph into different locomotion systems 
including (a) a wheel-like rolling system, (b) a snake-like 
undulatory locomotion system, and (c) a four-legged walk¬ 
ing system 

pies, independent robot or robotic modules can coop¬ 
erate to perform tasks that a single robot (or module) 
cannot perform. Robots can automatically couple to 
perform locomotion tasks (also Fig. 53.7) and manip¬ 
ulation tasks that either a single robot cannot perform, 
or that would require a special-purpose larger robot to 
perform. They can also coordinate to perform search 
and reconnaissance tasks exploiting the efficiency that 
is inherent in parallelism. They can also perform inde¬ 
pendent tasks that need to be coordinated. Examples in 
the manufacturing industry include, for example, fixtur- 
ing and welding. 

Besides being able to perform tasks that individual 
robots cannot perform, networked robots also result in 
improved efficiency. Networking gives each robot ac¬ 
cess to information outside its perception range. Tasks 
such as searching or mapping can, in principle, be 
performed faster with an increase in the number of 
robots. A speed up in manufacturing operations can be 
achieved by deploying multiple robots performing op¬ 
erations in parallel but in a coordinated fashion. 

Another advantage of using the network to connect 
robots is the ability to connect and harness physically 
removed assets. Mobile robots can react to informa¬ 
tion sensed by other mobile robots at a remote location. 
Industrial robots can adapt their end-effectors to new 
parts being manufactured upstream in the assembly 
line. Human users can use machines that are remotely 
located via the network. 

The ability to network robots also enables fault tol¬ 
erance in design. If robots can dynamically reconfigure 
themselves using the network, they are more tolerant 


to robot failures. This is seen in the Internet where 
multiple gateways, routers, and computers provide for 
a fault-tolerant system (although the Internet is not ro¬ 
bust in other ways). Similarly, robots that can plug and 
play can be swapped in and out automatically to provide 
for a robust operating environment. 

Finally, networked robots have the potential to pro¬ 
vide great synergy by bringing together components 
with complementary benefits and making the whole 
greater than the sum of the parts. 

Applications for networked robots abound. The US 
military routinely deploys unmanned vehicles that are 
reprogrammed remotely based on intelligence gathered 
by other unmanned vehicles, sometimes automatically. 
The deployment of satellites in space, often by astro¬ 
nauts in a shuttle with the shuttle robot arm, requires 
the coordination of complex instrumentation onboard 
the space shuttle, human operators on a ground station, 
the shuttle arm, and a human user on the shuttle. Home 
appliances now contain sensors and are becoming net¬ 
worked. As domestic and personal robots become more 
commonplace, it is natural to see these robots working 
with sensors and appliances in the house while co¬ 
operating with one or more human users. Networked 
robots will likely be used as critical ingredients in the 
environmental observatories of the future. Large-scale 
ecological monitoring precludes the use of monolithic 
infrastructure, and is envisioned to be built as a dis¬ 
tributed, networked robotic system. 

53.4.2 State of the Art and Potential 

The growth in networked robot systems is broad-based, 
across many industries. There is a strong connection 
between this industry and the industry connected to sen¬ 
sor networks. Sensor networks have been projected to 
grow dramatically in terms of commercialization and 
market value [53.31]. Robot networks are analogous 
to sensor networks except that they allow sensors to 
have mobility and allow the geographical distribution 
of the sensors to be adapted based on the information 
acquired. 

A system of robots, embedded computers, actua¬ 
tors, and sensors has tremendous potential in civilian, 
defense, and manufacturing applications. Nature pro¬ 
vides the proof of concept of what is possible [53.32], 
Group behaviors in nature can be found in organisms 
that are only microns to those that are several meters 
in length. There are numerous examples of simple ani¬ 
mals that execute simple behaviors with modest sensors 
and actuators but communicate with and sense nearest 
neighbors to enable complex emergent behaviors that 
are fundamental to navigation, foraging, hunting, con¬ 
structing nests, survival, and eventually growth. As seen 
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Fig. 53.8 Ants are able to cooperatively manipulate and 
transport objects often in large groups, without identified 
or labeled neighbors, and without centralized coordination 


in Fig. 53.8, relatively small agents are able to manip¬ 
ulate objects that are significantly larger in terms of 
size and payload by cooperating with fairly simple in¬ 
dividual behaviors. The coordination between agents is 
completely decentralized, allowing scaling up to large 
numbers of robots and large objects [53.33]. Individu¬ 
als do not recognize each other. In other words, there 
is no labeling or identification of robots. The number 
of agents in the team is not explicitly encoded. Agents 
are identical, enabling robustness to failures and modu¬ 
larity. There is minimal communication, and even that 
which is present is only between neighbors. Further¬ 
more, the optimal mode of group coordination may be 
scale dependent. Studies of wasps show strong evidence 
of centralized coordination among species with small 
colony sizes, but a distributed, decentralized coordina¬ 
tion in larger colonies [53.34]. All these attributes are 
relevant to networked robots. 

Biology has shown how simple decentralized be¬ 
haviors in unidentified individuals (e.g., insects and 
birds exhibiting swarming behaviors) can exhibit a wide 
array of seemingly intelligent group behaviors. Sim¬ 
ilarly networked robots can potentially communicate 
and cooperate with each other, and even though indi¬ 
vidual robots may not be sophisticated, it is possible for 
networked robots to provide a range of intelligent be¬ 
haviors that are beyond the scope of intelligent robots. 

The significance and potential impact of networked 
robots is apparent from the following examples. 

The manufacturing industry has always relied on in¬ 
tegration between sensors, actuators, material-handling 
equipment, and robots. Today companies are finding 
it easier to reconfigure existing infrastructure by net¬ 
working new robots and sensors with existing robots 
via wireless networks. There is also an increasing trend 
toward robots interacting with each other in operations 
like welding and machining, and robots cooperating 


with humans in assembly and material-handling tasks. 
Workcells consist of multiple robots, numerous sensors 
and controllers, automated guided vehicles, and one or 
two human operators working in a supervisory role. 
However, in most of these cells, the networked robots 
operate in a structured environment with very little vari¬ 
ation in configuration and/or operating conditions. 

There is a growing emphasis on networking robots 
in applications of field robotics, for example, in the 
mining industry. Like the manufacturing industry, oper¬ 
ating conditions are often unpleasant and the tasks are 
repetitive. However, these applications are less struc¬ 
tured and human operators play a more important role. 

In the health care industry, networks allow health 
care professionals to interact with their patients, other 
professionals, expensive diagnostic instruments, and 
surgical robots. Telemedicine is expected to provide 
a major growth impetus for remote networked robotic 
devices that will take the place of today’s stand-alone 
medical devices. 

There are already many commercial products, no¬ 
tably in lapan, where robots can be programmed via 
and communicate with cellular phones. For example, 
the MARON robot developed by Fujitsu lets a human 
user dial up their robot and instruct it to conduct sim¬ 
ple tasks including sending pictures back to the user 
via a cellular phone. Indeed these robots will inter¬ 
act with other sensors and actuators in the home - 
door openers equipped with Bluetooth cards and actua¬ 
tors and computer-controlled lighting, microwaves, and 
dishwashers. Indeed the Network Robot Forum [53.35] 
is already setting standards for how stationary sensors 
and actuators can interact with other robots in domestic 
and commercial settings. 

Environmental monitoring is a key application for 
networked robots. By exploiting mobility and commu¬ 
nication, robotic infrastructure enables observation and 
data collection at unprecedented scales in various as¬ 
pects of ecological monitoring. This is significant for 
environmental regulatory policies (e.g., clean air and 
water legislation), as well as an enabler of new scientific 
discovery. For example, it is possible to obtain maps 
of salinity gradients in oceans, temperature and humid¬ 
ity variations in forests, and chemical composition of 
air and water in different ecological systems [53.36]. 
In addition to mobile sensor networks, it is also pos¬ 
sible to use robots to deploy sensors and to retrieve 
information from the sensors. Mobile platforms allow 
the same sensor to collect data from multiple locations 
while communication allows the coordinated control 
and aggregation of information. Examples include sys¬ 
tems built for aquatic [53.37], terrestrial [53.38], and 
subsoil monitoring [53.39]. There are many efforts to 
developed networked underwater platforms [53.40-42]. 
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Networks of static and robotic devices have been devel¬ 
oped for aquatic monitoring [53.37] and to obtain high- 
resolution information on the spatial and temporal dis¬ 
tributions of plankton assemblages and concomitant en¬ 
vironmental parameters. The RiverNet project [53.43] 
at Rensselaer Polytechnic Institute (RPI) has focused 
on the development of robotic sensor networks for 
monitoring a river ecosystem. Recent work at Univer¬ 
sity of California, Los Angeles (UCLA), University of 
Southern California (USC), University of California, 
Riverside, and University of California, Merced on the 
networked infomechanical system project [53.38] has 
focused on the development of robotic networks for 
monitoring the forest canopy, with a view to provid¬ 
ing data for modeling canopy and undercover growth. 
Networked robotic mini-rhizotrons [53.39] are being 
deployed in the forest to monitor root growth in the 
soil. 

In the defense industry, countries like the USA have 
invested heavily in the concept of networked, geograph¬ 
ically distributed assets. Unmanned aerial vehicles like 
the Predators are operated remotely. Information from 
sensors on the Predators triggers the deployment of 
other vehicles and weapon systems at a different re¬ 
mote location and allows commanders in a third lo¬ 
cation to control and command all these assets. The 
US military engaged in the large Future Combat Sys¬ 
tems initiative to develop network-centric approaches 
to deploying autonomous vehicles. The network-centric 
tactical paradigms for modern warfare have created 
networked robots for defense and homeland security. 
While networked robots are already in operation, cur¬ 
rent approaches are limited to human users command¬ 
ing a single vehicle or sensor system. However, it 
takes many human operators (between 2—10 depend¬ 
ing on the complexity of the system) to deploy complex 
systems like unmanned aerial vehicles. A Predator un¬ 
manned aerial vehicle (UAV) is operated from a tactical 
control station, which may be on an aircraft carrier, with 
a basic crew of 3—10 operators. 

The eventual goal, however, is to enable a single 
human user to deploy networks of unmanned aerial, 
ground, surface, and underwater vehicles. There have 
been several recent demonstrations of multirobot sys¬ 
tems exploring urban environments [53.44,45] and the 
interiors of buildings [53.46, 47] to detect and track in¬ 
truders, and transmit all of the above information to 
a remote operator. These examples show that it is possi¬ 
ble to deploy networked robots using an off-the-shelf 
802.11b wireless network and have the team be re¬ 
motely tasked and monitored by a single operator. An 
example of a project with heterogeneous vehicles in 
an urban setting is shown in Fig. 53.9. An example 
of a project with heterogeneous vehicles in an indoor 



Fig. 53.9 A single operator commanding a network of 
aerial and ground vehicles from a command and control 
vehicle in an urban environment for scouting and recon¬ 
naissance in a recent demonstration by the University of 
Pennsylvania, Georgia Tech, and University of Southern 
California (after [53.48]) 

setting is shown in Fig. 53.10 wherein robots map an 
environment and deploy themselves to form a sensor 
network to detect intruders. 

Many research projects are addressing group be¬ 
haviors or collective intelligence by realizing swarming 
behaviors observed in nature. For example, the Euro¬ 
pean Union (EU) funded several EU-wide coordinated 
projects on collective intelligence or swarm intelli¬ 
gence. The I-Swarm project in Karlsruhe [53.49] and 
the Swarm-Bot project at Ecole Polytechnique Federale 
de Lausanne (EPFL) [53.50] are examples of swarm 
intelligence. The Laboratory for Analysis and Archi¬ 
tecture of Systems (LAAS) has a strong group in 
robotics and artificial intelligence. This group has had 
a long history of basic and applied research in multi¬ 
robot systems. The integration of multiple unmanned 
vehicles for applications such as terrain mapping and 
fire-fighting is addressed in [53.51]. A multi-university 
US project addressed the development of networked ve¬ 
hicles for swarming behaviors [53.52]. Projects such as 
these are exploring the scalability of the basic concepts 
to large numbers of robots, sensors, and actuators. 

53.4.3 Research Challenges 

While there are many successful embodiments of net¬ 
worked robots with applications to manufacturing in¬ 
dustry, the defense industry, space exploration, do¬ 
mestic assistance, and civilian infrastructure, there are 
significant challenges that have to be overcome. 

The problem of coordinating multiple autonomous 
units and making them cooperate creates problems at 
the intersection of communication, control, and percep¬ 
tion. Who should talk to whom and what information 
should be conveyed, and how? How does each unit 
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Fig. 53.10 Under the DARPA SDR program, a team from 
the University of Southern California, the University of 
Tennessee, and Science Applications and International 
Corporation (SAIC) demonstrated mapping, and intruder 
detection by a team of networked robots (after [53.46]) 

move in order to accomplish the task? How should 
the team members acquire information? How should 
the team aggregate information? These are all basic 
questions that need basic advances in control theory, 
perception, and networking. In addition, because hu¬ 
mans are part of the network (as in the case of the 
Internet), we have to devise an effective way for mul¬ 
tiple humans to be embedded in the network and 
command/control/monitor the network without worry¬ 
ing about the specificity of individual robots in the 
network. Thus the underlying research challenges he at 
the intersection of control theory, perception, and com¬ 
munication/networks, as shown in Fig. 53.11. 

It is also worth noting that robot networks are dy¬ 
namic, unlike networks of sensors, computers or ma¬ 
chines which might be networked together in a fixed 
topology. When a robot moves, its neighbors change and 
its relationship to the environment changes. As a con¬ 
sequence, the information it acquires and the actions it 
executes must change. Not only is the network topology 
dynamic, but the robot’s behavior also changes as the 
topology changes. It is very difficult to predict the per¬ 
formance of such dynamic robot networks, yet it is this 
analysis problem that designers of robot networks must 
solve before deploying the network. 

This notion of a changing topology inevitably leads 
us to complicated mathematical models. Traditionally, 
models of group behavior have been built on continu¬ 
ous models of dynamics of individuals, including local 
interactions with neighbors, and models of control and 
sensing with a fixed set of neighbors. While dynamics 
at the level of individual units may be adequately de¬ 
scribed by differential equations, the interactions with 
neighbors are best described by edges on a graph. Mod¬ 
eling, analysis, and control of such systems will require 
a comprehensive theoretical framework and new rep¬ 
resentational tools. New mathematical tools that marry 
dynamical system theory, switched systems, discrete 
mathematics, graph theory, and computational geom¬ 
etry are needed to solve the underlying problems. We 
need a design methodology for solving the inverse 



Fig. 53.11 The paradigm of networked robots introduces 
fundamental challenges at the intersection of control, per¬ 
ception, and communication that is of interest to the 
robotics, sensor networks, and artificial intelligence com¬ 
munities 

problem in navigation - behaviors for controlling in¬ 
dividuals to achieve a specified aggregate motion and 
shape of the group, and the application to active per¬ 
ception and coverage. An overview of some of these 
methods is provided in Sect. 53.4.4. 

Problems of perception have been studied exten¬ 
sively in the robotics community. However, the percep¬ 
tion problems in a system of networked, mobile sensor 
platforms bring a new set of challenges; for example, 
consider the problem of estimating the state of the net¬ 
work. State estimation requires the estimation of the 
state of robots and the environment based on local, 
limited-range sensory information. Localization of n 
vehicles in an m-dimensional configuration space re¬ 
quires 0({nm) k ) computations, where k is somewhere 
between 3 and 6, depending on the algorithm and 
domain-specific assumptions. The estimation problem 
is further exacerbated by the fact that not all robots in 
the network may be able to get the necessary informa¬ 
tion in a time-critical fashion. There are deep issues of 
representation and algorithmic development, which are 
discussed in Sect. 53.4.6. 

The paradigm of active perception [53.53] links 
the control of sensor platforms to perception, bring¬ 
ing control theory and perception together in a com¬ 
mon framework. Extending this paradigm to networked 
robots requires approaches of distributed control to be 
merged with decentralized estimation. Robots can move 
in order to localize themselves with respect to their 
neighbors, to localize their neighbors, and also to iden¬ 
tify, localize, and track features in the environment. 
These problems are discussed in Sect. 53.4.7. 

As discussed earlier, the communication network 
is central to the functioning of a network of robots. 
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However, if the network consists of mobile agents with 
transmitters and receivers with finite power, there is 
no guarantee that all agents can talk to teach other. 
Unlike a static sensor network, robots in a network 
can move toward each other to facilitate communica¬ 
tion and adaptively maintain a communication network. 
Some basic algorithmic problems and several pertinent 
results are provided in Sect. 53.4.8. 

53.4.4 Control 

The control of individual robots is critical to the perfor¬ 
mance and scope of robot networks. Indeed motion co¬ 
ordination algorithms have been proposed for the pur¬ 
pose of improving communication performance [53.54, 
55], localization [53.56,57], information integration, 
deployment [53.58], and coverage [53.59-61], among 
other tasks. Mobility allows the group of robots to self¬ 
deploy, and self-organize by relocating themselves in 
support of communication, sensing, or task needs; for 
example, they can reconfigure to guarantee a desired 
communication bandwidth, k-hop connectivity, or alge¬ 
braic connectivity, enabling message delivery from one 
robot to another. The group can also self-organize to 
position sensors so as to cover a desired area and adapt 
to shifts in the focus of monitoring activities. Control¬ 
ling sensor position also supports map making, tracking 
of objects and events, and goal-directed navigation for 
users of the network. Finally, mobility allows robots to 
accomplish tasks such as navigation, reconnaissance, 
transportation, and search and rescue. 

Given a group of mobile sensors, we would like 
to have distributed control capabilities that realize de¬ 
sirable global specifications. Thus, it is necessary to 
be able to automatically determine the necessary po¬ 
sition and orientation of the group members and/or 
the distribution of group members, and their motion to 
achieve the desired task. At a lower level, the robots 
must be able to use information from the communi¬ 
cation network and from their own sensors to derive 
local estimates, reason about the spatial network (their 
neighbors and their relationship to the environment), 
and then use the appropriate control policies to achieve 
the desired group specifications. We briefly outline the 
simplest mathematical model that is necessary to for¬ 
mulate such problems in order to provide a better sense 
of the underlying challenges. 

In a robot network, we have multiple agents or 
nodes in which each agent is a physical entity that 
can be a robot, a vehicle with actuators and sensors, 
a sensor platform (possibly static) or even a communi¬ 
cation relay node. Each agent A, is characterized by an 
identifier, i G I C Z, a state Xj G X , C R", and control in¬ 
puts M; G Uj C R, with/' : Xj x Uj — >■ TXj specifying the 


dynamics 

Xi =fi(Xi , Uj) . 


(53.1) 


The state v, will consist of the position (and orienta¬ 
tion), r, in some d-dimensional space, and its velocity, 
r ; : Xi = ( rj , rJ) T , with n = 2d. dV c (r,) and W s (r;) are 
neighborhoods of r that define the range and field 
of view of the communication hardware and sensors, 
respectively. 

A network of robots S consists of N agents with 
a sensing graph and a communications graph that 
is defined by the physical distribution of the agents. 
The sensing graph (and similarly the communications 
graph) is defined by a map E s : X 1 x X 2 ■ ■ ■ X N —> I x /, 
where the edges of the graph are formed dynamically 
depending on the physical proximity of pairs of agents. 
Specifically, the N xN adjacency matrix, 34 s (and sim¬ 
ilarly J4 C ) has entries 


1, if rj G ZAf s (r;) , 
0, otherwise. 


(53.2) 


Agent Aj has estimates of its own state and the states 
of neighbors (e.g., Aj), and these estimates are derived 
from information associated with edges in the sensing 
and communication graph 


Xj ,) = h(Xj,Zjj) , (53.3) 

where ztj represents measurements of the state of agent 
Aj available to A,- by sensing or communication chan¬ 
nels and h is the estimator used by A, . Note that zy may 
have dimension less than n and may therefore not con¬ 
tain complete information about xy = x, —Xj. Clearly the 
relative position vector denoted by ry = r,- — ^ and its 
magnitude are important quantities that may need to be 
estimated for biological and artificial agents. 

Finally, A, can encode n^ behaviors, which we 

will denote by = B\. Bi . B„ h . Each behavior Bj 

is a controller, a function kj :ExI ; -> Uj. All agents 
can be assigned identical or different behaviors. Each 
behavior represents a set of unsynchronized, locally 
executed computations (for control or estimation) be¬ 
ing carried out for some collective purpose, with each 
processor using in its computations only data from its 
neighboring processors. Furthermore, even for a fixed 
assignment of behaviors, each processor’s neighbors 
typically change with time because the processors are 
moving in and out of the sets ZA/” C and /Af s . Thus the 
methodology for modeling and analyzing such systems 
will require the merging of graph theory and dynamical 
system theory at a fundamental level. 
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The reader is directed to the many survey articles 
on this subject for further information. An overview 
of challenges for the controls community is presented 
in [53.62]. The underlying theory for networked mo¬ 
bile systems has been explored in the context of au¬ 
tomated highway systems [53.63], cooperative robot 
reconnaissance [53.46] and manipulation [53.64], for¬ 
mation flight control [53.65], and the control of groups 
of unmanned vehicles [53.45], Our goal in the fol¬ 
lowing sections is to explore the connections between 
communication, perception, and control. 

53.4.5 Communication for Control 

Communication networks allow physically discon¬ 
nected entities to exchange information. At the lowest 
level, when groups of vehicles coordinate their actions, 
communication allows vehicles to exchange state infor¬ 
mation [53.66-68]. At a higher level, robots can plan 
navigation and exploration tasks based on an integrated 
map of the world derived from information acquired 
from different robots [53.52]. 

The use of communication for control in the mul¬ 
tivehicle context has been addressed in the PATH 
project where formations of inline vehicles were stud¬ 
ied [53.63]. Problems of the stability of the forma¬ 
tion [53.69], the convergence of the formation to 
shapes [53.70], and the overall performance of the sys¬ 
tem [53.71] are of great interest. The performance of 
the system is directly influenced by the interconnections 
between agents. In addition to impacting on stabil¬ 
ity [53.63], feedback of states from different agents 
and feedforward information from the plans of differ¬ 
ent agents affects the rates at which the system of agents 
can respond to external stimuli [53.71] or to commands 
from human operators [53.72]. 

In addition, communication can be used for high- 
level control and planning of robots. There is great 
interest in using static sensor nodes as beacons to guide 
robot navigation. In [53.73], the problem of coverage 
and exploration of an unknown dynamic environment 
using a mobile robot is considered. An algorithm is 
presented which assumes that global information is not 
available (neither a map, nor global positioning system 
(GPS) information). The algorithm deploys a network 
of radio beacons that assists the robot in coverage. The 
network is also used by the robot for navigation. The 
deployed network can also be used for applications 
other than coverage (such as multirobot task allocation). 
A similar idea was presented using potential-field-based 
navigation in [53.52]. In this work the notion of no- 
go or danger areas was incorporated into the navigation 
cost function. Recent work along these lines with exper¬ 
imental data from sensor nodes is reported in [53.74], 


In such communication-enabled cooperative con¬ 
trol and planning (see also [53.75]), the communication 
network plays an important role in the creation of 
a shared representation of information. This notion of 
a shared representation is important to the scaling of 
coordinated control algorithms to large numbers of de¬ 
vices. For example, in [53.67], the information form 
of the Kalman filter is used to derive a framework 
for decentralizing estimation and fusion algorithms. 
This approach was shown to be applicable to multi¬ 
ple heterogenous ground and aerial platforms [53.56]. 
Such methodologies are transparent to the specificity 
and identity of the cooperating vehicles. This is be¬ 
cause vehicles share a common representation, which 
consists of a certainty grid that contains information 
about the probability of detection of targets, and an 
information vector-matrix pair that is used in the infor¬ 
mation form of the Kalman filter [53.45]. Observations 
are propagated through the network by changing both 
the certainty grid and the information vector/matrix. 
This allows each vehicle to choose the action that 
maximizes a utility function, which is the combined 
mutual information gain from onboard sensors to¬ 
wards the detection and localization of features in the 
environment. 

Thus, in summary, at the lowest level, communica¬ 
tion enables either partial or complete state feedback of 
the network and allows agents to exchange information 
for feedforward control. At the higher levels, agents can 
share information for planning and for control. This is 
also discussed in Sect. 53.4.6 where the communication 
network is shown to enable a network-centric approach 
to perception. 

53.4.6 Communication for Perception 

While individual robots have sensors and the ability to 
build maps and models by integrating sensory informa¬ 
tion, networked robots can exchange information and 
leverage sensory data, maps, and models from other 
robots. The challenge is to exploit communication for 
perception in tasks such as distributed mapping in the 
presence of the delays, limited bandwidth, and disrup¬ 
tion that are typical of communication networks. 

Distributed localization is the term used to de¬ 
scribe the merging of communication and perception 
for state estimation. Localization is an essential tool 
for the development of low-cost robot networks for use 
in location-aware applications and ubiquitous network¬ 
ing [53.76]. Location information is needed to track 
the placement of the nodes and to correlate the values 
measured by the node with their physical location. Dis¬ 
tributed computation and robustness in the presence of 
measurement noise are key ingredients for a practical 
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localization algorithm that will give reliable results over 
a large-scale network. 

The methods for distributed localization can be clas¬ 
sified into two broad classes: algorithms that rely on 
anchor nodes for localization and algorithms that use no 
beacons. Localization may be computed using range in¬ 
formation between nodes, bearing information, or both. 

In [53.54] a theoretical foundation for network lo¬ 
calization in terms of graph rigidity theory is provided. 
The problem is solved when nodes have perfect range 
information and it is show that a network has a unique 
localization if and only if its underlying graph is gener- 
ically globally rigid. In [53.77] the Cramer-Rao lower 
bound (CRLB) for network localization is derived. This 
work computes the expected error characteristics for 
an ideal algorithm, and compares this to the actual er¬ 
ror in an algorithm based on multilateration, drawing 
the important conclusion that the error introduced by 
the algorithm is just as important as the measurement 
error in assessing end-to-end localization accuracy. 
In [53.78] a distributed algorithm that uses no beacons 
and is guaranteed to compute correct location infor¬ 
mation under measurement noise for nodes that can 
range to neighbors is presented. This algorithm relies 
on the notion of robust quadrilaterals to compute ro¬ 
bustly a global system of coordinates among the nodes. 
The computation supports moving nodes. Extensions 
of this work to passive tracking have been discussed 
in [53.79]. Localization based on the propagation of lo¬ 
cation information from known reference nodes based 
on connectivity includes [53.80,81]. Mobility-assisted 
localization is introduced in [53.82]. Other techniques 
use distributed propagation of location information us¬ 
ing multilateration [53.77, 83]. In a recent paper [53.84] 
the problem of evaluating the rigidity of a planar net¬ 
work is treated while satisfying common objectives 
of real-world systems: decentralization, asynchronicity, 
and parallelization. 

Two approaches for cooperative relative localiza¬ 
tion of mobile robot teams are given in [53.85,86], 
Neither method uses GPS, landmarks, or maps of any 
kind; instead, robots make direct measurements of the 
relative pose of nearby robots and broadcast this in¬ 
formation to the team as a whole. In [53.85], each 
robot processes this information independently to gen¬ 
erate an egocentric estimate for the pose of other robots 
using a Bayesian formalism with a particle filter imple¬ 
mentation. In [53.86], maximum-likelihood estimation 
(MLE) and numerical optimization is used to achieve 
a similar result. 

A key issue is to be able to scale these computa¬ 
tions for building a shared representation to large num¬ 
bers of robots and sensors. This problem was studied 
in experiments under the US Defense Advanced Re¬ 


search Projects Agency (DARPA-funded software for 
distributed robotics (SDR) program. The goal of these 
experiments was to develop and demonstrate a multi¬ 
robot system capable of carrying out a specific mission. 
This required the ability to deploy a large number of 
robots into an unexplored building, map the building 
interior, detect and track intruders, and transmit all of 
the above information to a remote operator. A report on 
one set of experiments is presented in [53.46]. A tiered 
strategy for deploying the robots is described, where 
highly capable robots formed the first wave to enter 
and map a building, followed by a second wave which 
used the resulting map to self-deploy and monitor the 
environment for intruders. Both approaches relied ex¬ 
tensively on networking the robots using commercial 
802.11b wireless technology. This task involved both 
communication for building a shared representation as 
well control for perception. 

Another important set of problems arises when 
robot networks are used for identifying, localizing, 
and then tracking targets in a dynamic setting. An 
embedded stationary wireless sensor network is like 
a virtual sensor spread over a large geographical area. 
Such a network can provide information to mobile 
robots about remote locations. Robot networks allow 
this virtual sensor to move in response to external 
stimuli and to track moving targets. Indeed, it is pos¬ 
sible to cast this scenario as a pursuit-evasion game 
with robotic sensor networks [53.87]. For example, the 
Tenet project at USC addresses the design of network 
primitives and abstractions for tiered network archi¬ 
tectures, with robotic pursuit evasion as one of the 
target applications. Algorithms for guiding the sam¬ 
pling strategy of a robotic boat to model and locate 
phenomena of interest (e.g., hotspots) in aquatic en¬ 
vironments are discussed in [53.37]. The networked 
infomechanical systems (NIMS) project has focused 
on sensor-assisted techniques for mobile robot-based 
adaptive sampling for event response [53.88] and field 
reconstruction [53.89]. 

The information collected by the nodes in a sensor 
network can be processed at a central location or in a de¬ 
centralized fashion. Such in-network data processing 
techniques make better use of network communication 
and computation resources than centralized process¬ 
ing. This also enables the network to compute accurate 
and up-to-date global pictures of the global perception 
landscape that are available to all the robots in the sys¬ 
tem. Methods for in-network data processing with static 
nodes include artificial potential-field computation, gra¬ 
dient computations, particle filters, Bayesian inference, 
and signal processing. Algorithms have been developed 
for computing maps, paths, and predictors [53.52,73, 
90]. 
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A recent DARPA demonstration showed how com¬ 
munication networks can be used effectively in per¬ 
ception tasks involving heterogenous robots [53.44]. In 
cooperative search, identification, and localization un¬ 
manned aerial vehicles (UAVs) can be used to cover 
large areas, searching for targets. However, sensors on 
UAV s are typically limited in their accuracy of localiza¬ 
tion of targets on the ground. On the other hand, ground 
robots can be deployed to accurately locate ground tar¬ 
gets but have the disadvantage of not being able to move 
rapidly and see through obstacles such as buildings or 
fences. In [53.56], the synergy between these two de¬ 
vices is exploited by creating a seamless network of 
UAVs and unmanned ground vehicles (UGVs). As dis¬ 
cussed in Sect. 53.2, the key to such network-centric 
approaches for search and localization is a shared rep¬ 
resentation of state information, which in this case is 
easily scalable to large numbers of UAVs and UGVs 
and is transparent to the specificity of individual plat¬ 
forms. However, how to do this more generally and for 
more unstructured information remains an issue for fu¬ 
ture research. 

53.4.7 Control for Perception 

Networked mobile robots enable the exploration of 
dynamic environments and the recovery of three- 
dimensional information via distributed active percep¬ 
tion [53.53]. Since the nodes are mobile, a natural 
question is: where should the nodes be placed in or¬ 
der to ensure successful integration of information from 
multiple nodes, and to maximize the quality of the es¬ 
timates returned by the team? Since there is a cost 
associated with transmitting and processing data, it is 
important to consider which sensor readings should 
be used in the state estimation and what information 
should be communicated to the rest of the system. The 
quality of the information computed by the network de¬ 
pends on the locations of the sensor platforms both in 
an absolute and relative sense. The quality also depends 
on the noise characteristics of each sensor, and the com¬ 
munication network. 

A robot network goes well beyond a fixed sensor 
network, which can only collect data at fixed positions 
in space; for example, when an event is detected at 
a specific location it is possible to direct more sen¬ 
sors toward the location of observation of the event for 
more information (for example, higher-resolution data 
or higher sampling frequency). Reconfiguring the node 
locations for adaptive resolution sampling relies on dis¬ 
tributed control strategies. 

Various strategies have been introduced for control¬ 
ling mobile sensor network coverage. Mobile sensing 
agents are controlled using gradients of information- 


based objective functions [53.91]. Stability results are 
derived without concerns for the optimality of the 
network configuration, but local guarantees are pro¬ 
vided. Topology aware coordinated behavior is treated 
in [53.92]. A body of results reported in [53.93] 
and [53.94] describes decentralized control laws for po¬ 
sitioning mobile sensor networks optimally with respect 
to a known event distribution density function. This ap¬ 
proach is advantageous because it guarantees that the 
network (locally) minimizes a cost function relevant to 
the coverage problem. However, the control strategy 
requires that each agent have a complete knowledge 
of the event distribution density, thus it is not reac¬ 
tive to the sensed environment. The work by [53.95, 
96] generalizes these results to situations in which the 
nodes estimate rather than know ahead of time the event 
distribution density function. A local (decentralized) 
control law requires that each agent can measure the 
value and gradient of the distribution density function 
at its own position. This results in a sensor network 
that is reactive to its sensed environment while main¬ 
taining or seeking a near-optimal sensing configuration. 
In addition, the distribution density function approxi¬ 
mation yields a closed-form expression for the control 
law in terms of the vertices of an agent’s Voronoi 
region. This eliminates the need for the numerical in¬ 
tegration of a function over a polynomial domain at 
every time step, thereby providing a significant reduc¬ 
tion in computational overhead for each agent. Other 
work in event monitoring for unknown distributions 
includes [53.59]. Krause et al. [53.97] have recently 
proposed an approach for sensor placement that con¬ 
siders both the sensing quality and communication cost 
of imperfect sensing and communication components. 
They use a parametric model for link reception rate that 
assumes no acknowledgement and no temporal correla¬ 
tion of lossy links. 

Beginning with the art gallery problem, there have 
been multiple efforts to determine an optimal configu¬ 
ration of sensors to cover a given region [53.98-100]. 
A variant which allows the use of mobile sensors is 
known as the watchmen tours problem. In these ap¬ 
proaches the sensor model is abstract and not well 
suited to real environments and cameras. Distributed 
geometric optimization methods [53.94] have also been 
used for mobile sensor network reconfiguration. A re¬ 
lated class of methods is the use of estimation-theoretic 
optimization metrics and the application of informa¬ 
tion filters to coordinate network-wide motion [53.56]. 
There are other distributed optimization methods which 
use a distributed control law and show that it optimizes 
a global metric of interest, such as using a potential 
field or other linear control law based only on local 
neighbor interactions [53.101]. Research focusing on 
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the control of cameras with pan, tilt, and zoom ca¬ 
pabilities is due to [53.60,102,103]. In [53.102] an 
approach is developed to calibrate a pan-tilt-zoom 
camera automatically over its full zoom range and to 
build very high-resolution panoramas. In [53.60], the 
cameras are constantly moved to track observed targets, 
using a factor graph. A recent algorithm due to [53.104] 
significantly improves on this by positioning cameras to 
make the network better suited to detect and classify 
targets as they emerge. Pan-tilt-zoom cameras allow 
the construction of far more flexible vision systems than 
static cameras. 

53.4.8 Control for Communication 

In Sect. 53.4.5, we briefly discussed the benefits of 
using the communication network to synthesize and 
improve controller design. Conversely, the movement 
of robots affects the network and data transmission in 
the network. This gives rise to many challenges. If the 
controllers for individual robots are known, can we pro¬ 
vide guarantees about communication in the network 
and can we develop robust information routing and net¬ 
working algorithms in the presence of robot motion? 
Another challenge concerns how information propa¬ 
gates and diffuses in these networks. If the robots move 
under a given control model, how does a piece of in¬ 
formation propagate through the network and what can 
we say about when and where that piece of information 
will be heard? If we know the answers to such ques¬ 
tions, it may be possible to design controllers to realize 
desired communication network characteristics. 

One simple control strategy that can affect network 
performance is to control the robot motion to ensure 
messages are transmitted between designated nodes. 
The movement of robots in a network of robots and 
sensors may cause network partitioning when nodes go 
out of range. However, the ability of the robots to move 
in a controlled way also leads to an opportunity to ad¬ 
dress the information routing problem in disconnected 
networks by turning the robots into relay nodes. The 
key idea here is to enable the robot holding a current 
message to an unavailable destination to modify their 
trajectory in order to relay a message. This problem has 
been formulated as an optimization problem. The goal 
is to minimize the trajectory modifications necessary 
to send a message to its destination. Several solutions 
have been proposed depending on the information that 
is available to the robots. If the robots’ trajectories are 
known, path planning techniques can be used to com¬ 
pute who moves where to relay what. If the robots’ 
trajectories are not known, a distributed spanning tree 
can be created to enable robots to keep track of each 
other. Each robot is assigned a region of movement and 


a parent in the spanning tree. When the robot leaves its 
region, the parent is informed. When the robot moves 
too far away, the spanning tree is modified. 

Mobile robots can be used to create desired network 
topologies under suitable models of network commu¬ 
nication. If a robot is used to emplace nodes in an 
environment (or if sensor nodes robotically self-deploy) 
to build a network, the problem is referred to as deploy¬ 
ment. It is possible to control the motion of individual 
nodes to guarantee that a specified topology is main¬ 
tained [53.55]. It is also possible to reposition nodes 
with the explicit aim of changing the network topol¬ 
ogy - the so-called mobility-based topology control 
problem. 

A distributed algorithm for the deployment of mo¬ 
bile robot teams has been described by [53.105] using 
the concept of virtual pheromones: localized messages 
from one robot to another. These messages are used 
to generate either a gas expansion or a guided growth 
deployment model. Similar algorithms based on arti¬ 
ficial potential fields are described in [53.106,107], 
where the latter incorporates a connectivity constraint. 
An incremental deployment algorithm for mobile sen¬ 
sor networks is given in [53.58]; nodes are deployed 
one at a time into an unknown environment, with each 
node making use of information gathered by previously 
deployed nodes to determine its deployment location. 
The algorithm is designed to maximize network cover¬ 
age while ensuring that nodes retain line of sight with 
one another. 

Most work on network topology control has dealt 
with uncontrolled deployments, where there is no ex¬ 
plicit control of the positions of individual nodes. The 
primary mechanisms proposed are power control and 
sleep scheduling. These methods involve pruning an al¬ 
ready existing, well-connected communication graph in 
order to save power while ensuring that the resultant 
subgraph preserves connectivity. Given a network that 
is connected when all nodes are operating at maximum 
power, the aim of power control is to use the minimum 
power level at each node for which the network remains 
connected [53.108]. Given an overdeployed network, 
sleep scheduling seeks to activate a minimal subset 
of nodes to maintain connectivity and achieve other 
desired metrics [53.109]. In contrast, controlled de¬ 
ployments are feasible when the positions of individual 
nodes can be altered. Such deployments are interesting 
for two reasons. First, network topology with wireless 
communication relates directly to proximity relations 
and hence the position of the nodes. Second, there is in¬ 
creasing evidence that a large number of deployments 
are likely to involve careful, nonrandom placement of 
nodes. The positioning of nodes is controlled either by 
the nodes themselves or by external agents. Such net- 
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works present a different and interesting scenario for 
topology control since it is possible to exploit control of 
the motion and placement of the nodes to build efficient 
topologies. A local, completely decentralized technique 
for topology control using mobility is given in [53. 1 10]. 

An important application for networked robots is 
in monitoring and surveillance, where it is impor¬ 
tant that the robots cover the space while remaining 
within communication range [53.1 1 1—113], In a recent 
development [53.114] the problem of how to design 
communication models and scheduling protocols for 
choosing the appropriate path planning algorithms for 
robotic data collection is discussed. Probing environ¬ 
ment and adaptive sleeping protocol (PEAS) was one of 
the first attempts to address communication connectiv¬ 
ity and sensing coverage simultaneously using heuris¬ 
tic algorithms [53.115]. Wang et al. [53.116] proposed 
a new coverage configuration protocol (CCP) to pro¬ 
duce an approach that simultaneously optimizes cov¬ 
erage and connectivity while maximizing the number 
of nodes that are placed into sleep mode. Furthermore, 
they also identified three different classes of coverage- 
connectivity problems with respect to the ratio of radio 
and sensing ranges and recognized the critical ratio 

53.5 Swarm Robots 

Historically, some of the earliest work in multirobot 
systems [53.12, 13,118-125] dealt with large numbers 
of homogeneous robots, called swarms', swarm robotics 
continues to be a very active area of research. Most 
swarm approaches obtain inspiration from biological 
societies - particularly ants, bees, and birds - to develop 
similar behaviors in multirobot teams. Because biolog¬ 
ical societies are able to accomplish impressive group 
capabilities, such as the ability of termites to build large 
complex mounds, or the ability of ants to collectively 
carry large prey, robotics researchers aim to reproduce 
these capabilities in robot societies. 

Swarm robotics systems are often called collective 
robotics, indicating that individual robots are often un¬ 
aware of the actions of other robots in the system, other 
than information on proximity. These approaches aim 
to achieve a desired team-level global behavior from 
the interaction dynamics of individual robots follow¬ 
ing relatively simple local control laws. Swarm robotic 
systems typically involve very little explicit communi¬ 
cation between robots, and instead rely on stigmergy 
(i. e., communication through the world) to achieve 
emergent cooperation. Individual robots are assumed 
to have minimal capabilities, with little ability to solve 
meaningful tasks on their own. However, when grouped 
with other similar robots, they are collectively able to 


where the former range is twice as long as the latter. 
Zhang and Hou proved that, if the communication range 
is at least twice the sensing range, complete coverage 
of a convex area guarantees network communication 
connectivity, and then used this theorem as a basis for 
a localized density control algorithm [53. 109]. This was 
subsequently generalized to show that the condition that 
the communication range is twice the sensing range 
is sufficient and is the tight lower bound to guarantee 
that complete coverage preservation implies communi¬ 
cation connectivity among nodes if the original network 
topology is connected [53.117]. 

In summary, if the state of the communication 
network and the desired state of the communication 
network is known to each agent, it should be possible 
to synthesize distributed controllers to move agents to 
attain desired network characteristics. However, the as¬ 
sumptions on the global state are clearly not justified. 
Also, the desired motion to optimize network charac¬ 
teristics will conflict with the motion that is required to 
perform the desired task. However, as the brief discus¬ 
sion above illustrates, there are many interesting studies 
that point to promising directions for future work in this 
very fertile research field. 


achieve team-level tasks. Ideally, the entire team should 
be able to achieve much more than individual robots 
working alone (i. e., it is superadditive, meaning that 
the whole is bigger than the sum of the parts). These 
systems assume very large numbers of robots (at least 
dozens, and often hundreds or thousands) and explicitly 
address issues of scalability. Swarm robotic approaches 
achieve high levels of redundancy because robots are 
assumed to be identical, and thus interchangeable with 
each other. 

Many types of swarm behaviors have been studied, 
such as foraging, flocking, chaining, search, herding, 
aggregation, and containment. The majority of these 
swarm behaviors deal with spatially distributed multi¬ 
robot motions, requiring robots to coordinate motions 
either: 

1. Relative to other robots 

2. Relative to the environment 

3. Relative to external agents 

4. Relative to robots and the environment 

5. Relative to all (i. e., other robots, external agents, 
and the environment). 

Table 53.1 categorizes swarm robot behaviors ac¬ 
cording to these groupings, citing representative exam¬ 
ples of relevant research. 
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Table 53.1 Categories of swarm behaviors 


Relative motion requirements 

Swarm behaviors 

Relative to other robots 

Formations [53.126, 127], flocking [53.121], natural herding (as in herds of cattle), schooling, 
sorting [53.14], clumping [53.14], condensation, aggregation [53.128], dispersion [53.129] 

Relative to the environment 

Search [53.130], foraging [53.131], grazing, harvesting, deployment [53.58], coverage [53.132], 
localization [53.133], mapping [53.134], exploration [53.135] 

Relative to external agents 

Pursuit [53.136], predator-prey [53.137], target tracking [53.138], forced herding/shepherding 
(as in shepherding sheep) 

Relative to other robots and the 

environment 

Containment, orbiting, surrounding, perimeter search [53.139] 

Relative to other robots, 
external agents, and the 
environment 

Evasion, tactical overwatch, soccer [53.140] 


Much of the current research in swarm robotics 
is aimed at developing specific solutions to one or 
more of the swarm behaviors listed in Table 53.1 
Some of these swarm behaviors have 
received particular attention, notably formations, flock¬ 
ing, search, coverage, and foraging. Section 53.10 dis¬ 
cusses these behaviors in more detail. In general, most 
current work in the development of swarm behaviors 
is aimed not just at demonstrating group motions that 
are similar to biological systems, but also at under¬ 
standing the formal control theoretic principles that can 
predictably converge to the desired group behaviors, 
and remain in stable states. 

Demonstration of physical robot swarms is both 
a hardware and a software challenge. As discussed 
in Sect. 53.2, the first demonstrations were by 
Mataric [53.6], involving about 20 physical robots per¬ 
forming aggregation, dispersion, and flocking. This 
work defined composable basis behaviors as primitives 
for structuring more complex systems. In later research, 
McLurkin [53.141] developed an exten¬ 

sive catalog of swarm behavior software, and demon¬ 
strated these behaviors on about 100 physical robots 
(called the SwarmBot robots), developed by iRobot, as 
shown in Fig. 53.12. He created several group behav¬ 
iors, such as avoidManyRobots, disperseFromSource, 
disperseFromLeaves, disperseUniformly, computeAv- 
erageBearing, avoidManyRobots, followTheLeader, or- 
bitGroup, navigateGradient, clusterOnSource, and clus- 
terlntoGroups. A swarm of 108 robots used the devel¬ 
oped dispersion algorithms in an empty schoolhouse of 
area of about 300 m 2 , and were able to locate an object 
of interest and lead a human to its location [53.129]. 

The European Union has sponsored many swarm 
robot projects, leading toward decreasingly smaller- 
sized individual robots, and increasingly larger-sized 
robot teams. The I-SWARM project, for instance, aimed 
at developing millimeter-sized robots with full on¬ 
board sensing, computation, and power for perform¬ 
ing biologically inspired swarming behaviors, as well 



Fig. 53.12 The SwarmBot robots 


as collective perception tasks. This project was both 
a hardware and a software challenge, in that develop¬ 
ing microscale robots that are fully autonomous and can 
perform meaningful cooperative behaviors will require 
significant advances in the current state of the art. 

The EU-based SWARM-BOTS project studied new 
concepts in the design and implementation of self¬ 
organizing and self-assembling robots 
In this work [53.142], s-bot robots are developed that 
have grippers enabling the robots to create physical 
links with other s-bots or objects, thus creating as¬ 
semblies of robots. These assemblies can then work 
together for navigation across rough terrain, or to col¬ 
lectively transport objects. The s-bots are cylindrical, 
with a flexible arm and toothed gripper that can con¬ 
nect one s-bot to another. An interesting application 
of object transport using SWARM-BOTs is shown in 
Figure 53.13, which shows the robots self-assembling 
into four chains in order to pull a child across the floor 
(|<s>3EM!EDa). 

Another notable effort in swarm robotics research 
is the US multi-university SWARMS initiative led by 
the University of Pennsylvania. Research in this project 
aimed at developing a new system-theoretic frame- 
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Fig. 53.13 SWARM-BOTs self-assembling to move 
a child across the floor 


work for swarming, developing models of swarms and 
swarming behavior, analyzing swarm formation, stabil¬ 
ity, and robustness, synthesizing emergent behaviors for 
active perception and coverage, and developing algo¬ 
rithms for distributed localization. 

Besides the hardware challenges of dealing with 
large numbers of small robots, there are many import¬ 
ant software challenges that remain to be solved. From 
a practical perspective, a common approach to creat¬ 
ing homogeneous multirobot swarms is to hypothesize 
a possible local control law (or laws), and then study 
the resulting group behavior, iterating until the desired 
global behavior is obtained. However, the longer-term 
objective is to be able to both predict group perfor¬ 
mance based on known local control laws, and to 
generate local control laws based upon a desired global 
group behavior. 

More recent research has focused on the devel¬ 
opment of analytical techniques that can synthesize 
distributed controllers that achieve the desired macro¬ 
level system behaviors. Mather and Hsieh [53.143] 
address this challenge by proposing a technique that 
first identifies robot-robot interactions at the macro¬ 
scopic level; they then use this analysis to improve local 
robot control policies by filtering out spurious robot- 
robot interactions. Another top-down design approach 
is presented by Chen et al. [53.144] who show how 
to automatically synthesize control and communication 
strategies for a robot team based on global specifica¬ 
tions of the desired system-level behavior, stated using 
regular expressions. The resulting control strategies are 
formally proven to correctly achieve the desired global 
behavior. The work of Melo and Veloso [53.145] takes 


a different approach to the synthesis of local decision 
policies by making use of the decentralized sparse- 
interaction Markov decision process. This technique 
allows agents to recognize states when interactions with 
other robots might occur, thus enabling them to choose 
better motions based on possible future inter-robot ac¬ 
tions. Tsiotras and Castro [53.146] synthesize local 
controllers by generalizing the standard consensus al¬ 
gorithm, applied to geometric pattern formation. 

A common theme in most of the works cited above 
is that they first make use of formal methods to de¬ 
scribe the desired macro-level behavior, and then show 
how to use this macro-level goal to synthesize individ¬ 
ual robot controllers. Another use of formal methods is 
to show how the individual goals of robot team mem¬ 
bers can be considered collectively, with the objective 
of maximizing the system’s achievement of individual 
goals. Toward this end, game-theoretic techniques have 
been shown useful in a variety of distributed robot for¬ 
mulations. Cheng and Dasgupta [53.147] make use of 
the game-theoretic technique called Weighted Voting 
Games to address the problem of multi-robot team for¬ 
mation control amidst obstacles. Taheri et al. [53.148] 
also make use of game-theoretic principles, building 
upon the Local Interaction Game diffusion model to in¬ 
vestigate how a small number of agents can influence 
the global society’s behavior through local interactions. 

An interesting question in the design of distributed 
robot coordination mechanisms is the extent to which 
identical controllers can lead to diversity, specializa¬ 
tion, or changes in robot behavior. Hsieh et al. [53.149] 
study the emergence of specialization in robot swarms 
by making use of a distributed adaptation algorithm. 
They present a top-down analytical approach that de¬ 
fines the system equilibrium using waiting time param¬ 
eters, and then present adaptive optimization strategies 
that converge to the optimal configurations that achieve 
system equilibrium. Temporal changes in system-level 
swarm behavior are addressed by Hoff et al. [53.150] 
who show how a swarm can change and improve its for¬ 
aging behavior by switching between algorithms based 
on the environment in which the swarm finds itself. 

Once the distributed controller is synthesized, most 
of the works mentioned above presume that individual 
robots execute their controller successfully. The typi¬ 
cal presumption is that large swarms of interchangeable 
robots automatically result in robust and scalable swarm 
behavior. However, this presumption is challenged by 
Winfield and Nembrini [53.151], who illustrate that 
overall swarm reliability quickly falls in the presence of 
worst-case, partially failed robots. They conclude that 
future large scale swarm systems must develop new ap¬ 
proaches for achieving high levels of fault tolerance. 
One example approach is shown in loiMIllLliliiJ. 
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53.6 Modular Robotics 

The modular robotics field began with a paper pre¬ 
sented at International Conference on Robotics and 
Automation in the Spring of 1988 by Fukuda and Nak- 
agawa [53.152] describing the abstract concept of a re- 
configurable robotic system that can assume different 
shapes and envisioned a robot system composed of dif¬ 
ferent types of modules that can combine to accomplish 
a variety of tasks. Over the past twenty years, modu¬ 
lar robotics research developed many facets: hardware 
design; planning and control algorithms; the trade-off 
between hardware and algorithmic complexity; efficient 
simulation; and system integration. 

Modular robots are collections of physically con¬ 
nected, electromechanically active modules that, as 
a whole, form robotic systems that exhibit capabilities 
greater than those of the individual modules. Typically 
modular robots can change their shape or configuration 
in order to adapt to a variety of different tasks. For ex¬ 
ample, a collection of modules could reconfigure from 
a closed chain that rolls quickly over open ground to 
a legged robot that more easily traverses rough terrain. 
Modular robots are typically touted for their adaptabil¬ 
ity, their fault-tolerance, and the relative simplicity of 
the unit modules. Modular robotic systems can be de¬ 
scribed and classified on several axes using a variety of 
properties. In what follows, we choose the traditional 
route of classifying modular robotic systems by the ge¬ 
ometry of the system: chain, lattice, truss, or free form. 
For a more detailed history of the modular robotics 
field, consult [53.153, 154]. 

53.6.1 Chain Systems 

The defining characteristic of chain-type modular robot 
systems is the fact that the modules, when connected 
to their neighbors, are arranged in a chain. These 
chains may be one-dimensional, or two-dimensional, 
but three-dimensional chains are not as common. The 
fact that a chain-type modular robot is two-dimensional, 
or even one-dimensional, does not mean that it cannot 
operate in three dimensions. In fact, snake-like modular 
robots composed of segments with orthogonal joints are 
quite common. 

One of the first chain-type modular robotic systems 
was the polypod system developed by Yim [53.155, 156] 
The polypod system was composed of 
two types of modules: segments and nodes. It could form 
a variety of shapes including rolling loops and hexapods, 
and it went on to inspire many other chain-based sys¬ 
tems. One was the CONRO system [53.157-159] in 
which each module was composed of two orthogonal 
servo motors controlling each module’s pitch and yaw. 


Murata et al. developed the M-TRAN modular 
robotic system [53.160-163] which has undergone 
multiple revisions and improvements. In [53.161], 
Kamimura et al. employ a set of interconnected, out 
of phase oscillators to achieve walking gaits in the M- 
TRAN system. Marbach and Ijspeert improved upon 
the ability of systems like M-TRAN to generate gaits 
in real-time by applying function optimization to their 
modular system, YaMoR [53.164]. Murata et al. added 
cameras to the M-TRAN system so that a set of M- 
TRAN modules could separate, perform independent 
tasks, and then rejoin into a larger structure [53.163]. 

The ATRON system [53.165, 166] was developed 
to improve upon the M-TRAN. Lund et al. wanted to 
keep M-TRAN’s ability to form dense lattices while 
taking advantage of the two orthogonal degrees of free¬ 
dom, (pitch and yaw), found in the CONRO system. 
The Superbot system [53.167] also builds upon on the 
mechanical design of M-TRAN by adding an additional 
degree of rotational freedom between the two existing 
rotation axes. 

The PolyBot is chain-type modular robot [53.168, 
169] with a single rotational degree of freedom. Poly¬ 
Bot evolved into CKBot which has demonstrated the 
ability to reassemble itself after being accidentally or 
intentionally destroyed [53.170]. The Molecube sys¬ 
tem [53.171], developed by Lipson et al. is another 
example of a chain-type modular system with only one 
degree of freedom but still able to achieve interesting 
three-dimensional (3-D) configurations. Lipson et al. 
showed that a short chain of Molecube modules, along 
with some free modules, can self-replicate. 

Yim et al. designed another unique chain-type sys¬ 
tem named RATChET [53.172] which uses a connected 
chain of inter-latching right angle tetrahedrons to form 
structures. Neighboring RATChET modules latch to¬ 
gether when the angle between them passes some crit¬ 
ical value, and they unlatch through the use of shape 
memory alloy (SMA) springs when heated beyond 
70 °C. Interestingly, the RATChET modules possess no 
intelligence. Instead, they rely on an intelligent exter¬ 
nal actuator which rotates to control one end of the 
dangling chain. One unique property of the RATChET 
system is its relatively strength. 

53.6.2 Lattice Systems 

Lattice-type modular robot systems are collections of 
interconnected robotic modules in which the units are 
situated at the intersection points of a two or three 
dimensional grid. (A 1-D (one-dimensional) lattice sys¬ 
tem is simply a chain-type robot.) The main characteris- 
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tic separating a lattice system from a densely configured 
chain-type robot is the density of the interconnections 
between the modules. In a lattice-type system, each 
module is typically connected to all of its neighbors. In 
a dense chain-type system, two modules may be neigh¬ 
bors, but they will not be physically connected. 

Additionally, lattice-type systems tend to be built 
with modules that contain no rotational degrees of free¬ 
dom. While the modules in a lattice system typically 
have mechanisms which enable the modules to move 
relative to, and bond with, their neighbors, they gener¬ 
ally cannot bend themselves. In comparison, chain-type 
systems are often built from modules that contain one or 
more rotational degrees of free so that the modules can 
flex like links in a chain. There is some overlap between 
between the two types of system. 

Chirikjian et al. developed one of the first 
lattice-based modular robotic systems [53.173-175] 
(|4S>*ii!iiIE31) in which the modules are deformable 
hexagons capable of bonding with their neighbors. Oth¬ 
ers, such as Walter et al. [53. 176] further analyzed these 
hexagonal type systems to create distributed motion 
planners capable of reconfiguring the system from one 
state to another. 

Murata et al. were also early contributors to the 
development of lattice-based modular robotic systems 
with their development of a roughly hexagonal module 
capable of rolling around its neighbors in two dimen¬ 
sions [53.177, 178]. Kurokawa et al. presented a three 
dimensional adaptation [53.179] composed of cubes 
with six protruding arms capable of rotation. Yoshida 
et al. improved on this system with a new design that 
used shape-memory alloy actuators to rotate one robot 
module around the perimeter of a neighbor [53.180]. 

One of the simplest lattice systems is the the Digital 
Clay project [53.181]. The system was a set of com¬ 
pletely passive modules that relied on the user to make 
changes to it topology. The 2.5 cm rhombic dodecahe¬ 
drons were able to sense and communicate with their 
neighbors in order to create a virtual model of the phys¬ 
ical arrangement of modules. 

Rus et al. also explored the idea of 3-D mod¬ 
ules capable reconfiguration through a series of latch¬ 
ings, rotations, and unlatchings with the Molecule 
system [53.182-185]. In [53.186,187], Vona and Rus 
describe a different type of deformable lattice system. 
The Crystal system is composed of square modules able 
to expand and contract by a factor of two in the x-y 
plane. Suh et al. expanded on the Crystal concept with 
the Telecubes [53.188] that could move in three dimen¬ 
sions by expanding all six faces. 

Chiang and Chirikjian analyzed how to perform 
motion planning in a lattice of rigid cubic modules 
able to slide past each other [53.189]. The CHOBIE 


robot developed by Koseki [53.190] is able to actu¬ 
ally perform the sliding motion assumed by Chiang and 
Chirikjian in [53.189]. More recently. Ah developed the 
EM-Cube system [53.191] which is also capable of slid¬ 
ing motion. 

Another unique lattice is the I-Cube developed by 
Khoslae t al. [53.192, 193]. The 3-D I-Cube system con¬ 
sists of passive cubes which are connected by active links 
with three rotational degrees of freedom that are able to 
grab, reposition, and release the cubes. The 3-D I-Cube 
system was an improvement of the two-dimensional 
(2-D) system [53. 194] developed by Hosokawa et al. for 
rearranging cubic modules in a vertical plane. 

Goldstein et al. initiated the Claytronics project 
by publishing several papers [53.195,196] propos¬ 
ing lattice-based claytronic atoms or catoms. These 
vertically-oriented cylindrical robots, which were inca¬ 
pable of independent motion, used 24 electromagnets 
around their perimeters to achieve rolling locomotion 
about their neighbors. Goldstein et al. envisioned a sys¬ 
tem in which millions of smaller catoms could form 
arbitrary shapes using a randomized algorithm that 
avoided conveying a complete description of the shape 
to each module in the system. 

The catoms continue to evolve. One of the newest 
instantiations [53.197] employs hollow cylinders rolled 
from SiOi rectangles patterned with aluminum elec¬ 
trodes. The authors hope that two of these cylinders, 
when placed in close proximity with their axes aligned, 
will be able to rotate with respect to one another using 
electrostatic forces. Specifically, the electrodes, (which 
reside on the inside of each cylinder and are electrically 
isolated by the Si 02 ), will be charged so that they attract 
and repel mirror charges on the neighboring cylinder in 
a way that causes rotation. Currently, the system ap¬ 
pears to be constrained to form 2-D structures. The 
authors claim the completed system will have a yield 
strength similar to that of plastic and that the modules 
will be able to transfer power and communication sig¬ 
nals capacitively from neighbor to neighbor. 

The Claytronics project has proposed, but not yet 
demonstrated with hardware, the use of sub-millimeter 
intelligent particles as sensing and replication de¬ 
vices [53.198]. In particular, Pillai et al. present a the¬ 
oretical 3-D fax machine in which the object to be 
faxed is immersed in a container of intelligent particles 
that sense and encode the object’s dimensions. At the 
receiving end, these same Claytronic particles decode 
the shape description sent by the transmitter and bond 
together to replicate the original object. Unlike our ap¬ 
proach, Pillai’s approach is completely centralized and 
relies on an external computer for computation. 

White et al. developed hardware and algorithms for 
several 2-D stochastically-driven self-assembling sys- 
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terns [53.199]. To form specific shapes, each module is 
provided with a representation of the desired shape and 
decides, based on its location in the structure, whether 
to allow other modules to bond to its faces. Lipson et al. 
extended their 2-D system to 3-D [53.200-202] by using 
cubic modules suspended in turbulent fluid to achieve 
self-assembly and reconfiguration. As the free modules 
circulate in the fluid, they pass by a growing structure 
of assembled modules. When they come close enough, 
they are accreted onto the structure. The modules attract 
or repel each other with fluid suction or positive pres¬ 
sure. Early versions of the system used modules with 
interval values that could redirect these suction forces. 
More recently, Lipson 's group has worked to move the 
intelligence and actuation capabilities from the modules 
to the tank in which the modules circulate [53.203]. 

The Miche system [53.204] consisting of 45 mm cu¬ 
bic modules capable of mating with their neighbors us¬ 
ing mechanically switchable permanent magnets. Each 
module contains three switchable magnets, each of 
which mated with a steel face on a neighboring module. 
Because the connectors were gendered, any collection 
of modules had to be assembled by hand so that the con¬ 
nectors were always oriented correctly, but the system 
was capable of self-disassembling to form 3-D struc¬ 
tures. The Robot Pebbles are based, at 

least in principle, on the Miche modules. 

One of the newest lattice-type modular robotics 
is an aerial system composed of identical, hexagonal, 
single-rotor modules [53.205]. A group of modules may 
connect to form a flying platform with an arbitrary ar¬ 
rangement of multiple rotors. In addition to the ability 
to fly, each module contains wheels so that the system 
may self-reconfigure on the ground for the specific task 
at hand. 

53.6.3 Truss Systems 

Truss systems, as their name implies, are modular 
robotic systems in which the modules are nodes and 
edges in a truss structure. Both the trusses and con¬ 
nectors may be active in such systems. Unlike the 
lattice-based systems, truss-based systems do not need 
to operate on any regular lattice. Most truss-based sys¬ 
tems under development employ struts that expand or 
contract to achieve structural deformation. One of the 
first such system to do so was Tetrobot [53.206]. The 
Odin system, conceived by Lyder et al. [53.207,208] 
consists of three physically different types of modules: 
active strut modules capable of changing their length; 
passive strut modules of fixed length; and joint mod¬ 
ules. The biologically inspired Morpho system [53.209] 
developed by Nagpal et al. is similar to Odin. It also 
uses active links, passive links, and connector cubes. 


53.6.4 Free-Form Systems 

Free-form systems are able to aggregate modules in at 
least semi-arbitrary positions. One such system is the 
Slimebot [53.210,211]. The system consists of identi¬ 
cal vertical cylindrical modules that move on a horizon¬ 
tal plane. The perimeter of each module is covered by 
six gender-less hook and loop patches used to bond with 
neighboring modules. These patches oscillate radially 
in and out from the center of the body. By control¬ 
ling the frequency and phase of the oscillations between 
neighbors, the system can achieve aggregate motion in 
a given direction. 

Researchers are also developing algorithms for free¬ 
form systems. Funiak et al. developed a localization 
algorithm [53.212] that is capable of localizing tens- 
of-thousands of irregularly packed modules in 3-D. 
Rubenstein and Shen developed a number of shape for¬ 
mation algorithms for collections of two-dimensional 
modules. These algorithms allow an arbitrary-sized col¬ 
lection of modules to form arbitrary scale-independent 
shapes [53.213,214]. Once the shape is formed, mod¬ 
ules can be added to or removed from the system, and 
the system will reconfigure itself to incorporate the 
new modules. The resulting shape will grow or shrink, 
but its basic form will remain unchanged. Recently, 
Rubenstein et al. developed a 1000 modules hardware 
platform on which to deploy these algorithms [53.215]. 

Researchers have also explored the use of folding 
to create reconfigurable foldable systems [53.216,217]. 
These systems use flexible wiring and shape mem¬ 
ory alloy actuators embedded in composite sheets to 
programmatically create origami-inspired shapes. By 
controlling which actuators are energized, the system 
can form multiple different shapes. 

53.6.5 Self-Assembling Systems 

Self-assembling modular robotic systems are collec¬ 
tion of modules that are capable of autonomously 
coalescing and bonding with their neighbors to form 
a greater structure. The result is often robotic, but it 
need not be. Whether a system is capable of self¬ 
assembling is independent of whether it is free-form, 
a chain, a lattice, or a truss-based system. Almost all of 
aforementioned modular robot systems rely on human 
intervention to assemble. In an attempt to automate the 
process of creating intricate modular robotic systems, 
researchers have attempted to mimic and improve upon 
natural self-assembling systems. Whitesides et al. in¬ 
vestigated a wide variety of engineered self-assembling 
systems [53.218-220]. 

Miyashita et al. performed a more theoretical 
analysis of self-assembly using pie-shaped pieces 
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to form complete circles [53.221] from pie-shaped 
pieces. In the process, they followed Hosokawa et al.’s 
lead [53.222] and modeled the system as a chemical 
reaction. Shimizu and Suzuki developed a system of 
passive modules capable of self-repair when placed on 
a vibrating table [53.223]. 

Computer scientists have also investigated theoreti¬ 
cal aspects of self-assembly in the context of 2-D tiles 
which selectively bond with their neighbors to form 
simple well-defined shapes like squares [53.224—226]. 
Each side of every tile in the system has an associated 
bonding strength. When two tiles collide, they remain 
attached only if their cumulative bond strength exceeds 
a globally defined system entropy. To form a specific 
shape, one must design a set of tiles with the appropri¬ 
ate bonding strengths. 

Klavins et al. worked to develop intelligent self¬ 
assembling systems that employ triangular modules 
driven by oscillating fans on an air table to self- 
assemble different shapes [53.227]. The authors employ 
knowledge of the module’s local topology and in¬ 
ternal module state so that each module decides, in 
a distributed fashion, when to maintain or break a con¬ 
nection with its immediate neighbors. Griffith et al. also 
worked with intelligent modules capable of selective 
bonding to show that self-assembling systems may self- 
replicate [53.228]. 

Rus et al. [53.185] present the first generic rule- 
based approach to self-assembly, shape formation, and 
locomotion by reconfiguration. The rules can be used 
on any modular robot system that can implement the 
sliding cube model of relative motion. The result is an 
abstract set of rules for each of these tasks, that can be 
compiled down to module motions, taking into account 
how the physical module implementes translation and 
convex and concave transitions. 

Jones and Mataric [53.229] presented rule-based 
approach to self-assembly termed transition rule sets. 
In particular, they present a method that, given a goal 
structure, produces a set of rules shared among all mod¬ 
ules that govern when and where new modules are 
allowed to attach to the growing structure. Kelly and 
Zhang [53.230] expanded on this work by optimizing 
the size of the rule sets used to form a specific shape. 


53.7 Heterogeneity 

Robot heterogeneity can be defined in terms of vari¬ 
ety in robot behavior, morphology, performance quality, 
size, and cognition. In most large-scale multirobot sys¬ 
tems work, the benefits of parallelism, redundancy, and 
solutions distributed in space and time are obtained 


Werfel [53.23 1] also applied the idea of a transition rule 
set when studying the use of swarms to assemble com¬ 
plex structures from passive materials. 

Other groups have attempted to make self- 
assembly more deterministic. The MEMS (micro¬ 
electromechanical system) robots developed by Don¬ 
ald et al. [53.232,233] consists of thin (7—20|im), 
rectangular 260 pm x 60 pm), scratch-drive devices 
capable of moving on an insulating substrate embed¬ 
ded with electrodes. The authors used four of these 
robots to build larger composite structures. The Sitti 
group has developed a similar system of micro-meter 
sized robots [53.234]. Instead of using a scratch drive 
for locomotion, the robots are manipulated by exter¬ 
nal magnetic fields. The authors can electrostatically 
clamp any number of robots to the stage on which they 
move. With all but one robot immobilized, the remain¬ 
ing robot may be moved independently. The system 
naturally self-assembles because the robots contain per¬ 
manent magnets that attract their neighbors. 

The majority of existing self-assembly systems aim 
to form structures in one of two ways. Some systems 
such as [53.221,223-226] use a collection of applica¬ 
tion specific differentiated modules, that are only capa¬ 
ble of assembling in a particular fashion to form a spe¬ 
cific shape. In contrast, other systems such as [53.199- 
201,227,229-231,235] use completely generic mod¬ 
ules with more computation and communication ability 
embedded in each module. Both types of systems aim to 
form complex shapes in a direct manner: as these struc¬ 
tures grow from a single module, new modules are only 
allowed to attach to the structure in specific locations. 

An alternative approach eliminates many of the 
complexities of shape formation by active assembly 
by using dissasemly for shape formation. The Smart 
Pebble system [53.154, 204,236, 237] employs a set of 
distributed algorithms to perform two discrete steps: 
1) rely on stochastic forces to self-assemble a close- 
packed crystalline lattice of modules and 2) use the 
process of self-disassembly to remove the extra material 
from this block leaving behind the goal structure. By 
approaching shape formation in this manner, the entire 
shape-formation process is sped-up and the robustness 
of the system is increased. 


through the use of homogeneous robots, which are 
completely interchangeable (i. e., the swarm approach, 
as described in Sect. 53.4). However, certain complex 
applications of large-scale robot teams may require 
the simultaneous use of multiple types of sensors and 
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robots, all of which cannot be designed into a single 
type of robot. Some robots may need to be scaled to 
smaller sizes, which will limit their payloads, or cer¬ 
tain required sensors may be too expensive to duplicate 
across all robots on the team. Other robots may need 
to be large to carry application-specific payload or sen¬ 
sors, or to navigate long distances in a limited time. 
These applications, therefore, require the collaboration 
of large numbers of heterogeneous robots. 

The motivation for developing heterogeneity in 
multirobot teams is thus twofold: heterogeneity may be 
a design feature beneficial to particular applications, or 
heterogeneity may be a necessity. As a design feature, 
heterogeneity can offer economic benefits, since it can 
be easier to distribute varying capabilities across mul¬ 
tiple team members rather than to build many copies 
of monolithic robots. Heterogeneity can also offer en¬ 
gineering benefits, as it may simply be too difficult to 
design individual robots that incorporate all of the sens¬ 
ing, computational, and effector requirements of a given 
application. Heterogeneity in behavior may also arise in 
an emergent manner in physically homogeneous teams, 
as a result of behavior specialization. 

A second compelling reason to study heterogeneity 
is that it may be a necessity, in that it is nearly im¬ 
possible in practice to build a truly homogeneous robot 
team. The realities of individual robot design, construc¬ 
tion, and experience will inevitably cause a multirobot 
system to drift to heterogeneity over time. This is 
recognized by experienced roboticists, who have seen 
that several copies of the same model of robot can 
vary widely in capabilities due to differences in sensor 
tuning, calibration, etc. Over time, even minor initial 
differences among robots will grow due to individual 
robot drift and wear and tear. The implication is that, 
to employ robot teams effectively, we must understand 
diversity, predict how it will impact performance, and 
enable robots to adapt to the diverse capabilities of their 
peers. In fact, it is often advantageous to build diversity 
explicitly into the design of a robot team. 

There are a variety of research challenges in hetero¬ 
geneous multirobot systems. A particular challenge to 
achieving efficient autonomous control is when overlap 
in team member capabilities occurs, thus affecting task 
allocation or role assignments [53.238]. Techniques as 
described in Sect. 53.6 can typically deal with het¬ 
erogeneous robots for the purposes of task allocation. 
Another important topic in heterogeneity is how to rec¬ 
ognize and quantify heterogeneity in multirobot teams. 
Some types of heterogeneity can be evaluated quantita¬ 
tively, using metrics such as the social entropy metric 
developed by Batch [53.239]. Most research in hetero¬ 
geneous multirobot systems assumes that robots have 
a common language and a common understanding of 


symbols in their language; developing a common un¬ 
derstanding of communicated symbols among robots 
with different physical capabilities is a fundamental 
challenge, addressed by Jung and Zelinskv [53.240] 

As discussed in Sect. 53.2, one of the earliest 
research demonstrations of heterogeneity in physi¬ 
cal robot teams was in the development of the AL¬ 
LIANCE architecture by Parker [53.9]. This work 
demonstrated the ability of robots to compensate for 
heterogeneity in team members during task alloca¬ 
tion and execution. Murphy has studied heterogeneity 
in the context of marsupial robot deployment, where 
a mothership robot assists smaller robots in applications 
such as search and rescue [53.241] 

Grabowski et al. [53.134] developed modular milli- 
bots for surveillance and reconnaissance that could 
be composed of interchangeable sensor and effector 
components, thus creating a variety of different het¬ 
erogeneous teams. Simmons et al. [53.11] demonstrated 
the use of heterogeneous robots for autonomous as¬ 
sembly and construction tasks relevant to space appli¬ 
cations. Sukhatme et al. [53.242] demonstrated a he¬ 
licopter robot cooperating with two ground robots in 
tasks involving marsupial-inspired payload deployment 
and recovery, cooperative localization, and reconnais¬ 
sance and surveillance tasks, as shown in Fig. 53.14. 
Parker et al. [53.243] demonstrated assistive naviga¬ 
tion for sensor network deployment using a more 
intelligent leader robot for guiding navigationally chal¬ 
lenged simple sensor robots to goal locations, as part of 
a larger demonstration by Howard et al. [53.244] of 100 
robots performing exploration, mapping, deployment, 
and detection. Chaimowicze .t al. [53.245] demonstrated 
a team of aerial and ground robots cooperating for 
surveillance applications in urban environments. Parker 
and Tang [53.246] developed ASyMTRe (automated 



Fig. 53.14 Heterogeneous team of an air and two ground 
vehicles that can perform cooperative reconnaissance and 
surveillance 
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synthesis of multirobot task solutions through software 
reconfiguration), which enables heterogeneous robots 
to share sensory resources to enable the team to ac¬ 
complish tasks that would be impossible without tightly 
coupled sensor sharing. 

Many open research issues remain to be solved in 
heterogeneous multirobot teams; for example, the issue 
of optimal team design is a very challenging problem. 
Clearly, the required behavioral performance in a given 

53.8 Task Allocation 

In many multirobot applications, the mission of the team 
is defined as a set of tasks that must be completed. Each 
task can usually be worked on by a variety of differ¬ 
ent robots; conversely, each robot can usually work on 
a variety of different tasks. In many applications, a task 
is decomposed into independent subtasks [53.9], hier¬ 
archical task trees [53.247], or roles [53.11,245,248, 
249] either by a general autonomous planner or by the 
human designer. Independent subtasks or roles can be 
achieved concurrently, while subtasks in task trees are 
achieved according to their interdependence. Once the 
set of tasks or subtasks have been identified, the chal¬ 
lenge is to determine the preferred mapping of robots to 
tasks (or subtasks). This is the task allocation problem. 

The details of the task allocation problem can vary 
in many dimensions, such as the number of robots re¬ 
quired per task, the number of tasks a robot can work 
on at a time, the coordination dependencies among 
tasks, and the time frame for which task assignments 
are determined. Gerkey and Mataric [53.250] defined 
a taxonomy for task allocation that provides a way of 
distinguishing task allocation problems along these di¬ 
mensions, which is referred to as the multirobot task 
allocation (MRTA) taxonomy. 

53.8.1 Taxonomy for Task Allocation 

Generally, tasks are considered to be of two principal 
types: single-robot tasks (SR, according to the MRTA 
taxonomy) are those that require only one robot at 
a time, while multirobot tasks (MR) are those that re¬ 
quire more than one robot working on the same task 
at the same time. Commonly, single-robot tasks that 
have minimal task interdependencies are referred to as 
loosely coupled tasks, representing a weakly coopera¬ 
tive solution. On the other hand, multirobot tasks are 
often considered to be sets of subtasks that have strong 
interdependencies. These tasks are therefore often re¬ 
ferred to as tightly coupled tasks that require a strongly 
cooperative solution. The subtasks of a loosely coupled 
multirobot task require a high level of synchroniza- 


application dictates certain constraints on the physi¬ 
cal design of the robot team members. However, it is 
also clear that multiple choices may be made in de¬ 
signing a solution to a given application, based upon 
cost, robot availability, ease of software design, flexi¬ 
bility in robot use, and so forth. Designing an optimal 
robot team for a given application requires significant 
analysis and consideration of the tradeoffs in alternative 
strategies. 


tion or coordination between subtasks, meaning that 
each task must be aware of the current state of the co¬ 
ordinated subtasks within a small time delay. As this 
time delay becomes progressively larger, coordinated 
subtasks become more loosely coupled, representing 
weakly cooperative solutions. 

Robots can also be categorized as either single-task 
robots (ST), which work on only one task at a time or 
multitask robots (MT), which are able to make progress 
on more than one task at a time. Most commonly, task al¬ 
location problems assume robots are single-task robots, 
since more capable robots that perform multiple tasks in 
parallel are still beyond the current state of the art. 

Tasks can either be assigned to optimize the in¬ 
stantaneous allocation of tasks (IA), or to optimize the 
assignments into the future (TA, for time-extended as¬ 
signment). In the case of instantaneous assignment, 
no consideration is made for the effect of the cur¬ 
rent assignment on future assignments. Time-extended 
assignments attempt to assign tasks so that the perfor¬ 
mance of the team is optimized for the entire set of tasks 
that may be required, not just the current set of tasks that 
need to be achieved at the current time step. 

Using the MRTA taxonomy, triples of these abbre¬ 
viations are used to categorize various task allocation 
approaches, such as SR-ST-IA, which refers to an 
assignment problem in which single-robot tasks are as¬ 
signed once to single-task robots. Different variations 
of the task allocation problem have different computa¬ 
tional complexities. The easiest variant is the ST-SR- 
IA problem, which can be solved in polynomial time 
since it is an instance of the optimal assignment prob¬ 
lem [53.251]. Other variants are much more difficult, 
and do not have known polynomial time solutions. For 
example, the ST-MR-IA variant can be shown to be an 
instance of the set partitioning problem [53.252], which 
is strongly (VP-hard. The ST-MR-TA, MT-SR-IA, and 
MT-SR-TA variants have also all been shown to be 
/VP-hard problems. Because these problems are compu¬ 
tationally complex, most approaches to task allocation 
in multirobot teams generate approximate solutions. 
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53.8.2 Representative Approaches 

Approaches to task allocation in multirobot teams can 
be roughly divided into behavior-based approaches and 
market-based (sometimes called negotiation-style or 
auction-based) approaches. The following subsections 
describe some representative architectures for each of 
these general approaches. Refer to [53.250] for a com¬ 
parative analysis of some of these approaches, in terms 
of computation and communications requirements and 
solution quality. 

Behavior-Based Task Allocation 
Behavior-based approaches typically enable robots to 
determine task assignments without explicitly dis¬ 
cussing individual tasks. In these approaches, robots use 
knowledge of the current state of the robot team mis¬ 
sion, robot team member capabilities, and robot actions 
to decide, in a distributed fashion, which robot should 
perform which task. 

One of the earliest architectures for multirobot task 
allocation that was demonstrated on physical robots 
was the behavior-based ALLIANCE architecture [53.9] 
and the related L-ALLIANCE architecture [53.10]. AL¬ 
LIANCE addresses the ST-SR-IA and ST-SR-TA vari¬ 
ants of the task allocation problem without explicit com¬ 
munication among robots about tasks. As described in 
Sect. 53.2.2, ALLIANCE achieves adaptive action se¬ 
lection through the use of motivational behaviors, which 
are levels of impatience and acquiescence within each 
robot that determine its own and its teammates’ relative 
fitness for performing certain tasks. These motivations 
are calculated based upon the mission requirements, the 
activities and capabilities of teammates, and the robots’ 
internal states. These motivations effectively calculate 
utility measures for each robot-task pair. 

Another behavior-based approach to multirobot 
task allocation is broadcast of local eligibility 
(BLE) [53.253], which addresses the ST-SR-IA variant 
of task allocation. BLE uses a subsumption style be¬ 
havior control architecture [53.254] that allows robots 
to efficiently execute tasks by continuously broadcast¬ 
ing locally computed eligibilities and only selecting the 
robot with the best eligibility to perform the task. In 
this case, task allocation is achieved through behav¬ 
ior inhibition. BLE uses an assignment algorithm that 
is very similar to Botelho and Alami’s M+ architec¬ 
ture [53.255]. 

Market-Based Task Allocation 
Market-based (or negotiation-based) approaches typi¬ 
cally involve explicit communications between robots 
about the required tasks, in which robots bid for tasks 


based on their capabilities and availability. The nego¬ 
tiation process is based on market theory, in which 
the team seeks to optimize an objective function based 
upon individual robot utilities for performing particular 
tasks. The approaches typically greedily assign sub¬ 
tasks to the robot that can perform the task with the 
highest utility. 

Smith ’s contract net protocol (CNP) [53.256] was 
the first to address the problem of how agents can ne¬ 
gotiate to collectively solve a set of tasks. The use 
of a market-based approach specifically for multirobot 
task allocation was first developed by Botelho and 
Alami with their M+ architecture [53.255]. In the M+ 
approach, robots plan their own individual plans for the 
task they have been assigned. They then negotiate with 
other teammates to incrementally adapt their actions to 
suit the team as a whole, through the use of social rules 
that facilitate the merging of plans. 

Since these early developments, many alternative 
approaches to market-based task allocation have been 
developed. A thorough survey on the current state of 
the art in market-based techniques for multirobot task 
allocation is given in [53.257], comparing alternative 
approaches in terms of solution quality, scalability, 
dynamic events and environments, and heterogeneous 
teams. 

Most of the current approaches in market-based 
task allocation address the ST-SR problem variant, 
with some approaches (e.g., [53.11,258-260]) deal¬ 
ing with instantaneous assignment (IA), and others 
(e.g., [53.135,261-263]) addressing time-extended as¬ 
signments (TA). More recent methods are beginning to 
address the coaltion formation problem, which is the 
allocation of multirobot tasks (i. e., the MR-ST prob¬ 
lem variant), including [53.246, 264—269]. An example 
approach to the MR-MT problem variant is found 
in [53.270]. 

Some representative market-based techniques in¬ 
clude MURDOCH [53.258], TraderBots [53.247, 
263], and Hoplites [53.265]. The MURDOCH ap¬ 
proach [53.258] employs a resource-centric, publish- 
subscribe communication model to carry out auctions, 
which has the advantage of anonymous communication. 
In this approach a task is represented by the required 
resources, such as the environmental sensors. The meth¬ 
ods for how to use such a sensor to generate satisfactory 
results is preprogrammed into the robot. 

The TraderBots approach [53.247,263] applies 
market economy techniques for generating efficient 
and robust multirobot coordination in dynamic environ¬ 
ments. In a market economy, robots act based on selfish 
interests. A robot receives revenue and incurs cost when 
trying to accomplish a task. The goal is for robots to 
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trade tasks through auctions/negotiations such that the 
team profit (revenue minus cost) is optimized. 

The Hoplites approach [53.265] focuses on the se¬ 
lection of an appropriate joint plan for the team to 
execute by incorporating joint revenue and cost into 
the bid. This approach couples planning with passive 
and active coordination strategies, enabling robots to 
change coordination strategies as the needs of the task 


53.9 Learning 

Multirobot learning is the problem of learning new 
cooperative behaviors, or learning in the presence of 
other robots. The other robots in the environment, how¬ 
ever, have their own goals and may be learning in 
parallel [53.271]. The challenge is that having other 
robots in the environment violates the Markov prop¬ 
erty that is a fundamental assumption of single-robot 
learning approaches [53.271]. The multirobot learning 
problem is particularly challenging because it combines 
the difficulties of single-robot learning with multiagent 
learning. Particular difficulties that must be consid¬ 
ered in multirobot learning include continuous state 
and action spaces, exponential state spaces, distributed 
credit assignment, limited training time and insufficient 
training data, uncertainty in sensing and shared infor¬ 
mation, nondeterministic actions, difficulty in defining 
appropriate abstractions for learned information, and 
difficulty of merging information learned from differ¬ 
ent robot experiences. 

The types of applications that have been studied 
for multirobot learning include multitarget observa¬ 
tion [53.272,273], air fleet control [53.274], predator- 
prey [53.137,275,276], box pushing [53.277], forag¬ 
ing [53.23], and multirobot soccer [53.140,278]. Partic¬ 
ularly challenging domains for multirobot learning are 
those tasks that are inherently cooperative. Inherently 
cooperative tasks are those that cannot be decomposed 
into independent subtasks to be solved by individual 
robots. Instead, the utility of the action of one robot is 
dependent upon the current actions of the other team 
members. This type of task is a particular challenge in 
multirobot learning, due to the difficulty of assigning 
credit for the individual actions of the robot team mem¬ 
bers. 

The credit assignment problem is a particular chal¬ 
lenge, since it is difficult for a robot to determine 
whether the fitness (either good or bad) is due to its own 
actions, or due to the actions of another robot. As dis¬ 
cussed by Pugh and Martinoli in [53.279], this problem 
can be especially difficult in situations where robots do 


change. Strategies are predefined for a robot to accom¬ 
plish a selected plan. 

Some alternative approaches formulate the objects 
to be assigned as roles, which typically package a set 
of tasks and/or behaviors that a robot should undertake 
when acting in a particular role. Roles can then be dy¬ 
namically assigned to robots in a similar manner as in 
the auction-based approaches [53.1 1,248]. 


not explicitly share their intentions. Two different vari¬ 
ations of the credit assignment problem are common in 
multirobot learning. The first is when robots are learn¬ 
ing individual behaviors in the presence of other robots 
that can affect their performance. The second is when 
robots are attempting to learn a task with a shared fit¬ 
ness function. It can be difficult to determine how to 
decompose the fitness function to appropriately reward 
or penalize the contributions of individual robots. 

While learning has been explored extensively in 
the area of single-robot systems (see, for example, 
the discussion of learning in behavior-based systems 
in Chap. 13, and a discussion of fundamental learn¬ 
ing techniques in Chap. 15) and in multi agent sys¬ 
tems [53.280], much less work has been done in the area 
of mxAiirobot learning, although the topic is gaining in¬ 
creased interest. Much of the work to date has focused 
on reinforcement learning approaches. Some examples 
of this multirobot learning research include the work by 
Asada et al. [53.281], who propose a method for learn¬ 
ing new behaviors by coordinating previously learned 
behaviors using Q-learning, and apply it to soccer¬ 
playing robots. Mataric [53.8] introduces a method 
for combining basic behaviors into higher-level be¬ 
haviors through the use of unsupervised reinforcement 
learning, heterogeneous reward functions, and progress 
estimators. This mechanism was applied to a team 
of robots learning to perform a foraging task. Kubo 
and Kakazu [53.282] proposed another reinforcement 
learning mechanism that uses a progress value for 
determining reinforcement, and applied it to simu¬ 
lated ant colonies competing for food. Fernandez and 
Parker [53.272] apply a reinforcement learning algo¬ 
rithm that combines supervised function approximation 
with generalization methods based on state-space dis¬ 
cretization, and apply it to robots learning the multi¬ 
object tracking problem. Bowling and Veloso [53.271] 
developed a general-purpose, scalable learning algo¬ 
rithm called GraWoLF (gradient-based win or learn 
fast), which combines gradient-based policy learning 
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techniques with a variable learning rate, and demon¬ 
strated the results in the adversarial multirobot soccer 
application. 

Other multirobot learning approaches not based on 
reinforcement include Parker ’s L-ALLIANCE archi¬ 
tecture [53.10], which uses parameter tuning, based on 


53.10 Applications 

Many real-world applications can potentially bene¬ 
fit from the use of multiple mobile robot systems. 
Example applications include container management 
in ports [53.283], extraplanetary exploration [53.284], 
search and rescue [53.241], mineral mining, transporta¬ 
tion, industrial and household maintenance, construc¬ 
tion [53.11], hazardous waste cleanup [53.9], secu¬ 
rity [53.285, 286], agriculture, and warehouse manage¬ 
ment [53.287] ). Multiple robot systems 

are also used in the domain of localization, map¬ 
ping, and exploration; Chap. 46 mentions some of the 
work in multirobot systems applied to these problems. 
Parts F and G of this handbook outline many appli¬ 
cation areas that are relevant not only to single-robot 
systems, but also to multiple mobile robot systems. 
To date, relatively few real-world implementations of 
these multirobot systems have occurred, primarily due 
to the complexities of multiple robot systems and the 
relative newness of the supporting technologies. Nev¬ 
ertheless, many proof-of-principle demonstrations of 
physical multirobot systems have been achieved, and 
the expectation is that these systems will find their way 
into practical implementations as the technology con¬ 
tinues to mature. 

Research in multiple mobile robot systems is often 
explored in the context of common application test do¬ 
mains. While not yet elevated to the level of benchmark 
tasks, these common domains do provide opportuni¬ 
ties for researchers to compare and contrast alterna¬ 
tive strategies to multirobot control. Additionally, even 
though these common test domains are usually just 
laboratory experiments, they do have relevance to real- 
world applications. This section outlines these common 
application domains; see also [53.2] and [53.288] for 
a discussion of these domains and a more detailed list¬ 
ing of related research. 

53.10.1 Foraging and Coverage 

Foraging is a popular testing application for multirobot 
systems, particularly for those approaches that address 
swarm robotics, involving very large numbers of mobile 
robots. In the foraging domain, objects such as pucks 


statistical experience data, to learn the fitness of differ¬ 
ent heterogeneous robots in performing a set of tasks. 
Pugh and Martinoli [53.279] apply particle swarm op¬ 
timization techniques to distributed unsupervised robot 
learning in groups, for the task of learning obstacle 
avoidance. 


or simulated food pellets are distributed across the pla¬ 
nar terrain, and robots are tasked with collecting the 
objects and delivering them to one or more gathering 
locations, such as a home base. Foraging lends itself to 
the study of weakly cooperative robot systems, in that 
the actions of individual robots do not have to be tightly 
synchronized with each other. This task has tradition¬ 
ally been of interest in multirobot systems because of 
its close analogy to the biological systems that motivate 
swarm robotics research. However, it also has relevance 
to several real-world applications, such as toxic waste 
cleanup, search and rescue, and demining. Additionally, 
since foraging usually requires robots to completely ex¬ 
plore their terrain in order to discover the objects of 
interest, the coverage domain has similar issues to the 
foraging application. In coverage, robots are required 
to visit all areas of their environment, perhaps search¬ 
ing for objects (such as landmines) or executing some 
action in all parts of the environment (e.g., for floor 
cleaning). The coverage application also has real-world 
relevance to tasks such as demining, lawn care, environ¬ 
mental mapping, and agriculture. 

In foraging and coverage applications, a fundamen¬ 
tal question is how to enable the robots to explore their 
environments quickly without duplicating actions or in¬ 
terfering with each other. Alternative strategies can in¬ 
clude basic stigmergy [53.14], forming chains [53.120], 
and making use of heterogeneous robots [53.131]. 
Other research demonstrated in the foraging and/or cov¬ 
erage domain includes [53.23, 132,289-294], 

53.10.2 Flocking and Formations 

Coordinating the motions of robots relative to each 
other has been a topic of interest in multiple mobile 
robot systems since the inception of the field. In particu¬ 
lar, much attention has been paid to the flocking and for¬ 
mation control problems (l<s>SUUl!tiU, l-S£>M!tliLilE31). 
The flocking problem could be viewed as a subcase 
of the formation control problem, requiring robots to 
move together along some path in the aggregate, but 
with only minimal requirements for paths taken by spe¬ 
cific robots. Formations are stricter, requiring robots to 
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maintain certain relative positions as they move through 
the environment. In these problems, robots are assumed 
to have only minimal sensing, computation, effector, 
and communications capabilities. A key question in 
both flocking and formation control research is deter¬ 
mining the design of local control laws for each robot 
that generate the desired emergent collective behavior. 
Other issues include how robots cooperatively localize 
themselves to achieve formation control [53.133,295], 
and how paths can be planned for permutation-invariant 
multirobot formations [53.296]. 

Early solutions to the flocking problem in artifi¬ 
cial agents were generated by Reynolds [53.297] using 
a rule-based approach. Similar behavior- or rule-based 
approaches have been used physical robot demonstra¬ 
tions and studies, such as in [53.121, 298]. These earlier 
solutions were based on human-generated local con¬ 
trol rules that were demonstrated to work in practice. 
More recent work is based on control theoretic princi¬ 
ples, with a focus on proving stability and convergence 
properties in multirobot team behaviors. Examples of 
this work include [53.128, 299-307]. Refer to [53.308, 
309] for surveys of relevant control theoretic work. 

53.10.3 Object Transportation 

and Cooperative Manipulation 

Some of the earliest work in swarm robotics was aimed 
at the object transportation task [53.13, 123,310-313], 
which requires a team of robots to move an object 
from its current position in the environment to some 
goal destination The primary benefit 

of using collective robots for this task is that the indi¬ 
vidual robots can combine forces to move objects that 
are too heavy for individual robots working alone or in 
small teams. However, the task is not without its chal¬ 
lenges; it is non-trivial to design decentralized robot 
control algorithms that can effectively coordinate robot 
team members during object transportation. A further 
complication is that the interaction dynamics of the 
robots with the object can be sensitive to certain object 
geometries [53.314,315] and object rotations during 
transportation [53.315], thus exacerbating the control 
problem. 

Object transportation and cooperative manipula¬ 
tion are popular domains for demonstrating multirobot 
cooperation, because they offer a clear domain where 
close coordination and cooperation is required. A com¬ 
mon type of object transportation - box pushing - 
requires robot teams to move boxes from their start¬ 
ing positions to defined goal configurations, sometimes 
along specified paths. Typically, box pushing operates 
in the plane, and the assumption is made that the boxes 
are too heavy or too long to enable single robots to 


push alone. Sometimes there are several boxes to be 
moved, with ordering dependencies constraining the 
sequence of motions. Cooperative manipulation is sim¬ 
ilar, except it requires robots to lift and carry objects 
to a destination. This test bed domain lends itself to 
the study of strongly cooperative multirobot strategies, 
since robots often have to synchronize their actions to 
successfully execute these tasks. The domain of box 
pushing and cooperative manipulation is also popular 
because it has relevance to several real-world appli¬ 
cations [53.288], including warehouse stocking, truck 
loading and unloading, transporting large objects in 
industrial environments, and assembly of large-scale 
structures. 

Researchers usually emphasize different aspects of 
their cooperative control approach in the box push¬ 
ing and cooperative manipulation domain. For example, 
Kube and Zhang [53.13] (I^M’ll'H'ltTl) demonstrate 
how swarm-type cooperative control techniques could 
achieve box pushing (Fig. 53.15), Parker [53.10,316] 
illustrates aspects of adaptive task allocation and learn¬ 
ing, Donald et al. [53.317] (|o>®ilil£li!kl) illustrates 
concepts of information invariance and the interchange- 
ability of sensing, communication, and control, and 
Simmons et al. [53.1 1] demonstrate the feasibility of co¬ 
operative control for building planetary habitats. 

In general, the manipulation techniques used for 
collective object transportation can be grouped into 
three primary methods [53.318]: pushing, grasping, and 
caging. The pushing approach [53.10, 11, 13,316,317] 
requires contact between each robot and the object, in 
order to impart force in the goal direction; however, 
the robots are not physically connected with the ob¬ 
ject. In the grasping approach [53.123, 142,310-312, 
319-322], each robot in the team physically attaches to 
the object being transported. See for example Fig. 53.16 



Fig. 53.15 Collective pushing of lighted box 
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Fig. 53.16 Cooperative stick pulling 



Fig. 53.17 Collective transport via caging 


for cooperative stick pulling work of [53.320]. Finally, 
the caging approach [53.323-326] (|4S>*ii!liJfcUJi) in¬ 
volves robots encircling the object so that the object 
moves in the desired direction, even without the con¬ 
stant contact of all the robots with the object. See 
Fig. 53.17 for an example of collective transport via 
caging, from [53.323]. 

A closely related task is that of collective construc¬ 
tion and wall building. The objective of the collective 
construction and wall building task is for robots to 
build structures of a specified form, in either 2-D or 
3-D. This task is distinguished from self-reconfigurable 
robots, whose bodies themselves serve as the dynamic 
structure. We if el and Nagpal have extensively explored 
this topic (l<st>miiH!Xita), developing distributed algo¬ 
rithms that enable simplified robots to build structures 
based on provided blueprints, both in 2-D [53.327-329] 
and in 3-D [53.330]. In their 3-D approach, the sys¬ 
tem consists of idealized mobile robots that perform 
the construction, and smart blocks that serve as the pas¬ 
sive structure. The robots’ job is to provide the mobility, 
while the blocks’ role is to identify places in the grow¬ 
ing structure at which an additional block can be placed 
that is on the path toward obtaining the desired final 


structure. The goal of their work is to be able to deploy 
some number of robots and free blocks into a construc¬ 
tion zone, along with a single block that serves as a seed 
for the structure, and then have the construction to pro¬ 
ceed autonomously according to the provided blueprint 
of the desired structure. 

Hardware challenges of collective robot construc¬ 
tion are addressed by Terada and Murata [53.331]. In 
this work, a hardware design is proposed that defines 
passive building blocks, along with an assembler robot 
that constructs structures with the robots. Other related 
work on the topic of collective construction includes the 
work of Wciwerla et al. [53.332], in which robots use 
a behavior-based approach to build a linear wall using 
blocks equipped with either positive or negative Velcro, 
distinguished by block color. Their results show that 
adding 1 bit of state information to communicate the 
color of the last attached block provides a significant 
improvement in the collective performance. The work 
by Stewart and Russell [53.333,334] proposes a dis¬ 
tributed approach to building a loose wall structure with 
a robot swarm. 

Another type of construction is called blind bull¬ 
dozing, which is inspired by a behavior observed in 
certain ant colonies. Rather than constructing by accu¬ 
mulating materials, this approach achieves construction 
by removing materials. This task has practical appli¬ 
cation in site clearing, such as would be needed for 
planetary exploration [53.335]. Early ideas of this con¬ 
cept were discussed by Brooks et al. in [53.336], which 
argues for large numbers of small robots to be deliv¬ 
ered to the lunar surface for site preparation. Parker 
et al. [53.337], further develop this idea by proposing 
robots using force sensors to clear an area by pushing 
material to the edges of the work site. 

A significant body of additional research has been 
illustrated in this domain; representative examples in¬ 
clude [53.3, 6,123,258,284,310,323, 338-344], 

53.10.4 Multitarget Observation 

The domain of multitarget observation requires mul¬ 
tiple robots to monitor and/or observe multiple tar¬ 
gets moving through the environment. The objective 
is to maximize the amount of time, or the likeli¬ 
hood, that the targets remain in view by some team 
member throughout task execution. The task can be 
especially challenging if there are more targets than 
robots. This application domain can be useful for study¬ 
ing strongly cooperative task solutions, since robots 
have to coordinate their motions or the switching of 
targets to follow in order to maximize their objective. 
In the context of multiple mobile robot applications, 
the planar version of this test bed was first introduced 
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in [53.345] as cooperative multirobot observation of 
multiple moving targets (CMOMMT). Similar prob¬ 
lems have been studied by several researchers, and 
extended to more complex problems such as environ¬ 
ments with complex topography or three-dimensional 
versions for multiple aerial vehicle applications. This 
domain is also related to problems in other areas, such 
as art gallery algorithms, pursuit evasion, and sen¬ 
sor coverage. This domain has practical application in 
many security, surveillance, and reconnaissance prob¬ 
lems. Research applied to the multitarget observation 
problem in multirobot systems includes [53.138,253, 
346-353], 

53.10.5 Traffic Control and Multirobot Path 
Planning 

When multiple robots are operating in a shared envi¬ 
ronment, they must coordinate their actions to prevent 
interference. These problems typically arise when the 
space in which robots operate contains bottlenecks, 
such as networks of roadways, or when the robots take 
up a relatively large portion of the navigable space. 
In these problems, the open space can be viewed as 
a resource that robots must share as efficiently as possi¬ 
ble, avoiding collisions and deadlocks. In this domain, 
robots usually have their own individual goals, and must 
work with other robots to ensure that they receive use of 
the shared space to the extent needed to achieve their 
goals. In some variants, the entire paths of multiple 
robots need to be coordinated with each other; in other 
variants, robots must simply avoid interfering with each 
other. 

A variety of techniques have been introduced to ad¬ 
dress this problem, including traffic rules, subdividing 
the environment into single-ownership sections, and ge¬ 
ometric path planning ([53.354] for an overview). Many 
of the earliest research approaches to this problem were 
based on heuristic approaches, such as predefining mo¬ 
tion control (or traffic) rules that were shown to prevent 
deadlock [53.355-358], or using techniques similar to 
mutual exclusion in distributed computing [53.359, 
360]. These approaches have the benefit of minimiz¬ 
ing the planning cost for obtaining a solution. Other, 
more formal, techniques view the application as a ge¬ 
ometric multirobot path planning problem that can be 
solved precisely in configuration space-time. Chapter 7 
includes a discussion of motion planning for multiple 
robots relevant to this domain. While geometric motion 
planning approaches provide the most general solu¬ 
tions, they can often be too computationally intensive 
for practical application, impractical due to the dy¬ 
namic nature of the environment, or simply unnecessary 



Fig. 53.18 Legged robot teams competing in robot soccer 


for the problem at hand. In these cases, approxima¬ 
tion approaches may be sufficient, such as centralized 
techniques that limit the search space through roadmap¬ 
ping [53.361,362], and decoupled approaches that use 
either prioritized planning [53.363-365] (i. e., generat¬ 
ing robot paths one by one) or path coordination (i. e., 
first planning individual paths for robots, then handling 
collision avoidance). 

53.10.6 Soccer 

Since the inception of the RoboCup multirobot soc¬ 
cer domain as a proposed challenge problem for 
studying coordination and control in multirobot sys¬ 
tems [53.366], research in this domain has grown 
tremendously. This domain incorporates many chal¬ 
lenging aspects of multirobot control, including col¬ 
laboration, robot control architectures, strategy acqui¬ 
sition, real-time reasoning and action, sensor fusion, 
dealing with adversarial environments, cognitive mod¬ 
eling, and learning. Annual competitions show the 
ever-improving team capabilities of the robots in a var¬ 
iety of settings, as shown in Fig. 53.18. A key aspect 
of this domain that is not present in the other mul¬ 
tirobot test domains is that robots must operate in 
adversarial environments. This domain is also pop¬ 
ular because of its educational benefits, as it brings 
together students and researchers from across the 
world in competitions to win the RoboCup chal¬ 
lenges. The RoboCup competitions have added an 
additional search-and-rescue category to the competi¬ 
tion [53.367], which has also become a significant area 
of research (Chap. 66 for more details on this field). 
Annual proceedings of the RoboCup competitions 
document much of the research that is incorporated 
into the multirobot soccer teams. Some representative 
research works include [53.368-372] 
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53.11 Conclusions and Further Reading 


This chapter has surveyed the current state of the art 
in multirobot systems, examining architectures, com¬ 
munications issues, swarm robot systems, heteroge¬ 
neous teams, task allocation, learning, and applications. 
Clearly, significant advances have been made in the 
held in the last decade. The held is still an active area 
of research, however, since many open research issues 
still remain to be solved. Key open research questions 
remain in the broad areas of system integration, robust¬ 
ness, learning, scalability, generalization, and dealing 
with heterogeneity. 

For example, in the area of system integration, an 
open question is how to effectively allow robot teams 
to combine a spectrum of approaches toward achieving 
complete systems that can perform more than a limited 
set of tasks. We are a long way from creating ro¬ 
bust robot networks that can perform physical tasks 
in the real world. In the area of robustness, multi¬ 
robot teams still need improvements in the ability to 
degrade gracefully, to reason for fault tolerance, and 
to achieve complexity without escalating failure rates. 
The area of learning in multirobot teams is still in its 
infancy, with open questions including how to achieve 
continual learning in multirobot teams, how to facilitate 
the use of complex representations, and how to en¬ 
able humans to influence and/or understand the results 
of the team learning. Scalability is still a challeng¬ 
ing problem, in terms of more complex environments 
as well as ever-larger numbers of robots. We do not 
yet have a methodology for creating self-organizing 
robot networks that are robust to labeling (or num¬ 
bering), with completely decentralized controllers and 
estimators, and with provable emergent response. This 


requires basic research at the intersection of control, 
perception, and communication. Open issues in gen¬ 
eralization include enabling the robot team to reason 
about context and increasing the versatility of systems 
so that they can operate in a variety of different appli¬ 
cations. In dealing with heterogeneity, open questions 
include determining theoretical approaches to predict¬ 
ing system performance when all robots are not equal, 
and determining how to design a robot team optimally 
for a given application. Advances over the last decade 
have provided human users with the ability to inter¬ 
act with hundreds or thousands of computers on the 
Internet. It is necessary to develop similar network¬ 
centric approaches to interfacing, both for control and 
for monitoring. Finally, a major challenge is to create 
systems that are proactive and anticipate our needs and 
commands rather than reacting (with delays) to human 
commands. 

For further reading on the topic of multiple mobile 
robot systems, the reader is referred to survey articles 
in the field, including [53.2,288,373,374], Addition¬ 
ally, several special journal issues on this topic have 
appeared, including [53.1, 375-377]. Some taxonomies 
of multirobot systems are given in [53.26,288,378]. 
A variety of symposia and workshops have been held 
on a regular basis on the topic of multirobot systems, 
in particular the DARS (distributed autonomous robotic 
systems) series of symposia. Recent proceedings of 
these workshops and symposia include [53.379-388]. 
Additional texts on this topic include [53.389-391]. 
For some excellent further background on networked 
robotics we direct the reader to [53.31,35,46,52,55, 
75,90,104], 
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Pa rt F, Robots at Work, covers the advances in technol¬ 
ogy that are concerned with the growing area of robot 
applications, ranging from factory robotics, through 
a diverse array of industry applications such mining, 
agriculture, construction to health care and domestic 
robotics. The future vision for robotics is for the per¬ 
vasive application of robots. The robots of the future 
will perform all the dangerous, dirty, and dreary (DDD) 
tasks. Joe Engelberger, the pioneer of the robotics indus¬ 
try, wrote in his 1989 book Robotics in Service that the 
inspiration to write the book came as a reaction to a fore¬ 
cast study of robot applications, which predicted that 
in 1995 applications of robotics outside factories (the 
traditional domain of industrial robots) would account 
for less than 1% of total sales. Engelberger believed 
that this forecast was wrong, and he instead predicted 
that the non-industrial class of robot applications would 
become the largest class of robot applications. Engel- 
berger’s prediction has yet to come to pass. However, he 
did correctly foresee the growth in non-traditional appli¬ 
cations of robots. Previous parts of this Handbook show 
the great strides that robotics technology has made in 
the past 50 years. The technology has reached a level 
of maturity such that robots are now marching from the 
factories into held and service applications. 

The topics in Part F cover the essentials of what is 
required to create robots that can operate in all environ¬ 
ments and perform meaningful work. Part F describes 
ht-for-purpose robots and includes hardware design, 
control (of locomotion, manipulation, and interaction), 
perception, and user interfaces. The economic/social 
drivers for the particular applications are also dis¬ 
cussed. Part F builds on all of the previous parts of the 
Handbook. Robotics Foundations (Part A) and Design 
(Part B) are essential for providing the basic mech¬ 
anisms and control structures for any robot that is 
targeted for applications work. Sensing and Perception 
(Part C) and Manipulation and Interfaces (Part D) are 
critical capabilities for robots that need to interact with 
changing environments and perform manipulation tasks 
under human supervision. To fully utilize the potential 
of a working robot requires mobility and the ability to 
cooperate with other robotics, the basic technologies 
described in Moving in the Environment (Part E) are 
essential. 

Chapter 54, Industrial Robotics, gives a history 
and description of typical industrial robot applications. 
Most robots today can trace their origin to early indus¬ 
trial robot designs. All the important foundations for 
robot control were initially developed with industrial 
applications in mind. The chapter describes how robot 
with different mechanisms are designed to fit different 
applications. 


Chapter 55, Space Robots and Systems, deals with 
robotics technology for space applications. It covers 
two classes of robotics: orbital systems and opera¬ 
tion of planetary surfaces. Any unmanned spacecraft is 
a robotic spacecraft. However, space robots are more 
capable devices that can facilitate manipulation, assem¬ 
bly, assisting astronauts, and extending exploration to 
remote planets as surrogates for human explorers. This 
chapter covers the key issues in space robots and sys¬ 
tems: manipulation, mobility, sensing and perception, 
tele-operation and autonomy, and dealing with extreme 
environments. A historical overview and a review of the 
most recent technical advances for space robots is pro¬ 
vided. An extensive description of the mathematics for 
the control of robotic devices in microgravity environ¬ 
ments is also provided. 

Chapter 56, Robotics in Agriculture and Forestry, 
describes how robotics has made a impact in the agri¬ 
cultural and forestry industries. Agriculture is an in¬ 
dustry that is driven by productivity. Growing human 
population drives ever-increasing demands for greater 
food production. Robotics is a productivity technology 
that can be game changer in agriculture. The chapter fo¬ 
cuses on case studies where robotics has made a marked 
difference to agriculture and forestry. The case stud¬ 
ies cover the use of robotics in field crops, weeding, 
seeding, irrigation, orchard crops - fruit and vegeta¬ 
bles, forestry, and livestock applications - breeding, 
harvesting, slaughter and processing. The case stud¬ 
ies follow the progression of robotics technology from 
sensing through mobility to manipulation. The chapter 
emphasizes the agricultural field as a fruitful source for 
robotics applications, that are sufficiently demanding to 
require the development of new techniques, methods 
and technologies. 

Chapter 57, Robotics in Construction, describes the 
construction automation concepts that have been devel¬ 
oped and presents examples of construction robots that 
are in use and are in various stages of development. 
The chapter gives an overview of the construction in¬ 
dustry and discusses the concept of automation versus 
robotics. The chapter discusses robotics in applications 
that are on construction sites. The chapter deals with the 
growing number robotics and automation application of 
offsite construction tasks such as prefabrication. Un¬ 
solved technical challenges including interoperability, 
connection systems, tolerances, power, and communi¬ 
cations are discussed. 

Chapter 58, Robotics in Hazardous Applications, 
discusses robot technologies for dealing with difficult 
and dangerous environments. The technology solutions 
that are adopted depend on the nature and magnitude of 
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the hazards. When the magnitudes of hazards reach the 
point that human exposure represents a direct threat to 
life or has long-term health consequences, e.g., nuclear 
radiation, some form of remote operation must be used. 
This chapter describes robotics technologies for dealing 
with eradicating landmines, and hazardous materials 
handling, such as explosives disposal, and handling of 
dangerous biological and nuclear material. Enabling 
technologies to support demining and hazardous mate¬ 
rial handling are discussed, including mobile platforms, 
manipulator design, teleoperation and control, and the 
supply of energy and communication signals through 
reliable and robust tethers. The chapter ends with a dis¬ 
cussion of the advances that are required to progress the 
field. Progress is needed in mechatronics design, sens¬ 
ing, machine intelligence, and fully understanding the 
application and its associated economic factors. 

Chapter 59, Mining Robotics, discusses the ap¬ 
plication of robotics to the mining industry. Mining 
remains physically demanding and hazardous work. 
The scope for the application of robotics is great since 
mining requires the handling of enormous quantities 
of material in a cost-effective and safe manner. High 
operational costs, the need for greater productivity, 
and improved health and safety outcomes are powerful 
drivers for robotics. Robotics and automation is pro¬ 
viding the next step change in productivity and safety. 
This chapter reviews the modern mining practice and 
the technology drivers. It describes the enabling robot 
technologies associated with surface mining and un¬ 
derground mining. The chapter covers the technologies 
associated with digging, drilling, explosives handling, 
haulage, truck fleet management, autonomous mapping 
of mines and robotic rescue for recover in mining acci¬ 
dents. The chapter considers the challenges associated 
with mining in extreme environments such as undersea 
extraction of resources and the long-term possibilities 
of interplanetary exploration and mining. 

Chapter 60, Disaster Robotics, describes the state 
of the art of an emerging applications field. In recent 
times robots have served as extensions of responders to 
a disaster, providing real-time video and other sensory 
data about the situation. The technology is still emerg¬ 
ing and is beginning to be adopted by the international 
emergency response community. Robots are being used 
by fire departments and for bomb disposal. This chap¬ 
ter describes the basic nature of disasters, the impact on 
robot design, the types of robots actually used in disas¬ 
ters such as the Fukushima Nuclear disaster, promising 
designs, future concepts and benchmarks for evaluation 
of disaster robotics. The chapter discusses the funda¬ 
mental challenges facing disaster robotics, including 


the key challenge of evolution from a concept to the 
adoption as a solution technology. 

Chapter 61, Robot Surveillance and Security in¬ 
troduces surveillance and security robots for military 
and civil applications. The chapter describes the ap¬ 
plications environment that covers ground, aerial and 
maritime domains for a wide range of surveillance and 
security robots. A systems overview is provided of the 
mobility component, sensor payload, communication 
system, and operator control interface, together with 
a description of the enabling technologies. The chap¬ 
ter concludes with a review of current research topics 
and discusses the future direction of robotic surveil¬ 
lance and security. 

Chapter 62, Intelligent Vehicles, describes the 
emerging robotics application field of intelligent vehi¬ 
cles: motor vehicles that have autonomous functions 
and capabilities. The chapter describes why the devel¬ 
opment of intelligent vehicles is important and gives 
a brief history of the field and the potential benefits 
of the technology. It describes the enabling technolo¬ 
gies for intelligent vehicles to sense vehicle, environ¬ 
ment, and driver state, work with digital maps and 
satellite navigation, and communicating with intelligent 
transportation infrastructure. The chapter describes the 
challenges and solutions associated with road scene un¬ 
derstanding. Advanced driver assistance systems which 
use robotics technologies to create new safety and con¬ 
venience systems for motor vehicles, such as collision 
avoidance, lane keeping, parking assistance, as well 
as driver monitoring to mitigate against driver fatigue, 
inattention, and impairment are described. 

Chapter 63, Medical Robotics and Computer- 
Integrated Surgery describes the development of med¬ 
ical robotics. In the last 20 years starting with brain 
surgery, orthopaedics, endoscopic surgery, and micro¬ 
surgery, the field has expanded to include commer¬ 
cially marketed, clinically deployed systems and an 
active research community. This chapter provides a his¬ 
torical review of the field and discusses the major 
thrusts using examples from current and past research. 
Medical robots are described within the context of 
larger computer-integrated systems including presur- 
gical planning, intraoperative execution, and post¬ 
operative assessment and follow-up. The basic concepts 
of computer-integrated surgery including the critical 
factors affecting the deployment and acceptance of 
medical robots are described. An overview of medical 
robot systems is provided, including remote telesurgery 
and robotic surgical simulators. The chapter concludes 
with a discussion on future research directions. 
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Chapter 64, Rehabilitation and Health Care Robot¬ 
ics, describes robotic systems that assist persons who 
have a disability or provide rehabilitation therapy for 
persons. The chapter provides a historical review of the 
field. Physical therapy and training robots as well as 
robotic aids for people with disabilities are described. 
Advances in smart prostheses and orthoses for rehabili¬ 
tation robotics are also described. An overview of work 
in diagnosis and monitoring for rehabilitation is pro¬ 
vided. The chapter provides a solid understanding of 
the future challenges for rehabilitation and heath care 
robotics. 

Chapter 65, Domestic Robots, describes the tech¬ 
nology that everyone would one day like in their home, 
which will vacuum the house, clean the kitchen, load 
the dishwasher, or polish the shoes. In spite of hun¬ 
dreds of millions of potential users surprisingly not 
many domestic robots exist, with one notable excep¬ 
tion - the Roomba! This chapter examines what have 
been the success factors for commercializing domestic 
robots using cleaning robots as a representative ex¬ 
ample. The chapter reviews domestic floor cleaning 
robots, robotic pool cleaners, window cleaning robots, 
and robotic lawn mowing. The chapter also examines 


smart appliances such as sports robotics, and telepres¬ 
ence robots. A description of research projects for smart 
environments and smart homes is presented. The chap¬ 
ter describes the important underpinning technologies 
for domestic robots covering sensing, obstacle avoid¬ 
ance, localization, mapping and coverage. 

Chapter 66, Robotics Competitions and Chal¬ 
lenges, explores the use of competitions to accelerate 
robotics research and to promote education. Robot 
competitions have been shown to drive innovation in 
the robotics field. The chapter covers the two broad 
types of robot competition: human-inspired competi¬ 
tions and task based challenges. Human-inspired robot 
competitions, which are mostly sports contests, develop 
platforms that support problem solving. Task based 
challenges attract participants by presenting a tough 
new challenge for robotics technology. Three case stud¬ 
ies of robot competitions are presented Robot Soc¬ 
cer, the Unmanned Aerial Vehicles (UAV) Challenge 
and the DARPA (Defense Advanced Research Projects 
Agency) Grand Challenges. The case studies describe 
the organizational challenges for participants, the bene¬ 
fits and limitations of competitions and what it takes to 
have a good robot competition. 



1385 


Multimedia Contents 



54. Industrial Robotics 


Martin Hagele, Klas Nilsson, J. Norberto Pires, Rainer Bischoff 


Much of the technology that makes robots reli¬ 
able, human friendly, and adaptablefor numerous 
applications has emerged from manufacturers of 
industrial robots. With an estimated installation 
base in 2014 of about 1.5 million units, some 
171 000 new installations in that year and an 
annual turnover of the robotics industry estimated 
to be US$ 32 billion, industrial robots are by far the 
largest commercial application of robotics tech¬ 
nology today. 

The foundations for robot motion planning 
and control were initially developed with in¬ 
dustrial applications in mind. These applications 
deserve special attention in order to understand 
the origin of robotics science and to appreciate 
the many unsolved problems that still prevent 
the wider use of robots in today's agile manufac¬ 
turing environments. In this chapter, we present 
a brief history and descriptions of typical indus¬ 
trial robotics applications and at the same time we 
address current critical state-of-the-art techno¬ 
logical developments. We show how robots with 
different mechanisms fit different applications and 
how applications are further enabled by latest 
technologies, often adopted from technological 
fields outside manufacturing automation. 

We will first present a brief historical intro¬ 
duction to industrial robotics with a selection of 
contemporary application examples which at the 
same time refer to a critical key technology. Then, 
the basic principles that are used in industrial 
robotics and a review of programming methods 


54.1 Industrial Robotics: The Main Driver 

for Robotics Research and Application . 1386 

54.2 A Short History of Industrial Robots . 1386 

54.3 Industrial Robot Kinematics . 1392 

54.4 Typical Industrial Robot Applications .... 1393 

54.4.1 Handling . 1393 

54.4.2 Welding . 1396 

54.4.3 Assembly . 1398 

54.4.4 Painting . 1402 

54.4.5 Processing . 1402 

54.5 Safe Human-Robot Collaboration . 1405 

54.5.1 Overview of Basic 

Robot Safety Standards . 1405 

54.5.2 Types and Requirements 

of Human-Robot Collaboration .. 1406 

54.5.3 Examples of Human-Robot 

Collaboration . 1408 

54.6 Task Descriptions - 

Teaching and Programming . 1409 

54.7 System Integration . 1414 

54.8 Outlook and Long-Term Challenges . 1416 

Video-References . 1418 

References . 1418 


will be outlined. We will also introduce the topic 
of system integration particularly from a data in¬ 
tegration point of view. The chapter will be closed 
with an outlook based on a presentation of some 
unsolved problems that currently inhibit wider use 
of industrial robots. 
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54.1 Industrial Robotics: The Main Driver for Robotics Research 
and Application 


Even though robots are considered a cornerstone of 
today’s competitive manufacturing particularly in auto¬ 
mobile and related component assembly, there are still 
challenges to solve for manufacturing to efficiently re¬ 
spond to changing consumer behavior and global shifts 
in competitiveness. Furthermore, high-growth indus¬ 
tries (in electronics, food, logistics, and life-sciences) 
and emerging manufacturing processes (gluing, coat¬ 
ing, laser-based processes, precision assembly, fiber 
material processing) as well as fulfilling sustainabil¬ 
ity regulations will increasingly depend on advanced 
robot technology [54.1]. Additionally, the range of fea¬ 
sible applications could significantly increase if robots 
were easier to install, to integrate with other manu¬ 
facturing processes, and to program, particularly with 
adaptive sensing and automatic error recovery. Further 
challenges result from the integration of various types 
of controls (programmable logic controller (PLC), com¬ 
puter numerical control (CNC) sensors) with the robot 
controller, from close human-robot collaboration and 
fenceless production with both lightweight and heavy 
duty robots, and from an increasing need to save 
energy. 

Design and production of industrial robots on the 
one hand, and the planning, integration, and operation 
of robot work cells on the other hand are largely in¬ 
dependent engineering tasks. In order to be produced 
in sufficiently large quantities, a robot design should 
meet the requirements for the widest set of potential 
applications. As this is difficult to achieve in practice, 
various classes of robot designs regarding payload ca¬ 
pacity, number of robot axes, and workspace volume 
have emerged for application categories such as as¬ 
sembly, palletizing, painting, welding, machining, and 
general handling tasks. 


Generally, a robot workcell consists of one or more 
robots with controllers and so-called robot peripherals, 
e.g., grippers or tools, safety devices, sensors, and ma¬ 
terial transfer components for moving and presenting 
parts. Typically, the cost of a complete robot workcell 
is four to five times the cost of the robots alone; how¬ 
ever, efforts are underway to drastically reduce these 
costs through use of increased robot functionality and 
artificial intelligence [54.2]. A robot workcell is usually 
the result of customized planning, integration, program¬ 
ming, and configuration, requiring significant engineer¬ 
ing expertise. Standardized engineering methods, tools, 
and best-practice examples for specifying and designing 
robot workcells have become available to provide pre¬ 
dictable performance and to secure investments [54.3]. 

Today’s industrial robots are mainly rooted in the 
requirements of capital-intensive large-volume manu¬ 
facturing, typically defined by the automotive, electron¬ 
ics, and electrical goods industries which make up 80% 
of all robot installations. Future industrial robots will 
not be a mere extrapolation of today’s designs, but will 
rather follow new design principles addressing a much 
wider range of application areas and industries. At the 
same time, new technologies, particularly from the in¬ 
formation technology (IT) or the consumer domain will 
have an increasing impact on the design, performance, 
use and cost of future industrial robots. 

International and national standards now help to 
quantify robot performance and define safety precau¬ 
tions, geometry, and media interfaces [54.4], Most 
robots operate behind secure barriers to keep people 
at a safe distance. Recently, improved safety standards 
have allowed direct human-robot collaboration, permit¬ 
ting robots and human factory workers to share the 
same workspace [54.5, 6]. 


54.2 A Short History of Industrial Robots 


The invention of the industrial robot dates back to 
1954 when inventor George Devol filed a patent on 
a programmed article transfer (Fig. 54.1). After team¬ 
ing up with young engineer and entrepreneur Joseph 
Engelberger, the first robot company, Unimation, was 
founded. It put the first robot into service at a Gen¬ 
eral Motors plant in 1961 for extracting parts from 
a die-casting machine. Most of the hydraulically actu¬ 
ated Unimates were sold through the following years 
for workpiece handling and for spot-welding of car 


bodies [54.7]. Soon, many other companies started to 
develop and manufacture industrial robots in many 
industrial nations; an innovation-driven industry was 
born [54.8]. The first International Symposium on In¬ 
dustrial Robotics (now ISR) took place in Chicago in 
1970 and proved that robotics had become the field of 
activity of a vibrant research community. 

The breakthrough Stanford Arm was designed as 
a research prototype in 1969 by Victor Scheinman 
(Chap. 4). The six-degree-of-freedom (6-DOF) all- 
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Fig.54.1a,b The invention of the industrial robot, (a) This patent was the start of a joint effort between G. Devol and 
J. Engelberger to form the first robot company, Unimation, a fusion of the terms universal and automation. The company 
was acquired by Westinghouse in the late 1980s and subsequently taken up by Staubli in 1988. (b) The first Unimation 
performed a rather simple handling task in 1961 at a General Motors plant; other car manufacturers followed (courtesy 
of Smithsonian Institution Archives, Washington DC) 




Fig.54.2a,b The all-electric (a) IRB-6 and (b) a SCARA-type kinematic, (a) First introduced in 1973, the IRB-6 has 
been a breakthrough development as it was the first serially produced robot product, which combined all-electric-drives 
technology and a microcomputer for motion control and programming. The robot proved very robust, and life-times of 
more than 25 years in harsh productions were reported (courtesy of ABB Automation), (b) The selective compliance 
assembly robot arm (SCARA) is particularly suited for assembly tasks as it combines rigidity in the vertical axis and 
compliance in the horizontal axis. In 1978, the first Hirata AR-300 was put together. Depicted is the successor design, 
the AR-i350 (courtesy of HIRATA Robotics, Mainz) 
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Fig.54.3a,b The KUKA IR 601/60 (a) and the Unimation 
PUMA (programmable universal machine for assembly) 
560 (b). (a) In 1978, the novel 6 DOF KUKA robot fea¬ 
tured a parallel linkage for its second and third axes. At 
almost two tons of own weight, it could handle payloads 
of some 60 kg at maximum operating speed. The robot 
quickly became a workhorse for the automotive industry, 
(b) The six axis PUMA was inspired by the dexterity of 
a human arm. After its launch in 1979 by Unimation, it 
became one of the most popular arms and was used, due 
to its versatility and ease of use, as a reference in robotics 
research for many years 

electric manipulator was controlled by a state-of-the-art 
computer of the time, a DEC PDP-6. The nonanthro- 
pomorphic kinematic configuration with one prismatic 
and five rotational joints was configured such that the 
equations for solving the robot kinematics were simple 
enough to speed up computations. Drives consisted of 
direct-current (DC) electric motors, harmonic drive and 
spur gear reducers, potentiometers and tachometers for 
position and velocity feedback [54.9]. Subsequent robot 
designs were strongly influenced by Scheinman’s con¬ 
cepts (Figs. 54.2 and 54.3). 


In 1973, the company ASEA (now ABB) intro¬ 
duced the first microcomputer-controlled all-electric 
industrial robot, the IRB-6, which allowed continuous 
path motion (CP), a precondition for many applications 
such as arc welding or material removal (Fig. 54.2). In 
the 1970s, intense diffusion of robots into car manu¬ 
facturing set in mostly for (spot-)welding and handling 
applications (Fig. 54.3) [54.10]. 

In 1978, the selective compliance assembly robot 
arm (SCARA) was invented by Makino of Yamanashi 
University, Japan [54.11]. The ground-breaking four- 
axis low-cost design was perfectly suited for small parts 
assembly as the kinematic configuration allows fast and 
compliant arm motion (Fig. 54.4). Flexible assembly 
systems based on the SCARA robot in conjunction with 
compatible product designs (DFA) have contributed sig¬ 
nificantly to creating a boom in high-volume electron¬ 
ics production and consumer products [54.12], Further 
optimization of robot dynamics and accuracy led to 
the first direct-drive SCARA robot, the AdeptOne in 
1984 [54.13]. 

Requirements regarding a robot’s speed, accuracy 
and weight have led to novel kinematic and transmis¬ 
sion designs. An approach toward lightweight and stiff 
structures has been pursued since the 1980s by develop¬ 
ing parallel kinematic machines (PKM) which connect 
the machine’s basis with its end-effector by three to six 
parallel struts, see also Fig. 54.5. These so-called par¬ 
allel robots (Chap. 4 and 18) are particularly suited to 
achieve short cycle times (e.g., for picking), precision 
(e.g., for material removal), or handling high workloads 
(Fig. 54.5) and have found their niches in advanced 
manufacturing [54.14]. However, workspace volumes 
tend to be significantly smaller than those of serial or 
open kinematic chain robots which are comparable in 
size. 

Efforts of reducing mass and inertia of serial robot 
structures have been a primary research target, where 
the human arm with a weight-to-load ratio better 
than 1 : 1 was considered the ultimate benchmark. In 
2006, robot manufacturer KUKA introduced their LBR 
lightweight prototype robot, a compact 7-DOF robot 
arm with advanced torque-control capabilities which 
has recently been introduced in high-performance in¬ 
dustrial applications [54.15], An obvious next step in 
approaching human dexterity is the recent introduc¬ 
tion of two-armed robot designs with some recent 
developments being depicted in Fig. 54.7 [54.16]. In 
conjunction with a robot’s capability to support safe 
human-robot collaboration, new manufacturing con¬ 
cepts can be implemented which expand capabilities, 
productivity, and ergonomic quality to manual work¬ 
places [54.17], 
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Fig. 54.4 An automated video cassette recorder (VCR) assembly line (about 1989) with SCARAs carrying a turret 
with multigripper tools. Typically five parts are added by one robot before the VCR is moved to the next station of the 
automated assembly line 



Fig.54.5a-c Parallel robots are slowly diffusing into various fields of industrial application: (a) the Neos Tricept 600, 
(b) Fanuc F-200iB. (c) Adept Quattro. (a) In 1992, Neos Robotics represented with their Tricept robot range a concept 
to combine the stiffness of machine tools with the dexterity of a robot for heavy-duty applications such as in friction 
stir welding (FSW) or machining of aluminum for the aircraft industry, (b) The Fanuc F-200iB introduced in 2002 is 
a 6-DOF parallel robot particularly designed for welding gun handling, deflashing, or for assembly tasks (100 kg payload, 
±0.1 mm accuracy) in automotive assembly processes; (c) the Adept Quattro (introduced in 2007, following the ABB 
FlexPicker in 1998) is suited for high-speed applications in packaging, manufacturing, assembly, and material handling. 
The quad dual-link arm design forms an over-determined kinematic linkage, which in the wrist is converted to the forth 
axis of end-effector rotation by means of an internal transmission in the wrist 


In parallel to industrial robots, automated guided 
vehicles (AGV) have emerged. These mobile robots 
are used for moving workpieces or loading equip¬ 
ment following a predetermined or virtual path in 
industrial environments. Within the concept of auto¬ 
mated flexible manufacturing systems (FMS) AGVs 
have become an important part of their routing flex¬ 
ibility. Initially, AGVs relied on prepared floors such 


as embedded wires, magnets, or other tags for mo¬ 
tion guidance. Meanwhile, freely navigating AGVs 
along virtual trajectories are entering large-scale man¬ 
ufacturing and logistics. Usually, their navigation is 
based on laser scanners that provide an accurate two- 
or even three-dimensional map of the actual environ¬ 
ment for self-localization and obstacle avoidance. Early 
on, combinations of AGVs and robot arms were sug- 
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Fig.54.6a,b Mobile robots were introduced in the early 1980s for increased flexibility and reliability in factory logistics, 
(a) The MORO (1984) developed at Fraunhofer IPA was one of the first prototypes to combine a robot arm on a wire- 
bound mobile platform which follows a wire buried in the floor, (b) The KUKA omniRob features an omnidirectional 
platform and the LBR iiwa lightweight arm which form a highly kinematically redundant robot system (courtesy of 
KUKA) 



Fig.54.7a-f Examples of different designs of dual-arm robots (courtesy of (a) Motoman, (b) ABB, (c) Rethink Robotics, 
(d) Kawada Industries, (e) COMAU, (f) Seiko Epson) 
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Fig.54.8a-d Statistics of worldwide industrial robotics use (after [54.1]). (a) Estimated annual robot installations in 
selected countries (1000 units, estimate for 2015), (b) Number of multipurpose industrial robots (all types) per 10000 
employees in the automotive and in manufacturing industries 2014. (c) Estimated worldwide annual shipments of indus¬ 
trial robots in main application areas, (d) Estimated worldwide annual shipments of industrial robots in main industrial 
branches 


gested to automatically load and unload machine tools 
(Fig. 54.6). Safety and power supply have been an ob¬ 
stacle to these system’s diffusion in industrial practice. 
Currently, first solutions for mobile manipulation ap¬ 
pear [54.18]. 

The ability to use human and robot workers either 
interchangeably or in workspace sharing/collaboration 
scenarios in human workplaces motivated the design 
of anthropomorphic dual-arm robots (Fig. 54.7. Even 
though industrial acceptance initially has been low, ad¬ 
vances in programming comfort, securing safe human- 
robot coexistence/collaboration and system cost have 
led to significant interest in using dual arms in ag¬ 
ile manufacturing concepts, particularly in assembly 
and handling applications [54.19]. The dual-arm sys¬ 


tems suggest a new way of using powerful and lean 
type of robot which is easy to install by the man¬ 
ufacturing end-user with little adaptation of manual 
workplaces. 

Today, industrial robotics is seen as a central pillar 
to future manufacturing competitiveness and economic 
growth: 

• The International Federation of Robotics (IFR) es¬ 
timates that between 2000 and 2008 the robotics 
industry had created 8—10 million highly qualified 
jobs, either directly or indirectly. The prediction 
is that between 2012—2020 another 4 million jobs 
will be created in the robot ecosystem [54.20]. The 
extent of job creation by robotics has been dis- 
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cussed controversially. It is undisputed, however, 
that a wider use of robots in manufacturing is 
able to significantly strengthen a competitive posi¬ 
tion of a company or an industrial sector [54.21], 
Economically, manufacturing productivity gains are 
particularly effective for economic growth. There is 
no sustainable product innovation without manufac¬ 
turing competence which includes knowledge and 
practice of planning, designing, and operating ad¬ 
vanced robotic systems [54.22] (Fig. 54.8). 

• The average price for a robot in 2014 was in the or¬ 
der of US$ 46 800, which is about one-third of its 
equivalent price in 1990. At the same time, robot 
performance parameters such as speed, load capac¬ 
ity, and mean time between failures (MTBF) have 
dramatically improved. This means that automation 
has become more affordable, providing a faster re¬ 
turn on investment [54.1], 

• Traditionally, robot automation has not played a sig¬ 
nificant role in the implementation of lean manu¬ 
facturing strategies. However, efforts are underway 
to introduce industrial robots to lean, agile man¬ 
ufacturing. Characteristics are robot solutions that 
can be flexibly added to manufacturing systems 
on demand, that are significantly less expensive 
on a life-cycle-costing (LCC) basis than today’s 
systems due to reduced peripherals and systems 
integration (system out of the box ) [54.23]. With 
robots becoming commodities in manufacturing 
they might be used as intuitively and naturally as 
a handheld power tool today. This would imply 
intuitive and safe human-robot collaboration and 

54.3 Industrial Robot Kinematics 

By definition, an industrial robot is an automatically 
controlled, reprogrammable, multipurpose manipula¬ 
tor, programmable in three or more axes, which can 
be either fixed in place or mobile for use in indus¬ 
trial automation applications [54.28], Robots can be 
categorized according to their number of independent 
kinematic axes, and their mechanical structure which 
affect most of the robots’ kinematic properties, the com¬ 
putation methods used to determine joint motions, and 
the form and size of the robot workspace. Robot me¬ 
chanical structures are composed of links that are rigid 
bodies connecting neighboring (prismatic, rotary, cylin¬ 
drical or spherical) joints. The diagrams in Table 54.1 
show several common types of robot mechanical struc¬ 
tures. Of course, the workspace of industrial robots can 
be significantly expanded by placing the robot arm on 
an additional linear axis, sometimes reaching a length 
of more than 50 m, or even on mobile platforms. Fur- 


versatility due to advanced sensing, control, and 
embedding the robot set-up and operation in an IT 
infrastructure. 

• Factories of the future will represent a network of 
self-organizing cyber physical systems (CPS). As 
part of this industrial internet CPSs embed compu¬ 
tation, networking, and physical processes and can 
either represent manufacturing equipment such as 
machine tools, fixtures, trays, conveyors, tools, etc. 
or the workpiece which controls and memorizes its 
production. Robots are considered the centerpiece 
of future smart factories which combine manufac¬ 
turing agility, profitability, human ergonomics, and 
minimized resource consumption [54.24]. 

• Robots in assembly have not reached their pre¬ 
dicted installation potential mainly due to cheap 
labor cost and the lean assembly work systems 
which support highest flexibility and productivity at 
minimal waste. A reason is partly seen in the slow 
advances in dexterous manipulation for assembly 
tasks with industrial robustness. Here, torque-con- 
trolled lightweight robots [54.25] and two-armed 
robot systems have been proposed to imitate human 
ergonomics and task execution [54.26]. 

• New financing models such as leasing, pay by ser¬ 
vice will allow end-users to use robots on demand 
or to have manufacturing service providers to op¬ 
erate manufacturing lines on a pay-on-production 
basis [54.27], 

Figure 54.8 depicts some key figures on the recent 

extent of industrial robot diffusion into manufacturing. 


thermore, robot mechanical structures can be composed 
by joint modules which are connected by links to form 
task-specific designs. 

With advances in the state of the art in motion 
control and computer hardware processing capabilities, 
computation is much less a constraint on mechanism 
choice than it was for early robot designers. The choice 
of mechanical structure of the robot depends mostly on 
fundamental mechanical requirements such as payload 
and workspace size. Considering a given level of cost, 
there is usually a trade-off between workspace size and 
stiffness. To enable the robot to reach inside or around 
obstacles it is clearly advantageous to use an articulated 
mechanical design. 

Considering also the stiffness and accuracy (in 
a practical sense considering what is reasonable to 
build), the picture is more complex. Each of the first 
three types in Table 54.1 we refer to as serial kinematic 
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Table 54.1 Main categories of mechanical structures of industrial robots: Gantry is what a Cartesian coordinate robot is 
typically called with three prismatic joints, whose axes are coincident with a Cartesian coordinate system. The SCARA 
or selective compliance assembly robot arm has two parallel rotary joints to provide compliance in a selected plane. 
The articulated robot has three or more, typically six, rotary joints placed in series with their interconnecting links. The 
parallel link robot is characterized by links that form closed loop structures shown with prismatic joints, but can also have 
revolute joints such as the Delta robot (Fig. 54.5) (pictures courtesy of Giidel, ADEPT, ABB, PI Physik Instrumente) 


Category 

Robot main 
axes structure 


Workspace 

shape 


Technical 

example 


Gantry (or Cartesian) 




SCARA 


CiA 1 - 13 



1 prismatic and 
2(3) revolute joints 



I- 



Articulate Parallel 



prismatic axis 



machines (SKMs), while the last is a parallel kinematic 
machine (PKM). To obtain maximum stiffness, again 
for a certain minimum level of cost, the end-effector is 
better supported from different directions, and here the 
PKM has significant advantages. On the other hand, if 
high stiffness (but not low weight and high dexterity) 


is the main concern, a typical computerized numerical 
control (CNC) machine (e.g., for milling) is identical 
in principle to the gantry mechanism. There are also 
modular systems with servo-controlled actuators that 
can be used to build both robots with purpose-designed 
mechanisms. 


54.4 Typical Industrial Robot Applications 


Out of the many possible uses of industrial robots se¬ 
lected case studies on high-potential robot applications 
will be briefly described. Typical associated enabling 
technologies will be depicted. 

54.4.1 Handling 

Handling in robotics comprises numerous processes 
such as grasping, transporting, packaging, palletizing, 
and picking. As seen in Fig. 54.8 handling is the largest 


robot application field which is found in all branches 
of manufacturing and logistics. A central feature and 
major challenge in the engineering of robotic handling 
systems is the design of the gripper and associated 
grasping strategies given the physical properties of 
the workpiece, throughputs, and uncertainties regarding 
object geometry and location. Current high-potential 
application of robot handling systems are: tending of 
CNC machines for workerless shifts [54.29], palletiz¬ 
ing, and lifting of objects for ergonomic reasons or 
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Fig.54.9a-d Units of sausage are cut from strings, then placed into the thermo-formed cavities before applying lidding. 
The coordination of the robots and the optimization of the picking frequency require a selection of the best path for each 
robot. Missed sausages are fed back on the conveyor for another try. The shown 4-DOF parallel robot reaches cycle times 
of 1—3 Hz and can move payloads of up to 8 kg (courtesy of robomotion, Germany) 



Fig.54.10a-d Lightweight customized vacuum gripper (0.75 kg mass) through additive manufacturing. Cookies are 
delivered continuously on a belt, grasped from the belt, and batches of eight are put on a blister matrix before final 
packaging. The gripper’s spacing is pneumatically actuated and its rotation through the parallel robot’s central rotational 
axis (courtesy of robomotion, Germany) 


when limitations specified in load handling regulations 
are exceeded [54.30], for reasons of cleanliness as is 
typical in the food, pharmaceutical, and semiconduc¬ 
tor industries [54.31-33], avoiding monotonous work 
and psychological strain, and ensure logistics quality 
through workpiece or object tracking [54.34]. In the 
following, two use-cases of material handling will be 
highlighted, each in a different industrial domain, and 
based on specific industrial robot type and enabling 
technologies. 

Food Handling 

The food sector is claimed to have significant po¬ 
tential for the application of robots as fundamental 
change to productivity, product quality, and worker er¬ 
gonomics can be achieved [54.35]. In food automation, 
untouched by human hand entails critical requirements 
for robot automation such as the need for hygienic 
design, operational speed, ease of programming, and 
cost. In the past, these requirements had been difficult 
to achieve due to high throughputs, therefore requir¬ 
ing rapid grasps and fast robot motion, robust sensing 
for detecting object locations on conveyor belts. High 
speed at high flexibility apparently is a key in the indi¬ 


vidual handling of food objects. Therefore, fast SCARA 
and parallel robots have found wide acceptance in this 
field. 

An example of a packaging line in food produc¬ 
tion is depicted in Fig. 54.9 where cut mini salamis 
are delivered in four streams per conveyor belt in ran¬ 
dom sequence. The positions of the sausages on the 
translucent belt are determined by a computer vision 
system. The robot picks the sausage from the mea¬ 
sured position sequentially until the gripper holds three 
sausages which are then placed into cavities. With four 
parallel robots a maximum pick rate of 600 sausages 
per minute can be processed. Key of the application is 
its high-speed 2-D (two-dimensional) computer vision 
system which feeds the robot’s path planning for col¬ 
lision-free picks at a minimum loss rate of unpicked 
salami [54.37]. 

Recent efforts have led to customized designs of 
gripper systems through 3-D (three-dimensional) print¬ 
ing (additive manufacturing) which for instance in¬ 
cludes actuation through pneumatically driven bellows 
and low-wear metal joints. An example of a highly ac¬ 
tuated gripper based on 3-D-printing for use in food 
handling is depicted in Fig. 54.10. Additive manufac- 
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Fig.54.11a,b Procedure of a bin-picking method [54.36] (a) and gripper with additional degree of freedom for reaching 
deep into bins (b). Depicted is a 2-D laser scanner on a swiveling unit for acquiring the point cloud in parallel to 
the robot’s motion. The object detection itself consumes 0.5—2 s and is less time critical than the robot's motion and 
grasps 


turing processes seem to be perfectly suited to achieve 
higher flexibility in manufacturing automation [54.38]. 
Numerous materials have become available for dif¬ 
ferent additive manufacturing processes so that even 
specific manufacturing requirements can be matched. 
Initial doubts about gripper durability have been dis¬ 
pelled: lifetimes of more than 10 million load cycles for 
robot grippers manufactured on the basis of laser sin¬ 
tered polyamide have been reported. 

Bin-Picking 

Generally, industrial practice in robot workcell plan¬ 
ning aims at finding a compromise between reducing 
the variation of the workpiece location and the cost 
of sensor systems to compensate for residual variation 
or uncertainties. Today, nearly all parts arrive at robot 
workcells in a repeatable manner, either being stored in 
special carriers or magazines, or by being transported 
and oriented by vibrating devices that allow the parts 
to settle into a predictable orientation for proper robot 
grasping. However, cost and flexibility requirements in 
manufacturing automation will result in reducing cus¬ 
tomized parts magazines to more universal carriers, 
containers, or conveyor belts. If randomly oriented on 
a conveyor belt or in a carrier, parts have to be properly 
identified and located so that the robot can produce an 
collision-free grasp. 

The challenge of grasping partly or randomly or¬ 
dered parts by robot has been referred to as bin¬ 
picking and has been investigated by numerous re¬ 


searchers since the mid-1980s [54.39]. Even though an 
abundance of approaches has been presented, only re¬ 
cently bin-picking installations have found their way 
into daily manufacturing in significant numbers. Bin¬ 
picking algorithms follow a typical sequence of steps: 
initial point cloud data acquisition, object detection, 
pose estimation, collision-free path and grasp plan¬ 
ning, object grasping, and object placement. Most 
methods in bin-picking assume known geometrical 
representations (a computer-aided design CAD-model) 
of the workpiece in question including the specifi¬ 
cation of admissible grasps for applying template¬ 
matching methods [54.40,41]. Figure 54.11 depicts 
a variant of a fast template-matching method, which 
encompasses the following steps for detecting object 
poses: 

• For detecting the scene (e.g., a carrier or box filled 
randomly with workpieces) typically laser-based 
sensor systems are used for acquiring a sufficiently 
dense point cloud. The object pose detection then 
is considered as a combinatorial optimization prob¬ 
lem for which a construction heuristic is applied. 
For this heuristic tree search, a finite set of possible 
workpiece poses is initially derived from the search 
space. 

• In order to use a decision tree, the elements of the 
search set are split into two components: The first 
component describes a point of interest (POI) in 
the search space which is part of the workpiece 
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Fig.54.12a,b Car production (a) usually follows the illustrated steps along the assembly line: Stamping of the metal 
sheet into plates, fixing and alignment of the plates on trays, spot welding, painting the car body, and final assembly of 
the car body (doors, dashboard, windscreens, power-train seats, and tires). Car factories can host over 1000 industrial 
robots working typically three shifts per day (courtesy of PSA Peugeot Citroen, Paris and Art Movies, Paris). The Audi 
plant in Ingolstadt Germany (b) is highly automated. The picture shows spot welding robots along the body-in-white 
transfer line. Trays carrying car bodies pass through the robot garden 


surface. The second component describes possi¬ 
ble workpiece poses relative to a POI. The partial 
search quantities obtained thereby have a significant 
lower complexity compared to the original search 
set, since the points of interest can provide a con¬ 
straint on relative workpiece poses, thus restricting 
their assumed freedom of movement. 

• Typical tree search strategies such as best-first 
search can be used. In that case best-first search 
explores the search tree by expanding the most 
promising nodes first. These nodes are chosen 
according to a heuristic evaluation score, repre¬ 
senting the estimated distance from the node to 
a solution. 

• Final evaluation of the workpiece poses is pro¬ 
vided by a six-dimensional (6-D) Hough voting 
procedure, i. e., a generalized Hough transform. The 
features used for Hough voting are sensor measure¬ 
ments located relative to a POI. For all possible 
constellations of a sensor measurement relative to 
a point of interest, a probabilistic statement about 
possible workpiece poses can be made. Through 
the superposition of all probability statements, solu¬ 
tion candidates can be formed, which are subjected 
to a statistical test based on a quality rating. The 
obtained quality rating along with a given level of 
significance is used in order to decide about the ac¬ 
ceptance of a workpiece pose. 

The method is able to locate three to four work- 
pieces on average within 0.5—2 s, using a stan¬ 
dard desktop computer. Moreover, robot grippers are 
equipped with a seventh axis to allow grasping parts 


from corners of the bin. Furthermore, grippers should 
be formed in such a way that they may reach deep 
into the bin so that they offer only minimal collision 
volumes. 

54.4.2 Welding 

Welding is a manufacturing process that joins materi¬ 
als by applying heat, sometimes with pressure. Usually, 
workpiece material is melted at the process location of¬ 
ten with additional filler material. Typical robot-based 
welding processes are spot welding, particularly in 
car body assembly, and gas-shielded metal arc weld¬ 
ing (GMAW). With increasing compactness of laser 
sources and robot motion accuracy, laser welding is in 
the rise. 

Manual welding requires skilled workers, as small 
imperfections in the weld can lead to severe conse¬ 
quences. Furthermore, welders are exposed to haz¬ 
ardous working conditions (fumes, problematic er¬ 
gonomic working positions, heat, and noise) so that 
the use of robots has become beneficial in GMAW 
processes even for the smallest lot-sizes. Commonly, 
the automatic arc-welding process is based on a con¬ 
sumable wire electrode and a shielding gas that is fed 
through a welding gun. Modem welding robots are 
particularly suited through the following characteris¬ 
tics: 

• Computer control allows efficient programming of 
task sequences, robot motions, external actuators, 
sensors, and communication with external devices 
such as welding sources. 
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Fig.54.13a-c Offline programming of a spot welding workcell, (a) The robot workcell and the task execution are mod¬ 
eled on the basis of realistic robot models (geometry, kinematics, kinetics). (b) The shown laser tracker is a portable 
measurement system that relies on laser beams to accurately measure in a radial volume (accuracy of ±10 ppm = 
lOpm/m, up to 80 m in diameter, measuring rate up to 3000 points/s). If measured objects cannot be equipped with 
reflecting targets or reached by the tracker, handheld probes are tracked instead, (c) A tracker in use for interactively 
measuring the geometry of the robot workcell (courtesy of Leica, now Hexagon MI) 



Fig.54.14a-d GMAW welding of building trusses in lot-size one by robot. Illustration (a) shows the CAD drawing of a steel 
truss with relevant information for welding process, (b) one-half of the welding workcell with two welding robots working 
simultaneously on the truss when the neighboring truss on the other half is loaded or unloaded, (c) the laser-based seam finding 
and tracking sensors and (d) the welding robot (courtesy of Servo Robot, Canada; Goldbeck, Germany) 


• Free definition and parameterization of robot posi¬ 
tions or orientations, reference frames, and paths. 

• High repeatability and positioning accuracy of 
paths. Typically repeatability is some ±0.05 mm 
and positioning accuracy is better than ±1.0 mm. 
These values can be significantly improved through 
modern robot calibration methods [54.42]. 

• High speeds of the end-effector of up to 8m/s for 
quick approach and depart motions. 

• Typically, articulated robots have six DOF so that 
commanded orientations and positions in their 
workspace can be reached, which in the welding 
case means there is one DOF free for rotation 
around a rotational-symmetric welding tool. Ad¬ 
ditionally, workspace extensions by mounting the 
robot on a linear axis (seventh DOF) or even on 
mobile bases are common, especially for welding 
of large structures. 

• Typical payloads range 6—150 kg. Higher load ca¬ 
pacities are required for spot-welding guns (typi¬ 
cally > 50 kg) and their cable package. 

• Programmable logic controller (PLC) capabilities 
such that fast input/output control and synchroniz¬ 
ing actions within the robot workcell are accom¬ 
plished. 


• Interfacing to high-level factory control through 
factory communication networks. 

Electric current sources, torches, and peripheral 
devices for automatic cleaning and maintaining the 
GMAW torch (anti-splatter, wire-cutting, tool changer, 
etc.) are offered by specialized companies. Often sen¬ 
sors are used to track welding gaps and measure weld 
seams either before or synchronously with the weld¬ 
ing process, thus adapting the robot’s trajectory in the 
presence of workpiece variation and distortion. Also, 
collaborating robots have been introduced where one 
robot fixes and moves the workpiece in synchronization 
with another robot carrying a welding tool so that the 
weld can be performed with the pool of molten metal 
horizontal. 

Spot Welding in the Automotive Industry 
with Offline Programming 
Car manufacturing has been one of the key drivers in the 
technical development of industrial robots as the preci¬ 
sion handling of spot-welding guns was one of the first 
breakthrough use cases (Fig. 54. 12) body-in-white (i. e., 
unpainted car body) assembly is mostly done by robots, 
very much in contrast to the final assembly which is 
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Fig. 54.15 A robot-guided tool 
for handling and processing limb 
material, in this case a self-adhesive 
seal for car bodies 


dominated by manual work. Demands for faster cycle 
times have led to a concurrent and coordinated motion 
of the spot-welding gun and robot: the robot continues 
to move while the weld gun is simultaneously rotated 
about the electrode axis during welding [54.43]. 

Most of a spot welding robot’s programming 
is done using offline programming (OLP) packages 
(Fig. 54.13. A library of robots, devices, and ad¬ 
vanced CAD capabilities helps plan, program, vi¬ 
sualize, and optimize layouts and complete produc¬ 
tion cycles under assumed manufacturing conditions. 
Robot programs can be generated and downloaded 
to robots workcells. A critical step is the calibration 
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Fig.54.16a,b Design of the KUKA iiwa: (a) Shape and (b) integra¬ 
tion of joint mechatronics 


of the robot workcell with respect to the simula¬ 
tions [54.44]. 

Arc Welding in Metal Construction 
Normally steel constructions are designed using CAD 
programs that offer functions for GMAW-task defi¬ 
nitions such as welding parameters, multipass seams, 
weld beads sequencing, etc. This information may be 
used for automatically generating welding robot pro¬ 
grams, even in the case of lot-size one jobs. 

As an example, the generation principle of a weld¬ 
ing robot program is depicted in Fig. 54.14. Large- 
scale trusses of up to 15 m for large halls are welded- 
to-measure. The robot program is generated from the 
CAD drawing with process relevant information. Work- 
piece tolerances for example, induced by placing the 
steel components into the fixtures, by bending of the 
material under its own weight are compensated through 
active measurement. The robot-mounted sensor locates 
the weld seam by laser-based vision for shifting the 
generated programs in such a way that they match the 
actual weld seams. This calibration is automatically 
performed if expected and actual bead locations are 
within a range of ±2.5 cm. 

54.4.3 Assembly 

Assembly in manufacturing describes the combination 
of subsystems or components to systems of a higher 
complexity through joining. Assembly in manufactur¬ 
ing comprises four process groups: joining, handling, 
controlling, and auxiliary processes (cleaning, adjust¬ 
ment, marking, etc.) [54.45]. The composition of these 
four functions may vary depending on batch size, 
product, and throughput: from assembly workcells to 
high-throughput assembly lines. Assembly processes 
form up to 80% of a product’s manufacturing cost 
and this is where the greatest competitive advantage 
can be gained [54.46]. Therefore, optimization in as¬ 
sembly includes tightly interweaved aspects: Design 
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Table 54.2 Assembly subprocesses or modules and their implementation on the KUKA LBR iiwa 


Assembly 

subprocess 

Characterization 

Principle 

search', search 

Search motion type examples: 

Lissajous (force 

Linear Defined zigzag 


module which 
supports several 
search motions or 
search strategies 


peg in hole: 
execution of 
typical part 
joining motions 


gear, search : 
toothed gear 
joining motion 
and screw-in 
motion 


Linear 

Zigzag 

Spiral 

Sinus 

Lissajous 


driven motion) 


Commanding the search motion generation: 

• Position-based trajectory 

• Force-based trajectory 

• Combination of position and force-based trajectory 


Reduction of arbitrary joining part types to three ab¬ 
stract planar types: 

• Round (arbitrary axial orientation) 

• Triangular (defined axial orientation) 

• Rectangular (additionally defined workpiece coordi¬ 
nation system) 

Strategy of triangular types is common for numerous 
workpiece contours: 

• Orientation is given, tipping is executed in one step; 
object pivots around one corner 




yr 



Parts T and P of 
joining motion with 
local coordinate 
systems 





r yi 





1 Red dot\ Comer which 
enters first 

• Red dotted line : edge 
which follows in tip¬ 
ping 


• Meshing in toothed gears: Torque oscillations about 
gear axis and linear forward motion 

• Screw-in motion: fixed torque for torque-based 
screwing 

• Angle-controlled tightening required in most screw¬ 
ing process applications: command fixed torque, 
then turn by a defined angular increment. 





for Assembly (DFA), workcell and assembly line de¬ 
sign as well as logistics and manufacturing organiza¬ 
tion [54.12]. Early on, industrial robots were used in 
assembly automation, particularly in high-throughput 
manufacturing lines (Fig. 54.4). However, robots are 
increasingly used in highly flexible workcells and will 
enter agile lean manufacturing workplaces as versatile 
tools at the hands of the human worker. In the follow¬ 
ing, selective use cases of robots in assembly will be 


described by detailing on specific enabling technolo¬ 
gies. 

Assembly of Limb Material 
Numerous assembly processes include handling of limb 
materials such as rubber hoses, wire harnesses, etc. 
that have to be fixed in position in order to be joined 
(Fig. 54.15). Obviously stabilizing the material and 
securing process quality often result in ingenious grip- 
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Fig. 54.17 Set up and implementation of a centrifugal clutch assembly for a chain saw with a sequence diagram depicting 
the consecutive steps until tightening the clutch. Through the robot’s torque sensors in each of its links and an appropriate 
kinematic and dynamic model, the resulting forces at the tool tip are controlled 


per designs involving additional actuation and sensing 
functionalities. 

An example is the automatic application of self-ad¬ 
hesive seals as they easily lose their shape and can be 
stretched or compressed. Since manual application of 
adhesive seals to vehicle bodies or doors is sensitive 
and ergonomically problematic a robot-guided tool has 
to secure bonding of the material’s surface to the car 
body. The seal material is fed from a roll under correct 
tension and the tape, which covers the adhesive, is re¬ 
moved and stored in a small tank. At the tip of the tool 
a laser sensor follows the car body or door contour and 
an actuated roller produces a continuous normal force 
on the seal. Both the laser sensor and roller’s motion 
are translated into a tension free motion of the robot. In 
addition, a magazine on a flange ensures that the seal 
is correctly tensioned and a material reserve for one car 
door is provided [54.47], 

Here, the robot acts as slave to guide a tool which 
acts as both measuring unit and precision actuator with 
own master controller. Further efforts aim at embedding 
rich sensor and control modalities in the robot to ac¬ 
count for complex process control based on tactile and 
geometric information. 

Advanced Robot-Based 
Assembly Process Control 

Automation of advanced assembly processes depends 
on physical contact between the joined workpieces. 
For this contact formation to be controlled a robot 
should offer compliant motion control which is a con¬ 
trol method that modulates robot position and velocity 


based on measured or estimated joint torques or contact 
forces [54.48]. Subject of intense research efforts for 
a long time application packages for compliant force 
control in industrial robots which fulfill requirements 
regarding versatility, robustness, and ease of use in pro¬ 
gramming have become available during the last ten 
years [54.49]. The solutions are commonly based on 
a 6-DOF force-torque sensor which is attached to the 
robot flange. 

The fully torque-controlled DLR (Deutsches Zen- 
trum fur Luft- und Raumfahrt) lightweight robot broke 
new grounds as its 7-DOF redundant kinematic struc¬ 
ture, torque sensing in every joint and a variety of 
compliance modes allowed difficult assembly tasks 
with complex contact formation during the joining pro¬ 
cess [54.15]. DLR and KUKA managed to successfully 
go the strenuous road from the original LBR inven¬ 
tion, an idea made manifest in 1991, to a product, first 
applied in research and predevelopment of new indus¬ 
trial manufacturing concepts in a series of development 
steps: KUKA LBR3 (2006), LBR4 (2008), and LBR 
iiwa (2012) [54.25]. Figure 54.16 depicts the integrated 
mechatronic design of a joint with its unique joint- 
torque measurement. 

To simplify the programming of complex joining 
processes several assembly subprocess modules were 
developed of which three are depicted in Table 54.2: 
the search for contact formation and the execution of 
two typical joining motions (peg-in-hole, toothed gear 
joining). 

Figure 54.17 depicts a scenario of a force-based 
centrifugal clutch assembly for a chain saw. The robot 
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detects and locates the clutch workpieces on a tray by 
vision sensors. Now the grasped workpiece has to be 
joined and tightly screwed onto the shaft. In this case 
the robot’s tactile capabilities and compliant behavior 
are used to achieve a robust and quick assembly. Once 
a rough shaft center position estimate is acquired by 
the robot’s vision sensor, the robot approaches to this 
position and starts performing a search pattern until 
contact detection. The peg-in-hole process is followed 
by a screw-in motion for tightening the clutch. Alterna¬ 
tive designs may be based on 6-DOF industrial robots 
with a wrist-mounted force-torque sensor to measure 
process-induced forces on the robot, or simply estimate 
the forces from the motor torque control [54.50]. 

The sequence diagram shows a simplified execution 
trace of a spiral search of a circular shape. Listing 54.1 
lists part of the Java code in the KUKA Sunrise control 
system which identifies the three parts of the compliant 
assembly: main function, spiral search until collision, 
and border walk. 


Listing 5U.1 Extract from the code controlling the 
sequence of the centrifugal clutch assembly 
Main Program: 

Frame rough = camera.detectAssemblyTarget(); // Retrieve rough position from stereo vision 
// Move robot at 30 mm/s. Linear motion. Stop if force > 20 Newton 
ForceCondition forceCond = new ForceCondition(robot.getDefaultMotionFrame(), 20); 
robot.move(1in(rough).setCartVelocity(30).breakWhen(forceCond)); // First approximation 
Frame target = spiralSearch(rough,forceCond); // Obtain the assembly target 
assemble(Target); // Assembles the shaft at the given position 

Search Module. Returns the assembly target: 

Frame spiralSearch(Frame rough, ForceCondition forceCond){ // Spiral search until collision 
for (double phi = 0; phi < 10.0 * pi; phi +=pi / 90.0){ // 5 loops, 2° step 
Frame spiral = rough.copy(); 

Double radius = max_radius * phi / (10.0 * pi); 
spiral.setX(rough.getX + Math.cos (phi) * radius); 
spiral.setY(rough.getY + Math.sin(phi) * radius); 

robot.move(1in(spiral).setCartVelocity(30).breakWhen(forceCond)); 

if (motion.getFiredBreakConditionlnfo())return findCircleCenter (); // It collides 

} 

return null; // Target not found} 

Border walk to find shaft center: 

Frame findCircleCenter()\textbraceleft // Calculates the center of a circular shape 
ArrayList contact = new ArrayList<Frame> (); // Stores frames where collisions take place 
do { 

Frame next = getNextFrame(robot.getForce()); 

// Force feedback combined with search pattern such as a grid produces next frame 

robot.move(lin(next).breakWhen(forceCond)); // Move to next position, if possible 
if (motion.getFiredBreakConditionlnfo()) contact.add(robot.getDefaultMotionFrame()); 

} while (!areEnoughContacts(contact)); // Until enough contact points to identify center 
return getCircleCenter(contact); // Circle center can be determined with three points 
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Fig.54.18a-d High-speed rotating atomizer and a multirobot workcell for car body painting, (a) A Durr EcoBell2 paint 
gun which atomizes the paint material at the edge of the rotating bell disk by centrifugal forces. All current paint materials 
such as solvent-based or water-borne paints can be used in car production, (b) Multiple robots work in parallel for optimal 
throughput and accessibility of the car body. Most of the programming which includes the synchronization of the robots 
is performed offline, also with respect to optimal coverage of the painted surface, (c) Simulation of the painting process is 
critical for achieving highest yields (e.g., minimal overspray, uniform paint deposition, etc.) (d) Results from simulations 
for optimized program generation are shown (courtesy of Durr, Fraunhofer IPA, Stuttgart) 


54.4.4 Painting 

Hazardous working conditions for human operators 
motivated Trallfa, a Norwegian company, to develop 
simple spray-painting robots in 1969, particularly for 
spraying bumpers and other plastic parts in the auto¬ 
motive industry. Initially pneumatically driven for anti¬ 
explosion reasons, today’s robot designs are fully elec¬ 
tric. They also have hooks and grippers to open hoods 
and doors during the painting task. Hollow wrists that 
house gas and paint hoses allow fast and agile motions. 
Spray guns for robots have evolved dramatically for de¬ 
livering uniform quality using as little paint and solvent 
as possible and also for switching between different 
paint colors. Originally spray-painting robots replicated 
movements copied from human workers. Most of the 
programming for robot painting today is done offline as 
state-of-the-art programming systems offer integrated 
process simulations to optimize paint deposition, thick¬ 
ness, and coverage (Fig. 54.18). The simulation of the 


process is quite complex as different effects are taken 
into account such as turbulent flow field between atom¬ 
izer and target, static electrical field between rotating 
bell disk and target, charging of the paint droplets 
at the bell, space charge effects due to the flow of 
charged paint droplets, and Coulomb forces acting on 
the droplets [54.52]. 

54.4.5 Processing 

Material removal processes such as grinding, debur- 
ring, milling, and drilling are increasingly carried out 
by industrial robots with serial kinematics as they 
combine dexterity, versatility, and cost-effectiveness. 
The employed process tools are often combined with 
passive compliance or active force control as the 
workpiece geometry commonly exhibits tolerances in 
geometrical or material properties [54.53]. However, 
robot accuracy (±0.5 mm range) compares poorly to 
values in the ±0.01 mm range of typical machine 
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Fig.54.19a-d For small-to-medium-sized parts, a preferred configuration is to have the part grasped and guided by 
the robot relative to the fixed spindle, (a) The robot's gripper is mounted on a force-torque sensor to measure and 
limit process forces. For machining the edges of the workpiece (deflashing) the machining software package provides 
a machine-learning technique for optimising the motions [54.51]. (b) Influences on accuracy in robotic machining and 
methods for robot accuracy improvement. Still typical machine tool motion accuracies are in the order of ± 0.01 mm or 
better (courtesy of ABB, Fraunhofer IPA) 


tools [54.54]. The lower eigenfrequencies and damp¬ 
ing coefficients of mechanical structures should be 
as high as possible for precision: The lower eigen¬ 
frequencies of milling machines are in the range of 
50—lOOFlz as compared to 20—30Hz for typical in¬ 
dustrial robots [54.55,56]. Figure 54.19 depicts the 
factors that affect the robot’s accuracy in typical ma¬ 
terial removal applications. Figure 54.20 shows first 
eigenfrequencies, damping, and stiffness in Cartesian 


space. These characteristics are essential for the de¬ 
sign of machining processes and resulting workpiece 
quality. More robot-guided processes such as laser 
welding and laser cutting depend on achieving similar 
accuracies. 

Robot position accuracy results from geometric er¬ 
ror sources (deviations between actual robot structure 
and assumed Denavit-Hartenberg (DH) parameters) 
and nongeometric parameters (compliance of the me- 
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Fig.54.20a-c Robot dynamics in Cartesian space: (a) Damping and (b) first eigenfrequencies for a KUKA KR60 and 
(c) stiffness for a KR125 in a typical XZ-process plane (measured from the robot’s first axis) of the robot’s workspace 
(courtesy of ISG Stuttgart, Fraunhofer IPA) 
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Table 54.3 Characteristics of computerized numerical control (CNC) and robot controller (RC) 


Category 



Targeted 

application 

Machining, material 
removal 

Handling, assembly 

Motions 

Path-based, complex contour- 
oriented 

Point- or path-based and mo¬ 
tion time-oriented 

Programming 

On-site programming based 
on standardized program¬ 
ming language (G-code), ISO 
(International Organization 
for Standardization) 6983, 
use of computer aided manu¬ 
facturing (CAM) tools 

On-site teaching (teach 
pendant, editor) based on 
supplier specific languages, 
use of typical robot simula¬ 
tion environments 

Command 

Online interpreter, continu¬ 

Initial loading of programs, 

reading 

ous loading of instruction 

which are usually interpreted, 
sometimes compiled) 


Interpretation 


A CNC machine tool is single pur¬ 
pose; generally robots are universal 
machines 

Extended look-ahead of CNC 
controllers allows detailed path de¬ 
scription and adaption on » 100 via 
points (< 10 via points in most robot 
controllers) 

Whereas robots are traditionally pro¬ 
grammed manually on site, CNC 
controllers use CAM technology to 
generate complex paths automatically 
based on CAD data 


Program size in robot controllers 
limited by memory, CNC interprets 
programs online, may execute an un¬ 
limited number of commands 


CAD 


Interface (programming) |CAM 


1 

Interpreter RC-code 


Interpreter G-code 




1 — 

Tool radius compensation 

- — 

Motion planning 


Motion planning 

poses 


path driving 


Kinematic transformation 


-> Time-oriented 


-> Contour-oriented 

1 


1 

Path interpolation in 


Path interpolation in 

static state space 


dynamic state space 


Robot transformation, compensation, filter 


|_ Safety _| 

Robot axes I Q Robot controller 

I I I I I I I I NC controller 

m=i n~b n~b l~6~b 

Fig. 54.21 Structure of an NC-kernel integrated into a ro¬ 
bot controller (courtesy of ISG Stuttgart) 

chanical structure and gear play). In order to reduce the 
impact of the nongeometric parameters several stepwise 
approaches can be implemented: 

• In combination with force prediction or online 
measurements robot compliance can be compen¬ 
sated by means of joint stiffness models [54.57]. In 


drilling applications additional encoders are some¬ 
times mounted on the arm side of robotic joints in 
order to measure gear-induced joint compliance and 
backlash, or those effects can be estimated based on 
determined joint properties [54.58], so that a com¬ 
pensation can be achieved. 

• In combination with the geometric error calibration 
accuracies of better than ±0.2 mm for typical robots 
have been realized in larger robot workspaces of 3 x 
3 x 2m 3 [54.59], 

• For higher accuracies further sensor and actuators 
systems have been introduced. Using optical track¬ 
ing deviations of ±0.2 mm have been demonstrated 
in steel [54.60]. With additional actuation devia¬ 
tions could be reduced to ±0.1 mm [54.61]. 

Robot and CNC machine tool controls may have 
similar origins, but have taken different paths in de¬ 
velopments over the years as depicted in the following 
Table 54.3. 

Increasingly CNC controllers are used for robots in 
material removal applications for taking advantage of 
the well-established off-line programming tools in the 
CNC world and for improving motion accuracy of the 
robot for complex 3-D contours (Figs. 54.21 and 54.22. 
Modern robot controllers integrate so-called numerical 
control (NC) kernels which share components of the 
robot controller such as user interfaces, kinematic trans¬ 
formations, and safety functions. 
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N05 T1 (N: line or block number, Tl: selection of tool 1) 
N10 G41 (turn on cutter radius compensition (CRC), left) 
N20 G01 G90 X5 (G90: absolute position of x-axis, x=5) 

N30 G01 X10 (linear interpolation motion, x=10) 

N40 G02 X17 Y5 15 JO (circular interpolation about 
center, clockwise) 

N50 G01 X20 YO 
G60 G01 X15 Y-5 

N70 G03 X20 Y-10 12.5 J-2.5 (circular interpolation, 
about center, counter clockwise) 

N80 G01 X10 
N90 G01 X5 YO 


Programmed path 
Corrected path 

\ 

Circular transition 
(G26) 


1 1 Linear slope transition 
_J_J (G25) 


Fig. 54.22 (a) A simple CNC program 
(G-code, ISO 6983). A CNC program 
normally is machine independent and 
orients itself on the workpiece contour 
after processing, (b) Most CNC 
programs are generated automatically 
through CAM tools that transfer 
geometric information into executable 
G-code. The example shows generated 
tool trajectory generation for precision 
milling of a carbon fiber (CF) part and 
the finished workpiece (courtesy of 
Delcam, Birmingham; ISG, Stuttgart) 


54.5 Safe Human-Robot Collaboration 


Human-robot collaboration allows the combination of 
typical strengths of robots with some of the numer¬ 
ous strengths of humans. Typical strengths of industrial 
robots are high stamina, high payload capacity, pre¬ 
cision, and repeatability. Strengths of human workers 
that are unmatched by any machinery comprise flexibil¬ 
ity for new production tasks, creative problem-solving 
skills, and the ability to react to unforeseen situations. 

However, industrial robots have a significant poten¬ 
tial to harm humans. Therefore, standards for designing 
and operating industrial robot automation systems have 
been introduced and found international acceptance. 
Since 1999, efforts have been made to define measures, 
rules, and examples specifically for robots in collabora¬ 
tive modes. 


54.5.1 Overview of Basic 

Robot Safety Standards 

Machine safety standards provide guidelines for de¬ 
signing and operating any type of machinery. Their 
consideration is optional, but exhibits the straightfor¬ 
ward way of verifying the fulfillment of the fundamen¬ 
tal health and safety prerequisites of the Machinery 
Directive [54.63,64]. Generally, safety standards are 
classified into three categories (Fig. 54.23): 

• Type A standards: basic standards that define fun¬ 
damental principles to achieve machine safety. 

• Type B standards: generic standards that regulate 
either a specific safety aspect, e.g., separating dis- 



Fig. 54.23 Different types of standards and a simplified iterative risk assessment procedure. See ISO 12100 [54.62, 
Fig. 1 and 2] for a more detailed schematic representation of risk reduction process 
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tances, or specific protective devices, e.g., opto¬ 
electronic protective equipment like laser scanners 
or emergency stop (ISO 13850), and are valid for 
a spectrum of machines. 

• Type C standards: machine standards that list de¬ 
tailed safety requirements for a specific type of 
machine, e.g., industrial robots. 

For robotic safety the following standards are of 
highest importance: 

• ISO 12100 [54.62]: Type A standard listing general 
principles of machine safety, in particular the risk 
assessment process. 

• ISO 13849 [54.65] andlEC 62061 [54.66,67]: Type 
B safety standards governing the design of control 
systems with safety functions. 

• ISO 13855 and ISO 13857 [54.68,69]: Safety 
distances for separating and nonseparating safety 
equipment, e.g., fences, light curtains, and laser 
scanners. 

• ISO 10218-1/-2 [54.5,6]: Type C standards specif¬ 
ically covering safety of industrial robots and robot 
system integration. 

For the setup of any robot installation, an iterative 
risk assessment process as defined by ISO 12100 has 
to be conducted. Its workflow as depicted in Fig. 54.23 
starts with a functional and geometric design of the ma¬ 
chine followed by the definition of the machine limits 
including spatial boundaries and boundaries regarding 
usage, e.g., typical tasks, operator qualification, and en¬ 
vironmental conditions. The process continues with an 
identification of the tasks which are consecutively as¬ 
sessed on its risks. Methods for risk estimation are, for 
example, risk trees as presented in Fig. 54.24. Thereby, 
any individual hazard is rated on its risk level by quan¬ 
titatively considering the potential injury severity, the 


exposition to the hazard, and the possibility to avoid it. 
The risk estimation leads to a decision if all hazards 
have been adequately addressed in the machine design. 
If this is not the case, the machine design is modified to 
reduce the specific risk and the risk assessment process 
is repeated. 

According to ISO 12100, measures to reduce risks 
need to be carried out according to the following 
priorities: 

• Risk reduction through inherently safe design 

• Risk reduction through safeguards and protective 
devices 

• Risk reduction through information for use (e.g., 
work instructions, instructions to wear protective 
equipment). 

The central safety standard family for robot safety is 
ISO 10218. Part 1 addresses the safety requirements of 
the robots, complemented by part 2, which focuses on 
the robot system, i. e., the complete integrated machine 
performing a production task. The second revision of 
part 1 and the first revision of part 2 were released 
in 2011 and replace any former robot safety standard, 
such as EN 775. These standards for the first time de¬ 
fine human-robot-collaboration as a specific form of 
a robotic application in an industrial setting and pro¬ 
vide guidelines for setting up such collaborative robot 
systems. 

54.5.2 Types and Requirements 

of Human-Robot Collaboration 

ISO 10218-1:2011 contains specific requirements on 
human-robot-collaboration. It defines four types of 
collaborative operations in which a human can collab¬ 
orate with a robot in automatic mode, as depicted in 
Fig. 54.25. All controller functions for safe human- 
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Legend risk parameters: 

S severity of injury 

51 slight (normally reversible injury) 

52 serious (normally irreversible injury or death) 

F frequency and/or exposure to hazard 

FI seldom-to-less-often and/or exposure time is short 

F2 frequent-to-continuous and/or exposure time is long 
P possibility of avoiding hazard or limiting harm 
PI possible under specific conditions 

P2 scarcely possible 

L low contribution to risk reduction 

H high contribution to risk reduction 

PL r required performance defined in terms of probability 
of dangerous failures per hour. Examples: 

10 -5 < a < KT 4 (dangerous failure per hour) 

KT 8 < e < 1(T 7 (1/h) 


Fig. 54.24 Risk graph for determining required PL r for safety function according to EN ISO 13849-1 (after [54.65]) 














































Industrial Robotics I 54-5 Safe Human-Robot Collaboration 1407 



0 Type 3 dec l/f(v ro bob ^human )5 

if d < d m then stop 


d) Type 4 hazard induced 

by_robot — hazardanowed 




Fig.54.25a-d Modes of human-robot 
collaboration according to ISO 10218. 
(a) Stop on access with automatic 
task resumption, (b) hand-guiding 

(c) separation and speed reduction, 

(d) monitoring and power and force 
limiting 


robot collaboration have to comply with performance 
level PLj with category 3 structures (ISO 13849-1) or 
SIL 2 (IEC 62061) if the safety assessment does not 
yield a differing requirement: 

• Safety-rated monitored stop (Fig. 54.25 Type 1): 
The robot is stopped upon access of the human to 
the collaborative workspace with the robot drives 
still in control. A so-called safety controller, now 
offered by most robot manufacturers, assures the 
standstill of the robot. The robot task can be auto¬ 
matically resumed as soon as the human operator 
has left the collaborative workspace. Human and 
robot share the same workspace, but the robot does 
not move while the human is present. 

• Hand-guiding (Fig. 54.25 Type 2): This type of 
operation implies a direct physical interaction 
between human and robot with full control of 
the human over the robot movement. The human 
guides the robot through a direct input device 
(e.g., a handle) at or near the end-effector while 
activating a three-position enabling device (three- 
position hold-to run device). Thereby the position 
of the worker within the collaborative workspace 
is defined. A safety controller for delimiting the 
robot speed to a specified threshold is required. 
Hand-guiding in combination with graphic support 
through icons or 3-D simulation is in particular 
suitable for intuitive programming of industrial 
robots [54.70] during automatic mode of the robot. 

• Speed and separation monitoring (Type 3): The 
relative speed between robot and human as well as 
their distance are actively monitored. If the human 
is present, the robot has to maintain a safe com¬ 
bination of speed and distance to the human to be 
able to stop any hazardous motion before a contact 
with the human may occur. Again safety controllers 
in conjunction with safe surveillance sensors (in¬ 


cluding safe sensor data processing) are needed to 
supervise speed and position of the robot for human 




Fig.54.26a,b Intuitive instruction of a welding robot by lead- 
through programming: (a) the worker guides the robot by a flange- 
mounted (B) handle (A) while pressing safety switches (three- 
position hold-to run button) (C) on either side. The force exerted on 
the handle is measured by a force-torque sensor (E) and translated 
into robot motion. The seam tracking sensor (F) simultaneously 
measures the workpiece contour for precisely localizing and ex¬ 
tracting weld seams. The recorded robot motion is visualized and 
edited in a simulation environment before task execution, (b) The 
safety relevant part of the system contains a three-position hold-to- 
run device, a laser scanner for safety monitoring during automatic 
program execution and a speed monitoring function during hand- 
guiding is an operational feature which does not have to be realized 
with a specific safety integrity level (courtesy of Fraunhofer IPA) 
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motion capturing. Currently, relatively few sensors 
in industrial automation offer this capability with 
the required safety integrity level. However, such 
systems will be available in the future, then being 
able to feature novel safety strategies for dynamic 
distance control [54.71]. 

• Power and force limiting (Type 4): The mechanical 
hazard potential of the robot is sufficiently reduced 
to allow a direct, physical interaction of human 
and robot without an additional safety controller. 
This is achieved by appropriately limiting collision 
forces through the design of the robot system such 
that in the event of a contact between the human 
and a robot biomechanical tolerance limits are not 
exceeded. 

The ISO 10218 standards in their latest revision as 
of 2011 explicitly demand that speeds, distances, pow¬ 
ers and forces are to be sufficiently limited, but does 
not give precise threshold values for these limitations. 
These threshold values need to be determined through 
risk assessment for a specific application that is fore¬ 
seen with the robot system. Currently, the technical 
committee ISO/TC184/SC2 Robots and robotic devices 
develops a technical specification ISO/TS 15066 (in 
committee draft (CD) status of December 2015). It has 
been drafted to offer more guidance on the risk assess¬ 
ment for collaborative robots and will be released in 
the near future [54.72]. The procedure to achieve safety 
outlined in TS 15066 is new for machine safety (not 
only for robot safety) and is expected to make its way 
into further standards, especially into the second revi¬ 
sion of ISO 10218-2. 

For the first time, ISO/TS 15066 introduces tol¬ 
erance values for the physical strain of the human 
body. These strain thresholds include maximum forces 
and maximum pressures for different body parts that 
can be sustained without suffering from pain or even 
injury. The risk assessment process involves determin¬ 
ing body regions with risk of contact depending on 
the specific applications and workflows. Based on this 
information, it can be proved experimentally or in sim¬ 
ulation that the given severity threshold values are not 
exceeded due to limited mechanical or robot control pa¬ 
rameters [54.73,74]. The technical specification aims 
at transforming highly complex biomechanical injury 
thresholds into controllable robot performance limits. 
One of the principles used for dynamic collision analy¬ 
sis is based on the fundamental robot dynamics theory 
which enables the representation of the reflected in¬ 
ertia of a robot system at any point along the robot 
structure. Drop-test results from medical injury assess¬ 


ment have been made available for designing robot 
controllers [54.75]. 

Part 2 of the ISO 10218 standard provides guide¬ 
lines regarding the aspects that need to be taken into 
account during robot workcell setup and system inte¬ 
gration. It states the need not only to assess the robot 
itself, but to take into account the complete application, 
particularly the end-effector, process, environment, and 
typical work tasks. This is of special importance for col¬ 
laborative human-robot operation. As the application 
always needs to be taken into account, it is not possible 
to design a safe robot, but only to provide robots that 
are equipped with safety features for setting up a safe 
collaborative operation. 

54.5.3 Examples of Human-Robot 
Collaboration 

In the following, two examples for human-robot col¬ 
laboration are given. 

Intuitive Instruction of a Welding Robot 
by Lead-Through Programming 
Figure 54.26 presents a robot welding workcell which 
is designed for small batch size production. An intuitive 
teaching process implementing hand-guiding accord¬ 
ing to ISO 10218 (Type 2 in Fig. 54.25) significantly 
speeds up robot programming time. The presence of 
the human during the collaborative operation is mon¬ 
itored via a laser-scanner activating safety zones and 
safe speed reduction of the robot safety controller dur¬ 
ing the hand-guiding operation. The recorded motion 
can be executed automatically after switchover as soon 
as the human is out of the collaborative workspace. 

Collaborative Assembly of Battery-Cases 
In this case, sensitive tasks are carried out by the hu¬ 
man, while strenuous tasks are executed automatically 
by a small payload robot (Fig. 54.27 for the cell design). 
Thereby, the safety concept comprises the require¬ 
ments for speed and separation monitoring (Type 1 
in Fig. 54.25) from ISO 10218 and involves two light 
curtains with a signal processing on a safety PLC and 
a robot with a safety controller. Switchover between 
the safety zones is activated upon detection of human 
presence in a collaborative workplace area. Each safety 
zone statically monitors the minimum possible distance 
to the robot’s active workspace thereof deriving the 
maximum possible robot speed. Such a system reduces 
space requirements for a robot installation while en¬ 
hancing the flexibility of the assembly process due to 
the collaborative nature of this assembly process. 
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Fig. 54.27 The robot’s 
workcell is divided into 
three segments by two light 
curtains which trigger in 
which area the robot works 
at lower velocity when 
collaborating with the worker 
in the collaboration spaces 


- Teaching and Programming 


54.6 Task Descriptions 

Although desirable, a robot cannot be instructed in 
the same way that one would instruct a skilled human 
worker how to carry out a task. With skilled human 
workers knowing the applications, devices, processes, 
and the general requirements on the product to be man¬ 
ufactured, we would only need to summarize what 
needs to be done. Humans have, with or without aware¬ 
ness thereof, extensive knowledge about motions, phys¬ 
ical effects, cause-effect relationships, learned proce¬ 
dures, etc., and they maintain such knowledge for their 
own good. A human is skilled because physical capabil¬ 
ities are combined with reuse of that leamed/maintained 
knowledge, and the purpose of the manufacturing oper¬ 
ation can be explained and understood. 

Robots are not capable of performing such knowl¬ 
edge-based behaviors in a productive manner. Instead, 
instructions have to be quite explicit and motion ori¬ 
ented, and even motion planning is rarely used since 
it is difficult to encode much of the required back¬ 
ground knowledge. We could aim for programming 
principles that resemble instructing a (totally) unskilled 
worker, telling precisely how every aspect of the task 
is to be performed. Such an explicit way of instruct¬ 
ing the robot then should be human friendly, but the 
performance requirements motivated by productivity 
needs imply that methods for defining the task need 
to reflect machine/robot properties considering prod¬ 
uct data and production processes. Existing approaches 
include: 

• Manually guiding the robot to the positions of inter¬ 
est, or even along the desired paths or trajectories if 
human accuracy is enough (Fig. 54.26). 

• Having simple ways to make use of CAD data 
whenever available (Fig. 54.13). 

• Using different complementary modalities (paths of 
communication between the human and the robot), 
such as pointing devices and 3-D graphics. 


• Choreographing the task movements, for instance 
loops and conditions, without requiring extensive 
programming competencies. 

• Means of describing acceptable variation, e.g., ac¬ 
ceptable deviations from the nominal path or POI. 

• Specification of how external sensing should be 
used for path adaptations or for handling unknown 
variations. 

Most of these items still mean that the robot is rather 
dumb, and the knowledge of solutions is maintained 
among human experts. Consequently, robots are time- 
consuming and hard to instruct for productive indus¬ 
trial tasks, and the created solutions are not reusable 
since the knowledge is not explicitly represented in 
a way that is useful for the robot. To embody knowledge 
in the robot system is the way to make task defini¬ 
tion more efficient, which can be better understood if 
we review the origins and the progress so far. Initially, 
mainly during the 1970s and 1980s, there were some 
painting robots that could be programmed by man¬ 
ual guidance. This was possible due to the following 
abilities: 

• Applications such as painting permitted the use of 
a lightweight arm, including the end-effector, possi¬ 
ble balanced with respect to gravity if needed. 

• The accuracy requirements were (compared to to¬ 
day) modest so it was possible to use back-drivable 
drive trains and actuators, and the definition of the 
motions could then be done manually by the op¬ 
erator moving the end-effector along the path. The 
recorded poses, including the timing/speed informa¬ 
tion, defined the programmed motion. 

• Since no inverse kinematics was needed during pro¬ 
gramming or real-time operation, it was not a prob¬ 
lem from a computing point of view to use arm 
kinematics without singularities in the workspace. 
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• The requirements of optimality of painting motions 
were also modest, compared to recent years when 
environmental conditions (for nature and workers) 
called for minimal use of paint. 

It is often referred to as an inevitable problem 
that there are singularities to be handled within the 
workspace. However, to simplify the kinematics and 
its inverse from a software point of view (e.g., during 
the 1980s considering the power of the microproces¬ 
sors and algorithms available at that time), robots were 
actually designed to have simple (to compute) inverse 
kinematics. For instance, the wrist orientation was de¬ 
coupled from the translation by the arm by using wrist 
axes that intersected with the arm axes. The resulting 
singularities within the workspace could be managed by 
restrictions on wrist orientations, but an unfortunate im¬ 
plication was that robot arms were no longer back-driv- 
able (close to the singularities) when designs were (due 
to engineering and repeatability requirements) adopted 
to standard industrial controllers. Then with the devel¬ 
opment of microprocessor-based industrial controllers 
and the definition of motions based on jogging (manual 
moves by for instance using a joystick) and CAD data, 
the means of robot programming became closer to com¬ 
puter programming (extended with motion primitives). 

Robot programming languages and environments 
have traditionally been separated into online program¬ 
ming (using the actual robot in situ) and OLP (using 
software tools without occupying the robot). With the 
increasing power of OLP tools, their emerging abil¬ 
ity to connect to the physical robot, and the increasing 
level of software functions that are embedded into the 
robot control system, online programming is now un¬ 
usual, except to verify and manually adjust programs 
generated offline. Of course, it is economically impor¬ 
tant to minimize downtime for robot programming, and 
advanced sensor-based applications may be too hard 
to develop without access to the true dynamics of the 
physical workcell for fine-tuning. Nevertheless, robot 
languages and software tools must provide for both 
methods of programming. 

Even though robot languages from different man¬ 
ufacturers look similar, there are semantic differences 
that have to do with both the meaning when programs 
are running (the robot performing its operations) and 
the way the robot is instructed. The need to ensure 
that existing robot programs can operate with replace¬ 
ment robots and controllers, and also to make use 
of existing knowledge in robot programmers and in¬ 
corporated into OLP software, requires manufacturers 
to continue supporting their original proprietary lan¬ 
guages. Features such as backward execution (at least of 
motion statements) and interactive editing during inter¬ 


pretation by the robot (in combination with restrictions 
to make the programming simpler) also make robot 
languages different from conventional computer pro¬ 
gramming languages. 

During the last decade and in current developments, 
the trend has been to focus on the tool (robot end-ef¬ 
fector) and on the process knowledge needed for the 
manufacturing process, and to let the operator express 
the robot task in such terms. This development results 
in a need for an increased level of abstraction to sim¬ 
plify the programming, reflecting the fact that the so- 
called robot programmer knows the production process 
very well, but has quite limited programming skills. To 
understand why such a high level of abstraction has not 
come into widespread usage, we may compare this with 
the early days of industrial robotics when there was no 
kinematics software built into the controllers, and hence 
the robots were programmed via joint-space motions. 
(Kinematics here deals with the relation between robot 
motors/joints and the end-effector motions.) 

Built-in inverse kinematics permits tool motion to 
be specified in Cartesian coordinates, which is clearly 
a great simplification in many applications. That is, the 
robot user could focus more on the work to be carried 
out by the robot and less on the robot itself. How¬ 
ever, robot properties such as joint limits cannot be 
neglected. Until the beginning of the 1990s, robots did 
not follow programmed trajectories very well at high 
speeds or accelerations, and full accuracy could only 
be achieved at low speed. Modern robot systems with 
high-performance model-based motion control perform 
their tasks with much greater accuracy at high speed 
due to model-based control features (Chap. 6). 

There are increasing opportunities to raise the level 
of abstraction to simplify the use of robots even more 
by encoding more knowledge about the robot, tools, 
the process, and workcells into control systems. This 
is a gradual development with possibilities depending 
on the application; when knowledge is not encoded the 
robot programmer needs to be more aware of robot 
properties and how to handle them. Reasons for not 
including such knowledge typically lie in the fact that 
it is simply too difficult to do so or because the robot 
programmer does neither have the competence nor the 
tools nor reasons to do so. With reference to Table 54.4, 
the following example explains that type of trade-off 
and what it means in terms of centricity. 

A machine part consisting of steel plates and pipes 
is to be manufactured by means of welding, and there 
is robot and process equipment available for production 
of this (and other) types of workpieces: 

• A product-centric system would generate configu¬ 
rations and robot programs, and instruct the oper- 
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Table 54.4 Robot-related centricity, ranging from a high-level view of the work to be accomplished in terms of the product to 
manufacture, to a low-level robot motion view that, in practice, constraints what manufacturing operations can be performed 



Product Description of the final shape and assemblies of the workpieces, in terms of that 
product; the robot system plans the operations 

Process 

Tool 

Arm 

Joint For each specified location, the joint angles are specified, so straight-line motions 
are difficult; the robot provides coordinated servo control 


Based on known sequences of specific manufacturing operations, each of these is 
specified in terms of their processes parameters 

The motion of the robot-held tool is specified in terms of programs or manual guid 
ance; the user knows the process it accomplishes 

The robot arm and its end-plate for tool mounting is programmed how to move in 
Cartesian space 


* 


ator for whatever manual assistance that might be 
needed (e.g. clamping and fixturing). The system 
would determine the welding data such as the type 
of welding and how many passes for each seam. 

• A process-centric system would instead accept in¬ 
put in terms of the welding parameters to use, 
including the order of the welds. The system would 
select input signals to the process equipment (such 
as what output voltage the robot controller should 
set such that the welding will be done with a certain 
current), and robot programs specifying the motions 
of the welding torch would be generated. 

• A tool-centric way of programming would require 
the operator to set up the process manually by 
configuring the equipment in terms of their native 
settings, and appropriate tool data would be config¬ 
ured such that the robot controller can accomplish 
the programmed motions, which are specified by 
giving coordinates and motions data referring to the 
end-effector. 

• An arm-centric system is similar to the tool-centric 
approach in that Cartesian and straight-line mo¬ 
tions can be (and need to be) explicitly specified/ 
programmed by the programmer, but extra work is 
needed since the robot does not support a general 
tool frame. 

• A joint-centric style would be needed if one of the 
very early robot systems is being used, requiring 
a straight line to be programmed by lots of joint- 
space poses close to each other. 

Thus, the question is, are you programming robot 
joint servos to make the robot provide a service for 
you, are you programming/commanding an arm how to 
move a tool, is it the tool that is made programmable by 
means of the robot, is it manufacturing services that are 
ordered by specifying the desired process parameters, 
or is it an intelligent system that simply can produce 
your product? Still today in industry, there are reasons 


to think and program considering all five levels, al¬ 
though the tool-centric alternative is kind of a baseline. 
Hence tool-data needs to be maintained in the robot pro¬ 
gram, as in the next example below. 

As a final goal related to the product-centric view, 
so-called task-level programming is desirable. This has 
been a goal since the 1980s, and implicitly also since 
the very beginning of robotics. It would mean that the 
user simply tells the robot what should be done in com¬ 
mon terms and the robot would know how to do it, but it 
would require extensive knowledge about the environ¬ 
ment and so-called machine intelligence. The need for 
extensive modeling of the environment of the robot is 
well known. Sensing of the environment is costly, but 
in an industrial environment it should only be needed 
occasionally. Modeling has to encompass full compo¬ 
nent dynamics and the limitations of the manufacturing 
process. With these difficulties, task-level programming 
is not yet achievable in practice. If we limit our domain, 
however, such as to machining where motion planning 
is already used for so-called machine tools, generating 
the robot programs from product descriptions is feasible 
since the process (with tolerances due to robot compli¬ 
ance) is modeled. 

As indicated in the application examples, it is now 
common practice to generate robot programs from 
geometric data in CAD files. That is, the CAD appli¬ 
cation could be the environment used for specifying 
how the robot should perform the required operations 
on the specified parts, which is the CAM part of 
CAD/CAM (Computer Aided Manufacturing, often in 
term of a back-end to the CAD system, and hence 
usually not explicitly referred to). CAM is not quite 
task-level programming since human operators do the 
overall planning. CAD software packages are powerful 
3-D tools that are very common among manufacturing 
companies. Consequently, using these tools for robot 
programming is desirable since the operator may start 
the OLP of the necessary manufacturing operations us- 
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ing the 3-D model of the product, even before selecting 
a robot. 

There is an ongoing competition along the so-called 
vertical integration from product data down to machine 
operations, including at least the following three ap¬ 
proaches: 

• CAD providers increasingly offer advanced robot- 
aware functions, thereby expanding their field of 
service and expertise from a higher (CAD) to 
a lower (robot) level. An example of a robot-aware 
CAM system is depicted in Fig. 54.22. 

• Robot providers have developed programming tools 
with CAD data import and various motions plan¬ 
ning modules, and thus, expanded their offerings 
to higher levels. One such software is shown in 
Fig. 54.29. 

• A further development is to include a CNC kernel 
into the robot controller, as suggested in Fig. 54.21, 
making the robot behave as a CNC machine. 

These approaches as such are more complemen¬ 
tary than competing, with the competing stakeholders 
presently exploring what benefits can be gained for cus¬ 
tomers and business in each type of application. 


Listing 5U.2 Example pick-and-place operation in 
the ABB robot programming language Rapid 

PROC PicklnPallet() 

MoveJ Offs(pPicklnPallet,-500,0,500),v2000,zl00,tGripper\WObj:=wobjPalletStatic; 
MoveL RelTool(pPicklnPallet,0,0,-100),v2000,zl00,tGripper\WObj:=wobjPallet; 

MoveL RelTool(pPicklnPallet,0,0,0),vlOO,fine,tGripper\WObj:=wobjPallet; 

Grip; 

MoveL RelTool(pPicklnPallet,0,0,-100),v500,zl0,tGripper\WObj:=wobjPallet; 

MoveJDO Offs(pPicklnPallet,-500,0,500),v2000,zl00,tGripper\WObj:=wobjPalletStatic, 
doNewObj ect,1; 

ENDPROC 

PROC PlaceAtOutFeeder() 

MoveJ Offs(pDropOutFeeder,0,0,200),v2000,zl0,tGripper\WObj:=wobjOutFeeder; 

WaitDI diOutFeederReady,1; 

MoveL Offs(pDropOutFeeder,0,0,0),v200,fine,tGripper\WObj:=wobjOutFeeder; 

Release; 

MoveLDO Offs(pDropOutFeeder,0,0,200),v500,zl0, 
tGripper\WObj:=wobjOutFeeder,doStartOutFeeder,1; 

MoveL pFromOutFeeder,v2 0 0 0,z10,tGripper\WObj:=wobj Machine; 

ENDPROC 

PROC Grip() 

SetDO doGrip,1; 

WaitDI diGripped,l; 

ENDPROC 
PROC Release() 

SetDO doGrip,0; 

WaitDI diGripped,0; 

ENDPROC 
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Fig. 54.28 Workflow according to 
the PowerMill software from Delcam 
with its robot module. Note that 
step 4 includes also simulation of 
robot properties such as closeness to 
singularities 



Fig. 54.29 Robot program¬ 
ming environment, with 
a virtual controller (console 
to the lower right) for each 
robot including the model- 
based embedded control 
software compiled for the 
personal computer (PC). To 
the left there are the instances 
of all objects depicted as 
a tree structure, ranging from 
complete machines down to 
the bits of the I/O signals 


At step 5 in Fig. 54.28, the motion specification is 
generated and expressed in the native language of the 
robot. On the other hand, the tool path could be ex¬ 
pressed in machine-independent, standardized G-code. 
A further step would be to integrate a machine tool in¬ 
terface and control into the robot controller which is 
depicted in Fig. 54.22. What approach to take is an area 
of current research and development (R&D) and busi¬ 
ness decisions, but most robots are programmed in their 


own language only, since the needed system functions 
as a consequence of Table 54.4. 

To exemplify task definitions based on a native 
robot language, consider the application depicted in 
Fig. 54.29, which includes bin-picking, handling, ma¬ 
chine tending, conveyer tracking, and palletizing. List¬ 
ing 54.2 lists a sample program with some of the 
functions from that application programmed in ABB’s 
proprietary robot language Rapid. The program being 
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tool-centric shows in the tool argument of the Move 
statements, the tGripper that is a reference to the tool 
data including the tool frame. At the same time, it is 
arm-centric in the way that the execution of motions 
is performed per arm (with support for synchronized 
execution for multiarm robots), as expressed by the 
MoveL statements. Correspondingly, the MoveJ per¬ 
forms a joint-space motion. 

Note that Grip and Release are in separate proce¬ 
dures where execution waits until a hardware acknowl¬ 
edged signal is obtained. For Move statements L denote 
Linear, J means Jointspace, functions Offs and RelTool 
compute target poses based on other determined poses 
(i. e., frames). The v -constants are velocities in mm s~ \ 
and z-constants (or fine) specify how close to the target 
pose the motion should be. Dl refers to digital input and 
DO to digital output. The WObj is the work object, i. e., 
the specification of the base frame of the motions. 

When programming motion behaviors like a search 
strategy, which could be considered being a motion 
primitive from an application point of view, use of the 
end-user language of that robot might not be the best 
option. Instead a computer programming language can 
be more appropriate as shown in Listing 54.1. In the 
ABB case, the strategies are implemented as part of an 
application package as for assembly, and the code is 
internally written in C (cannot be viewed or changed 

54.7 System Integration 

It is interesting to note that connecting different work¬ 
cell devices with each other, and integrating them into 
a working system, is hardly mentioned in the robotics 
literature. Nevertheless, in actual nontrivial installa¬ 
tions, this part typically represents, apart from the cost 
for peripheral device, roughly half of the overall in¬ 
stallation costs. The automation scenario includes all 
the problems of integrating computers and their periph¬ 
erals, plus additional issues that have to do with the 
variety of (electrically and mechanically incompatible) 
devices and their interaction with the physical environ¬ 
ment (including their inaccuracies, tolerances, and un¬ 
modeled physical effects such as backlash and friction). 
The number of variations is enormous so it is often not 
possible to create reusable solutions. In total, this re¬ 
sults in a need for extensive engineering to put a robot 
to work (Table 54.5). This engineering is what we call 
system integration. 

Carrying out system integration according to cur¬ 
rent practice is not a scientific problem as such (al¬ 
though how to improve the situation is), but the ob¬ 
stacles it comprises form a barrier to applying ad¬ 
vanced sensor-based control for improved flexibility, as 


by the user, but has optimized real-time performance) 
while the functionality is exposed to the robot user via 
special statements like FCRefSpiral [54.51] and more to 
accomplish the principles of Table 54.2. A variety of so¬ 
lutions are available for different robot brands [54.76]. 

Considering a complete setup with multiple robots, 
either with a few robots and some peripherals like 
in Fig. 54.9 or a complete manufacturing line as in 
Fig. 54.12, there are several robot programs that need 
to be programmed to work together and hence there 
is a coordination and complexity problem. A useful 
approach is to use a service-oriented approach, consid¬ 
ering robots as servers providing services according to 
a set of programs that are exposed on the factory or 
production-cell network which means programming is 
done by: 

• Building services (using several technologies) 
which are available, discoverable or not, remotely 
accessible by the application programmer 

• Building applications that coordinate and use those 
services. 

For such system to be programmable, however, the 
equipment (typically from different suppliers) has to be 
configured and interconnected according to the some¬ 
times overlooked art of robot system integration. 


Table 54.5 Stages of system integration, typically carried 
out in the order listed 


Physical 

Selecting equipment based on dimensioning 
for mechanical size, load, and stress 


Mechanical interfacing (locations, adapter 
plates, etc.) 


Electrical power supply (voltages and cur¬ 
rents for robots, effectors, feeders, etc.) 


Connections for analog signals (shielding, 
scaling, currents, binary levels, etc.) 


Safety design and risk assessment 

Communication 

Interconnections for single-bit digital I/O 


Byte-wise data communication, including 
latencies and bit rates 


Transfer of byte sequences 

Configuration 

Configuration of messages between inter¬ 
acting devices 


Establishment of services 


Tuning for performance and resource uti¬ 
lization 

Application 

Definition of application-level functions/ 
services 

Task 

Application programming, using the 
application-level services 
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Fig. 54.30 Manufacturing cell with a multicontroller ar¬ 
chitecture with dedicated controllers (robot motions, pro¬ 
duction sequences, positioning systems, processes and 
worker safety). In modern robot controllers, such as the 
KUKA KR C4, these controllers are replaced by soft¬ 
ware tasks of a single multifunctional robot controller, 
thus reducing investment costs on controllers and com¬ 
munication, simplifying engineering, programming and 
diagnosis, and enhancing process quality and shortening 
cycle time ► 

is needed in small series production. In particular, in 
future types of applications using external sensing and 
high-performance feedback control within the workcell, 
system integration will be an even bigger problem since 
it includes tuning of the feedback too. 

For large series production (e.g. in the automotive 
industry), the engineering cost of system integration is 
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less of a problem since its cost per manufactured part is 
small. On the other hand, the trend toward smaller series 
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Fig. 54.31 Control modules and interfaces in a typical (but open) robot controller. Several of the named network tech¬ 
nologies are based on Ethernet, ranging from normal LAN to different real-time protocols (courtesy of COMAU) 
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of customized products, or products with many variants 
that are not kept in stock, calls for high flexibility and 
short changeover times. Flexibility in this context refers 
to variable product variants, batch sizes, and process pa¬ 
rameters. In particular, this is a problem in small and 
medium-sized productions, but the trend is similar for 
larger enterprises as flexibility requirements are contin¬ 
uously on the rise. One may think that simply by using 
standards for input/output (I/O) and well-defined inter¬ 
faces, integration should be just a matter of connecting 
things together and run the system. 

Of course, the object-oriented software solution 
would be to have a gripper class with operations grasp 
and release. That would be appropriate for a robot simu¬ 
lator in pure software, but for integration of real systems 
such encapsulation of data (abstract data types) would 
introduce practical difficulties because: 

• Values are explicit and accomplished by external (in 
this case) hardware, and for testing and debugging 
we need to access and measure them. 

• Online operator interfaces permit direct manipula¬ 
tion of values, including reading of output values. 


The use of functions of object methods (so-called 
setiers and gefiers) would only complicate the pic¬ 
ture; maintaining consistency with the external de¬ 
vices would be no simpler. 

Already in small installations, the complexity of 
system integration becomes apparent. So-called vertical 
integration means integration of low-level devices with 
a high-level factory control system, whereas integration 
of peripherals on (for instance) workcell level is called 
horizontal integration (Fig. 54.30). 

Lack of self-descriptive and self-contained data de¬ 
scriptions that are also useful at the real-time level 
further increases the integration effort since data inter¬ 
faces/conversions typically have to be manually writ¬ 
ten. In some cases, as illustrated in Fig. 54.31, there 
are powerful software tools available for the inte¬ 
gration of the robot user level and the engineering 
level [54.77]. The fully (on all levels) integrated and 
nonproprietary system such as the recently discussed 
Industry4.0 or industrial Internet initiatives is still is 
a challenge, particularly for small and medium-sized 
productions [54.24], 


54.8 Outlook and Long-Term Challenges 


The widespread use of industrial robots in standard, 
large-scale production such as the automotive indus¬ 
try, where robots perform repetitive tasks in well-known 
environments, resulted in the common opinion that in¬ 
dustrial robotics is a solved problem. This opinion was 
underpinned by the robot systems’ impressive auto¬ 
matic performance, based on advanced semiautomatic 
programming and resulting in an unbeatable product 
quality when compared to manual labor. However, 
large-scale production comprises only a minor part of 
the work needed on an industrial scale in any wealthy 
society, especially considering the number of compa¬ 
nies and the variety of applications and processes that 
could and should be automated for productivity, health 
and sustainability reasons. 

Global prosperity and wealth requires resource-effi¬ 
cient and human-assistive robots. The challenges today 
are to recognize and overcome the barriers that are cur¬ 
rently preventing robots from being more widely used, 
especially in small and medium-sized manufacturing. 

Taking a closer look at the scientific and technolog¬ 
ical barriers, we find the following challenges: 

• Human-friendly task specification , including intu¬ 
itive ways of expressing permitted/normal/expected 
variations and errors. There are numerous upcom¬ 
ing and promising techniques for user-friendly 


human-robot interaction (such as speech, gestures, 
manual guidance, and so on), but the focus is still 
on specification of the nominal task rather than 
the complete task which is capable of handling 
foreseen deviations from the nominal case (Part G, 
Robots and Humans). Taking care of these expected 
deviations can account for up to 80% of the total 
programming time. 

• Intuitive human-robot interaction The foreseen 
variations, and the unforeseen variations experi¬ 
enced during robot work, are difficult to manage. 
When instructing a human, he/she has an extensive 
and typically implicit knowledge about the work 
and the involved processes. To teach a robot, it is 
an issue both how to realize what the robot does not 
know, and how to convey the missing information 
efficiently. Furthermore, a person should feel com¬ 
fortable and familiar with the robot’s functionality 
and operation during all operational modes includ¬ 
ing maintenance and error recovery. Research and 
development toward industrial robotics usability 
and ergonomics for increasing acceptance and 
operator efficiency has been of surprisingly low 
activity in the past. 

• Efficient mobile manipulation. Successful imple¬ 
mentations and systems are available for both 
mobility and for manipulation, but accomplished 
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in different systems and using different types of 
(typically incompatible) platforms. A first step 
would be to accomplish mobile manipulation at all, 
combining all degrees of freedom of both the mo¬ 
bile base and arm(s), intelligently exploiting system 
redundancies to achieve a given task. A second step 
providing truly dependable autonomous navigation 
(with adaptive, but predictable understanding of 
constraints, such as frequent environmental changes 
and typical shop-floor dynamics, and appropriate 
sensor-based reactions) dexterous manipulation 
with multifingered grippers, and robust force/torque 
interaction with environments (that have unknown 
stiffness). As a concluding step, all this needs to 
be done with decent performance using reasonably 
priced hardware, and with human-robot interfaces 
according to the previous items. 

• Low-cost components including low-cost actuation. 
Actuation of high-performance robots represent 
about two-third of the overall robot cost, and 
improved modularity often results in a higher 
total hardware cost (due to less opportunities for 
mechatronic optimization). On the other hand, cost- 
optimized (with respect to certain applications) 
systems result in more-specialized components 
and smaller volumes, with higher costs for small 
series production of those components. Since future 
robotics and automation solutions might provide the 
needed cost efficiency for small series customized 
components, we can interpret this as a boot-strap¬ 
ping problem, involving both technical and business 
aspects. The starting point is probably new core 
components that can fit into many types of systems 
and applications, calling for more mechatronics 
research and synergies with other products. 

• Composition of subsystems. In most successful 
fields of engineering, the principle of superposition 
holds, meaning that problems can be divided into 
subproblems and that the solutions can then be su¬ 
perimposed (added/combined) onto each other such 
that the total solution comprises a solution to the 
overall problem. These principles are of key impor¬ 
tance in physics and mathematics, and within engi¬ 
neering some examples are solid-state mechanics, 
thermal dynamics, civil engineering, and electron¬ 
ics. However, there is no such thing for software, 
and therefore not for mechatronics (which includes 
software) or robotics (programmable mechatronics) 
either. Thus, composition of un-encapsulated 
subsystems is costly in terms of engineering effort. 
Even worse, the same applies to encapsulated soft¬ 
ware modules and subsystems. For efficiency, sys¬ 
tem interconnections should go directly to known 
(and hopefully standardized) interfaces, to avoid 


the indirections and extra load (weight, mainte¬ 
nance, etc.) of intermediate adapters (applying to 
both mechanics for end-effector mounting and to 
software). Interfaces can be agreed upon, but the 
development of new versions typically maintains 
backward compatibility (newer devices can be con¬ 
nected to old controller), while including the reuse 
of devices calls for mechanisms for forward com¬ 
patibility (automatic upgrade based on meta in¬ 
formation of new interfaces) to cover the case 
that a device is connected to a robot that is not 
equipped with all the legacy or vendor-specific 
code. 

• Embodiment of engineering and research results. 
Use or deployment of new technical solutions today 
still starts from scratch, including analysis, under¬ 
standing, implementation, testing, and so on. This is 
the same as for many other technical areas, but the 
exceptional wide variety of technologies involved 
with robotics and the need for flexibility and up¬ 
grading makes it especially important in this field. 
Embodiment into components is one approach, but 
knowledge can be applicable to engineering, de¬ 
ployment, and operation, so the representation and 
the principle of usage are two important issues. Im¬ 
proved methods are less useful if they are overly 
domain specific or if engineering is experienced to 
be significantly more complicated. Software is im¬ 
perative, as well as platform and context dependent, 
while know-how is more declarative and symbolic. 
Thus, there is still a long way to go for efficient 
robotics engineering and reuse of know-how. 

• Open dependable systems. Systems need to be open 
to permit extensions by third parties, since there is 
no way for system providers to foresee all upcom¬ 
ing needs in a variety of new application areas. On 
the other hand, systems need to be closed such that 
the correctness of certain functions can be ensured. 
Extensive modularization in terms of hardware and 
supervisory software make systems more expensive 
and less flexible (contrary to the needs of open¬ 
ness). Highly restrictive frameworks and means of 
programming will not be accepted for widespread 
use within short-time-to-market development. Most 
software modules do not come with formal spec¬ 
ification, and there is less understanding of such 
needs. Thus, systems engineering is a key prob¬ 
lem. 

• Sustainable manufacturing. Manufacturing is about 
transformation of resources into products, and pro¬ 
ductivity (low cost and high performance) is a must. 
One aspect of sustainability deals with energy-effi¬ 
ciency of the devices that are used for production. 
Saving energy can constrain the path planning of an 
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individual and a whole fleet of robots, and even shift 
the operation of energy-consuming tasks in periods 
of the day where energy cost are low. A second as¬ 
pect of sustainability deals with recycling of scarce 
resources in terms of materials and noble earths. In 
most cases, this can be achieved by crushing the 
product and sorting the materials, but in some cases 
disassembly and automatic sorting of specific parts 
are needed. There is, therefore, a need for robots 


in recycling and de-manufacturing. Based on future 
solutions to the above items, this is then a robot ap¬ 
plication challenge. 

An overall issue is how both industry and academia 
can combine their efforts such that sound business can 
be combined with scientific research so that future de¬ 
velopments overcome the barriers that are formed by 
the above challenges. 
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SMErobotics project video 

available from http://handbookofrobotics.org/view-chapter/54/videodetails/260 
SMErobot video coffee break (English) 

available from http://handbookofrobotics.org/view-chapter/54/videodetails/261 
SMErobot final project video 

available from http://handbookofrobotics.org/view-chapter/54/videodetails/262 
SMErobot - New parallel kinematic with unique concepts for demanding handling 
and process applications 

available from http://handbookofrobotics.org/view-chapter/54/videodetails/265 
SMErobot D4 The woodworking assistant 

available from http://handbookofrobotics.org/view-chapter/54/videodetails/266 
SMErobotics demonstrator D1 assembly with dual-arm industrial manipulators 
available from http://handbookofrobotics.org/view-chapter/54/videodetails/380 
SMErobotics demonstrator D2 human-robot cooperation in wooden house production 
available from http://handbookofrobotics.org/view-chapter/54/videodetails/381 
SMErobotics demonstrator D3 assembly with sensitive compliant robot arms 
available from http://handbookofrobotics.org/view-chapter/54/videodetails/382 
SMErobotics demonstrator D4 welding robot assistant 

available from http://handbookofrobotics.org/view-chapter/54/videodetails/383 
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55. Space Robotics 


Kazuya Yoshida, Brian Wilcox, Gerd Hirzinger, Roberto Lampariello 


In the space community, any unmanned space¬ 
craft can be called a robotic spacecraft. However, 
Space Robots are considered to be more capable 
devices that can facilitate manipulation, assem¬ 
bling, or servicing functions in orbit as assistants 
to astronauts, or to extend the areas and abilities 
of exploration on remote planets as surrogates for 
human explorers. 

In this chapter, a concise digest of the histori¬ 
cal overview and technical advances of two distinct 
types of space robotic systems, orbital robots and 
surface robots, is provided. In particular, Sect. 55.1 
describes orbital robots, and Sect. 55.2 describes 
surface robots. In Sect. 55.3, the mathematical 
modeling of the dynamics and control using ref¬ 
erence equations are discussed. Finally, advanced 
topics for future space exploration missions are 
addressed in Sect. 55.4. 
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55.1 Historical Developments and Advances of Orbital Robotic Systems 


Key issues in space robots and systems are character¬ 
ized as follows: 

Manipulation - Although manipulation is a basic tech¬ 
nology in robotics, microgravity in the orbital en¬ 
vironment requires special attention to the motion 
dynamics of manipulator arms and objects being 
handled. Reaction dynamics that affect the base 
body, impact dynamics when the robotic hand con¬ 
tacts an object to be handled, and vibration dynam¬ 
ics due to structural flexibility are included in this 
issue. 

Mobility - The ability for locomotion is particularly 
important in exploration robots (rovers) that travel 
on the surface of a remote planet. These surfaces are 
natural and rough, and thus challenging to traverse. 
Sensing and perception, traction mechanics, and ve¬ 
hicle dynamics, control and navigation; all of these 
mobile robotics technologies must be demonstrated 
in a natural untouched environment. 

Teleoperation and Autonomy - There is a significant 
time delay between a robotic system at a work site 
and a human operator in an operation room on the 
earth. In earlier orbital robotics demonstrations, the 
latency was typically 5 s, but can be several tens of 
minutes, or even hours for planetary missions. Teler¬ 
obotics technology is therefore an indispensable 
ingredient in space robotics, and the introduction of 
autonomy is a reasonable consequence. 

Extreme Environments - In addition to the micrograv¬ 
ity environment that affects the manipulator dy¬ 
namics or the natural and rough terrain that affects 
surface mobility, there are a number of issues related 
to extreme space environments that are challenging 
and must be solved in order to enable practical engi¬ 
neering applications. Such issues include extremely 
high or low temperatures, high vacuum or high pres¬ 
sure, corrosive atmospheres, ionizing radiation, and 
very fine dust. 

The first robotic manipulator arm used in the or¬ 
bital environment is the shuttle remote manipulator 
system (SRMS). It was successfully demonstrated in 
the STS-2 mission in 1981 and was operational until 
the end of the shuttle era. This success opened a new 
era of orbital robotics and inspired a number of mis¬ 
sion concepts to the research community. One ultimate 
goal that has been discussed intensively after the early 
1980s is the application to the rescue and servicing 
of malfunctioning spacecraft by a robotic free-flyer or 
free-flying space robot (e.g., ARAMIS report [55.1], 


Fig. 55.1). In later years, manned service missions 
were conducted for the capture-repair-deploy procedure 
of malfunctioning satellites (Intelsat 603 by STS-49, 
for example) and for the maintenance of the Hubble 
space telescope (STS-61, 82, 103, and 109). For all of 
the examples, the Space Shuttle, a manned spacecraft 
with dedicated maneuverability, was used. However, 
unmanned servicing missions have not yet become 
operational. Although there were several demonstra¬ 
tion flights, such as ETS-VII and Orbital Express (to 
be elaborated later), the practical technologies for un¬ 
manned satellite servicing missions await solutions to 
future challenges. 

55.1.1 Robotic Arms for Assistance 
of Human Space Flight 

Space Shuttle Remote Manipulator System 
Onboard the space shuttle, the SRMS, or Canadarm, 
is a mechanical arm that maneuvers a payload from 
the payload bay of the space shuttle orbiter to its de¬ 
ployment position and then releases it [55.2]. It can 
also grapple a free-flying payload, maneuver it to the 
payload bay of the orbiter, and berth it back into the 
orbiter. The SRMS was first used on the second Space 
Shuttle mission STS-2, launched in 1981. Since then, 
it was used more than 100 times during space shuttle 
flight missions, performing such payload deployment 
or berthing as well as assisting human extra vehicular 
activities (EVAs). Servicing and maintenance missions 
to the Hubble space telescope and construction tasks 
of the International Space Station (ISS) have also been 



Fig. 55.1 A conceptual design of telerobotic servicer (af¬ 
ter [55.1]) 
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Fig. 55.2 Space Shuttle remote manipulator system (SRMS) (after [55.2]) 


successfully carried out by the cooperative use of the 
SRMS with human EVAs. 

As depicted in Fig. 55.2, the SRMS arm was 15 m 
long and had 6-degrees of freedom (DOF), comprising 
shoulder yaw and pitch joints, an elbow pitch joint, and 
wrist pitch, yaw, and roll joints. Attached to the end of 
the arm was a special gripper system called the stan¬ 
dard end effector (SEE), which was designed to grapple 
a pole-like fixture (GF) attached to the payload. 

By attaching a foothold at the end point, the arm 
could serve as a mobile platform for an astronaut’s 
EVAs (Fig. 55.3). 

After the Space Shuttle COLUMBIA accident dur¬ 
ing STS-107, NASA outfitted the SRMS with the 
orbiter boom sensor system - a boom containing instru¬ 
ments to inspect the exterior of the shuttle for damage 
to the thermal protection system [55.3]. 

ISS Mounted Manipulator Systems 
The ISS is the largest international technology project, 
with 15 countries making significant cooperative con¬ 
tributions. The ISS is an outpost of human presence in 
space, as well as a flying laboratory with substantial fa¬ 
cilities for science and engineering research. In order 



Fig. 55.3 Space shuttle remote manipulator system 
(SRMS) used as a platform for an astronaut’s extra 
vehicular activity in the shuttle cargo bay 
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to facilitate various activities on the station, there are 
several robotic systems, some of which are already op¬ 
erational, while others are ready for launch. 

The space station remote manipulator system (SS- 
RMS), or Canadarm 2 (Fig. 55.4), was the next gen¬ 
eration of SRMS for use on the ISS [55.4]. Launched 
in 2001 during STS-100 (ISS assembly flight 6A), 
the SSRMS has played a key role in the construction 
and maintenance of the ISS both by assisting astro¬ 
nauts during EVAs and using the SRMS on the Shuttle 
to hand over a payload from a Shuttle to the SS¬ 
RMS. The arm is 17.6m long when fully extended 
and has 7-DOF. Latching end effectors, through which 
power, data, and video can be transmitted to and from 
the arm, are attached to both ends. The SSRMS is 
self-relocatable using an inch-worm like movement 
with alternate grappling of power data grapple fixtures 
(PDGFs), which are installed over the station’s exterior 
surfaces to provide the power, data, and video, as well 
as a foothold. 

As another mobility aid for the SSRMS to cover 
wider areas of ISS, mobile base system (MBS) was 
added in 2002 by STS-111 (ISS assembly flight UF-2). 
The MBS provides lateral mobility as it traverses the 
rails on the main trusses [55.5], 

The special purpose dexterous manipulator 
(SPDM), or Dextre, which is attached at the end of the 
SSRMS, is a capable mini-arm system to facilitate the 
delicate assembly tasks currently handled by astronauts 
during EVAs. The SPDM is a dual arm manipulator 
system, where each manipulator (with 3 m length) has 
7-DOF and is mounted on a one degree-of-freedom 
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Fig. 55.4 Space station remote manipulator system (SSRMS) (af¬ 
ter [55.4]) 


body joint. Each arm has a special tool mechanism 
dedicated to the handling of standardized orbital 
replacement units (ORUs). The arms are teleoperated 
from a Robotic Workstation (RWS) inside the space 
station [55.6]. 

The European Space Agency (ESA) will also pro¬ 
vide a robotic manipulator system for the ISS, the Eu¬ 
ropean robotic arm (ERA), and will be used mainly to 
work on the Russian segments of the station [55.7]. The 
arm is 11.3 m long and has 7-DOF. The basic configu¬ 
ration and functionality are similar to SSRMS [55.8]. 

In lapan, the Japanese experiment module remote 
manipulator system (JEMRMS), as shown in Fig. 55.5, 
was developed by the Japan Space Exploration Agency 
(JAXA) [55.9-11], The arm was launched with the 
STS-124 Mission in 2008 and is now operative on the 
Japanese module of the ISS. JEMRMS comprises two 
components: the main arm, a 9.9 m long, six-degree- 
of-freedom arm, and the small fine arm, a 1.9 m long, 
six-degree-of-freedom arm. 

Unlike the SSRMS or the ERA, the main arm 
does not have self-relocation capability, but is fitted 
with a small fine arm, with which JEMRMS can form 
a serial 12-degree-of-freedom macro-micro manipula¬ 
tor system. After installation, the arm is used to handle 
and relocate the components for the experiments and 
observations on the exposed facility. 

55.1.2 Future-Oriented Space Robot 
Experiments 

R0TEX 

The Robot Technology Experiment (ROTEX) devel¬ 
oped by the German Aerospace Agency (DLR), is one 
of the important milestones of robotics technology in 
space [55.12], as it demonstrated the first (remote) 
ground control of a space robot with of up to 6 s 
round-trip signal delay using geostationary relay satel¬ 
lites. A multisensory robotic arm was flown on Space 
Shuttle COLUMBIA (STS-55) in 1993. Although the 
robot worked inside a work cell on the shuttle, sev¬ 
eral key technologies, such as a multisensory gripper, 
teleoperation from the ground and by the astronauts, 
shared autonomy, and time-delay compensation by use 
of a predictive graphic display were successfully tested 
(Fig. 55.6 and 

Presumably the most spectacular experiment in RO¬ 
TEX was the fully automatic grasping of a small 
free-floating cube with flattened edges by the ground 
computers who evaluated the stereo images from the 
robot gripper, estimated the motion, predicted for the 
above-mentioned 6 s and sent up the commands for 
grasping (l<S2jKJltli!C£Ui). 
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ETS-VII 

Engineering Test Satellite VII (ETS-VII), shown in 
Fig. 55.7, is another milestone in the development of 
robotics technology in space, particularly in the area of 
satellite servicing. ETS-VII was an unmanned space¬ 
craft developed and launched by the National Space De¬ 
velopment Agency of Japan (NASDA, currently JAXA) 
in November 1997. A number of experiments were 
successfully conducted using a 2m long, six-degree- 
of-freedom manipulator arm mounted on its carrier 
satellite. 

The mission objective of ETS-VII was to test free- 
flying robotics technology and to demonstrate its utility 
in unmanned orbital operation and servicing tasks. The 
mission consisted of two subtasks: autonomous ren¬ 
dezvous/docking (RVD) and a number of robot experi¬ 
ments (RBT). The robot experiments included: (1) tele¬ 
operation from the ground with a large time delay; (2) 
robotic servicing task demonstrations such as ORU ex¬ 
change and deployment of a space structure; (3) dynam¬ 
ically coordinated control between the manipulator re¬ 
action and the satellite attitude response; and (4) capture 
and berthing of a cooperative target satellite [55.13]. 

The communication time delay due to radio propa¬ 
gation (speed-of-light) is relatively small, for example 
0.25 s for a round trip to geostationary Earth orbit 
(GEO). However, to have a global coverage of com¬ 
munication in low Earth orbit (LEO) operations, the 
signals are transmitted via multiple nodes including 
data relay satellites located at GEO and ground sta¬ 
tions. This makes the transmission distance longer, and 
even more additional delays are added at each node. As 
a result, the cumulative delay becomes some seconds, 
actually 5—7 s in case of ETS-VII mission, more or less 
the same as in the ROTEX experiment. However, it is 
important to state that these delays have been unneces¬ 
sarily long as the optimal communication infrastructure 
was (and is still) missing. Figure 55.8 shows that for 
a low orbit robot satellite (with typically 1.5 h orbital 
period) half of the orbit (i. e., around 45 min) would 
have communication from the ground station via one 
relay satellite yielding approximately 600 ms round trip 
delay (including computational delays). For a robot in 
geostationary orbit having permanent communication 
to the ground station, the round trip delay would be only 
300 ms. 

In ETS-VII, opportunities for academic experi¬ 
ments were also opened to Japanese universities and 
European institutions (e.g., DLR and ESA), and impor¬ 
tant flight data were obtained that validate the concepts 
and theories for free-flying space robots [55.14,15]. 
As an example, DLR performed experiments aiming 
at demonstrating robot control methods for performing 
ORU exchange tasks (see for example |43>M3H3tE3l) 
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Fig. 55.5 (a) Japan Experiment Module (JEM) on the ISS and 
(b) the JEMRMS manipulator system 


and dedicated satellite attitude control via swimming or 
waiving motions of the robot arm. 

Ranger 

Ranger is a teleoperated space robot being devel¬ 
oped at the University of Maryland, Space Systems 
Laboratory [55.16]. Ranger consists of two seven- 
degree-of-freedom manipulators with interchangeable 
end effectors to perform such tasks as changeout of or¬ 
bital replacement units (ORUs) in orbit. Also discussed 
was the changeout of the electronics controller unit 
(ECU) of the Hubble space telescope, which previously 
required human EVA. A number of tests and demon- 
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Fig. 55.6 ROTEX manipulator arm onboard the Spacelab 
D2 mission, the first remotely controlled space robot 



Fig. 55.7 Japanese Engineering Test Satellite ETS-VII 


strations for servicing missions have been conducted at 
the University of Maryland Neutral Buoyancy Facility 
(Fig. 55.9). Originally designed for a free-flying flight 
experiment, Ranger had been redesigned for a shuttle 
flight experiment, but ultimately was not manifested on 
a shuttle flight. 

Orbital Express 

The Orbital Express Space Operations Architecture 
program is a DARPA program developed to validate 
the technical feasibility of robotic on-orbit refueling 
and reconfiguration of satellites, as well as autonomous 
rendezvous, docking, and manipulator berthing [55.17], 
The system consists of the autonomous space trans¬ 
port robotic operations (ASTRO) vehicle, developed by 
Boeing Integrated Defense Systems, and a prototype 
modular next-generation serviceable satellite, NextSat, 
developed by Ball Aerospace. The ASTRO vehicle is 
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Fig. 55.8 Minimal Roundtrip signal propagation delay to 
robots in LEO is around 0.6 s, to robots in GEO around 
0.3 s 



Fig. 55.9 Neutral buoyancy test of the Ranger telerobotic 
shuttle experiment 


equipped with a robotic arm to perform satellite capture 
and ORU exchange operations (Fig. 55.10). 

After its launch in March 2007, various mission 
scenarios were conducted. These scenarios include (1) 
visual inspection, fuel transfer, and ORU exchange on 
NextSat using ASTRO’s manipulator arm when both 
spacecrafts are connected, (2) separation of NextSat 
from ASTRO, orbital maneuvers by ASTRO, and fly- 
around, rendezvous, and docking with NextSat, and (3) 
capture of NextSat using ASTRO’s manipulator arm. 

These scenarios were successfully completed by 
July 2007, with ASTRO’s onboard autonomy using on¬ 
board cameras and advanced video guidance system. 
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Fig. 55.10 Orbital express flight mission configuration 


ROKVISS and Delayed Teleoperation 
As a precursor demonstration of the planned German 
Orbital Servicing demonstration mission DEOS (see 
below), the German Aerospace Agency (DLR) has de¬ 
veloped and flown a 0.5 m long, two-degree-of-freedom 
manipulator arm with a dedicated test bench, called 
robotic components verification on the ISS (ROKVISS) 
(Fig. 55.11) [55.18]. 

Using the 7—8 min contact window of the ISS over¬ 
flight through a dedicated real-time space link, resulting 
in a roundtrip delay as small as 20 ms, this was the first 
high-fidelity telepresence system in the history of space 
flight that allowed force feedback teleoperation. A large 
amount of data documenting the evolution of electrical 
and mechanical properties of the robot (such as sensor 
accuracy, friction, and motor parameters) has been col¬ 
lected during the six year mission. 

ROKVISS was launched by an unmanned Russian 
progress transport vehicle in 2004 and installed on the 
outer platform of the Russian segment of the station in 
early 2005 (l<s«M£*iil). 

After six years, the system was still fully functional, 
even though it had undergone only low-cost qualifi¬ 
cation, i. e., most of the electronic components were 
off-the-shelf and the system as a whole had been qual¬ 
ified by radiation, vibration, and temperature tests. The 
long-time verification of these joints for outer space has 
been a second main goal of ROKVISS (l<sa>)Kilil£Itil). 
Toward the end of the mission, ROKVISS was teleop- 
erated from the private home of the project leader using 
standard internet communication to DLR’s professional 
ground station. Round-trip delay went then up to around 
400 ms still allowing telepresence with force-reflection. 
At the end of 2011, ROKVISS was brought back to 
Earth for completion of verification and validation tests 



Fig. 55.11 ROKVISS - 6 years on ISS 


Although the numbers of joints was small, chal¬ 
lenging experiments of telepresence were conducted, 
in which human operators from ground teleoperated 
the arm using a force feedback joystick and stereo vi¬ 
sion images. The ROKVISS arm, on the other hand, 
was equipped with joint torque sensors, allowing full 
impedance control and advanced bilateral control tech¬ 
niques. The secondary goal of the experiment was the 
space qualification of the joint drives, which are the key 
components of DLR’s torque-controlled lightweight 
robots [55.19]. 

No electrical or mechanical damages were visible, 
and the main observation was that joint friction had 
nearly doubled in vacuum on ISS, but went back to its 
original value when the arm was back on earth again. 
In the framework of ROKVISS and the accompanying 
investigations, important basic clarifications of the gen¬ 
eral telepresence potentials and prerequisites could be 
achieved. 

Already in the early days of space telerobotics as 
pushed forward by JPL, there have been estimates that 
humans might be able to master signal delays of up to 
1 s in the visual system (i. e., looking at delayed images) 
and up to 500 ms with the haptic system from the pro¬ 
prioceptive point of view. 

The interesting, fairly recent result is here [55.20, 
21], where feasibility of force-reflecting teleoperation 
with communication delays as high as 650 ms has been 
proven. By using the so-called time domain passiv¬ 
ity control approach, the mechanical energy of the 
system is observed and controlled in real time, such 
that the system is passive for any given communi¬ 
cation channel characteristics, including varying time 
delays and packet loss. These results have been veri¬ 
fied with a unique teleperesence experiment, where two 
torque-controlled light weight arms located in Oberp- 
faffenhofen (Germany) were connected through a real 
relayed communication link that used the geostationary 
satellite ARTEMIS, ground station communication an- 
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Fig. 55.12 Force-reflecting telepresence demos at DLR 


tennas located in Garching (Germany) and a data mirror 
in Redu (Belgium) [55.22], 

Recently, telepresence experiments with a fairly 
complex multidegree-of-freedom (DOF) system have 
been performed [55.23]. The system consisted of the 
DLR’s Space JUSTIN humanoid robot and a human- 
machine interface based on two torque-controlled light 
weight arms as a force reflecting hand controller 
(haptic device) (Fig. 55.12 and l<gf'ii >»**«) Com¬ 
plex tasks that required with high levels of dexter¬ 
ity, such (un)screwing with a screwdriver and solder¬ 
ing, were performed with time delays up to 500 ms 

It is therefore concluded that teleoperation with 
force feedback is possible over the whole earth orbit. 

The DEOS Demonstration Project 
The German space robotics-demonstration mission 
DEOS, based on the ROKVISS-joint torque-controlled 
arm technology, aims at the demonstration of the matu¬ 
rity and availability of key technologies as needed for 
an orbit servicing. DEOS is comprised of two satel- 



Fig. 55.13 DEOS - orbital servicing experiment with 
client {left) and servicer {right) 


lites, servicer and client (Fig. 55.13). Its main goal 
is to find, approach, and capture an uncontrolled and 
uncooperative satellite in LEO (l<sa>3di2DEEEI). After 
successful capturing some typical repair and mainte¬ 
nance tasks are planned to be demonstrated before the 
servicing satellite enters the atmosphere for a controlled 
descent together with the captured satellite. Project de¬ 
velopments are currently being pursued to advance the 
technological readiness level, in view of future mis¬ 
sions. Developments also include adaptations to Active 
Debris Removal missions in study, such as e.Deorbit 
(ESA), for the deorbiting of the ENVISAT satellite. 

Robonaut and JUSTIN 

Humanoid (human-like) robots have been developed 
to conduct human-compatible dexterous tasks. NASA’s 
Robonaut and DLR’s JUSTIN are such representative 
examples. These robots are elaborated in Sect. 55.4.2. 


55.2 Historical Developments and Advances of Surface Robotic Systems 


The research on surface exploration rovers began in 
the mid-1960s, with an initiative (that never flew) 
for an unmanned rover for the Surveyor lunar lan¬ 
ders and a manned rover (Moon buggy) for the hu¬ 
man landers in the United States. In the same period, 
research and development began for a teleoperated 
rover named Lunokhod in the Soviet Union. Both 
the Apollo-manned rover and the Lunokhod-unmanned 
rover were successfully demonstrated in the early 1970s 
on Moon [55.24], In the 1990s, the exploration tar¬ 
get had expanded to Mars, and in 1997, the Mars 
Pathfinder Mission successfully deployed a microrover 
named Sojourner that safely traversed the rocky field 
adjacent to the landing site by autonomously avoid¬ 
ing obstacles [55.25, 26], Following this success, today. 


autonomous robotic vehicles are considered indispens¬ 
able technology for planetary exploration. The twin 
Mars Exploration Rovers, Sprit and Opportunity, were 
launched in 2003, and have had remarkable success in 
terms of remaining operational in the harsh environ¬ 
ment of Mars for over three years. Each has traveled 
more than 5000 m and has made significant scientific 
discoveries using on-board instruments [55.27,28]. 

55.2.1 Teleoperated Rovers 

The first remotely operated robotic space surface ve¬ 
hicle was Lunokhod (Fig. 55.14) [55.24], Lunokhod 1 
landed on the Moon on November 17, 1970 as a payload 
on the lander Luna-17, and Lunokhod 2 landed on the 
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Fig. 55.14 Lunokhod 


Moon on January 16, 1973. Both were 8-wheeled skid- 
steered vehicles having a mass of about 840 kg, where 
almost all the components were in a pressurized bathtub 
thermal enclosure with a lid that closed over the tub to 
allow it to survive the deep cold (« 100 K) of the long 
lunar nights using only the heat emitted by small pel¬ 
lets of radioisotope. On the inside of the lid were solar 
arrays which recharged batteries during the day as re¬ 
quired to maintain operation of the vehicle. Lunokhod 

1 operated for 322 Earth days, traversing over 10.5 km 
during that period, and returned over 20000 TV im¬ 
ages, 200 high-resolution panoramas, and the results of 
more than 500 soil penetrometer tests and 25 soil analy¬ 
ses using its x-ray fluorescence spectrometer. Lunokhod 

2 operated for about 4 months, having traversed more 
than 37 km, with the mission officially terminated on 
June 4, 1973. It has been reported that Lunokhod 2 was 
lost prematurely when it began sliding down a crater 
slope and hasty commands were sent in response which 
ultimately caused end-of-mission. 

Each of the eight wheels on the Lunokhod vehi¬ 
cles were 0.51 m in diameter and 0.2 m wide, giving an 
effective ground pressure of less than 5 kPa based on 
an assumed sinkage of 3 cm. Each wheel had a brush- 
type DC motor, a planetary gear reduction, a brake, and 
a dis-engagement mechanism allowing it to free-wheel 


in the event of some problem with the motor or gears. 
The mobility commands to the vehicle included two 
speeds forward or backward, braking, and turning to the 
right or the left either while moving or in place. 

The vehicles had both gyroscope and accelerome¬ 
ter-based tilt sensors which could automatically stop the 
vehicle in the event of excessive tilt of the chassis. Typ¬ 
ical mobility commands specified a time duration over 
which the motors would run, and then stop. Precision 
turning commands specified the angle through which 
the vehicle should turn. These commands were termi¬ 
nated when the specified turn angle had been reached 
according to the heading gyroscope. Odometry was de¬ 
termined by a ninth small wheel which was unpowered 
and lightly loaded and used only to determine over-the- 
ground distance. There was an on-board current over¬ 
load system, and motor currents, pitch and roll mea¬ 
surements, distance traversed, and many component 
temperatures were telemetered to the ground operators. 

The Lunakod crew consisted of a driver, a navigator, 
a lead engineer, an operator for steering the pencil- 
beam communication antenna, and a crew commander. 
The driver viewed a monoscopic television image from 
the vehicle, and gave the appropriate commands (turn, 
proceed , stop , or back up) along with their associated 
parametric value in terms of duration or angle. The nav¬ 
igator viewed displays of telemetry from the vehicles’ 
course gyroscope, gyrovertical sensor, and odometer, 
and was responsible for calculating the trajectory of the 
vehicle and laying out the route to be followed. Thus, 
the driver was responsible for vehicle stability about its 
center of mass and the navigator was responsible for 
the trajectory of that center of mass. The lead engineer 
(assisted by many specialists as required), was respon¬ 
sible for assessing the health of the on-board systems. 
The lead engineer provided both routine updates on 
energy supply, thermal conditions, etc., as well as possi¬ 
ble emergency alerts such as extreme motor currents or 
chassis tilt. The pencil-beam antenna operator oversaw 
the functioning of an independent ground-based closed- 
loop control system that servoed the antenna to always 
point at Earth, independent of the vehicle motion. The 
crew commander supervised the implementation and 
execution of the overall plan, gave any detailed com¬ 
mands for making actual contact with the surface (e.g., 
by the penetrometer) and also could override any com¬ 
mand to the vehicle as he viewed the same information 
as the driver. 

The entire driving system was tested extensively 
prior to the Lunokhod 1 mission at a lunodrome hav¬ 
ing simulated lunar terrain which proved to be more 
challenging than that actually encountered during the 
Lunokhod 1 mission. Despite this, the operators of 
Lunokhod 1 said they encountered a dangerous situ- 
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ation (unforeseen entrance into a crater, rolling onto 
a rock, etc.) slightly more often than once per kilometer. 
This was attributed to inadequate driving experience, 
the modest quality of the television images, and the 
poor illumination conditions on the Moon. The driving 
direction was often selected primarily to give the best 
images; even so, the operators reported fictitious dan¬ 
gers caused by varying illumination conditions. In the 
first three months (lunar days) of operation, the vehicle 
traversed 5224 m in 49 h of driving using 1695 driving 
commands, including about 500 turns. Sixteen signals 
were sent for protection against excessive tilt during 
that time; approximately 140 craters were traversed at 
maximum slope angles of 30°. 

With the approximately 2.6 s speed-of-light delay, 
the operators stated that control experience confirmed 
the desirability of movement in a starting-stopping 
regime with mandatory stopping each few meters. The 
soil properties were found to differ substantially even in 
terrain sectors not very distant from one another. The 
soil penetrometer determined that the upper layer of 
regolith varied from a stiffest where the penetrometer 
required about 16 kg (Earth weight) of force to pen¬ 
etrate about 26 mm, to a weakest measurement where 
only 3 kg of Earth weight caused a penetration of about 
39 mm. The cone penetrometer had a base diameter of 
50 mm and a cone height of 44 mm. Thus the upper 
layer of regolith had a rate of increase of loadbearing 
strength ranging from about 400kPam~ 1 for the weak¬ 
est soil to about 3MPam~ 1 for the stiffest soil. Crater 
walls and the immediate ejecta blanket around craters 
generally exhibited the weakest soil. Below 5—10 cm of 
penetration depth, the regolith generally became rapidly 
stiffer. The mean value of wheel slippage for the first 
three lunar days was about 10%. On horizontal terrain, 
the slippage ranged from zero to 15% depending on 
the surface irregularities and ground inhomogeneity. On 
crater slopes, the slip increased to 20—30%. The spe¬ 
cific resistance of the Lunokhod wheels was generally 
in the range of 0.05—0.25, while the specific free trac¬ 
tion (the ratio of traction to weight) was in the range of 
0.2—0.41. The crater distribution in the area explored 
by Lunokhod 1 was found to be closely approximated 
by the formula N(D) = AIX s , where N(D) is the num¬ 
ber of craters larger than diameter D meters per hectare 
of lunar surface, A is a scale factor found to be about 
250, and S is the distribution exponent, found to be 
about 1.4 [55.24], 

55.2.2 Autonomous Rovers 

In the mid-1960s, research began on a lunar rover at the 
US Jet Propulsion Laboratory (JPL) in Pasadena, Cal¬ 
ifornia, when it was proposed to put a small rover on 


the Surveyor lunar landers. These landers were led by 
JPL (based on a system-level contract to Hughes), and 
were designed to land softly on the Moon to establish 
the safety of such landing prior to the Apollo landers 
with humans aboard. At the time it was speculated (no¬ 
tably by T. Gold) that the Moon might be covered in 
a thick layer of soft dust that would swallow any lan¬ 
der. In 1963, JPL issued a contract to build a small 
rover concept prototype in support of the Surveyor 
program to the General Motors Defense Research Lab¬ 
oratories in Goleta, California. That GM facility had 
recently hired Bekker, who was considered the father 
of off-road locomotion, having written several seminal 
textbooks on the subject, and having introduced many 
of the key concepts relating soil properties to off-road 
vehicle performance that are still used today [55.29, 30] 
(Sect. 55.3.12). 

Bekker and his team proposed an articulated 6- 
wheeled vehicle based on a novel 3 cab configuration 
with an axial spring-steel suspension. This vehicle ex¬ 
hibited remarkable mobility, being able to climb ver¬ 
tical steps up to 3 wheel radii high, and crossing 
crevasses 3 wheel radii wide. Notable people working 
with Bekker were Larenc Pavlics, who went on to lead 
the development of the mobility system for the Apollo 
lunar rover (under contract from Boeing), and Lred Jin- 
dra, who developed the underlying equations describing 
the mobility of the 6-wheeled articulated vehicle that 
were later used by Don Bickler in conceiving of the 
rocker bogie chassis used on Sojourner and the Mars 
Exploration Rovers. Bekker and his team proposed the 
6 wheeled articulated vehicle after experimenting with 
many types of vehicles, including multitracked vehi¬ 
cles, screw-type vehicles (for fine powdered terrain), 
and others. The 6-wheeled vehicle demonstrated in 
scale-model testing superior performance in both soft 
and rocky terrain. 

They built and delivered two vehicles that were 
about 2 m long with approximately 0.5 m wheel diam¬ 
eters. Those vehicles were used in testing throughout 
the 1960s and early 1970s to conduct simulated opera¬ 
tions to determine how such vehicles could actually be 
used on the Moon. One key issue was that the speed-of- 
light round trip from the Moon (about 3 s) precluded 
direct driving of the vehicle. Perhaps most annoying 
was the fact that, during vehicle motion, the highly di¬ 
rectional radio antenna used to communicate with Earth 
would lose its pointing, and so communications would 
briefly be lost. This meant that operators driving the 
rover would be confronted with a series of still im¬ 
ages, instead of a stream of moving images. It was 
quickly realized that much of an operator’s situation 
awareness and depth perception needed to drive a ve¬ 
hicle with a monocular camera comes from motion. It 
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was very difficult to drive from frozen monocular im¬ 
ages. A crude form of stereo was incorporated where 
the camera mast was raised and lowered slightly and 
the operator could switch back and forth between the 
two views. 

Following the successful landing of several of the 
Surveyor spacecraft, and the discovery that all landing 
sites seemed to have relatively firm soil, it was con¬ 
cluded that the Surveyor lunar rover was not needed. 
As a result, the prototype was used for research into the 
early 1970s, and subsequently restored for use again in 
research in the 1980s, becoming the first vehicle to be 
outfitted with waypoint navigation of the sort later used 
on the Sojourner and MER missions. 

About the time Viking was conceived and devel¬ 
oped, JPL began the 1984 Mars rover effort. (1984 
was an energetically favorable launch opportunity from 
Earth to Mars, and the next likely major mission oppor¬ 
tunity after Viking.) Two testbed vehicles were devel¬ 
oped, a software prototype and a hardware prototype. 
The software prototype had a Stanford arm, designed 
by Vic Scheinman (who went on to design the Unima- 
tion PUMA arm and many other famous early robotic 
devices). This was the only 1.5-scale Stanford arm ever 
built. Lewis and Bejczy became well-known in robotics 
for solving the kinematics of this arm, one of is not 
the first full kinematics ever done in robotics up to that 
time, e.g., [55.31]. A stereo pan-tilt head was imple¬ 
mented and equipped with the first solid-state cameras 
to become available. A number of very important works 
were published in the 1977 International Joint Confer¬ 
ence on artificial intelligence, e.g., [55.32, 33]. The first 
hand-eye-locomotion coordination was done with this 
vehicle, where a rock was designated in a stereo image, 
and the vehicle maneuvered autonomously to a point 
where the arm could reach out and pick up the rock. 
One of the first demonstrations of pin in hole insertion 
and other dexterous manipulations were also done with 
this system in the 1970s. 

The hardware prototype was built using elastic loop 
wheels made by Lockheed [55.34]. The vehicle was bat¬ 
tery powered and controlled via a handheld RC unit of 
the type used by hobbyists. 

In late 1982, JPL had a contract with the US Army 
to study the use of robotic vehicles in support of the 
US Army. During this study, Brian Wilcox at JPL pro¬ 
posed a technique to reduce the need for a real-time 
video link or high-bandwidth communication channel 
between the vehicle and the operator. This technique 
(which became known as computer-aided remote driv¬ 
ing, or CARD) [55.35] required the transmission of 
a single stereo image from the vehicle to the operator, 
so the operator could designate waypoints in that image 
using a 3-D cursor. By use of a single stereo image in¬ 


stead of a continuous stream of monocular images, the 
amount of information that needed to be transmitted by 
the vehicle was reduced by orders of magnitude. JPL 
first demonstrated CARD on the resurrected surveyor 
lunar rover vehicle (SLRV, which had been painted 
baby-blue and so became known as the Blue Rover, 
Fig. 55.15a), and later on a modified Humvee. During 
field tests in the Mojave desert in 1988, CARD was 
demonstrated on the Humvee with path designations 
of 100 m per stereo image, and with time-to-designate 
each path of only a few seconds. 

As the CARD work was ongoing, an internally 
funded effort at JPL demonstrated a concept called 
semiautonomous navigation (SAN). This concept in¬ 
volved humans on Earth designating global paths using 
maps of the sort that could be developed from orbiter 
imagery, and then having the vehicle autonomously re¬ 
fine and execute a local path that avoids hazards. The 
moderate success of that effort led to a NASA funded 
effort, leading to the development of a new vehicle, 
called Robby (Fig. 55.15b). Robby was a larger vehicle 
that could support the on-board computing and power 



Fig.55.15a,b Six-wheel articulated body rovers developed 
by JPL (a) SLRV and (b) Robby 
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needed for untethered operation. (The SLRV had been 
tethered to a VAX 11/750 minicomputer over a 1500 
foot tether during arroyo field testing of CARD.) For the 
first time (in 1990), an autonomous vehicle had made 
a traverse through an obstacle field that was faster than 
a rover could have done on Mars using human path des¬ 
ignation done on Earth. 

However, Robby had a severe public relations prob¬ 
lem - it was perceived as too large. Of course, none 
of the computers or power systems had been miniatur¬ 
ized or lightweighted - it was composed entirely of the 
lowest-cost components that could do the job. How¬ 
ever, because it was the size of a large automobile, 
observers and NASA management got the impression 
that future rovers would be car-sized or even truck¬ 
sized vehicles. This was compounded by the Mars rover 
sample return (MRSR) study done by JPL in the late- 
1980s which suggested a mass for the rover of 882 kg. 
An independent study of the MRSR study by Science 
Applications International, Inc. (SAIC) estimated the 
overall cost at $13B. When word of this outrageous 
price tag filtered around NASA Headquarters and into 
the Congressional Staff, MRSR was summarily killed. 
Robby died along with it. At about the same time, 
NASA funded Carnegie Mellon University to develop 
Ambler (Fig. 55. 16), a large walking robot that was able 
to autonomously choose safe footfall locations, also as 
a testbed Mars Rover [55.36,37]. Ambler had a simi¬ 
lar public-relations problem, being about the same mass 
as Robby. that the NASA management community was 
very skeptical that such large systems could affordably 
be flown to Mars. Both Robby and Ambler had all-on¬ 
board power and computing systems, which at that time 
were not sufficiently miniaturized to make autonomous 
rovers credible for actual flight missions. Moore’s law 



Fig. 55.16 Ambler 


was not only causing the computing technology to 
become miniaturized at a high rate, but also the en¬ 
ergy required per computing instruction was dropping 
rapidly. This meant that early systems devoted most of 
their power to computing rather than to motive power. 
Later systems, such as the Mars exploration rovers, 
have a more nearly equal balance between power for 
mobility and power for computation. Future systems 
will presumably devote the majority of their power to 
mobility as opposed to computation. 

Soon thereafter, the Mars Environmental Survey 
(MESUR) mission set was proposed, as a lower cost 
alternative to a sample return mission. The MESUR 
Pathfinder mission was proposed as a first test of what 
was envisioned as a network of 16—20 surface stations 
to provide global coverage of Mars. A small rover was 
proposed to the Mars Science Working group [55.38, 
39]. A very short-term development effort culminated 
in a demonstration in July 1992 of a 4 kg rover that 
could move to directed points on the surface nearby 
a lander using stereo designation of waypoints in a 3-D 
display of frozen images taken from a lander mast 
camera pair (Fig. 55.17). This demonstration was suf¬ 
ficiently successful that a similar rover was manifested 
for the Mars Pathfinder mission. The Pathfinder rover 
(Fig. 55.18) was later named Sojourner, and became 
the first autonomous vehicle to traverse the surface of 
another planet, using a hazard detection and avoid¬ 
ance system to move safely between waypoints through 
a rockfield [55.25,26]. The hazard detection system 
avoided obstacles, and also was used to position the 
vehicle accurately in front of rocks. Sojourner oper¬ 
ated successfully for 83 Mars days (until the failure of 
the lander, which was acting as a communications re¬ 
lay between the rover and Earth). Sojourner examined 
approximately a dozen rock and soil samples with its 
Alpha-Proton-x-ray spectrometer, which gives the ele¬ 
mental composition of the rocks and soil. The success 
of Sojourner led directly to the decision to build the 
twin Mars exploration rovers launched in 2003. Both 
Sojourner and the subsequent Mars exploration rovers 
Spirit and Opportunity, and the Mars Science Labora¬ 
tory Curiosity, use waypoint designation in stereo im¬ 
ages by the human operator together with autonomous 
hazard detection and avoidance to keep the rovers safe 
if they should wander off the designated path. 

During the 1992—1993 summer season in Antarc¬ 
tica, the Dante I robot, built by Carnegie-Mellon Uni¬ 
versity and funded by NASA, attempted to rappel into 
the caldera of the active volcano Mt. Erebus. Dante was 
a walking robot, and was the first serious attempt to 
make a robot rappel down a grade that was too steep to 
traverse using purely frictional contact. Unfortunately, 
the extreme cold (even in the summer) compounded 
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Fig. 55.17 Rocky 4 



Fig. 55.18 The Pathfinder rover. Sojourner 


by human error caused a kink in the fiber-optic um¬ 
bilical to snag going through an eyelet, breaking the 
high-bandwidth fiber communications on which the 
system depended. The fiber could not be repaired in 
the field, and so the mission was aborted. Undaunted, 
in the summer of 1994, Dante II (Fig. 55.19) made 
a successful rappel into the caldera of Mt. Spur in 
Alaska, exploring the active vents on the crater floor 
in a way that would be unsafe in the extreme if done 
with human explorers. The Dante robot series demon¬ 
strated that rappelling, especially when combined with 
legged locomotion, allows robots to conduct explo¬ 
ration to extremely hazardous sites in ways that humans 
cannot. 

In 1984, NASA started the Telerobotics Research 
program [55.40,41]. This program demonstrated var¬ 
ious aspects of on-orbit assembly, maintenance, and 
servicing. Some highlights of this activity were the 
automated tracking and grappling of a free-spinning 
satellite (suspended with a counterweight and gimbal 



Fig. 55.19 Dante II at Mt. Spur in Alaska 


for realistic reactions under external forces), connection 
of a flight-like fluid coupler, and many busy box func¬ 
tions such as door opening, threaded fastener mating 
and demating, use of power tools, dual-arm manipu¬ 
lation of a simulated hatch cover and flexible thermal 
blanket, etc., by various control approaches ranging 
from force-reflecting teleoperation to fully autonomous 
sequences. This activity ended in about 1990. 

55.2.3 Research Systems 


There have been many mobile robots built by govern¬ 
ment, university, and industrial groups whose objective 
was to develop new technologies for planetary surface 
exploration, or to excite students or young engineers 
about the possibilities in that area. Carnegie Mel¬ 
lon University developed the Ambler, Dante, Nomad, 
Hyperion, Zoe, and Icebreaker robot series. The Jet 
Propulsion Laboratory (JPL), Draper Labs, MIT, San- 
dia National Lab, and Martin-Marietta (later Lockheed- 
Martin) each built more than one planetary surface 
robot testbed. The Marsokhod chassis built by VNII 
Transmach of St. Petersburg, Russia was used by re¬ 
search groups there and also in Toulouse (LAAS and 
CNRS) [55.42,43] as well as the NASA Ames Re¬ 
search Center and McDonnell Douglas Corporation 
(later part of the Boeing Company) in the US. 
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These research platforms have been used for two ba¬ 
sic avenues of research. One avenue is to perfect safe 
driving techniques on planetary surfaces, despite the 
speed-of-light latencies inherent in robotic exploration 
of the planets. This includes the waypoint navigation 
technology developed at JPL in the 1980s, where frozen 
stereo images are used to plan a possibly lengthy se¬ 
ries of waypoints or activity sites, and then executed 
with various sorts of reflexive hazard avoidance or saf- 
ing techniques, such as used on the Sojourner rover 
on Mars in 1997. The other is to develop higher level 
autonomy for improved science data return or mis¬ 
sion robustness. Technologies in this latter category 
include mission planners that attempt to optimize routes 
and activity sequences based on time, limits to peak 
power, total energy, expected temperature, illumina¬ 
tion angles, availability of communications, and others. 



Fig. 55.20 The Mars exploration rovers. Spirit and Oppor¬ 
tunity, with a manipulator arm in front 



Fig. 55.21 Rocky 7 


Automated classification of possible science targets 
based on clustering of spectral data, figure-ground seg¬ 
mentation of rocks, and other approaches have been 
attempted with some success. At the time of this writ¬ 
ing, some of these technologies have been uploaded to 
the twin Mars exploration rovers Spirit and Opportu¬ 
nity (Fig. 55.20) [55.27], including automated detection 
of temporary events of scientific interest such as dust- 
devils and clouds [55.44]. The Spirit rover was lost 
when it broke through a surface crust into a small 
crater filled with fine dust in late 2009, with attempts 
to free it continuing until mid 2010. The Mars Sci¬ 
ence Laboratory Curiosity was landed in August 2012 
and seeks to explore Mt. Sharpe in the center of Gale 
Crater. 

The ESA with a number of contractors, e.g., AS- 
TRIUM and DLR has been working for several years 
on a 6-wheeled Mars rover ExoMars that is supposed 
to be flown after 2018 as well as on a next lunar lan¬ 
der NLL Rover, but it is not finally clear, when and in 
which configuration the mission might be realized. 

55.2.4 Sensing and Perception 

In the 1980s, most planetary rover sensing research 
was based on laser ranging or stereo vision. Stereo 
vision was too computationally intensive for early low- 
power, radiation-hard processors, so the Sojourner Mars 
rover used a simple form of laser ranging to determine 
which areas were safe to traverse. Between the launch 
of Sojourner (1996) and the Mars exploration rovers 
(2003) sufficient progress had been made in radiation- 
hardened flight processors that stereo vision was used 
for hazard detection on MER [55.45], mostly in experi¬ 
ments conducted with the Rocky-7 rover (Fig. 55.21). 
This allowed much larger numbers of range points 
to be incorporated into the hazard-detection algorithm 
(thousands of points, instead of the 20 discrete range 
points used by Sojourner). Perception of hazards on 
Sojourner was based on simple computations of av¬ 
erage slope and roughness over the 4x5 array of 
range measurements, as well as the maximum height 
differences. 

The two MER rovers and the MSL rover use a more 
sophisticated evaluation of the safety of the rovers 
along a large number of candidate arcs from its cur¬ 
rent location. Many other algorithms for perception of 
terrain hazards have been used with reasonable suc¬ 
cess by various organizations. Today it is probably fair 
to say that the unsolved problems lie not in the area 
of geometric hazards (e.g., hazards that can be eval¬ 
uated completely based on accurate knowledge of the 
shape of the terrain) but rather in the area of nonge¬ 
ometric hazards (e.g., hazards where uncertainties in 
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the load-bearing or frictional properties of the terrain 
determine the safety of a proposed traverse). Accu¬ 
rate estimation of load-bearing or friction properties of 
terrain by remote sensing is a very challenging task 
that will not be completely solved anytime soon, if 
ever. 

55.2.5 Estimation 

Most estimation for planetary surface exploration re¬ 
lates to the internal state of the robot, or its position, 
pose and kinematic configuration with respect to the en¬ 
vironment. Internal state sensors such as encoders on 
any active or passive articulations in the vehicle are 
used, along with a kinematic model and inertial sen¬ 
sors such as accelerometers and gyroscopes, to estimate 
the pose of the vehicle in inertial space. Perceptual al¬ 
gorithms such as surface reconstruction from clouds of 
range-points as developed by stereo vision put terrain 
geometry estimates into this same representation. Head¬ 
ing in inertial space is generally the most difficult to 
reliably estimate, due to the lack of navigation aids such 
as the global positioning system, or any easily mea¬ 
sured heading reference such as a global magnetic field. 
Integration of rate-gyro data is used to maintain local 
attitude during motion while accurate estimation of the 
rotation axis for Mars is possible by long integrations of 
3 axis rate-gyro data while the vehicle is stopped. Sim¬ 
ilar approaches are probably not feasible for the Moon, 
because of its slow rotation rate. Imaging of the solar 
disk or constellations of stars at precisely known times 
can be combined with stored models of the rotation of 
the planet to allow accurate estimation of the complete 
pose of the vehicle in inertial space. Kalman filtering or 
related techniques are generally employed to reduce the 
effects of measurement noise. 


55.3 Mathematical Modeling 

Broadly speaking, both on-orbit manipulators and sur¬ 
face mobile robots are considered to be common ar¬ 
ticulated body systems with a moving base. One point 
that clearly distinguishes them from other ground-based 
robots, such as industrial manipulators, is the existence 
of a moving base. 

55.3.1 Space Robot 

as an Articulated Body System 

The robotic systems discussed in this chapter comprise 
one or multiple articulated limbs mounted on a base 
body that has a dynamic coupling with these limbs. 


55.2.6 Manipulators for In-Situ Science 

The Mars exploration rovers were the first planetary 
exploration vehicles to have general-purpose manipula¬ 
tors. (Lunokhod had a single-purpose soil penetrometer, 
and Sojourner had a single degree-of-freedom device 
to place an Alpha-Proton X-Ray spectrometer in direct 
contact with the terrain.) The MER arms each have 5°- 
of-freedom and a reach of over 1 m. Accurate gravity- 
sag models of the lightweight arm allow the precise po¬ 
sition to be predicted in advance of any command to 
deploy an instrument, and contact sensors allow the arm 
to stop before any excessive forces build up in the rel¬ 
atively flexible arm. Future arms for planetary surface 
operations, especially any proposed assembly, mainte¬ 
nance, or servicing tasks as part of the proposed lunar 
outpost, will require force sensing to protect the stronger 
but much more rigid arms from damage, as well as to 
allow controlled forces to be applied to the terrain or 
workpieces. Of course, there is a huge body of knowl¬ 
edge associated with industrial robot arms, and undersea 
robotics (e.g., for the offshore oil industry), but such 
arms are generally very heavy, fast, and stiff compared 
with credible systems for planetary surface use. Delicate 
force control has rarely been applied to industrial set¬ 
tings. Space hardware is necessarily very lightweight, 
and so both the arms and the workpieces will need to 
have well-resolved force sensing and control to prevent 
damage to one or both. Because of severe limits on both 
mass and power, and to avoid unnecessary risk, space 
manipulation tends to be slow. Historically, this means 
the gear ratio between each motor and the corresponding 
output shaft is very large, making the use of motor cur¬ 
rent as an estimator for output torque very problematic. 
Other low-mass and robust means for accurate sensing 
of applied forces in the space environment are needed. 


Typical styles of such moving base systems are cate¬ 
gorized into the following groups [55.46]. 

Free-Floating Manipulator Systems 
A space free-flyer that has one or more manipulator 
arms, as illustrated schematically in Fig. 55.22a, is 
a typical example of this group. When operating the 
manipulator arm(s), the position and orientation of the 
base spacecraft fluctuates due to the manipulator reac¬ 
tion. The kinetic momentum of the system is conserved 
if no external forces or moments are applied, and the 
conservation law for this system governs the reaction 
dynamics. Coordination or isolation, between the base 
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Fig.55.22a-d Four basic types for moving base robots: 
(a) Free-floating manipulator system, (b) macro-micro 
manipulator system, (c) flexible structure-based manipula¬ 
tor system, and (d) surface locomotive robot system 


and manipulator dynamics is key to advanced motion 
control. 


Macro-Micro Manipulator Systems 
A robotic system that comprises a relatively small arm 
(microarm) for fine manipulation mounted on a rela¬ 
tively large arm (macro-arm) for coarse positioning, is 
called a macro-micro manipulator system. The SSRMS 
(Canadarm2) and the SPDM (Dextre) system, as well 
as the JEMRMS on the Japanese module of the ISS, 
are good examples. Here, the connecting interface at the 
end point of the macro arm or the root of the micro arm 
is modeled as the base body (Fig. 55.22b). A free-flying 
space robot may be treated in this group when its base 
body is actively controlled by actuators that produce ex¬ 


ternal forces and moments, such as gas-jet thrusters. In 
this case, these actuators can be modeled as a virtual 
macro-arm [55.47], 

Flexible-Based Manipulator Systems 
If the macro arm behaves as a passive flexible (elas¬ 
tic) structure in a macro-micro manipulator system, the 
system is considered to be a flexible-based manipula¬ 
tor (Fig. 55.22c). Such a situation can be observed in 
the operation of the ISS, when the SSRMS is servo or 
brake locked after its coarse operation. Here, the issue is 
that the base body (the root of the micro arm, or the end 
of the macro arm, according to the definition above) is 
subject to vibrations that will be excited by the reaction 
of the micro arm. 

Mobile Robots with Articulated Limbs 
Mobile robots for surface locomotion have the same 
structure in terms of the dynamics equation as the 
above groups. This group includes wheeled vehicles, 
walking (articulated limb) robots, and their hybrids. 
In a wheeled vehicle, suspension mechanisms, if any, 
are also modeled as articulated limb systems. The 
forces and moments yielded by contact with ground, 
or planetary surface, govern the motion of the system 
(Fig 55.22d). 

55.3.2 Equations for Free-Floating 
Manipulator Systems 

Let us first consider a free-floating system with a sin¬ 
gle or multiple manipulator arm(s) mounted on a base 
spacecraft. The pioneering work in the mathematical 
models of this type of space manipulator systems were 
conducted in late 1980s and early 1990s and collected 
in the book [55.48], published in 1993. In this section, 
the models that are widely accepted today are intro¬ 
duced. 

The base body, termed link 0, is floating in inertial 
space without any external forces or moments. At the 
end point of the arm(s), external forces/moments may 
apply. For such a system, the equation of motion is ex¬ 
pressed as follows 



(55.1) 


The kinematic relationship among jth, Jfb> and <f> is ex¬ 
pressed using Jacobian matrices as 


Lh — Jm0 T Jl>fb , (55.2) 

3th — Jm0 T j 1 n + *J b-fh T jh-f- (55.3) 
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Where the notations are listed as follows: 

jCb G K 6 : position/orientation of the base 

<j> eW: joint angle of the arm 

jCh G K 6 *: position/orientation of the end point(s) 

ig 8": joint torque of the arm 

Fh G R 6A : external forces/moments on the end point(s) 

n: number of total joints 

k: number of manipulator arms, 

and Hb. H m , and Hb m are inertia matrices for the base 
body, manipulator arm, and the coupling between the 
base and the arm, respectively, and c p and c q are non¬ 
linear Coriolis and centrifugal forces, respectively. 

For a free-floating manipulator in orbit, the gravity 
forces exerted on the system can be neglected, and so 
the nonlinear term becomes 

Cb — Hb-^b t Hbm0b • 

Integrating the upper set of equation in (55.1) with re¬ 
spect to time, we obtain the total momentum of the 
system as 

L = J jjF h At = H b Xb + H bm 0 • (55.4) 

For the case in which reaction wheels are mounted 
on the base body, they are included as additional ma¬ 
nipulator arms. 

55.3.3 Generalized Jacobian 
and Inertia Matrices 

From (55.2) and (55.4), the coordinates of the ma¬ 
nipulator base ib, which are passive and unactuated 
coordinates, can be eliminated, as follows 

ih = J0+*hO. (55.5) 

where 

J = J m -J b H^ 1 H bm , (55.6) 

and 

i h0 =J b H 1 7 1 X. (55.7) 

Since Hb is the inertia tensor of a single rigid body (the 
manipulator base), it is always positive definite, and so 
its inverse exists. 

The matrix J was first introduced in [55.49, 50] and 
is referred to as the generalized jacobian matrix. In 
its original definition, it was assumed that no external 


forces/moments acted on the system. If Fh = 0, then 
the term iho becomes constant, and, in particular, if the 
system has zero initial momentum ±ho = 0, then (55.5) 
becomes simple. However, note that in the derivation of 
(55.6), zero or constant momentum is not a necessary 
condition. 

Using this matrix, the manipulator hand can be op¬ 
erated under a resolved motion-rate control or resolved 
acceleration control in inertial space. Thanks to the gen¬ 
eralized Jacobian, although the reactive base motion 
occurs during the operation, the hand is not disturbed 
by the motion. 

From the upper and lower sets of equations in 
(55.1), ib can be eliminated to obtain the following ex¬ 
pression 

Htj) +c=r + JF h , (55.8) 

where 

H = H m -H^ m H- | H bm . (55.9) 

The matrix H is known as the generalized inertia matrix 
for space manipulators [55.48]. This matrix represents 
the inertia property of the system in the joint space and 
can be mapped onto the Cartesian space using the gen¬ 
eralized Jacobian matrix 

G = JH 'j T (55.10) 

The matrix G is referred to as the inversed inertia 
tensor for space manipulators and is useful for the 
discussion of impact dynamics when a space manip¬ 
ulator collides with or captures a floating target in 
orbit [55.51]. 

The generalized Jacobian matrix (GJM) is a use¬ 
ful concept, with which the manipulator end point can 
perform positioning or trajectory tracking control by 
a simple control algorithm regardless of the attitude de¬ 
viation during the operation. 

A simplified laboratory demonstration was car¬ 
ried out using a two-dimensional free-floating test bed 
called EFFORTS [55.52]. To simulate the motion in 
a microgravity environment, a robot model was floated 
on a thin film of pressurized air on a horizontal plate, so 
that frictionless motion with momentum conservation 
was achieved. 

Figure 55.23 depicts the test bed and a typical ex¬ 
perimental result. For Fig. 55.23b, the control command 
was given to the floating robot by 

0 = J” 1 ^ , 


(55.11) 
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Fig.55.23a,b Laboratory test bed for a free-floating space 
robot: (a) The EFFORTS test bed, (b) a target capture result 


where id is the desired velocity of the manipulator end 
point, the value of which was given and updated by on¬ 
line measurement of the end point position x b and the 
target position x t , as follows 


id 


*t~*h 

At 


(55.12) 


where At is the time interval of the on-line control loop. 
The desired end point velocity was simply resolved into 
the joint velocity by (55.1 1). 

The result clearly shows that the manipulator end 
point properly reached the target in an optimum man¬ 
ner, although the robot base rotated considerably due to 
the manipulator reaction. Note that, since the target was 
stationary in this example, the resulting motion trace 
was a straight line. However, thanks to the on-line con¬ 
trol. the manipulator was also able to track and reach 
a moving target with the same control law. 

The validity and effectiveness of the GJM-based 
manipulator control were also demonstrated in orbit by 
Japanese ETS-VII mission [55.14]. 


55.3.4 Linear and Angular Momenta 

The integral of the upper set of equation in (55.1) gives 
a momentum equation, as shown in (55.4), which is 
composed of the linear and angular momenta. The lin¬ 
ear part is expressed as 

H b v b + H bm <f>=P, (55.13) 

where v b is the linear velocity of the base, P is the ini¬ 
tial linear momentum, and the inertia matrices with the 
mark of 0 are the corresponding components for the 
linear momentum [55.48]. When the linear momentum 
is further integrated, the result verifies the principle that 
the mass centroid of the entire system either remains 
stationary or translates with a constant velocity. 

The angular momentum equation, however, does 
not have a second integral, and therefore provides 
a first-order nonholonomic constraint [55.53]. The 
equation can be expressed as 

H b « b + H bm 0=L, (55.14) 

where &> b is the angular velocity of the base, L is the 
initial angular momentum, and the inertia matrices with 
the mark of 0 are the corresponding components for 
the angular momentum [55.48]; H bm 0 represents the 
angular momentum generated by the manipulator mo¬ 
tion. 

Equation (55.14) can be solved for w b with zero ini¬ 
tial angular momentum 

Q) b = -H^H bm 0 . (55.15) 

This expression describes the resulting disturbance mo¬ 
tion of the base when there is joint motion 0 in the 
manipulator arm. 

There are a number of points worth discussion when 
analyzing this equation. The magnitudes and direc¬ 
tions of the maximum and minimum disturbances can 
be obtained from the singular value decomposition of 
the matrix (— Hjj“ H bm J and displayed on the map. 
Such a map is called a disturbance map [55.54,55]. 
Equation (55.15) is also used for the feed-forward com¬ 
pensation in the coordinated manipulator/base control 
model [55.56,57]. 

55.3.5 Virtual Manipulator 

The concept of the virtual manipulator (VM) is an 
augmented kinematic representation that considers the 
base motion due to reaction forces or moments. The 
model is based on the fact that the mass centroid of the 
entire system does not move in the free-floating sys¬ 
tem without any external forces [55.58]. The mobility 
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(Normalized manipulability value = index number/40) 

Fig.55.24a,b Manipulability distribution in the work space (af¬ 
ter [55.60]) 


of the end point of the arm is decreased by the base 
motion. In the VM representation, such mobility degra¬ 
dation is expressed by virtually shrinking the length 
of the real arm according to the mass property. Note 
that the VM considers only linear momentum conser¬ 
vation. If the differential expression of VM is obtained 
using a Jacobian matrix, the Jacobian is not a conven¬ 
tional kinematic Jacobian, but rather a version of the 
generalized Jacobian defined by the combination of the 
kinematic equation (55.2), and the linear momentum 
equation (55.13). 

55.3.6 Dynamic Singularity 

Dynamic singularities are singular configurations in 
which the manipulator end point loses mobility in some 
inertial direction [55.59]. Dynamic singularities are not 
found in earth-based manipulators, but rather occur 
in free-floating space manipulator systems due to the 
coupling dynamics between the arm and the base. Dy¬ 
namic singularities coincide with the singularities of the 
generalized Jacobian matrix determined by (55.6). The 
singular value decomposition (SVD) of a manipulator 
Jacobian matrix provides manipulability analysis. Like¬ 
wise, the SVD of the generalized Jacobian matrix yields 
the manipulability analysis of a free-floating space ma¬ 
nipulator [55.60]. Figure 55.24 shows the comparison 
of the manipulability distribution between a 2-DOF 
ground-based manipulator and a 2-DOF floating manip¬ 
ulator, from which the degradation of the manipulability 
is observed in the space arm due to the dynamic cou¬ 
pling. 

55.3.7 Reaction Null-Space (RNS) 

Front a practical point of view, any change in the base 
attitude is undesirable. As such, manipulator motion 
planning methods that minimize the base attitude dis¬ 
turbance have been investigated extensively. Analysis 
of the angular momentum equation reveals that the ulti¬ 
mate goal of achieving zero disturbance is possible. 

The following is the angular momentum equation 
with zero initial angular momentum L = 0 and the zero 
attitude disturbance &> b = 0 given in (55.14) 

H bm 0=O. (55.16) 

This equation yields the following null-space solution 

0 = (I-H+H bm K. (55.17) 

The joint motion given by (55.17) is guaranteed not to 
disturb the base attitude. Here, the vector £ e R" is ar¬ 
bitrary and the null-space of the inertia matrix H bm e 
R 3Xn is called the reaction null-space (RNS) [55.61]. 


The DOF for £ is n — 3. For example, if a manip¬ 
ulator arm mounted on a free-floating space robot has 
6-DOF, i. e., n = 6, then 3-DOF remain in the reaction 
null-space. These DOFs can be specified by introducing 
additional motion criteria, such as end point positioning 
of the arm. Such manipulator operation that produces 
no reaction in the base is called the reactionless manip¬ 
ulation [55.62]. 

The validity and effectiveness of the RNS-based re¬ 
actionless manipulation were demonstrated in orbit by 
Japanese ETS-VII mission [55.14], 


This subsection addresses the generation of feasible 
trajectories for a free-flying robot for executing typi¬ 
cal point-to-point or grasping tasks. The subject falls 
under the problem domain of motion planning, with 
the aim of satisfying motion constraints which gen¬ 
erally cannot be satisfied with use of local methods 
alone (feedback control and model predictive control). 
A trajectory resulting from the motion planning is fed 
to a tracking controller, which accounts for any mod¬ 
eling errors, to accomplish the task in question. This 
approach also aims at providing autonomous skills for 
supporting a human ground operator. 

A typical task of interest here is that of a point- 
to-point maneuver of a robot manipulator mounted on 
a servicer satellite, to bring its end-effector in some de¬ 
sired inertial position and orientation. This task may 
require actuation of the servicer, or may, if possible, be 
preferably executed in free-floating mode, to avoid is¬ 
sues related to the use of the on-board thrusters, such 
as fuel expenditure. In this context, to minimize the 
attitude change of the servicer resulting from the ma¬ 
nipulator motion, a noticeable fundamental result for 
a point-to-point maneuver, derived from nonlinear opti¬ 
mization theory, is the V-maneuver [55.63]. Intuitively, 


55.3.8 Motion Planning Issues 
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the attitude change is minimized by making the robot 
first move radially inward, toward the system center of 
mass, before turning and then radially outward, to reach 
the new desired final position. 

A second task that has received much attention is 
the grasping of a free tumbling target, like a defected 
satellite. Currently, there are a number of space pro¬ 
grams worldwide which are addressing this task, like 
the preparation of the demonstration mission DEOS of 
the DLR (Sect. 55.1.2) for grasping a small satellite in 
LEO, the e.Deorbit study of the ESA for the deorbit¬ 
ing of the defected ENVISAT satellite and PHOENIX 
of DARPA for grasping a geostationary satellite in the 
graveyard orbit. Different approaches can be found in 
the literature to tackle this problem, to include [55.64, 
65] for feedback control. [55.66] for model predic¬ 
tive control, [55.67] for optimal control in Cartesian 
space and [55.68] for nonlinear optimization, to men¬ 
tion some. 

Due to the nonholonomic nature of the dynamics 
of a free-floating robot (Sect. 55.3.4), in order to sat¬ 
isfy all relevant motion constraints, the motion planning 
problems above can only be solved through numerical 
integration. Note in fact, that the final system configu¬ 
ration for a given final end-effector position in inertial 
space is function of the whole path taken by the robot 
throughout the motion. 

Principally, the motion planning tasks above may be 
formulated as an optimal control problem of the type 

min r(<j>(f),r{t),tf), (55.18) 

4,0(0 

subject to 

g(f f ,0(t)) = O (55.19) 

h(t f , 0 ( 0 ) < 0 , (55.20) 

for 0<t<tf and where tf is the final time; 0(f) is 
the vector of joint positions; r(f) is the vector of 
joint torques; f is a predefined cost function; g are 
equality constraints to include, for example, the state 
transition equations; h are inequality constraints, for ex¬ 
ample the joint box constraints on position, velocity and 
torque, or collision avoidance constraints. Other motion 
constraints may include inequality constraints on the 
end-effector forces during contact, or other operational 
constraints. 

This consists of an infinite dimensional problem, 
in the given time interval, which generally cannot be 
solved in closed form. The direct shooting optimiza¬ 
tion methods lend themselves well to solving these 
problems iteratively [55.69], where for example the in¬ 
dependent DOF (in the case of a free-floating system. 


the robot joint angles) are parameterized in time as 

0 = 0(f,p) (55.21) 

with p G R", for n optimization parameters, where 0 
may be a polynomial function, a B-spline function, 
or any other. This way, the problem becomes that of 
finding a suitable value for the parameters p which 
satisfies all motion constraints (feasibility), as well as 
perhaps minimizes a cost function (optimality), like 
the time-to-collision [55.70], the end time [55.67], or 
the mechanical energy [55.68]. The problem is as such 
transformed into a finite dimensional problem and can 
be solved as a nonlinear programming problem (NLP) 
with classical numerical iterative methods such as se¬ 
quential quadratic programming (SQP). 

For example, for a Cartesian point-to-point prob¬ 
lem, we have the following supplementary equality 
constrains at the end time 

4 

X e = f W(t' P )dt = X e des , (55.22) 

0 

where X e e R" denotes the end-effector pose (array ma¬ 
trix of dimension DOFe x 1). Note that the constraint 
itself has 00 D0F+3 - D °Fe so i u q ons f or th e system con¬ 
figuration (for DOF robot joints), however these all 
imply a specific base body attitude as well as a specific 
robot configuration. Solving the problem of reaching 
a specific system configuration is a hard nonholonomic 
control problem, which is avoided by solving the con¬ 
straint thorough the integral above and an adequate 
parameter set p. 

Extensions of this are necessary to treat the grasp¬ 
ing task. The latter can ideally be separated into three 
phases: approach, tracking and, stabilization. The first 
comprises a point-to-point maneuver, however with 
a nonzero end velocity. The second comprises a track¬ 
ing maneuver, in which the Cartesian motion of the 
robot end-effector is dictated by the tumbling motion 
and the geometry of the target as it follows the grasp¬ 
ing point and homes in onto it, to finally grasp it. Note 
that this phase aims at minimizing the impact between 
the end-effector and the target. The third phase involves 
a robot joint velocity decay maneuver, once the target 
is grasped. This formulation results in a multiple-phase 
problem, for which the boundaries between the phases 
introduce supplementary motion constraints in the mo¬ 
tion planning problem. 

From a methodological point of view, note that there 
is no simple measure to determine if and when the 
grasping point on the target will be reachable from the 
current robot configuration (Fig. 55.25) and whether 
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Fig. 55.25 Orbital scenario: servicer satellite with 7-DOF 
manipulator and target satellite with solar panels. Coordi¬ 
nate system of a predefined grasping point on target ring 
shown 

the trajectory which derives from a local control law 
will be feasible at all times (accounting for the mo¬ 
tion constraints listed above). It is also necessary to 
provide information on the time synchronization be¬ 
tween the motion of the grasping point on the target 
and that of the robot. Furthermore, the nonlinear na¬ 
ture of the robot kinematics needs to be exploited to 
favor a successful grasp. These considerations speak 
in favor of the use of a reference trajectory, which is 
computed off-line by means of a global search, based 
on a motion prediction of the target and on a ge¬ 
ometric model of the same target and of the space 
robot [55.68]. 

It is also well known that optimization methods 
suffer from convergence issues (arising from local min¬ 
ima), if a judicious initial guess is not available. It is 
for this reason that a look-up table approach is nec¬ 
essary [55.68] in order to provide a sufficiently high 
probability of convergence for a given grasping task. 

The necessity of a look-up table also arises from 
the long computation times of the optimization pro¬ 
cess, which result from the aforementioned necessity 
to integrate the equations of motion for any given op¬ 
timization iteration. This time is generally reduced if 
a solution close to the sought one is given as a starting 
point. An attempt to eliminate the necessity of integrat¬ 
ing the equations of motion was also made in [55.71], 
where a differentially flat representation for a free- 
floating robot was sought. In such a formulation, there 
are as many differential equations as there are indepen¬ 
dent state variables, or flat variables, and as such, any 
parameterization of the flat variables is solution of the 
equations of motion. Such a representation was found 
for the case in which the robot has three joints and the 
center of mass of its load lies on the rotation axis of the 
last joint. 


It is also of interest to make some considerations 
on practical technological issues which can influence 
one or the other future research direction. With regard 
to the minimization of the servicer attitude change dur¬ 
ing robotic operations, it is worth realizing that this is 
generally only an issue for maintaining the communi¬ 
cation link from low Earth orbit with a geostationary 
satellite, for which a high pointing accuracy is required 
(note that for a communication link from low Earth 
orbit to ground instead, an omnidirectional antenna is 
sufficient). Therefore, rather than limiting the robot 
workspace to its reaction null-space to avoid any at¬ 
titude change at all, in which case the resulting robot 
movements would generally be confined to the extent 
of becoming of little practical use, a simple technolog¬ 
ical solution is possible through the implementation of 
a gimbal joint for the communication antenna on the 
servicer. 

Another simple technological solution to a theoreti¬ 
cally involved problem is that of controlling the attitude 
of the servicer by means of adequate closed-loop ma¬ 
neuvers of the robot. As is well known, reaction wheels 
on a satellite achieve the same result, also with the same 
principle of momentum transfer, but however with a far 
simpler control law than that necessary for a robot. It is 
then important to realize that, although reaction wheels 
were until today far too small for any useful robotic 
application, larger reaction wheels are currently being 
developed, for example, in the context of the BIROS 
mission of the DLR, which is planned to fly in 2014. 
Although these will still not be able to fully compensate 
the typical robot dynamic coupling terms (if wanting to 
fully stabilize the servicer attitude), they will however 
be able to allow significant attitude slew maneuvers of 
the servicer quickly, without having to resort to the use 
of the robot or of the thrusters. Furthermore, they will 
be useful to reduce the robot singularities, due to the 
enhanced system’s actuation. 

55.3.9 Equations for Flexible-Based 
Manipulator Systems 

Next, let us consider a flexible-based manipulator sys¬ 
tem in which a single or multiple manipulator arm(s) 
are supported by a flexible beam or a spring and damper 
(visco-elastic) system. For such a system, the equation 
of motion is obtained using the following variables: 

.*b G R 6 : position/orientation of the base 

0 el": joint angle of the arm 

JCh G R 64 ': position/orientation of the end point(s) 

Fb e R 6 : forces/moments to deflect the flexible base 

rel": joint torque of the arm 

Fh G R 64 : external force/torque on the end point(s). 
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f Hb Hbm \ b \ , (C b 'j 

V H bm H m jUj + UJ“ 



*h = Jm0 + Jb*b , (55.24) 

*h = Jm0 + j m 0 + Jb*b + ib-Tb • (55.25) 

Here, with the gravitational force g in Cartesian space, 
the term c b is generally expressed as 

fb =/(Xb,0.ib.0)+g(*b.0) • (55.26) 

The difference from the equation of a free-floating 
manipulator system (55.1) is the existence of the base 
constraint force F b . Let Db and Kb be the matrix rep¬ 
resenting the damping and spring factors, respectively, 
of the flexible-base. The constraint forces and moments 
Ft, are then expressed as 

F b = -D b ir b -K b Z\x b . (55.27) 

Since the base is constrained, the total momentum is 
not conserved, and it might be meaningless to check the 
system momentum. However, it is important to consider 
the partial momentum £ m for the part of the manipula¬ 
tor arm 

£ m = H bm 0, (55.28) 

which is termed the coupling momentum [55.72]. Its 
time derivative describes the forces and moments F m , 
which are yielded by the dynamic reaction from the ma¬ 
nipulator arm to the base 

f''rn = H bm 0 + Hbm0 ■ (55.29) 

Using Ft, and F m , the upper set of equation in 
(55.23) is rearranged as 

H b j? b + D b L b + K b Axt, = -g-F m +ilFt ,. (55.30) 

Equation (55.23) or (55.30) is a familiar expression 
for flexible-base manipulators [55.73,74]. 

55.3.10 Advanced Control for Flexible 
Structure Based Manipulators 

In this subsection, advanced control issues for a macro¬ 
micro space manipulator are discussed. The SSRMS 
attached to the SPDM (Fig. 55.4) and the JEMRMS 
(Fig. 55.5) are examples. Operation modes for this class 
of space manipulators include coarse motion by the 
macro (long-reach) component and fine manipulation 


by the micro component. In normal cases, these two 
control modes are executed exclusively. Namely, while 
one component is active the other component should 
be servo (or brake) locked. Thus, during the operation 
of the micro component, the macro component behaves 
just as a passive base. 

Due to the flexible nature of the space long-reach 
arm, the macro part is subject to vibrations. These vi¬ 
brations can be excited during coarse positioning and 
may remain for a long time after each operation. In 
fine manipulation, the macro arm behaves as a pas¬ 
sive flexible structure, but then vibrations can be excited 
by reactions from the motion of the micro arm. These 
motions degrade the control accuracy and operational 
performance of the system. In practice, the booms are 
usually sufficiently stiff, but flexibility comes mainly 
from the low stiffness at the joints and gear trains. 
Moreover, lightweight and microgravity characteristics 
make the structure sensitive to yield vibrations, and the 
surrounding vacuum, or the lack of air viscosity, pro¬ 
vides a reduced damping effect to the structure. 

Conventionally, the vibration issue has been man¬ 
aged for SRMS and SSRMS by the operational skill 
of well-trained astronauts and by limiting the maxi¬ 
mum operational velocity according to the inertia of the 
handling object. However, if an advanced controller is 
introduced on the ISS, the training time for astronauts 
will be reduced and the operational speed can be in¬ 
creased. 

Here, the following two subtasks are considered in 
dealing with this issue: 

• Suppression of the vibrations of the flexible base 

• End point control in the presence of vibrations. 

To suppress the vibrations of the macro arm (flex¬ 
ible base), the coupled dynamics is effectively used. 
Such control is called coupled vibration suppression 
control [55.75]. The coupled dynamics is a solution 
space of the micro arm motion with maximum coupling 
with the vibration dynamics of the macro arm. Note 
that this solution space is perpendicular to the reaction 
null-space introduced in Sect. 55.3.7. Since the spaces 
are orthogonal, the coupled vibration suppression con¬ 
trol and reactionless manipulation can be superimposed 
without any mutual interference. 

The motion command of the micro arm to suppress 
the vibrations is determined with a feedback of the lin¬ 
ear and angular velocity of the end point of the macro 
armi: b 

0 = H+ H b G b i b , (55.31) 

where (•)"*“ denotes the right pseudo-inverse, and G b is 
a positive definite gain matrix. 
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If written in the form of a joint torque input, the 
vibration control law is expressed as follows 

r = H m H+ G b i: b . (55.32) 

In the presence of redundancy in the micro arm, 
(55.31) can be extended to control with a null-space 
component 


ij> m = H+(H b Gbi: b - H bm 0 m ) + .Prns? , (55.33) 

where t, is an n-DOF arbitrary vector and Prns = (E — 
Hb+Hbm) is a projector onto a null space of the coupling 
inertia matrix Hb m . When the micro arm is operated us¬ 
ing (55.33), the closed-loop system is expressed as 

Hb^b + HbGbib + KbAtb = Fb + J^Fh . (55.34) 

Equation (55.34) represents a second-order damped vi¬ 
bration system. With no force input, i. e., Fj, = Fj, = 0, 
the vibrations converge to zero with a proper choice of 
the gain matrix Gb- 

For the determination of vector £, feedback control 
to reduce the positioning error of the micro arm end 
point is considered. The error vector is defined as 

^h=*h-*h- (55.35) 

After some derivation, the control law for the joint 
torque of the micro arm is obtained in the following 
form [55.75,76] 

r = ((Jm) + H m P RNS ) + (Kp5 r h - K d J m 0) - G m 0 , 

(55.36) 

where K p , K d , and G m are positive definite gain matri¬ 
ces. 

Figure 55.26 shows a block diagram for the control 
system described by (55.32) and (55.36). 

As a simplified demonstration, a planar system with 
a four-joint redundant manipulator arm atop a flexible 
beam is considered. Figure 55.27 shows the vibra¬ 
tion amplitude of the flexible beam after an impulsive 
external force. The graph labeled w/o vs depicts the vi¬ 
brations of the beam without any manipulator control 
but with natural damping. The graph labeled with vs 
depicts the case in which vibration suppression con¬ 
trol given by (55.32) is applied, where the vibrations 
are damped quickly. 

In addition, Fig. 55.28 shows the end point motion 
of the manipulator arm during the control. The graph 
labeled w/o RNS is the case of using (55.32), where 
the base vibrations were successfully suppressed but 



Fig. 55.26 Block diagram for simultaneous vibration suppression 
and manipulator end point control for a flexible-structure-mounted 
manipulator system 


y deflection (m) 



y deflection (m) 



Time (s) 


Fig. 55.28 Positioning error of the manipulator end tip 

the position of the manipulator end point was deflected 
by this suppression behavior. The graph labeled with 
RNS depicts the case in which both vibration suppres¬ 
sion control given by (55.32) and the end point control 
given by (55.36) are applied simultaneously. This last 
case shows that the vibrations were damped success¬ 
fully and that the positioning error of the manipulator 
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end point converged to zero. This is a result of the re¬ 
dundancy of the arm. 

Here, note that the proposed control method re¬ 
quires precise information on dynamic characteristics, 
such as the inertia parameters of the arms and the 
handled payload, if any. To achieve more practical ap¬ 
plications, the proposed method must be extended to 
schemes for parameter identification [55.77] and adap¬ 
tive control [55.78], with which the convergence of 
the control is guaranteed even with imprecise a priori 
knowledge of the dynamic parameters [55.76]. 

55.3.11 Contact Dynamics 

and Impedance Control 

The capture and retrieval operation of a floating and 
tumbling target, such as a malfunctioning satellite, by 
a manipulator arm mounted on a servicing robot (called 
a chaser) can be decomposed into the following three 
phases: 

1. Approaching phase (before contact with the target) 

2. Contact/impact phase (at the moment of contact) 

3. Post contact phase (after contact or grasping). 

If the contact is impulsive, the first and the third 
phases are discontinued by the impulsive phenomena 
of the second phase. Understanding of this impulsive 
phenomena is indispensable when designing a compre¬ 
hensive capture control scheme. In this subsection, the 
formulation of impact dynamics is first considered, and 
impedance control (which is useful to minimize the im¬ 
pact forces and prolong the contact duration) is then 
discussed. 

Let us consider a chain of rigid links composed of 
n + 1 bodies freely floating in inertial space. As dis¬ 
cussed in Sect. 55.3.3, the equation of motion for this 
type of system becomes (55.8). Here, the impulsive 
contact force is assumed to be applied at the manipu¬ 
lator end tip and is expressed as F h = (/J,wJ) T e R 6 . 
This impulsive force also yields a change in the system 
momenta (Pi, Ll ) T e R 6 , expressed as 

//> g W wE ow* g w 0 \ 

\LJ V 0 ig/ V w g/ V w g x ig w g/ 

= ( E °V /h l 

Ugh e; \n h J 

(55.37) 

where E is a 3 x 3 identity matrix, and the symbols with 
the suffix g indicate the corresponding values observed 
around the mass centroid of the n + 1 link system. 

In the above equations, (55.8) describes an inter¬ 
nal joint motion (termed local motion) of the system. 


whereas (55.37) describes the overall motion (termed 
global motion) about the centroid of the system. As 
a result of force input F h, the floating chain induces 
both the local motion around its articulated joints and 
the global translation and rotation with respect to the 
centroid. 

From (55.8) and (55.37), the acceleration of the ma¬ 
nipulator end tip ah can be expressed in the inertial 
frame as 

a h = G *F h +d* , (55.38) 


where 

G* = JH-'jT + RhM- Rj , (55.39) 



and d* is a velocity dependent term. 

Equations (55.38)-(55.40) are expressions for the 
motion of the hand (the point at which collision occurs) 
induced by the impact force F^, where the matrix G*, 
which is the augmented version of (55.10), represents 
the dynamic characteristics of this system. 

Further augmentation for the inverted inertia ma¬ 
trix has been discussed for the case in which the 
contact duration is not considered to be infinitesi¬ 
mal [55.51]. 

Now let us assume the case in which two free- 
floating chains, A and B, with dynamic characteristics 
GJ and Gg collide with each other at their respective 
hands (end points) and an impact force F h is induced 
by this collision. 

The equations of motion at the instance of collision 
are 

GlE h = f UhA )-rfX (55.41) 

V^hA/ 

for the chain A, and 

GS(-E h ) = f VhB V rf B (55-42) 

V«hB/ 


for the chain B, where the subscripts A and B indicate 
the label of the chain. 

Assuming that G^ and Gg remain constant during 
the infinitesimal contact duration and the velocity- 
dependent terms d ^ and </g are small and negligible, 
integration of (55.41) and (55.42) yields 


t+St 



t 


(55.43) 


(55.44) 
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where { } indicates the velocity after the collision. Inte¬ 
gration of F h 


(V) 

V W hA / 


= (cr i +G B *- 1 )- 


t+St 

Fh = litn / Fh d t 

St-+ 0 J 


(55.45) 


represents the impulse (force-time product) acting on 
both chains. Providing that the total momenta of the two 
systems are strictly conserved before and after the col¬ 
lision, we obtain the following expression from (55.43) 
and (55.44) 


(Ga + Gg) F h 



In general collision analysis, the coefficient of resti¬ 
tution (elasticity factor) associated with the relative 
velocities before and after the collision is often em¬ 
ployed [55.79]. If we accept this restitution coefficient 
for 6-DOF linear and angular velocities, the relation¬ 
ship between relative velocities before and after contact 
becomes 



(55.47) 


where 


t = diag(ei . e^) , 0 < e, < 1 (55.48) 


x 


(E 6 + e)Gg —1 



+ (G£- 1 -eGg- 1 ) 


— K / WhA 


^hA 


where suffixes A and B are interchangeable. 


(55.53) 


These expressions are considered to be an augmen¬ 
tation of the impact theory for a two-point-mass system 
into articulated body systems. 

Impedance control is a concept by which we can 
control the manipulator end tip so as to obtain the de¬ 
sired mechanical impedance characteristics. Such con¬ 
trol is useful to alter the dynamic characteristics of the 
arm during the contact phase. In a special case, the de¬ 
sired impedance of the manipulator end tip (hand) may 
be tuned to achieve impedance matching with the col¬ 
liding target object so that the hand can easily maintain 
stable contact with the target [55.80]. 

Let M d , D ( |. and K d be the matrices for the desired 
impedance properties of inertia, viscosity, and stiffness, 
respectively, measured at the manipulator end point. 
The equation of motion for the desired system is then 
expressed as 


M d x h + DdZ\x h + K d Z\x h = F h . (55.54) 

From (55.8) and (55.54), the impedance control 
law for a free-floating manipulator system is obtained 
as [55.81] 

r h = H*J _1 {Mj _1 (D d Z\i h + K d A*: h — F h ) 


is the restitution coefficient matrix. 

Substituting (55.47) into (55.46), the impulse in¬ 
duced by this collision can be expressed only by the 
relative velocity of two points before the contact 


F h = (E 6 + €)G|r 1 V h AB, 

(55.49) 

where 


g^=g:+g;, 

(55.50) 

-V (vm\ (v hA \ 

ThAB = UJ"UJ- 

(55.51) 


Using the introduced notation, the magnitude of im¬ 
pulse is expressed as 

II F h || = 

\j (E6 + O^abG^-^G^ 1 V hAB (E 6 + €) , 

(55.52) 


- J0-Xgh}-J T F h + c* . (55.55) 

The usefulness of the impedance control in free- 
flying space robots has been discussed in [55.81-83]. 

55.3.12 Dynamics of Mobile Robots 

The equation of motion for a mobile robot that has mul¬ 
tiple articulated limbs, such as that shown in Fig. 55.29, 



and the velocity after collision becomes 


Fig. 55.29 Schematic model of a mobile robot 
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is given by the following equation 


55.3.13 Wheel Traction Mechanics 


( H b 

Hbml 



/z,\ 


( c b > 

H bml 

H m i 1 

H m i£ 


0i 

+ 

^ml 

vhL* 

Ku ••• 

H m u/ 


UJ 


k) 


( F b\ 

(3 If 

ex 

\ 




*1 


iL^e: 


+ 


\T„/ VjmA^'cxt/ 


(55.56) 


where the symbols have the following meanings: 
k number of limbs 

Xb G R 6 : position/orientation of the base 
<j> = ( (j)J ,..., G R": articulated joint angles 

x e x = (xj x i’ • • •, eR 6 ^: position/orientation of the 

end points 

Fb G R 6 : forces/moments directly apply on the base 
r = (tJ, ..., tJ) t g R": joint articulated torque 
F ex = (Fjxi, • ■ ■ , F , J xA .) t g R“: external 
forces/moments on the end points. 


Note that for Fig. 55.29, F ex ; = \fh, n^,] T . 

Comparing the above equations with (55.1), no 
difference is observed in the mathematical structure. 
The gravity force on the vehicle main body and the 
configuration-dependent gravity terms of the articulated 
bodies are included in <4, and c mi , respectively. In prac¬ 
tice, however, one substantial difference is the existence 
of ground contact forces/moments at the end point of 
each limb. Unlike floating target capture discussed in 
Sect. 55.3.10, the contact is not considered impulsive, 
but instead continues for a nonnegligible period of time. 
In such cases, a well-accepted approach is to explicitly 
evaluate the contact forces/moments F ex , according to 
the virtual penetration of the end point into the collided 
object or the ground surface [55.79]. 

In cases in which each limb has a wheel on its 
end terminal, rather than the point penetration model, 
a wheel traction model will be adopted to evaluate 
F e x- For planetary exploration missions, rovers (mobile 
robots) are expected to travel over natural rough terrain. 
A number of studies have examined the modeling of 
tire traction forces on loose soil, called regolith (where 
there is no organic component) [55.84-98]. Particularly, 
these studies investigate the soil mechanics called ter- 
ramechanics to understand the tractive forces generated 
by wheels. 

In the following subsection, models for wheel trac¬ 
tion mechanics are summarized. 


Terramechanics is the study of soil properties, specif¬ 
ically the interaction of wheeled, legged, or tracked 
vehicles and various surfaces. For the modeling of 
the wheel traction forces and the analysis of the ve¬ 
hicle mobility, textbooks written by Bekker [55.29, 
30] and Wong [55.99] are good references. Although 
these books were written in the 1960s and 1970s, ba¬ 
sic formulae from these books are frequently cited by 
researchers even today [55.88]. In this subsection, the 
models for a rigid wheel on loose soil are summarized. 


Slip Ratio and Slip Angle 

Slips are generally observed when a rover travels on 
loose soil. In particular, slips in the lateral direction are 
observed during steering or slope-traversing maneuvers. 

The slip in the longitudinal direction is measured 
by slip ratio s, which is defined as a function of the 
longitudinal traveling velocity v x of the wheel and the 
circumference velocity of the wheel rco (r is the wheel 
radius and u> is the angular velocity of the wheel) 


ru> — v x 


s = 


ru> 

ra> — v x 


if |ra>| > | v x | : driving 
if|ra>| < | v x | : braking . 


(55.57) 


The slip ratio takes a value from — 1 to 1. 

On the other hand, the slip in the lateral direction is 
measured by slip angle /J, which is defined in terms of 
v x and the lateral traveling velocity ly as 



Note that the above definitions, (55.57) and (55.58) 
have been traditionally used in the vehicle community 
as standards. However, planetary rovers in a challeng¬ 
ing terrain, such as Spirit and Opportunity on Mars, 
experienced the cases in which the rovers slip backward 
while attempting to drive up hill, or travel faster than the 
wheel’s circumference velocity in downhill driving. In 
these cases, the slip ratio can exceed the range from — 1 
to 1. Also while traversing side slopes, the case may 
arise in which Vy > 0 but v x is nearly 0, making the 
definition (55.58) nearly singular. Therefore, these defi¬ 
nitions are needed to be further discussed for the rovers 
in very loose terrain. 


Wheel-Soil Contact Angle 
Figure 55.30 depicts a schematic model of a rigid wheel 
contacting loose soil. In the figure, the angle from the 
surface normal to the point at which the wheel initially 
makes contact with the soil (Z AOB) is defined as the 
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Fig. 55.30 Wheel contact angles 


entry angle. The angle from the surface normal to the 
point at which the wheel departs from the soil (Z BOC 
in Fig. 55.30) is the exit angle. The wheel contact region 
on loose soil is represented from the entry angle to the 
exit angle. 

The entry angle 9{ is geometrically described in 
terms of wheel sinkage h as 

h\ 

1--J • (55.59) 



The exit angle 9 X is described using the wheel sink- 
age ratio A, which denotes the ratio between the forward 
and rear sinkage of the wheel 


9 r = cos 1 



(55.60) 


The value of A depends on the soil characteristics, the 
wheel surface pattern, and the slip ratio. It becomes 
smaller than 1.0 when the soil compaction occurs, but 
can be greater than 1.0 when the soil is dug up by the 
wheel and transported to the rear region of the wheel. 



First, the wheel sinkage h(9) at an arbitrary wheel 
angle 6 is geometrically given by 

h(9) = r(cos 9 — cos0 S ) , (55.62) 

where 9 S is the static contact angle. Then, substituting 
(55.62) into (55.61) yields 

p(9) = r n ^ + kd, j (cos 8 - cos 9 S )' 1 . (55.63) 

The wheel eventually sinks into the soil until the stress 
from the soil balances the vertical load W on the wheel. 


W = 


9 S 

J p(6)br cos 9&9 


-9 S 


9s 

= r""*" 1 ( k c + kffcb) J (cos 9 — cos 9 S )" cos 6 d# . 
—9 S 

(55.64) 


Wheel Sinkage 

The amount of wheel sinkage is constituted by static 
and dynamic components. The static sinkage depends 
on the vertical load on the wheel, while the dynamic 
sinkage is caused by the rotation of the wheel. 

According to the equation formulated by Bek- 
ker [55.29], the static stress p(h) generated under a flat 
plate, which has a sinkage h and a width b, is calculated 
as 



where k c and k$ are pressure-sinkage modules, and n 
is the sinkage exponent. Applying (55.61) to the wheel, 
as shown in Fig. 55.31, the static sinkage is evaluated 
as follows. 


Using this equation, the static contact angle 9 S is evalu¬ 
ated for the given W. In practice, (55.64) does not yield 
a closed form solution for 8 S , although 9 S can be evalu¬ 
ated numerically. 

Finally, the static sinkage h s is obtained by substi¬ 
tuting 9 S into the following equation 

h s = r(l — cos 0 S ) . (55.65) 

Flowever, as illustrated in Fig. 55.32, the dynamic 
sinkage becomes a complicated function depending on 
the slip ratio of the wheel, the wheel surface pattern, 
and the soil characteristics. Although it is difficult to 
obtain an analytical form for the dynamic sinkage, it 
is again possible to evaluate the dynamic sinkage nu¬ 
merically, using the condition W = F z , where F z is the 
normal force given by (55.76), which will be presented 
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later herein. The force F- increases with the wheel sink- 
age because the area of the contact patch increases 
accordingly. 

Stress Distribution Under the Wheel 
Based on terramechanics models, the stress distribution 
under the rotating wheel can be modeled as shown in 
Fig. 55.33. 

The normal stress a (9) is determined by the follow¬ 
ing equation [55.87, 88] 


cr(9) = r n ^ -- + k (cos 6 — cos 6f) n 
for 9 m < 9 < Of , 


„ fk c \ ( 

9-Or 

a(9) = r | — +kcpj jcos 

9f-- a - UOf -9 m ) 

Vm ~ Vr 

— cos Of}" for 0 t 

< 9 < e m . 


( 55 . 66 ) 



Note that the above equations are based on Bekker’s 
formula, as given in (55.61), and they become 
equivalent to the Wong-Reece model for normal 
stress [55.100] when 11 = 1. Also note that by lin¬ 
earizing this distribution, Iagnemma et al. [55.84,88] 
developed a Kalman-filter-based method to estimate the 
soil parameters. 

The term 9 m is the specific wheel angle at which the 
normal stress is maximized 

9 m = (a 0 + ais)0 f , (55.67) 

where a 0 and ci\ are parameters that depend on the 
wheel-soil interaction. Their values are generally as¬ 
sumed to be ao 0.4 and 0 < ai < 0.3 [55.100]. 

The maximum terrain shear force is a function of 
the terrain cohesion c and internal friction angle </>, and 
can be computed from Coulomb’s equation 

Anax((^) —c T tr max (d) tan (p . (55.68) 

Based on the above equation, the shear stresses un¬ 
der the rotating wheel, x x (0) and x y (9), are written as 
follows [55.101] 

x x (9) = [c + ct(9) tan $] (l -e~ jAd)/k A , (55.69) 

x y (9) = [c + a(9) tan0] ^1 — e - * <6l)/ ^ , (55.70) 

where k x and k y are the shear deformation moduli in 
each direction. In addition, j x and j y , which are the soil 
deformations in each direction, can be formulated as 
a function of the wheel angle 9 with the slip ratio and 
the slip angle, respectively [55.89, 100] 

j x (9) = r[9{ — 9 — (1 — i)(sin0f — sin#)] , (55.71) 

j y (0) = r(\-s)(9f-0) tan . (55.72) 

Drawbar Puli: F x 

Using the normal stress a(9) and the shear stress in 
the x direction x x (9), the drawbar pull F x , which is the 
net traction force exerted from the soil to the wheel, is 
calculated as the integral from the entry angle Of to the 
exit angle 9 r [55.100] 

ft 

F x = rb J [x x (9) cos 9 — a(9) sin 9] dO . (55.73) 

ft 

Side Force: F y 

The side force F y appears in the lateral direction of the 
wheel when the vehicle makes steering maneuvers or 
traverses a side slope. The side force is decomposed 
into two components [55.89] 


Fig. 55.33 Stress distribution model under a wheel 


F y — F u + F s , 
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where F„ is the force produced by the shear stress in 
the y direction r y ( 6 ) underneath the wheel and F s is the 
reaction force generated by the bulldozing phenomenon 
on a side face of the wheel. The above equation can be 


rewritten as 


II 

' 

rbT y { 0 ) +/?b[r — h{9) cos 9] 

Or 

. F ° 


(55.74) 


Bulldozing direction 



Here, Hegedus ’s bulldozing resistance estima¬ 
tion [55.102] is employed to evaluate the side face force 
F s . As shown in Fig. 55.34, a bulldozing resistance 
Rb is generated on a unit width blade when the blade 
moves toward the soil. According to Hegedus’s theory, 
the bulldozed area is defined by a destructive phase that 
is modeled by a planar surface. In the case of a horizon¬ 
tally placed wheel, the angle of approach a' should be 
zero; R b can then be calculated as a function of wheel 
sinkage h{ 6 ) 


Rb(h) = D\ 


ch( 6 ) + D 2 


Pdh 2 (oy 

2 


(55.75) 


where 


D\(X C , cj>) = cotX c + tan(X c + cp) , 
cot 2 X c 

D 2 (X c ,<t>)=cotX c + -. 

coUp 

In the above equations, p<j denotes the soil density. 
Based on Bekker’ s theory [55.29], the destructive angle 
X c can be approximated as 



Normal Force: F z 

The normal force F- is obtained in the same manner as 
for the (55.73) [55.100] 

e f 

F z = rb J [r t (0) sin 6 + 0 ( 8 ) cos 6 ] d9 , (55.76) 

Or 

which should balance the normal load of the wheel in 
a static condition. 


Fig. 55.34 Estimation model of the bulldozing resistance 

Motion dynamics simulation for a vehicle traveling 
over loose soil can be performed by plugging the forces 
F x , F y , and F, obtained from the above equations into 
the equation of motion (55.5). 

A better understanding of the soil-wheel contact 
and traction mechanics is important in order to improve 
the navigation and control behavior of exploration 
rovers, in terms of minimization of wheel slippage, for 
example. Reducing the wheel slippage will increase the 
power efficiency of surface locomotion, decrease the er¬ 
rors in path tracking maneuvers, and decrease the risks 
of wheel spinning and sinking, which can cause immo¬ 
bilization of the vehicle. 

One key in realizing such advanced control of 
slippage minimization is determining how to properly 
estimate the slip ratios and slip angles in real time us¬ 
ing on-board sensors. The slip ratio is determined by the 
ratio between the wheel spinning velocity and the trav¬ 
eling velocity of the vehicle, but proper sensing of the 
velocity of the vehicle is usually difficult. One simple 
solution is to use a free wheel specialized for traveling 
velocity measurement. Another solution is to employ 
inertial sensors, which are however usually subject to 
noise and drift. 

An alternative, but promising possibility is visual 
odometry, which is based on optical flow or feature 
tracking in the sequence of optical images. Actually, 
this technique has been applied to the Mars exploration 
rovers, Spirit, and Opportunity, in their long range nav¬ 
igation, and verified to be very useful. Particularly, the 
algorithm based on feature detection and tracking us¬ 
ing stereo pair of cameras provides reliable results with 
good accuracy for the estimation of driving distance as 
well as the wheel slippage [55.28]. 


Part F | 55.3 

















Part F | 55.4 


1452 Part F 


Robots at Work 


55.4 Future Directions of Orbital and Surface Robotic Systems 


55.4.1 Robotic Maintenance 
and Service Missions 

For many years, we have sent satellites and other sys¬ 
tems into space without caring too much about what 
might happen at the end of their life cycle. There has 
been recently awareness about the dramatic increase of 
space debris and the danger of a fatal chain reaction in¬ 
creased in collisions. Generally speaking, space debris 
removal may become a prerequisite for future space- 
flight. Space systems above approx. 600 km flight path 
altitude are not currently accessible to astronauts by 
means of present transport systems and therefore are 
excluded from any kind of human removal, repair, or 
maintenance. 

In contrast, satellites equipped with robotic arms 
or humanoid robonauts may be remotely controlled 
or only supervised from Earth in any orbit including 
the geostationary one. In the future, they should be 
able to support astronauts during routine and mainte¬ 
nance work on space stations, capture uncontrollably 
tumbling satellites, prolong their life-time by repair or 
refuel, deorbit or relocate them if necessary. Efficient 
telerobotic and telepresence technologies allow us to 
select flexibly the appropriate level of robot autonomy 
within shared autonomy frameworks between ground 
operator and space robot. 

Telepresence technologies as mentioned above 
make sure that by real-time feedback of stereo im¬ 
ages and force/torque information, the operator on 
the ground gets the feeling as though he was actu¬ 
ally working at the remote site. High-quality telep¬ 
resence requires low round-trip communication time 
delays. The challenge here is twofold: (a) to pro¬ 
vide the mentioned technically feasible communica¬ 
tion infrastructure, and (b) to apply the mentioned 
optimized delay-compensating telepresence technolo¬ 
gies which yield satisfactory haptic feedback up to 
650 ms delay. For large robots mounted on a car¬ 
rier satellite, their dynamic interactions, including 
the physical contacts when grasping a target, have 
to be mastered (Sect. 55.3). Autonomous skills are 
needed for supporting a human ground operator in 
performing the risky task of grasping and stabiliz¬ 
ing noncooperative tumbling targets (satellite or space 
debris). 

Due to the high cost of space validation missions, 
simulation capabilities are crucial for the develop¬ 
ment and verification of on-orbit servicing systems. 
This applies both for the required hardware simula¬ 
tion facilities including sensors and illumination ef¬ 
fects as well as for dynamics modeling techniques 


and software tools. Various hardware simulators us¬ 
ing industrial robots for simulation of chaser and tar¬ 
get satellite motion and the dynamic interaction with 
the space-robot have been built up not only in DLR 
(Fig. 55.35). 

Robotic maintenance and service missions for space 
infrastructure have been a long-term dream in the 
space robotics community since their conceptual de¬ 
signs were first published in ARAMIS report in the 
early 1980s (Fig. 55.1) [55.1]. 

ROTEX, ETS-VII, Ranger, and ASTRO that were 
introduced in the earlier section are technological de¬ 
velopments toward this goal, but robotic maintenance 
and service missions have not become routinely opera¬ 
tional yet (A good comparative study of orbital robotic 
missions is provided by [55.103]). The Hubble space 
telescope (HST) is a huge space telescope which has 
the capability to be serviced in orbit, but it has been 
visited by Space Shuttle and serviced (components ex¬ 
changed and trouble fixed) only by human EVA. After 
the COLUMBIA accident in 2003, NASA seriously 
considered the possibility of robotic maintenance of 
the HST, investigating available technologies and se¬ 
lecting a prime contractor of the mission development. 
Figure 55.36 depicts one possible configuration for the 
robotic rescue mission. Ultimately, it was decided to 
perform this last servicing mission with human astro¬ 
nauts. Maintenance of the HST involves tasks that are 
too complicated to be done by a robot, because the HST 
itself was designed for human-based maintenance and 
not specifically designed for robots. 

Robonaut, which is described in the following sub¬ 
section, is therefore considered as an interesting option 
to be capable in conducting practical maintenance and 
service missions, due to its compatibility and similar 
level of dexterity with human astronauts. 



Fig. 55.35 OOS-SIM, a hardware-in-the-loop simulator in 
DLR 
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Fig. 55.36 A conceptual drawing for robotic rescue of 
Hubble space telescope 


55.4.2 Robonaut and JUSTIN 


Robonaut is a humanoid robot designed by the Robot 
Systems Technology Branch at NASA Johnson Space 
Center in a collaborative effort with DARPA. The 
Robonaut project seeks to develop and demonstrate 
a robotic system that can function as an EVA astro¬ 
naut equivalent. Robonaut jumps generations ahead by 
eliminating the robotic scars (e.g., special robotic grap¬ 
ples and targets), but it still keeps the human operator in 
the control loop through its telepresence control system. 
Robonaut is designed to be used for EVA tasks (extra¬ 
vehicular activities or space walks), i. e., those which 
were not specifically designed for robots. 

A key challenge is to build machines that can help 
humans work and explore in space. Working side by 
side with humans, or going where the risks are too great 
for people, machines like Robonaut will expand capa¬ 
bilities for construction and discovery. Over the past 
five decades, space flight hardware has been designed 
for human servicing. Space walks are planned for most 
of the assembly missions for the ISS, and they are a key 
contingency for resolving on-orbit failures. To maintain 
compatibility with existing EVA tools and equipments. 


a humanoid shape and an assumed level of human per¬ 
formance (at least a human in a space suit) are required 
for this robotic surrogate. 

The manipulator and dexterous hand have been de¬ 
veloped with a substantial investment in mechatronics 
design. The arm structure has embedded avionics el¬ 
ements within each link, reducing cabling and noise 
interference. Robonaut has been designed based on 
a biologically inspired approach. For example, it uses 
a chordate neurological system in data management, 
bringing all feedback to a central nervous system, 
where even low-level servo control is performed. Such 
a biologically inspired approach is extended to left- 
right computational symmetry, sensor and power du¬ 
ality and kinematical redundancy, enabling learning 
and optimization in mechanical, electrical and software 
forms. 

Robonaut has a broad mix of sensors including ther¬ 
mal, position, tactile, force and torque instrumentation, 
with over 150 sensors per arm. The control system 
for Robonaut includes an onboard, real-time CPU with 
miniature data acquisition and power management. Off- 
board guidance is delivered with human supervision us¬ 
ing a telepresence control station with human tracking. 

Robonaut 2 (Fig. 55.37), the latest generation of 
the Robonaut family, launched to the ISS aboard Space 
Shuttle Discovery on the STS-133 mission in Febru¬ 
ary 2011. It is the first humanoid robot in space, and 
although its initial job is demonstrating its capabili¬ 
ties inside the space station, the goal is that through 
upgrades and advancements it will one day venture 
outside the station to help spacewalkers make repairs 
or additions to the station. Robonaut 2 is a dexterous, 
anthropomorphic robotic torso that has significant tech¬ 
nical improvements over its predecessor to make it a far 
more valuable tool for astronauts. Upgrades include: in¬ 
creased force sensing, greater range of motion, higher 
bandwidth, and improved dexterity. Robonaut 2’s inte¬ 
grated mechatronic design results in a more compact 
and robust distributed control system with a fraction of 
the wiring of the original Robonaut. 



Fig.55.37a-c NASA’s Robonaut family: (a) Robonaut 2, (b) Zero-G Leg for surface inspection of ISS, (c) Centaur with 
a surface mobility system 
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Fig.55.38a,b Robonaut 2 onboard 
ISS: (a) measuring airflow, (b) shaking 
hands with ISS Commander Dan 
Burbank 


Robonaut 2, also called R2, has completed many 
firsts during its two years on the ISS. During its ini¬ 
tial checkout, it used American sign language to say 
Hello World. R2 illustrated its unique control system 
design that permits it to work directly with astronauts 
by shaking hands with ISS Commander Dan Burbank 
(Fig. 55.38). More recently, it has been using standard 
crew tools to measure airflow and demonstrate its abil¬ 
ity to perform autonomous inventory scans. As part of 
gaining experience that will be useful once R2 starts 
working on the outside of the Space Station, on-board 
crew have successfully demonstrated teleoperation. Us¬ 
ing a variety of sensors that track human hand, arm and 
neck motion, astronaut Tom Marshburn, while also on¬ 
board the station, became the first person to remotely 
control R2 to have it catch a free flying object inside 
the ISS. 

One potential application of Robonaut technology is 
a regular monitoring and contingent maintenance work 
of human habitant modules of the space station. Fig¬ 
ure 55.37b depicts such an application where Robonaut 
crawls on the surface of the station module by using 
hand rails which were originally designed for human 
EVA. 

The application of the humanoid robot is not limited 
to orbital tasks. Figure 55.37c depict an idea to combine 
the humanoid torso on a surface mobility system, which 
shall be useful for robotic planetary explorations. 

DLR’s anthropomorphic JUSTIN is based on high- 
fidelity joint-torque-controlled light weight-technology 
and adjustable whole-body compliance in Cartesian 
space. JUSTIN on the mobile platform (Fig. 55.39) 
has actuated joints and torque controlled sensors. 
With JUSTIN’s upper body the new delay compen¬ 
sating technologies have been verified using copies of 
JUSTIN’s light weight arms as force reflecting hand- 
controllers up to delays of slightly more than 700 ms. 


The European space agency ESA too, is push¬ 
ing forward robonaut-type concepts, e.g., via testing 
a dexterous 4-finger-hand DEXHAND as developed in 
contract by DLR (Fig. 55.40) In spring 2012, the DEX¬ 
HAND successfully passed the acceptance test and is 
delivered to ESA. 

55.4.3 Aerial Platforms 

There are three planetary candidates for aerial robotic 
systems: Venus, Mars, and Titan (a moon of Sat¬ 
urn) [55.104, 105], Venus has a very dense but hot 
atmosphere (460 °C and 65kgm“" 3 at the surface), and 
so can easily float relatively heavy payloads. Mars has 
a very thin and cold atmosphere (somewhat variable but 
often —100 °C and 0.02 kgm -3 ). Titan has an atmo¬ 
sphere even colder than Mars (f« 100K) but about 50% 
denser than Earth’s atmosphere. Thus very different 
vehicles have been envisioned for the three candidate 
mission targets. On Venus, buoyant devices are gener¬ 
ally considered, especially those that can continuously 
or periodically rise high enough to reach moderate tem¬ 
peratures where conventional electronics can survive. 
One candidate is to use a phase-change fluid as part of 
the buoyant system, so that the fluid can condense in 
the cool upper atmosphere and be trapped in a pres¬ 
sure vessel, causing a loss of buoyancy and allowing 
the vehicle to descend, possibly all the way to the sur¬ 
face. After a brief stay, and before the heat flux to the 
interior of the device destroys all the sensitive equip¬ 
ment, a valve would be opened so that the phase change 
fluid can evaporate, increasing the buoyancy and allow¬ 
ing the craft to ascend to the cool upper atmosphere. 
After a suitable period of heat rejection into this cool 
zone, the process can be repeated, perhaps indefinitely. 
The density of the Venus atmosphere is sufficiently high 
that powered dirigibles can be used, so that the buoy- 
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ant vehicles can use propulsion and steering to reach 
particular locations in the atmosphere or on the sur¬ 
face [55.106]. 

In contrast, the Mars atmosphere is too thin for 
powered dirigibles to work (at least with the power- 
to-weight ratio of any current propulsion technology). 
Balloon aerobots could be deployed in the Mars atmo¬ 
sphere, and could ascend and descend, but probably 
could not be steered precisely to specific locations, at 
least not by use of a propulsion system. Polar bal¬ 
loons could circumnavigate either pole many times, 
or equatorial balloons could make one partial circuit 
around the planet, until they impact the Tharsis Bulge, 
a North-South string of high-altitude volcanoes that 
represents an essentially impenetrable barrier to any 
equatorial balloon having a reasonable payload. Be¬ 
cause of the problems with lighter-than-air vehicles 
in the thin Mars atmosphere, there has been consid¬ 
erable study of airplanes for use in exploring Mars. 
Aircraft can be designed to have reasonable lift-to-drag 
ratios in the Mars atmosphere, so that their performance 
is not too different from airplanes on Earth. Most of¬ 
ten considered are gliders that deploy directly from an 
aeroshell that comes in to the Mars atmosphere at hy¬ 
personic velocity, and then proceed to glide hundreds or 
a thousand kilometers before impact. One common mis¬ 
sion concept is to fly down the great Valles Marineris 
canyon, taking high-resolution imagery and spectrom¬ 
etry of the walls of that canyon. Powered aircraft have 
also been considered, including those that land and re¬ 
generate their propellant (e.g., using solar power and 
atmospheric CO 2 ) so as to be able to make multiple 
flights. 

On Titan, like Venus, buoyant devices are gener¬ 
ally considered more attractive than surface vehicles 
(although helicopters have been proposed). Also like 
Venus, the atmosphere on Titan contains many ob¬ 
scuring particles and aerosols so that high-resolution 
imaging over a broad spectrum is only possible by 
getting close to the surface. This makes balloons or 
powered dirigibles very attractive. On Venus the ex¬ 
treme surface temperature makes it challenging to make 
a surface vehicle operate for any extended duration. On 
Titan, there is a significant risk that some sort of hy¬ 
drocarbon goo exists on the surface that might foul any 
surface vehicle. Thus both Titan and Venus are consid¬ 
ered especially attractive targets for the use of aerobots, 
especially in the form of powered dirigibles. Navigation 
of such aerobots presumably would be accomplished 
primarily by sensing the terrain and navigating relative 
to any landmarks that can be discerned. When these 
vehicles operate in the upper atmosphere, they can aug¬ 
ment their position knowledge by means of sun or star 



Fig.55.39a,b DLR’ JUSTIN, wheeled version (a) and legged ver¬ 
sion (b) 



Fig. 55.40 DEXHAND 


tracking (as referenced to the local vertical). Deeper 
in the atmosphere, this may not be possible. One key 
issue is whether direct communications to Earth are en¬ 
visioned, or relay via satellite. If there is a satellite in 
orbit, it can provide considerable radio-navigation as¬ 
sistance and relatively frequent communications when 
the aerobot is on the side away from the Earth (both 
Venus and Titan spin very slowly). But a satellite 
relay is expensive, so the least expensive options re¬ 
quire that the dirigible have a large high-gain antenna 
(usually presumed to be inside the gas bag). Radio- 
based servo pointing at the Earth will provide precise 
navigation information (again along with precise mea¬ 
surements of local vertical). However, when the aerobot 
goes out-of-sight beyond the limb of the planet, it may 
spend days or weeks out of communications with the 
Earth. This is probably the situation calling for the 
highest degree of autonomy of any that have been en¬ 
visioned in robotic planetary exploration of the solar 
system. 
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55.4.4 Mobility Concepts 

and Subsurface Platforms 

For high mobility on Moon, planets, and asteroids there 
is still not a final answer which technology would be 
optimal. Although multilegged crawlers (e.g., DLR’s 
six-legged version as shown in Fig. 55.41) seem to be 
the best alternative for investigating steep craters, 4- 
wheeled rovers may climb up and down unbelievably 
steep slopes. May be wheel-leg combinations as re¬ 
alized in JPL’s ATF1LETE (Fig. 55.42) or in DLR’s 
conceptual design (Fig. 55.43) will turn out to be the 
optimal solution. 

Precise autonomous landing based on visual data is 
a prerequisite for exploration, closely related to the Fu¬ 
ture Space Systems program. 

DLR’s main interest however aims at fast loco¬ 
motion by local autonomy thus (including collision 
avoidance and real-time path planning) circumventing 
the problem of long signal delays from 3 s (Moon) 
to 15—30 min (Mars). Stereo cameras with field-pro¬ 
grammable gate array (FPGA) processor chips are ca¬ 
pable of modeling the environment in 3-D real time, 


L / 



Fig. 55.41 DLR’s six-legged crawler 



Fig. 55.42 JPL’s ATHLETE 


using e.g., the so-called semiglobal matching (SGM). 
Thus the goal of moving up to 10 km per hour seems 
realizable now. 

Other modes of mobility may be superior when 
gravity is e.g., only 10000 times smaller than that 
on earth as is the case on some asteroids, e.g., for 
a Japanese mission Hayabusa 2 a jumping shoe-box is 
developed by DLR using just a small excentric motor 
that causes moderate hopping motions over a few 100 m 
without reaching the fairly low escape velocity. 

Subsurface exploration of planetary bodies holds 
great promise: it is believed that a liquid-water aquifer 
may exist at significant depths on Mars, and perhaps 
an under-ice ocean on Europa and Ganymede which 
probably represent the best possible locations within the 
solar system to look for extant (as opposed to extinct) 
extraterrestrial life. Also, in the lunar polar dark craters 
there is some evidence of the existence of water ice or 
other volatiles, and perhaps there exists a layered ge¬ 
ologic record of impacts in the Earth-Moon system in 
these cold-traps. Even access to a depth of a few meters 
holds the promise of reaching pristine scientific sam¬ 
ples that have not been exposed to thermal cycling or 
ionizing radiation [55.1 07]. 

The prevailing wisdom has been that traditional 
sorts of drilling rigs are required to access deep un¬ 
derground, involving drill towers, multisegmented drill 
strings, large robotic systems to serve the function of 
a terrestrial drilling crew, and large power systems. 
Also, terrestrial drilling is usually done using large 
amounts of fluids (water, air, or mud) to flush away cut¬ 
tings and to cool and lubricate the cutter. The NASA 
Mars Technology Program has funded contractors that 
demonstrated reaching 10 m of depth in a realistic set¬ 
ting with segmented drill strings without the use of 
fluids. While this is much less than needed to reach the 
putative liquid water, it is much more than is reachable 
by previous techniques [55.108]. 



Fig. 55.43 Modular rover concept 
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Other approaches have been proposed such as 
Moles or Inchworms that could be relatively self- 
contained and yet might reach great depths without 
the mass and complexity of a large drill tower and 
segmented drill string. A key issue is that it appears 
that the needed energy cannot be stored on-board such 
self-contained drills, at least if it is stored as chemical 
energy. This is because drilling through terrain requires 
that some of the chemical bonds that hold the terrain 
together be broken, and so if the energy of chemi¬ 
cal bonds is used to provide that power, then a given 
volume of chemical energy storage can only advance 
some fixed ratio of its length into the terrain, where 
the ratio is determined by the efficiency in taking bond 
energies of one sort to break bonds of a different sort. 
Based on these considerations, it appears unlikely that 
a completely self-contained subsurface vehicle could 
advance more than perhaps a hundred times its own 
length. Unless nuclear power sources are considered 
(and they have been), this requires some sort of tether 
to the surface to provide a nearly unlimited source 
of energy. Another problem for subsurface vehicles is 
that rock tends to expand when it is pulverized (in 


a process called comminution). Nonporous rock typ¬ 
ically expands in volume by a few tens of percent 
when excavated, which means that fully self-contained 
subsurface vehicles have a severe conservation of vol¬ 
ume problem. In principle the rock can be compressed 
back into its original volume, but this generally re¬ 
quires pressures much greater than the compressive 
strength of the original rock. The energy required to do 
this is much larger than the energy required to exca¬ 
vate the rock in the first place, and would become the 
dominant use of energy in an already energy-intensive 
effort. 

As a result, it is generally assumed that any sub¬ 
surface vehicle must keep some access tunnel open to 
the surface so that the excess volume of cuttings can 
be transported out. If this tunnel is available, then it 
seems that a means for getting power from the surface 
is also available, so that self-contained nuclear power 
is not needed. Subsurface vehicles with diameters as 
small as one or a few centimeters have been proposed 
that could potentially reach great depths within the mass 
and power constraints of feasible planetary robotic ex¬ 
ploration missions. 


55.5 Conclusions and Further Reading 


Space robotics as a field is still in its infancy. The 
speed-of-light delays inherent in remote space opera¬ 
tions makes problematic the master-slave teleoperation 
approach that has been very useful in the undersea 
and nuclear industries. Space robotics lacks the highly 
repetitive operations in a tightly structured environment 
that characterize industrial robotics. Hardware handled 
by space robots is very delicate and expensive. All three 
of these considerations have led to the fact that rela¬ 
tively few space robots have been flown, they have been 
very slow in operation, and only a small variety of tasks 
have been attempted. Nonetheless, the potential rewards 
of space robotics are great - exploring the solar sys¬ 
tem, creating vast space telescopes that may unlock the 
secrets of the universe, and enabling any viable space 
industries all seem to require major use of space robots. 
The scale of the solar system is not so great (a few 


light-hours) that human intelligence cannot always sup¬ 
plement even the most remote space robot that becomes 
confused or stuck. Indeed, for the Moon (with only 
a few seconds of time delay) it seems that hazard avoid¬ 
ance and reliable closure of force-feedback loops is all 
that is required to make a highly useful robotic system. 
For Mars (with tens of minutes of time delay), along 
with hazard avoidance and force loop closure, it seems 
that robust anomaly detection (with modest reflexive 
safing procedures) and perhaps scientific-novelty de¬ 
tection are probably all that is needed. High levels of 
autonomy are enhancing but not enabling for work in 
the inner solar system, and become more and more de¬ 
sirable for robots that are sent farther into the outer solar 
system. 

For further reading, the following materials are sug¬ 
gested [55.109-114], 
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ItstjMI'H'tfU DLR ROTEX: The first remotely controlled space robot 

available from http://handbookofrobotics.org/view-chapter/55/videodetails/330 
l<gj>»im'Kia DLR predictive simulation compensating 6 seconds round-trip delay 

available from http://handbookofrobotics.org/view-chapter/55/videodetails/331 
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DLR GETEX manipulation experiments on ETSVII 

available from http://handbookofrobotics.org/view-chapter/55/videodetails/332 
DLR ROKVISS animation 

available from http://handbookofrobotics.org/view-chapter/55/videodetails/333 
DLR ROKVISS camera images pulling spring 

available from http://handbookofrobotics.org/view-chapter/55/videodetails/334 
DLR ROKVISS disassembly 

available from http://handbookofrobotics.org/view-chapter/55/videodetails/336 
DLR telepresence demo remove cover 

available from http://handbookofrobotics.org/view-chapter/55/videodetails/337 
DLR telepresence demo with time delay 

available from http://handbookofrobotics.org/view-chapter/55/videodetails/338 
DLR DEOS demonstration mission simulation 

available from http://handbookofrobotics.org/view-chapter/55/videodetails/339 
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56. Robotics in Agriculture and Forestry 


Marcel Bergerman, John Billingsley, John Reid, Eldert van Henten 


Robotics for agriculture and forestry (A&F) rep¬ 
resents the ultimate application of one of our 
society's latest and most advanced innovations 
to its most ancient and important industries. 
Over the course of history, mechanization and au¬ 
tomation increased crop output several orders of 
magnitude, enabling a geometric growth in pop¬ 
ulation and an increase in quality of life across 
the globe. Rapid population growth and rising 
incomes in developing countries, however, re¬ 
quire ever larger amounts of A&F output. This 
chapter addresses robotics for A&F in the form 
of case studies where robotics is being success¬ 
fully applied to solve well-identified problems. 
With respect to plant crops, the focus is on the 
in-field or in-farm tasks necessary to guarantee 
a quality crop and, generally speaking, end at 
harvest time. In the livestock domain, the focus 
is on breeding and nurturing, exploiting, harvest¬ 
ing, and slaughtering and processing. The chapter 
is organized in four main sections. The first one 
explains the scope, in particular, what aspects 
of robotics for A&F are dealt with in the chap¬ 
ter. The second one discusses the challenges and 
opportunities associated with the application of 
robotics to A&F. The third section is the core of 
the chapter, presenting twenty case studies that 
showcase (mostly) mature applications of robotics 
in various agricultural and forestry domains. The 
case studies are not meant to be comprehen¬ 
sive but instead to give the reader a general 
overview of how robotics has been applied to 
A&F in the last 10 years. The fourth section con¬ 
cludes the chapter with a discussion on specific 
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Robotics for agriculture and forestry represents the ulti¬ 
mate application of one of our society’s latest and most 
advanced innovations to its most ancient industries. 
Since the dawn of civilization, agriculture and forestry 
(A&F for short) remain chief among humankind’s most 
important economic activities, providing the food, feed, 
fiber, and fuel necessary for our survival. (In this chap¬ 
ter, agriculture is understood as in the Merriam-Webster 
definition: 

the science, art, or practice of cultivating the soil, 
producing crops, and raising livestock and in vary¬ 
ing degrees the preparation and marketing of the 
resulting products. 

Therefore, the term crop may be used here to denote any 
product of an agricultural or forestry process, includ¬ 
ing grains, cereals, fruit, vegetables, nuts, trees, beef, 
wool, etc. Whenever necessary, we will differentiate 
plant from animal products appropriately.) 

Over the course of history, mechanization and au¬ 
tomation - from the manual ploughs of yore to the 
modern combines of today - increased crop output 
several orders of magnitude. This, in turn, enabled 
a geometric growth in population and a correspond¬ 
ing increase in quality of life across the globe. Rapid 
population growth and rising incomes in developing 
countries, however, require ever larger amounts of A&F 
output. Scientists predict that agricultural production 
must double to meet the demands of nine billion people 
in 2050 [56.1-3]. Clearly, this cannot be achieved by 
simply doubling the inputs (land, water, seeds, labor, 
etc.) because of constrained resources and environ¬ 
mental concerns. Therefore, the efficiency of the A&F 
system must increase in a sustainable and consistent 
manner. 

56.1 Section Scope 

Before presenting the case studies that showcase recent 
advances in robotics applied to A&F, we must define 
the scope of the chapter, both in terms of crop produc¬ 
tion processes and robotic technologies. 

A typical crop production cycle includes sev¬ 
eral processes, among them field preparation, seed¬ 
ing/breeding, transplanting, planting, growing, main¬ 
tenance (including attaching plants to support struc¬ 
tures, disbudding, removing leaves, pruning tree limbs 
and shoots, thinning blossoms and fruit, nurturing ani¬ 
mals, etc.), exploiting/harvesting/slaughtering, sorting, 
and packing. In all of them, internal transport of peo¬ 
ple, machines, and produce play a role. Single-harvest 
crops such as lettuce must be replaced once harvested; 


According to Reid [56.4], global agricultural total 
factor productivity (TFP), or the output per unit of to¬ 
tal resources used in production, must increase from 
the current 1.4 to 1.75 to double agricultural output 
by 2050. This requires significant scientific, techno¬ 
logic, and management advances in all of the factors 
that impact TFP - seeds, soil, water, fertilizers, herbi¬ 
cides, insecticides, crop architecture, cultural practices, 
automation, labor, public policy, etc. While robotics is 
but one of these factors, it is one with the potential to 
effect A&F in a broad, systemic way, and contribute 
significantly to meeting our future needs. 

In this chapter we address the field of agricul¬ 
tural and forestry robotics from the point of view of 
the applications it enables, rather than from the point 
of view of the elementary technologies it comprises. 
Furthermore, we chose to present a limited number 
of case studies where robotics is being successfully 
applied to solve well-identified problems, rather than 
a comprehensive survey of all work reported in the lit¬ 
erature. We believe the former is more meaningful as it 
showcases top-down, problem-oriented solutions ( mar¬ 
ket pull) rather than bottom-up, technology-led ones 
(technology push). We focus on examples from the last 
five to ten years, as they represent work that leverages 
the most recent advances in sensors and computing. 

We address applications of agricultural and forestry 
robotics in the form of twenty case studies where 
robotics is being successfully applied to solve well- 
identified problems. With respect to plant crops, the 
focus is on the in-field or in-farm tasks necessary to 
guarantee a quality crop and, generally speaking, end at 
harvest time, before the crop is transported to a packing 
plant or warehouse. In the livestock domain, the focus 
is on breeding and nurturing, exploiting, harvesting, and 
slaughtering and processing. 


multiple-harvest crops such as apples, tomatoes, and 
roses last a year or even several years before they need 
to replanted. Depending on the crop, machinery with 
varying levels of automation exists for some or all of 
these processes. In grain and cereal production, for ex¬ 
ample, farmers have access to commercial machines 
for tilling, seeding, transplanting, spraying, irrigating, 
and harvesting. In fresh fruits and vegetables, on the 
other hand, mechanization and automation are more 
prevalent at the early and late stages of the produc¬ 
tion cycle [56.5], with crop maintenance and harvest 
remaining for the most part manual tasks. 

With respect to plant crops, this chapter’s scope 
is limited to the in-field or in-farm tasks necessary 
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to guarantee a quality crop and, generally speaking, 
end at harvest time, before the crop is transported to 
a packing plant or warehouse. Of course, there are 
many opportunities for robotics in those latter environ¬ 
ments, e.g., automatic sorting and grading. We exclude 
them from our presentation as they are currently much 
more the focus of automation efforts than of robotics 
per se. In future editions of the Handbook, we will re¬ 
visit this separation as robotics advances to post-harvest 
tasks. 

In the livestock domain, much of farming concerns 
the nurturing, exploitation, harvesting, and slaughtering 
of animals. Slaughtering extends beyond the killing of 
the animal to its division into marketable portions, and 
subsequent packing and marketing. The essential pro¬ 
cesses considered in this section pertain to land animals 
and birds and include: 

56.2 Challenges and Opportunities 

Since the end of the industrial revolution, (arguably) the 
three most significant impacts robotics and automation 
have had on agriculture and forestry are: 

1. Precision agriculture, or the use of sensors to pre¬ 
cisely control when and where to apply inputs such 
as fertilizers and water 

2. Auto-guidance on field crop machinery, which to¬ 
day can drive down a field with an accuracy 
unattainable by human drivers 

3. Machines that harvest fruits and vegetables for pro¬ 
cessing (e.g., tomato paste and orange juice). 

Academic and commercial researchers are now fo¬ 
cusing on the next wave of sensing, mobility, and ma¬ 
nipulation technologies that promise to increase A&F 
output and productivity. 

Sensing entails measuring crop temperature, hu¬ 
midity, pH, wetness, image, range, and other physical 
attributes, and combining and analyzing the data for 
specific purposes. One example is a camera-based sys¬ 
tem that takes pictures of an apple orchard a few 
weeks before harvest and produces an accurate crop 
yield estimate that growers can use to plan and manage 
the harvest operation [56.6,7]. The utility of sensing 
to A&F is that it enables decision-making at a level 
unattainable by human sensing alone, because the lat¬ 
ter is either inherently inaccurate or slow or both. 

Mobility relates to various levels of vehicle au¬ 
tomation that enable driverless (or driver-assistive) 
field coverage; the most common example is a global 
positioning system/global navigation satellite system 
(GPS/GLS)-guided combine that harvests corn with 
minimal intersection between passes on the field, thus 


• Breeding and nurturing: Livestock can range from 
poultry to cattle in feedlots, including pigs and other 
animals. Their care involves environment and be¬ 
havior monitoring, plus fodder distribution. 

• Exploiting: In many cases, products are obtained 
from the live animals. Sheep are shorn for wool, 
cows are milked, chickens lay eggs, and bees pro¬ 
duce honey. 

• Harvesting: Mustering and collecting free-range 
cattle, feral pigs, and other unconstrained animals. 

• Slaughtering and processing: Cattle, poultry, pigs, 
other animals are killed subject to strict regulations, 
then divided into marketable portions. 

In a future edition of the Handbook fishing should 
be included, since many opportunities for robotics can 
be imagined there. 


optimizing coverage and minimizing fuel consump¬ 
tion [56.8]. More recently, auto-guidance has started to 
migrate to orchard vehicles as well, albeit here other 
navigation sensors may be required because of poor 
satellite reception under thick canopies 
lanawinmuMt [ n A&F, automated vehicles equipped 
with the appropriate implements enable (semi-) au¬ 
tonomous seeding, spraying, mowing, weed removal, 
harvesting, and animal feeding, among other opera¬ 
tions dda M'JMi'HCl Mobility requires 

some level of sensor-based perception, provided by 
GPS/GNSS, inertial units, cameras, ladars, radars, etc. 
These sensors are not to be confused with those de¬ 
scribed in the previous paragraph, although there cer¬ 
tainly are situations where a sensor can perform double 
duty of sensing for decision making and for navigation. 
In general, robotic mobility technology is currently less 
advanced than sensing. 

Manipulation refers to the various operations per¬ 
formed directly on the crop, including pruning, thin¬ 
ning, harvesting, tree training, leaf probing, tree cutting, 
weed removal, etc. In general, this technology requires 
more sophisticated sensor-based perception than mobil¬ 
ity, and in terms of field deployment is less advanced 
than either sensing or mobility. 

Two domains that are particularly challenging for 
robotics research are orchard crops and crops in pro¬ 
tected cultivation. (We use the term orchard liberally 
to include grape vineyards and orange groves, among 
other similar environments where crops are grown in 
well-defined rows.) These are highly valuable crops, 
potentially generating one to three orders of magnitude 
more revenue per acre than field crops. They are also 
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characterized by the need for intensive cultivation and 
skilled labor. Consider, for example, apple production 
in the US - no less than 50 to 60% of the variable 
costs to produce an apple is due to labor [56.9]. On 
top of that, labor is needed in bursts - in Washing¬ 
ton state, in the US, up to seven times more workers 
are needed during the fall harvest season than during 
the winter pruning season. In many developed coun¬ 
tries, labor availability for manual orchard operations 
is a challenge, putting significant pressure on growers 
to find innovative solutions to address their labor re¬ 
quirements. In the 2011 Tree Fruit Industry Perspective 
publication, industry leaders with the US Northwest 
Farm Credit Services say that: 

[...] seasonal labor force utilized by the tree fruit 
industry for thinning and harvest operations will al¬ 
ways be an issue of concern [...]. Labor shortages 
at various times throughout the growing season 
have occurred and will likely occur in the future. 
The perishable nature of the tree fruit mandates 
a strict timeliness in field operations, and the gener¬ 
ally narrow harvest windows do not accommodate 
labor shortages at critical stages. [...] New tech¬ 
nology [...] could reduce labor requirements sub¬ 
stantially over the next five to 10 years. Specifically, 
such technology would include the use of platforms 
and mechanical-harvest methods in the orchards, 
and increased use of robotics [...]. 

Partly in response to the high labor costs and partly 
to increase production efficiency, the tree fruit industry 
is moving in the direction of highly structured planting 
architectures. Whereas before an apple grower had two 
to four hundred trees per acre, she now has 1200 apple 
trees per acre in a fruit wall configuration that is much 
more conducive to automation (Fig. 56. la). This has led 
to the development of autonomous vehicles and tractors 
that drive from garage to block and traverse tree rows 
for hours on end, including turning from row to row, 



Fig. 56.1 (a) Modem apple orchard with trees arranged to form 
a fruit wall, (b) Vine tomato in a protected cultivation environment. 
These new architectures open the door to robotics-based production 
technologies that increase efficiency and reduce labor costs 


without any human assistance [56.10]. When equipped 
with the appropriate implements, these robotic vehicles 
can mow, spray, and collect tree and crop data for inven¬ 
tory management; and when configured as platforms, 
they can carry workers pruning, thinning, training trees, 
and harvesting the top part of the trees, thus eliminating 
the inefficiencies and injuries related to ladders. In the 
future, manipulators mounted on such vehicles will be 
able to probe plants for phenotyping purposes, and au¬ 
tomatically prune, thin, and harvest. 

A protected cultivation system is a powerful in¬ 
strument to produce crops (Figure 56.1b). Greenhouses 
protect crops from unfavorable climate conditions and 
pests and offer the opportunity to modify the climate 
to create an environment that is optimal in terms of 
both crop quality and quantity. A protected cultivation 
is an intensive production method with high investment 
and operational costs, therefore permitting only the pro¬ 
duction of high value fruit and vegetable crops such as 
tomatoes, sweet peppers and cucumbers, flowers such 
as roses, chrysanthemums and gerberas and many types 
of potted plants. In the past decades, in western soci¬ 
eties, this type of production has been confronted with 
increasing size of production facilities, increasing la¬ 
bor costs, reduced availability of sufficiently skilled 
labor, health problems of the employees due to heavy 
and repetitive tasks, and growing competition on the 
national and international markets. Automation and 
robotics are considered to be a way to address these is¬ 
sues. Additionally, growing concerns with food safety 
call for such technology. Last but not least, more and 
more a precision horticulture approach is adopted in 
which plants are treated on an individual basis, to im¬ 
prove quantity and quality of crop production whilst 
using resources as efficiently as possible. Given the cur¬ 
rent constraints on human labor, this has led to an even 
stronger call for automation and robotics. 

Within the context of agricultural production, in 
terms of production area, protected cultivation is world¬ 
wide a relatively small business. Total area in use with 
protected cultivation worldwide is estimated at roughly 
740000ha [56.11]. In terms of added value, however, 
protected cultivation plays a much more important role. 
In The Netherlands at a few percent of the area avail¬ 
able for agricultural production, horticulture produces 
roughly 35% of the economic return of total agricultural 
production. In that case production is very capital and 
labor intensive. On a global scale, the potential of pro¬ 
tected cultivation is being increasingly acknowledged. 
Yet, technology levels still vary widely, largely based 
on significant differences in local conditions in terms of 
market, economy, resource availability, etc. 

Robotics research in protected cultivation has a his¬ 
tory of some thirty years [56.12], focused mainly on 
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harvesting and chemical spraying operations. Robotic 
harvesting focused mainly on tomatoes, cucumbers, 
eggplants, sweet peppers, and strawberries, for which 
multiple examples of research prototypes have been de¬ 
veloped [56.12] (losSMSEH). Single examples are 
known of a harvesting robot for roses [56.13] and ger- 
bera [56.14]. Spraying robots have been reviewed in 
detail in [56.15]. Only two robotic systems, other than 
for harvesting or spraying, were found in the literature: 
a leaf picking robot for cucumber [56.16] and a robot 
for bagging, thinning, and spraying in grapes [56.17]. 
In protected cultivation robots have not yet been devel¬ 
oped for actions like pruning, disbudding and attaching 
of plants to support structures. 


56.3 Case Studies 

In this section we present a variety of case studies where 
robotics technologies are being applied successfully to 
problems in the agriculture and forestry sectors. The 
presentation follows a somewhat loose categorization 
by application domain, i. e., field crops, orchard crops, 
protected cultivation, forestry, and livestock. The point 
here is to group examples that are domain-specific but 
allow for the discussion of those that straddle across 
domains. Within each domain, the case studies follow 
(again, loosely) the progression from sensing to mobil¬ 
ity to manipulation. 

56.3.1 Optimized Coverage 
for Arable Farming 

Agricultural researchers and practitioners have long- 
desired to have the capability to follow well-defined 
traffic lanes with common-width equipment sys¬ 
tems to minimize soil compaction effects on plant 
growth [56.18]. The advent of automation and control 
technology (e.g., GNSS, automatic steering) eliminated 
the need for complex processes to define the controlled- 
traffic lanes by making it possible for agricultural 
machine systems to follow precise paths spatially and 
temporally. 


Challenges in livestock production are of a more 
practical nature. When cattle roam freely they can be 
difficult to find and muster. Spotting by helicopter is 
expensive so there is great potential for the use of un¬ 
manned aerial vehicles. The method of collecting feral 
pigs and native kangaroos is usually by shooting, sup¬ 
plying markets in Germany and Russia respectively. 
There could in theory be an opportunity for the applica¬ 
tion of robotics, but it is difficult to see how this could 
be compatible with safety. It has been suggested that 
cattle could be located by carrying a transmitter. Ra¬ 
dio communication involves considerable distances, so 
the use of GPS/GNSS collars with transmitting systems 
would pose interesting power and battery problems. 


With the rapid adoption of automatic guidance 
systems, automated path planning has great potential 
to further optimize field operations. Field operations 
should be done in a manner that minimizes time and 
travel over field surfaces and is coordinated with spe¬ 
cific field operations, machine characteristics, and to¬ 
pographical features of arable lands. To reach this goal, 
Jin and Tang [56. 19] proposed an optimal coverage path 
planning (OCPP) algorithm where coverage is repre¬ 
sented by a geometric model. To determine the full cov¬ 
erage pattern of a given field by using boustrophedon 
paths, it is necessary to know whether and how to de¬ 
compose a field into sub-regions and how to determine 
the travel direction within each sub-region. The search 
mechanism is guided by a customized cost function re¬ 
sulting from the analysis of different headland turning 
types and implemented with a divide-and-conquer strat¬ 
egy. The complexity of the algorithm is 0(n 3 log(n 3 )) 
for a field with n edges. In order to reduce the total 
turning cost, the number of turns needed to be mini¬ 
mized. Besides, turns with relatively high operational 
costs needed to be avoided. Fields of irregular shapes 
had inefficiencies related to headland turns when head¬ 
lands were at an angle to the machine. Two-dimensional 
field examples with complexity ranging from a sim- 




Fig. 56.2 Results obtained with 
the optimal coverage path planning 
algorithm for two-dimensional (2-D) 
terrains; the inner polygons indicate 
non-traversable obstacles 
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Fig.56.3a-c Results obtained with the optimal coverage path planning algorithm for a 3-D terrain with terraces and 
valleys, (a) Aerial image, (b) Topographic map. (c) Optimal coverage path 


pie convex shape to an irregular polygonal shape that 
has multiple obstacles within its interior were tested 
with the OCPP algorithm (Fig. 56.2). The results show 
that, in the most extreme two-dimensional cases, OCPP 
saved up to 16% in the number of turns and 15% 
in headland turning cost. There were no cases where 
OCPP outputted solutions worse than those adopted by 
farmers. 

When optimizing coverage path over three-dimen¬ 
sional (3-D) terrains, more factors need to be consid¬ 
ered, including headland turning cost, soil erosion cost, 
and skipped area cost. Jin and Tang developed an ana¬ 
lytical 3-D terrain model with B-splines; and analyzed 
different categories of coverage costs on 3-D terrains 
and developed methods to quantify soil erosion cost and 
curving path cost corresponding to a particular coverage 
solution. Similar to the 2-D coverage path optimiza¬ 
tion, terrain decomposition and classification methods 
are used to divide a field into sub-regions with similar 
field attributes and comparatively smooth boundaries. 
The divide-and-conquer strategy was also applied in the 
3-D terrain coverage planning. The most appropriate 
path direction of each region was the one that achieved 
the minimum coverage cost. A seed curve searching 
algorithm was successfully developed and applied to 
several practical farm fields with various topographic 


features (Fig. 56.3). The 3-D planning algorithm has 
shown its superiority on 3-D terrain fields compared 
with the 2-D planner. On the tested fields, on aver¬ 
age the 3-D version saved 10.3% on headland turning 
cost, 24.7% on soil erosion cost, 81.2% on skipped area 
cost, and 22.0% on the weighted sum of these costs, 
where their corresponding weights were 1, 1, and 0.5, 
respectively. In one of the regions, in particular, the 
3-D planning algorithm generated a result with only 
30.5% of the soil erosion caused by the 2-D planning 
algorithm. It was also observed that the skipped area 
resulting from the sharp turning curvature in 3-D plan¬ 
ning is generally much smaller than the skipped area 
between paths when projecting 2-D planning results to 
a 3-D surface. 

56.3.2 Weed Control 

Weeds compete with the production crop for light, wa¬ 
ter, and nutrients and can have a detrimental impact 
on crop yield if uncontrolled [56.20]. For these rea¬ 
sons chemical and mechanical weed control have long 
received attention of agricultural engineers. Compared 
to current methods, robotics offers the opportunity to 
improve this important production task in two ways: 
greater precision when done mechanically, and reduced 
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Fig. 56.4 A robot to detect and control broad-leaved dock 
in grassland. Weeds are detected with a texture-based 
algorithm using images from a color camera (A). Nav¬ 
igation is GPS-based (B). A diesel engine (C) provides 
hydraulic power for the weeding implement (D), which can 
be moved laterally along the rail (E) 

emissions and environmental runoff when done chem¬ 
ically. This section describes three examples of the 
application of robotics to weed control; for an overview 
of the field see [56.20]. 

Broad-leaved dock is a troublesome weed in grass¬ 
land. When not controlled it may reach a high density 
and considerably reduce grass yield. In response to a re¬ 
quest of seventeen organic Dutch dairy farmers, a robot 
was constructed to detect and control broad-leaved dock 
in grassland [56.21] (Fig. 56.4) It con¬ 

sists of a custom platform with four independently 
driven wheels powered by a 36 kW diesel engine. The 
wheels are fitted with golf cart tires that provide traction 
on grass with minimal impact on the sod. Reduction 
gears ensure high torque, allow centimeter-precision 
forward movement, and limit maximum speed to a safe 
3 mph. Skid steering was deemed sufficiently precise 


under the circumstances. Traversal of a large, feature¬ 
less pasture was achieved by using real-time kinematic 
GPS (RTK-GPS) to follow a pre-defined path consist¬ 
ing of parallel segments connected by half-circle turns 
at the ends. Because weeds and grass are both green, 
a texture-based image analysis method was used to de¬ 
tect the former [56.22,23], The weeder proposed by 
Riesenhuber was used [56.24]. The method consists of 
a rod weeder driven vertically into the ground to frag¬ 
ment the weed’s taproot. 

Volunteer potatoes are the leftovers of the previ¬ 
ous year’s harvest. After a mild winter these potatoes 
will sprout and constitute a serious weed. They not 
only compete with the production crop, but also po¬ 
tentially carry diseases. In The Netherlands, legislation 
requires that they be removed annually by July 1st. 
This labor-intensive task naturally called for automa¬ 
tion. As a joint effort of industry, policy makers, and 
scientists, a project to detect and control volunteer 
potato plants was initiated [56.25-27]. A proof-of- 
principle machine for sugar beet fields has been built 
and tested (Fig. 56.5). Machine vision-based detection 
at 100 mm 2 precision was combined with a micro¬ 
sprayer with five needles and a working width of 0.2 m. 
The accuracy of the system was ±14 mm in the lon¬ 
gitudinal direction and ±7.5 mm in the transversal 
direction. The main error source was the variability 
in micro-sprayer droplet velocity that caused longi¬ 
tudinal errors. Still, 77% of volunteer plants with 
a size larger than 1200 mm 2 were successfully con¬ 
trolled at velocities up to 0.8 ms -1 . Within the seed 
line, glyphosate was applied on weed potato plants, ac¬ 
companied with up to 1.0% unwanted killed sugar beet 
plants. 

The Intelligent Autonomous Weeder (Fig. 56.6) is 
a four-wheel steer, four-wheel drive platform to be 
used for autonomous weeding operations in arable 
farming [56.28-30]. The platform combines dual GPS- 



Fig.56.5a-d Precision 
removal of volunteer potatoes 
in sugar beets, (a) Computer 
vision-based detection of 
the potato weeds {red) 
among the sugar beet plants 
(green), (b) A sugar beet field 
infested with potato weeds. 

(c) A micro-spray system. 

(d) The removal device pulled 
by a tractor 
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Fig. 56.6 The Intelligent Autonomous Weeder operates in 
arable farming land 


based navigation with computer vision-based row fol¬ 
lowing. The four wheel steering construction offers 
supreme maneuverability, which is not only an advan¬ 
tage when precise operation within the crop is consid¬ 
ered, but also offers the opportunity of very compact 
head land turns. 

56.3.3 High Precision Seeding 

The premise behind this work is that, if an autonomous 
agricultural machine could accurately follow a prede¬ 
fined path to carry out seeding, then the same machine 
could drive in the field throughout the entire growing 
season carrying out subsequent tasks - e.g., weeding, 
fertilizing, spraying, etc. - without the need for repeated 
crop location sensing (l<£4Kii>H!l£lB). The primary 
challenge is to develop centimeter-level path tracking 
accuracy along straight runs by the mobile agricul¬ 
tural machines, and even greater accuracy (1—2 cm) in 
agricultural tool manipulation, e.g., controlling seeding 
tines for seed placement. 

While there are semi-automated seeding systems 
currently on the market, they suffer from a number 
of problems with major implications for environmental 
sustainability, productivity, and economic return: 

• They are passive, i. e., unable to take action to cor¬ 
rect the seeding tines’ path. 

• The seeding implements are towed by a tractor 
which is GPS-guided, but with insufficient accuracy 
(usually 40 cm); moreover, accurate path following 
by a tractor does not guarantee accurate seed place¬ 
ment by the implement. 

• The tractor does not sense deviation of the seeding 
implement’s path, causing the crop to be laid out 
unpredictably. 


• Even when the system senses implement deviation, 
it does not have the ability to correct the seeding im¬ 
plement’s path, let alone the position of the seeding 
tines. 

• Tractors used in current systems are massive, there¬ 
fore must use fixed tracks to limit ground com¬ 
paction (however still causing compaction of up to 
20% of arable land). 

• Insufficient seeding accuracy does not permit inter¬ 
row cropping in alternating seasons - a technique 
that utilizes remnant nutrients in inter-row spaces, 
and 

• Present day systems do not have fully autonomous 
operational capability. 

Katupitiya et al. [56.31] built and delivered a sys¬ 
tem that advanced the field in the following ways 

(Fig. 56.7): 

• An active seeder equipped with sophisticated con¬ 
trol systems that can take corrective action against 
path deviations while ensuring high accuracy seed¬ 
ing tine position control. 

• High accuracy localization of the seeder and tines 
achieved via precise GPS, high-precision sensors, 
data fusion, and control software to locate/position 
the seeding tines. Control systems include those that 
micro-adjust the seeding tines with respect to the 
main seeder frame. This allows the seeding tine 
positions to follow a more accurate path than the 
seeder itself. 

• The seeder is force-controlled and self-propelled, 
allowing for the size of the tractor to be greatly re¬ 
duced. Smaller tractors also mean smaller wheels, 
so the tractor wheel width can be smaller than 
the inter-crop row width; hence the tractor can 
use the inter-row space as its wheel tracks with¬ 
out squashing the crop for subsequent operations. 



Fig. 56.7 Precision seeding system for broad acre farming 
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Fig. 56.8 Autonomous crop yield estimation system hardware, 
composed of concentric cameras and ring flashes for controlled il¬ 
lumination during nighttime operations 


These result in the complete elimination of ground 
compaction. 

• The level of automation integrated into the tractor- 
seeder pair is such that the entire system is readily 
autonomous with no operator required. 

• The seeder is modular, with either single or multiple 
units operating around the clock. A special purpose 
tandem, non-linear adaptive pursuit path tracking 
algorithm is used to control the steering of the trac¬ 
tor and the seeder wheels. The seeder has its wheels 
under force control based on the tension at the off- 
axle hitch point. 

56.3.4 Crop Yield Estimation 

One common desire of all fruit growers is knowledge of 
the crop yield. Accurate yield prediction helps growers 
improve fruit quality and reduce operating cost by mak¬ 
ing better decisions on intensity of fruit thinning and 
size of the harvest labor force. It benefits the packing 
industry as well, because managers can use estimation 
results to optimize packing and storage capacity. 

Typically, yield estimation is performed based on 
historical data, weather conditions, and workers man¬ 
ually counting fruit in multiple sampling locations. 
Manually gathering samples is a time-consuming, la¬ 
bor-intensive, and inaccurate process, and the number 
of samples is usually far too small to capture the magni¬ 
tude of the variation in yield across each block. Growers 
are searching for an automated and efficient alterna¬ 
tive that can accurately capture spatial variation in 
yield. 

To deal with this need, Nuske et al. [56.6,7] de¬ 
veloped a computer vision-based system to detect and 
count fruit. The system uses a camera rig for image ac¬ 
quisition, working at nighttime with controlled artificial 
lighting to reduce the variance of natural illumination. 
An autonomous vehicle is used as the support platform 
for automated data collection (Fig. 56.8). The system 
scans both sides of each row of trees or vines, detect¬ 
ing fruit captured within the image sequence, and then 
generates yield estimates. 

The accuracy of the system was demonstrated by 
comparing its crop yield estimation with ground-truth 
recorded via careful and tedious manual measurements. 
Its end result is a yield map that closely resembles 
the true spatial distribution of yield, which growers 
can utilize to make critical production management 
decisions. 

The yield estimation system was deployed in 
a number of vineyards, apple orchards, and strawberry 
ranches, and the results show that the system works well 
in a variety of crops and training structures (Figs. 56.9 
and 56.10). 


Fig.56.9a-c Output of the crop yield estimation sys¬ 
tem in vineyards (a), apple orchards (c), and strawberry 
ranches (b) 


Current irrigation practice within the agricultural com¬ 
munity often dictates over-watering crops as opposed 
to under-watering. This results in wasted resource, in¬ 
creased leaching of fertilizers, and an increase in crop 
disease. There are crop models that determine adequate 
irrigation quantities, but they are seldom used as the 
constant modifying of irrigation parameters is tedious 
and difficult. 

Kohanbash et al. [56.32] used wireless sensor net¬ 
works (WSN’s) to monitor environmental conditions in 
real time and adjust irrigation parameters on the fly. 
WSN’s can do basic set-point control where irrigation is 
enabled every time the soil moisture goes below a pre¬ 
defined threshold. More advanced control methods are 
also possible by integrating crop water use models into 
the system. WSN systems communicate with a central 
base station that can be connected to the Internet, al¬ 
lowing remote access for viewing crop condition and 
modifying irrigation settings. Having a central base sta- 


56.3.5 Precision Irrigation 
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Fig.56.10a,b Spatial yield 
map collected by the system: 

(a) Truth manual count, 

(b) automated count 


Table 56.1 Irrigation improvements at a greenhouse by us¬ 
ing a wireless sensor network system (after [56.32]) 


Condition 

Irrigations per day 

Before WSN system installed (baseline) 

8 

With WSN system (monitoring) 

4-5 

With WSN system (monitoring and 
control) 

2-3 


tion also allows growers the ability to monitor and 
analyze the crop growing environment for trends and 
long-term changes. 

Early results (Tab. 56.1) show water savings of up 
to 75%, fertilizer leaching reduced to near zero, in¬ 
creased crop quality, speed of crop growth increased 
by over 50%, and reduced occurrence of crop disease. 
In addition to these savings, growers can use the data 
from WSN systems to tailor their crop for specific mar¬ 
kets. For example, by adjusting the irrigation set-point 


value, a grower can choose between more expensive 
(Grade A) product, or less expensive (Grade B) prod¬ 
uct that they can sell more of. 

56.3.6 Tree Fruit Production 

Autonomous orchard vehicles will radically transform 
tree fruit production by automating maintenance op¬ 
erations such as mowing and spraying and augment¬ 
ing workers pruning, thinning, and harvesting. From 
a robotics point of view, these applications can be 
realized with a relatively simple, yet challenging, ca¬ 
pability: driving along one row of trees, turning at the 
end of the row, and entering the next one. Challenges 
involved include reliably sensing the trees in the pres¬ 
ence of sloped terrain, branches, tall grass, and missing 
trees; localizing the vehicle in the orchard; following 
trajectories inside and outside the rows; and avoiding 


Table 56.2 Control modes enabled by the family of autonomous orchard vehicles in Fig. 56.11, from least to most 
complex 


How it works 

Mule mode 

Scaffold mode 

Pace mode 

The vehicle drives 
along rows of trees as 
workers harvest fruit, 
placing them in bins 
on the vehicle 

Farm workers stand on the vehicle 

while it self-steers in the row 

The vehicle autonomously drives an 
entire block at a time without requiring 
any further interaction 

Production tasks 

enabled 

Fruit harvesting 

Pruning, fruit and blossom thin¬ 
ning, tree maintenance, harvesting, 
pheromone dispenser placing 

Mowing, spraying, and scouting for 
disease, insects, and crop yield estima¬ 
tion 

Autonomous 

functionalities 

Row following (continuous or stop-and-go), end-of-row detec¬ 
tion, obstacle detection 

Row following, end-of-row detection, 
turn and enter new row, obstacle detection 

Vehicle speed 

0.2—2mph 

0.1—0.2mph 

2—5 mph 

Interface 

Control box with 

buttons 

Control box and foot pedals 

Handheld tablet or smartphone 

Permanent infra¬ 
structure installed in 

orchard 

None 

None 

None 
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Fig. 56.11 A family of autonomous orchard vehicles ca¬ 
pable of driving among tree rows, based on the Toro 
Workman series. To keep cost low, the vehicles use only 
a laser rangefinder and steering and wheel encoder for per¬ 
ception and navigation 

obstacles. Additionally, the added cost of the autonomy 
components should be as low as possible, to make such 
vehicles commercially viable. 

Figure 56.11 shows a family of autonomous orchard 
vehicles with a common sensing and computing infras¬ 
tructure that allows them to cover entire orchard blocks 
continuously for several hours (l<s>MMSEDH). To keep 
cost low, they are not equipped with GPS/GNSS or in¬ 
ertial navigation systems; instead, perception and navi¬ 
gation is achieved with a laser rangefinder mounted on 
the front of the vehicle and steering and wheel encoders. 
Table 56.2 summarizes the three autonomous control 
modes in which the vehicles can operate. 

From 2008 to 2012 the vehicles drove a com¬ 
bined 350 km in experimental and commercial or¬ 
chards, vineyards, and nurseries in several US states, 
including many in the largest apple producer in the 
United States, Washington state. The longest run cov¬ 
ered 25 km over five hours [56.10,33]. Time trials 
conducted by The Pennsylvania and Washington State 
Universities Extension educators showed that work¬ 
ers onboard an autonomous orchard platform can be 
twice as fast as workers using ladders or on foot when 
working on the top portion of apple and peach trees 
(Fig. 56.12) [56.34], 

56.3.7 Vehicle Formation Control 

Multiple robotic agricultural vehicles driving in forma¬ 
tion offer capabilities beyond those possible by a sin¬ 
gle vehicle. Lenain et al. [56.35] developed a control 
architecture that enables accurate and stable control of 
several robots in a given and possibly variable forma¬ 
tion. The method is based on a path tracking framework, 
defining the relative robot’s positions as a lateral dis¬ 



Fig. 56.12 (a) Autonomous orchard vehicle, (b) Workers on foot 
harvest apples and deposit them on bins towed by the vehicle, elim¬ 
inating the need for and the cost of a tractor driver, (c) In this version 
the vehicle is equipped with a lift platform from where workers can 
prune, thin, tie trees, and place pheromone dispensers up to twice 
as fast as workers performing the same operations on ladders. Users 
on the platform control the vehicle from an interface (d) that com¬ 
municates with the onboard computer wirelessly 


tance with respect to a given path, and the longitudinal 
distance along this path. Perturbations due to motion 
on natural environments (e.g., poor grip conditions and 
uneven terrain topography) are compensated with a non¬ 
linear model-based controller that includes an observer 
for wheel sideslip. In the formation control point of view 
the lateral error is no longer regulated to zero but to a de¬ 
sired set-point, potentially variable and accounting for 
other vehicle deviations. In addition, the velocity of each 
vehicle is controlled in order to ensure a desired curvi¬ 
linear distance with respect to the others, and imposing 
a desired speed for one of them. Similarly to lateral dy¬ 
namic, the desired distance may be defined as varying, 
and error may be constituted of a mix between longitu¬ 
dinal errors of one vehicle with respect to the other. 

This approach was tested in the field with two elec¬ 
trical off-road robots (Fig. 56.13a). They are equipped 
with RTK-GPS providing an accuracy of ±2 cm, and 
are able to communicate their positions among each 
other using wireless communications. Weighing around 
500 kg each, these two-meter long vehicles are able to 
reach speeds of 4 m s —1 and climb up to 20 degrees ter¬ 
rain. In this example, the first vehicle has to follow the 
trajectory depicted in black, composed of a straight line 
along a 15 degree slope on grass at a speed of 2ms -1 , 
followed by a half turn coming back on flat ground. The 
second vehicle must follow the first one with a desired 
curvilinear distance of 10 m and a desired lateral 
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a) b) Y coordinates (m) 




Fig. 56.13 (a) Two agricul¬ 
tural vehicles in formation 
control. (b) Reference trajec¬ 
tory and actual path achieved 
by the two vehicles 



Fig. 56.14 Date tree tracking 
system adapted to a commer¬ 
cial sprayer. The system aims 
the sprayer nozzle toward 
the trees more efficiently and 
safely than a human operator 


deviation of 1 m. The positions of the first and second 
vehicles are shown in blue and red dots in Fig. 56.13b, 
respectively. In these conditions, the accuracy of rela¬ 
tive positioning is within ±15 cm despite the challeng¬ 
ing terrain conditions (low grip, uneven terrain, curved 


trajectory). This work demonstrated the capability of 
maintaining a formation of several vehicles, compat¬ 
ible with the conditions encountered in agriculture, 
allowing to consider the introduction of multi-vehicle 
platoons in the field for precision agriculture. 


1 " 


6 = arctan 
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X is measured using the 
proximity sensor that is 
mounted on the wheel 

•Auto turn toward next 
tree at X = y 


Driving 

direction 


Spraying 
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D\ = Distance 
between trees 
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D2 = Distance from trees 
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Fig. 56.15 Calculation of the desired spray angle 


56.3.8 Date Palm Tree Spraying 

Dates palm tree spraying is normally done manually by 
a team of three workers from a platform 18 m or more 
above the ground. In the past, many accidents have oc¬ 
curred due to instability of the platform when in a lifted 
position. Degania Sprayers Company (Israel) developed 
a sprayer with a tall air cannon and a pan-tilt unit at 
the end to control the air flow and spray direction. This 
system, however, requires a worker to manually aim 
the spray toward the tree. Shapiro et al. [56.36] devel¬ 
oped an automatic tree tracking system for the sprayer 
(Fig. 56.14). 

The tracking system is based on an ultrasonic 
range sensor to detect the trees and a proximity sen- 
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sor mounted on the sprayer’s wheel to measure distance 
traveled. A human driver is responsible for maintain¬ 
ing the sprayer a distance Di = 3.5m from the trees. 
Knowing the average distance between trees in the row 
( D\ = 9 m), it is possible to compute the desired spray¬ 
ing angle as 8 = tan - '((Di — x)/D 2 ), where x is the 
distance traveled since the last tree (Fig. 56.15). This 
value is fed fed into a proportional-integral-derivative 
(PID) controller that outputs a pulse-width modulation 
(PWM) signal to an electrically-controlled hydraulic 
valve that controls the spraying angle. The authors 
opted to use an on/off valve driven by a PWM signal in¬ 
stead of a proportional valve to reduce the system cost 
and therefore make it more attractive to farmers. The 
rotation angle of the sprayer is measured by a poten¬ 
tiometer and used as feedback for the PID controller. 
When the sprayer reaches the midpoint between two 
trees, i.e., x = D\/2, the sprayer is set to rotate to¬ 
ward the next tree in the row and start spraying it. 
The tracking algorithm is implemented on an Arduino 
microcontroller. 

The system was built and deployed in date farms, 
showing good tracking results. It replaces a human 
worker which previously had to perform a dull task 
controlling the sprayer to track the trees using a joy¬ 
stick. A preliminary economic analysis indicates that 
the additional cost of the sprayer tracking system can 
be recovered in one harvest season. 

56.3.9 Plant Probing 

In large botanic experimentation fields, treatments (wa¬ 
tering, nutrients, sunlight) that optimize certain desired 
aspects (growth, appearance) need to be determined. 
Towards this aim, experiments entailing many repetitive 
actions need to be conducted. For example, measure¬ 
ments and samples from leaves must be taken regularly 
and some pruning may need to be performed. For 
these tasks robots would be very handy, but difficul¬ 
ties arise from the complex structure and deformable 
nature of plants, which do not only change appear¬ 
ance through growing, but whose leaves also move on 
a daily and sometimes hourly basis. Even though recent 
advances in depth sensors, deformable object model¬ 
ing, and autonomous mobile manipulation have brought 
this goal in reach for robotic applications [56.37, 38], 
many problems still exist, in particular concerning ro¬ 
bust recognition and localization of plant parts (leaves, 
flowers, fruits, stems), and robot manipulation under 
weakly constrained conditions in natural environments. 

In this context, the European project GARNICS 
(gardening with a cognitive system) aims at 3-D sensing 
of plant growth and building perceptual representations 
for learning the links to actions of a robot gardener. 



Fig. 56.16 (a) Chlorophyll measurements with a robot-mounted 
spad meter, (b) Cutting of leaf disks with a robot-mounted, custom- 
designed cutting tool. Both a color camera and a ToF camera are 
mounted on the end-effector of the arm. (c) Close-up of prob¬ 
ing: Chlorophyll measurements of a tobacco plant. The lightweight 
time-of-flight camera is seen at the top 




Fig. 56.17 (a) Color image, (b) Infrared image, (c) ToF 
depth, (d) Segmentation, (e) Surface models, (f) Color- 
coded leaf confidences and extracted probing points 


Plant sensing and control is addressed by combining 
active vision with appropriate perceptual representa¬ 
tions, which are essential for cognitive interactions 

An application of robotized phenotyping related to 
this project is the accurate placing of a measurement 
tool on a leaf in order to either cut sample disks from 
the leaf, or to measure chlorophyll content. The robotic 
arm is equipped with a time-of-flight (TOF) camera 
and a measurement tool (Fig. 56.16) [56.39-41], In 
this approach, image segmentation and model fitting are 
employed to recognize and localize single leaves from 
depth information. 3-D data is combined with color or 
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Fig.56.18a-c A cucumber harvesting robot, (a) The robot in the 
greenhouse, (b) Unprocessed image of cucumber crop in the near 
infrared spectrum, (c) Detected cucumbers 



Fig.56.19a,b Cucumber leaf removal robot, (a) Robot op¬ 
erational in the crop, (b) End-effector (after [56.16]) 


infrared images and used to segment the data into sur¬ 
face patches, which are assumed to correspond to actual 
plant leaves [56.42]. 

In this approach, a next-best view strategy was pro¬ 
posed for finding a non-obstructed and frontal view of 
the leaf [56.41]. Initially, the robot arm is moved to 
a position from which a general view of the plant is 
obtained. The depth and infrared images acquired from 
this position are segmented into their composite sur¬ 
faces, leaf model contours are fitted to the extracted 


segments, and the validity of the fit and the graspabil- 
ity of the leaf are measured (Fig. 56.17). A target leaf 
is selected and the robot moves the camera to a closer, 
frontal view. The validity of the target and the graspa- 
bility are re-evaluated. If the leaf is considered to be 
suitable for sampling based on these criteria, the prob¬ 
ing tool is placed onto the leaf following a two-step 
path. If the target is not considered suitable for probing, 
another target leaf (from the general view) is selected 
and the procedure is repeated. 

The method is based on several assumptions: 

i) The boundaries of leaves are visible in the infrared- 
intensity image 

ii) The leaf surfaces can be modeled by a basic 
quadratic function 

iii) Leaves of a specific plant type can be described by 
a common 2-D contour 

iv) Leaves are large enough to allow analysis by a ToF 
camera, and 

v) The leaves are static during probing. 

These assumptions may be violated under certain 
conditions, and further research will have to be under¬ 
taken to solve the various problems originating mostly 
from the complex and deformable nature of plants. 

56.3.10 Cucumber Harvesting 

A prototype robotic harvester for cucumbers is pre¬ 
sented in Fig. 56.18. The machine consists of a mobile 
platform that runs on rails (|o>®iL!l£*l!t*). These rails 
are commonly used in greenhouses in The Netherlands 
for the purpose of internal transport, but they are also 
used as a hot water heating system of the greenhouse. 
Harvesting requires functional steps such as the detec¬ 
tion and localization of the fruit and assessment of its 
ripeness. In case of the cucumber harvester, the differ¬ 
ent reflection properties in the near infra-red spectrum 
were exploited to detect green cucumbers in the green 
environment [56.43-45]. Whether the cucumber was 
ready for harvest was identified based on an estima¬ 
tion of its weight. Since cucumbers consist almost 95% 
of water, the weight estimation was achieved by esti¬ 
mating the volume of the fruits. Stereovision principles 
were then used to locate the fruits to be harvested in 
the 3-D environment. For that purpose the camera was 
shifted 50 mm on a linear slide and two images of the 
same scene were taken and processed. A Mitsubishi 
RV-E2 manipulator was used to steer the gripper-cutter 
mechanism to the fruit and transport the harvested fruit 
back to a storage crate. Collision-free motion planning 
based on the A* algorithm was used to steer the ma¬ 
nipulator during the harvesting operation [56.44]. The 
cutter consisted of a suction cup on a parallel gripper 
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that grabbed the peduncle of the fruit. (The peduncle is 
a stem segment that connects the fruit to the main stem 
of the plant.) Then the action of a suction cup immobi¬ 
lized the fruit in the gripper. A special thermal cutting 
device was used to separate the fruit from the plant. The 
high temperature of the cutting device also prevented 
the potential transport of viruses from one plant to the 
other during the harvesting process. For a successful 
harvest this machine needed 65.2 s on average. The suc¬ 
cess rate was 74.4% [56.43]. 

56.3.11 Cucumber Leaf Removal 

Harvesting has received considerable attention in 
robotics research focused on protected cultivation. Yet, 
this is not the only time- and labor-consuming cul¬ 
tivation operation. In cucumber production, amongst 
others, removal of old non-productive leaves in the 
lower regions of the plant is a time-consuming task. 
Figure 56.19 shows the same platform used for har¬ 
vesting, but in this case hardware and software were 
used for removal of leaves from the plants [56.16] 
(|4Scan4iitLtl). In this system, the camera system is 
used to identify and locate the main stem of the plants. 
The gripper is sent to the plant and moved upwards. 
Leaves encountered during this upward motion are sep¬ 
arated from the plant using the similar thermal cutting 
device as used for harvesting. An interesting feature of 
this machine is that with slight modifications of soft¬ 
ware and hardware, two greenhouse operations could 
be performed. 

56.3.12 Rose Harvesting 

In recent years a harvesting robot for roses has been 
developed and tested under practical circumstances in 
The Netherlands. The prototype is shown in Fig. 56.20. 
In this case, the rose plants are grown on moveable 
benches. Thus plants move to the robot instead of the 
robot moving to the plants in the greenhouse. During 
the harvest cycle, a camera system travels over the rose 


plants and locates the roses to be harvested. Then the 
harvesting operation is performed with two manipu¬ 
lators. One manipulator grips the rose just below the 
flower and pulls it gently aside to generate space for the 
second manipulator to travel down the stem towards the 
point where it will cut the stem. This manipulator carries 
a small-sized stereovision system that is used for real¬ 
time tracking of the stem during this downwards motion. 
Upon arrival, the manipulator deploys a small scissor- 
type cutter that will cut the stem. Then, finally, the rose 
is pulled out and put in a storage by the first manipula¬ 
tor whilst the second manipulator moves out of the crop 
and proceeds with the next harvest cycle. Details of this 
system have been covered in [56.13], 

56.3.13 Strawberry Harvesting 

In Japan, the market of strawberries is large, as large 
as the market of tomatoes, cucumbers, and mandarin 
oranges. The potentially high-economic return of this 
product together with the high labor intensity of pro¬ 
cesses like harvesting, explains the long tradition of 
research on robotic strawberry harvesting. As shown in 
Fig. 56.21, the robot consists of a 4 degrees-of-freedom 
(DOF) cylindrical manipulator. The robot carries 3 
charge-coupled device (CCD) cameras. A square LED- 
array is used for illumination of the scene. Two cameras 
provide stereo vision for detection and localization of 
the fruits. Once a fruit is detected, the end-effector 
is positioned in front of the fruit. The third camera, 
mounted on the end-effector, is then used to detect 
the peduncle, i. e., the fruit stem, and calculate its in¬ 
clination. Based on this data, the orientation of the 
end-effector is modified with a tilt mechanism and it 
then approaches the fruit. A successful approach of the 
strawberry is detected by a reflection-type light sensor 
in the end-effector. Upon successful completion of this 
motion, the peduncle is grasped and the stem is cut with 
a scissor-type cutting mechanism. A suitable manipula¬ 
tor motion then sends the harvested strawberry to a tray. 
This procedure is repeated for all detected fruits at the 



Fig.56.20a,b A rose harvesting robot, 
(a) The robot with the rose crop on 
a mobile growing system, (b) The 
end-effector with stereo vision system 
and a scissors-type cutting device 
(courtesy Jentjens Machinetechniek 
BV, Veghel, The Netherlands) 
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Fig. 56.21 A strawberry harvesting robot (courtesy IAM- 
Brain, Japan) 


current position of the robot. After all picking attempts 
are completed the full robot platform is moved 210 mm 
with a gantry-type transportation system running be¬ 
low the strawberry benches. The current prototype has 
achieved a picking speed of 6.3 s with a success rate of 
52.6%. For more details refer to [56.46,47]. 

56.3.14 Pot Handling in Nurseries 
and Greenhouses 

Nursery and greenhouse (N&G) farms in the United 
States produce over two billion potted plants annually 
(Fig. 56.22a). During the course of production plants 
are moved several times - distributed onto indoor or 


outdoor growing beds, repositioned to recover space as 
orders are filled, and collected for bulk transport. Un¬ 
til recently, only scarce manual labor was suitable for 
these jobs (Fig. 56.22b). 

Flarvest Automation’s recently-marketed HV- 
100 (Haney) robot automates critical N&G tasks 
(Fig. 56.23a). Plants are lifted and transported using 
a one degree-of-freedom manipulator coupled with 
a one degree-of-freedom gripper. The mobility system 
employs two differentially-controlled drive wheels 
balanced by a front roller. 

Flarvey uses a laser rangefinder to identify the con¬ 
tainers in which plants are grown. This sensor has 
a horizontal field-of-view greater than 180 degrees and 
can detect poorly reflecting plant containers from at 
least 4m- even under bright sunlight. The robot also 
uses the laser rangefinder to detect obstacles and robot 
teammates. 

Four boundary sensors, two forward-pointing, two 
rear-pointing, are used to find and follow retroreflective 
tape that marks the edge of the bed. The tape marker 
performs double duty as both the robot’s global refer¬ 
ence and as part of the user interface. By positioning the 
boundary marker workers indicate to the robot where 
plants should be placed. 

A user interface consisting of a dial and buttons is 
located on the back of the electronics box. The inter¬ 
face enables users to input the desired plant spacing, 
bed width, spacing pattern (hexagonal or rectangular) 
and the number of aisles the robot should instantiate. 

Robust operation of the robot is enabled by a be¬ 
havior-based programming scheme. Figure 56.23b il¬ 
lustrates spacing, a task that gives each plant sufficient 
room to grow without interfering with its neighbors. 
The closely packed plants in the foreground have 
been delivered to the growing bed and placed on the 
ground. Robot A identifies plant containers using the 
rangefinder; it selects for pick up the container farthest 
down field. After closing on and capturing a container 
with its gripper the robot will turn toward the boundary 
B. Nearing the tape marker the two forward boundary 
sensors on the robot will detect the tape and compute 
the relative angle between robot and tape. This enables 



Fig.56.22a,b Potted plants destined 
for garden stores and landscapers 
are grown in containers on immense 
nursery farms. Large amounts of stoop 
labor is needed to place, maintain, and 
retrieve these plants 
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Fig. 56.23 (a) HV-100 ( Harvey ) nursery and greenhouse automation robot with main components labeled: (A) laser 
rangefinder, (B) boundary sensor (two of four), (C) gripper, (D) emergency stop flag (pulling the flag stops the robot), 
(E) electronics box, with user interface components mounted on the back, and (F) passive roller to balance the robot, 
(b) Two robots operating as a team perform the spacing task. Robot A approaches the source plants in the foreground. 
Boundary B marks the edge of the bed. Robot C carries a plant toward the destination 


the robot to turn and align with the tape, keeping the 
tape on its left side. 

After acquiring the boundary marker the robot will 
servo along the tape as robot C is doing. When the plant 
destination comes within view the laser rangefinder is 
used to identify the next empty space in the pattern 
of spaced plants. The robot then computes an efficient 
path to the put-down point, moves to that position, and 
drops off the plant. Afterward, it turns back toward the 
source of plants and repeats the process. The robot uses 
dead reckoning to find its way to the vicinity of the 
source plants. Following this strategy robots can work 
singly or in teams of various sizes. 

56.3.15 Precision Forestry 

About 30% of the land mass of the earth is covered by 
forests. In addition to providing the raw materials for 
furniture, paper, clothing, and heating, forests provide 
habitats to diverse animal species and form the source 
of livelihood for many different human settlements. In 
2012, in Germany alone, an estimated 1.3 million jobs 
in the wood processing industry generated a revenue 
of more than 180 billion euros. To explore forests in 
an environmentally-friendly but still economical way is 
thus a major issue. Robotics is currently being used to 
preserve the forest and secure the jobs in forestry and 
related industries. 

Today, work in forests is already highly mecha¬ 
nized, and in the last decade, mobile robotics know-how 
combined with new virtual reality and remote sens¬ 
ing techniques paved the way for a new robotics view 
onto work machines in the forest. Wood harvesters 
(Fig. 56.24) and forwarders, advanced work machines 
for log cutting and transport, are currently a major aim 
of automation efforts [56.45,48,49]. Based on the in- 



Fig. 56.24 (a) A simulated wood harvester equipped with 
simulated laser scanners in a virtual testbed used for algo¬ 
rithm development and evaluation, (b) The physical har¬ 
vester in the woods, equipped with laser scanners mounted 
to the right and left door of the cabin 

sight that accurate machine localization cannot solely 
be based on GPS, mobile robotics capabilities for local¬ 
ization and navigation have been introduced. Measured 
GPS errors of up to 50 m due to signal absorption in 
the canopy and to multi-path effects make GPS practi¬ 
cally useless for precise localization and navigation. In 
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Fig. 56.25 (a) Top-down view of the 
machine, particles ( gray dots), trees 
from global tree map ( light blue dots) 
and trees from local tree map (dark 
blue dots), (b) Particles accumulate 
near the left edge of the cabin of 
the forest machine and indicate 
the newly calculated position in 
comparison to the GPS-based position 
(white rectangle below the harvester), 
(c) A semantic world model turned 
into a virtual forest. 


the VisualGPS approach, the GPS position only serves 
as a starting point for a combined Kalman filter and 
Monte Carlo localization algorithm based on optical 
range measurements by laser scanners 
The approach yields a machine position with an ac¬ 
curacy of 0.5 m and thus provides a sound basis for 
the development of navigation and (semi-) autonomous 
logging procedures [56.50, 51]. 

Practical experiments show that map building based 
on simultaneous localization and mapping (SLAM) 
techniques [56.52] are not very applicable in these en¬ 
vironments, because the resulting errors in the map are 
not bounded and thus are not well-suited for large area 
operation and for a matching against parcel borders. 
Instead, the highly accurate position estimation from V?- 
sualGPS builds on a previously generated map of single 
trees. A multi-sensor fusion approach helps to build this 
map based on airborne and satellite imagery in multiple 
spectral ranges as well as on airborne laser scanning. 
The multispectral imagery data provides the basis for 
the tree species determination which is solved as an ad¬ 
vanced pattern classification problem [56.53]. The next 
step, single tree delineation, is based on airborne im¬ 
agery in combination with 3-D surface data from the 
laser scans. A modified watershed algorithm [56.54] de¬ 
lineates the tree crowns. From the tree crowns’ sizes 
and species information, the geo-referenced stem posi¬ 
tion and even the trees’ diameters can be deduced. This 
not only allows for generating a global, geo-referenced 
tree map, but is also sufficient for generating a semantic 
world model, the virtual forest (Fig. 56.25). 

Each work machine in the forest also builds lo¬ 
cal maps of visible trees using machine-mounted laser 
scanners and a compass. For localization purposes, 
these local maps are matched against the previously 
generated global map of trees by means of a particle fil¬ 
ter. For a moving machine, the prediction step is carried 
out by a Kalman filter [56.50, 5 1], 

Figure 56.25 shows the two important results of 
the global map building process. The figures on the 


left show just the tree map with the white noise in the 
upper image denoting the particles representing poten¬ 
tial poses for the Monte Carlo localization process. In 
the lower image, the particles have converged against 
the machine’s correct position. The right image gives 
an impression of the virtual forest generated from the 
same data that was used for the tree map. This vir¬ 
tual forest is used to visualize the deduced information 
(e.g., tree species, tree height, crown shape and size, 
etc.) in a high-end virtual reality representation. This 
approach follows a general trend in robotic applications 
for natural or space environments: the development 
of virtual testbeds for world model information vi¬ 
sualization in an intuitively comprehensible manner. 
Advanced virtual testbeds then provide the simulation 
capabilities to develop and test the robotic application 
against the virtual world in order to save time and 
costs [56.55]. 

The developed laser rangefinder-based VisualGPS 
algorithms are being enhanced by optical stereo image 
recognition capabilities and are being ported to mobile 
platforms like the Seekur Jr. This is due to the fact that 
the work that started as an automation effort for work 
machines in the forest has recently turned into an envi¬ 
ronment monitoring project which aids the forester to 
inspect, protect, and attend to the forest with increased 
efficiency and effectiveness. 

56.3.16 Semi-Automation 
of Forwarder Crane 

In cut-to-length harvesting, which is the predominant 
tree harvesting method in Europe, delimbed and bucked 
timber is collected from the forest to the roadside by 
a forwarder. For an operator of such a machine, a large 
part of the work cycle consists of maneuvering the 
onboard hydraulic crane (Fig. 56.26). The operator con¬ 
trols the links individually using two joysticks. The 
redundant kinematic design is necessary for dexterity 
and the large active workspace of the machine, but it 
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Fig. 56.26 (a) Valmet 830 forwarder with redundant kinematic design of the hydraulic crane, (b) The operator uses two 
joysticks to control each link individually 


also makes the crane control difficult to learn and to 
perform in an efficient way. 

Considering the technological progress and capabil¬ 
ities of today’s forestry machines, the current manual 
control approach creates bottlenecks in the forwarding 
process. Semi-automation of these operations would 
thus be beneficial for productivity. Automation of some 
repetitive motions is also desired in order to support the 
operator and to reduce the physical and cognitive work¬ 
load. 

To this end, a small-scale hydraulic forwarder crane 
has been installed in a laboratory environment [56.56]. 
The crane is equipped with position and pressure sen¬ 
sors, as well as with electronics and software for rapid 
prototyping of automated control strategies. The same 
equipment has been installed on a commercial Valmet 
830 forwarder by Komatsu Forest for field testing. 

Using these platforms, new feedback control meth¬ 
ods and trajectory planning procedures have been de¬ 
veloped to construct and implement time-efficient mo¬ 
tions [56.57]. For a given geometric path, the speed and 
relative use of the different crane links along this path 
can be shaped to achieve motions with optimal perfor¬ 
mance within the limitations of the machine. Velocity 
constraints for the individual joints are particularly re¬ 
strictive in hydraulic manipulators. 

In [56.58], trajectories planned with this approach 
are compared with recorded motions by professional 
human operators with respect to execution time. Re¬ 
sults show that the performance can be improved 
significantly by path-constrained replanning of human- 
operated motions (Fig. 56.27). Additional replanning 
of the geometrical path, along with efficient velocity 
profiles along the path, can further improve the time ef¬ 
ficiency of the crane motions. 

Tracking the trajectories with feedback control re¬ 
quires sensors for measuring the joint positions. Instal¬ 
lation of such sensors may not be desired due to high 


costs or durability problems in the rough outdoor en¬ 
vironment. Using the approach in [56.59], motions can 
be found that are robust to uncertainties in initial con¬ 
ditions and therefore possible to perform in open loop. 
The end effector position along the trajectory can also 
be estimated using an observer and signals available 
from pressure sensors. Such design can be a more con¬ 
venient solution in the harsh conditions of the forest. 

The introduction of new tasks or new ways to 
perform tasks means new challenges for human-ma¬ 
chine interaction (HMI). Semi-autonomous operation 
requires the automated parts to be well integrated with 
the manual work. In order to facilitate cooperation in 
manipulator control between the human and computer, 
the work distribution and work transitions between 
them is important. Hansson and Servin [56.60] de¬ 
scribes an implementation of shared and traded control 
for this setup. The results from user tests show that 
semi-autonomous operation reduces the workload and 
shows significant potential to improve the productivity 
of inexperienced operators. 

A reliable framework of sensors and low-level con¬ 
trol, along with efficient motion planning strategies, 
allows for development of more advanced interaction 
technologies. One such possible future scenario is tele¬ 
operation, that presents several advantages to both ma¬ 
chine owners and operators. Firstly, it opens up for 
redesign of the machine with removal of the cabin 
which saves weight and cost. Secondly, the working en¬ 
vironment is improved with reduction of the noise and 
vibration levels. A virtual environment-based system 
for teleoperation of a forwarder crane was demonstrated 
in [56.61]; follow-up work is investigating how a virtual 
environment can be a useful tool for presenting feed¬ 
back to the operator in different scenarios. This method 
does however require environment reconstruction to lo¬ 
cate logs and potential obstacles, both of which are 
considered in [56.62]. 


Part F | 56.3 









Part F | 56.3 


1482 Part F 


Robots at Work 



Fig.56.27a-c Path-constrained replanning of human-operated motions, (a) Example of two crane motions performed by 
a human operator: from the load bunk to the logs (A-B) and back (B-C). (b) Velocity profiles for the distance 9 along the 
path A-B. A different distribution of the work between the joints allows for a higher velocity along the same path. The 
gray area indicates velocity constraints not to be violated, (c) The resulting evolution of joint coordinates with respect to 
time. The motion with a time-efficient velocity profile is clearly faster than the one of the driver 


56.3.17 Livestock Breeding 
and Nurturing 

In some cases animal breeding is a proactive process, 
such as in hatcheries where optimum conditions must 
be managed. In others nature is allowed to take its 
course. Sometimes intervention may be needed in the 


birthing process, but in the Australian outback cattle 
must take their chances. Nevertheless the newborn must 
be marked with transponder tags under the national live¬ 
stock identification scheme, NLIS [56.63]. 

Other species such as kangaroos, feral pigs, wild 
camels and horses can breed and run wild until har¬ 
vested or culled. When creatures are confined, tasks 



Fig. 56.28 Lely luno robot 


Fig. 56.29 Lely Vector feeder 
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include environmental monitoring and control, feeding, 
cleaning and growth monitoring. 

A Dutch company, Lely [56.64], is active in market¬ 
ing products that have a strong robotic element. When 
cattle are located in stalls, the Juno [56.65] patrols 
the laneway at the side of the pen, pushing fodder to¬ 
wards the rails to put it in reach of the feeding cattle 
(Fig. 56.28). 

The Vector [56.66] goes even further. A mobile 
robot carrying a hopper, it navigates automatically be¬ 
tween the bam that holds the feed and the stalls where 
the cattle are waiting (Fig. 56.29). 

Other equipment in Australian feedlots estimates 
the weight gain of the cattle, such as a system that reads 
the NLIS tag of a beast at a water trough and records 
the weight imposed by the front legs. 

In the Australian outback, similar sensing systems 
perform a variety of functions. Water is scarce, so 
watering points can be fenced, giving access only 
via a laneway that can be monitored. Some earlier 
projects [56.67] have involved species identification to 
control access by means of an automatically operated 
gate, but similar technology has valuable use in cattle 
production. 

Again the NLIS tag is read, while a walk over 
weighing system notes the weight gain of the animal. 
Vision can be used to identify cows that are followed by 
a calf. Now automatic gates can direct these to a sepa¬ 
rate paddock for tagging the calves. 

Many of the opportunities are for automation, which 
may be a precursor for robotics. Several projects have 
involved machine-vision monitoring of cattle [56.68] 
or of pigs in piggeries [56.69]. The weight-gain is 
estimated by visual means using precision cameras 
(Fig. 56.30). Other analyses concern behavior [56.70]. 

Mobile robots can be involved in sheds where 
chickens are bred for meat, rather than laying. A mobile 
robot has been proposed for moving among the poul¬ 
try to monitor air quality including temperature, relative 
humidity and the concentrations of ammonia and dust. 



Fig. 56.30 Measuring the live weight of cattle with vision 


Australian honeybees must also be protected from 
pests, carried by bees on arriving ships. Bait boxes are 
deployed at ports around Australia to attract any such 
swarms. A remote monitoring system with camera sens¬ 
ing gives an early alert to the apiary officer [56.71]. 

56.3.18 Livestock Exploitation 

Traditional farming practices milk cows twice a day; 
in morning and in the evening. Automatic milking par¬ 
lors with robotic milking stations are being widely 
adopted to improve diary productivity and convenience 
(Fig. 56.31). These allow the cows to self-determine 
when to come in for milking and feeding. In an auto¬ 
mated milking station, teats are located and attachment 



Fig. 56.31 Robot milking 



Fig. 56.32 Engineers demonstrate how their pioneering 
sheep shearing robot works at the University of Western 
Australia in 1980 (courtesy National Archives of Australia: 
A6180, 23/9/80/1) 
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of milking systems is automatic. Yield is monitored 
and udders are automatically inspected for injury and 
disease. Human intervention is minimal. The Lely com¬ 
pany is active in this area [56.72]. 

Once a year sheep must be shorn of their wool. 
In the past, the University of Western Australia devel¬ 
oped an automatic shearing system known as the Shear 
Magic [56.73] (Fig. 56.32). The commercial exploita¬ 
tion was not a success and there seems to have been 
little activity in this area in recent years. 



Fig. 56.33 Prototype computer vision system for scanning 
the egg collection belt for potential blockages 



Fig. 56.34 Precision Pastoral’s mustering method 


Eggs must be collected from battery hens. As they 
are laid, they roll from the cages onto a conveyer belt. 
Among other robotic applications in egg production, 
machine vision is used to detect foreign bodies or dam¬ 
aged eggs [56.74] (Fig. 56.33). 

56.3.19 Livestock Harvesting, 

Slaughtering, 
and Processing 

The system already described for monitoring cat¬ 
tle that approach a waterhole is also the subject of 
a funded project for mustering. Cattle, selected by 
means of their NLIS tags, are diverted into a com¬ 
pound from which they can be collected for trans¬ 
port to a feedlot, in preparation for slaughter [56.75] 
(Fig. 56.34). 

The vision-based system that can control access to 
waterholes can also be used for the collection of feral 
animals such as pigs. There are said to be more wild 
camels in Australia than in Arabia and these too can be 
captured by such a system. 

At the start of the 1990s, Fututech was hailed as 
the future of slaughterhouses [56.76]. Robotic sys¬ 
tems to be installed at the Kilcoy pastoral company in 
Queensland would automate the whole process from 
the knocking box to the chillroom. Construction started 
in 1992, but by June 1994 the project was abandoned, 
at a cost of over $40 million [56.77]. It was a project 
that was ahead of its time, relying on a central com¬ 
puter and kilometers of cabling rather than distributed 
intelligence. 

But much is different now. Robots have become 
commonplace in abattoirs, supported by sophisticated 
sensing systems to locate skeletal features [56.78] 
(Fig. 56.35). Pigs probably outnumber cattle and the 
processed animals include sheep and goats. Deboning 
systems are marketed for products including chicken 
legs and ham bones [56.79, 80]. 

For slaughtering quantity, the lead must be held by 
poultry, where a single plant can process up to 4000 
birds per hour [56.81]. 



Fig. 56.35 Lamb processing 
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56.3.20 Aerial-Based Precision Agriculture 

Unmanned aerial vehicles (UAVs) have recently begun 
being applied to precision agriculture. This is a new 
development and results are still preliminary, albeit 
very promising (l<s>Milil£ii!M). In fact, the Association 
for Unmanned Vehicle Systems International forecasts 
that 80% of all UAVs sold in the United States in 
the 2015-2025 period will go to serve the agricultural 
market [56.82]. Even if the actual figure turns out not 
to be that high, there is no question that roboticists 
and agricultural engineers are investing significant time 
and resources to understanding how UAVs can improve 
agricultural efficiency and reduce costs. 

Dong et al. [56.83] use high-resolution images and 
on-board sensor data from an aerial vehicle (Fig. 56.36) 
to create a sequence of dense 3-D reconstructions of 
the crop over time. This four-dimensional (4-D) spatio- 
temporal reconstruction is used to segment the canopies 
and estimate how the crown radius and height of each 
plant evolve. The processing pipeline was tested on data 
collected in a field planted with corn, broccoli, and cab¬ 
bage in Tifton, GA, USA (Fig. 56.37). 



Fig. 56.36 AscTec Pelican quadrotor used for data collec¬ 
tion over early season sweet com 


In many potato growing areas, the potato crop 
does not senesce naturally and therefore tuber matu¬ 
ration is artificially induced by killing the haulm 10 
to 25 days prior to harvest. Reglone is a widely-used 
potato haulm killing herbicide. Building on the knowl¬ 
edge of the relationship between potato crop biomass 
status, expressed in weighted difference vegetation 
index (WDVI) [56.84], and the minimum effective 


a) 3-D reconstructions 

Week 1 Week 2 Week 3 



b) Canopy segmentation 




Canopy 


d) Statistics 



Fig.56.37a-d Dense 3-D reconstruction of the corn stalks, 
used to estimate how the crown radius and the height of 
each plant evolve over time, (a) 3-D reconstruction of crop, 
(b) canopy segmentation, (c) crop analysis and (d) crop 
statistics 
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Fig. 56.38 Operator launching an unmanned aircraft over a potato field via a launch rope. The plane then flies au¬ 
tonomously following a pre-programmed flight path 
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Fig. 56.39 (a) Weighted difference 
vegetation index (WDVI) map of 
a potato field; the legend on the 
bottom represents the WDVI, which 
varies from zero (no crop at all) to one 
(theoretical maximum). A vigorous 
potato crop has a value in the 
0.6—0.7 range, (b) Reglone dose task 
map; the legend represents dose in 
liters per hectare 


dose of Reglone, Kempenaar et al. [56.85] success¬ 
fully demonstrated the use of crop biomass imaging 
with a multi-spectral camera underneath an unmanned 
aircraft (Fig. 56.38) for variable rate application. The 



Fig. 56.40 Fixed-wing UAV launched over a black-grass- 
infested wheat field 


WDVI map was converted into a 33 m x 10 m dose grid 
map (Fig. 56.39) adjusted to the boom width of the 
field sprayer. On the field, the average use was 0.9 liters 
of Reglone per ha, with satisfactory efficacy. Standard 
practice would have been the use of 2 liters per ha. The 
use of unmanned aerial imaging thus prompted a sav¬ 
ings of more than 50%, without loss in potato haulm 
killing efficacy, harvestability of the potatoes, and final 
product quality. 

Wheat is the most widely grown arable crop in the 
UK, covering around 1.6 million hectares and produc¬ 
ing 11.9 million metric tons in 2013. As with other 
crops, wheat has its share of weed competitors, one of 
which is black-grass. This prolific weed is highly com¬ 
petitive and increasingly resistant to chemical control, 
making it one of the biggest challenges facing UK’s 
agricultural sector. 

Researchers from URSULA Agriculture have 
demonstrated a system that combines multispectral 
sensing with UAVs to capture imagery at 10 cm resolu- 
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tion (Fig. 56.40) [56.86]. Using the unique spectral and 
morphological properties of the black-grass, they de¬ 
veloped classification processes that are applied to the 
image to automatically delineate the black-grass from 
the host crop. 

For the farmer and agronomist, the output is sup¬ 
plied in the form of visual maps and spatially attributed 
data points that are transferable through farm planning 
software to precision guided machinery (Fig. 56.41). 
URSULA Agriculture’s imagery analysis is able to 
not only identify the location and in-field extent of 
black-grass infestation, but also the density and area 
of black-grass plants, all of which will have a bear¬ 
ing on control decisions. The information provides the 
farmer with vital data to help contain black-grass in¬ 
festation in the current growing season. Crucially, it 
also helps with long-term control by informing man¬ 
agement methods such as higher seed rates to increase 
competition in black-grass hotspots, alternative cultiva¬ 
tion techniques, or adjusting cropping strategies, all of 



Fig.56.41a-b Black-grass analysis stack: High-resolution mul- 
tispectral image (a), black-grass delineation (b), and density 
map (c) 


which can help control black-grass alongside chemical 
herbicides. 


56.4 Conclusion 

The robotics community has made great strides toward 
introducing robotic systems in agriculture and forestry. 
The next ten years will bring improved sensing and 
mobility, and some advances in manipulation for crop 
production. 

In orchard crops, there is still a need for improve¬ 
ment in algorithms that calculate crop yield and canopy 
volume or that automatically detect insects and disease. 
Here the breakthroughs may end up coming from physi¬ 
cists and engineers producing novel imaging systems 
rather than from roboticists. The continued conversion 
of existing orchards to the tree wall architecture means 
that autonomous orchard vehicles and platforms will 
soon have access to the vast majority of the fruit pro¬ 
duction acreage, at least in developed countries. 

Manipulation in orchard environments is a grow¬ 
ing and challenging field. The long-sought harvesting 
robot is still an object of fiction, especially when it 
comes to its economic feasibility. In the apple indus¬ 
try, for example, the best human workers are capable 
of picking 40 to 60 pieces of fruit per minute, keeping 
bruising down to a few percent of the volume picked. 
Our community is relatively far from achieving this 
level of performance, but we do not need to get there 
in one shot. Many value-added operations, including 
pruning, thinning, and harvesting, could benefit from 
an automated manipulation-based solution. Manipula¬ 
tion should be introduced into low-precision tasks first, 
where economic gains can be more readily realized, be¬ 


fore the technology is refined and directed at tasks such 
as apple picking. One example is automatic grape shoot 
and fruit thinning according to each individual vine’s 
crop load [56.87]. This is in contrast to today’s methods, 
where crop load management is conducted uniformly 
over the entire vineyard, leading to significant varia¬ 
tions in yield. Achieving vine-level manipulation in an 
economically sound way would help move the grape in¬ 
dustry in the direction of variable rate management, so 
that one day, each vine, leaf, and fruit will be treated 
individually according to its own needs of water, nutri¬ 
ents, light, etc. [56.88]. 

Despite more than thirty years of research, robotic 
systems for plant maintenance operations in protected 
cultivation have not become commercially available. 
Clearly, the complexity of the operation is an issue. 
This is due to the highly unstructured working environ¬ 
ment, the inherent natural variability of the crop, and the 
unfavorable environmental conditions, e.g., strong vari¬ 
ations in lighting. In this domain, robust sensing and 
perception is key to successful deployments. This in¬ 
cludes aspects such as detection of fruits, stems, and 
leaves, determination of their properties, e.g., ripeness, 
and, finally, their accurate localization in the 3-D work¬ 
ing environment. Object detection can be quite a chal¬ 
lenge as in some cases it boils down to finding a green 
object in a green background. When color differences 
are more pronounced detection can be easier, but still 
in many cases object occlusion poses another problem. 
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This does not only prevent proper detection of the fruits, 
but also adds difficulty in the proper assessment of their 
properties. 

In the cluttered environments of protected culti¬ 
vation, manipulation is also quite challenging. While 
performing a task, the robot should prevent damage to 
the crop as this will immediately reduce its value. Still, 
the robot should be allowed to hit objects such as leaves. 
Humans tend to have quite intensive contact with the 
crop during operations. To mimic this kind of behavior 
requires knowledge about whether objects should not be 
touched at all or whether they might be gently touched 
and pushed aside, employing a kind of compliant mo¬ 
tion. 

Generally speaking, robotics-based crop produc¬ 
tion can be seen as an instantiation of the hand-eye 
coordination problem, or the integration of effective 
sensing, perception, and intelligent manipulation; in 
other words, those aspects of a well-trained human 
that are hard to reproduce. Development of technol¬ 
ogy to mimic human behavior is one way to go, but 
it will always be limited by the environmental con¬ 
ditions found at the time the robot was introduced. 
A systemic approach where the crop production sys¬ 
tem is designed top-down with the robotic system as 
just one of its components is a much more promising 
route, yet one that is still in its infancy. For example, 
growing systems can be modified to make detection and 
the approach of fruits and flowers easier, thus allow¬ 
ing for more simple design and operation of automated 
systems. Likewise, cultural practices may be modified 
to include the robot as part of the task [56.89, 90]. 
Human-robot interaction might be an interesting and 
cost-effective intermediate step - allowing for human 


guidance and supervision only when the machine needs 
assistance. Such collaborative approaches facilitate data 
collection in real working environments, offering the 
opportunity for learning and improving algorithms, thus 
paving the way for fully autonomous operations in the 
future. 

Another important aspect of the successful intro¬ 
duction of robots in agriculture and forestry is that 
related to the socio-economic barriers to adoption. In 
addition to being able to execute a task correctly, the 
robot must do it in a cost-effective way. Such effi¬ 
ciency can only be demonstrated via numerous field 
trials, which are time-consuming and costly. Further¬ 
more, the robotic system must have a total ownership 
cost, including acquisition, maintenance, user training, 
and disposal, that is less than the financial gains lever¬ 
aged through its introduction in the production process. 
Finally, one fundamental aspect is safety. Not only must 
the hardware and software be designed and validated 
based on explicit safety requirements, there must be 
standards and regulations that dictate how and when 
robots and humans can interact. For an example of 
a study on the socio-economic barriers to adoption in 
the apple industry, see [56.91]. 

The mechanization enabled by the Industrial Rev¬ 
olution and the automation enabled by the information 
technology era have revolutionized agricultural produc¬ 
tion to the point where a single farmer can produce 
grains to feed one hundred people. In the next fifty 
years, we will witness a similar occurrence in fruits, 
vegetables, and other crops, thanks to the advances in 
robotic technologies that our community is continually 
developing for and applying to agriculture and forestry 
(|<s>M2Ell!E*). 
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Autonomous orchard tractors 

available from http://handbookofrobotics.org/view-chapter/56/videodetails/26 
Autonomous orchard vehicle for specialty crop production 
available from http://handbookofrobotics.org/view-chapter/56/videodetails/91 
Autonomous utility vehicle - R Gator 

available from http://handbookofrobotics.org/view-chapter/56/videodetails/93 
Automatic plant probing 

available from http://handbookofrobotics.org/view-chapter/56/videodetails/95 
Visual GPS - High accuracy localization for forestry machinery 
available from http://handbookofrobotics.org/view-chapter/56/videodetails/96 
Smart Seeder: An autonomous high accuracy seed planter for broad acre crops 
available from http://handbookofrobotics.org/view-chapter/56/videodetails/131 
A robot for harvesting sweet-pepper in greenhouses 

available from http://handbookofrobotics.org/view-chapter/56/videodetails/304 

Ladybird: An intelligent farm robot for the vegetable industry 

available from http://handbookofrobotics.org/view-chapter/56/videodetails/305 

An automated mobile platform for orchard scanning and for soil, yield, and flower mapping 

available from http://handbookofrobotics.org/view-chapter/56/videodetails/306 
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A mini unmanned aerial system for remote sensing in agriculture 

available from http://handbookofrobotics.org/view-chapter/56/videodetails/307 

An autonomous cucumber harvester 

available from http://handbookofrobotics.org/view-chapter/56/videodetails/308 
An autonomous robot for de-leafing cucumber plants 

available from http://handbookofrobotics.org/view-chapter/56/videodetails/309 
The Intelligent Autonomous Weeder 

available from http://handbookofrobotics.org/view-chapter/56/videodetaiis/310 
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57. Robotics in Construction 


Kamel S. Saidi, Thomas Bock, Christos Georgoulas 


This chapter introduces various construction au¬ 
tomation concepts that have been developed over 
the past few decades and presents examples of 
construction robots that are in current use (as of 
2006) and/or in various stages of research and de¬ 
velopment. Section 57.1 presents an overview of 
the construction industry, which includes descrip¬ 
tions of the industry, the types of construction, 
and the typical construction project. The industry 
overview also discusses the concept of automa¬ 
tion versus robotics in construction and breaks 
down the concept of robotics in construction into 
several levels of autonomy as well as other cate¬ 
gories. Section 57.2 discusses some of the offsite 
applications of robotics in construction (such as 
for prefabrication), while Sect. 57.3 discusses the 
use of robots that perform a single task at the 
construction site. Section 57.4 introduces the con¬ 
cept of an integrated robotized construction site 
in which multiple robots/machines collaborate to 
build an entire structure. Section 57.5 discusses un¬ 
solved technical problems in construction robotics, 
which include interoperability, connection sys¬ 
tems, tolerances, and power and communications. 
Finally, Sect. 57.6 discusses future directions in 
construction robotics and Sect. 57.7 gives some 
conclusions and suggests resources for further 
reading. 
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Construction is a ubiquitous human activity that re- manufacturing in that the production activity normally 
lates to the creation or realization of physical artifacts occurs in a field setting and is undertaken in the open 
or custom-made capital goods. It is distinguished from air, on natural terrain, and often with naturally oc- 
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curring materials. Typically, building and construction 
products are large in scale and unique in form. More¬ 
over, the environment or field setting is typically unique 
and requires a, rather ad hoc, factory to be synthesized 
on site. 

Over the centuries, various forms of machines and me¬ 
chanical engineering systems have been introduced into 
the construction engineering domain and into the build¬ 
ing and construction industry to increase production 
efficiency. In common with the fields of agriculture, 
mining, and forestry, the long-term trend has been for 
these fields to become increasingly mechanized [57.1]. 

In the last few decades, with the decrease in the rel¬ 
ative cost of machinery to labor and with the globaliza¬ 


tion of markets, the construction industry has become 
significantly more capital intensive and large-scale ma¬ 
chinery systems and pieces of construction plant - 
such as tunnel-boring machines and very large tower 
cranes - have become commonplace. This trend to 
mechanization is likely to continue with the progres¬ 
sive introduction of computer-controlled construction 
machinery and flexible manufacturing concepts into the 
industry. 

With the relatively recent development of the micro¬ 
processor and the availability of low-cost computer and 
sensing technology, construction robots have become 
a technical and economic possibility and this form of 
technology is now gradually being used in the industry. 


57.1 Overview 

The application of robots in construction traditionally 
falls under construction automation .As the term im¬ 
plies, the field of construction automation is focused 
on automating construction processes, and the use of 
robots is but one aspect of automation. Construction 
processes also fall within several categories best de¬ 
scribed through a brief introduction to the construction 
industry. 

57.1.1 Industry Description 

The construction industry typically accounts for 5% 
of a country’s gross domestic product (GDP) or gross 
value added (GVA) and employs a significant portion 
of the country’s workforce. Table 57.1 presents some of 
these statistics for the USA, the European Union (EU), 
Japan and China [57.2-8]. Worldwide, construction in¬ 
dustry spending was estimated at approximately 11 % of 
the world’s GDP at the end of the 20th century [57.9]. 

Construction is considered by many to be techno¬ 
logically behind other industries, such as manufactur- 


Table 57.1 Value of construction put in place and number 
of construction workers for the USA, EU, Japan and China 


Country 

Data period (year) 

USA 

2012 

EU 

2009 

Japan 

2011 

China 

2011 

Value of construction put 
in place as percentage of 
national GDP/GVA (%) 

5 

7 

10 

7 

Number of construction 
workers as percentage of the 
total national workforce (%) 

5 

11 

10 

5 

Total number of workers 

in the national workforce 
(million) 

112 

175 

63 

784 


ing. In manufacturing, a product is designed for mass 
production, whereas construction products (or projects) 
are usually one-off and unique [57.10]. Thus the ef¬ 
ficiencies achieved through mass production are not 
easily achieved in construction. 

Other often attributed reasons for the construction 
industry’s technological lag are the industry’s fragmen¬ 
tation and aversion to the risks associated with the 
introduction of new technologies [57.10, 11]. For ex¬ 
ample, in the US in 2010 firms with fewer than twenty 
employees employed approximately 40% of all con¬ 
struction workers, and 63% of all construction workers 
worked for specialty trade contractors who accounted 
for 64% of construction firms [57.10]. Specialty trade 
contractors are usually subcontractors on a project and 
are not responsible for the overall outcome of the 
project. 

In addition, unlike their manufacturing counter¬ 
parts, construction sites are for the most part unstruc¬ 
tured, cluttered, and congested, making them difficult 
environments for robots to operate in. Furthermore, 
human workers are also present in large numbers on 
a construction project, making safety a paramount con¬ 
cern. 

Types of Construction 

Construction projects are usually classified as resi¬ 
dential, commercial, industrial, or civil. Residential 
construction generally involves single-family homes 
or large apartment buildings; commercial focuses on 
building structures such as office and retail space, ware¬ 
houses, and so on; industrial is involved in building 
factories, power plants, and other similar structures; and 
civil construction focuses on public infrastructure such 
as highways, bridges, tunnels, and dams. 
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The Typical Construction Project 
A construction project typically goes through six major 
phases, as shown in Fig. 57.1. Some projects may go 
through different variations of the sequence shown, but 
most projects include planning, design, construction, 
and operation phases [57.4, 10]. Phase 1 begins when 
a need for a project arises and the requirements are iden¬ 
tified. Phase 2 involves developing alternative project 
plans that could meet the identified needs and evaluat¬ 
ing the technological and economic feasibility of each 
alternative. Phase 3 develops detailed engineering de¬ 
signs and specifications for the plan selected in phase 2. 
The construction of the facility from ground-breaking 
through to final inspection takes place in phase 4 of the 
project. The facility is occupied and commences oper¬ 
ation in phase 5 and continues operating until it is time 
to shut down and dismantle the facility once it becomes 
obsolete (phase 6). 

The actual physical work (building, operating, 
maintaining, and dismantling) on a construction project 
takes place in the last three phases. Although these are 
the only phases in which machines are used, construc¬ 
tion automation can also take place during the other 
phases of a project, as described in Sect. 57.1.2. 

During the various phases of a construction project, 
several stakeholders may be involved at any time. The 
major stakeholders include the following: the owner 
and operator, whose needs initiated the project; the ar¬ 
chitect and engineer, who have the task of translating 
the owner’s needs into an aesthetically pleasing and 
structurally sound design; and the general contractor, 
whose task is to translate the design into a physical 
structure. In addition several other stakeholders may 
be involved either independently from the above stake¬ 
holders or as part of their organizations; for example, 
the constructor may employ a project manager, a con¬ 
struction manager, a site superintendent, and others, and 
he/she will typically subcontract major activities such 
as excavation and concrete pouring. The subcontrac¬ 
tors and sub-stakeholders are often the ones who use 
a new technology and can either make or break its 
implementation. 

Although the builder is the most likely user of 
robots on a construction project, the actual work on 
site is often conducted by subcontractors who often are 
reluctant or financially unable to use advanced tech¬ 
nologies which have not been entirely adopted by in¬ 
dustry. A constructor’s goal is often to meet the owner’s 
requirements by the most efficient and least risky meth¬ 
ods possible. Hence, traditional construction methods 
that have stood the test of time are preferred. Nev¬ 
ertheless, it has often been anecdotally reported that 
the owner has the power to require the use of certain 
technologies on a construction project, since that shifts 



Fig. 57.1 Phases of a typical construction project 

part of the risk and cost associated with the use of 
the technology to the owner. This phenomenon may be 
partially responsible for the widespread penetration of 
laser scanning technology into the construction industry 
that is currently taking place [57.12]. 

Typical Construction Processes 
From a purely physical-world and practical point of 
view, construction may be viewed as being comprised 
of a finite number of elementary processes [57. 13- 
lb], which may be summarized by the following 
list [57.11]: 

• Attaching 

• Building 

• Coating 

• Concreting 

• Connecting 

• Covering 

• Cutting 

• Digging 

• Finishing 

• Inlaying 

• Inspecting 

• Jointing 

• Measuring 

• Placing 

• Planning 

• Positioning 

• Spraying 

• Spreading. 

Most of these processes can also be grouped into 
three predominant types of functional operators as fol¬ 
lows: 

• Materials handling (by bulk and unit load) 

• Materials shaping (cutting, breaking, compacting, 
and machining) 

• Structural joining. 

These functional operators are typically each ap¬ 
plied to multiple operands. Common operands in build- 
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ing and civil engineering are steel and other metals, 
concrete, timber, earth and rock, masonry, plastic and 
glass, cement, aggregate, epoxy resin, bitumen, and 
other bulk and formed materials. 

57.1.2 Automation in Construction 

Construction automation describes the field of research 
and development focused on automating construction 
processes, and the use of robots is but one aspect of 
that field. In short, construction automation deals with 
applying the principles of industrial automation to the 
construction sector, whether in building construction, 
civil engineering (roadways, dams, bridges, etc.), or 
in prefabrication of construction components [57.17], 
This can be viewed as an extension to research in 
field service robots generally designed to replace or as¬ 
sist humans in a specific construction-related task or 
function. 

From a historical perspective, research in construc¬ 
tion robotics and automation started in the 1980s with 
the introduction of single-purpose robots (principally 
remotely controlled, or teleoperated, machines). The 
Japanese led this effort, driven primarily out of con¬ 
cern for societal demographics, which showed a sig¬ 
nificant future shortfall in personnel available for the 
construction labor pool [57.18]. In the US the princi¬ 
pal related work involved developing remote control 
or teleoperated machinery for hazardous work that re¬ 
quired modified construction equipment. Example ap¬ 
plications include robots developed for rapid runway 
repair and unexploded ordinance removal. In the EU, 
research was focused on the development of large-size 
masonry (brick laying, assembly) robots for residential 
and industrial building construction. 

During the next decade, as research on task-specific 
construction robots continued, large Japanese construc¬ 
tion firms introduced on-site factories for high-rise con¬ 
struction. These construction systems included just-in- 
time delivery of components, automated part tracking 
and material handling, robotic connection and assem¬ 
bly, and centralized control in an enclosed or semi- 
enclosed environment. It is reported that the systems en¬ 
able better working conditions (weather invariant) and 
reduced project completion time [57.19]. Other benefits 
include improved productivity and quality, though over¬ 
all construction costs are not necessarily lower [57.20], 

Advanced concepts in integrated residential con¬ 
struction automation were developed as part of the EU 
FutureHome project. In this construction concept, each 
structure consists of several high-quality, prefabricated 
three-dimensional (3-D) modules and two-dimensional 
(2-D) panels which are assembled production-style on 
site [57.21]. An analogue of this approach for auto¬ 


mated residential construction has been commercial¬ 
ized in Japan [57.22], 

New methods for collecting, processing, analyz¬ 
ing, and communicating construction information are 
a significant area of construction automation re¬ 
search [57.23], This research includes data interoper¬ 
ability and exchange through the design, construction, 
operations, maintenance, and decommission phases of 
a capital project [57.24]; advanced sensors for assess¬ 
ing the status of the construction process [57.25-27]; 
visualization systems for planning construction events, 
verifying constructability, and maintaining site situ¬ 
ational awareness [57.28-30]; and information mod¬ 
els which extend traditional computer-aided design 
(CAD) modeling to combine both the physical (geo¬ 
metric) and functional characteristics of building com¬ 
ponents [57.31]. 

Finally, three notable, large-scale, EU projects at¬ 
tempted to integrate construction automation into the 
way construction projects are executed: the ManuBuild 
(open building manufacturing) project dealt with, 
among other things, the development of mobile field 
factories, including robots, for on-site modular con¬ 
struction [57.32]; the I3CON (industrialized, integrated, 
intelligent, construction) project dealt with the in¬ 
door automation and robotization of buildings [57.33]; 
and the Tunconstruct project dealt with the robotiza¬ 
tion of inspection and maintenance operations in tun¬ 
nels [57.34], 

57.1.3 Classification of Robotics 
in Construction 

Construction robotics is an advanced form of mecha¬ 
nization (automation) in which an endeavour is made 
to automate some industrially important operation and 
thereby reduce the cost of this operation by either re¬ 
moving a human operator from the control loop , or 
enhance operational efficiency through machine control 
systems. Due to the nature of construction work, most 
robots which have been developed for the construction 
industry are either mobile or relocatable systems. Some 
platforms, such as floor-finishing robots and machine- 
controlled earthmovers require mobility as a specific 
function of the work process to be performed. Oth¬ 
ers, such as wall and ceiling panel manipulators, re¬ 
quire some level of mobility to extend their operating 
workspace. 

Robot terminology can vary depending upon the 
research discipline. For this chapter two broad classi¬ 
fications of construction robotics are onsite robots and 
offsite robots. These are distinguished by whether they 
are applied at the construction site or at a factory or pre¬ 
fabrication facility. In addition, a further distinction can 
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be made depending on whether a construction robot is 
intended to be used for a single task (onsite or offsite) 
or whether multiple robots are integrated into an auto¬ 
mated construction site. 

Single Task Construction Robots Versus 
Integrated Robotized Construction Sites 
After the initial experiments in large-scale industrial¬ 
ized, automated and robotized pre-fabrication of system 
houses were conducted successfully in Japan, and the 
first products (e.g. Sekisui Ml) also proved successful in 
the market, in 1975 a Japanese construction contractor, 
Shimizu Corporation in Tokyo, set up a research group 
for construction robots. The goal was now no longer 
the mere shifting of complexity into a structured envi¬ 
ronment as in prefabrication, but the development and 
deployment of systems that could be used on the con¬ 
struction site to create structures and buildings. The fo¬ 
cus initially was set on simple systems in the form of 
so-called single task construction robots that can exe¬ 
cute a single, specific construction task in a repetitive 
manner. 

Single-task construction robots are systems that 
support workers in executing one specific construction 
process or task (e.g. digging, concrete levelling, con¬ 
crete finishing, painting) or completely supplement the 
physical activity of the human workers necessary to per¬ 
form this one process or task. Further, the processes and 
tasks they support or supplement can be allocated to 
a specific profession or craft. In addition, the processes 
and tasks for which single task robots were developed 
had in common that they entailed frequent, repetitive 
activities. Common characteristics of single-task con¬ 
struction robots are as follows: 

• Highly specialized not only for a profession but 
even for a task within a profession (e.g., concrete 
pouring, levelling and finishing. 

• Enhanced productivity compared to conventional 
labour. E.g., according to [57.17] the conventional 
labour productivity rate for concrete floor finishing 
is between 100m 3 /h and 120m 3 /h, whereas the 
productivity using machines is between 300m 3 /h 
and 800m 3 /h. 

• Improved quality through precise control of func¬ 
tions and operations (e.g., uniform distribution of 
paint) and by allowing real time monitoring (and 
recording) of the operation. 

• Improved working conditions by removing work¬ 
ers from dangerous environments and reducing the 
amount of and heavy physical work. 

• Most robots allow various operation modes such as 
autonomous or sensor guided, pre-programmed, or 
tele-operated. 


• Reduced material consumption through precise 
control of material delivery and collection and reuse 
of unused material. 

• Most robots have simple yet robust sensor technolo¬ 
gies, such as gyroscopes, simple laser systems, or 
touch/pressure sensors. 

• Most robots only require 1 or 2 persons to operate. 

The evolution of industrialized and automated 
building prefabrication during the 1970s along with 
the development of single tasks robots, envisioned the 
concept of integrated robotized construction sites. This 
concept combined prefabrication technology (process¬ 
ing of prefabricated components instead of parts in 
order to reduce complexity on the site), single task 
construction robotic systems, and moving or stationary 
site factories that were able to assemble the building’s 
main structure (e.g., steel frame or concrete structure) 
almost automatically. As those site factories not only 
automated parts of the construction process but also 
integrated prefabricated component technology and sin¬ 
gle task automation can thus be called as integrated 
robotized construction sites. 

A major reason for the transition into integrated 
robotized construction sites was that the construc¬ 
tion companies realized that single task construction 
robots which were not networked or embedded within 
a greater infrastructure, turned out to be incompatible 
with the way buildings were designed and built. Sin¬ 
gle task robots were designed to execute certain tasks 
meanwhile construction workers were not allowed to 
interfere significantly with the robots activity. However, 
it turned out that under these premises, only a very 
few number of robots could be used efficiently. The 
constraints for the workers, the necessary safety regula¬ 
tions, coupled with the unforeseen, unpredictable, and 
dynamic processes at the construction site, led to the 
implementation of individual robots working in paral¬ 
lel. Although single task construction robots achieved 
a high throughput, significant time had to be spent 
on-site for transportation, preparation, programming, 
configuration, etc. Besides the mentioned frictions sin¬ 
gle task construction robots caused on (conventional) 
construction sites, the emergence of concepts for inte¬ 
grated sites was also nurtured by the new technological 
possibilities. Important for the integration of such sys¬ 
tems into larger and coordinated automated systems 
was the development of systems that allowed control¬ 
ling and monitoring an uninterrupted flow of informa¬ 
tion and material on-site, between individual automated 
entities that are involved in the final assembly of the 
building. 

The transition to integrated sites can also be ex¬ 
plained from an evolutionary view. In many industries 
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the evolution from more workshop like production with 
individual and only loosely coupled production enti¬ 
ties or stations, to flow-line like or production line like 
systems with stable processes and continuous material 
flow was part of the evolution of other well estab¬ 
lished and known industries,, i. e., automotive, com¬ 
puter industry. 

Finally, three general categories of construction 
robots are introduced. The first class is teleoperated 
systems, which for simplicity includes remote control 
systems. The general distinction between the two terms 
is whether or not the equipment must be operated in line 
of sight from the human controller [57.35]. The second 
category, programmable construction machines (PCM), 
includes most construction equipment that is outfitted 
with sensors and mechanisms to augment operation by 
an onboard human operator. The final category, intelli¬ 
gent systems, relates to unmanned construction robots 
which operate either in a semi- or fully autonomous 
mode. In Sect. 57.3 this classification will be expanded 
to include examples based on various generic activities, 
materials handled (operand types), levels of onboard in¬ 
telligence, levels of commercialization, and levels of 
system integration and computer integration. 

Teleoperated Systems in Construction 
In established engineering terminology, the term tele¬ 
operation refers to the remote control of machines and 
systems. In teleoperation (loosely referred to as teler¬ 
obotics) the control of the machine is accomplished by 
the use of remote control means such an umbilical cord 
or wireless control. Teleoperation ideas and methods 
are used extensively in the space and nuclear industries 
(Chaps. 55 and 58, respectively). 

In telerobotics, the machine does not operate au¬ 
tonomously but is under the control of a human. Data 
sensing and interpretation and cognitive activities such 
as task planning are done by the operator. 

Recently, many telerobotic devices have appeared 
in the construction and mining industries. These ma¬ 
chines have evolved in response to industrial situations 
where there is danger to the operator and where remote- 
controlled machinery is necessary (e.g., teleoperated 
small compactors). Situations of this kind occur in the 
construction, demolition, and mining industries and in 
other hazardous locations. 

The technology for telerobotics in construction is 
well established, with a number of excellent examples 
of such activity available, for example, a sophisticated 
model-based supervisory-control-type distributed tele¬ 
operation system for the construction of a trench for 


a diversion dam in a lava field in Japan using a fleet 
of heavy earthmoving machines [57.36]. 

Programmable Construction Machines 
A software-programmable construction machine is 
what most people would consider to be a robot. The 
operator of this type of machine is able to vary the 
task to be accomplished within certain constraints either 
by choosing from a preprogrammed menu of functions 
or by teaching the machine a new function. Variations 
in the task to be accomplished could be as simple as 
slight changes in the driving speed based on the cur¬ 
rent load for an automated forklift, or as complex as 
a change from being able to pick and place steel beams 
and columns using an autonomous crane to being able 
to deliver concrete autonomously using the same crane. 

Generally, software-programmable construction 
machines are identical to traditional construction ma¬ 
chines (such as an excavator), but have been modified 
to be controllable through a computer (similar to the 
way in which traditional manufacturing machines - 
such as mills and lathes - evolved into computer 
numerically controlled (CNC) machines). 

Software-programmable construction machines can 
make use of an electronic representation of a portion 
of the construction site where their work is to be con¬ 
ducted in order to control all or part of the machine’s 
operation. A commercialized example is stakeless grad¬ 
ing, where data from a 3-D model is used in combina¬ 
tion with global positioning systems (GPS) and/or laser 
measurement systems to automate the blade control for 
bulldozers and motor-graders. 

Intelligent Systems in Construction 
As opposed to a teleoperated or a software-pro¬ 
grammable construction machine a fully autonomous 
construction robot is expected to accomplish its task, 
within a defined scope, without human intervention. 
A semiautonomous construction robot would be ex¬ 
pected to accomplish its task with some level of 
planning interaction conducted with a human super¬ 
visor [57.35]. In each case the construction robot is 
expected to adapt to its sensed environment, formu¬ 
late plans for the execution of its task, and replan as 
necessary (with possibly some human assistance in the 
semiautonomous mode). The intelligent construction 
robot should also be able to determine when its task¬ 
ing is not executable and request assistance. 

Example research in intelligent construction sys¬ 
tems includes autonomous excavation [57.37,38] and 
autonomous crane operations [57.30, 39-41]. 
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The infusion of technologies (such as computer aided 
design (CAD), computer aided manufacturing (CAM)) 
into the construction industry from other industries 
(such as the automotive, airplane and shipbuilding in¬ 
dustries) are the clearest proof for a shift in building 
strategy. Fully automated concrete fabrication is cur¬ 
rently a reality that are implemented by factories with 
customized product delivery in order to adapt to the 
changing requirements of the market. There are two ma¬ 
jor industrialized production methods for prefabricated 
concrete: Production methods that (a) use stationary 
single formwork and (b) that use mobile formwork. 

57.2.1 Robotics in Component Production 

Production methods using stationary single formwork 
comprise mobile workflows such as cleaning, pour¬ 
ing/casting and moving the final product to the storage, 
while the formwork is stationary during the whole pro¬ 
cess of prefabrication. In production methods using 
mobile formwork, the work posts are stationary while 
the formwork is mobile, moving into the various pre¬ 
fabrication posts of the assembly line. 

Concrete Production 

A plant for combined manufacture of 2-D and 3-D pre¬ 
cast concrete elements includes overhead gantry cranes 
and electric hoists, used to transport consumable mate¬ 
rials (such as lattice girders, steel coils, and parts to be 


inserted/installed) as well as to lift elements from the 
concreting station into the storage facility or onto trans¬ 
port pallets. Pallets start at the discharging station, from 
where they proceed via the pallet cleaning station to the 
first workstation. There, by means of a de-palletizing 
device attached to a panel-stacking crane, the finished 
flooring elements are lifted off the pallets and stacked 
directly within the working range of the discharging ve¬ 
hicle. A cleaning and plotting (magazining, cleaning, 
plotting (MCP)) robot is used for a variety of tasks: pick¬ 
ing up, insertion of latitudinal anchors, cleaning of pal¬ 
lets, full scale plotting of elements, and installation of 
latitudinal anchors (Fig. 57.2). Figure 57.3 shows fur¬ 
ther examples of robotic concrete component manufac¬ 
turing and handling. 

Brickwork Component Production 
Since the late 1970s, due to the lack of skilled workers 
in construction and increasing costs of buildings, ratio¬ 
nalization developments were started. They mainly took 
place in the masonry and the formwork sector, where no 
ready solutions from other countries were available. In 
Germany and also in most parts of Europe an often used 
and still loved building material is brickwork. Most of 
the building projects are constructed with this material. 
Due to the lack of brickwork in Japan (not resistant 
to earthquakes if not reinforced), and missing develop¬ 
ments in other countries, main efforts were made in this 
field. 



Fig. 57.2 (a) Pallet in concreting station; (b) bridge crane; (c) pallets molding systems; (d) demolding-depalletizing 
device; (e) cleaning, measuring and oiling (after [57.42]) 
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Fig. 57.3 (a) Automated 
concrete pipe manufacturing 
machines (courtesy Hawkeye 
Group USA); (b) precast 
concrete handling robot 
(courtesy Halo, Inc.) 


In the last few decades activities to develop ma¬ 
chines that support the laying of bricks and even 
automated robots have been increasing, especially in 
Germany. Brick laying is a very labor-intensive activity, 
which can lead to significant health issues and early re¬ 
tirement [57.43, 44]. Consequently a very small number 
of skilled workers is available nowadays. This in turn 
is reflected in ultimately high prices and wages, which 
will make brick building in some time prohibitively ex¬ 
pensive. Therefore it should just be a question of time 
before only houses are put on the market that have not 
been erected on site by a team of bricklayers, but by 
a robot in the production shop. 

Early Trials of Robotic Assembly 
of Modular Blocks 

One idea for Robot Oriented Design (ROD) [57.45- 
47], in building construction is to change the conven¬ 
tional construction works adaptable to robotics so that 
construction system and advanced automation and/or 
robotics are co-adapted. Thus a structural system for 
the wall erection named SMAS (Solid Material As¬ 
sembly System) [57.48] was proposed and developed 
already in the 80s in Japan. SMAS is a of reinforced 
masonry construction system. A standard building com¬ 
ponent of this system, 30 cm x 30 cm x 18 cm in size 
and 20 kg in weight, is made of pre-cast concrete and 
includes cross-shaped steel bar inside each component 
for the reinforced of structural wall. Components are 
positioned automatically by the robot one by one with¬ 
out arrangements of conventional bonding. Following 
the positioning of each components, steel bars are con¬ 
nected to those of adjacent components also by the 
robot. The joint type of steel bar for vertical direction 
is mechanical, and that for lateral direction is over¬ 
lapping. Concrete is grouted from the top of the wall 
which is erected one storey high (about 3 m). The newly 
developed operating hand is installed to the mother 
robot (6-articulation-type robot) which was developed 


for a wide variety of applications in factory use and a se¬ 
ries of experiments for wall erection were carried out. 

The rapid progress being achieved today in the 
modernization and industrialization of building con¬ 
struction technology has triggered a trend to reduce 
the complicate works at construction site and increas¬ 
ingly produce building components at factories. It is 
obviously that the prefabrication has been successful in 
up-grading the quality of the building and in shorten¬ 
ing the construction period. The sizes of those building 
components such as prefabricated structural members 
are also becoming larger to simplify assembling work 
at the construction site. 

However, these movements are not necessarily ori¬ 
ented toward the introduction of robots. The heavy and 
large components are difficult to be operated by the 
robots and the complicated assemble techniques are 
sometimes too skilful for robots. Meanwhile, when one 
looks at the prefabrication of compact and lightweight 
structural components as a means of accomplishing 
construction work more efficiently, robots can be used 
for assembling these structural members efficiently. 

From this point of view SMAS structural system was 
developed and proposed as a robot-oriented construc¬ 
tion system. A masonry structure has not been consid¬ 
ered as a major structural system in Japan because of 
earthquakes; however, it becomes to be recognized as 
a flexible structural system applied to various building 
designs when properly reinforced. In addition, it has ad¬ 
vantages in the construction cost and in the construction 
period. The SMAS system itself was designed as a mod¬ 
ular system for the purpose that single components could 
be developed further in the future. All system compo¬ 
nents including stones are developed as complementary 
parts according to the guidelines of Robot Oriented De¬ 
sign (ROD) (Fig. 57.4). These robotic approaches were 
meant to be applied on-site, but technologically they 
were proven to be not effective. Thus, they became better 
suited for offsite assembly operations. 
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Fig. 57.4 (a) Building 
component developed by 
guidelines of Robot Oriented 
Design (ROD), (b) SMAS 
total system, (c) robot end- 
effector/gripper with optical 
fiber sensor guiding screwing 
mechanism 


Automated Brickwork Plants 
The adoption of CAD/CAM systems combined with 
implementation of Enterprise Resource Planning (ERP) 
Solutions has made the implementation of advanced 
technically refined, fully automatic production plants 
possible: 

• ffigh degree of information and communications 
technology (ICT) integrated and automated/inter¬ 
connected processes 

• Integration of as many devices as possible by ICT 
(interoperability) 

• CAD/CAM Systems 

• Integration of processes by ERP Systems 

• Just in Time and Just in Sequence supporting cus¬ 
tomization. 

Brickwork Robot Plant SuBA 
Crucial technical developments preceded the construc¬ 
tion of a fully automatic brickwork machine. The pro¬ 
totype of the automatic brickwork plant was developed 
by the company SiiBA in Hockenheim, and the com¬ 
pany Windhoff AG in Rheine at the beginning of the 
1990s. The production of brickwork panes is appro¬ 
priate for a capacity of 300 m 2 net area of brickwork 
panes - without windows and door recesses - in a shift 
of eight hours. The employment of CAD in the archi¬ 


tect’s offices made it possible to transfer the large data 
set for the production of brickwork panes directly with¬ 
out manual input over CAM to the brickwork robots. 
Considering the large number of data for the automat, 
which was necessary for the production of a brickwork 
pane, this was one of the most important problems of 
the automation of bricking at that time. Figures 57.5 
and 57.6 give an overview over the most important 
SiiBA factory modules and automation processes. 

Brickwork Robot Plant Winkelmann 

(Horizontal Brickwork Panel Production) 
Today’s fully automated and highly robotized brick¬ 
work plants can be distinguished into two basic types: 
horizontal and vertical brickwork panel production. The 
brickwork robot plant Winkelmann is a characteristic 
example for horizontal panel production. All single de¬ 
vices are equipped with Microsystems, interconnected 
and part of a systemic logistic network. CAD/CAM 
guarantees efficient data processing between planning 
section and production. After the delivery standard 
palettes bricks are brought into processing order by an 
automated palletizing system with a robot for distribu¬ 
tion of bricks (Fig. 57.7a). After that the bricks are taken 
up by automated de-palletizing system (Fig. 57.7b), 
which supplies horizontal brickwork layering robot sta¬ 
tion (Fig. 57.8a). Insertion of reinforcement and house 



Fig. 57.5 (a) Automated 
reinforcement (horizontal) 
station; (b) mortar dispensing 
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Fig. 57.6 (a) Brickwork positioning; (b) brickwork alignment; (c) automated reinforcement (vertical) station 



Fig. 57.7 (a) Automated 
palletizing system with robot 
for distribution of bricks; 

(b) automated de-palletizing 
system 



Fig. 57.8 (a) Automated 
horizontal brickwork layering 
robot station; (b) insertion 
of reinforcement and house 
infrastructure 


infrastructure as well as plastering are done stationary 
in the factory (Fig. 57.8b) until a finalized and dried-out 
brickwork panel is delivered to the construction site. 

Brickwork Robot Plant Leonhard Weiss 

(Vertical Brickwork Panel Production) 

The brickwork robot plant Leonhard Weiss is a char¬ 
acteristic example for vertical brickwork panel produc¬ 
tion. Many processes as for example the palletizing and 
de-palletizing to bring the bricks into factory order are 


similar to the horizontal type. Yet the central process of 
positioning the bricks in given order is done vertically 
layer by layer. High accuracy robots combined with lin¬ 
ear axis are here in charge for the exact positioning. The 
automated vertical brickwork layering has some advan¬ 
tages in terms of efficient use of the factory area and 
moreover the firmness of the wall could be improved 
whereas the exact positioning of reinforcement, cables 
and other elements of the house infrastructure is done 
easier with the horizontal type. CAD/CAM combined 
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Fig. 57.9 (a) Sequencing station; (b) robotic assembly system for vertical brickwork layering; (c,d) final brickwork panel 
product 



Fig.57.10a,b 3-D adjustable auto¬ 
mated assembly welding unit (Sekisui 
Heim) 



Fig. 57.11 (a) On the 400 m 
production-line, the steel frame 
units ( chassis ) pass several worksta¬ 
tions; (b) 10 Insertion of prepared 
elements and by suppliers (e.g., Toto, 
Inax) prefabricated bath modules into 
the chassis unit 


with variable production and ICT integrated produc¬ 
tion allows individually fabricated brickwork panels 
and necessary variations. Figure 57.9 shows the various 
steps of the automated vertical brickwork panel con¬ 
struction process. 

Steel Component Production 
Sekisui Heim introduced an automated steel frame pro¬ 
duction. One of the basic features is the automated as¬ 
sembling and welding station. Ceiling elements, floor¬ 
ing elements and columns are fed into this station, 
followed by automatic welding into a frame, which is 


used as chassis and bearing structure during the further 
completion process on the production line (Fig. 57.10). 
After the automated welding process, the steel frame 
chassis is streaming through the factory from work step 
to work step, until all installations have been completed 
(Fig. 57.1 la). The factories of Sekisui and Toyota have 
gates on both sides of the assembly lines in order to 
receive material, parts, components and prefabricated 
bath or kitchen modules, required for the customized 
production of individual units. All of them arrive just- 
in-time and just-in-sequence by cooperating suppliers 
(Fig. 57.11b). 
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57.3 Onsite Applications of Single Task Construction Robots 


The construction of larger buildings that use steel as 
the structural support system can involve a high to¬ 
tal amount of welds. If the design of the columns and 
beams can be adjusted in order to reduce the amount 
and variety of welds, welding becomes a highly repeti¬ 
tive operation suitable for being automated. 

57.3.1 Steel Welding 

Automated welding is able to control and guarantee 
the quality of the connection between the welded parts 
to a level similar to (and sometimes better than) that 
achieved by professional human welders. Manual weld¬ 
ing of these and other types of connections requires 
highly specialized skills that are currently in short 
supply. Furthermore, conventional welding can have 
damaging long-term effects on a worker’s vision. Si¬ 
multaneous automated welding on a beam (e.g., from 
two or three coordinated positions) is even able to en¬ 
sure that the steel component is not distorted by the 
welding operation itself and thus improves the quality 
of the constructed structures. In Fig. 57. 12a the Shimizu 
steel-welding robot is depicted. The robot can automat¬ 
ically weld a column including the corner portions. The 
configuration of a joint is detected by a laser sensor, and 
the welding is performed in an optimized way referring 
to a database. In Fig. 57.12b, the Obayashi Corporation 
Steel welding robot operates in a similar principle, us¬ 


ing a circular clamp-on type fastener to the column. The 
welding process is computer-controlled, though work¬ 
ers are still involved in overseeing operations at least 
for the time being. 

57.3.2 Reinforcement Manufacturing 
and Positioning 

Concrete reinforcement operations involve cutting and 
bending of rebar (reinforcement bars), precise (relative 
to each other) arrangement of those rebar, binding of re¬ 
bar and final positioning of the rebar elements or mesh 
on a floor or in a mold or formwork system. These 
tasks are labor intensive and hard on a worker’s body 
(e.g., back accidents and damage to a worker’s mus¬ 
culoskeletal system occur about ten times more during 
reinforcement work than during painting work [57.49]). 
Automated systems mitigate the risks and impacts on 
the health of workers and enhance the quality of the 
reinforced concrete structures. Figure 57.13a shows 
a reinforcement production robot and Fig. 57.13b shows 
a teleoperated rebar placing robot. 

57.3.3 Concrete Distribution 

Concrete distribution systems are used to distribute 
mixed concrete with uniform quality over large sur¬ 
faces or over formwork systems. Concrete distribution 



Fig.57.12a,b Steel welding robots. 

(a) Shimizu Corp.; (b) Obayashi Corp 



Fig. 57.13 (a) Automated reinforce¬ 
ment production and (b) teleoperated 
rebar placement (after [57.17]) 
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Fig. 57.14 (a) Takenaka Corp. Automatic Concrete Distributor DB Robo: (b) Tokyu Concrete Distribution Robots 



covers the continuous supply with concrete (pumps, 
hoses), a system which slides in a certain pattern over 
the area where the concrete has to be distributed and 
a concrete ejection system. Systems can be operated 
manually, tele-operated and sometimes sensor-guided 


Fig. 57.15 Concrete slipform road paving machine (cour¬ 
tesy GOMACO Corp.) ◄ 

or fully automatic. Depending on the surface where 
the concrete needs to be distributed, concrete distri¬ 
bution systems can be truck mounted, stationary or 
mobile. In Fig. 57.14a, a track mounted automatic 
concrete distributor robot (DB Robo) can be seen. In 
Fig. 57.14b a mobile concrete distributor system can be 
seen, which due to its wheel-based platform, enables 
an enhanced operating range. Figures 57.15 and 57.16 
show other examples of automated concrete distribution 
systems. 

57.3.4 Robots for Customized Construction 
On-Site 

Several masonry-wall erecting robots have been devel¬ 
oped to-date. Examples of these are the SMAS in lapan 



Fig. 57.16 (a) Programmable, articulated-boom machine for the pumped delivery of fresh concrete (after [57.50]) and 
(b) teleoperated concrete spraying robot (courtesy MEYCO Equipment) 
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and the ROCCO robot at the University of Stuttgart. 
Many of these systems are still in the prototype stage. 
The basic concepts that these systems have in common 
are as follows: 

• Autonomous mobility on the construction site 

• Sensor systems determine the robot’s position and 
orientation in its environment 

• Automatic pick up of the bricks from the pallets 

• Automatic application of mortar 

• Automatic positioning of the bricks. 

ROCCO is a Robot Construction System [57.51, 
52], for Computer Integrated Construction (CIC). Sev¬ 
eral companies and institutes participated in this EU 
funded project in an inter-disciplinary and international 
approach with experts in the fields of construction 
technology, mechanical and electrical engineering and 
information technology from Germany, Spain and Bel¬ 
gium. The goal of the project was the development of 
a computer integrated robot system, which also contains 
a continuous solution in the ICT (Information Commu¬ 
nication Technology) for all steps from the architectural 
design to the automated assembly of the components on 
the construction site. 

The main emphasis of the project lies on the real¬ 
ization of a mobile robot system for construction site 
operation as well as on the integration of a computer 
based system for work preparation and quality control. 
In the work preparation phase the necessary data is gen¬ 
erated for the pre-fabrication and customization of the 
masonry blocks, for the construction site layout and for 
the automatic robot program generation. 

Based on the CAD-representation of the building, 
first the walls are divided automatically into the nec¬ 
essary blocks. In the next step the optimal working 



Fig. 57.17 (a) Robot for residential buildings; (b) robot for indus¬ 


trial use with long reach manipulator 


positions of the mobile robot as well as the positions 
of the pallets and the arrangement of the blocks on the 
pallets are automatically calculated. With this informa¬ 
tion the necessary non-standard and standard blocks can 
be produced and palletized. Finally the robot programs 
are generated automatically out of the calculated geom¬ 
etry information. The user interface on the construction 
site is graphically interactive and enables the user to 
partially re-program the generated robot programs to 
deal with the uncertainties of the construction process 
without the need to learn a specific robot programming 
language. 

To test different approaches based on the construc¬ 
tion application (residential buildings vs. industrial) and 
the sensor integration (autonomous vehicle vs. long 
reach), two systems were developed within the project. 
The first one, for residential buildings, has a reach of 

4.5 m and it is able to handle up to 400 kg. This robot 
is placed over an autonomous vehicle that allows the 
movement on the construction site. Fig. 57. 17a. Its main 
task is the erection of walls in residential buildings. 
The second one is able to handle up to 1000 kg with 
a reach of 8.5 m. This robot. Fig. 57.17b, is placed over 
a towable platform and its main task is the erection of 
external walls in industrial buildings with typical height 
of up to 8 m and standardized layout. 

57.3.5 Robotic Interior Finishing 

Interior finishing work is work conducted inside a struc¬ 
ture (e.g., an office building) to complete construction 
of the space within the structure (e.g., painting and 
false ceiling and gypsum board installation). Due to 
the confined nature of interior finishing work traditional 
material handling equipment (such as cranes) cannot be 
used. The following four systems are examples of inte¬ 
rior finishing robots: 

• The Shimizu CFR 1 (Fig. 57.18a) could be manu¬ 
ally and tele-operated and allowed the installation of 
suspended ceiling panels to a height of up to 3.5 m. 
The robot’s compliant gripper mechanism was able 
to finely adjust the positions of the panels. 

• The Tokyu ceiling panel installation robot (Fig. 
57.18b) could position and adjust the panels and 
also fire nails into the panels to fasten them to the 
underlying ceiling system. 

• Two variants (Fig. 57.18c,d) of interior finishing 
modular mobile robot, developed in Technical Uni¬ 
versity of Munich. 

• The Komatsu Mighty Hand LH (Fig. 57.18e) could 
position and adjust interior wall panels, glass pan¬ 
els, door frames, window casings and the outside 
walls (up to 350 kg) with high accuracy. 
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Fig. 57.18 (a) Shimizu CFR 1 - Interior finishing robot; (b) the Tokyu ceiling panel installation robot; (c) BR2 TU 
Munich - modular ceiling drilling robot system type 1; (d) BR2 TU Munich - modular ceiling drilling robot system 
type 2; (e) Kajima interior wall setting - KOMATSU LH 150 


Concrete Finishing 

Floor finishing is one of the most critical construction 
processes in which construction workers carry or guide 
trowels over an unfinished, wet concrete floor for sev¬ 
eral hours in a stooped posture. In order to reduce these 
physical demands on the workers and to ensure a more 
uniform finish quality, various companies developed 
and employed concepts for robots that can perform 
the concrete-leveling task: the Flat-Kun by Shimizu 
(800m 2 /h), the Kote King by Kajima (500m 2 /h), the 
Surfing Robo by Takenaka (300m 2 /h), the Obayashi 
made by Mitsubishi (500m 2 /h) and the Floor Travel¬ 
ing Robot MF1E by Hazama (300m 2 /h). Each of these 
single task construction robots was able to operate on 
a floor where it was set up into any desired direction 
(e.g., not only move backwards and forwards and turn¬ 
ing but also able to rotate 360° in place). 

The finishing mechanism consisted, in most cases, 
of automatically controlled and operated rotating trow¬ 
els. The degree of autonomy ranged from systems with 
human-machine interfaces for tele-operation to systems 
that could generate motions themselves and to pre¬ 


programming of paths for the robot to follow. In many 
cases gyroscopes and rotating laser levels assisted nav¬ 
igation and motion planning at a low level. After an 
intensive research and development phase that lasted 
until 1985, the first concrete finishing robots where de¬ 
ployed commercially to finish concrete floors in office 
buildings, factories, warehouses and shopping centers. 
The use of the robot systems became efficient once 
a floor plan allowed for a working area that could be 
processed without interruption of more than 500 to 
600 m 2 . Examples of commercial systems are depicted 
below in Fig. 57.19. 

Tile Placement 

The exterior of many buildings are sometimes finished 
with weather-resistant tiles. In Japan single-family 
buildings, factories, offices and high-rise buildings are 
often finished with tiles. Tiles are relatively small build¬ 
ing elements compared to the total surface area of 
a building and thus a very large number of tiles have 
to be installed using the same, repetitive process, which 
involves applying mortar and positioning the tiles on 
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Fig. 57.19 (a) Hazama robot; (b) Kajima Kote King robot; (c) Tak- 
enaka Surf Robo; (d) auto-leveling concrete screed machine (cour¬ 
tesy Somero Enterprises, Inc.) 



Fig.57.20a,b Hazama tile setting robot; (a) robot upon operation; 
(b) finished tile setting 


top of the mortar. This and the fact that building fa¬ 
cades are generally difficult to access make the use 
of automated systems feasible. Hazama’s tile-setting 
robot (Fig. 57.20) also showed that accuracy could be 
enhanced and that the laying of patterns could be ac- 



Fig.57.21a,b Shimizu fireproof coating robots 



Fig. 57.22 Concrete panel installation robot (courtesy of 
Fujita Research) 


complished without dramatically increasing construc¬ 
tion time. 

Fireproof Coating 

In many countries building regulations require that 
steel structures be coated with fire proof materials 
and/or a fire-retarding paint. The application of the 



Fig.57.23a-c Facade painting robots, (a) Shimizu; (b) Kajima; (c) Taisei 
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Fig.57.24a-c Kajima facade 
inspection robot (a,b); 
Comatec Robosoft glass 
cleaning robot (c) 



Fig.57.25a-f Earthmoving: 
(a) Teleoperated excavation 
system (courtesy Fujita 
Research); (b) computer 
assisted road compacting 
system (after [57.53]); 

(c) automated grading 
system (courtesy Caterpillar, 
Inc. ); (d) automated drag line 
control system (after [57.54]); 
(e) autonomous excavator 
robot (after [57.55]); (f) auto¬ 
nomous off-road dump truck 
(after [57.56]) 


fire-roof coating and paint can only be done af¬ 
ter the steel structure has been erected and joined 
on site (e.g., by welding) in order not to inter¬ 
fere with the joining process and in order to avoid 
any damage to the fire-proof coating. Therefore, it 
is not possible to shift the coating operations to 
upstream production steps that can be performed 


in structured factory environments where high effi¬ 
ciency can be achieved. Due to seismic considera¬ 
tions, most high-rise construction involves the use of 
steel structures and thus, the development and em¬ 
ployment of automated and robotic systems that are 
able to apply the fire-proof coatings after the struc¬ 
ture is erected took place in Japan. Robots devel- 
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oped by Shimizu (Fig. 57.21) and others were mostly 
autonomous. 

57.3.6 Robotic Facade Operation 

Facade operations involve the installation of windows, 
complete facade elements or building exterior walls. 
Facade elements are, in modem architecture and es¬ 
pecially in high-rise constmction, decoupled from the 
main load-bearing concrete or steel structure and can 
thus be considered as a type of add-ons. Facade instal¬ 
lation operations are complex operations that involve 
the accurate positioning of heavy parts or elements at 
locations that are difficult to access (e.g., high eleva¬ 
tions without scaffolding). This involves risk of injury 
(and thus extensive safety measures must be taken) 
and of damaging the building or the elements them¬ 
selves. Furthermore, the positioning and alignment of 
prefabricated facade elements requires precision and 
low tolerances. A widespread trend (since the 1980s) 
of designing large buildings as monolithic structures 
that repeat the same or similar facade elements, has 
been a major motivation for investment in automated 
or robotic systems. Figure 57.22 shows an example of 
a tele-operated facade installation robot. 

Painting 

Facade-painting robots were developed to simplify the 
painting or re-painting of high-rise buildings during 
construction and operation. Facade painting robots have 
a particular advantage in that they can keep the paint fin¬ 
ish quality constant. They usually have multiple spray 
nozzles and the spray area is either encapsulated or 
hermetically sealed in order to avoid streaking. A fur¬ 
ther advantage of facade painting robots is the fact that 
workers are not exposed to harmful paint fumes and 
vapours. Single task, facade-painting robots use one of 
the following three different strategies to move along 
the facade: 

• Cable-suspended cage/gondola systems 

• Rail guided systems 

• Vacuum or other adhesion technology. 

The use of fa§ade-painting robots was not con¬ 
sidered efficient for facades with an area < 2000 m 2 . 
Facade-painting robots were thus used primarily to 
paint large facades of warehouses and skyscrapers. Fa¬ 
cades to be processed were required to have a low 
curvature and wherever possible no corners or lugs 
which can hinder the operation of the robot. Further, 
the design of window frames as well as the num¬ 
ber of windows and the area they cover impact the 
feasibility and efficiency of facade-painting robots. Be¬ 
tween 1984 and 1988 various companies introduced 


facade-painting robots. Shimizu and Kajima Corpo¬ 
rations of Japan both applied the principle of the 
suspended cage or gondola (Fig. 57. 23a, b, respec¬ 
tively). The Taisei Corporation of Japan used a rail- 
guided system and combined it with the gondola ap¬ 
proach for their robot (Fig. 57.23c). The fastest of 
the systems (Kajima’s robot) worked with a speed of 
290m 2 /h during the application of the primer paint, 
200m 2 /h for the base coat and 290m 2 /h for the top 
coat. 



Fig.57.26a-c Road maintenance: (a) Teleoperated pothole 
patching robot (courtesy Leeboy); (b) semi-autonomous 
road crack sealing robot (after [57.57]); (c) semi- 
autonomous road crack sealing robot (after [57.58]) 






Robotics in Construction I 57.4 Integrated Robotized Construction Sites 1511 



Fig.57.27a-c Material handling: (a) six degree of freedom robotic crane (after [57.59]); (b) large scale pipe manipulator; (c) large 
manipulator system (courtesy of Shimizu Corp.) 


Robotized Inspection and Maintenance 
Facades of high-rise buildings are in many cases 
equipped with tiles or other surface panels that have to 
be inspected regularly during the building’s life-cycle in 
order to detect structural damage and in order to replace 
tiles or panels that might fall from the facade. Typically, 
workers access those tiles or panels via cages or gondo¬ 
las suspended from the roof of the buildings. This work 
process was considered by Japanese construction firms 
as monotonous, inefficient and dangerous. Further, 
since the method for identifying damaged tiles or pan¬ 
els involves listening to the sound a tile or panel makes 
when gently impacted by a handheld tool, the sounds 
were difficult to classify at high elevations due to wind 
noise. Therefore, with substantial financial commit¬ 
ment, autonomous facade-inspection and maintenance 
robots were developed. Between 1985 and 1988, six 
different facade-inspection robots were developed in 
Japan by the Kajima, Takenaka, Obayashi, Taisei, Tam- 
agawa and Seki Corporations. For the inspection of the 
facade of a 40 m high building (3000 m 2 facade) an in¬ 
spection robot needed in average 8 h including sa 1 h for 
task as preparation, configuration, conversions, disman¬ 
tling and cleaning of the robot (Fig. 57.24a,b). 

Today robotic systems are not only developed for 
the construction of buildings but also for the opera¬ 
tion, maintenance and decommissioning of buildings 
and other construction products. These types of ser¬ 
vice robots were developed for the construction sector 
for inspection of nuclear power plants, exterior walls of 


high-rise buildings and cleaning of high-rise facades or 
glass roofs (Fig. 57.24c). 

57.3.7 Earthmoving 

Earthmoving is a construction process for prepar¬ 
ing project sites for construction by digging, grading, 
trenching, scraping and other similar tasks. There have 
been significant developments in many earthmoving ar¬ 
eas and certain tasks (such as grading) can now be 
completely automated. Figure 57.25 shows several ex¬ 
amples of automated earthmoving equipment some of 
which were still in the research and development phase 
while other are already commercial products. 

57.3.8 Road Maintenance 

Figure 57.26 shows three examples of machines that 
perform pothole repairs and crack sealing on road with 
various degrees of automation. Figure 57.26a is a com¬ 
mercial product while the other two machines were 
research projects undertaken by departments of trans¬ 
portation in the US. 

57.3.9 Material Handling 

The transport and handling of materials is a critical ac¬ 
tivity at most construction sites. Figure 57.27 shows 
examples of material handling robots developed under 
different research efforts. 


57.4 Integrated Robotized Construction Sites 


The first prototypes for mainly automated high-rise 
construction sites were put into operation in 1990 and 
1991 by Shimizu (Fig. 57.28) after five years in devel¬ 
opment and a financial outlay of almost sixteen million 
Euros. Since then, twenty automated high-rise sites 
have been implemented by different companies (Taisei, 
Takenaka, Kajima, Maeda, and Kumagai) [57.60,61], 


57.4.1 Robotic Roof Field Factory Approach 

An integrated robotized construction site involves the 
use of semi- and fully-automated storage, transport 
and assembly equipment and/or robots that are used to 
erect a building almost completely automatically. It is 
an attempt to improve the sequencing of construction 


Part F | 57.4 





Part F | 57.4 


1512 Part F 


Robots at Work 



Fig. 57.28 (a) Robotic trolleys for 
transporting and positioning of beams, 
columns, floor panels, building 
services units and facades, in Shimizu 
SMART system; (b) SMART roof 
field factory view 



Fig.57.29a-c Obayashi ABCS 
(a) early construction phase; (b) inter¬ 
mediate construction phase; (c) final 
construction phase 


processes and construction site management by using 
real-time control. This includes an unbroken flow of in¬ 
formation from the planning and design phase through 
programming the on-site robots and controlling and 
monitoring the construction operations. 

The robotic roof held factory approach is typically 
implemented once the building foundations have been 
laid. The production equipment, on which the steel con¬ 



Fig.57.30a,b The AMURAD Construction Strategy: floor-wise up- 
pushing of the whole building. Kajima, Japan (a) early construction 
phase; (b) later construction phase 


struction has been installed with assembly and transport 
robots, is then covered completely with a roof made of 
plastic him. Depending on the roof system used, this 
process takes from three to six weeks after which the 
robots go into production. Due to the lack of space 
around building sites in Japan, steel and concrete plants 
are also often installed to supply parts in 10 min cycles 
on a just-in-time basis. 

Prefabricated parts are checked and then placed in 
specihc depots at the foot of the building or in the 
building itself where they are readily accessible by the 
robots. This is where the automated construction pro¬ 
cess actually starts. As many as 22 robots equipped with 
automatic crane winches deliver the columns, supports, 
Boors, ceilings, walls and other elements to the floor 
of the steel skeleton under construction. These compo¬ 
nents are then positioned and fixed into place almost 
completely automatically. The steel columns and sup¬ 
ports are joined together by welding robots after they 
have been positioned. The position and quality of the 
welding seams are monitored with lasers. 

In the Obayashi ACBS (Automatic Constructions 
Building System) (Fig. 57.29 and in I43JEM5E3), 
once a storey has been finished, the whole support struc¬ 
ture, which rests on four columns, is pushed upwards 
by hydraulic presses to the next storey over a 1.5 h pe¬ 
riod. Fully extended, the support structure is 25 m high; 
retracted it measures 4.5 m. Once everything has been 
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moved up, work starts on the next storey. By construct¬ 
ing the topmost storey of the high-rise building as the 
roof at the beginning of the building process, the site 
is closed off in all directions, considerably reducing the 
effect of the weather and any damage it might cause. 

57.4.2 Robotic Field Factory on-Ground 
Approach 

The Amurad-system (Fig. 57.30), developed by the Ka¬ 
jima Corporation, is a way of construction based on the 
idea of after the first floor is built (which is the top 
floor), it is pushed up one floor at a time with the aid 
of large hydraulic cylinders. The plumbing, electrical 
and mechanical equipment and the interior fittings and 
cladding of the facade then begin. This process is re¬ 
peated until the building is completed, where the last 
floor to be built becomes the ground floor. In order to 
realize the AMURAD System three automated systems 
were developed for pushing up the whole building, for 
transporting and assembling and for material handling. 

57.4.3 Robotic Deconstruction and Robotic 
Recycling 

Controlled deconstruction supported by robotic sys¬ 
tems can be combined with component reuse systems 
since some elements such as structural steel have long 
life spans, and total recycling (i. e., scrapping, melt¬ 
ing and recasting) consumes large amounts of energy. 
In a component reuse system all structural building 
components are accepted as trade-ins. Therefore, the 
deconstruction process comprises a reversed version of 
the construction process, which if it is based on industri¬ 
alized fabrication, all deconstructed parts and elements 
could be directly reintroduced into the fabrication sys¬ 
tems. The DARUMA system by Kajima Corporation 
(Fig. 57.31) enables such a controlled deconstruction 
process. Deconstructed structural steel elements are 
transported to special dismantling factories just-in-time 



Fig.57.31a,b Systemic and automated floor-by-floor building de- 
construction, Kajima, DARUMA System, Japan (a) early decon¬ 
struction phase; (b) later deconstruction phase 

and just-in-sequence, where the old joints and finishes 
can be dismantled under factory conditions and fed 
into advanced reuse cycles. Modularized and standard¬ 
ized structural units can be inspected and renewed and 
then equipped with new finishes according to each 
customer’s needs. Component Reuse systems could be 
connected to Advanced Construction ERP Systems and 
companies could match customers who want to sell 
their modular building for reuse and customers willing 
to buy reused building modules for further customiza¬ 
tion. Renewed building components are reorganized 
and customized in the factory and transported to other 
building sites. A combined system of controlled decon¬ 
struction and component reuse saves large amounts of 
materials and energy. 

By adopting integrated industrialization processes 
the construction industry would have the chance to 
address all parameters relevant for sustainable eco¬ 
nomic, environmental and social development. In¬ 
novative industrialization-oriented architectural design 
structures, appropriate modularization and standardiza¬ 
tion of building structures, logistics, equipment and 



Fig. 57.32 (a) Teleoperated 
demolition robot (courtesy 
Brokk AB); (b) water-jet 
demolition robot (courtesy 
Conjet AB) 
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processes, could serve as a fundamental integration 
framework. Customized prefabrication could further be 
able to supply construction sites with individualized el¬ 
ements. Hyper-flexible robotic systems could support 
a limited amount of trained workers to perform posi¬ 
tioning, joining and finishing operations. Construction 
ERP systems could support organization as well as 
a lean and demand oriented construction based on just- 
in-time and just-in-sequence resource supply mean¬ 


while locally based factories grant identity and reduced 
logistic effort. Moreover integrated industrialization 
would not only be limited to the fabrication but also link 
systems of controlled deconstruction and component 
reuse to a network for continuous resource circulation. 

In addition to fully automated deconstruction sys¬ 
tems, a number of teleoperated demolition robots have 
been widely adopted in the construction industry. Two 
examples of these are shown in Fig. 57.32. 


57.5 Currently Unsolved Technical Problems 


A recent study found that the US construction in¬ 
dustry does not realize approximately 15 billion US 
dollars per year in potential savings due to inadequate 
interoperability related to information exchange and 
management practices [57.62], 

57.5.1 Interoperability 

Although the lack of interoperability between the vari¬ 
ous information systems used in construction is a sig¬ 
nificant source of inefficiency for the industry, it is 
also a roadblock to the use of automated systems in 
construction. Automated systems need electronic infor¬ 
mation on past, current, and/or projected future states 
of a construction project to function efficiently. 

For example, in order for a robotic crane to pick 
a steel beam from the site and deliver it to its target loca¬ 
tion, the robot must be able to know that the steel beam 
has been delivered to the site, as well as its current po¬ 
sition and orientation. While information on the current 
inventory of parts on a site may be available on paper, 
it is rarely available electronically unless someone en¬ 
ters it manually into some computer system, which in 
turn may not be compatible with other systems used 
in that project. In many of the examples of automated 
construction technologies presented in this chapter, cus¬ 
tom electronic databases and/or data formats have to be 
devised to demonstrate the robot’s operation. In some 
cases, the electronic information had to be entered man¬ 
ually from paper into a computer for the robot to work 
correctly. 

In addition to information exchange and manage¬ 
ment, many of the measurement instruments and sen¬ 
sors used in construction are not interoperable. This 
problem is not limited to the construction sector but 
is a relatively large problem in many robotic appli¬ 
cations where different types of sensors are used. 
Several efforts are underway to make sensors inter¬ 
operable [57.63] and construction equipment in gen¬ 
eral [57.64], but the issues have yet to be resolved. 


57.5.2 Structural Connection Systems 

Traditionally, structural member connections in con¬ 
struction have been designed for human installation. 
Whether using bolted, welded, or other types of con¬ 
nections, manual labor is usually involved in guiding 
the mating parts together and in establishing the con¬ 
nection. 

For example, in structural steel erection, workers 
perched on the structure typically guide a crane opera¬ 
tor through visual or auditory cues in order to maneuver 
a steel beam (or column) into place. The workers must 
then physically manipulate the beam in order to align 
corresponding surfaces for bolting or welding. Once the 
correct beam pose has been achieved, it must be main¬ 
tained while the workers temporarily fasten the beam to 
the structure. The workers then release the beam from 
the crane and permanently fasten the beam to the struc¬ 
ture at a later stage. 

For automated (or robotic) construction to work, 
new connections that are more amenable to automation 



Fig. 57.33 A drop-in, shear-load-only, steel connection 
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b) 



Fig.57.34a,b A piping and electrical 
drop-in connector shown (a) disas¬ 
sembled and (b) assembled 


must be designed. These connections need not mimic 
traditional, human-installed connections, but should 
be optimized for use with robots instead. For exam¬ 
ple, the Lehigh University Advanced Technology for 
Large Structural Systems (ATLSS) Center designed 
a gravity-load-only shear steel connector [57.65] back 
in the early 1990s that is more suitable for automated 
construction (Fig. 57.33). This type of male-female 
connector for automatic assembly of building mod¬ 
ules allows, together with an adequate control strategy, 
small tolerances, which permit assembly by automatic 
cranes [57.66]. 

An example of a nonstructural connector that is 
more amenable to automation is the piping and elec¬ 
trical connector developed for the FutureHome project, 
shown in Fig. 57.34 [57.21,67], 

Although automated welding has been applied in 
some limited form in construction, it is generally used 
to replace manual welding without changing the design 
of the parts being welded. In other words, no significant 
change in the way in which construction components 
are designed has occurred as a result of automated 
welding. Moreover, apart from a few examples of au¬ 
tomated welding applied at the construction site, most 
of the other limited applications are done at the compo¬ 
nent fabrication facility. 

57.5.3 Tolerances 

Specifications for tolerances in the construction sector 
exist for most types of construction, however, they are 
not always achieved in practice [57.68]. For example, 
it has been stated that one of the biggest sources of 
problems in structural steel erection in the US is that 
fabricated pieces are often out of tolerance, and that this 
is only discovered during installation at the construc¬ 
tion site [57.69]. However, since the finished facility 
must meet the design tolerances before it is accepted, 
this shifts the burden to workers and supervisors during 
construction. The workers are expected to handle prob¬ 
lems as they come up rather than expect all fabricated 
construction components to be within tolerance. Since 
most construction projects are under tight schedules, it 
is often preferable to fix these kinds of problems on site 


rather than wait for replacement components to be fab¬ 
ricated and delivered. 

However, tolerance problems are not all due to 
fabrication errors. Installation problems are also re¬ 
sponsible for out-of-tolerance problems, for example, 
anchor rod installation has been an area of concern for 
structural steel erection. (Anchor rods (or bolts) are the 
connection interface between the concrete foundation 
and the structural steel columns. The rods are usually 
installed by the concrete foundation crew before the 
concrete is dry.) The situation of anchor rod patterns 
that do not match the hole patterns in a mating column 
is an identified problem in construction [57.26,70]. 

Given the relatively loose achievable tolerances in 
construction, the application of robotics in construc¬ 
tion faces an uphill battle. This, in addition to the 
unstructured nature of the construction site environ¬ 
ment, requires that robots either be highly intelligent in 
order to correctly interpret and react to their surround¬ 
ings or to be human assisted. However, site structure 
and tolerances are expected to improve as pressures to 
reduce costs and improve productivity continue to rise. 
Improvements in site organization and construction tol¬ 
erances have already been proved to be achievable in 
a few cases, as has been demonstrated in Japan [57.71]. 
However, the economic case for these demonstration 
projects has yet to be made. 

57.5.4 Power and Communications 
in the Field 

Unlike manufacturing environments, in which spe¬ 
cially designed factories are outfitted with the necessary 
power and communications installations, a construction 
project often begins before such resources have been in¬ 
stalled at the site. Therefore, robots with large power 
requirements that need to communicate with supervi¬ 
sory systems located off site would be challenging to 
implement without significant added cost. 

Although communications technologies have ad¬ 
vanced significantly in the last few decades, it is still 
considered difficult to maintain a reliable local-area 
network at a construction site and to connect that net¬ 
work to the Internet. The use of cellular telephones 
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with a press-to-talk (the digital version of the tradi¬ 
tional two-way radio communication method) feature 
has largely replaced the traditional two-way radios 
that also revolutionized on-site construction commu¬ 
nications. However, in order for construction sites 
to become more automated, reliable interference-free 
high-bandwidth networks must be able to carry data 
transmission between sensors, machines, and supervi¬ 
sory systems. 

57.5.5 Sensing 

Field measurements are an integral part of the construc¬ 
tion process. The tape measure and the transit have 
been used in construction for decades for measuring 
distances and angles, respectively. However, construc¬ 
tion measurements are not limited to distances and 
angles, but can also include measurements of installed 
quantities, percentages of completion of activities, and 
so on. All of these measurements are necessary to be 
able to lay out the site where a facility is to be built, 

57.6 Future Directions 

As previously discussed, developing new methods for 
collecting, processing, analyzing, and communicating 
construction information is a significant area of con¬ 
struction automation research, and will dominate near- 
term efforts. As this construction information becomes 
more readily accessible, automation of processes can be 
enabled directly from the combination of design infor¬ 
mation and current site status that is accurately captured 
and shared. Resource tracking will become ubiquitous 
and just-in-time delivery of needed materials and equip¬ 
ment will happen throughout the site. 

Advances in mobility (humanoid robotics, smart 
cars, legged locomotion, etc.) will enable ever more 


to measure the conformance of the as-built facility to 
the intended design, and to monitor safety, productivity, 
and progress. 

In the past few decades, more advanced means 
of making measurements on construction sites have 
come into play. These include (but are not limited 
to) total stations, GPS, indoor GPS, ultra-wide band, 
laser scanners, ground-penetrating radar, equipment 
and structural health monitoring sensors, concrete ma¬ 
turity meters and radio frequency identification [57.72- 
84], These technologies, as well as others that have 
not yet been adopted in the construction industry, are 
crucial for enabling more automation and robotics in 
construction. Nevertheless, despite all of these advances 
in technology, one of the biggest challenges for im¬ 
plementing robotics at the construction site is to be 
able to provide accurate and up-to-date information to 
the robots about where everything is and what other 
equipment are doing. This challenge will eventually be 
solved by the introduction of more sophisticated sens¬ 
ing and perception systems. 


automated material handling on the job site. These 
advances will require better control systems for the con¬ 
struction robots that can provide high payload and good 
positioning accuracy. The increased use of robots at 
the construction site will also drive research into safety 
systems for construction robots working around human 
workers and other machines. 

Perhaps most important, more extensive automatic 
design systems will enable more prefabrication of 
building components and new methods of assembling 
those components on site, which in turn will provide 
the promise of faster, better, and cheaper construction 
robotics first envisioned in the 1980s. 


57.7 Conclusions and Further Reading 

The application of robotics to construction has yet to i 
catch up with other industries such as automobile man- i 
ufacturing. Construction presents a unique challenge 1 
for robotic applications. The construction environment t 
is cluttered, unstructured, and teaming with human a 
workers. In addition, construction processes are usually i; 
labor intensive and have to accommodate wide mar- t 
gins of error in the constructed facility. The application i; 
of robotics in construction to date has been limited s 
to commercial teleoperated and programmable ma¬ 
chines. Autonomous or semiautonomous machines are I 
currently mostly limited to research projects within var- e 


ious nonconstruction organizations. With the increase 
in competition throughout the global construction mar¬ 
ket, construction companies are on the lookout for ways 
to improve productivity, quality, and safety. The use of 
automation and robotics is one answer that the industry 
is slowly turning toward. However, before these poten¬ 
tial solutions can be successfully applied, much work 
is needed to improve construction tolerances, develop 
standards, and achieve real-time site status monitoring. 

The International Association for Automation and 
Robotics in Construction holds an annual confer¬ 
ence (the International Symposium on Automation and 
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Robotics in Construction) at which researchers can 
present the latest developments in the field. The pro¬ 
ceedings from this conference hold a wealth of informa¬ 
tion on the state of the art of the field and are accessible 
to the general public through IAARC’s website (www. 
iaarc.org). 

There are several journals that publish articles 
on various aspects of automation in construction. 


Most notable of these are Automation in Construction, 
Computer-Aided Civil and Infrastructure Engineering, 
the Journal of Computing in Civil Engineering, and 
the Journal of Construction Engineering and Man¬ 
agement. In addition, some journals publish special 
editions on robotics in construction. Some examples 
are Autonomous Robots and the Journal of Advanced 
Robotic Systems. 


Video-References 


Obayashi ACBS (Automatic Constructions Building System) 

available from http://handbookofrobotics.org/view-chapter/57/videodetails/272 
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58. Robotics in Hazardous Applications 


James Trevelyan, William R. Hamel, Sung-Chul Kang 


Robotics researchers have worked hard to real¬ 
ize a long-awaited vision: machines that can 
eliminate the need for people to work in haz¬ 
ardous environments. Chapter 60 is framed by 
the vision of disaster response: search and res¬ 
cue robots carrying people from burning buildings 
or tunneling through collapsed rock falls to reach 
trapped miners. In this chapter we review tangi¬ 
ble progress towards robots that perform routine 
work in places too dangerous for humans. Re¬ 
searchers still have many challenges ahead of 
them but there has been remarkable progress in 
some areas. Hazardous environments present spe¬ 
cial challenges for the accomplishment of desired 
tasks depending on the nature and magnitude of 
the hazards. Hazards may be present in the form of 
radiation, toxic contamination, falling objects or 
potential explosions. Technology that specialized 
engineering companies can develop and sell with¬ 
out active help from researchers marks the frontier 
of commercial feasibility. Just inside this border 
lie teleoperated robots for explosive ordnance dis¬ 
posal (EOD) and for underwater engineering work. 
Even with the typical tenfold disadvantage in ma¬ 
nipulation performance imposed by the limits of 
today's telepresence and teleoperation technology, 
in terms of human dexterity and speed, robots 
often can offer a more cost-effective solution. 
However, most routine applications in hazardous 
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environments still lie far beyond the feasibility 
frontier. Fire fighting, remediating nuclear con¬ 
tamination, reactor decommissioning, tunneling, 
underwater engineering, underground mining and 
clearance of landmines and unexploded ordnance 
still present many unsolved problems. 


58.1 Operation in Hazardous Environments: 
The Need for a Robotics Solution 


Routine work in hazardous environments presents many 
challenges for robotic devices that depend on both the 
tasks and the hazards. Hazards may be present in the 
form of temperature extremes, radiation, toxic fumes 


and materials, potential explosions, risk of electrocu¬ 
tion, the absence of breathable air in space, and the 
high pressure of underwater environments. When the 
magnitudes of one or more hazards reach the point 
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that human exposure would either represent a direct 
threat to life or unacceptable long-term health conse¬ 
quences, some form of remote operations that separate 
humans from the hazards must be employed. A long- 
extant example of such operations is the nuclear indus¬ 
try where telerobotics has become a well-established 
technique for operations in environments with high lev¬ 
els of nuclear radiation. Many of the technical roots 
of modem robotics technology can be traced back to 
nuclear remote handling manipulators and support sys¬ 
tems. Remote handling and operations concepts using 
engineered systems that allow humans to perform haz¬ 
ardous work while staying in a safe environment have 
evolved since the 1940s. Today, such remotely operated 
systems are widely used and recently they have become 
routinely used in explosives disposal, security opera¬ 
tions, handling of dangerous biological materials, and 
routine inspection of industrial systems such as power 
lines, substations, sewers, piping systems, pressure ves¬ 
sels and many others. 

A remote handling system will generally involve 
subsystems for mobility, manipulation, tooling, sens¬ 
ing, and human-machine interfacing (Fig. 58.1). Nom¬ 
inal operations involve the collective workings of these 
subsystems to accomplish remote operational goals. 
Any remote handling system will eventually experience 
some aspect of off-nominal operation that may be the 
result of unexpected environmental events or system 
malfunctions. The fundamental idea is to connect the 
human operator to the remote environment via a power 
and signal infrastructure that allows effective operation 
in the remote environment: mechanized devices and 
sensor systems that allow the human’s perception and 
action capabilities to be projected into the hazardous 
environment to perform remote operations. The more 


realistic this projection, the more natural and effective 
the human remote control will be. 

Because of their inherent complexity and the very 
nature of remote operations, operator training is a major 
challenge that requires the use of simulations and cold 
testing facilities that provide operators with comprehen¬ 
sive and realistic training. Such training will typically 
encompass all aspects of nominal and anticipated off- 
nominal operations. 

Remote handling systems themselves will even¬ 
tually experience equipment failures. Remote recov¬ 
ery/maintenance/operation of failed remote systems 
must be an integral part of their basic design and op¬ 
erational features. Hardware/software features must be 
provided for the analysis of, recovery from, and correc¬ 
tion of problems. 

Robotics researchers have worked hard to realize 
a long-awaited vision: machines that enable people 
to perform routine work without exposure to haz¬ 
ards. In this chapter we review progress and some of 
the current and future challenges that lie ahead for 
researchers. 

Technology that specialized engineering compa¬ 
nies can develop and sell without active help from 
researchers marks the frontier of commercial feasi¬ 
bility. Just inside this border lie teleoperated robots 
for explosive ordnance disposal (EOD) [58.1,2] and 
for underwater engineering work. Even with the typi¬ 
cal tenfold disadvantage in manipulation performance 
imposed by the limits of today’s telepresence and tele¬ 
operation technology, compared with human dexterity 
and speed, robots can often offer a more cost-effective 
solution. Naturally, if people need to wear protective 
suits to protect them from hazards, their endurance and 
dexterity will be restricted. 



Fig. 58.1 Basic subsystems of 
a remote handling system 
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Fig. 58.2 Remotely operated vehicles designed for underwater engineering work, demonstrating an ingeniously sim¬ 
ple technical arrangement compared with conventional free-swimming vehicles, (courtesy of Total Marine Technology, 
Australia) 


Ingenuity stimulated by commercial incentives can 
greatly reduce the operating costs of remotely operated 
equipment. Figure 58.2 illustrates the elegant simplic¬ 
ity of the NOMAD remotely operated vehicle (ROV) 
that pulls itself deeper with a simple electric winch. It 
can be left in place overnight or during rough weather. 
Free-swimming robots are much more complicated and 
expensive and have to be hauled out of the water when 
not under active control. 

Limited autonomy or autonomous operation for 
a restricted time can relieve operator fatigue and allows 
unmanned aerial vehicles (UAVs, often referred to as 
drones) to fly extended reconnaissance missions with 
occasional precision weapon delivery. Such missions 
would be too hazardous or too politically sensitive for 
manned aircraft. 

Even after the first decade of the 21st century, with 
60 years of remote operating experience, most haz¬ 


ardous applications still lie far beyond the frontier of 
commercial feasbility. Fire fighting, remediating high- 
level nuclear contamination [58.3], reactor decommis¬ 
sioning, tunneling, underground mining, and most land¬ 
mine and unexploded ordnance problems still present 
many research challenges. Most attempts to use first- 
generation mine rescue robots reported in the press in 
2000 and 2001 have merely created distractions for the 
people who regularly have to risk their lives saving 
their trapped colleagues (e.g., the Australian Numbat 
robot [58.4]). The Fukushima Daiichi nuclear reactor 
disaster triggered by the March 2011 tsunami that dev¬ 
astated parts of lapan serves as a reality check on 
our capacity to use robots effectively for hazardous 
tasks. Beyond the carefully engineered environments 
that characterize the nuclear material handling indus¬ 
try, the disappointing performance of several robots has 
demonstrated just how far we still have to go. 


58.2 Applications 

Potential applications of robotic systems in hazardous 
environments encompass an extremely wide spectrum, 
limited only by human imagination. The solutions for 
these different environments are equally diverse. In 
general such applications involve unique challenges as¬ 
sociated with the uncertainty and unstructured nature 
of the associated tasks. In this discussion, two appli¬ 
cation areas that are very different have been selected 


to give the reader a deeper sense of technology evolu¬ 
tion, accomplishments, and remaining challenges. The 
first application we will discuss is landmine eradi¬ 
cation, sometimes referred to as demining, a current 
application domain with humanitarian importance and 
extremely difficult and hazardous outdoor conditions. 
The second application area is hazardous nuclear and 
biological material handling, a decades-old industry 
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that has strongly influenced many aspects of robot ma¬ 
nipulation and mobility research and development. We 
conclude this section with a brief discussion on the 
lessons learned from the Fukushima Daiichi disaster, 
lessons that help to strengthen our argument that we are 
still at a very early stage of development in robotics for 
routine operations in hazardous environments. 

58.2.1 Eradicating Landmines 

Efforts to construct a practical robotic device to help 
with landmine clearance have met with only limited 
success. While remote control technology has enabled 
some existing machines and vehicles to be used in 
hazardous situations, we are still a long way from 
achieving reliable robotic mine clearance. It is instruc¬ 
tive to understand why many expectations turned out to 
be hopelessly optimistic. 

Landmines, a simple type of victim-activated explo¬ 
sive device, were used extensively in Europe and North 
Africa during the Second World War between 1939 and 
1945. Extensive clearance operations in 1945 and 1946 
removed nearly all of the landmines then in use [58.5, 
part 1, pp. 15-25]. 

Landmines were used extensively in subsequent 
decades. Along with antipersonnel cluster bombs, they 
caused extensive civilian and military casualties in Viet¬ 
nam and Cambodia from the 1960s onwards. However, 
it was not until their widespread use in Afghanistan, 



Fig. 58.3 Antipersonnel fragmentation mine (left) and buried blast 
mine (right) 


Angola, Cambodia, and several other countries in the 
1980s that they were recognized as a major humanitar¬ 
ian problem. Landmines blocked aid efforts needed to 
rebuild communities following civil conflicts. 

The Red Cross and the International Campaign to 
Ban Landmines (ICBL) successfully promoted a ban 
on the use of landmines that came into effect in 1997 
as the Ottawa Treaty. The knowledge that thousands 
of children were losing their legs, even their lives, 
motivated hundreds of researchers to develop new 
technologies to help eliminate this threat. By 2000 
ICBL estimated that over 80 countries were affected 
by landmines and other explosive remnants of war 
such as cluster bombs. Although several countries that 
have not signed the Ottawa Treaty still have extensive 
stocks of landmines, the treaty has been effective 
in restricting the use of landmines more through 
peer pressure. A few countries and several non-state 
actors continue to deploy landmines. However, other 
explosive remnants of war such as cluster bombs and 
other munitions have become, yet again, an increasing 
problem in Iraq, Afghanistan, Libya and Syria. 

There are several basic types of landmines and 
unexploded ordnance (UXO) that continue to cause 
problems in many countries. 

Antipersonnel (AP) blast mines, made predomi¬ 
nantly of plastic with small metal firing pins and det¬ 
onator cases, typically contain between 20 and 100 g of 
explosive (Fig. 58.3). These mines only cause extensive 
injuries when they detonate within a few centimetres of 
a person. Typically they lie buried just below the ground 
surface and are activated when the victim steps on top of 
the mine. Shattered fragments of bone pass through the 
flesh of the leg at high velocity. If the victim survives 
long enough to reach hospital, amputation above or be¬ 
low the knee usually saves his or her life but the victim 
will need prosthetic legs, replaced at regular intervals. 

Antipersonnel fragmentation mines contain simi¬ 
lar quantities of explosive with a thick metal case that 
breaks into high-velocity fragments when the mine 
explodes just above the ground surface. Older fragmen¬ 
tation mines were mounted on posts; some varieties lie 
buried but jump into the air when activated and explode 
at waist height, killing or seriously wounding victims 
up to 200 m away. These mines are much easier to de¬ 
tect so they are often protected by nearby antipersonnel 
blast mines to deter theft. 

Anti-vehicle (AV) or anti-tank mines (ATs), made 
predominantly of plastic with small metal firing pins 
and detonator cases, are large versions of the AP blast 
mine and typically contain 5—10 kg of explosive. Some 
have a thick metal plate on top that can penetrate 50 cm 
of armor plate on the underside of a tank. These mines 
cause significant damage even to mine-resistant vehi- 
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cles which have blast-resistant hulls, offset wheels, and 
additional protection for occupants. 

Air-dispersed munitions such as cluster bombs 
(CBs) were not intended to be victim-activated. Several 
hundred are released at one time from a single canis¬ 
ter and they are designed to explode on impact with the 
ground. Typically between 5 and 25% fail to explode 
immediately and lie in a partially triggered state either 
on or just below the ground surface. Some will deto¬ 
nate in response to electromagnetic fields from metal 
detectors, and others will detonate with the slightest 
movement. Most explode like powerful fragmentation 
mines with a lethal radius of up to 200 m. 

Improvised explosive devices (IEDs), often in the 
form of roadside bombs, are increasingly used by insur¬ 
gent groups fighting organized military forces (asym¬ 
metric warfare). They are often made from large UXOs 
fitted with remote controlled detonators l<s>MSH3iE3l. 
Ironically the UXO is often unintentionally donated by 
the same organized military forces who become the 
targets of these devices. IED detection and clearance 
has become one of the highest priorities for military 
robotics research. 

Evolution of Landmine Clearance Techniques 
Removing landmines is usually both laborious and 
hazardous. It is important to distinguish between hu¬ 
manitarian mine clearance and military mine clearance 
methods (sometimes called breeching). Military mine 
clearance has to work fast, in all conditions (even un¬ 
der fire), and therefore it is unrealistic to aim for 100% 
clearance. In humanitarian operations there is less time 
pressure and work can be suspended in unfavorable 
conditions, and the aim is 100% clearance. Since the 
1990s, the expectation in industrialized countries of 
low casualties from military operations often demands 
very high clearance standards even for urgent military 
operations. 

Humanitarian mine clearance typically starts years, 
perhaps decades, after the mines were laid. The mines 
lie buried or hidden from view. They deter people from 
entering the land so vegetation often grows thickly 
(Fig. 58.4). Drainage systems rapidly become clogged, 
denying access in wet conditions. 

The traditional manual method for removing land¬ 
mines has been to use a metal detector to locate metal 
fragments close to the ground surface and then to care¬ 
fully check each metal fragment to see if it is associated 
with a mine or explosive device Any trip¬ 

wires and vegetation have to be removed, with great 
care, before a metal detector can be used. In many 
areas deminers have to investigate hundreds or thou¬ 
sands of metal fragments for every mine found. Manual 
mine clearance also requires careful organization and 



Fig. 58.4 Typical ruined house overgrown by vegetation 
in a village in northern Croatia, possibly containing mines 
or booby traps. The entire village population was forced to 
leave in 1991 and the houses were looted and intentionally 
severely damaged. Vegetation problems like this must be 
taken into account in considering practical mine and UXO 
clearance devices. August 1999 (courtesy of J. Trevelyan) 

marking of the ground to ensure safety and thorough 
clearance. Currently it is still the method that guaran¬ 
tees the lowest risk of residual mine contamination but 
it is expensive, typically costing US$ 1-5 /m 2 . 

Armored mine clearance machines using hammers 
mounted on the end of rapidly spinning chains (flails) 
first appeared in the 1940s but have not been able to 
neutralize mines with sufficient reliability for most hu¬ 
manitarian applications [58.6]. 

In the late 1990s commercial mine clearance or¬ 
ganizations operating in thick vegetation in Bosnia 
Herzegovina and Croatia realized that flails spinning 
just above the ground could rapidly remove vegetation 
and trip wires to prepare the ground for manual clear¬ 
ance, often assisted by mine detection dogs. Clearance 
costs have been reduced by up to 80% (particularly in 
thick vegetation) using different combinations of ma¬ 
chines, detection dogs, and manual clearance. 

Ground milling machines use metal drums stud¬ 
ded with hard cutters that shred buried objects. They 
require more power than flails but can operate with 
greater levels of reliability (Fig. 58.5). Both flails and 
ground milling machines have been extensively used in 
Croatia to recover large areas of formerly productive 
agricultural land. Both kinds of machines can with¬ 
stand a limited number of AT and moderate-size UXO 
explosions before main bearings and other compo¬ 
nents need to be replaced I 

Several of these machines 
have remote operation capabilities (Fig. 58.6). 

Naturally, machines operate best on flat or gently 
sloping ground, which is also the land that is most valu¬ 
able for agriculture and human habitation. Thick forest 
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Fig. 58.5 Flail machine using hammers on the ends of spinning chains to clear vegetation and tripwires. This machine 
will also detonate a proportion of buried mines {inset) (courtesy of Scanjack AB, Sweden) 


Fig. 58.6 Bozena teleoperated demining vehicle (courtesy of Way 
Industry, Slovakia) 

and mountainous terrain still requires traditional man¬ 
ual clearance and, in most countries, will not be cleared 
of mines for a long time, if ever. 

Mechanized clearance methods continue to evolve 
with improvements to machines and techniques. Ma¬ 
chines can be used for survey, risk assessment, and risk 
reduction tasks to help determine the need for more 
expensive manual clearance methods. Mine action pro¬ 
grams are gradually shifting from an emphasis on total 
clearance in the 1990s to one of progressive prioritized 
risk reduction involving a series of measures including 
high-security fences, mechanized survey and risk re¬ 
duction methods, and selective manual clearance [58.5, 
part 4]. Protective measures applied to agricultural 
machinery offer cheaper alternatives in low-AT-risk ar¬ 
eas [58.7]. 


Evolution of Demining Research Priorities 
Technological development in landmine clearance from 
within the demining community has mainly been driven 
by the search for improved productivity. Many of the 
comments in this section are based on numerous discus¬ 
sions with experienced demining personnel who have 
tried new technologies in the field. References have 
been cited where further detailed written information 
is available. 

In the mid-1990s there was the expectation that, 
with sufficient research, advanced technology detectors 
could replace eddy-current metal detector technology 
that had been in use since the 1940s. Metal detectors 
also react to metal fragments in the ground. A detec¬ 
tor that could confirm the presence of explosive, it was 
thought, would save having to investigate all these false 
alarms. The most promising line of research seemed 
to be data fusion: combining signals from a metal 
detector, ground-penetrating radar, infrared detectors, 
thermal neutron detectors, and even acoustic detectors. 
Astute observers at research conferences have pointed 
out that these signals are often well correlated, even 
in the presence of false alarms: producing a reliable 
detector was going to be hard work. Their forecasts 
turned out to be very accurate. Only one such detec¬ 
tor is currently in operation: the handheld standoff mine 
detection system (HSTAMIDS) detector used by US 
military forces in Afghanistan employs a combination 
of ground-penetrating radar and eddy-current metal de¬ 
tection. Little information on its effectiveness has been 
released and no independent trials have been reported. 
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Experienced research groups report that ground-pene¬ 
trating radar requires accurate alignment of the detector 
with the ground surface (to eliminate ground surface re¬ 
turns) and also with the target center point to enable 
the target to be characterized reliably. If the principal 
metal component of the landmine coincides with its ge¬ 
ometric center, a common feature of minimum metal 
mines, the metal detector can be used for alignment. 
However this is not always the case and one cannot 
guarantee the absence of other metal fragments near 
the mine. Ground-penetrating radar provides confusing 
returns in very dry or very wet conditions and is also 
susceptible to false alarm indications from underground 
discontinuities such as stones, sticks, animal burrows 
etc. Publishers mostly downplay these difficulties and 
prefer only to report positive results, not false negatives. 
These issues only emerge from discussions with devel¬ 
opers who have seriously evaluated technology in field 
conditions. 

The major performance improvements in sens¬ 
ing have been obtained by compensating eddy-current 
metal detectors for soil magnetization, enabling them 
to work in a much wider range of soil conditions. Im¬ 
provements in sensitivity can help with minimum metal 
mines but can also result in a large number of false 
alarms from smaller metal fragments. Metal detector ar¬ 
rays have been fitted to vehicles to speed up clearance 
of paved areas and roads [58.8]. 

By the late 1990s slow progress with sensors had 
become more apparent and research priorities after 
2000 gradually turned to mine detection dogs and large 
demining machines. 

The Afghanistan Mine Action Center (MACA) 
started using mine detection dogs around 1993 but 
it was not until 1998 that this program was running 
effectively. There were several difficulties. The first 
challenge was that close association between humans 
and dogs was socially unacceptable in Afghanistan. The 
second challenge was to devise ways to use dogs and 
manual mine clearance in an effective combination pro¬ 
viding reliable clearance with high productivity. This 
was much the greater challenge, but by 1998 the cost of 
clearance using dogs was around one-third the cost of 
manual clearance. It was then that the problems started 
to appear: the occasional missed mine that could not 
be explained by lack of organization or failure to follow 
procedures. At the same time, carefully controlled trials 
of mine detection dogs in Bosnia had returned highly 
variable results. On several occasions dogs had walked 
past blocks of trinitrotoluene (TNT) lying almost visi¬ 
ble in the ground. Yet, at the same time, a number of 
commercial demining agencies were routinely declar¬ 
ing land free of mines using similar dogs. In late 1999 


the Bosnian Mine Action Center ran a carefully con¬ 
trolled test in which around 80% of the dogs failed to 
achieve the required performance standard. The results 
were hotly contested at the time and the international 
community organized a systematic trial of mine detec¬ 
tion dogs through the Geneva International Centre for 
Humanitarian Demining (GICHD). 

By 2001 it was apparent that there had been lit¬ 
tle scientific research on the fundamental physiological 
mechanisms that enable dogs to locate sources of ex¬ 
plosive vapor. Dogs had been able to find mines using 
explosives (such as high melting point explosives like 
HMX) with vapor pressure far below measurable detec¬ 
tion thresholds. The mechanism by which TNT vapor 
and its breakdown products reach the ground surface 
was the subject of considerable scientific debate. By 
2003 a systematic trial in Afghanistan, scientific studies 
at SANDIA Laboratories in the USA and in Scan¬ 
dinavia, explosive trace detection studies with dogs 
at Auburn University, and several other investigations 
provided some insight into this problem for the first 
time [58.9]. However, the precise physiological mech¬ 
anisms for canine explosive detection remain unclear, 
especially for lower-vapor-pressure explosives. We still 
do not know for sure whether dogs are reacting to va¬ 
por, minute particles of explosive suspended in the air, 
biochemical breakdown products, or a combination. 

In 2003 a US company, NOMADICS, demonstrated 
the FIDO detector, the first that could reliably measure 
the presence of TNT vapor with greater sensitivity than 
a highly trained dog. However field trials showed that 
TNT vapor could be detected everywhere in a mine con¬ 
taminated area. An explosive vapor sensor was just the 
beginning of the story and warns of a complex cognitive 
task ahead in using the patterns of variation to localize 
targets. 

By 2004 the international community realized that 
the early confidence in a breakthrough resulting from 
advanced sensor technology, demining machinery, and 
mine detection dogs had been misplaced. GICHD com¬ 
missioned a detailed study of manual demining to see 
whether productivity improvements could be made. 
A systematic series of trials were conducted in Africa to 
determine the effectiveness of several innovations such 
as magnets and rakes. The final report, issued in 2005, 
revealed that greatly improved productivity was possi¬ 
ble but would depend more on improving contracting 
arrangements, management, and training than technol¬ 
ogy- 

The New York attacks in September 2001 funda¬ 
mentally changed research priorities. Removing unex¬ 
ploded ordnance, particularly cluster bombs, became 
the top priority for the next 12 months in Afghanistan 
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(Fig. 58.7). Since then the deteriorating security and 
political situation in Afghanistan has focused mine 
clearance agencies more on maintaining security for 
their own workforce than trying to improve productivity 
and safety. 

Resistance to the US and international occupation 
of Iraq and the easy availability of explosives both from 
former Iraqi armed forces and unexploded ordnance 
from US military operations led to the proliferation 
of improvised explosive devices (IED) to attack or¬ 
ganized military forces and police. These have now 
become the main threat and the focus for much of the 
funding, and operational and research expertise for¬ 
merly available to support mine clearance operations. 
This development has also placed ordnance disposal 
teams at the front line for the first time, rather than 
working in well-protected secure areas. Iraqi insurgent 
groups attack ordnance disposal teams both because 
they are attempting to disarm some of the insurgents’ 
most effective weapons and also because they remove 
the main sources of explosives available to insurgent 
groups. 

Improvised explosive devices, when detected, are 
often investigated and neutralized using remotely op¬ 
erated robots. While there are non-destructive methods 
to neutralize IEDs, the fastest method usually involves 
placing a small shaped charge device close to, and ac¬ 
curately aligned with the device. Operational details 
remain confidential to reduce the risk that IEDs will be 
modified to defeat current neutralization methods. 

Paradoxically it is this development that has enabled 
robotics to make a greater contribution to the problem 
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Fig. 58.7 Unexploded BLU-97 cluster bombs in Afgha¬ 
nistan early in 2002. Two of the small yellow canisters with 
parachutes still attached lie visible in the foreground. Oth¬ 
ers lie in the houses in the distance. Some may lie up to 
40 cm below the ground surface. Some can detonate when 
a metal detector or mobile phone is used nearby. These 
devices have a kill radius of 200 m and can sometimes 
be set off by a strong gust of wind. They readily attract 
the curiosity of children (courtesy of G. Zahachewsky and 
N. Spencer) 


by through improvements in remote manipulation tech¬ 
nology. These improvements come more in the form of 
low-cost commercial off-the-shelf components (mobile 
platform, motors, TV cameras etc.) than from funda¬ 
mental research advances. Improvements are still being 
made: improved remote manipulation, blast surviv¬ 
ability, operator interface improvements, and mobility 
improvements have all contributed significantly to per¬ 
formance and reduced operating costs. 

Advances in Demining Robotics Research 
In a brief survey it is not possible to mention every con¬ 
tribution. We have attempted to provide a sample of 
research reports that illustrate the main achievements 
and we present brief technical discussions on mobility 
and manipulation dexterity. 

Teleoperation remains the only robotics technology 
that has been used in practical application in field condi¬ 
tions [58.10-12], even with devices such as iRobot and 
machines such as Mine Wolf. Robotics research has not 
yet been able to make a significant contribution to mine 
and unexploded clearance work. However, the problems 
posed by landmine clearance have stimulated new re¬ 
search results that could have other applications. 

Robotics researchers started their efforts in the early 
1990s, for example, Havlik and Trevelyan indepen¬ 
dently proposed suspended cable robots to work in 
minefields [58.13]. However, both have later argued in 
favor of alternative solutions [58.10, 14]. 

Nicoud [58.15] wrote one of the first surveys ex¬ 
ploring the possibility of using robotics technology for 
landmine clearance. Developments in robotics research 
since the mid 1990s have been motivated both by a gen¬ 
uine desire to help combat a serious humanitarian issue 
and also by a desire to find a justification for more fun¬ 
damental research. However, most researchers still have 
not learned lessons from the field such as the need to re¬ 
move vegetation and the variety of situations in which 
deminers find themselves [58.16]. 

One of the most prolific research areas justified in 
part by humanitarian demining problems has been path 
planning for autonomous agents [58.17], Probabilistic 
approaches were explored by [58.18]. Some researchers 
have proposed multiple robot solutions, even swarms of 
robots [58.19] and so-called immune systems [58.20], 
Autonomous search and mapping algorithms have also 
been explored [58.21], including even three-dimen¬ 
sional (3-D) search techniques for locating underground 
chemical sources [58.22], 

Landmine clearance has also stimulated develop¬ 
ments in autonomous robot vehicles, with many ex¬ 
amples in the literature [58.23]. Tracked vehicles have 
been proposed for working in rubble and built environ¬ 
ments [58.24]. Walking vehicles have been proposed, 
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particularly for difficult terrain in countries like Bosnia 
and Afghanistan [58.25] even to the extent of exam¬ 
ining how damaged robots with missing legs could 
extricate themselves from a mined area [58.26]. 

The desire to keep human operators away from 
the risk of handling unexploded ordnance has stim¬ 
ulated research on artificial hands and telemanipula¬ 
tion [58.27]. Purely mechanical devices have also been 
explored [58.28]. Several researchers have provided de¬ 
tailed results of tests with manipulators mounted on 
autonomous vehicles [58.29] (Fig. 58.8). 

Robotics solutions have also been proposed partly 
to overcome the limitations of handheld sensors such as 
ground-penetrating radar. A robot manipulator can con¬ 
trol the motion of the sensor so much more precisely, 
opening the possibility of synthetic aperture techniques 
for both metal detection and radar [58.30]. 

Military research agencies in the United States, 
Australia, Britain, and Canada devoted large research 
budgets to the problem of road clearance since AT 
mines and IEDs accounted for a large proportion 
of military casualties in Somalia, Bosnia, Iraq and 
Afghanistan. Insurgent forces had demonstrated that 
they could bring organized military forces to a complete 
standstill overnight simply by laying anti-vehicle mines 
in a few road potholes. With most roads in poor con¬ 
dition there was no easy way to detect that the mines 
had been laid. Insurgents would typically use mines to 
stop leading vehicles in a convoy in order to increase 
the effectiveness of an ambush. 

Most research teams proposed one or more vehicles 
carrying multiple sensors including ground-penetrating 
radar, metal detector arrays, passive and active infrared, 
and even some acoustic arrays. Some sensor arrange¬ 
ments were designed to look forward sufficiently far to 
allow the carrier vehicle to stop before reaching a mine. 
Others were to be carried by lightweight remotely con¬ 
trolled lead vehicles. Military planners calculated that 
search speeds of ss 30 km/h would be required to be 
able to check roads daily in time for supply convoys 
to use the roads in daylight hours. Many different ar¬ 
rangements have been reported [58.31]. Other teams 
proposed teleoperated devices for landmine and ex¬ 
plosive ordnance detection and neutralization but few 
anticipated the requirement to operate under armed at¬ 
tack. Typical requirements envisaged secure rear area 
mine clearance in combat situations and peace-keep¬ 
ing on roads and tracks [58.32] or force protection 
roles [58.33]. Most of these efforts were initiated in 
the late 1990s but by 2005 it had become apparent 
to military planners that vehicle protection rather than 
mine detection was a more practical solution. Much 
of today’s vehicle protection technology originated in 
southern Africa with further development in Australia, 



Fig. 58.8 Gryphon experimental robot on trial in Cambo¬ 
dia (after [58.29] courtesy of S. Hirose) 


the UK, and other countries, stimulated in part by South 
African expatriates. 

One way to reduce the distance between researchers 
and field problems, at least in terms of geographic 
distance, has been to promote research in landmine- 
affected countries such as Sri Lanka and Colom¬ 
bia [58.34]. However this is not easy. Most countries 
affected by landmines have been disrupted by social 
conflict and destabilization that led to the military con¬ 
flicts in which landmines were used. This makes it 
difficult for local people to create sufficient economic 
and physical security for researchers to pursue their 
work. 

Future Prospects for Robotic Demining 
What are the challenges for robotics researchers work¬ 
ing on landmine clearance and other hazardous appli¬ 
cations in the future? 

We need further advances in mechatronics design, 
sensing, and accurate understanding of the problems to 
be solved using robots. 

The best starting point for research is to witness 
people undertaking hazardous work in several differ¬ 
ent situations. Nuclear accidents, mine disasters, and 
burning buildings are usually off-limits to researchers. 
However, mine clearance operations are readily ac¬ 
cessible in many countries. Though many researchers 
think a visit would be far too hazardous, there is no 
better way to appreciate the practical difficulties in¬ 
volved. Photographs taken at mine clearance operations 
are available to provide researchers with a web site for 
reference purposes, partly in answer to this need to un¬ 
derstand the practical realities [58.35]. 

One of the main motivations for robotics re¬ 
searchers has been the perception that mine clearance 
is a hazardous occupation and that it would be more 
preferable for robots to be exposed to minefield risks 
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than human beings [58.36]. While mine clearance is 
certainly a hazardous occupation it is not necessarily 
dangerous. Accident records show that mine clearance 
in Afghanistan in 1998 resulted in about half the rate of 
injury of the United States forestry industry and about 
one third the rate of injury for the United States build¬ 
ing construction industry per 100000 working hours. 
Mine clearance agencies use advanced techniques to 
improve safety when possible [58.37], In terms of 
deaths, demining is considerably less hazardous than 
mining, construction of building foundations, and es¬ 
pecially offshore drilling rigs [58.38, pp. 11-14], 

Another motivation for research is to reduce deaths 
and injuries among local people who have to live with 
the daily threat of landmines and unexploded ordnance. 
Again, there are misperceptions of risk. The incidence 
of death and injury from mine explosions is often very 
small compared with disease, for example. The main 
priorities for local people tend to be improvements for 
water and food supplies, education, sanitation and phys¬ 
ical security: landmine clearance is usually a much 
lower priority and it is often hard to justify significant 
local resources. 

It is also important that robotics researchers intend¬ 
ing to contribute to the solution of this problem under¬ 
stand the relatively small size of humanitarian demining 
operations, which have been funded from a combined 
international humanitarian aid budget of approximately 
US$400 million. These programs spend an estimated 
$ 20 million annually on all equipment needs. The mar¬ 
ket for specialized humanitarian demining detectors is 
therefore very small and manufacturers cannot afford 
research and development specifically to support hu¬ 
manitarian demining solutions [58.39]. Adapting tech¬ 
nology developed for other purposes, such as military 
equipment or civil engineering construction machinery, 
is more likely to be feasible. 

From 1990 to 2014 there were significant improve¬ 
ment in mine clearance techniques but overall progress 
is still slow and robotics may well provide the final 
solution in the long term. There are still opportuni¬ 
ties to develop robotic techniques that could ultimately 
provide the only cost-effective method for finally elim¬ 
inating this menace. 

58.2.2 Hazardous Materials Handling 
and Operations 

The oldest application of robotics-related technology to 
hazardous environments is various aspects of remote 
nuclear operations dating back to the beginning of se¬ 
rious work on atomic physics in the early 1940s. This 
section discusses hazardous materials and operations in 
the context of nuclear applications and some extrap¬ 


olations to other domains. These applications run the 
spectrum from low- to high-fidelity manipulation and 
multiple mobility modes. 

In the 1940s, research in atomic physics led to a new 
era in remote handling as scientists sought to explore 
the nature of materials involving ionizing radiation. As 
experiments became more complex, mechanical manip¬ 
ulator systems were created which allowed operators 
to perform increasingly complex tasks safely behind 
thick biological shielding I^JEDHESI, 

|4ScJtniSEIll. These mechanical sys¬ 
tems then evolved into electrical systems that allowed 
larger work volumes to be considered. Incredible en¬ 
gineering achievement occurred in a 15 year period 
within the Remote Control Division of the Argonne 
National Laboratory. Even though this era represented 
tremendous technical achievement, it went further in 
illustrating the intrinsic complexity of remote opera¬ 
tions. The equivalent work performance achieved with 
sophisticated teleoperated remote systems is poor in 
comparison to what human workers can achieve with 
direct contact operations and common tools. Typically, 
this form of teleoperation (i. e., manual control over 
a physical distance or barrier) is ten to hundreds of 
times slower than the equivalent operations performed 
directly by a human. Remote operations are extremely 
expensive and time consuming and have been the con¬ 
tinual target of engineering improvements over the 
years. 

Many research and development efforts have fo¬ 
cused on different avenues for improving the work effi¬ 
ciency of teleoperated remote operations l<s>M3El$EEl. 
These efforts have included the development of better 
manipulators, control stations, control algorithms, etc., 
all intended to enhance reliability and maintainability. 
In the late 1960s and early 1970s, as digital electronics 
became more cost effective, interest began to emerge 
in the integration of automation with teleoperation as 
a scheme to effectively increase remote operations work 
efficiency. It was around this time that industrial robot 
concepts were also introduced. Combining selective 
automation of specific subtasks with traditional teleop¬ 
eration offers the potential to reduce labor requirements 
and to improve the quality of repetitive task executions. 
This integration of automation with teleoperation be¬ 
came the foundation of what is now termed telerobotics. 
From the 1970s until today, telerobotics has been an 
active area of research and development in many dif¬ 
ferent domains that include nuclear, space, and military 
applications. Unlike manufacturing automation, remote 
operations in hazardous and unstructured work task en¬ 
vironments necessitate human-in-the-loop control, or 
teleoperation, at least as a backup, to assure safe op¬ 
erations. Teleoperation related to hazardous materials 
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and operations, in the most general sense, involve mo¬ 
bility and the use of manipulators to handle objects and 
tools to accomplish useful work in predominantly un¬ 
structured and uncertain environments. 

Over the years, there have been numerous papers, 
books, and reports written concerning technical chal¬ 
lenges, issues, and solutions. Readers will find Vertut 
and Coiffet [58.40] and Slutski [58.41] provide com¬ 
prehensive and general information about manipulators 
and systems The performance and design 

of effective teleoperated systems are strong functions 
of human factors and Kraiss [58.42] and Johnsen and 
Corliss [58.43] provide good discussions of key princi¬ 
ples. Finally, control system hierarchies and structures 
are explained in depth by Sheridan [58.44]. A final 
emphasis to all readers would be to fully explore the 
annals of the American Nuclear Society’s Remote Sys¬ 
tems Division, later renamed Robotics and Remote 
Systems Division (RRSD). Over the decades, this divi¬ 
sion hosted numerous topical meetings and workshops 
on all aspects of remote operations. A starting point is 
http://rrsd.ans.org/pages/topicals.html. 

Past Perspectives 

Initially, science and experiments involving dangerous 
radiation levels were accomplished through the inno¬ 
vative use of shielding walls, long-handled tools, and 
mirrors. As the tasks became more difficult, mechanical 
arms that could do a better job in emulating human mo¬ 
tions were pursued. These ideas represented schemes 
whereby human sensing and handling capabilities could 
be more completely projected into the remote work 
environment [58.45]. One of the first challenges that 
the Argonne group tackled was the development of 
the mechanical master-slave manipulator. The basic 
concept was to create a mechanism that would have 
a master controller side where an operator could pro¬ 
vide position and orientation commands to a slave- 
side mechanism/linkage system that would replicate 
motions and forces in the remote work area. It was 
felt that force reflection to and from the master and 
slave systems was essential for the operator’s sensory 
awareness of the task execution. Experimentation has 
repeatedly verified the significance of both kinesthetic 
and tactile feedback in performing more complicated 
tasks [58.46]. 

Today, master-slave manipulator (MSMs) are used 
around the world in nuclear, biological, and other types 
of hazardous remote experimentation and operations. 
(They have also become increasingly used for so-called 
robotic surgery.) The remote work efficiency of a dual¬ 
arm MSM with shielded-window viewing is around 5— 
10 times slower than equivalent direct human contact 
operations. Because the master and slave sections of 


MSM’s are mechanically coupled through the metal- 
tape drive transmission, the physical separation that 
can exist between the safe operating area and the haz¬ 
ardous remote work area is limited to a maximum 
of Ri 10 m. Because of this characteristic and their kine¬ 
matics, MSMs are restrictive in many applications and 
often have constrained the physical design of remote 
cells. The Argonne group and others recognized that it 
would be much better to have the equivalent of a fly¬ 
by-wire MSM in which the physical separation of the 
master and slave could be larger. This need led to the 
development of electrical master-slave manipulators 
(EMSs) that are commonly called electrical servoma- 
nipulators [58.47,48]. Research and Development on 
the EMS began in the late 1940s and continued into the 
early 1950s. 

The Argonne group was at that time limited by 
the available electrical control technology. Nonethe¬ 
less, they made noteworthy progress toward integrated 
systems, as depicted in Fig. 58.9. The prototype sys¬ 
tem shown is a dual-arm anthropomorphic system with 
head-aiming remote television viewing and bilateral 
force reflection. After the Argonne Remote Control 
Division was disbanded, limited research and devel¬ 
opment occurred until the 1970s when commercial 
nuclear power growth was driving a number of re¬ 
search programs in the US, West Germany, France, 
and Japan. During this time, the programs in the US 
and France were focused on electrical servomanipu- 
lator systems which incorporated emerging micropro¬ 
cessor technology. The Central Research Laboratories 
model M2 system, shown in Fig. 58.10, was jointly 
developed with the Oak Ridge National Laboratory 
and was the first force reflecting servomanipulator sys¬ 
tem to use distributed digital electronics to implement 
position-position force reflection with multiplexed se¬ 
rial communications between the master and slave. The 



Fig. 58.9 An integrated electrical master-slave manipula¬ 
tor system (courtesy of Oak Ridge National Laboratory) 
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Fig. 58.10 CRL Model M2 manipulator system perform¬ 
ing space truss assembly (courtesy of Oak Ridge National 
Laboratory) 


model M2 system was used over the years to explore 
remote operations for military, space, and nuclear ap¬ 
plications. 

The development of the advanced servomanipula- 
tor (ASM) followed the M2 in an effort to improve the 
remote maintainability of the manipulators themselves. 
This work was one of the earliest modular robotics ef¬ 
forts. The motivation for this work was to reduce main¬ 
tenance technician radiation exposure and to increase 
the overall availability of the remote maintenance sys¬ 
tem. The ASM was designed from the beginning to 
provide a foundation for telerobotics in addition to ef¬ 
fective teleoperation [58.49] (Fig. 58.11). 

At the time of the M2 and ASM developments, 
Jean Vertut and his colleagues in the Commissariat a 
l’Energie Atomique (CEA) focused their research on 
the development of telerobotic functionality for their 
MA-23 electrical servomanipulator systems. This re¬ 
search appears to include some of the earliest experi¬ 
mental demonstrations of telerobotic functions [58.40]. 
They called their concept computer-assisted teleopera¬ 
tion and it included both operator assists and robotic 
teach/playback functions. Operator assists included 
software jigs and fixtures designed for the improve- 



Fig. 58.11 Advanced servomanipulator system (courtesy 
of Oak Ridge National Laboratory) 


ment of the remote operation of tools such as saws, 
drills, etc. 

In the 1980s and 1990s as nuclear power activities 
began to decline, nuclear remote operations technol¬ 
ogy was migrated into other areas such as space and 
the military. Nuclear remote operations experience with 
teleoperation influenced the Space Shuttle remote ma¬ 
nipulator system and the short-lived Flight Telerobotic 
Servicer program. 

In the mid 1990s, substantial interest developed in 
the application of remote handling systems to deal with 
problems in hazardous site/facility remediation, such as 
those associated with defunct nuclear facilities in the 
US and in Eastern Europe. The intrinsic complexity 
of remote clean up operations continues to drive re¬ 
search and development in all aspects of teleoperated 
and telerobotic systems. Challenges and accomplish¬ 
ments in this application area are discussed in the next 
section. 

Hazardous Site Clean Up Applications 
Robotic and remote systems have been used in assess¬ 
ing the status of sites contaminated with hazardous 















Robotics in Hazardous Applications I 58.2 Applications 1533 


materials. These surveys are essential in planning and 
executing subsequent clean up operations. Such sys¬ 
tems have been used most extensively in nuclear waste 
sites around the world because remote techniques are 
more common in many nuclear operations. There has 
been some use of robotic survey systems in chemical 
and biological hazard situations as well. 

Survey Systems. Numerous robotic survey systems 
have been developed and used. The basic idea is to inte¬ 
grate a sensor suite, appropriate for the contaminants of 
interest, with a suitable mobile platform that incorpo¬ 
rates remote and/or autonomous driving functions and 
requisite navigation and control functions. The desired 
output of the survey process is a precise map of contam¬ 
inant locations and concentrations. Such systems have 
been developed for both outside and inside operations. 

It was common practice in many industries for 
a number of decades to bury hazardous wastes in 
earthen trenches in isolated burial sites l<£aM2EB3EH, 
|43>M3EE!EEi, loSEEHiKTl. Usually, useful records of 
what materials were buried at what locations either did 
not exist or were not accurate. In fact, the general con¬ 
ditions of such buried waste sites are often unknown 
to the extent that human entrance is not allowed. As 
a result, the first step toward remediation is to quan¬ 
titatively assess the physical and hazard conditions of 
the site. The mobile robot shown in Fig. 58.12 is an 
example of a survey system used to evaluate nuclear 
waste buried waste sites. The robot’s location is moni¬ 
tored with Global Positioning System (GPS). The suite 
of sensors includes eddy-current probes and ground- 
penetrating radar that reveal density contours, radia¬ 
tion detectors, and gas emissions monitors. This system 
was an initial prototype that could be operated from 
a remote driving station or operated in autonomous pro¬ 
grammed trajectory mode. The system was controlled 
using a radio communication link. Its unique feature is 
that it was designed with minimal use of ferromagnetic 
materials to minimize interference with the magnetic 
subsurface sensors. 

The mobile characterization system shown in 
Fig. 58.13 is a similar concept for radiation survey 
of floored areas. This particular system uses a Cyber¬ 
motion commercial mobile platform, triangulation of 
optical fiducial markers for localization, and operates 
primarily in a full autonomous mode. The objective in 
this application is to reduce labor costs and eliminate 
error-prone tedious human operations. 

Excavation Systems. The actual remediation of 
buried wastes ultimately leads to digging and object¬ 
handling type operations. Various types of systems in- 



Fig. 58.12 Prototype buried waste survey robot (courtesy 
of Los Alamos National Laboratory) 



Fig. 58.13 Mobile characterization system (courtesy of 
Oak Ridge National Laboratory) 


volving integrated mobility and manipulation-like func¬ 
tions have been developed. One of the most popular and 
low-cost approaches has been to retrofit conventional 
excavation equipment with sensor and actuators that al¬ 
low remote and robotics operations. Such systems have 
been used in explosive ordnance disposal also. The sys¬ 
tem shown in Fig. 58.14 is called the teleoperated small 
emplacement excavator (TSEE) and is a prototype re¬ 
motely operable backhoe for such applications. The unit 
included a multi-camera remote viewing system that is 
the primary basis for teleoperation. Radio and tether 
communications connections are used to allow the hu¬ 
man operators to be displaced up to several kilometers 
from the hazardous operations. 

Deactivation and Decommissioning Systems. In 
recent years, one of the most complex forms of haz¬ 
ardous operations is the deactivation and decommis¬ 
sioning (D&D) of defunct facilities where nuclear ra¬ 
diation or toxicity hazards preclude human presence. 
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Fig. 58.14 Remotely operated excavator (courtesy of Oak Ridge National Laboratory) 


D&D can be thought of as remote demolition for the 
most part. Some operations are crude such as knock¬ 
ing down building structures and debris removal. Other 
operations may involve careful disassembly of equip¬ 
ment and devices, size reduction and packaging of 
handling/storage. These operations are essentially the 
inverse of remote maintenance and require the dex¬ 
terous use of tools and handling of objects. Tools 
include saws, hydraulic shears, impact wrenches, and 
the like. Planning and situational awareness are very 
important in demolition-type operations because the 
physical layout and stability of the basic environment is 
changing during the process. Effective remote viewing 
(adjustable multiview capabilities) and acoustic sensing 
are essential. Audio feedback from the remote envi¬ 
ronment provides the human operator with the ability 
to monitor the normalcy of tooling operations such as 
sawing. D&D applications require manipulators that are 
both dexterous and massive. Experience has shown that 
payload capacities on the order of 100 kg are needed 
to deal with debris handling, typically with manipula¬ 
tors of the type shown in Fig. 58.15. Figure 58.16 is 
a frame view from one of the remote viewing cameras 
used to operate a similar system that was used to D&D 
an old experimental nuclear reactor. A manipulator is 
being used to teleoperate a conventional circular saw to 
segment a large-diameter aluminum tank. This operator 
view of the remote environment provides a realistic per¬ 
ception of the task environment lighting conditions and 
complexity. 

In recent years, numerous remote manipulation sys¬ 
tems for nuclear applications have been developed 
around industrial robots. These systems are augmented 
with sensors and master controller arrangements to fa¬ 
cilitate teleoperation. This approach offers significant 
cost savings and the level of teleoperation performance 
achievable in complex task environments has steadily 
increased [58.50-54]. 


Fig. 58.16 Operator’s view of a remote sawing task (cour¬ 
tesy of Oak Ridge National Laboratory) 

Applications in Large Scale 
Science Experiments 

High radiation levels resulting from materials activation 
in the target areas of high-energy particle accelera- 



Fig. 58.15 Dual-arm D&D manipulator system (courtesy 
of Oak Ridge National Laboratory) 
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Fig. 58.17 SNS Telerob EMSM-2B Dual arm servomani- 
plator 



Fig. 58.18 Telerob EMSM-2B master control station 


tor experiments often require remote operations. Since 
these are unique scientific experiments, they involve 
complex and specialized equipment that results in some 
of the most challenging remote operations in the world. 
Nonetheless these highly specialized remote operations 
still involve the basic elements of remote operations 
summarized in Fig. 58.1. The most recent example of 
this class of remote systems is the comprehensive re¬ 
mote handling system developed as an integral part of 
the Spallation Neutron Source (SNS) located at the Oak 
Ridge National Laboratory [58.55]. 

The SNS is a neutron scattering facility in which 
uniquely high-flux neutron beams are generated by im¬ 
pinging high-energy protons on to a mercury target. 
The mercury is activated to the extent that the contact 
dose rate in the associated piping and vessels can be 
over 10 5 R/h and are long lived. The design and op¬ 
eration of this target system are such that it must be 
periodically replaced or refurbished. The mercury tar- 



Fig. 58.19 SNS remote handling control room 



Fig. 58.20 SNS pedestal manipulator system 


get system and its support infrastructure are located in 
a shielded hot cell that includes a comprehensive re¬ 
mote handling system. The hot cell remote handling 
system is comprised of combinations of conventional 
mechanical master/slave manipulators, overhead cranes 
and a state of the art dual arm telerobot. Details of the 
overall system are provided in [58.55]. The focus of this 
discussion will be the telerobotic system. 

Figures 58.17 and 58.18 show a dual arm force re¬ 
flecting servomanipulator system, Telerob EMSM-2B, 
that is mobile in 3-D throughout the hot cell by virtue 
of a overhead bridge ( X,Y motions) and rigid telescop¬ 
ing tube (Z motion). The manipulators are 6 dof and 
each have a 245 N continuous capacity and 445 N peak 
capacity. A 2200 N auxilary hoist is integrated into the 
bridge trolley to facilitate handling heavy objects with 
the telerobot. The bridge system can be teleoperated or 
robotically pre-programmed. A second dual arm servo- 
manipulator package (Fig. 58.20) is part of a mobile 
pedestal mounted system that is specifically designed 
in the high bay area over the target hot cell. The con¬ 
trol room. Fig. 58.19 includes 21 flat panel displays 
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Fig. 58.21 Fukushima Daiichi 
Nuclear Plant after tsunami had 
inundated entire site to depth of 
over 13 m and subsequent hydrogen 
explosions, (courtesy of Japan Atomic 
Energy Agency) 


that can be used with any of the 11 remote cameras lo¬ 
cated in the hot cell. The telerobot operator is located 
at the far end of the room and can operate either stand¬ 
ing or sitting. This overall system is functionally similar 
in concept to advanced servomanipulator systems dis¬ 
cussed in the earlier sections. The system has been in 
operation since 2006 and has been successfully used in 
several major mercury target diagnostic/refurbishment 
campaigns. The SNS remote system has shown that the 
electromechanical equivalent of traditional MSM’s with 
full remote hot cell mobility is practical and reliable. 
Larger scale nuclear facilities involving higher level of 
radiation are certain to follow the remote handling phi¬ 
losophy represented by this system. Because this class 
of electrical servomanipulators have all of the real-time 
digital control features of modem robots, greater use of 
programmed and intelligent control concepts for certain 
types of remote tasks is expected in future. 

58.2.3 Lessons Learned from the 
Fukushima Daiichi Disaster 

Japan has not only been at the forefront in the devel¬ 
opment of robots to help in disaster situations, but has 
also, sadly, experienced a major recent disaster of the 
kind anticipated by robotics researchers and engineers 
for decades previously. This disaster provided a real¬ 
ity check for researchers and exposed just how far we 
still need to progress before robots can be relied upon 
to perform routinely in hazardous environments. 

The March 2011 tsunami precipitated the Fuku¬ 
shima Daiichi nuclear reactor disaster (Figs. 58.21 
and 58.22). Even though the reactors were shut down 
at the time, they still required significant cooling wa¬ 
ter supplies as they had been only recently shut down. 



Fig. 58.22 Underwater reconnaissance robot being eval¬ 
uated in search for human remains following the March 
tsunami, Japan, (courtesy of Prof. Shigeo Hirose) 


The tsunami destroyed the diesel back-up generators re¬ 
quired to keep the cooling water pumps operating. The 
resulting temperature rise in the reactors led to gas ex¬ 
plosions that severely damaged the reactors, their cool¬ 
ing systems and the containment buildings, spreading 
radioactive contamination over a large area of nearby 
land and ocean. 

As Kawatsuma and his colleagues explained, many 
different kinds of robots had been developed as a result 
of earlier research and development programs [58.56], 
Unfortunately, at the time of the disaster most of these 
robots were either inoperative, unable to be repaired 
and made ready, or unsuitable because of the need 
for tether cables, or reliance on Wi-Fi communica¬ 
tions which could not be operated satisfactorily in the 
contaminated nuclear environment (Chap. 60.4). Sev¬ 
eral teleoperated construction machines developed for 
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the 1991 Mt. Fugen volcanic disaster were success¬ 
fully deployed to clear contaminated wreckage and 
debris Robots originally developed for 

reconnaissance work, primarily investigation of unex¬ 
ploded ordnance, were supplied by British and Ameri¬ 
can sources l<s MBa , 

However only limited operations could be completed 
with these robots as they could not reach many of the 
places where they were needed. In the first months 
following the disaster, only one robot, the Japanese de¬ 
signed Quince from Chiba Institute of Technology, was 
successfully deployed inside the reactor buildings for 
several weeks before it failed with a wiring fault. At the 
time of writing, it had not yet been recovered for repairs 

Since then more robots 
have been successfully deployed to the site, though still 
with limited operational capabilities. 

The disaster exposed the need for prior planning and 
continuous training, rehearsal and cooperation between 
research and development agencies, defence disaster 
relief agencies, robotic systems manufacturers and en¬ 
gineers at the hazardous facilities where the robots 


58.3 Enabling Technologies 

There are a number of technology issues that are key 
to the design and implementation of mobile robots that 
can operate effectively in different types of hazardous 
environments. A few of them are touched upon here to 
give the reader additional insight into the foundational 
aspects of this class of robots. 

58.3.1 Mobility Issues 

Hazardous applications such as fire fighting, EOD, 
and demining require the ability to operate on un¬ 
even ground in indoor and outdoor environments 
where a normal wheel-type robot cannot easily op¬ 
erate. Most mobile mechanisms are classified into 
wheel type [58.57-60], track type [58.61-66], and leg 
type [58.67]. For negotiability in uneven ground, the 
wheel and track types need additional linkages with 
ground adaptation. There are two kinds of this adap¬ 
tation: active and passive adaptation. Active adaptation 
uses an additional actuator to alter the linkage’s motion 
while in passive ones the linkage motion is controlled 
by the ground conditions and gravity effects. 

Stability Conditions 

Stability should be considered in the design process to 
prevent rollover on uneven ground including stairways, 
steps, and natural terrain. Stability can be investigated 


would be used. Rather than individual robots, systems 
of which robots form a part need to be developed, 
including shielded vehicles to allow close-up moni¬ 
toring and control by human operators 
Continuous updating of robot systems is needed: some 
robots that could have been deployed relied on ob¬ 
solete electronic and computer components that could 
not be replaced. Following this disaster, we can expect 
a significantly increased effort to deploy robots over the 
coming years to deal with the consequences of similar 
events. 

The Fukushima Daiichi disaster and other opera¬ 
tions following the March 2011 tsunami demonstrated 
that, as in the other applications discussed above, robots 
can still only be expected to perform a niche role. 
Robots have only performed consistently in such en¬ 
vironments after decades of development, and invest¬ 
ment in developing trained operators with appropri¬ 
ate maintenance and support facilities. Our ability to 
realistically predict the capabilities of robotic opera¬ 
tions in unforeseen hazardous environments is still very 
limited. 


in terms of three parameters: center of mass, support¬ 
ing area, and stability margin. Static stability requires 
that the center of mass remains inside the supporting 
area, as shown in Fig. 58.23. The supporting area is 
a polygon built by the edges connecting each contact 
point on the horizontal plane. A conventional vehicle 
has limitations in rugged terrain due to a fixed center of 
mass with respect to the body coordinates. The center 
of mass greatly affects the stability margin (the mini¬ 
mum length between the center of mass and edges of 
a supporting area). If the center of mass is located out 
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a) Deformable wheels 



Fig. 58.24 Wheel configurations 



Fig.58.25a,b Mobile robots with active adaptation mechanisms: 
(a) Packbot (after [58.68]), (b) Andros Mark V-Al (after [58.64]) 


of the supporting area, rollover can occur. The stability 
margin of a conventional vehicle is mainly determined 
by the terrain inclination. Many vehicles are designed 
to have a low center of mass, thereby obtaining a large 
stability margin on a slope. Furthermore, a vehicle is of¬ 
ten designed to have multiple bodies to overcome this 
limitation. For a multiple-link mechanism, the center of 
mass varies and the relative motion of links when trav¬ 
eling over the landform also alters the supporting area. 

Active Adaptation Mechanisms 
An active adaptation mechanism uses an additional ac¬ 
tuator to generate motion of the adaptation linkage 
on rugged ground. Although additional actuators and 


linkages are required, the adaptation mechanism can 
overcome variously shaped rugged landforms. 

It is known that a vehicle can negotiate an obsta¬ 
cle which is smaller than the wheel radius. One may 
think that a vehicle which has large wheels can pass 
over rough terrain like a monster truck. However, most 
applications require a small vehicle to drive between 
obstacles. 

A deformable wheel or its equivalent mechanism 
can be adopted: insufficiently inflated wheels, as shown 
in Fig. 58.24a, adapt to irregular terrain by deformation 
of wheels. However, their size should be big enough 
to negotiate rough terrain, so this concept still has size 
limitations. Their equivalent mechanisms are relatively 
compact and yet they need sophisticated articulation 
mechanisms, as shown in Fig. 58.24b. Two wheels are 
attached to a link that rotates about a main body. There¬ 
fore, revolution and rotation of wheels are made as 
shown in Fig. 58.24b. If rotation is provided by actu¬ 
ators at the main body (i. e., an active mechanism), the 
vehicle has an equivalent wheel whose radius is equal 
to that of the rotation. The vehicle in Fig. 58.24b can 
pass a step by carriage rotation, although the radius of 
a wheel is smaller than the height of the step. 

Many track-type mechanisms have multiple link¬ 
age structures with active adaptation [58.63-66,68]. 
When traveling over uneven ground, these mechanisms 
can negotiate rugged ground by changing the config¬ 
uration of multiple track mechanisms, for example, 
typical multiple-tracked robots with active adaptation 
such as Andros [58.64] and Packbot [58.68], as shown 
in Fig. 58.25a and 58.25b, commonly uses a small ac¬ 
tive additional track called a flipper. When moving on 
relatively even ground the robot can reduce undesired 
track resistance by lifting up the flippers. When climb¬ 
ing up stairs, touching the flippers down to the ground 
can increase the stability by enlarging the support¬ 
ing area. Snake-like robots use active interconnecting 
joints [58.65] and have the potential for mobility and 
manipulation. 

Passive Mechanisms 

Passive adaptation mechanisms do not use any addi¬ 
tional actuators; rather they simply utilize the gravity 
effect to generate adaptation to irregular ground. This 
approach is less adaptive but enables the operator to 
drive the robot easily because he (or she) does not 
have to take care of controlling the adaptation mech¬ 
anism. Passive mechanisms usually include various 
kinds of passive linkages activated by irregular ground 
conditions. 

For the wheeled vehicle shown in Fig. 58.24b, 
a passive mechanism can be designed in which no addi¬ 
tional actuators are needed, for example, when a wheel 
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Fig.58.26a,b Mobile platforms with passive adaptation: 
(a) Solero (after [58.60]) and (b) Robhaz-dt3 


meets a sill whose height is greater than the radius 
of a wheel, it will become stuck. Then high torque is 
exerted at the wheel since the stall torque is usually 
greater than the normal rotation torque of the wheel. 
In this case, a closed linkage without any actuator ex¬ 
cept the driving wheels can generate a passive motion to 
climb over a step [58.58], for example, a smart mobile 
robot with passive adaptation called Solero (depicted in 
Fig. 58.26a) can climb up an obstacle by lowering the 
instant center of rotation of the passive four-bar linkage 
when coming into contact with the vertical surface of 
the obstacle [58.60]. 

Similarly, multiple bodies with tracked and chained 
passive joints can also adapt to irregular terrain. The 
track type is inherently insensitive to the unevenness 
of ground I4S&MM5EE1. Thus passively chained mul¬ 
tiple-tracked bodies can provide greater reliability and 
better capabilities in ground adaptation. For example, 
a mobile robot with a passively chained double-tracked 
mechanism called Robhaz (shown in Fig. 58.26b) is 
a practical design with simply adapted multiple tracked 
bodies with passive joints [58.66]. As depicted in 



Fig.58.27a,b Passive adaptation of double-tracked mech¬ 
anism: (a) positive deflection and (b) negative deflection 

Fig. 58.26, two tracks of the Robhaz rotate positively 
or and negatively according to the contact situation 
between the track and the stair surface. Based on 
this behavior, in terms of the lower center of mass 
(Fig. 58.27a) and the supporting area (Fig. 58.27b) 
better stability in stair climbing than that of a single- 
tracked vehicle can be achieved. 

58.3.2 Manipulator Design and Control 
for Hazardous Object Handling 

Manipulation is essential for EOD missions. Gener¬ 
ally, EOD missions have two phases: (1) approach, and 
(2) object manipulation to achieve ordnance neutral¬ 
ization. The second phase is generally controlled by 
teleoperation. Both the human’s careful control and in¬ 
nate intelligence, plus appropriate manipulator design 
and control, are needed for such dangerous tasks. 

Design Requirements 

Unlike industrial manipulators, the design of manipula¬ 
tors for many robots in hazardous environments does 
not require very fast movement. Instead, high pay- 
load, light weight, and compactness are more important. 
Therefore, many manipulator designers locate actua¬ 
tors and reducers, which typically have high weight 
and large volume, on the base or lower links. This re¬ 
duces the weight of the moving parts of the manipulator 
and offers better stability and dynamic control proper¬ 
ties. Consequently, they have a thin profile, as shown 
in Fig. 58. 28a, b. This characteristic is usually associ¬ 
ated with explosive ordnance disposal-type robots that 
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Fig.58.28a,b Hazardous application robots with manip¬ 
ulator: (a) Packbot with manipulator and (b) Robhaz-dt2 
with manipulator 


require access to congested areas and normal human 
living spaces. Mobile robots used in other hazardous 
environments can be much larger, depending on the spe¬ 
cific applications. 

58.3.3 Control for Hazardous Tasks 

In most cases, the operator does not have much in¬ 
formation about the environment around the robot and 
the manipulator. He may have very limited vision, and 
unexpected collisions between the manipulator and ob¬ 
stacles can easily happen, even if he (or she) tries to 
be very careful. The robot should be able to handle 
these risks by appropriate design of the manipulator 
and good control algorithms. There are several ways to 
solve these problems. 

Proximity sensors that detect nearby objects can 
be used to avoid a collision before it occurs [58.69]. 
However using many proximity sensors necessitates 
large wiring bundles and complex manipulator design. 
Force control is also a good way to handle collisions. 


When a collision occurs, the manipulator can move 
away from the collision point by using force/torque 
sensor data. Often a six-axis force sensor attached at 
the end-effector of the manipulator is used for the 
force control [58.70]. Many researchers have devel¬ 
oped stiffness/compliance/impedance control methods 
for a manipulator with a force sensor. However, this 
method can only handle a limited collision zone - only 
the end-effector or the manipulator hand - because the 
sensor cannot sense contact with intermediate links, 
while a collision could occur at any point on the ma¬ 
nipulator. For this reason, some manipulators use joint 
torque sensors at all of their joints [58.71], When us¬ 
ing a joint torque sensor, contact at any point on the 
manipulator can be sensed, however, gravity compen¬ 
sation and errors in the transformation from the joints 
to the Cartesian space introduce further difficulties. 

The stability of telerobotic systems, particularly 
when force feedback or reflection is involved, remains 
a critical issue. Systems involving significant data com¬ 
munication time delays are even more difficult. Na¬ 
tional Aeronautics and Space Administration (NASA) 
groups have studied stability and bandwidth issues nu¬ 
merous times over the years; one of the more recent 
studies is provided by Uebel et al. [58.72]. Energy 
methods have been applied to this problem showing 
promise for enhanced stability and performance [58.4, 
73], 

Master Controllers 
for Teleoperated Manipulation 
In teleoperation, the robot is designed to be a faithful 
slave to deal with a dangerous task while the operator 
uses a control interface to direct the slave from a safe 
location. The user interface usually provides a means 
for: 

• Sending position commands to the manipulator. 

• Providing contact force feedback or force reflection 

to the user. 

Ideally, the design goal of a user interface is to make 
it transparent, so the operator feels as if he or she is 
directly manipulating the object handled by the slave 
manipulator. To achieve this transparency, there are sev¬ 
eral design issues: 

• Simplicity: 

- All indicators are unified as one scene. 

- All input button and joystick are integrated into 
one haptic device. 

• Intuitiveness: 

- High-level command by speech recognition. 

- Human-friendly feedback such as graphical dis¬ 
plays and human voice. 
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Fig. 58.29 Appearance of wearable multimodal user interface 


Waist belt 



Base mechanism 

Prismatic joint using coupled 3 pieces of links 


Fig. 58.30 Picture of wearable haptic device 


- Motion command matching between the haptic 
device and the slave in Cartesian space. 

• Portability: 

- Small/lightweight and human-friendly design 
for wearability 

- Tetherless operation without communication 
lines and additional power. 

An example of wearable multimodal user inter¬ 
face is provided by [58.74], as shown in Fig. 58.29. 
The operator wears a head-mounted display (HMD), 
head tracker, and headset to interact with the slave. 
A six-degree-of-freedom haptic master is attached on 
his waist together with the standalone controller, and 
the operator grips its handle to telemanipulate in Carte¬ 
sian space. With wearable hardware, the user can walk 
around to gain a better view during teleoperation using 
wireless communication. It is composed of major three 
operator interfaces. 

Speech and Auditory Interfaces 
An operator usually sends two types of commands to 
the robot: selection and continuous motion commands. 
For example, the selection between mobile and manip¬ 
ulation mode, the reset of the robot arm and mobile 
base, the on/off and reset of pan-tilt motors, speed 
selection for the mobile platform, selection among in¬ 
stalled cameras are defined in the selection commands. 
Voice commands offer convenience. When the operator 
says a word that has been predefined as a command, 
the speech interface can sense that word and, if the 
speech recognition system successfully recognizes it, 
the recognized command pops up on the HMD for 
confirmation. Finally, the operator decides whether to 
execute or cancel the command with a confirmation but¬ 
ton on the haptic master. 

The auditory interface synthesizes the human voice 
using a speech synthesis engine. It can warn of the ap¬ 
proach of an obstacle by sound, or reveal the distance 
and direction to a laser-designated object nearby. 

Wearable Haptic Interfaces 
A wearable haptic device is shown in Fig. 58.29. The 
base linkage is designed as a serial RRP mechanism to 
measure a translation, and an RRR z—y—z rotation mech¬ 
anism is attached at the end of the base linkage. 

Figure 58.30 shows a wearable haptic master device 
for teleoperation of mobility and manipulation systems. 
It has six degrees of freedom for motion input and three 
degrees of freedom for force feedback. 

To achieve a compact design and reduce its weight, 
a tendon-driven mechanism is designed into each joint. 
Due to weight constraints, only three actuators are in¬ 
stalled for force feedback, and each actuator is specially 


designed to fit the joint. Because a passive actua¬ 
tor is better than an active actuator with respect to 
power density (power per unit volume or weight), small 
magnetorheological (MR) brakes have been developed. 
They are installed at each joint of the base linkage for 
force feedback. Also a compact brake drive with current 
feedback capability has been designed, which reduces 
the response time of the MR brake. The controller is 
packed into a bag that is attached to the back side of the 
waist. 

Visual Interfaces 

The visual interface integrates the robot’s view, the sta¬ 
tus of sensed data, and the status of speech commands. 
Since the slave includes a stereoscopic camera, the user 
sees a three-dimensional view on the head-mounted dis¬ 
play (HMD). The operator wears a head tracker and it 
generates a pan/tilt command from the two-degree-of- 
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freedom head motion. In the integrated system, head 
tracker data is used to command the direction of the re¬ 
mote viewing camera, thus the user easily looks around 
the environment of the robot, as shown in Fig. 58.31. 

Moreover, the head tracker is useful to indicate an 
object as a target. The operator moves his head and 
looks at a target to get information of direction and dis¬ 
tance, then triggers a laser displacement sensor that is 
placed in parallel with the pan-tilt camera. A virtual 
reality overlay is prepared with the reported data, as 
shown Fig. 58.31. The recognized speech command is 
highlighted on the left for confirmation. These are over¬ 
laid on the remote video source pictures and unified 
into a single scene. Finally the operator sees the stereo¬ 
scopic picture and the status of the robot at a glance 
on the immersive HMD. The overlaid view is shown in 
Fig. 58.31c. 

58.3.4 Data Communications 

Robots used in hazardous environments are almost 
always mobile so that they can move about an area of in¬ 
terest with flexibility. The combination of mobility and 
intrinsic operation across a physical or hazard barrier 
introduces unique problems with respect to bidirec¬ 
tional data communications (Fig. 58.1). Communica¬ 
tions from the operator location to the robot is necessary 
to achieve remote control or teleoperation. Communi¬ 


a) b) 



Fig.58.31a-c Appearance of integrated visual interface: (a) camera 
view (b) overlay source view (c) user view 


cations from the robot to the operator are essential for 
situational awareness. The transmission of the electrical 
and/or optical signals comprising such data communi¬ 
cations can be quite complex due to a host of factors 
such as penetration of physical barriers, signal transport 
delays, signal attenuation, and data throughput require¬ 
ments. Data throughput requirements are dominated by 
the feedback of visual imagery or remote camera views 
from the mobile system. Channel capacity necessary for 
audio feedback and control are small in comparison. If 
the mobile system involves precision manipulation with 
force feedback, bidirectional data channels on the order 
of 1 Mb/s are required. 

For a mobile robot with high-fidelity manipula¬ 
tion, multichannel remote viewing, and other sensors, 
on the order of hundreds of Mb/s are required for 
the data communications path from the remote envi¬ 
ronment to the operator station while on the order of 
ten Mb/ s are needed for the bidirectional control link. 
Data throughput requirements can be reduced through 
data compression hardware provided there is no signifi¬ 
cant increase in latency (time delay) which can degrade 
control by humans if the delay exceeds 100 ms. The 
choice of communication means is not easy: while wire¬ 
less communication seems attractive, when multiple 
robot systems need to cooperate in a shared space, the 
communication systems need to be designed with this 
in mind. All too often, communication systems have 
evolved to support only a single robot system within 
a given operating region. Tethered cables provide high 
bandwidth, but pose extreme difficulties in unstructured 
environments, and when multiple robot systems need to 
work together (Table 58.1). 

58.3.5 Energetics 

Another key challenge in fielding mobile robots in 
hazardous environments is in the area of energetics in¬ 
cluding power supply, consumption, conversion, and 
management. The specifics of the given environment 
often restrict the types of power supply/conversion that 
are permissible, for example, the use of combustible 
fuels and internal combustion engines is seldom al¬ 
lowed within coal mines, refineries, gas plants, offshore 
platforms and nuclear facilities but can be entirely ac¬ 
ceptable in demining and other outdoor operations. 
The most common power systems used for research 
robots are electrochemical batteries and electric mo¬ 
tor drives. The power and energy density map [58.75] 
given in Fig. 58.32 shows the fundamental situation 
for mobile robots used in hazardous environments. For 
perspective, a small passenger automobile is estimated 
to need a minimum of 200Wh/km to meet road de¬ 
mands, which in turn translates into 500Wh/kg. This 
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Table 58.1 Example data communications requirements (minimum) 


Type 

Single channel resolution 

Typical standard system 

Standard black-and-white remote television (typical 
for rad hard technology): 

600 x 400 pixels 

30frames/s 

12 bit gray scale 

% 10 Mb/s per viewing channel 

30—50 Mb/s for 3—5 channels 

Color remote television: 

red, blue, and green = 3 x black and white 

% 30 Mb/s per viewing channel 

90—150 Mb/s for 3—5 channels 

Color high definition (HD) 1080p television: 

1920 x 1080 pixels 

60 frames 

~ 1.5 Gb/s per viewing channel 
with encoding compression 
~ 3 Gb/s uncompressed 

5.5—15 Gb/s for 3—5 channels 

Control 

12 bit resolution input and output 

200 Hz sampling rate 

% 4.8 kb/s per control channel 

48 kb/s - ten control servo channels 

Audio feedback 

15 kHz signal capture 

12 bit resolution 

% 180 kb/s per audio channel 

540 kb/s - three remote micro¬ 
phones 


is why hybrid electric vehicles that combine spark ig¬ 
nition engines with battery-powered electric motors are 
becoming popular. A large mobile robot with dual ma¬ 
nipulators operating in the payload range of hundreds of 
kg would have power requirements similar to these and 
would be well above the 100—200Wh/kg range that 
batteries can provide. A force-reflecting six-degree-of- 
freedom robot manipulator with a 20 kg payload will 
have a peak power consumption on the order of 10 kW. 
If a particular application domain is amenable to the 
use of spark ignition engines, the order of magnitude 
of power density compared with any type of battery- 
powered system makes implementation feasible. If pure 
battery power is necessary, then considerable attention 

Power density (W/kg) 



Fig. 58.32 Power and energy density for various power 
storage and supply systems 


must be given to the design, including state of charge 
monitoring and mission provisions for recharging. In 
the future, emerging fuel-cell technology may provide 
better solutions for mobile robot power systems. 

58.3.6 System Architectures 

for Real-Time Mission Control 

Because of the intrinsic nature of hazardous environ¬ 
ments where uncertainty and lack of geometric struc¬ 
ture are prevalent in the task environment, shared con¬ 
trol architectures are commonly used. The degree of 
human interaction runs the gamut from pure teleoper¬ 
ation (manual control) to high-level intelligent control 
operations (autonomous). For example, most low-cost 
EOD type robots utilize simple manual control archi¬ 
tectures, while at the other end of the spectrum, the 
Mars rovers operate as autonomous agents responding 
to mission-level commands from human operators on 
the earth. The basic architectures for real-time mission 
control are summarized in Fig. 58.33 [58.76]. Also, 
see Sheridan for a comprehensive discussion of ar¬ 
chitecture principles and design [58.44]. Between the 
manual and autonomous control exist combinations of 
computer assistance and semiautonomous (i. e., selec¬ 
tive and in situ task automation) functions intended to 
improve remote work performance by either reducing 
operator workload or allocating tasks more suitable for 
computer control to autonomous execution. It is im¬ 
portant to recognize that, as one moves from manual 
control toward autonomous control, the data through¬ 
put rates for communication and control are reduced 
because high-level commands are comparatively small 
packets of information while force-reflection manual 
control of a manipulator, for example, requires visual 
feedback and high-bandwidth control interconnection 
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a) Direct manual 
control with no 
computer-aided 
(Type 1) 


b) Indirect manual 
control with 
computer-aided 
(Type 2) 

c) Parallel type 

d) Serial type 

e) Parallel and 
serial combined 

f) Autonomous 
(Type 6) 

Semi-autonomous control (Type 3-5) 



Fig. 58.33 System control architectures 


of the slave and master controller. The data communi¬ 
cation bandwidth available influences control architec¬ 
ture. Untethered undersea applications usually require 
semiautonomous architectures because of the band¬ 
width limitations of acoustic communications links. 

As intelligent systems research advances, it is ex¬ 
pected that, for hazardous situations, robots will em¬ 


ploy semiautonomous and autonomous architectures to 
a greater degree as a fundamental approach to over¬ 
all performance optimization. Operators will be able to 
choose the control mode based on the task needs and the 
likelihood that autonomous execution can be reliably 
achieved. In this human-centered way, task execution 
mode will be matched to best achievable performance. 


58.4 Conclusions and Further Reading 

If we are to learn anything from the experience of the 
last decade we must appreciate that robotics research 
is just one step in the development of tools that ex¬ 
tend human capabilities. This search for improved tools 
is as old as humanity itself, so great patience is re¬ 
quired [58.77], 

Researchers need to make significant advances on 
four different fronts: mechatronics design, sensing, ma¬ 
chine intelligence, and problem understanding: 

• Mechatronics designers have to trade gains in pre¬ 
cision, dexterity, mobility, and strength for losses 
in endurance and reliability. We are mostly well 
short of biological (e.g., human) capabilities ex¬ 
cept for high-precision applications. Machines have 
much the same environmental tolerance as peo¬ 
ple. Machines need special precautions against 


heat and cold outside the temperature range of 
0—35 °C, and operation beyond —60 or +60 °C 
is usually impractical. Dust, radiation, low or zero 
air pressure, fumes, biological agents, even insects 
can be fatal for machines without special design 
features, which may result in performance reduc¬ 
tions. Maintenance or repair work may necessitate 
decontamination before people can work on the 
machinery. 

• While electronic sensors can go far beyond biologi¬ 
cal capabilities, hazardous applications still present 
problems far beyond present capabilities as we have 
seen in the case of landmine applications. Radiation 
levels that are lethal for humans can also quickly 
kill electronic sensors. Extreme heat or cold limits 
sensor performance, as can contamination, and even 
insects. 
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• Advances in usable machine intelligence have been 
much more difficult than many expected. The only 
real progress has been achieved with capabilities 
often associated with clever people: logic, math¬ 
ematical manipulation, and playing games. While 
recent advances have enabled a remarkable de¬ 
gree of understanding for some insect perception 
and motor capabilities, the social behaviour of in¬ 
sects still largely defies present-day understanding. 
However, teleoperation and supervisory control can 
make up for this deficit in the medium term. Short¬ 
comings in machine intelligence represent less of 
a barrier than the other three fronts. 

• Understanding the intended application and eco¬ 
nomic factors has proved to be equally important for 
researchers. Early success from exploiting specific 
opportunities can generate confidence that inspires 
others to persevere with more difficult general prob¬ 
lems. Failures in understanding the desired task 
performance, particularly with respect to economic 
factors, can undermine confidence and the credibil¬ 
ity of research efforts. Attempts to develop land¬ 
mine clearance robots provides a useful case study 
that illustrates some of the difficulties in developing 
robots for hazardous applications. 

Video-References 


There is no question that future teleoperators will 
become hybrid telerobotic systems that allow seamless 
transfer between manual and autonomous operations. 
Specific tasks will be selectively automated by opera¬ 
tors for the purposes of enhancing quality and/or re¬ 
ducing subtask execution times. Early systems will con¬ 
tinue to incorporate high levels of human interactivity 
to accommodate the limitations of current intelligent 
systems technology. Real-time execution of complex 
control algorithms and virtual-reality-like graphics en¬ 
gines (which are really useful) have become routine. Ro¬ 
bust machine learning that allows human task execution 
skills to be captured through learning by observation or 
imitation is gradually becoming realizable. Advances in 
mass production of microelectrical mechanical systems 
(MEMSs) technologies has greatly reduced the cost of 
sensing devices. In the next decade, telerobots for haz¬ 
ardous environments will become smarter and their re¬ 
lationship with their human operators will become more 
cooperative and less interactive. Human operators will 
supervise the operation of multiple telerobots: at the mo¬ 
ment, the number of support people required for real 
operations is a multiple of the number of robots. 

Readers are directed to the selection of references 
provided in this chapter for further reading. 
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UNMACCA: Demining Afghanistan 

available from http://handbookofrobotics.org/view-chapter/58/videodetails/571 
IED hunters 

available from http://handbookofrobotics.org/view-chapter/58/videodetails/572 
Bozena 5 remotely operated robot vehicle 

available from http://handbookofrobotics.org/view-chapter/58/videodetails/574 
DALMATINO 

available from http://handbookofrobotics.org/view-chapter/58/videodetails/575 
PT-400 D:Mine 

available from http://handbookofrobotics.org/view-chapter/58/videodetails/576 
DIGGER DTR demining destroying anti-tank mines 

available from http://handbookofrobotics.org/view-chapter/58/videodetails/577 
Remote-control heavy equipment used in debris removal at Fukushima reactor 3 
available from http://handbookofrobotics.org/view-chapter/58/videodetails/578 
iRobots used to examine interior of Fukushima powerplant 
available from http://handbookofrobotics.org/view-chapter/58/videodetails/579 
iRobots inspecting interior of Fukushima powerplant 

available from http://handbookofrobotics.org/view-chapter/58/videodetails/580 
Robot being used to carry vacuum cleaner head at Fukishima powerplant 
available from http://handbookofrobotics.org/view-chapter/58/videodetails/581 
Views of robot control screen - inspecting Fukushima power plant 
available from http://handbookofrobotics.org/view-chapter/58/videodetails/582 
Promotional video of robot for cleaning up Fukushima 

available from http://handbookofrobotics.org/view-chapter/58/videodetails/583 
Sukura robot developed for reconnaissance missions inside nuclear reactor buildings 
available from http://handbookofrobotics.org/view-chapter/58/videodetails/584 
HD Stock Footage 1950s atomic power plants - nuclear reactors 
available from http://handbookofrobotics.org/view-chapter/58/videodetails/586 
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Radioactive material handling 1954 

available from http://handbookofrobotics.org/view-chapter/58/videodetails/587 
Nuclear manipulator remote handling equipment (1960) 
available from http://handbookofrobotics.org/view-chapter/58/videodetails/588 
1961 nuclear reactor meltdown: The SL-1 accident - United States Army Documentary 
available from http://handbookofrobotics.org/view-chapter/58/videodetails/589 
Jean Vertut master-slave manipulator arms 

available from http://handbookofrobotics.org/view-chapter/58/videodetails/590 
NanoMag magnetic crawler for remote inspection 

available from http://handbookofrobotics.org/view-chapter/58/videodetails/591 
Remote handling and inspection with the VT450 

available from http://handbookofrobotics.org/view-chapter/58/videodetaiis/592 
Robots answer nuclear waste challenges at SRS 

available from http://handbookofrobotics.org/view-chapter/58/videodetails/593 


References 


58.1 S. Kang, C. Cho, J. Lee, D. Ryu, C. Park, K.-C. Shin, 
M. Kim: R0BHAZ-DT2: Design and integration of 
passive double tracked mobile manipulator system 
for explosive ordnance disposal, IEEE/RSJ Int. Conf. 
Intell. Robots Syst. (2003) 

58.2 A. Kron, G. Schmidt, B. Petzold, M.l. Zah, P. Hinter- 
seer, E. Steinbach: Disposal of explosive ordnances 
by use of a bimanual haptic telepresence system, 
IEEE Int. Conf. Robotics Autom. (2004) 

58.3 W.R. Hamel, R.L. Cress: Elements of Telerobotics 
Necessary for Waste Clean Up Automation, IEEE Int. 
Conf. Robotics Autom. (2001) 

58.4 J.C. Ralston, D.W. Hainsworth, D.C. Reid, D.L. An¬ 
derson, R.J. McPhee: Recent advances in remote 
coal mining machine sensing, guidance, teleop¬ 
eration and field robotics, Robotica 19(5), 513-526 
( 2001 ) 

58.5 GICHD: A Study of Manual Mine Clearance (Interna¬ 
tional Centre for Humanitarian Demining, Geneva 
2005) 

58.6 GICHD: A Study of Mechanical Application in Demi¬ 
ning (Generva International Centre for Humanitar¬ 
ian Demining, Geneva 2004) 

58.7 J.P. Trevelyan, S. Tilli, B. Parks, H.C. Teng: Farm¬ 
ing minefields: Economics of remediating land 
with moderate landmine and UXO concentra¬ 
tions, Demining Technol. Inform. Forum J. 1(3), 
( 2002 ) 

58.8 C.G. Bruschini, B. Gross: A survey on sensor tech¬ 
nology for landmine detection, J. Mine Action 2(1), 
(1998) 

58.9 A. Goth, I.G. McLean, J.P. Trevelyan: How do dogs 
detect landmines? A summary of research re¬ 
sults. In: Mine Detection Dogs: Training, Operations 
and Odour Detection, ed. by I.G. McLean (Geneva 
International Centre for Humanitarian Demining, 
Geneva 2003) 

58.10 S. Havlik: A modular concept of the robotic vehicle 
for demining operations, Autonom. Robots 18(3), 
253-262 (2005) 

58.11 J.P. Wetzel: Robotic applications in humanitarian 
demining, Proc. 9th Bienn. Conf. Eng. Constr. Oper. 
Chall. Environ. Earth Sp. (2004) 


58.12 R. Bogue: Detecting mines and lEDs: What are the 
prospects for robots?, Ind. Robot 38(5), 456-460 
( 2011 ) 

58.13 J.P. Trevelyan: A suspended device for humanitar¬ 
ian demining, EUREL Int. Conf. Detect. Abandoned 
Land Mines Humanit. Imperative Seek. Tech. Solut. 
(1996) 

58.14 J. Trevelyan: Robots: A premature solution for the 
land mine problem, Proc. 8th Int. Symp. Robotics 
Res. (1998) 

58.15 J.-D. Nicoud: Vehicles and robots for humanitarian 
demining, Ind. Robot 24(2), 164-168 (1997) 

58.16 J.P. Trevelyan: Landmine research: Technology so¬ 
lutions looking for problems, Proc. SPIE Detect. 
Remediat. Technol. Mines Minelike Targets IX 
(2004) 

58.17 H.M. Choset, E.U. Acar, A.A. Rizzi, J.E. Luntz: Sensor- 
based planning: exact cellular decompositions in 
terms of critical points, Proc. SPIE Mobile Robots XV 
Teiemanip. Telepresence Technol. VII (2000) 

58.18 Y. Zhang, M. Schervish, E.U. Acar, H. Choset: Proba¬ 
bilistic methods for robotic landmine search, Proc. 
RSJ/IEEE Int. Conf. Intell. Robots Syst. (2001) 

58.19 S. Singh, S. Thayer: Inspired by immunity, Nature 
415 , 468 - 470 (2002) 

58.20 S. Sathyanath, F. Sahin: Application of artificial im¬ 
mune system based intelligent multi agent model 
to a mine detection problem, IEEE Int. Conf. Syst. 
Man Cybern. (2002) 

58.21 D. Goldberg, M.J. Mataric: Maximizing reward in 
a non-stationary mobile robot environment, Au¬ 
tonom. Agents Multi-Agent Syst. 6(3), 287-316 
(2003) 

58.22 R.A. Russell: Locating underground chemical 
sources by tracking chemical gradients in 3 dimen¬ 
sions, IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS) 
(2004) 

58.23 D.C. Conner, P.R. Kedrowski, C.F. Reinholtz, 
J.S. Bay: Improved dead reckoning using caster 
wheel sensing on a differentially steered three¬ 
wheeled autonomous vehicle, Proc. SPIE Mobile 
Robots XV Teiemanip. Telepresence Technol. VII 
( 2000 ) 














Robotics in Hazardous Applications I References 1547 


58.24 S.-D. Kim, C.-H. Lee, S.-J. Yoon, H.-K. Jeong, S.- 

C. Kang, M.-S. Kim, Y.-K. Kwalc Variable configura¬ 
tion tracked mobile robot for demining operations, 
ASME Des. Eng. Tech. Conf. Comput. Inf. Eng. Conf. 
(2004) 

58.25 K. Nonami, R. Yuasa, D. Waterman, S. Amano, 
H. Ono: Preliminary design and feasibility study 
of a 6-degree of freedom robot for excavation 
of unexploded landmine, Autonom. Robots 18(3), 
293-301 (2005) 

58.26 Y.-J. Lee, S. Hirose: Three-legged walking for 
fault-tolerant locomotion of demining quadruped 
robots, Adv. Robotics 16(5), 415-426 (2002) 

58.27 T. Wojtara, K. Nonami, H. Shao, R. Yuasa, S. Amano, 

D. Waterman, Y. Nobumoto: Hydraulic master-slave 
land mine clearance robothand controlled by pulse 
modulation, Mechatronics 15(5), 589-609 (2005) 

58.28 N. Furihata, S. Hirose: Development of mine hands: 
Extended prodder for protected demining opera¬ 
tion, Autonom. Robots 18(3), 337-350 (2005) 

58.29 P. Debenest, E.F. Fukushima, Y. Tojo, S. Hirose: 
A new approach to humanitarian demining: Part 
2: Development and analysis of pantographic ma¬ 
nipulator, Autonom. Robots 18(3), 323-336 (2005) 

58.30 H. Yabushita, M. Kanehama, Y. Hirata, K. Ko- 

suge: 3-D ground adaptive synthetic aperture radar 
for landmine detection, IEEE/RSJ Int. Conf. Intel!. 
Robots Syst. (IR0S) (2005) 

58.31 A.A. Faust, R.H. Chesney, Y. Das, J.E. McFee, 

K.L. Russel: Canadian teleoperated landmine de¬ 
tection systems. Part I: The improved landmine 
detection project, Int. J. Syst. Sci. 36(9), 511-528 
(2005) 

58.32 A.A. Faust, R.H. Chesney, Y. Das, J.E. McFee, 

K.L. Russel: Canadian teleoperated landmine de¬ 
tection systems. Part II: Antipersonnel landmine 
detection, Int. J. Syst. Sci. 36(9), 529-543 (2005) 

58.33 K.N. Zachery, G.M. Schultz, L.M. Collins: Force pro¬ 
tection demining system (FPDS) detection subsys¬ 
tem, Proc. SPIE Detect. Remediat. Technol. Mines 
Minelike Targets X (2005) 

58.34 J. Coronado-Vergara, G. Avina-Cervantes, M. Devy, 
C. Parra: Towards landmine detection using arti¬ 
ficial vision, IEEE/RSJ Int. Conf. Intell. Robots Syst. 
(2005) 

58.35 J.P. Trevelyan: Demining Research at the University 
of Western Australia, http://www.mech.uwa.edu. 
au/jpt/demining (Univ. Western Australia, Perth 
2000 ) 

58.36 S. Rajasekharan, C. Kambhampati: The current 
opinion on the use of robots for landmine detec¬ 
tion, IEEE Int. Conf. Robotics Autom. (2003) 

58.37 J.P. Trevelyan: Reducing Accidents in Demining: 
Achievements in Afghanistan, J. Mine Action 1, IQ- 
17 (2000) 

58.38 T. Lardner (Ed.): A Study of Manual Mine Clear¬ 
ance: Part 4 - Risk Assessment and Risk Manage¬ 
ment (Geneva International Centre for Humanitar¬ 
ian Demining, Geneva 2005) 

58.39 P. Newnham, D. Daniels: The market for advanced 
humanitarian mine detectors, Proc. SPIE Detect. 
Remediat. Technol. Mines Minelike Targets VI (2001) 


58.40 J. Vertut, P. Coiffet: Teleoperation and robotics, 
evolution and development, Robot Technol. 3A, 
302-307 (1985) 

58.41 L.l. Slutski: Remote Manipulation Systems: Quality 
Evaluation and Improvement (Kluwer, Dordrecht 
1998) 

58.42 K.F. Kraiss: Advanced Man-Machine Interaction: 
Fundamentals and Implementation (Springer, 
Berlin, Heidelberg 2006) 

58.43 E.G. Johnsen, W.R. Corliss: Human Factors Applica¬ 
tions in Teleoperator Design and Operation (Wiley, 
New York 1971) 

58.44 T.B. Sheridan: Telerobotics, Automation, and Hu¬ 
man Supervisory Control, Technology and Indus¬ 
trial Arts (MIT Press, Cambridge 1992) p. 432 

58.45 W.R. Hamel: Sensor-based planning and control in 
telerobotics. In: Control in Robotics and Automa¬ 
tion, Sensor-Based Integration, ed. by B.K. Ghosh, 
N. Xi, T.J. Tarn (Academic, New York 1999) pp. 285— 
309 

58.46 J.V. Draper: Effects of force reflection on servo- 
manipulator performance, Int. Top. Meet. Robotics 
Remote Handl. Hostile Environ. (1987) 

58.47 R. Goertz: Manipulator system development atANL, 
12th Conf. Remote Syst. Technol. (1962) 

58.48 R. Goertz: Some work on manipulator systems at 
ANL, past, present and a look at the future, Sem. 
Remot. Oper. Spec. Equip. (1964) 

58.49 D.P. Kuban, H.L. Martin: An advanced remotely 
maintainable servomanipulator concept, Natl. Top. 
Meet. Robotics Remote Handl. Hostile Environ. 
(1984) 

58.50 D. Sands: Cost effective robotics in the nuclear in¬ 
dustry, Ind. Robot 33(3), 170-173 (2006) 

58.51 S. Sanders: Remote operations for fusion using 
teleoperation, Ind. Robot 33(3), 174-177 (2006) 

58.52 B.L. Luk, K.P. Liu, A.A. Collie, D.S. Cooke, S. Chen: 
Teleoperated climbing and mobile service robots 
for remote inspection and maintenance in 
nuclear industry, Ind. Robot 33(3), 194-204 
(2006) 

58.53 P. Desbats, F. Geffard, G. Piolain, A. Coudray: Force- 
feedback teleoperation of an industrial robot in 
a nuclear spent fuel reprocessing plant, Ind. Robot 
33(3), 178-186 (2006) 

58.54 B. De Jong, E. Faulring, J.E. Colgate, M. Peshkin, 
H. Kang, Y.S. Park, T. Ewing: Lessons learned from 
a novel teleoperation testbed, Ind. Robot 33(3), 
187-193 (2006) 

58.55 M. Rennich, E. Bradley, T. Burgess, V. Graves: Spal¬ 
lation neutron source remote handling implemen¬ 
tation, 1st Jt. Emerg. Prep. Response/Robotics Re¬ 
mote Syst. Top. Meet. (2006) 

58.56 S. Kawatsuma, M. Fukushima, T. Okada: Emergency 
response by robots to Fukushima-Daiichi accident: 
Summary and lessons learned, Ind. Robot 39(5), 
428-435 (2012) 

58.57 S. Hirose, H. Kuwahara, Y. Wakabayashi, N. Yosh- 
ioka: The Mobility Design Concepts/Characteristics 
and Ground Testing of an Offset-Wheel Rover Vehi¬ 
cle, Int. Conf. Mobile Planet. Robots Rover Roundup 
(1997) 


Part F | 58 



Part F | 58 


1548 Part F 


Robots at Work 


58.58 T. Kagiwada: Robot design for stair navigation, Jpn. 
Soc. Mech. Eng. Int. J. C 39(3), 629-635 (1996) 

58.59 R. Volpe, R. Ohm, R. Petras, J. Welch, J. Blaram, 
R. Ivlev: A prototype manipulation system for Mars 
rover, IEEE/RSJ Int. Conf. Intell. Robots Syst. (1997) 

58.60 R. Siegwart, P. Lamon.T. Estier, M. Lauria, R. Piguet: 
Innovative design for wheeled locomotion in rough 
terrain, Robot. Autonom. Syst. 40, 151-162 (2002) 

58.61 K. Yoneda, Y. Ota, S. Hirose: Development of a hi- 
grip stair climbing crawler with hysteresis compli¬ 
ant blocks, 4th Int. Conf. Climbing Walk. Robots 
(CLAWAR) (2001) 

58.62 S. Hirose, T. Sensu, S. Aoki: The TAQT carrier: A prac¬ 
tical terrain-adaptive quadru-track carrier robot, 
IEEE/RSJ Int. Conf. Intell. Robots Syst. (1992) 

58.63 H. Schempf, E. Mutschler, C. Piepgras, J. Warwick, 
B. Chemel, S. Boehmke, W. Crowley, R. Fuchs, 
J. Guyot: Pandora: Autonomous Urban Robotic Re¬ 
connaissance System, IEEE Int. Conf. Robotics Au- 
tom. (1999) 

58.64 J.B. Coughlan: Small All Terrain Mobile Robot (Remo 
Tec, Clinton 1991) 

58.65 T. Takayama, S. Hirose: Development of Souryu-I 
connected crawler vehicle for inspection of narrow 
and winding space, 26th Annu. Conf. IEEE Ind. Elec¬ 
tron. Soc. (2000) 

58.66 C.H. Cho, W.S. Lee, S. Kang, M.S. Kim, J.B. Song: 
Rough-terrain negotiable mobile platform with 
passively adaptive double-tracks and its applica¬ 
tion to rescue missions, Adv. Robotics 19(4), 459- 
475 (2005) 

58.67 K. Kato, S. Hirose: Development of the quadruped 
walking robot, TITAN-IX-mechanical design con¬ 


cept and application for the humanitarian de- 
mining robot, Adv. Robotics 15(2), 191-204 (2001) 

58.68 iRobotlnc.: http://www.irobot.com/ 

58.69 E. Cheung, V.J. Lumelsky: Proximity sensing in robot 
manipulator motion planning: System and im¬ 
plementation issues, IEEE Trans. Robot. Autom. 5, 
740-751 (1989) 

58.70 B.R. Shetty, M.H. Ang Jr.: Active compliance control 
of a PUMA 560 robot, IEEE Int. Conf. Robotics Autom. 
(1996) 

58.71 B.-S. Kim, S.-K. Yun, S. Kang, C.-S. Hwang, M.- 
S. Kim, J.-B. Song: Development of a joint torque 
sensor fully integrated with an actuator, Int. Conf. 
Control Autom. Syst. (2005) 

58.72 M. Uebel, M.S. Ali, I. Minis: The Effect of Bandwidth 
on Telerobot System Performance (Goddard Space- 
flight Center, Greenbelt1991) 

58.73 G. Niemeyer: Telemanipulation with time delays, 
Int. J. Robotics Res. 23(9), 873-890 (2004) 

58.74 D. Ryu, S. Kang, M. Kim, J.-B. Song: Multi-modal 
user interface for teleoperation of R0BHAZ-DT2 field 
robot system, IEEE/RSJ Int. Conf. Intell. Robots Syst. 
(IROS) (2004) 

58.75 A.K. Skula, A.S. Arico, V. Antonucci: An appraisal of 
electric automobile power sources, Renew. Sustain. 
Energy Rev. 5,137-155 (2001) 

58.76 K.W. Ong, G. Seet, S.K. Sim: Sharing and trading in 
a human-robot system. In: Cutting Edge Robotics, 
ed. by V. Kordic, A. Lazinica, M. Merdan (pro liter- 
atur, Augsburg 2005) pp. 467-496 

58.77 J. Trevelyan: Redefining robotics for the new mil¬ 
lennium, Int. J. Robotics Res. 18(12), 1211-1223 
(1999) 



1549 


Multimedia Contents 



59. Robotics in Mining 


Joshua A. Marshall, Adrian Bonchis, Eduardo Nebot, Steven Scheding 


This chapter presents an overview of the state 
of the art in mining robotics, from surface to 
underground applications, and beyond. Min¬ 
ing is the practice of extracting resources for 
utilitarian purposes. Today, the international 
business of mining is a heavily mechanized 
industry that exploits the use of large diesel 
and electric equipment. These machines must 
operate in harsh, dynamic, and uncertain en¬ 
vironments such as, for example, in the high 
arctic, in extreme desert climates, and in deep 
underground tunnel networks where it can be 
very hot and humid. Applications of robotics in 
mining are broad and include robotic dozing, 
excavation, and haulage, robotic mapping and 
surveying, as well as robotic drilling and explo¬ 
sives handling. This chapter describes how many 
of these applications involve unique technical 
challenges for field roboticists. However, there 
are compelling reasons to advance the disci¬ 
pline of mining robotics, which include not only 
a desire on the part of miners to improve pro¬ 
ductivity, safety, and lower costs, but also out 
of a need to meet product demands by access¬ 
ing orebodies situated in increasingly challenging 
conditions. 
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There was a time, in recent history, when improved 
productivity and lower mining costs could be achieved 
purely by the economies of scale that arise from the 
use of larger equipment [59.1]. However, this era is 
likely over. Today’s mining companies and equipment 
manufacturers have been making renewed efforts in the 


pursuit of new and innovative approaches to the busi¬ 
ness of resource exploration and mining. 

Among these efforts has been a move toward increas¬ 
ingly autonomous mobile equipment and processes. For 
example, major suppliers of mining equipment for both 
underground and surface mining operations now offer 
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robotic driving, dumping, and other materials handling 
functionalities. What is more, some global mining com¬ 
panies are now operating their own technology-focused 
divisions, each with goals of bringing new technologi¬ 
cal developments to their mines worldwide. 


59.1 Modern Mining Practice 

Mining is an ancient and broad practice, dating back 
to Palaeolithic times (43 000 years ago) [59.2] and in¬ 
volves the extraction of materials from the Earth’s crust 
for utilitarian purposes. Today we mine minerals (e.g., 
gemstones, metals, salt, coal, and many others), aggre¬ 
gates, natural gas, and petroleum. These resources are 
used, for instance, to make the tools we use, shelter, 
food products, medicine, and clothing. Try to imagine, 
for a moment, a life where you endeavored to use (or 
build a robot from) absolutely nothing derived from 
mining. We benefit greatly from mining, but there is 
also increasing pressure and international recognition 
that the often negative environmental, safety, and social 
implications of mining and mineral processing must be 
curtailed and balanced with economic and lifestyle con¬ 
siderations. Robotics can play a vital role in all of these 
efforts. 

In this section, we briefly provide some introductory 
background about the fundamental stages of mining, 
common to most operations, and discuss what drives 
the field of mining robotics, as well as its unique chal¬ 
lenges, technical and otherwise. 

59.1.1 Stages of Mining 

Almost all mining operations have five fundamental 
stages [59.3]: 

1. Prospecting 

2. Exploration 

3. Development 

4. Exploitation 

5. Reclamation. 

Prospecting involves the initial search for valu¬ 
able materials. This stage is normally carried out by 
geologists, and deposits may be located at or below 
the surface. Prospecting usually happens by either di¬ 
rect visual examination or by indirect methods (e.g., 
through the use of geophysical techniques) that look 
for anomalies in seismic, magnetic, electrical, electro¬ 
magnetic, and/or radiometric variables of the Earth. 
Geochemistry and geobotany techniques are also em¬ 
ployed. Although robotics has not yet played a pivotal 


Finally, there exist several emerging frontiers for 
mining, where robotics is already playing a critical role. 
These include not only very deep and high altitude 
mining, but also undersea mining and extra-terrestrial 
mining. 


role in the search for minerals on the Earth, plane¬ 
tary geologists have been collaborating with roboticists 
for planetary prospecting; e.g., [59.4] and references 
therein. 

In the exploration phase, the objective is to deter¬ 
mine as accurately as possible the size ( tonnage ) and 
value (grade) of a deposit. This process is sometimes 
called delineation and more often than not involves 
collecting samples of material by drilling. Similar to 
prospecting, robotic exploration drills for mining are 
not yet common for surface exploration on the Earth. 
However, there has been a significant amount of work 
on robotic drilling in extreme environments, such as un¬ 
dersea and planetary exploration; see [59.5] for a good 
introduction and ample references. 

The development and exploitation phases of min¬ 
ing are where tools and techniques from robotics have 
been, to date, applied most. Development is the work 
necessary to bring a mine into production, and usual 
entails planning, design, construction (e.g., overburden 
removal, shaft sinking, underground access) and the 
installation of mine services (e.g., power, water, ven¬ 
tilation, etc.). 

It is during the exploitation phase that the resource 
is physically extracted, processed, and shipped to buy¬ 
ers. How exploitation occurs depends on the selected 
mining methods , which depend on many factors. These 
include the spatial characteristics of the deposit, ge¬ 
ologic conditions, geotechnical conditions, economic 
considerations, technological considerations (e.g.. the 
availability of advanced tools), and environmental con¬ 
siderations. Generally speaking, mining methods can be 
divided into two categories: 

1. Surface mining methods; 

2. Underground mining methods. 

Surface-mining methods are used when the deposit 
lies near the surface, are usually very large scale op¬ 
erations and have high production rates. Underground 
methods are used when the deposit is deeply buried. 
Such methods have lower production rates and are usu¬ 
ally more technically challenging and hazardous for 
mine workers. 
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Figure 59.1 shows examples of robotic vehi¬ 
cles used in surface and underground mining. These 
technologies, among others, are discussed further in 
Sects. 59.2 and 59.3. 

Mining must strive to meet the economic and envi¬ 
ronmental needs of the present while at the same time 
ensuring (or enhancing) the ability of future genera¬ 
tions to meet their own needs. Mine reclamation is the 
process of closing a mine, recontouring, revegetating, 
and restoring the land and water to an acceptable post¬ 
activity state. The objective is to minimize any adverse 
effects on the environment or threat to human health 
and safety. Some reclamation activities involve the con¬ 
struction and monitoring of tailings impoundments, 
where waste from mining and resource processing ac¬ 
tivities is placed. 

Recently, tools from robotics have been used 
to conduct bathymetric surveys of tailings ponds, 
which are unique environments with very specific 
safety requirements. Figure 59.2 shows a robotic mar¬ 
itime research vessel performing a bathymetric sur¬ 
vey of a mine tailings pond at a potash mining op¬ 
eration in eastern Canada. Performing robotic sur¬ 
veys has been shown to be safer and more cost- 
effective [59.6]. 



Fig. 59.1 (a) Examples of robotic vehicles in underground 
and surface mining. Unmanned surface haul trucks (Photo: 
Mobular Mining Systems, 2011); (b) an autonomous un¬ 
derground load-haul-dump (LHD) machine at the Kvam- 
torp mine, Sweden (Photo: Joshua Marshall, 2007) 


59.1.2 Technology Drivers in Mining 

In order to better understand the potential for applica¬ 
tions of robotics in mining, one must appreciate what 
drives the development of equipment technology in the 
mining industry. Mining is a dynamic process that must 
deal with the uncertainties of outdoor environments, 
and is usually conducted in as a series of interacting 
unit operations (e.g., drilling, blasting, loading, hauling, 
and processing). The harsh, extensive, and unstructured 
environments found in mining often preclude the appli¬ 
cation of existing techniques from other industries (e.g., 
from manufacturing robotics) [59.7, p. 806]. However, 
the recent growth of mobile and field robotics has re¬ 
sulted in several opportunities for robotics in mining. 
These opportunities stem mainly from a set of primary 
technology drivers: 

• Working Environment. Mines are often developed 
in harsh and remote areas. For example, several 
companies operate mines and processing facili¬ 
ties at high altitudes. Also, as mines strive to 
operate at greater depths, extremely high ambi¬ 
ent temperatures and humidity pose greater health 
risks [59.8]. Robotics may serve to minimize infras¬ 
tructure needs and physically remove people from 
such hostile environments. 

• Labor Shortage. Miners have reported difficulties 
finding the skilled labor needed to support their 
operations [59.9]. The newest generation of work¬ 
ers is technologically astute, but holds a different 
attitude toward physical labor than previous genera¬ 
tions. Robotization may be what is needed to entice 
this new generation into mining. 

• Health and Safety. With increasing mine depths, 
growing equipment sizes and speeds, and the tight- 



Fig. 59.2 Robotic maritime research vessel conducting 
a bathymetric survey of a mine tailings pond with overlaid 
example bathymetry map (Photos: Clearpath Robotics, 
2013) 
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Productivity index 



Fig. 59.3 Experimental estimates of multifactor and labor 
productivity for the mining industry in Australia (1986— 
2011) showing decreasing productivity since 2001 (af¬ 
ter [59.10]) 

ening of government regulations, it stands to reason 
that the deployment of almost any new technology 
that enhances the inherent safety of mining will be 
viewed positively at all levels. 

• Equipment Maintenance. Maintenance costs and 
machine failures can make up a very significant 
portion of mine operating costs [59.11]. Moreover, 
operators often push equipment to its performance 
limits. Recent wisdom from the field of reliabil¬ 
ity engineering suggests that the automation (or 


partial automation) of mobile equipment may help 
to significantly reduce downtime and maintenance 
costs [59.12], 

Operational Efficiency. There are a great number 
of areas in which efficiency improvements might 
be realized by the application robotics in mining. 
The most obvious lies in the fact that production 
time is lost during shift changes and breaks, re¬ 
sulting in much less than the desired 24/7 runtime. 
Not as obvious, and more difficult to quantify, is 
the less-than-optimal way in which underground 
mines typically operate. Without global and real¬ 
time control over the assignment (and movement) 
of machinery, the ability to optimize production 
performance through, for example, reduced dilu¬ 
tion and increased recovery is also very diffi¬ 
cult [59.1]. 

• Sustainability. Environmental and social responsi¬ 
bility are expected of modern mining companies 
and equipment suppliers. The alleviation of emis¬ 
sions (e.g., through the coordination and optimiza¬ 
tion of fleets) or reduction in power consumption 
by reduced operating demands (e.g., ventilation 
support) might be realized by way of real-time mon¬ 
itoring and automation of equipment. 

Figure 59.3 shows multifactor and labor productiv¬ 
ity for the mining industry in Australia between 1986 
and 2011. Arguably encompassing all of the factors 
above, this data shows a worrisome decrease in produc¬ 
tivity since the turn of the century, and some believe that 
robotics might help to reverse this trend [59.13]. 


59.2 Surface Mining 

Surface mining is characterized by high tonnage op¬ 
erations, massive mobile equipment, and challenges 
that can include difficult environments (e.g., dust, fog, 
and extreme weather) together with remote locales and 
a strict need for keeping costs low. 

59.2.1 Automated Haulage 

Haulage is the process of moving material from one lo¬ 
cation to another. In the surface mining context, this is 
typically achieved using haul trucks (Fig. 59. la), which 
move material from the point of blasting/excavation to 
the point of processing or stockpiling. The material be¬ 
ing moved is either the desired economic quantity or 
overburden, and the destination is usually dependent on 
the load in the truck. 

Haulage is a repetitive process, and the one to which 
automation can bring significant benefits, particularly 


with respect to safety. Between 1989 and 1991, haul 
trucks were involved in 42% of the accidents and 60% 
of the fatalities in surface mining [59.14]. As with 
most mining equipment, there are also potential benefits 
of automation in the areas of productivity, repeatabil¬ 
ity, and reduced maintenance costs. Although it is too 
early in the history of robotic mining equipment to 
precisely define these benefits, the lessons from port 
automation (which use machines of similar scale, al¬ 
beit in a more benign environment) have shown that 
maintenance costs of autonomous straddle carriers (in 
particular tyre wear and fuel costs) are reduced by ap¬ 
proximately one-third [59.15] over their conventionally 
operated counterparts. 

Autonomous Haul Trucks 

Autonomous haul trucks can be thought of as mobile 
robots operating in a difficult, dynamic environment. As 
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such, they are required to have systems to address the 
standard issues of sensing and perception, situational 
awareness, localization, and control. The unique chal¬ 
lenges presented in surface mining not present in other 
robotics applications include: 

• Scale (haul trucks can carry up to 4001 of material). 

• Mining pits often restrict the view of the sky, which 
can dramatically decrease global navigation satellite 
systems (GLS) reliability. 

• Surface mining environments are constantly chang¬ 
ing, which limits the ability to install fixed infras¬ 
tructure. 

• Mixed manually operated and automated fleets 
of machines, which places strict requirements on 
safety and integrity. 

• All-weather operational requirements. 

In the commercial sector, both Caterpillar and Ko¬ 
matsu had demonstrated the ability to automate haul 
trucks as early as the mid-1 990s [59.16]. However, it 
was not until 2008 that the first commercial deploy¬ 
ment of Komatsu driverless trucks was realised in the 
Gaby Mine, operated by Codelco in Chile [59.17]. This 
was followed closely by a deployment by Rio Tinto at 
their West Angelas mine in the Pilbara, Western Aus¬ 
tralia [59.18] d-s&Xihliilfl), where they have operated 
continuously since and have moved more than 50 mil¬ 
lion t of waste. 

The Komatsu trucks operate using a high-precision 
global positioning system (GPS) as the primary local¬ 
ization sensor, radar sensors for obstacle detection and 
situational awareness, a proprietary wireless communi¬ 
cations system, and are given tasks (load, haul, dump, 
etc.) using a proprietary scheduling system developed 
by Modular Mining Systems [59.18]. 

At the time of writing, similar systems by Cater¬ 
pillar [59.19] and Hitachi [59.20] are also in various 
stages of trial and deployment, although there is scant 
information available publicly regarding the specifics of 
their designs and operation. 

The automation of mining haul trucks is still an 
area of active research, however, primarily due to 
the challenging nature of the mining environment as 
previously noted. New positioning systems [59.21], 
situational awareness systems (Sect. 59.2.10), and plan¬ 
ning and scheduling algorithms [59.22] are all un¬ 
der active development, and are intended to improve 
robustness, safety, and productivity, respectively. If 
each of these challenges can be addressed, robotic 
haulage technology will be able to applied in a much 
greater range of operational scenarios than is currently 
possible. 


59.2.2 Fleet Management 

Fleet management is a catch-all term that may de¬ 
fine a number of different technologies used in mining. 
Commonly though, fleet management is divided into 
three main tasks: 

1. Position (and perhaps materials) monitoring 

2. Production monitoring 

3. Equipment task assignment. 

Fleet management solutions are typically deployed 
to an office environment and allow operators to quickly 
survey large amounts of mine data so that appropri¬ 
ate actions can be taken in real-time on the mine- 
site. Computerized fleet management must integrate 
and evolve with the deployment of robotic mining 
machines. 

Position Monitoring 

The position of mobile equipment is commonly re¬ 
ported via wireless communications to a mine or re¬ 
gion’s database, where it can be queried by a visualiza¬ 
tion application. This provides a real-time picture of the 
movement of all equipment that has been enabled for 
such operation and allows an operator to quickly assess 
whether planned operations are proceeding as expected. 
Material movements may also be monitored in a similar 
fashion, but the material type (ore or waste) and grade 
must usually be entered manually by an operator. How¬ 
ever some parameters, such as weight, are measured by 
the system. 

Production Monitoring 

Given that vast quantities of data may exist (partic¬ 
ularly from robotic machines) that give a historical 
account of the evolution of a mine, or group of mines, 
this data can be used to provide detailed reports to 
mine management, and may also be useful as part of 
a continual productivity improvement process. Exam¬ 
ples of production monitoring data may include ma¬ 
chine cycle times, average maintenance cycles, machine 
failure events, payloads, or combinations of parame¬ 
ters (e.g., machine failure events vs. machine payload 
weight). 

Equipment Task Assignment 
At a typical mine, the equipment task assignment prob¬ 
lem is addressed in a manner very similar to that of 
the urban taxi dispatch problem: A centralized author¬ 
ity decides, based on some predefined high-level goals, 
which machines must visit certain locations on the 
mine-site and perform some useful action. A machine 
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operator is generally given these instructions through 
a user interface located in the machine’s cabin, which 
must be acknowledged and accepted prior to carrying 
out the task. The most typical implementation of the 
task assignment problem addresses the haulage sce¬ 
nario: when should a particular haul truck approach 
a given shovel, and when and where should it dump 
its subsequent load? Haulage is most commonly ad¬ 
dressed, because it is here that even small changes 
in efficiency can have significant effects on overall 
profit. 

Commercial Solutions 

There are many commercial solutions for fleet manage¬ 
ment in the surface mining. The dominant solution is 
Modular Mining’s DISPATCH system [59.23], which 
was developed in the 1970s. It implements an optimiza¬ 
tion algorithm to allocate trucks to shovels and dump 
sites, and contains modules for the other tasks of pro¬ 
duction and positioning monitoring. Modular Mining 
joined the Komatsu group of companies in 1996 and 
helped to deploy the FrontRunner autonomous haulage 
system described previously [59.24]. 

Other commercial systems include Caterpillar’s 
MineStar software [59.25], Wenco’s fleet management 
solution [59.26], Leica Geosystems Jigsaw suite of 
software and hardware [59.27], and Devex’s Smart- 
Mine system [59.28]. These systems all now offer very 
similar overall functionality and differ mainly in imple¬ 
mentation details. 

Fleet Management Research 
Although commercial systems for fleet management ex¬ 
ist, there is still a substantial amount of research being 
undertaken, primarily in the area of task assignment 
optimization. Most commercial systems optimize task 
allocation using a heuristic that defines costs for allow¬ 
ing a shovel to become idle and/or allowing haul trucks 
to wait in a queue. In an ideal or optimal system, all 
items of equipment are fully utilized. However in prac¬ 
tice, a higher cost is usually given to shovel idle time. 
The paper [59.29] gives a good overview of many of the 
commonly used heuristics used in task allocation opti¬ 
mization. 

More recently, it has been acknowledged that 
heuristic, deterministic optimizers may not always pro¬ 
duce truly optimal task assignments. This is primarily 
due to their inability to account for variability in the 
duty cycles of the items of equipment (i. e., they are 
stochastic rather than deterministic in nature), and any 
unforeseen discrete events, such as equipment failure, 
road blockage or operator error. The papers [59.22, 30- 
33] all describe recent approaches to overcoming these 
hurdles. 


59.2.3 Robotic Digging 

Mass excavation is one of the most important opera¬ 
tions in mining. Being extremely sensitive to economies 
of scale and requiring a significant capital investment, 
it became a target application for robotic technologies 
starting in the mid-1990s. Surface mining typically uses 
large hydraulic excavators, hydraulic shovels, and elec¬ 
tric rope shovels (Fig. 59.4). See also Sect. 59.3.3, 
which focuses on underground robotic loading. 

Research in robotic digging involves systems de¬ 
sign (including systems of systems), traditional robotics 
topics (sensing, planning, control), and tool-ground 
interaction problems. A framework for an intelligent 
earthwork system is suggested by [59.34], which dis¬ 
cusses factors that can affect earthwork operation 
performance, identify key emerging technologies to 
support the implementation, the system architecture, 
and the system control strategy. A comprehensive re¬ 
view by [59.35] examined various aspects of sens¬ 
ing, planning, and control as they apply to earthmov- 
ing automation. Later developments in these topics 
are further discussed in [59.36]. Irrespective of the 
manned/unmanned aspect of the operation, a typical 
digging phase involves a number of critical steps: plan¬ 
ning the bucket trajectory in contact with the ground, 
bank or muck pile; detecting when the bucket is full; 
and, detecting incipient stall and averting it. An analy¬ 
sis of autonomy requirements for each of these steps is 
detailed in [59.37] for the particular case of an electric 
rope shovel. 

In other work, authors of [59.38,39] proposed 
an autonomous excavation system for front-end-loader 
style machines that uses bucket force feedback, fuzzy 
logic, and neural networks for control. In their ap¬ 
proach, a set of fundamental bucket action sequences. 



Fig. 59.4 A rope shovel dumping into a haul truck (cour¬ 
tesy of Commonwealth Scientific and Industrial Research 
Organisation (CSIRO), 2006) 
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typically used by human operators, was compiled for 
use by the controller. A reactive approach, using fuzzy 
behaviours, was designed to act on force data to assess 
the excavation status and determine an appropriate con¬ 
trol input. Experimental results, using a programmable 
universal machine for assembly (PUMA) 560 arm, were 
reported. Many more researchers have studied the 
robotic loading problem for homogeneous materials, 
such as soils and regolith. The majority of researchers 
have advocated the use of manipulator impedance or 
compliant motion control [59.40-43] and much of this 
work has focused on estimating the properties of the 
media [59.44] and on computing the resistive forces act¬ 
ing on the budget during loading [59.45]. 

Invariably, all researchers agree that the biggest 
challenge to robotic excavation is the interaction be¬ 
tween the tool and the terrain (ground, pile). This 
interaction is shaped by the properties of the media 
(e.g., density and hardness), the rock pile geometry, 
and the distribution of particle sizes and shapes. The 
problem is compounded by the fact that it is difficult 
to predetermine the exact nature of interactions prior 
to the execution of any particular excavation opera¬ 
tion [59.46]. Significant effort was dedicated to model¬ 
ing the interaction between the bucket and the terrain in 
excavation. This requires estimating the soil parameters 
online, since these parameters can vary significantly 
with the strata being excavated. The classical funda¬ 
mental equation of earthmoving [59.47] used to deter¬ 
mine the forces required for digging will fail to predict 
these forces if the soil parameters are not estimated cor¬ 
rectly. Models for automated excavation using online 
estimation of soil parameters are discussed in [59.48]. 

Situational awareness, which includes both work¬ 
space and machine awareness, is another key aspect of 
robotic digging. Workspace awareness implies a knowl¬ 
edge of the terrain surrounding the digging machine. At 
a primary level, this knowledge is gathered using a sen¬ 
sor suite retrofitted to the machine, which in a typical 
configuration consists of a number of ranging sensors, 
an inertial measurement unit, and a GNSS unit. The 
acquired raw data is subsequently processed by higher 
level processes to detect the location and configuration 
of the dig and dump regions, and the location of other 
mobile equipment operating the workspace (e.g., haul 
trucks, utility vehicles). Laser rangefinders offer ade¬ 
quate resolution to segment the scene. An example of 
a workspace image in a form of a three-dimensional 
(3-D) point cloud acquired with a long range laser scan¬ 
ner installed on a shovel is shown in Figure 59.5. Color 
is assigned to the points based on intensity returns from 
the laser scanner. 

Machine self-awareness is primarily concerned with 
determining the position and orientation of the bucket 


with respect to a referential fixed with respect to the 
machine. It also includes all the usual status data for 
the main onboard systems: power plant, transmission, 
tool actuation, etc.. The position and orientation of the 
bucket are usually available from encoders and incli¬ 
nometers. A better solution, however, is to use suitably 
placed ranging sensors that are able to track the bucket 
in all positions and orientations [59.49], This latter 
method avoids the errors typically associated with back¬ 
lash in joints and dynamic effects in inclinometers. The 
placement of the ranging sensors on the machine should 
consider the need to provide an image of the ground un¬ 
der the boom. 

Autonomous digging systems offer the additional 
benefit of enabling dig-to-plan operation and reconcil¬ 
ing actual versus planned volumes by providing ac¬ 
curate real-time survey information. In surface mining 
operations, the floor is usually mined and therefore dis¬ 
turbed before accurate surveys are executed, and the 
floor is graded on an as-needed basis to facilitate the 
movement of the mobile equipment operating in the 
pit [59.49]. 

59.2.4 Robotic Dozing 

Dozers are used in most surface mines. Operating 
a dozer under conditions that are dynamic, with hazards 
that can vary and be subtle and difficult to recognize, 
adds to the complexity of the tasks that workers perform 
on these machines. Many of these tasks require oper- 



Fig. 59.5 A top view of a 3-D point cloud coloured by in¬ 
tensity returns from a long range laser scanner installed on 
a mining shovel loading a haul truck (courtesy of CSIRO, 
2008) 
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ators that are highly skilled in ground-engaging tool 
control and machine positioning. In addition, dozer op¬ 
erators are exposed to a variety of risk factors that may 
lead to health problems, including whole-body vibra¬ 
tions, awkward postural requirements, noise, and shift 
work. While machine guidance systems are now ubiq¬ 
uitous, they still require an operator in the cab and 
have known problems associated with their reliance on 
GNSS-based localization. As alternatives, some equip¬ 
ment manufacturers offer tele-remote-controlled and 
autonomous dozers, which eliminate the need for an on¬ 
board operator. 

Remote control solutions for dozers have been in 
use for two decades. They cannot perform in poor visi¬ 
bility conditions (dust, fog, night operation, heavy rain) 
and do not offer the possibility of automating machine 
guidance and tool control tasks otherwise available if 
operating the dozer from the cab. Other problems in¬ 
clude loss of situational awareness, inaccurate attitude 
judgement, and inaccurate depth perception. More re¬ 
cent commercial remote control offerings include force 
feedback. In the case of tele-operated control with 
negligible latencies, imparting force feedback to the op¬ 
erator has the benefit of allowing the operator to feel and 
control the machine. 

In view of the limitations of remote control tech¬ 
nologies, research on dozer automation has focused 
on semi-autonomous and autonomous system retrofits. 
Autonomous systems have already made significant in¬ 
roads in the mining industry, and the acceptance for 
this technology is growing. Leica Geosystems offers an 
autonomous ripping solution for track and wheel doz¬ 
ers. Their system, called Leica J 3 dozer autorip, provides 
a number of automatic functions, which include engine 
start/stop engine, automatic control for steering, throt¬ 
tle, transmission and ripper, accessories control, and an 
emergency fail-safe system [59.50]. The system also 
incorporates a path planner based on area coverage al¬ 
gorithms. 

59.2.5 Autonomous Blasthole Drilling 

Blasthole drilling is a mining process typically em¬ 
ployed during the exploitation phase of surface mining. 
To excavate a material it must first be blasted to fracture 
the rock mass into small volumes suitable for haulage 
and processing. This is achieved by drilling a matrix of 
holes from the surface down into the rock mass; these 
holes are then loaded with explosives and blasted. The 
resultant fractured rock mass can be excavated by a va¬ 
riety of machines. The matrix of holes (or drill pattern) 
and the amount and type of explosives used are op¬ 
timized to produce a desired rock size distribution in 
order to minimize the cost of processing; however, there 


can be significant variation due to the variable nature of 
the underlying geology. 

Blasthole drill rigs are usually large tracked vehicles 
with an articulated mast that contains the main drilling 
components: the drill motor, drill rods, and drill bit. To 
drill to depths longer than the mast, drill rods can be 
concatenated to form a long drill string as drilling is 
progressing. Like many home drills, most blasthole drill 
rigs are capable of rotary drilling, as well as hammer 
(or percussion) drilling. Rotary is typically preferred for 
softer materials, with hammer drilling primarily used 
for harder materials. 

There are several facets of drilling amenable to au¬ 
tomation: tramming (the process of moving the entire 
rig from one hole location to another), drilling, and the 
management of drill consumables (such as rod or bit 
changes). A fully autonomous drill rig should have all 
of these functions automated. The promise of automa¬ 
tion is increased productivity and more consistency in 
the placement and angle of drill holes, which in turn 
should lead to better blasting outcomes. Consistency 
in post-blast fragmentation results in a significant eco¬ 
nomic impact. 

There are several high-profile mining equipment 
manufacturers and companies that have developed fully 
automated drill rigs. Atlas Copco’s Rig Control Sys¬ 
tem (RCS) has reportedly made significant impact in 
increasing average tramming speeds (from 0.8 m/min 
to 2.6 m/min) and drilling efficiency (a savings of 1 h 
per 12 holes drilled) [59.51]. Flanders [59.51], and Rio 
Tinto [59.52] also have fully automated blast hole drill 
rigs in production. 

59.2.6 Telerobotic Rock Breaking 

Drilling and blasting results are often far from ideal and 
the resulting fragments of rock (called shot muck or just 
muck) can be larger than the projected size. Some of 
the shot muck can be too big for handling with a dig¬ 
ging machine, and thus requires a secondary blasting. 
Other boulders can be handled by the digging and haul¬ 
ing machines but are still too large to fit in the mouth of 
a primary crusher. In this latter case, rockbreakers are 
used to achieve a further reduction in ore and rock size. 

A rockbreaker consists of a large 4-DOF (degree of 
freedom) serial link manipulator arm that is fitted with 
a hydraulic hammer (Fig. 59.6), and is usually posi¬ 
tioned in front of a run-of-mine (ROM) bin. The bin 
is fitted with horizontal bars at the bottom that prevent 
oversized rocks from entering the crusher below (this 
arrangement is called a grizzly). The size of the grill 
openings is made in such a manner that only rocks small 
enough to be crushed may pass through. Larger rocks 
do not fit and therefore must be broken. Haul trucks car- 
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Fig.59.6a,b Rock breaker at rest over 
the ROM bin (a) and breaking a rock 
on grizzly (b) (courtesy of CSIRO, 
2007) 


rying ore from a nearby quarry dump their load into the 
ROM bin. Typically, an operator uses a line-of-sight re¬ 
mote control to operate the arm. 

Rio Tinto is pursuing a programme to automate the 
vast majority of its Pilbara iron ore operations in West¬ 
ern Australia, and control the operations from the Re¬ 
mote Operations Centre (ROC) in Perth, situated over 
1000 km away from the Pilbara region. This effort in¬ 
cludes the development of a telerobotic control system 
for the primary rockbreaker at the West Angelas mine 
in collaboration with the CSIRO [59.53], This system 
addresses known drawbacks in teleoperation (limited 
communication bandwidth, high latency in the real¬ 
time video feedback, lack of spatial situational aware¬ 
ness, etc.) by improving the intelligence of the con¬ 
trol system at the remote/machine end (e.g., Cartesian 
motion and collision avoidance capability) and by pro¬ 
viding the operator with a mixed-reality interface that 
combines live video with 3-D computer visualization. 

Each rockbreaker is fitted with a number of sensors 
to detect its pose in space, and actuated by signals that 
bypass the existing remote-control system. PTU (pan, 
tilt, zoom) cameras are fitted to poles on either side 
of the ROM bin and a pair of high-resolution digital 



Fig. 59.7 A mixed reality user interface combines live 
video and 3-D stereo reconstruction of the rocks above the 
grizzly (courtesy of CSIRO, 2008) 


stereo video cameras were mounted below. At the re¬ 
mote end, in the control room, the operator is presented 
with an overview of the rockbreaker from a wide-angle 
video stream augmented by a synthetic computer image 
(Fig. 59.7). The operator is able to walk around the rock¬ 
breaker in the virtual world and inspect the rocks from 
different angles, enabling the operator to determine the 
appropriate breaking strategy. The operator deploys the 
arm using commands from a joystick, and as the arm is 
commanded to move, the motion of the arm is replicated 
in the 3-D scene. Simultaneously, both PTZ cameras fol¬ 
low the tip of the hammer. When the operator is ready 
to break the rock, he/she can switch his/her attention to 
the live video stream, which can be used to monitor the 
breaking of the rock. Once complete, the arm can be au¬ 
tomatically sent to the rest position. 

59.2.7 Automated Loading Unit 
and Truck Interactions 

The main loading units used in surface mining are elec¬ 
tric and hydraulic shovels and hydraulic excavators. 
Given the large volumes of material that need to be 
moved, and the comparatively small size of loading and 
hauling units, cycle times are closely monitored. In the 
continuous pursuit of efficiency, mining engineers dis¬ 
sect cycles to look for potential savings. One area of 
interest is truck spotting, which is the process of ma¬ 
noeuvring the haul truck into a position and orientation 
that is ideal for the loading unit to dump material into 
the truck’s tray (Fig. 59.4). 

The ideal truck placement for loading has both 
productivity and safety implications. When a mining 
shovel is used for loading, the haul truck must be po¬ 
sitioned in such a manner that the swing motion of the 
shovel is minimized and that the shovel boom does not 
need to lower excessively to reach the tray. If the haul 
truck is not placed optimally, the loader needs to repo¬ 
sition itself to tip with a full load in the bucket, which 
results in lost time. Lowering the loader arm excessively 
and/or an uneven distribution of load in the haul truck 
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tray may result in injuries to the operators and property 
damage. 

In open-pit mining, the shovel operator usually 
guides the haul trucks to the desired location (spotting) 
by placing the bucket in the location that is optimal for 
the dumping phase, and waits for the truck to move into 
position. A key requirement for achieving automatic 
spotting is the availability of a robust localizer, that de¬ 
termines the relative pose between the shovel and the 
truck. Currently, most of the commercial guidance so¬ 
lutions are based on the use of GNSS and GNSS-aided 
inertial navigation systems on both the shovel and the 
truck. This results in an indirect determination of the 
relative pose between the shovel and the truck. 

These systems, however, do not provide a robust so¬ 
lution. For the haul truck, the proximity to the shovel 
boom and crowd arm can lead to a degradation of the 
GNSS solution due to multipath, while the placement 
of the truck near highwalls leads to a partial obstruction 
of the sky, with a loss of satellite tracking and a sig¬ 
nificant geometric dilution of precision. The guidance 
system on the shovel can be impacted detrimentally by 
the proximity to highwalls. Imaging sensors (e.g., laser 
scanners) can be installed on both the shovel for a direct 
determination of the relative pose between the shovel 
and the truck [59.55]. However, this solution has po¬ 
tential pitfalls, that are related to the sensitivity of the 
imaging sensors to environmental conditions such as 
dust, rain, and variable illumination during the day (or 
during the 24 hour day/night cycle in cases where cam¬ 
eras are used). 

Some researchers have looked at modalities that 
mitigate these pitfalls. One possible solution, suggested 



Fig. 59.8 Traffic management zones around a shovel in 
a double-spotting configuration, when the shovel loads on 
both sides (after [59.54]) 


in [59.54], is to divide the space around the shovel into 
three zones: the outer zone, the handover zone, and the 
work zone (Fig. 59.8). 

The relative size of the zones depends on the truck 
and shovel geometries, and the chosen safe operating 
speed limits within each zone. As the truck crosses into 
the handover zone from the outer zone, the spotting 
system flags the truck as active and begins sending com¬ 
mands to it. In this zone, the relative pose between the 
shovel and truck is determined indirectly, based on the 
method discussed above. In the handover zone the truck 
is sufficiently far from the shovel and the highwall, and 
therefore the localization errors due to multipath and 
sky occlusion are minimized. Once the vehicle crosses 
into the work zone, the spotting system begins comput¬ 
ing the relative pose using the imaging sensors on the 
shovel. 

Since the sensors on the shovel are also able to 
see the bench and other obstacles in the work zone, 
the shovel is able to direct the truck around obsta¬ 
cles as they appear in the work zone, and park the 
trucks as close to the bench as practical. In other words, 
since the shovel is actively aware of its environment, 
it is able to deal with changes to the environment as 
they occur. The spotting system generates an approach 
path that will bring the truck in the optimum posi¬ 
tion for loading, where this optimum is determined 
in relation to the characteristics of the shovel and the 
configuration of the dig face. The path is also opti¬ 
mized with respect to the positioning time. Spotting 
times may be further minimized by double spotting, 
when the shovel loads on both sides, as depicted in 
Fig. 59.8. 

The above discussion focused on shovel-truck in¬ 
teractions. Similar issues aise when hydraulic exca¬ 
vators are used for loading. Comprehensive research 
results in autonomous truck loading using hydraulic ex¬ 
cavators are discussed in [59.56]. 

59.2.8 Dragline Automation 

Mining dragline excavators are massive electrically 
powered machines used in open-pit coal mining to 
remove overburden and uncover coal. A dragline com¬ 
prises a rotating platform that supports the house (con¬ 
sisting of the engine room and operator cabin), boom 
and a bucket rigging structure, as illustrated in Fig. 59.9. 
The house rotates on a base known as the tub which 
rests on the ground. The mechanism has three degrees 
of freedom: 

1. Rotation with respect to the tub; 

2. Hoist by a cable passing over sheaves at the tip of 

the boom; 
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3. Drag by a cable passing over sheaves at the base of 

the boom. 

Digging is controlled using only the drag and hoist 
ropes. When the bucket is filled it is hoisted clear of the 
ground and swung to the dump position by swinging the 
house and boom. The human operator fills the bucket 
by dragging it through the ground, lifts and swings it 
to the spoil pile, dumps it, and then returns to the dig 
point. Each cycle moves up to 1001 of overburden, and 
typically takes 60 s. 

High capital costs are pushing the owners of these 
machines to continually look for avenues to maxi¬ 
mize productivity (m 3 / shift) in the presence of operator 
variability and maintenance costs (around 30% of op¬ 
erating cost) [59.2]. From this perspective, dragline 
operations seem ripe for deploying robotic technolo¬ 
gies. However, the idea of completely removing the 
human operator from the dragline seat, while tempting 
from a research perspective, it is not widely entertained 
in the industry. 

Swing control and digital terrain mapping are the 
two robotic technologies at the core of the dragline au¬ 
tomation efforts. During dragline operation, the bucket 
naturally tends to swing with respect to the boom. 
A skilled operator can coordinate the boom slew mo¬ 
tion so as to control the natural tendency of the bucket 
to swing during dumping and spotting operations. 
A novice operator typically requires six months training 
to become proficient. This warrants the development 
of a swing automation system to improve the produc¬ 
tivity of the dig-to-dump and dump-to-dig phases of 


a dragline operation. Automatic swing controllers of¬ 
fer the additional benefit of minimizing stress fatigue 
on the boom. 

An implementation of a dragline swing controller 
is discussed in [59.57]. The control system moves 
the bucket between operator-specified dig, and dump 
points, passing through intermediate (via) points. The 
rather rigid programming of control (dig, via, and 
dump) points has a number of disadvantages. Firstly, as 
the spoil pile grows the dump point must be raised. If 
the automation system does not contain a model of spoil 
growth, the operator has to make adjustments by re¬ 
training the system, resulting in time losses. Secondly, 
if the operator does not correctly identify a via point 
at a critical area, such as the top of the high wall, the 
swing automation system will not know the high-wall 
is there and may attempt to swing the bucket through it. 
Augmenting the automation system with a digital ter¬ 
rain mapping capability provides a common solution to 
these problems. 

One implementation of a digital terrain mapping so¬ 
lution for draglines is described in [59.58]. Maps were 
created using data from an real-time kinematic (RTK) 
GNSS system and a two-dimensional (2-D) laser scan¬ 
ner. The laser scanner was mounted at boom tip in such 
a way that the scanning plane intersected the ground un¬ 
derneath the boom, and tilted by about 7 0 with respect 
to the vertical. Tilting the laser minimizes the likeli¬ 
hood of the bucket and ropes blocking the laser’s view. 
The terrain is scanned as the dragline swings during 
the normal operation cycle, a typical map is shown in 
Fig. 59.10. 
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Fig. 59.10 Image of a digital terrain map created from 
a long range laser scanner installed at the tip of a boom 
dragline (courtesy of CSIRO, 2005) 


The implementation of a digital terrain mapping 
function in dragline automation systems also enables 
the measurement of dragline productivity in real time. 
Commercial dragline monitoring systems can only 
measure the approximate weight of each bucket, and 
the approximate disengage and dump points. They can¬ 
not provide indications of where the where the spoil 
is located in relation to the dragline, of the actual vol¬ 
ume of overburden moved, or how much re-handle took 
place [59.58]. 

59.2.9 Machine Positioning 
and Terrain Mapping 

The task of establishing and maintaining the vehicle 
positioning (i. e., position and orientation) relative to 
an external frame of reference is a fundamental prob¬ 
lem in autonomous vehicle navigation and control, and 
this hold true for mining applications. High-accuracy 
positioning technology is a dynamic area of research 
and development, spurred on by the advances in satel¬ 
lite positioning as well as by the current challenges 
for the technology in mining systems, especially in 
surface mining operations. The following initiatives 
are underway to improve the availability and robust¬ 
ness of current technologies, which have and will 
further enable the application of robotics in surface 
mining: 

• Public GNSS systems (e.g., GPS and GLS 
(global navigation satellite system)) 

• Augmented public GNSS systems (e.g., GPS with 
WAAS (wide area augmentation system)) 

• Locally augmented GNSS systems (e.g., DGPS 
(differential global positioning system) or RTK 
GPS) 


• Pseudolite augmented GNSS systems (i. e., through 

the installation of Earth-based transceivers) 

• Closed systems (i. e., wholly proprietary systems 

relying on a constellation of locally installed Earth- 

based pseudolites). 

For example, in the mining domain, Leica Geosys¬ 
tems offers the Jigsaw Positioning System (JPS), 
a product that offers a significant breakthrough in ad¬ 
dressing the major shortcomings of GNSS [59.59]. 
Based on a technology developed by the Locata Cor¬ 
poration [59.60], JPS is a ground-based network for 
positioning that in principle mirrors the functionality of 
GNSS. It circumvents the fallacies of GNSS by using 
a terrestrial network of transmitters that can be placed 
in optimal locations with respect to the mobiles being 
tracked. 

Different from pseudolites, an example of tech¬ 
nology for positioning in GNSS-denied environments 
is CSIRO’s wireless ad-hoc system for positioning 
(WASP) wireless tracking technology. The system can 
be configured as a mesh network and offers a unique 
combination of high accuracy, high resistance to mul¬ 
tipath interference, low-cost hardware, and ability to 
be rapidly deployed [59.61]. The accuracy of WASP 
depends upon the radio propagation environment, and 
is typically better than 0.25 m under line-of-sight con¬ 
ditions, even in the presence of substantial multipath 
interference. The maximum range over which WASP 
nodes can measure range and communicate is typically 
over 400 m, and can exceed 1 km. As WASP can form 
a mesh network, the overall network size is not limited. 
The maximum location update rate is 200 Hz; however, 
there is a tradeoff between the number of nodes and up¬ 
date rate, and a typical update rate is 1—10 Hz. 

Terrain Mapping and Surveying 
The penetration of robotic and autonomous systems 
in mining, coupled with new computer aided tools for 
mine modeling and reconciliation, requires more accu¬ 
rate and timely information about the current state of 
the mine, and at higher spatial and temporal resolutions. 
In the traditional surveying framework, one obvious so¬ 
lution is to increase the number of surveys (reduce the 
time gap between successive surveys) and the density 
of survey points, while reducing the costs per survey. 
To address this, a number of vehicle-based scanning 
systems have been recently introduced to the market. 
The mode of operation ranges from stop-and-go-type 
systems (such as the Maptek I-Site), to continuous scan¬ 
ning mobile systems, such as those offered by Riegl 
(VMX 250), Topcon (IP-S2), and Trimble (MX-8). In 
general, these systems integrate high-end LIDAR (light 
detection and ranging) with GNSS-aided INS (inertial 
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Fig. 59.11 A 3-D map of a surface mine created from a mo¬ 
bile mapping system installed on a four wheel drive (4WD) 
vehicle (courtesy of CSIRO, 2012) 

navigation system) and carry a significant price tag, 
which can limit deployment to a single dedicated sur¬ 
vey vehicle. 

On the other hand, consider the application of si¬ 
multaneous localization and mapping (SLAM) in min¬ 
ing. SLAM is a mature area in mobile robotics and 
efforts are underway to transfer the technology to min¬ 
ing. Data-fusion techniques used in SLAM make it 
possible to use lower cost mapping sensors, which in 
turn enables mining companies to deploy the equipment 
on a multitude of utility vehicles and/or heavy-duty 
mining equipment (Fig. 59.11). As a result, the vast 
majority of survey work can be performed during the 
normal operation of the equipment in pits, on haul 
roads, or other site areas, without directly exposing 
surveying personnel to the perils of conducting survey 
work in high-risk areas. 

59.2.10 Mine Safety 

The increase in demand for resources has posed new 
challenges to increase productivity, while at the same 



Fig. 59.12 Complexity of mining equipment interactions 
affected by adverse environmental conditions (courtesy of 
CSIRO, 2012) 


time continually improving the standard of safety. 
Robotics is beginning to play a role in this. There are 
inherent risks associated with manually operating ve¬ 
hicles in mining environments. Operators work long 
shifts and the view from the driver’s cabin can be very 
restricted. Furthermore, vehicle interactions are inher¬ 
ently difficult due to the size of machines, and can be 
further complicated by environmental conditions such 
as dust, fog, and snow (Fig. 59.12). 

Each year there are hundreds of mine haulage ac¬ 
cidents that result in significant numbers of injuries, 
even deaths, and large financial costs due to machine 
downtime and repair of equipment. According to the 
United States Mine Safety and Health Administration 
(MSHA) [59.62], from 2002 to 2008 the United States 
coal industry alone averaged 1206 accidents per year 
involving mobile equipment. 

During the past few years, a number of approaches 
have been introduced to assist operators in becom¬ 
ing more aware of safety risks in the local environ¬ 
ment. These technologies and methods include prox¬ 
imity awareness technology (PAT), proximity detection 
technology (PDT) and collision avoidance technology 
(CAT). It is important to note that, in surface mining, 
these existing technologies do not perform any control 
action. They merely provide information to the driver, 
who remains in total control of the vehicle. 

Detection of Threats 

In general, most safety incidents are from vehicle in¬ 
teractions with other vehicles, fixed infrastructure, or 
personnel. A large proportion of these incidents happen 
at low speed. This is most likely to occur on vehi¬ 
cle startup, or at the loading areas with interactions 
between trucks, shovels, loaders, and other ancillary 
equipment. These accidents are usually of low impact 
but can cause serious damage to equipment and signifi¬ 
cant loss of production. 

The fundamental approach to reducing the likeli¬ 
hood of low-speed incidents is to provide the operator 
of large vehicles with aids to see into blind areas 
such at side or back of the vehicle. These aids are 
normally based on standard video or infrared cam¬ 
eras. The next level of risk reduction is achieved by 
incorporating automatic detection of resources and peo¬ 
ple in close proximity by utilizing range and bear¬ 
ing sensors. Radar is a preferred sensor modality 
for bad environmental conditions. Wide beam radars 
are very common but they are prone to false alarm. 
New recent developments mitigate this problem by us¬ 
ing electronic-mechanical scanning narrow beam radar 
technology [59.63,64]. 

High-speed incidents occur less frequently than at 
low speed, but often result in serious injuries and fatal- 
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Fig. 59.13 Graphical representation of the state of the 
world close to the vehicle, including other vehicles in prox¬ 
imity, maximum risk, most likely paths, and risk areas 
(context) (courtesy of Eduardo Nebot, 2011) 

ities. The time available to stop the vehicle or perform 
an evasive maneuver is reduced and consequently long- 
range detection of threats is necessary in order to take 
early preventive action. 

This is addressed by incorporating vehicle-to- 
vehicle (V2V) communication with all nearby vehicles 
at sufficient range to provide early warning of potential 
threats. V2V networks can also form a mesh structure. 
The fundamental advantage of this approach is that no 
fixed infrastructure is required, and communication be¬ 
tween vehicles can be guaranteed anywhere in the site 
when vehicles are in radio range. 

Positioning, Context, and Road Maps 
Positioning is one of the fundamental components of 
a comprehensive safety system. The incorporation of 


GNSS systems, integrated with velocity and inertial 
observations, allows for the evaluation of reliable time- 
to-collision (TTC), especially when combined with 
haulage road maps information. 

Determining the risk of a situation is entirely de¬ 
pendent on the context of the situation. One important 
situation context is the location, or more specifically 
the type of area the vehicle is currently operating. The 
expected behavior and driver intent [59.65, 66] varies 
depending on whether they are driving on a road, park¬ 
ing in a designated area, or dumping ore in a crusher. 
Map information is required to allow the system to vary 
the information provided to the operator depending on 
the location. An example is shown in Fig. 59.13, where 
the map allows the operator to become aware of the 
risks, and complexity of the situation, independent of 
the weather or geometry of the terrain. 

Monitoring and Design 

The ultimate goal of any safety system is to find a com¬ 
plete solution to the safety problem by introducing new 
technology, methods, and algorithms. It is very unlikely 
that this can be achieved with a single technology alone, 
perhaps with the exception of a fully autonomous sys¬ 
tem implementation. An active area of research and 
development aims to use both reactive and proactive 
approaches to solve the safety problem. At the reac¬ 
tive level, technology is used to assist the operator in 
recognizing and avoiding the safety risks. At the proac¬ 
tive level, managers are provided with tools to monitor, 
analyze, design, and improve the safety of the mining 
operation. This new comprehensive approach to safety 
has the potential to eliminate all accidents in the mine 
by integrating technology, procedures and intelligent al¬ 
gorithms to make the mine safer and more productive. 


59.3 Underground Mining 

The especially hazardous and generally unpleasant na¬ 
ture of underground mining environments, operator 
safety and fatigue, labor costs, and the repetitive nature 
of the ubiquitous load-haul-dump cycle all motivate 
autonomous and semi-autonomous and teleoperation- 
based solutions for underground machines and pro¬ 
cesses. 

59.3.1 Telerobotic Operations 

In mining, remote control typically refers to the control 
of a machine where the operator has a direct line-of- 
sight view of the machine. Remote control is normally 
used only for excavation (or mucking), not for driving 


(or tramming) of loaders, for instance. Remote control 
systems are widely used, for example, when loaders 
must access underground drawpoints where there is 
a risk of falling ground. 

By remotely operating equipment or through the 
partially autonomous operation of equipment, for ex¬ 
ample, it stands to reason that the exposure of personnel 
to unsafe circumstances might be significantly reduced. 
This was argued to be true at Inco Ltd. during the 
1990s [59.67]. 

Telerobotic systems are those in which a human re¬ 
mains part of the control loop, although at a distance 
and not generally within direct view of the system be¬ 
ing controlled. Many mining companies and equipment 
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manufacturers are now finding benefit in the application 
of telerobotics, particularly to underground equipment 
such as LHD machines (Fig. 59.1b) and drills [59.67- 
69]. This is normally done by the real-time transmission 
of video and control signals to and from a remote con¬ 
trol room; Figure 59.14, which shows a commercial 
teleoperation station. 

However, teleoperated equipment may not be as 
productive as manually operated equipment, for ex¬ 
ample, due to visibility and issues related to reduced 
operator feedback. Recently, some researchers have 
proposed the development of semi-autonomous tele¬ 
operation systems, whereby the operator’s actions are 
filtered by a machine control system that regulates op¬ 
erator commands with some degree of partial or local 
autonomy [59.70]. 

59.3.2 Autonomous Tramming 

Autonomous tramming refers to automating the driv¬ 
ing task of underground mining vehicles, which is 
often a repetitive activity. Most robotic tramming sys¬ 
tems have been developed for LHD machines, such as 
the one shown in Fig. 59.1b, although some suppliers 
also have systems available for their underground haul 
trucks. 

Several challenges exist that make the autonomous 
tramming problem difficult. For example, the large in¬ 
ertia of these vehicles and their hydraulically actuated, 
center-articulated steering mechanisms make them dif¬ 
ficult to control at high speeds. Another is the problem 
of precise and real-time underground localization in the 
absence of global infrastructure (most notably, GNSS 
available on the surface). 

Underground Navigation 

Early autonomous tramming systems worked by out¬ 
fitting the mine with signal-emitting cables [59.71], 



Fig. 59.14 Teleoperator’s station for control of under¬ 
ground robotic vehicles at the Kemi mine, Finland (cour¬ 
tesy of Atlas Copco, 2009) 


reflective strips [59.72], light-emitting ropes [59.73], 
or reflective tape [59.74,75]. Other systems required 
beacons, placed at strategic locations throughout the 
mine, which allowed the vehicle to estimate its posi¬ 
tion by measuring angles to the beacons and comparing 
the measured angles to those expected from a map of 
known beacon locations [59.76]. 

One drawback to these early systems is the time- 
consuming task of installing, localizing, and maintain¬ 
ing the necessary infrastructure, particularly as the mine 
advances. To the best of our knowledge, none of these 
systems ever saw widespread use in industry. The state 
of the art in underground autonomous vehicle systems 
is infrastructureless. 

One of the first infrastructureless technologies was 
one initially developed by an Inco Ltd. spin off. The 
patent [59.77] describes the use of a map consisting 
of a set of interlinking nodes intended to represent the 
topology of the underground passageway environment. 
The nodes themselves are located at points of interest, 
such as at intersections, dead ends, etc. Thus, a topo¬ 
logical map was used for navigation, which relied on an 
algorithm capable of classifying all of the possible tun¬ 
nel geometries encountered in the mine. Steering the 
vehicle was then done by a lower-level reactive algo¬ 
rithm, which tried to keep the machine near the tunnel 
center. A range-sensing device was used to sense the 
distance to walls. 

This topological and reactive approach to under¬ 
ground navigation and control was tried again af¬ 
ter [59.77]; see also [59.78-81]. Recently, these meth¬ 
ods have also been combined with radio frequency 
identification (RFID) tags to provide a global refer¬ 
ence [59.82]. This general approach is what formed 
the basis for one of the first truly commercial au¬ 
tonomous LHD systems, which is now offer as a prod¬ 
uct by Caterpillar Inc. (see the following). However, 
the main drawback to this technology is that it re¬ 
lies on classifying tunnel and intersection topology. 
As roboticists know, misclassifications can happen in 
practice. 

Inspired by advances in mobile robotics research, 
some considered the application of alternative map- 
based localization algorithms - the idea being to match 
sensor measurements to a map rather than by classify¬ 
ing tunnel topology [59.83,84]. In most cases, a pla¬ 
nar poly-line representation of the mine was used to 
help the vehicle navigate. This is the approach taken 
by Sandvik AB in their commercial offering. More 
recently. Atlas Copco Rock Drills AB entered the un¬ 
derground automation game by applying an approach 
similar in philosophy, which generates sequences of 
occupancy grid maps and subsequently uses these to es¬ 
timate the vehicle’s position in real time by way of an 


Part F | 59.3 








Part F | 59.3 


1564 Part F 


Robots at Work 


unscented Kalman filter (UKF)-based algorithm [59.85, 

86 ], 

Vehicle Control 

A machine control system capable of performing at or 
better than a human operator is necessary in order for 
robotic tramming systems to compete in industry with 
manual operation. Many researchers have proposed ar¬ 
ticulated vehicle steering controllers based solely on ve¬ 
hicle kinematics [59.87-90], while few have considered 
dynamics [59.91]. Others have contended that wheel 
slip is significant and should be explicitly accounted 
for [59.92]. The paper [59.86] describes a control ar¬ 
chitecture and implementation that permits high-speed 
tramming by handling hydraulic vehicle steering and 
driveline dynamics by explicitly forcing a bandwidth 
separation between low-level machine/actuator dynam¬ 
ics and the path-tracking control problem, which is 
solved at the kinematic level. 

Commercial Products 

Although product descriptions are not the primary focus 
of this chapter, it is worth briefly mentioning that there 
are (at the time of writing) three main commercially 
available underground vehicle automation products: 
Caterpillar MineStar system for underground [59.93], 
Sandvik AutoMine [59.94], and the Atlas Copco ST14 
ARV [59.85] 

Mobile equipment vendors, not third-party suppli¬ 
ers, sell all three. All three systems are infrastructure 
free. They all operate by scanning the vehicle’s local 
environment with a 2-D SICK laser rangefinder and 
combining this data with wheel encoder information. 
All three systems operate in conjunction with a high- 
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Fig. 59.15 Scooptram (or LHD) components for robotic 
tramming, including SICK scanning laser rangefinders 
(front and rear), articulation angle encoder, drive shaft en¬ 
coder, onboard cameras, and computing subsystems (cour¬ 
tesy of Atlas Copco, 2009) 


speed wireless network. What is more also all perform 
autonomous dumping. A schematic of the Atlas Copco 
product is shown in Fig. 59.15, highlighting the sensor 
and control components used for robotic tramming. 

59.3.3 Robotic Loading 

Despite a significant amount of research effort in the 
field of robotic excavation, none has yet resulted in the 
widespread development or adoption of autonomous or 
semi-autonomous excavation technologies in the min¬ 
ing industry. Although this section focuses on loading 
from an underground mining perspective, robotic load¬ 
ing is also applicable to surface mining operations 
(Sect. 59.2.3). 

Robotic excavation is especially challenging be¬ 
cause of the dynamic and unpredictable nature of the 
bucket-rock interactions. Also, a dig controller must 
manage not only the motion of the excavation arms 
(e.g., boom and bucket links), but also the penetra¬ 
tion rate as determined by the motion of the mobile 
platform. As a result, performance is highly influenced 
by the conditions of interaction between the machine 
and its environment. For example, the forces that act 
on a bucket as it is actuated to penetrate a rock pile 
may vary significantly depending upon the properties 
of the media (e.g., density and hardness), the rock 
pile geometry, and the distribution of particle sizes and 
shapes [59.95, p. 562]. 

It is important to distinguish between excava¬ 
tion in homogeneous materials, such as soil, sand, 
or regolith, and the challenge of excavating frag¬ 
mented rock as in mining, which usually has a sig¬ 
nificant distribution of particle sizes. Fragmented rock 
may consist of both fines (i. e., very small parti- 



Fig. 59.16 EJC 9T LHD machine conducting robotic ex¬ 
cavation experiments in fragmented rock at the engineer¬ 
ing facilities of Sandvik Mining and Construction Canada, 
Burlington, Ontario (courtesy of Joshua Marshall, 2000) 
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cles), oversize (i. e., particles too large for the exca¬ 
vation tool), and a range of sizes in between. Fig¬ 
ure 59.16 shows an EJC 9T LHD machine con¬ 
ducting robotic digging experiments in fragmented 
rock. 

The robotic loading problem can be split into two 
fundamental tasks [59.96]: 

1. Dig planning 

2. Dig execution and control. 

Dig planning constitutes the problem of deciding 
where and what to dig, possibly considering the geome¬ 
try of the rock pile, the distribution of particles, and the 
physical characteristics of the loader and bucket. Dig 
control refers to algorithms for modifying plans based 
on the nature of the media encountered, so as to effi¬ 
ciently fill the loader’s bucket. This task of the problem 
is particularly challenging in fragmented rock. 

Dig Planning 

Machine vision has been proposed as an enabling tech¬ 
nology for dig planning. The paper [59.97] describes 
a laboratory-scale excavation system that utilized cam¬ 
era data for control and navigation. In [59.98], a scale- 
model system was built to mimic the motions of an 
LHD and different loading strategies were developed 
that depend on sensed information about the rock pile. 
Similar work, described in [59.99], employed a vision 
system to obtain images of the rock pile. In their ap¬ 
proach, these images were used to plan the excavation 
task based on an estimated contour of the rock pile. In 
more recent work, the authors of [59.100] conducted 
an experimental study of two novel approaches for 
selecting an attack pose from 3-D data, with the objec¬ 
tive of eventually using this for the robotic loading of 
sand. 

Dig Execution and Control 
The dig execution and control tasks have been tack¬ 
led by several robotics researchers during the past two 
decades. Much of this work has targeted surface mining 
applications (Sect 59.2.3), although some concepts are 
transferrable to underground. Targeting underground, 
in [59. 101] it was suggested that the trajectory of an ex¬ 
cavator’s bucket through the rock pile should not have 
priority in a devised control scheme, since the objective 
is to effectively fill the bucket, not to follow a prede¬ 
termined path. This was explored further in [59.46, 95], 
which proposed an algorithm for robotic digging that 
would respond to bucket-rock interaction forces sensed 
as changing pressures the excavators bucket cylinder by 
commanding a change in the cylinders retraction ve¬ 
locity, rather than its position. This strategy was based 


on full-scale experiments with an LHD. Together with 
Atlas Copco AB, researchers at Queen’s University 
in Kingston, Canada have recently demonstrated this 
admittance-based robotic loading strategy by way of 
full-scale experiments on a modified Kubota loader (on 
surface) and an ST14 LHD (underground), as shown in 

59.3.4 Longwall Automation 

Longwall mining is one of the main extraction methods 
for underground coal mining. This method is highly ef¬ 
ficient in cases where thick seam reserves are present, 
with high-quality coal at shallow depth and benign 
geological conditions [59.102]. In the longwall pro¬ 
cess, a shearer - a machine with large rotating cutting 
drums - is driven back and forth across the seam, with 
each pass taking a massive slice of coal. 

Figure 59.17 shows a schematic representation of 
a longwall mining equipment installation. Two long, 
horizontal and permanent tunnels known as gate roads 
are cut into a coal seam to form the main boundaries for 
a large rectangular block of coal, known as a longwall 
panel. The shearer travels in this panel along the face 
on a structure also housing an armored (or articulated) 
face conveyor (AFC) that carries the coal to one end 
of the face. The coal is then removed via other convey¬ 
ing systems to the surface. The roof is supported over 
this operation by a number of adjacent powered roof 
supports (also known as shields or chocks) which are 
each connected to the AFC. After the shearer passes, 
the chocks release from the roof, advance under hy¬ 
draulic power into the cavity created by the removal 
of the coal, reposition the AFC, and re-support the 
roof [59.2]. 



Fig. 59.17 Schematic of a longwall mining equipment in¬ 
stallation (courtesy of CSIRO, 2008) 
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A concerted effort to deliver automation solutions 
for longwall mining saw the establishment of the Long- 
wall Automation Steering Committee (LASC) in 2000, 
comprising Australian Coal Association Research Pro¬ 
gram (ACARP) industry representatives, equipment 
manufacturers, research providers and mine safety au¬ 
thorities. With funding from ACARP, CSIRO and the 
Cooperative Research Centre for Mining Technology 
and Equipment (CMTE), now CRC Mining, embarked 
in a major project to deliver automation solutions for 
coal cutting and loading, maintaining face geometry 
and manipulating roof supports without human inter¬ 
vention. 

Three issues were identified as critical for longwall 
automation [59.103]: Face alignment, horizon control, 
and creep control. Until recently, efforts to develop au¬ 
tomatic controllers for face alignment were hindered 
by the lack of an automatic and reliable method for 
measuring the actual geometric profile of the long¬ 
wall face. The solution provided in this project em¬ 
ployed a high-end inertial navigation system to provide 
accurate shearer position and attitude information in 
real time. The system is shearer-mounted and is re¬ 
ferred to as the Shearer Position Measurement System 
(SPMS). One of the major outcomes of this project was 
the development of open specifications for the SPMS, 
and as a direct result, all the major longwall equip¬ 
ment manufacturers now offer automatic face alignment 
systems. 

The second significant technological challenge 
tackled by LASC was achieving enhanced horizon 
control (EHC). While face alignment occurs in the 
horizontal plane, horizon control occurs in the verti¬ 
cal plane, and is significantly more difficult. In order 
to maximize the extraction of coal and minimize the 
extraction of waste, the roof and floor cutting hori¬ 
zons need to stay within the coal seam. The EHC 
incorporates two independent horizon controllers: one 
for the floor and one for the roof. The set points for 
the two controllers are generated by an integrated cut 
model, which sits at the core of the EHC. The cut 
model incorporates real-time information from navi¬ 
gation sensors, coal interface detectors, seam tracking 
sensors, as well as off-line geological and geotechnical 
information. 

Finally, creep control refers to maintaining the lat¬ 
eral position of the longwall equipment in the panel. 
As the longwall mining system progresses, the assem¬ 
bly of supports should not creep toward either of the 
two gate roads. The LASC solution measures creep us¬ 
ing laser range finders that scan in a horizontal plane 
in the panel and roadway directions. The creep infor¬ 
mation is provided to the longwall shearer automation 
processing system and generates corrections in the face 


profile in the form of a lead to either of the gate 
ends [59.103]. 

59.3.5 Robotic Explosives Loading 

Many tasks in mining and construction require the driv¬ 
ing of horizontal and inclined tunnels (or drifts) by 
drilling and blasting for rock fragmentation. Currently, 
drilling and blasting also remains the primary method 
for recovering ore from blocks (or stopes ) in under¬ 
ground hard-rock mining. Blasting engineers design 
a blast hole pattern that specifies the size and location 
of the holes in order to achieve the desired rock frag¬ 
mentation and minimize the damage to the rock mass 
left behind. 

Blast hole charging is a two-step operation. In the 
first step, the operator places a primer assembly at the 
end of a flexible hose, inserts the hose into the blast 
hole, and pushes the hose until the primer reaches the 
end of the hole. A primer is a compact device contain¬ 
ing high strength, sensitive explosives used to safely 
initiate the main column in the blast hole at a controlled 
time provided by the detonator. Initiation requires elec¬ 
tric energy, usually delivered by detonating cords. The 
hole pushing is mechanized. In step two, an explosive 
emulsion is pumped into the hole through a hose, the 
hose is retracted as the explosive gradually fills the hole, 
leaving the trailing detonator wire. Once a ring of blast 
holes is filled, the detonator wires are joined and the 
rock is blasted. 

A robotic explosive charging system (RECS) devel¬ 
oped by the CSIRO [59.104] incorporates technologies 
that make the operation of loading holes safer with 
comparable or better productivity than otherwise pos¬ 
sible. The system incorporates mixed teleoperation and 
robotic capabilities that allow the operator to load the 
entire pattern from the comfort and safety of the cab 
of the loading truck. The prototype RECS was de¬ 
veloped for a mechanical manipulator derived from 
a Palfinger truck crane. The system is able to lo¬ 
cate blast holes using the laser rangefinder attached 
to the arm end effector. An automatic arm pose con¬ 
trol function is used to achieve a sweeping motion 
of the arm, during which the laser rangefinder col¬ 
lects images of the tunnel. These images are sub¬ 
sequently processed to extract blast hole locations, 
and presented to the operator via a human-machine 
interface. 

Once the holes are identified in the scan, they are 
matched with the holes present in the blast plan and 
the operator is ready to proceed with the charging. 
The robot arm is sequenced to pick up the primer 
assembly from a magazine, and transfers it to the 
hole. The primer ends up at a pre-set offset from the 
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hole collar. In teleoperation mode the operator uses 
in-cab joysticks to insert the primer and the hose in 
the hole. The video system provides the operator with 
real-time video images of the relative positioning be¬ 
tween the tip of the end effector and the blast hole 
collar. 

In the work of [59.104], hole identification was car¬ 
ried out for 89 and 102 mm blast holes. While the hole 
diameter is used as a parameter in the identification 
algorithm, determining the hole length was not a re¬ 
quirement (this is not to say that in-situ validation of 
drilled versus planned is not important). During scan¬ 
ning, the robotic arm moves the end effector back and 
forth through the tunnel. The end effector is sequen¬ 
tially rotated, such that the laser is capable of scanning 
a full 360° pattern in the tunnel. Successful hole de¬ 
tection is influenced by the relative alignment between 
the laser beam and the hole axis. The larger the mis¬ 
alignment, the greater the chance of generating a false 
negative. Figure 59.18 shows a typical 3-D profile used 
for hole identification. 

Visual servoing requires locating the tool in relation 
to the target hole. Cameras mounted on the end effector 
provide estimates of the location of blast holes with re¬ 
spect to the end effector. The control methodology does 
not require extensive knowledge of the manipulator 
physical parameters, and uses a minimal set of tuning 
parameters, to allow operators to make adjustments if 
and when required. However, finding an adequate so¬ 
lution raises a number of significant challenges when 
using a typical truck crane as a robotic arm. 



I-1-1-1-1-1-1 

0 12 m 


Fig. 59.18 A 3-D tunnel profile generated for identifying 
blast holes in a tunnel (courtesy of CSIRO, 2006) 


59.3.6 Underground Mapping, Surveying, 
and Positioning 

Mapping, surveying, and real-time equipment position¬ 
ing are core services in many industries, and min¬ 
ing is no exception. In surface mining, the advent of 
the satellite-based GPS in the 1980s led to funda¬ 
mental changes in the way everyday operations are 
carried out [59.105], with new applications being de¬ 
veloped even more recently (Sect. 59.2). Based on 
GNSS, accurate site surveys, information systems, and 
robotic tools that enhance safety, improve productivity, 
and cost-saving maintenance operations, for example, 
have become commonplace in surface mining [59.1, 
106], 

However, similar progress has been made more 
slowly in underground mining because no directly com¬ 
parable positioning technology currently exists for the 
accurate and real-time localization of mobile equip¬ 
ment in underground mining operations. This section 
discusses some tools and techniques from the field of 
robotics that have found their way into mining for 
such purposes as surveying, mapping, and equipment 
localization. 

Underground Mapping and Surveying 
At present, manually generated survey maps in min¬ 
ing are used for the design and construction of 
structures, ventilation, power, drainage, haulage plan¬ 
ning, and tracking of development and ore extraction 
progress [59.107]. However, traditional surveying tech¬ 
niques are slow and laborious, involving many man¬ 
ual steps, infrastructure installation (e.g., retroreflective 
markers), and repeated measurements to obtain the nec¬ 
essary accuracy for the given application. Conventional 
survey maps are invariably a coarse approximation 
of the actual mine structure because, while individual 
observations may be accurate, few measurements are 
taken (i. e., poor resolution). Also, surveying is a reg¬ 
ulated practice; thus the use of new techniques from 
robotics requires that current standards be updated. 

Several recent advances out of the robotic mapping 
community show promise for applications in under¬ 
ground mining, and we mention just a select few here 
as examples. Authors from Carnegie Mellon University 
(CMU) [59.108, 109] describe robotic systems devel¬ 
oped for acquiring volumetric maps of underground 
mine workings; specifically abandoned mines where 
ground conditions may be hazardous. SLAM results, 
based on data from 2-D scanning laser rangefinders, 
were reported from data obtained in two mines: a re¬ 
search mine in Bruceton, PA, and an abandoned coal 
mine in Burgettstown, PA. 


Part F | 59.3 





Part F | 59.4 


1568 Part F 


Robots at Work 



Fig. 59.19 3-D SLAM-generated map of a decline at the 
Northparkes mine (after [59.112]); the gaps on the floor 
and ceiling are due to the blindspot between two vertically 
oriented SICK laser rangefinders; electrical cables can be 
seen along the left side wall (courtesy of So Jung Yun, 
2012) 

Some have developed techniques for robotic explo¬ 
ration and topological mapping [59.1 10] by focusing on 
the detection and matching of underground mine tunnel 
intersections as a basis for the creation of topological 
maps. 

New scan registration techniques, such as 
3-D-NDT (three-dimensional normal distributions 
transform) [59.111], have also been conceived specif¬ 
ically for underground mine mapping. Authors from 
the AASS in Sweden compared their results with the 
popular iterative closest point (ICP) algorithm on actual 
mine data. 

Recently, researchers at CSIRO [59.112] demon¬ 
strated a 3-D SLAM solution, consisting of a spin¬ 
ning 2-D scanning laser rangefinder and industrial-grade 
micro-electromechanical system (MEMS) inertial mea¬ 
surement unit (IMU). The system was mounted on 
a site vehicle at the Northparkes mine (Parkes, NSW) 
that continuously acquired data at typical mine driv¬ 
ing speeds. The deployed system mapped over 17 km of 
mine tunnels in under two hours, resulting in a dense 
and accurate geo-referenced 3-D surface model. Fig¬ 
ure 59.19 shows an example of the generated surface 
model. 


Underground Positioning and Tracking 
The satellite-based GPS uses radio frequency sig¬ 
nals that cannot usefully penetrate significant obstacles 
(e.g., rock); see any reference on the fundamentals 
of GPS [59.113,114]. Although high-sensitivity re¬ 
ceivers are being developed [59.114], these will al¬ 
most certainly not function underground. For build¬ 
ings, some have proposed the use of signal re¬ 
peaters [59.115], but this is again not practical for 
most underground mine environments. These chal¬ 
lenges necessitate a different approach for underground 
mines. 

As a result, several alternatives for underground 
mining have appeared in recent years, many of which 
have limited functionality or are, as of yet, unproven. 
A recent summary, found in [59.1 16], presents a survey 
of the most commonly cited approaches; 

• Employing RFID tags for event-based tracking of 
equipment and personnel [59.117]. 

• Use of radio signals (e.g., installed Wi-Fi ac¬ 
cess points) for signal strength-based position¬ 
ing [59.118, 119], which is very difficult in under¬ 
ground mines due to severe multipath and shadow¬ 
ing, as well as non-line-of-sigh propagation and the 
need for too many devices in order to obtain GPS- 
like accuracies. 

• Dead reckoning using inertial sensors (e.g., IMUs) 
and odometry for only short-term machine tracking. 

• Rock-penetrating very low frequency electromag¬ 
netic signals, generally limited to near sur¬ 
face (< 200m) [59.120] with questionable accu¬ 
racy [59.121], 

However, none of these options yet offer a solu¬ 
tion comparable to that provided by GNSS on sur¬ 
face. Another issue is that mining practitioners often 
do not appreciate the technical limitations of these 
approaches [59.116,122]. As a result, some have re¬ 
cently proposed the adaptation of map-based localiza¬ 
tion techniques for underground mining equipment. For 
example, [59.123] describes a method for large-scale 
mapping that employs a combination of RFID tags (as 
landmarks) and occupancy grid mapping specifically 
designed to facilitate real-time underground vehicle lo¬ 
calization. 


59.4 Challenges and Industry Acceptance 


Despite the latest uptake of robotics technologies in 
mining, significant challenges remain. Next, we briefly 
review the most significant ones that are likely to affect 
the diffusion rate of autonomous systems in mining. 


59.4.1 Technological Challenges 

One expression that describes the current situation of 
robotics in mining is islands of automation. This re- 
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fleets the significant integration issues that exist in re¬ 
lation to fully autonomous and mixed autonomous- 
manual/remote equipment. The root cause of the prob¬ 
lem is a lack of standards to which the various equipment 
manufacturers should align. Although steps are now be¬ 
ing taken to develop these standards (Sect. 59.5.3), the 
situation is likely to persist for a number of years. In 
addition to the time it will take to develop the required 
standards, there will be a significant lag at the equipment 
manufacturers’ end, associated with the rolling out of 
new products that align with these new standards. 

Other challenges to the implementation of robotics 
technology in mining include: 

• Reliability, availability, and fail-safe operation with 
graceful failure 

• Ability to sustain productivity at current or better 
levels 

• Mining in extreme environments and difficulties in 
developing the required supporting infrastructure 

• Current design methodologies for mines and mining 
systems are not geared to support robotic systems 

• Deploying systems in the field and the fact that ev¬ 
ery mine is different 

• Provision of technical support, shortfall in skilled 
labor, lack of adequate training 

• Poorly designed or challenging man-machine inter¬ 
faces. 


59.4.2 Socioeconomic Challenges 

The resource industry has a conservative history, and 
the implementation of new technology and processes 
must overcome significant inertia. In general, the at¬ 
titude of people, from those at the working face to 
senior managers and company executives, mirrors the 
attitude of the workforce in the manufacturing in¬ 
dustry in the early days of industrial robotics; i. e., 
skepticism about technology and fear of losing one’s 
job. The generational change is likely to alleviate the 
fear of technology, but robots will be regarded by 
many in the workforce as a competitor for years to 
come. 

Some researchers in human factors also point 
to the over-reliance on the technology by operators 
as a possible outcome of spreading robotic systems 
in the industry. As a common trend across sev¬ 
eral domains, it was noted that automation and new 
technologies can sometimes result in operators en¬ 
gaging in more risky behaviours in automated sys¬ 
tems [59.124], 

Legal issues and insurance are two other factors that 
could create significant headwinds to the rapid prolif¬ 
eration of autonomous systems in mining or in other 
civilian activity for that matter. Actuarial analysis is 
very hard to do for a green field, such as commercial 
autonomous operations. 


59.5 Challenges, Outlook, and Conclusion 


This section briefly summarizes some of the primary 
challenges for robotics in mining, as well as exam¬ 
ines some emerging themes within the field of min¬ 
ing robotics, beginning with those on the Earth and 
followed by a short discussion about planetary explo¬ 
ration and extra-terrestrial mining, all of which will 
rely heavily on robotic tools. The section ends with 
a short conclusion about the future outlook for mining 
robotics. 

59.5.1 Technical Challenges 

Despite the recent uptake of robotics technologies in 
mining, significant challenges remain. One expression 
that could be used to describe the current status of 
robotics in mining is islands of automation. This re¬ 
flects the significant integration issues that exist in 
relation to fully autonomous, as well as mixed au¬ 
tonomous/manual/remote robotic equipment. One spe¬ 
cific challenge, for example, is the lack of stan¬ 
dards to which the various equipment manufacturers 
and technology developers should align. Although, 


steps are now being taken to develop these standards 
(Sect. 59.5.3), it may take some time before equipment 
manufacturers roll out commercial products that adhere 
to these standards. 

Some other key challenges for miming robotics in¬ 
clude: 

• Reliability, availability, and fail-safe operation with 
graceful and safe failure modes for very large equip¬ 
ment and (often) electric or hydraulically actuated 
equipment. 

• Ability to sustain productivity at current or better 
levels when compared with current methods and 
systems. 

• Mining in extreme and unstructured environments 
(e.g., far north, very deep underground, in harsh 
weather) and the difficulties in developing and sup¬ 
porting the necessary infrastructure in these envi¬ 
ronments. 

• Current mining methods and design methodolo¬ 
gies need to evolve so as to inherently incorporate 
robotic systems. 
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• Field robotics deployment challenges and the fact 
that every mine is different. 

• Provision of technical support in remote areas, 
shortages of skilled labor, and the need to re-train 
existing personnel. 

• Developing effective man-machine interfaces. 

• Safely managing the colocation of robotic machines 
and human workers. 

59.5.2 Socio-Economic Challenges 

For the most part, the resource industry has a con¬ 
servative history and the implementation of new tech¬ 
nologies and processes must often overcome significant 
resistance to change. In many cases, the attitudes of 
people - from those at the working face to senior 
managers and company executives - mirror the atti¬ 
tudes of the workforce in the manufacturing industry 
in the early days of industrial robotics; i. e., skepticism 
about technology and fear of losing one’s job. How¬ 
ever, generational changes are likely to alleviate these 
challenges. 

Some researchers in human factors engineering also 
point to the risk of over-reliance on technology by oper¬ 
ators as a possible negative outcome of the proliferation 
of robotic systems. As a common trend across several 
domains, it was noted that automation and new tech¬ 
nologies can sometimes result in operators engaging in 
more risky behaviours [59.124], which would not be 
looked favorably upon in mining. 

59.5.3 Emerging Frontiers 

There are several emerging frontiers for mining 
robotics, from the creation of standards to robots 
that improve safety in mining, and future-looking 
robots for mining asteroids and other extra-terrestrial 
bodies. 

Emerging Standards for Mining Robotics 
and Automation 

Due to the increasing prevalence of automation in 
mining, there is an immediate need to formulate stan¬ 
dards for robotic equipment such that data may be 
exchanged freely among systems developed by differ¬ 
ent manufacturers. Currently, a tension exists between 
manufacturers, for whom it may seem the commer¬ 
cial imperative to design closed vertically integrated 
systems, and the mine operators, who would like any 
system they purchase to interoperate with all existing 
items of equipment. 

There currently exists a data interchange standard 
known as the International Rock Excavation Data Ex¬ 
change Standard (IREDES) [59.125]. This standard 


uses the extensible markup language (XML) to define 
a three-level architecture: 

1. Administration level that defines reusable data ob¬ 
jects such as coordinate systems 

2. Application profile level that covers general infor¬ 
mation for one application purpose 

3. Equipment profile level that has detailed equipment 
specific information. 

The IREDES standard currently covers drill rigs and 
LHD machines, with many prospective application pro¬ 
files under active development. 

There is also an ISO (International Organization for 
Standardization) standard called Earth-moving machin¬ 
ery - Autonomous machine safety (ISO 17757) [59.126] 
currently under active development. However, at the 
time of writing, there is no public information regarding 
this standard apart from its December 2012 approval. 
The standard will likely cover the risks inherent in large 
equipment automation, and common ways that those 
risks can be mitigated through design and/or procedure. 

It is still early days for the standardization of au¬ 
tonomous mining machines, and current efforts are far 
from complete. However, it is a worthy goal, and one 
which will only increase in importance as more mining 
equipment is automated. 

Mine Rescue Robots 

Mining is inherently risky. Although infrequent today, 
many fatalities have occurred in mining and the ne¬ 
cessity of a mine rescue strategy at every operation - 
particularly in underground mining - is clear. So why 
not use robots to assist with mine rescue? 

Although not mainstream, mine rescue robots have 
been under development at a few research institutions, 
including at Sandia National Laboratories where re¬ 
searchers have focused on coal mines under funding 
from the United States National Institute for Occupa¬ 
tional Safety and Health (NIOSH) [59.127], In coal 
mining, the risk of explosions is high and poisonous 
gases may be present, particularly after a disaster. Their 
tracked Gemini Scout mine rescue robot is approxi¬ 
mately 1.2 m long, equipped with an infrared camera 
(to see through smoke and dust) and gas sensors that 
allow rescuers to assess the situation before attempting 
a rescue. 

More about mine rescue robots can be found 
in [59.128], including reference to the first known mine 
rescue robot prototype at the CSIRO, called Num- 
bat [59.129]. 

Robotic Undersea Mining 

There is a vast body of robotics research related to un¬ 
dersea applications and remotely operated underwater 
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vehicles (ROVs) (Chap. 51 of this book is dedicated 
to this topic). However, much of this work does not 
yet directly target undersea mining. Nevertheless, ad¬ 
vances in robotics - specifically computer-based map¬ 
ping, multivehicle coordination, as well as telerobotic 
drilling and excavation - are likely to make ocean floor 
mining a reality in the very near future. 

Although the pursuit of marine mining dates back 
to the 1870s, large-scale operations have not yet been 
realized at least in part due to technological barri¬ 
ers [59.130]. Today, several venture mining compa¬ 
nies, including most notably Vancouver-based Nautilus 
Minerals Inc. (TSE:NUS), are targeting polymetallic 
seafloor massive sulphide deposits. However, these re¬ 
sources are only accessible by using ROVs equipped 
with telerobotic seafloor production tools to disaggre¬ 
gate and collect the rock from the seafloor [59.131]. 
Thus, robotics is poised to play a significant role in un¬ 
dersea mining. 

Planetary Exploration and Mining 
Perhaps the next great frontier for mining is that of plan¬ 
etary exploration and resource extraction, whether for 
in-situ resource utilization (e.g., toward development of 
the Moon) or for return to the Earth (e.g., mining of as¬ 
teroids for valuable materials). 

Several enabling technologies that support au¬ 
tonomous off-world robotic excavation for large-scale 
habitat construction and resource extraction have al¬ 
ready been developed. For example, researchers from 
the National Aeronautics and Space Administration 
(NASA) and CSIRO demonstrated remote excavation 
of a trench as an example of site preparation and re- 
golith mining [59.132]. This remote excavation system 
consisted of a 1 /7-th scale dragline excavator located 
at a CSIRO test facility in Australia, a control inter¬ 
face located at a NASA facility in the Unites States, 
and a communication network between the two. The 
tested use case scenario involved remote initiation of 
terrain mapping, and a click-to-dig user interface that 
allowed the specification of material transfer in terms 
of dig and dump points on the acquired terrain map. 
The terrain map was used to generate a collision-free 


terrain-skimming path from dig to dump. The onboard 
path planner generated a bucket trajectory that included 
engaging the dig face (bank), filling the bucket and dis¬ 
engaging the bank. Tool force sensing was used as an 
input to the planner in order to constrain the dig speed 
and dig trajectory to remain within the operational lim¬ 
its of the machine and ensure adequate material flow 
into the bucket. The system successfully completed 50 
consecutive cycles without any operator intervention. 
The average cycle time was approximately 63 s with the 
entire mission taking 52 min to complete. 

There is also a thriving space exploration commu¬ 
nity, particularly within the mobile robotics community, 
and there is a complete chapter of this book dedi¬ 
cated to space robotics, Chap. 55. However, it is worth 
mentioning here that mining and planetary resource 
extraction have much in common and there is increas¬ 
ing cooperation between planetary geologists, mining 
equipment suppliers, and roboticists in problems re¬ 
lated to Lunar and Mars exploration (e.g., see [59.4, 5, 
133] and references therein for examples). Moreover, 
new commercial ventures, such as the widely publi¬ 
cized Planetary Resources, Inc., have been established 
with the aim of pursuing commercial extra-terrestrial 
resource discovery and utilization. 

59.5.4 Conclusion 

The desire to improve productivity, safety, and lower 
the costs of mining is a key motivator for the use of 
robotics in the mining industry. As highlighted in this 
chapter, several robotics-related advances have been 
made in recent history, for both surface and under¬ 
ground mining environments and, after a long courting 
period with robotics and automation, many mining 
companies are embracing robotics as a credible and 
practical tool. Moreover, with the emergence of field 
robotics as an increasingly popular subfield within the 
robotics research community, as well as the recent for¬ 
mation of networks, centers, and institutes that focus 
specifically on field robotics (some on mining), the 
domain of mining robotics appears poised to undergo 
accelerated growth on the Earth, undersea, and beyond. 


Video-References 


Autonomous tramming 

available from http://handbookofrobotics.org/view-chapter/59/videodetails/142 
l-gtfr'II'Jfm Autonomous haulage system 

available from http://handbookofrobotics.org/view-chapter/59/videodetails/145 
|4S>M302BE1 Autonomous loading of fragmented rock 

available from http://handbookofrobotics.org/view-chapter/59/videodetails/718 
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60. Disaster Robotics 


Robin R. Murphy, Satoshi Tadokoro, Alexander Kleiner 


Rescue robots have been used in at least 28 dis¬ 
asters in six countries since the first deployment 
to the 9/11 World Trade Center collapse. All types 
of robots have been used (land, sea, and aerial) 
and for all phases of a disaster (prevention, re¬ 
sponse, and recovery). This chapter will cover the 
basic characteristics of disasters and their impact 
on robotic design, and describe the robots ac¬ 
tually used in disasters to date, with a special 
focus on Fukushima Daiichi, which is providing 
a rich proving ground for robotics. The chapter cov¬ 
ers promising robot designs (e.g., snakes, legged 
locomotion) and concepts (e.g., robot teams or 
swarms, sensor networks), as well as progress 
and open issues in autonomy. The methods of 
evaluation in benchmarks for rescue robotics are 
discussed and the chapter concludes with a dis¬ 
cussion of the fundamental problems and open 
issues facing rescue robotics, and their evolution 
from an interesting idea to widespread adoption. 
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Robots serve as extensions of emergency profession¬ 
als. During the immediate response to a disaster, robots 
can provide real-time data about the event and per¬ 
haps even directly aid in inserting mitigation devices 


and sensors, turning valves, and moving equipment. As 
the immediate life-saving and stabilization phase of the 
disaster ends, robots can help with the economic re¬ 
covery. Robots for disaster are an emerging technology, 
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and have not yet been widely adopted by the interna¬ 
tional emergency response community. However, as of 
2012, they have been used in 28 disasters in Cyprus, 
Haiti, Italy, Japan, New Zealand, and the United States 
primarily for mining disasters and structural collapses 
or infrastructure damage. Robots have been used for 
hazardous materials accidents, often referred to as 
chemical, biological, nuclear, radiological, or explosive 
(CBRNE) events, most recently the Fukushima Daiichi 
nuclear accident. Both structural and CBRNE events 
are generally handled by urban search and rescue teams 
unless an industry-specific team assumes command. As 
a result of the breadth of the term urban search and res¬ 
cue contributes to a misperception that disaster robotics 
is limited to urban or constructed environments. For ex¬ 
ample, although technically an urban event, wilderness 
search and rescue is often conducted by a local urban 
search and rescue team. In general, disaster robotics 
connotes extraordinary situations requiring special op¬ 
erations; fire fighting is not considered as part of disas¬ 
ter robotics as it is a routine emergency. 

Disaster robotics can contribute to broad area 
activities, local incidents, and routine emergencies, 


60.1 Overview 

Disaster response is always a race against time, to move 
as fast as possible to reach all potential survivors and 
yet move slowly enough to avoid creating additional 
collapses, damage, risk to rescuers and victims, or con¬ 
tention over airspace. The primary motivation is to save 
lives; robots can assist in meeting this goal either by 
interacting directly with victims or structures or au¬ 
tomating support activities. 

60.1.1 Motivation 

Disasters are defined as a discrete meteorological, geo¬ 
logical, or manmade event, that exceeds local resources 
to respond and contain. Disasters are distinct from an 
ongoing condition such as global warming. In practice, 
a disaster means that an agency has to engage additional 
specially trained experts and equipment. For example, 
a chemical spill may be routine and handled with exist¬ 
ing personnel trained in hazardous materials or it may 
be horrific, such as the 1984 Bhopal disaster that killed 
thousands of people. 

Disaster activities are often referred to by phases, 
though there is no single accepted model in emergency 
management. After an event, public officials are re¬ 
sponsible for the life-saving response and economic 
recovery activities focused on maximizing benefit to the 


though this chapter will focus on the use of robots 
for disaster events. Interest in robotics for survey¬ 
ing wildland fires and flooding is emerging, and the 
few documented uses of robots are reported in this 
chapter. Robots are also being increased use in lo¬ 
cal incidents which do not qualify as true disas¬ 
ters. For example, several fire rescue departments in 
Japan and the United Kingdom routinely use small 
underwater robots for water-based search and recov¬ 
ery of drowned victims and the interest in the use 
of aerial vehicles for wilderness search and rescue is 
growing. 

Disaster robots have not been widely adopted, but 
their use is increasing as the technology matures. The 
general lack of adoption is to be expected since the tech¬ 
nology is new and the concept of operations for these 
novel technologies take time to evolve. Rescue appli¬ 
cations are often similar enough to military operations 
that the same platforms can be adapted; however, some 
tasks are significantly different than their military coun¬ 
terpart, some are unique to rescue, and the human-robot 
interaction for civilian response diverges from military 
patterns of use. 


largest population. Nongovernmental agencies may join 
with other officials and citizens in humanitarian relief 
efforts which are generally focused on benefiting indi¬ 
viduals. Emergency professionals are also tasked with 
prevention of disasters and preparation should there be 
sufficient warning. 

Disaster response activities are typically executed 
by the urban search and rescue (USAR) task force 
within a fire rescue agency who work with law en¬ 
forcement, transportation officials, and others. USAR 
is often used synonymously with any response activ¬ 
ity. The term urban can be misleading as it sounds like 
it is restricted to building collapses. USAR includes 
hazardous material response, as these materials most 
certainly have something to do with built structures. 
USAR includes wilderness search and rescue which of¬ 
ten involves searching tunnels and abandoned mines, 
avalanches, not just locating lost hikers. It also includes 
water-based search and rescue which deals with sav¬ 
ing victims of floods or high currents (also known as 
swift water rescue) or in the aftermath of traffic ac¬ 
cidents where victims are trapped in cars that have 
plunged into a river or bay. Water-based search and 
rescue does not necessarily have urban structures but 
there is a similarity of rescue techniques and re-use of 
equipment. 
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Historically, disaster robotics concentrated on the 
response phase of a disaster. The disaster robotics com¬ 
munity began to form in 1995 as the outcome of the 
tragic loss of life in the Hanshin-Awajii earthquake 
in Kobe, Japan, and the bombing of the Murrah Fed¬ 
eral Building in Oklahoma City, United States [60.1, 
2], In both cases, it was easy to imagine how small 
ground robots may have been able to enter rubble to 
find trapped victims. As a result, research efforts began 
in individual laboratories. Two mobile robot competi¬ 
tions (the AAAI (Association for the Advancement of 
Artificial Intelligence) mobile robot competition in the 
United States, and the RoboCup rescue league interna¬ 
tionally) were started in the late 1990s to engage the 
scientific community in rescue research. The first use of 
robots for response was at the 2001 World Trade Center 
(WTC) disaster, where ground robots were used to pen¬ 
etrate the deeply packed rubble. Since then robots have 
been used for response to earthquakes, hurricanes, mine 
collapses, bridge collapses, explosions, and flooding. 

The deployment of disaster robots during the re¬ 
sponse phase offers many potential benefits. The 2010 
World Disasters report [60.3] suggests just how many 
lives have been, and will be, impacted by urban disas¬ 
ters. Over 1.1 million people were reported killed from 
2000 to 2009, with the total amount of disaster-related 
damage estimated at 986.7 billion US dollars. Of the 
victims in urban disasters, only a small fraction may 
actually survive. Consider from [60.4,5] that the ma¬ 
jority of survivors (80%) of urban disasters are surface 
victims, that is, the people lying on the surface of the 
rubble or readily visible. However, only 20% of sur¬ 
vivors of urban disasters come from the interior of the 
rubble, yet the interior is often where the majority of 
victims are located, providing motivation for robots that 
can explore deep within collapses. The mortality rate 
increases and peaks after 48 h, meaning that survivors 
who are not extricated in the first 48 h after the event are 
unlikely to survive beyond a few weeks in the hospital. 

However, disaster robotics is no longer focused on 
response but has expanded to recovery. Recovery ac¬ 
tivities consist of extricating any remaining bodies of 
victims, re-establishing normal operations in the com¬ 
munity, continuity of government, and resumption of 
the economy. The activities are typically carried out 
by various local, provincial, state, and federal agencies 
as well as insurers and banks. Each municipality may 
have different priorities in rebuilding schools, hospitals, 
ports, and industry and there is usually no centralized 
authority. 

The benefits of disaster robotics for the recovery 
phase were seen in the ongoing aftermath of 2011 
Tohoku earthquake and related Fukushima Daiichi nu¬ 
clear accident. The tsunami generated by the earthquake 


swept thousands of people into the sea. It also severely 
damaged many fishing ports and scattered debris and 
pollutants throughout fishing areas. Teams of roboti¬ 
cists used underwater robots to search for victims and 
to help re-open ports and clear fishing beds. The de¬ 
commissioning of the Fukushima Daiichi facility will 
make use of robots, possibly some of those used in the 
response. 

Disaster robotics is also addressing prevention and 
preparedness. As with recovery operations, prevention 
and preparedness activities are similarly distributed 
among many groups. It should be noted that the fire 
rescue and law enforcement teams will be the first re¬ 
sponders and whatever equipment they use locally will 
be likely to be used at a disaster. Therefore, the dis¬ 
aster robotics community has begun pushing for the 
adoption of robots into daily use by fire rescue and 
bomb squads so that the responders will be familiar and 
comfortable with the technology [60.6]. A lesson from 
Fukushima was that robots built by the nuclear industry 
in the aftermath of Chernobyl and Three Mile Island 
had not been used in decades, with no one expert in 
their use. 

60.1.2 Disaster Robot Tasks 

While the overall motivation for rescue robotics is to 
save lives, the motivation for specific robot designs and 
capabilities depend on their potential tasks. The types 
of tasks that have been proposed for rescue robots are 
described below. Some of the above tasks are similar to 
tasks for military robots, especially search and recon¬ 
naissance and mapping, but many are unique or have 
a different flavor. For example, structural inspection, 
rubble removal, and adaptively shoring rubble are res¬ 
cue specific. Tasks such as casualty extraction appear 
to be similar but are significantly different. Consider 
that a wounded soldier is unlikely to have a spinal cord 
injury and is likely to be in a space large enough for 
a human to work in, so a robot entering the area and 
dragging the soldier to safety is appropriate. However, 
a crushed victim of a building collapse is physically 
trapped or pinned in a small space, requiring rubble 
removal, and the victim’s spinal cord must be immobi¬ 
lized before extraction; clearly victim extraction in the 
search-and-rescue domain is more challenging. 

Search is a concentrated activity in the interior of 
a structure, in caves or tunnels, or wilderness and aims 
to find a victim or potential hazards. The motivation for 
the search task is speed and completeness without in¬ 
creasing risk to victims or rescuers. 

Reconnaissance and mapping is broader than 
search. It provides responders with general situation 
awareness and creates a reference of the destroyed en- 
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vironment. The goal is speedy coverage of a large area 
of interest at the appropriate resolution. 

Rubble removal can be expedited by robotic ma¬ 
chinery or exoskeletons. The motivation is to move 
heavier rubble faster than could be done manually, but 
with a smaller footprint than that of a traditional con¬ 
struction crane. 

Structural inspection may be either conducted on 
the interior (e.g., to help rescuers understand the nature 
of the rubble in order prevent secondary collapses that 
may further injure survivors) or on the exterior (e.g., to 
determine whether a structure is safe to enter). Robots 
provide a means of getting structural sensor payloads 
closer and in far more favorable viewing angles. 

In situ medical assessment and intervention are 
needed to permit doctors and paramedics to interact ver¬ 
bally with victims, visually inspect the victim or apply 
diagnostic sensors, or to provide life support by trans¬ 
porting fluids and medication through narrow tubing. 
The objective is to provide telepresence for medical 
personnel during the 4-10 h that it usually takes to 
extricate a victim [60.4,5]. The lack of medical inter¬ 
vention was a major problem at the Oklahoma City 
bombing [60.7]. Preliminary work has been conducted 
for victim management, most notably in telemedicine 
(also called medical reachback) [60.8], general assess¬ 
ment and triage [60.9], and comforting trapped victims 
as per the Survivor Buddy project [60.10]. 

Medically sensitive extrication and evacuation of 
casualties may be needed to help provide medical as¬ 
sistance while victims are still in the disaster area, also 
known as the hot zone. In the case of a chemical, bio¬ 
logical, or radiological event, the number of victims is 
expected to exceed the number that can be carried out 
by human rescuers in their highly restrictive protective 
gear; this makes robot carriers attractive. Since medi¬ 
cal doctors may not be permitted inside the hot zone, 
which can extend for kilometers, robot carriers that 
support telemedicine may be of huge benefit. Remov¬ 
ing survivors, particularly from a hazardous material 
event where there may be many people immobilized 
or disoriented is another area that is getting increasing 
attention. The US Army Telemedicine and Advanced 
Technology Research Center has been leading initia¬ 
tives in robots such as the Vecna BEAR for autonomous 
casualty extraction [60.11]. Other efforts such as those 
by Yim and Laucharoen [60.12] have focused on civil¬ 
ian casualty extraction which requires stabilization of 
the spine. 

Acting as a mobile beacon or repeater to extend 
wireless communication ranges, enable localization of 
personnel based on radio signal transmissions by pro¬ 
viding more receivers, and to serve as landmarks to 
allow rescuers to localize themselves. Active or passive 


transponders such as wireless sensor nodes and RFIDs 
(radio-frequency identification) can be deployed in the 
field. They can also be used to support the localization 
of personnel, but even more to leave mission-related 
data such as the location of nearby victims and hazards 
in the field [60.13]. 

Serving as a surrogate for a team member, such as 
a safety officer or a logistics person. In this task, the 
robot works side-by-side with rescuers, for example, 
a group breaching rubble deep within the interior of 
a disaster may have difficulty using a radio to request 
additional resources because of noise. However, a team 
member outside of the rubble can see and hear through 
the robot the state of progress and anticipate needs. The 
objective is to use robots to speed up and reduce the 
demands of tasks, even if they are done by humans. 

Adaptively shoring unstable rubble to expedite the 
extrication process. Rubble removal is often hindered 
by the need to adopt a conservative pace in order to 
prevent a secondary collapse that might further injure 
a trapped survivor. 

Providing logistics support by automating the trans¬ 
portation of equipment and supplies from storage areas 
to teams or distribution points within the hot zone. 

60.1.3 Types of Disaster Robots 

Rescue robots are needed to help quickly locate, as¬ 
sess, stabilize, and extricate victims who cannot be 
easily reached. They typically do this by extending the 
rescuers’ ability to see and act. On the ground, small un¬ 
manned ground vehicles (UGVs) can enable rescuers to 
find and interact with trapped victims in voids that are 
too small or too dangerous for human or canine entry. 
Large UGVs can accomplish tasks such as removing 
large rubble faster than humans. In the air, unmanned 
aerial vehicles (UAVs) robots extend the senses of the 
responders by providing a bird’s eye view of the situa¬ 
tion. In the water, unmanned marine vehicles (UMVs), 
either underwater vehicles (UUVs) or surface vehicles 
(US Vs) robots can similarly extend and enhance the 
rescuers’ senses. 

Rescue robots can be broadly categorized into types 
based on modality and size [60.14], though other tax¬ 
onomies that mix modality, size, and task have been 
proposed [60.15]. There are three modalities of robots: 
ground, aerial, and marine. The modality impacts on 
the basic design and capabilities of the robot. Within 
each modality, rescue robots can be further described 
as one of three sizes: man-packable, man-portable, and 
maxi. The size of the robot impacts both on the tasks 
for which it is suited and how soon after a disaster it 
might be used. In order to be man-packable, the en¬ 
tire robot system, including the control unit, batteries. 
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and tools, must fit into one or two backpacks. Man- 
packable robots are more likely to be used immediately 
after a disaster since they can be carried by responders 
over debris and up and down ladders into the core of 
the disaster, while larger equipment must wait for paths 
to be carved out. The next larger size is man-portable; 
these are robots that can be carried a short distance 
by two people or on a small all-terrain vehicle. Man- 
portable robots function in human-sized spaces such as 
mine tunnels. They can be used as accessibility within 
the hot zone improves or outside the hot zone for logis¬ 
tics support. Maxi-sized robots require trailers or other 
special transportation logistics. This inhibits their inser¬ 
tion into the hot zone. 

Aerial rescue robots represent the most advanced 
robotics technology in use, and new concepts continue 
to emerge. Aerial vehicles can be further subdivided 
into fixed wing (plane-like), rotary wing (helicopters). 


lighter-than-air (blimp), and tethered (kite) platforms. 
Fixed-wing UAVs typically travel long distances and 
can circle points of interest, while rotary-wing plat¬ 
forms can hover and require a small area to launch and 
land. Lighter-than-air vehicles may be tethered, similar 
to a kite. Underwater vehicles may also be subdivided 
into tethered and untethered platforms. 

Large unmanned helicopters, such as the Yamaha 
R-Max, continue to be adapted for commercial rescue- 
and-recovery missions, including carrying heavy pay- 
loads capable of estimating the amount of rubble and 
debris generated by a disaster. Small fixed-wing plat¬ 
forms for tactical military use may have great benefit to 
the response community. However, UAVs are drawing 
the attention of agencies that control airspace and may 
become heavily regulated in the future. For example, 
the small UAV used at Haiti in 2010 flew in violation of 
airspace restrictions. 
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The type of disaster as well as the general pattern of 
activity influence the choice of robot platforms and pay- 
loads. 

60.2.1 Natural Disasters 

Natural disasters, such as earthquakes, tsunamis, hurri¬ 
canes and typhoons, volcanoes, avalanches, landslides, 
and floods, present many challenges for rescue robots. 
Natural disasters are usually geographically distributed, 
perhaps affecting a 200 km or more radius around the 
epicenter of the event. The sheer size of the affected 
area presents many challenges to the emergency re¬ 
sponse. The primary impact of natural disasters is on 
residences, light commercial buildings, sea walls and 
canals, and transportation and communications infras¬ 
tructure. This implies that rescuers have thousands of 
structures to check quickly for survivors, but those 
structures will be fairly small and amenable to man¬ 
ual and canine search. Besides the sheer volume of 
structures to check, communication disruptions prevent 
rescuers from getting timely information as to the state 
of transportation access and the general needs of an 
area. However, designing robots to meet these chal¬ 
lenges is important because natural disasters provide 
the most hope of a large number of survivors. Unin¬ 
jured survivors may simply be stranded and can survive 
for up to 72 h. Natural disasters have generally favored 
a bird’s eye view from a UAV invaluable in establish¬ 
ing situation awareness or UMVs in determining hidden 
conditions of critical transportation, power, and com¬ 
munication infrastructure. 


60.2.2 Manmade Disasters 

In comparison to natural disasters, manmade disasters 
(such as a terrorist bombing or an industrial accident) 
occur in a small area, though may have significant 
wide-area impacts (e.g., radiation, chemical release). 
The challenge is often not how to see the entire ex¬ 
ternal extent of the damage, but rather to see what is 
not visible: the interior of the rubble, the location and 
condition of survivors, the state of potentially danger¬ 
ous utilities (e.g., electricity, gas lines) [60.14, 16, 17], 
and hazardous conditions such as radiation [60. 1 8]. The 
communications and power infrastructure usually exist 
within a 10 km range and cell phones generally work 
outside of the directly affected area. Voids in the rubble 
may be irregular in shape and vertical in orientation. 
Wireless communications in the interior of the rub¬ 
ble are unpredictable, and generally nonexistent due to 
the large amount of steel within commercial structures, 
but the combination of irregular voids and sharp rub¬ 
ble do not favor the use of fiber optic cables. Visibility 
is difficult as there is no lighting and everything may 
be covered with layers of gray dust, further hampering 
recognition of victims, potential hazards, or accurate 
mapping. The interiors may be wet or contain standing 
water due to water lines, sewers, and sprinkler systems. 
Survivors are more likely to be in dire need of medi¬ 
cal attention. Small ground robots that can enter deep 
into the interior of the rubble have been used most fre¬ 
quently for structural collapses, mid-sized robots for 
CBRNE events, and large, intrinsically safe robots for 
mine collapses. 
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60.2.3 General Pattern of Activity During 
the Response Phase 

In order to understand how to apply robotics to disas¬ 
ter response (as opposed to recovery or prevention and 
preparedness), it is helpful to understand the general 
pattern of activity, which can be summarized as [60.19, 
20 ]: 

1. Responders become aware of the existence of vic¬ 
tims. This awareness may be generated by informa¬ 
tion from family, neighbors, and colleagues, an un¬ 
derstanding of demographic patterns (e.g., at night, 
apartment buildings will be heavily occupied, while 
during a work day, office buildings will be occu¬ 
pied), or by a systematic search. 

2. The response command staff attempt to understand 
the disaster site. They investigate the site for con¬ 
ditions such as hazardous materials, the risk posed 
to the rescuers themselves, any pending threats 
to trapped victims, and resource restrictions such 
as barriers to transporting resources to the site, 


nearby usable equipment and materials that can be 
exploited, and any other barriers to a timely res¬ 
cue. 

3. The command staff plans the operations. 

4. Search and reconnaissance teams are sent to map 
the situation and assess environmental conditions. 
Accurate estimates of the need for emergency med¬ 
ical intervention are highly desirable in order to 
optimize allocation of medication personnel in the 
field and to prepare ambulances and hospitals. (In 
the case of the Kobe earthquake, this stage took the 
longest time [60.19].) 

5. Excavation of rubble to extract victims begins. Note 
that removal of rubble in search and rescue differs 
from construction removal of rubble, because the 
safety of the victims is the top priority. 

6. Responders gain access to victims and apply emer¬ 
gency medicine in situ. 

7. Victims are transferred to hospitals. 

8. Field teams report activities periodically, usually at 
the end of the shift, and the command staff modifies 
or replans accordingly. 


60.3 Robots Actually Used at Disasters 


Small land, aerial, or marine vehicles have been used 
at 28 disasters and numerous local events, starting with 
the first use of UGVs at the 2001 World Trade Cen¬ 
ter collapse, see Table 60.1. The majority, 16, have 
taken place in the United States, followed by Japan (3), 
Italy (3), New Zealand (2), Haiti (1), Thailand (1), 
Cyrus (1), and Canada (1). The May 2013 rescue by 
a UAV guiding searchers to an injured motorist who 
had wandered from his car in Canada is the first re¬ 
ported live save of a person by a robot. However, as 
noted earlier, directly saving lives is not the only mo¬ 
tivation for a robot. Robots have received in general 
high marks for completing their missions. Table 60.1 
shows 37 reported deployments of robots for a disas¬ 
ter or well-known local incident. The deployments are 
grouped by major general disaster type of event: meteo¬ 
rological (e.g.. hurricane, earthquake), geological event 
(e.g., earthquake, tsunami), man-made (e.g., terrorism, 
industrial accident), or mining (either due to a geologi¬ 
cal or a man-made event). Mine and manmade disasters 
are most frequent, followed by geological and mete¬ 
orological events. In 7 of the 37 events, robots were 
present but were not used or could not reach the area 
of interest and thus considered not successful. UGVs 
have been used most frequently, followed by UAVs, 
and UMVs. While the USA initially led in reported 
adoptions, Europe and Asia are beginning to deploy 


robots. This section discusses the deployments by the 
four disaster types, while the following section high¬ 
lights the 2011 Fukushima Daiichi nuclear accident as it 
highlights the types of robots, missions, and challenges 
posed during both the response and recovery phases of 
a disaster. 

60.3.1 Meteorological Events 

Robots were used at six meteorological events and were 
successful at five: 

• 2005 Hurricane Katrina (USA), where an Inuktun 
ASR Xtreme UGV [60.21], an Aerovironment 
Raven fixed wing UAV and a Like90 helicopter 
UAV, and a Silver Fox fixed wing UAV were 
used during the initial response phase for search, 
reconnaissance and mapping, and structural in¬ 
spection [60.22]. Later an iSENSYS IP3 was 
used for structural inspection during the recovery 
phase [60.23]. The UGV searched an apartment 
building unsafe for human entry and the UAV s sur¬ 
veyed areas inaccessible by car. The two UAVs used 
during the response are shown in Fig. 60.1 - a bat¬ 
tery-powered fixed-wing and a battery-powered 
rotary-wing (a Like90 T-Rex miniature helicopter 
modified for stability in high winds) UAVs. 
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Table 60.1 The modalities and numbers of robots deployed to actual disasters and whether they were successful or not 


Disaster or notable incident 

Meteoro¬ 

logical 

Disaster type of event 
Geo- Manmade 

logical 

Mining 

R 

Ground 

obots use 

Aerial 

d 

Marine 

Success 

2001 

World Trade Center (USA) 



✓ 


4 



/ 

2001 

Jim Walters No. 5 mine (USA) 




✓ 

1 



X 

2002 

Barrick Gold Dee mine (USA) 




✓ 

1 



/ 

2004 

Brown’s Fork mine (USA) 




✓ 

1 



/ 

2004 

Niigati Chuetsu earthquake 
(Japan) 


✓ 




1 


/ 

2004 

Hurricane Charley (USA) 

/ 




1 



X 

2004 

Excel #3 Mine (USA) 




✓ 

1 



/ 

2005 

DR No. 1 mine (USA) 




/ 

1 



/ 

2005 

McClane Canyon mine (USA) 




✓ 

1 



/ 

2005 

La Conchita mudslides (USA) 


✓ 



1 



X 

2005 

Hurricane Katrina (USA) 

X 




1 

3 


/ 

2005 

Hurricane Wilma (USA) 

/ 





1 

1 

/ 

2006 

Sago mine (USA) 




✓ 

1 



X 

2006 

California wildfires (USA) 

/ 





1 


/ 

2007 

Midas gold mine (USA) 




/ 

2 



/ 

2007 

Crandall Canyon mine (USA) 




✓ 

1 



/ 

2007 

1-35 Minnesota bridge collapse 
(USA) 



✓ 




2 

/ 

2007 

Berkman Plaza II collapse 
(USA) 



✓ 


2 

1 


/ 

2008 

Hurricane Ike (USA) 

/ 






1 

/ 

2009 

Cologne State Archives collapse 
(Germany) 



✓ 


2 



X 

2009 

L’aquila earthquake (Italy) 


/ 




1 


/ 

2010 

Haiti earthquake (Haiti) 


✓ 




1 

1 

/ 

2010 

Wangjialing coal mine (China) 




/ 

1 



X 

2010 

Upper Big Branch mine (USA) 




✓ 

1 



X 

2010 

Deepwater Horizon (USA) 



✓ 




16 

/ 

2010 

Prospect Towers (USA) 



✓ 


2 



/ 

2010 

Missing balloonists (Italy) 



✓ 




1 

/ 

2010 

Pike River mine (NZ) 




✓ 

2 



/ 

2011 

Christchurch earthquake (NZ) 


/ 



1 

1 


/ 

2011 

Tohoku earthquake (Japan) 


/ 



3 

1 


/ 

2011 

Tohoku tsunami (Japan) 


✓ 





9 

/ 

2011 

Fukushima nuclear emergency 
(Japan) 



✓ 


7 

2 


/ 

2011 

Naval base explosion (Cyprus) 



✓ 



2 


/ 

2011 

Thailand floods (Thailand) 

/ 





2 


/ 

2012 

Costa Concordia (Italy) 



✓ 




2 

/ 

2012 

Finale Emilia earthquake (Italy) 


/ 




2 


/ 

2012 

Missing person RCMP (Canada) 



✓ 



1 


/ 

Total 

37 

6 

8 

11 

12 

23 

13 

8 

37 


• 2005 Hurricane Wilma, Ft. Myers, Florida (USA), 
where an AEOS unmanned marine surface ve¬ 
hicle (Fig. 60.2) and an iSENSYS T-Rex un¬ 
manned aerial vehicle were used independently and 
cooperatively, to assist with the recovery phase 
for structural inspection of buildings, docks, and 
piers [60.24]. The robots provided useful infor¬ 


mation but the USV experienced significant wire¬ 
less communications and GPS (global position¬ 
ing system) signals near and underneath large 
structures. 

• 2006-2010 California wildfires (USA), where the 
National Aeronautics and Space Administration 
Ikhana Global Hawk was used to provide recon- 
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Fig.60.1a,b Man-packable UAVs used to search por¬ 
tions of Mississippi during the hurricane Katrina re¬ 
sponse: (a) an Aerovironment Raven fixed-wing UAV 
and (b) an iSENSYS IP3 rotary-wing UAV (courtesy of 
CRASAR) 


Fig. 60.2 Unmanned surface vehicle exploring the bridge 
to Marco Island, Florida (courtesy of CRASAR) 


naissance and mapping of 57 fires [60.25]. Ikhana 
was primarily used to gather data to develop new 
payloads but did contribute in 2010 to tactical 
response. 

• 2008 Hurricane Ike, Rollover Pass Bridge, Texas 
(USA), where an AEOS unmanned marine sur¬ 
face vehicle, a VideoRay remotely operated vehicle 
(ROV), and a YSI Oceanmapper autonomous un¬ 
derwater vehicle (AUV) were used for structural 
inspection of the Rollover Pass bridge [60.26]. The 
USV was able to confirm the lack of scour around 
the pilings while the VideoRay could not be con¬ 
trolled in the high currents and the YSI AUV could 
not accurately preplan its path without colliding 
with the structure. 

• 2011 Thailand floods (Thailand), where two un¬ 
manned UAVs provided reconnaissance and map¬ 
ping of flood plains, allowing officials to concen¬ 
trate evacuation or shoring efforts during the pre¬ 
vention and preparedness phases [60.27]. The UAVs 
are credited with mitigating damage to Bangkok 
and loss of life. 

UGVs were present at 2004 Hurricane Charley 
(USA) but not used for the house-to-house inspec¬ 
tion tasks that made up the bulk of ground opera¬ 
tions [60.28]. 

60.3.2 Geological Events 

Robots have been used at eight geological events (6 
earthquakes, 1 landslide, 1 tsunami). They were suc¬ 
cessful at seven events: 

• 2004 Niigata-Chuetsu Earthquake, where a variant 
of the Soryu III snake robot was used for search in 
collapsed houses. The snake robot was able to move 
into constricted passageways. Figure 60.3 shows the 
robot on site. 


Fig. 60.3 IRS Soryu robot searching a house destroyed by 
the Niigata-Chuetsu earthquake (courtesy IRS) 
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Fig.60.4a-c Investigation of damaged building by Quince 



Fig. 60.5 (a) Docking system 
of Quince and Pelican, and (b) 3-D 
(three-dimensional) map measurement 
by collaboration 



Fig.60.6a-c Finale Emilia earthquake: (a) the UGV platform deployed in the destroyed cathedral, (b) Reconstruction of 
the cathedral from laser measurements, (c) and image data 


• 2009 L’aquila earthquake, L’aquila (Italy), where 
a customized AscTec quadrotor UAV was used for 
structural inspection. The UAV was able to fly suc¬ 
cessfully close to buildings and windows. 

• 2010 Haiti earthquake (Haiti), where a SeaBotix 
LBV ROV was used for underwater reconnaissance 
and mapping of debris in shipping lanes and an El- 
bit Skylark UAV was used for reconnaissance and 
mapping of orphanages. Both the robots provide 
valuable information. 

• 2011 Christchurch earthquake, Christchurch (New 
Zealand), where an iRobot Packbot UGV and a Par¬ 
rot AR.drone UAV were used for structural inspec¬ 
tion of the Christchurch catholic cathedral [60.29]. 
The Packbot was successful but the UAV’s camera 


did not face upward and the ceiling could not be ex¬ 
amined. 

• 2011 Tohoku earthquake, Sendai (Japan), where 
a KOHGA3 and Quince UGVs (Fig. 60.4) and a Ke- 
naf and Quince UGVs were used cooperatively with 
a Pelican UAV [60.30] (Fig. 60.5) for structural 
inspection of damaged buildings. The robots were 
able to provide information to structural engineers. 

• 2011 Tohoku tsunami (Japan), where nine UMVs 
were used for reconnaissance and mapping, and 
victim recovery during the recovery phase. Five 
different teams deployed UMV: A SeaBotix SAR- 
bot, SeaBotix LBV-300, SeaMor, and Access AC- 
ROV were deployed by a joint IRS-CRASAR team 
in April 2011 and later again in October adding 
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Fig. 60.7 Robots available at World Trade Center disaster. 
Ovals indicate the three types of robots used in the first two 
weeks (courtesy CRASAR) 


a YSI Oceanmapper [60.31]; the IRS-CRASAR 
team was credited with reopening the port of Mi- 
namisanriku. Ura from Tokyo University fielded an 
unnamed ROV which recovered two dead bodies 
from an undisclosed location [60.32], Prof. Shi- 
geo Hirose at Tokyo Institute of Technology also 
created an experimental ROV and deployed with 
the Japanese Ground Self-Defense Forces [60.32], 
IDEA Consultants (Japan) deployed their Mitsui 
ROV. Shibuya Diving Industry bought and used 
a VideoRay Pro 4, though the extent of the use is 
not known [60.33]. All UMVs were ROVs with the 
exception of the YSI Oceanmapper. 

• 2012 Finale Emilia earthquake (Italy), where two 
custom unmanned aerial vehicles and two UGVs 
were used at Mirandola, Italy, for structural in¬ 
spection of the exteriors and interiors of two 
churches that had not been entered due to safety 
reasons [60.34] (Fig. 60.6). The robots were able 
to provide information to the Italian National Fire 
Corps and state archaeologistics. 

An Inunktun VGTV Xtreme UGV was unsuccess¬ 
fully deployed at the La Conchita mudslide to search 
for possible survivors in houses collaterally damaged 
by the mudslide [60.35]. The robot failed on its two runs 
within two minutes and four minutes, respectively, due 
to severe mud and thick shag carpeting. 

60.3.3 Manmade Events 

Robots were at nine manmade events. They were suc¬ 
cessful at eight events: 

• 2001 World Trade Center collapse (USA) [60.36], 
where Inuktun Micro-Tracks, Inuktun Micro- 
VGTV, Foster-Miller mini-Talon, and Foster-Miller 


Talon were used for search and reconnaissance and 
mapping in the interiors of the collapsed buildings 
(Fig. 60.7). The robots were able to penetrate farther 
than nonrobotic tools in spaces too small for human 
or canine entry. 

• 2007 1-35 Bridge collapse, Minneapolis, Min¬ 
nesota (USA) [60.37,38], where two unspecified 
unmanned marine vehicles were used for search 
and reconnaissance and mapping underwater. The 
ROVs appeared to provide views without risking 
human divers in the fast moving currents. 

• 2007 Berkman Plaza II parking garage collapse, 
Jacksonville, Florida (USA) [60.39,40], where an 
Inuktun Micro-VGTV was used during the rescue 
phase for search and during the recovery phase an 
Active Scope Camera and an Inuktun ASR Xtreme 
were used for structural inspection along with an 
iSENSYS IP3 UAV for reconnaissance and map¬ 
ping and structural inspection. The Micro-VGTV 
ruled out the presence of a survivor in areas too dan¬ 
gerous for a human to enter. The ASC and Xtreme 
provided structural forensic information that re¬ 
solved legal liability and insurance coverage costs. 
The UAV provided imagery that led structural ex¬ 
perts to hypothesize the cause of the collapse. 

• 2010 Prospect Towers parking garage collapse, 
Hackensack, New Jersey (USA) [60.22], where an 
Inuktun VersaTrax 100 and an Inuktun VGTV- 
Xtreme were deployed for search. The robots were 
used to see vehicle information, license plates, 
makes and models in areas not safe for responders. 

• 2011 Fukushima Daiichi nuclear accident 
(Japan) [60.18,41], where seven UGVs and 
two UAVs were used for search, reconnaissance 
and mapping, and structural inspection. The robots 
provided damage assessment and radiological 
surveys without exposing workers to radiation. 
A UGV and a UAV were lost during the first year, 
but only after they provided important information. 

• 2011 Evangelos Florakis naval base explosion 
(Cyprus) [60.17], where an AscTec Falcon and 
AscTec Hummingbird were used to inspect the 
damage to the adjacent Vasilikospower plant with¬ 
out requiring engineers to risk exposure to live 
unexploded ordinance. The UAVs were used for 
structural inspection. The operations were success¬ 
ful and led to a rapid repair of the power plant. 

• 2012 Costa Concordia (Italy) [60.42,43], where 
an Ageotec Perseo ROV and a VideoRay ROV, 
possible other unreported ROVs, were used for re¬ 
connaissance and mapping, structural inspections, 
and for victim recovery operations from the sunken 
cruise liner. The ROVs were considered essential in 
understanding the situation under the water. 
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• 2013 Missing person Royal Canadian Mounted Po¬ 
lice (Canada) [60.44], where a Draganflyer X4-ES 
UAV with a thermal camera was used by the Royal 
Canadian Mounted Police to find an unconscious 
driver who had wandered from a car wreck. This 
is considered the first reported live save by a robot. 

Robots were requested and present, but not used at 
the 2009 State Archives Building Collapse in Cologne, 
Germany, due in part to operator safety [60.45]. 

60.3.4 Mine Disasters 

Robots were at 12 mining disasters. They were success¬ 
ful at seven events: 

• 2002 Barrick Gold Dee mine, Elko, Nevada; 2004 
Excel #3 mine, Pikesville, Kentucky; 2005 DR#1 
mine, McClure, Virginia; 2005 McClane Canyon 
mine, Grand Junction, Colorado (USA), where 



Fig. 60.8 The mine-permissible variant of the Wolverine 
robot used at the 2006 Sago Mine Disaster (courtesy of the 
US Mine Health and Safety Administration) 



Fig. 60.9 The Inuktun VGTV-Xtreme robot being low¬ 
ered into the Midas gold mine (courtesy of CRASAR) 


a mine-permissible Remotec Wolverine (Fig. 60.8) 
was deployed the US Mine Safety and Health Ad¬ 
ministration (MSHA) used by to reopen mines that 
had been closed [60.46]. At the Barrick Gold Dee 
mine, the V-2 was deployed from the surface down 
a 16° slope and was able to navigate and to take 
continuous gas samples. At the Alliance Resources’ 
Partners Excel #3 (coal) mine the robot was able 
to penetrate 230 m into the mine and successfully 
completed the objective of providing an assessment 
of the situation. At the DR#1 Dixon-Russell (coal) 
mine the robot was able to penetrate 210 to 250 m 
into the mine with a slope of 18° and the robot arm 
was used to move and realign ceiling supports in 
order to progress into the mine. At the McClane 
Canyon (coal) mine trials were conducted to estab¬ 
lish manipulation capabilities. In this case, the robot 
was tasked to close five doors and pull out timbers 
holding up a mine fan. The robot was generally un¬ 
successful with manipulation tasks. 

• 2007 Midas gold mine, Midas, Nevada (USA), 
where an Allen-Vanguard from Fallon Naval Air 
Station and a Inuktun VGTV-Xtreme variant from 
the Center for Robot-Assisted Search and Rescue 
(CRASAR) were lowered into the mine and used for 
victim recovery [60.47], Mine-permissible robots 
were not needed as the gold mine was not emitting 
methane. The Allen-Vanguard that was lowered into 
the void and found the machinery but not the victim. 
The Xtreme was able to penetrate 35 m vertically 
into the void (Fig. 60.9). The robot scanned the area 
where the body was recovered but did not have suf¬ 
ficient lighting to actually see the body. 

• 2007 Crandall Canyon mine, Huntington, Utah 
(USA), where a custom Inuktun mine Cavern 
Crawler robot was deployed by US mine Safety and 



Fig. 60.10 The Inuktun mine crawler robot being lowered 
into a borehole at the Crandall Canyon mine (courtesy of 
CRASAR) 
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Health Administration to search for missing miners 
(Fig. 60.10) [60.47]. The robot had to travel over 
430 m through a 22 cm diameter borehole. Four at¬ 
tempts were made to enter the mine through two 
different boreholes, but only one was successful. In 
the fourth run, the robot was able to travel through 
the borehole then search about 2 m on the mine floor 
which was largely impassable due to the debris and 
drilling tailings. The robot provided no sign of the 
miners. 

• 2010 Pike River coal mine, Greymouth (New 
Zealand), where two unknown New Zealand De¬ 
fence Force bomb squad robots and Western Aus¬ 
tralia Water company pipeline inspection robot 
were used for search and recovery operations. The 
first New Zealand Defence Force (NZDF) robot 
ailed after reaching 550 m into the mine due to 
falling water, but was successfully restarted and 
moved out of the way of the second robot before 
running out of battery power. The first and second 
robots were presumed destroyed in a second explo¬ 
sion that ended any expectations of survivors. The 


pipeline inspection robot was later used for recov¬ 
ery operations. 

Robots were unsuccessful at two events. At the 
2004 Browns Fork mine, Hazard, Kentucky, disaster, 
the Wolverine robot was too tall to go into the area of 
interest to conduct a search. At the 2006 Sago mine Dis¬ 
aster, Sago, West Virginia (USA), the robot was only 
able to penetrate about 700 m into the mine before being 
damaged when it accidentally was driven off the path 
and flipped over [60.46]. 

Robots were on-site but not used at three events: 
the 2001 Jim Walters #5 mine fire (USA) as the 
robot was not mine-permissible, the 2010 Wangjialing 
coal mine (China) presumably because the unidentified 
robot could not transit high water, and the 2010 Upper 
Big Branch mine (USA) where the restricted confines 
posed too great a risk of failure. Robots were requested 
for the 2010 San Jose Copper-Gold mine (Chile) and 
the 2011 San Juan De Sabinas coal mine (Mexico) dis¬ 
asters but robots were not small enough and thus were 
not sent. 


60.4 Robots at the Fukushima-Daiichi Nuclear Power Plant Accident 


The Fukushima-Daiichi nuclear power plant accident 
merits special attention because robots have been used 
for both the response and recovery phases. The use of 
robots have been sustained over months and years, and 
a wide variety of robots have been used. 

Control over the reactors at the Fukushima-Daiichi 
nuclear power plant was lost on March 11, 2011, 
after a 14 m high tsunami from the Eastern Japan 
Earthquake submerged the facility. Units 1-3 melted 
down, hydrogen exploded at the high floors of nu¬ 
clear reactor buildings of Units 1, 3, and 4, and huge 
amounts of radioactive materials were released to the 
wide area centered by Fukushima Prefecture. Robots 
were used during the prolonged response phase for 
assessment of the emergency situation and mitiga¬ 
tion of the event by recovering the cooling systems 
to enable a cool shutdown. After four months, the 
immediate response turned to preparation for decom¬ 
mission as the reactors were stabilized. The recovery 
phase requires operations in the contaminated area; 
therefore, many robotic systems have been, and are 
continuing to be, used for minimizing workers’ ex¬ 
posure to radiation by substituting for human opera¬ 
tions. In the first year, ground and aerial robots were 
used for exterior operations outside the buildings and 
ground robots for interior operations inside the reactor 
buildings. 


60.4.1 Exterior Operations 

Robots were used outside the buildings and on the 
grounds by different teams for four applications: debris 
removal, structural inspection, radiological surveys, and 
mitigation efforts. 

Debris Removal 

by Unmanned Construction Machines 
There was a large amount of contaminated debris 
outside the building producing a high dose rate of 
a few hundred mSv/h, preventing workers accessing 
the buildings. Debris removal operations were con¬ 
ducted starting April 1, 2011 for nine months using 
an adaptation of unmanned construction robot ma¬ 
chines had been developed for the restoration work 
after the Unzen Fugen-dake Volcano eruption in 1991 
(Fig. 60.11). Debris was gathered by two backhoes and 
a bulldozer, with wireless teleoperation, then loaded 
to containers on two remote-controlled crawler dump 
trucks, and carried to storage yards of contaminated 
materials. The crawler dump trucks were teleoperated 
from a remote operator car, with command and control 
signals and data from camera and sensors transmitted 
via a remote-controlled radio relay car. Seven camera 
cars were connected to the system by wire, and ex¬ 
ternal image views supported the task execution. As 
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Fig.60.11a,b Unmanned construction machines used at Fukushima-Daiichi (courtesy of Unmanned Construction Ma¬ 
chines Association) 


a result, debris of 20 000 m 3 was removed from the area 
56 000 m 2 wide, and the radiation level reduced signifi¬ 
cantly to safely accessible level for the workers. 

Structural Assessment and Radiological 

Surveys from T-Hawk UAV 
A team led by Westinghouse used Honeywell T- 
Hawks from April 10, 2011, to the end of July, 2011 
(Fig. 60.12). The team flew approximately 40 missions 
with two objectives originally. One was to investigate 
the state of the reactor and turbine buildings and to 
acquire video of the general area in order to support 
initial assessments of the incident and to help plan for 
debris removal. The second was to take radiological 
samples to produce a survey map. The objectives later 
expanded to off-site debris field survey, survey of the 
coastal breakwater structure, a gamma radiation sur¬ 
vey at specific locations of interest to TEPCO, and 
airborne-particulate sampling (flying into plumes) at se¬ 
lect locations. 

Radiation Source Measurement 

by Gamma Cameras 

The Japan Atomic Energy Agency (JAEA) developed 
a robot control car TEAM NIPPON with a gamma cam¬ 
era, a 3-D camera, a thermo camera, and a QinetiQ 
Talon robot that were provided by Idaho National Lab¬ 
oratory, a US federal laboratory that conducts research 
in nuclear reactors, in collaboration with Tohoku Uni¬ 
versity. It was used from May 5, 2011 for measuring 
radiation of the outdoor debris, and provided visualized 
data as shown in Fig. 60.13 [60.18,41]. 

Remote Control of Concrete Pumping Cars 
Concrete pumping cars had been used for pouring wa¬ 
ter from outside in order to cool used-fuel pools on the 


top floor of the exploded nuclear reactor buildings. In 
order to reduce the radiation exposure of workers who 
control their long arms, the cars were fitted for remote 
teleoperation by using remote controllers, cameras, and 
wireless equipment on May 2011. 

60.4.2 Interior Operations 

in Nuclear Reactor Buildings 

UGVs continue to be used for operations in the inte¬ 
rior of the reactor buildings. UGVs were used to survey 
the interiors (iRobot Packbot, IRS Quince), monitor for 
gamma radiation (JAEA J-3), remove debris (QinetiQ 
Bobcat and Talon, Brokk), and experiment with decon¬ 
tamination (iRobot Warrior). 

Survey of Nuclear Reactor Buildings 

by PackBots 

On April 17, 2011, TEPCO workers and six engineers 
from iRobot used two donated PackBots to open the 
double hatch doors of Units 3 and 1, providing the first 
entry into those buildings (Fig. 60.14). The TEPCO 
workers carried the robots to the front of the hatch, and 
pointed an antenna to the robots from the glass win¬ 
dow of the hatch. Based on the readings of radiation 
dose rate, temperature, humidity, and gas concentra¬ 
tion, humans were allowed to enter the nuclear reactor 
buildings to begin the cool shutdown process. On the 
following day, the same mission was tried in Unit 2, but 
the PackBot could not complete it. The camera window 
of PackBot misted over under high humidity, because 
the vapor from boiled water on the basement floor did 
not escape from the building. Prior to the actual op¬ 
erations, TEPCO workers were intensively trained on 
remote teleoperation starting on March 24, 2011, and 
the scenario was verified in Unit 5, which was not dam- 
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Fig.60.12a,b UAV used at Fukushima-Daiichi: (a) T-Hawk and (b) top view of nuclear reactor building of Unit 1 (pho¬ 
tographs courtesy of TEPCO) 
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Fig.60.13a,b Gamma radiation 
measurement by UGVs: (a) TEAM 
NIPPON and (b) visualized radiation 
source by gamma camera (courtesy of 
JAEA) 



Fig.60.14a,b Operations of PackBots in Unit 1 (courtesy of TEPCO) 


aged. The PackBots continue to be used for monitoring 
and light tasks in the buildings. 

Survey of Nuclear Reactor Buildings by Quince 
The Quince robot was used in addition to the iRobot 
Packbots in June, 2011 (Fig. 60.14), and is now being 
used in cooperation with the Packbots for various mis¬ 
sions as only Quince could climb the steps to the upper 
floors. Quince had been developed for CBRNE disaster 
response by a team from the International Rescue Sys¬ 


tem Institute including Chiba Institute of Technology 
and Tohoku University. One unit of Quince was lent 
to TEPCO for free after radiation-proof test, wireless 
communication test, intensive refinement of reliability, 
and training exercises of operators [60.48]. Quince has 
been used for surveillance of the nuclear reactor build¬ 
ing since June 24, 2011. On July 8, it entered Unit 2 
and measured the dose rate on the second and third 
floors, then sampled dust for identification of nuclide 
(Fig. 60.15). On July 26, valves and pipes of a spray 
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Fig.60.15a,b Quince at Fukushima: (a) robot platform and (b) its operator station image captured in the mission on July 
8, 2011 (courtesy of TEPCO) 


cooling system were visually inspected by the high- 
density camera and dose rate was measured in Unit 3. 
Redundant cooling system was established on the ba¬ 
sis of these data. On October 20, Quince surveyed fully 
on the third floor of Unit 2, then climbed the stairs up 
to the fifth floor (operation floor) for monitoring. The 
high dose rate, condition of the crane, etc. were investi¬ 
gated on the high floors at first. However, it stopped on 


the third floor on its return home because its commu¬ 
nication tether was hooked and cut. If Quince had not 
been applied, it would have been delayed to examine 
situation of the upper floors, and the risk would have 
become higher because of late cool shutdown. Quince 
2 and 3 were remodeled and lent for free of charge to 
TEPCO from 2012, and are used in cooperation with 
PackBots for various missions. 


60.5 Lessons Learned, Challenges, and Novel Approaches 


The experiences with disaster robotics since 2001 pro¬ 
vide key insights into technical challenges, such as 
mobility, communications, sensing, control, and hu¬ 
man factors, but also into sociotechnical issues such 
as training and procedures for transporting and decon¬ 
taminating robots. This section highlights the general 
lessons learned and the fundamental problems posed 
in mobility for UGVs, UAVs, and UMVs, communica¬ 
tions, control, sensors and sensing, power, human-robot 
interaction, multirobot team coordination, and other 
issues. 

60.5.1 Mobility: UGVs 

Mobility remains a major problem for all modalities of 
rescue robots, but especially for ground vehicles. The 
challenges for ground robots stem from the complexity 
of the environment, which is an unpredictable combina¬ 
tion of vertical and horizontal elements with unknown 
surface characteristics and obstacles. For example, at 
Fukushima, the original configuration of the iRobot 
Packbot could not climb the steep metal staircases. In 
other cases, a robot could not cross a catwalk or climb 


stairs due to rubble and the lack of confidence that the 
robot could negotiate the debris. Robots such as the 
Inuktun VGTV-Xtreme at the La Conchita mudslides 
could not handle the terrain, while the Allen-Vanguard 
was too heavy for vertical entry into the Midas gold 
mine. 

The field is currently lacking any useful character¬ 
izations of rubble to facilitate better design. However, 
even without a complete understanding of rubble en¬ 
vironments, it is clear that more work is needed in 
actuation and mechanical design as well as in algo¬ 
rithms that would enable the robot to adapt its mobility 
style to the current terrain (also known as task shap¬ 
ing [60.49]). 

The state of the practice in UGVs for disasters are 
polymorphic tracked vehicles, with a movement toward 
tracked robots with manipulators (e.g., the Wolverine 
used for mine disasters, the robots at Fukushima) and 
two instances of snake-like robots (Niigata-Chuetsu 
earthquake, Berkman Plaza II collapse). Wheeled plat¬ 
forms are severely limited by the roughness of the 
terrain and the need to overcome obstacles, steps, 
ramps, but combination wheeled and tracked vehicles 
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Fig.60.16a,b Examples of legged and crawler robots, (a) 
Hexapod (legged) robot from the RHex project travers¬ 
ing a portion of the National Institute of Standards and 
Technology (NIST) test bed (courtesy of R. Sheh) and (b) 
Terminatorbot being tested in rescue test bed (courtesy of 
R. Voyles) 

are commercially available. Basic research in legged 
robots (Chap. 48), wheeled robots (Chap. 49), mi¬ 
cro/nanorobots (Chap. 27), and manipulators will ben¬ 
efit disaster robotics. 

Free serpentine robots such as the Soryu III used 
at the rubble at the Niigata-Chuetsu earthquake pro¬ 
vide a fundamentally different style of mobility, while 
fixed-based snakes can be used in tandem with a more 
traditional platform as a highly flexible sensor manip¬ 
ulator [60.50]. Examples of both types of snake robots 
are shown in Figs. 60.3 and 60.19. 

In order to overcome the difficulties posed by un¬ 
known terrain, novel legged robots and crawler robots 
have been proposed. In addition, some types of crawler 
robots can climb walls and reach locations that would 
otherwise be very hard to reach. Legs are interesting 
because they exploit biomimetic principles. The RHex 
hexapod robot [60.51], proposed for search and res¬ 
cue among other potential applications, is shown in 


Fig. 60.16 climbing random step fields. Crawler robots 
such as the Terminatorbot [60.52] use their arms or legs 
to pull themselves through the rubble. The Terminator¬ 
bot is designed to withdraw itself into a cylinder that 
can be inserted through one of the small boreholes com¬ 
monly drilled by responders to get through walls, then 
open up and begin moving. Other types of crawlers in¬ 
clude lizard- or gecko-like robots that adhere to walls; 
these types are promising but have not been tested for 
the dusty, wet, and irregular conditions found in disas¬ 
ters. 

Another novel concept for UGVs is that of smart 
tools, particularly lifts that can help stabilize collapsed 
structures during extrication [60.53,54], Extrication is 
one of the most time-consuming activities in rescue. 
Rescuers must proceed cautiously when removing rub¬ 
ble in order to prevent secondary collapses or slides 
that would further injure the survivors. Experiments 
suggest that roboticized lifts or shoring mechanisms 
would be able to sense and respond fast enough to small 
movements in the rubble to adaptively maintain stabil¬ 
ity [60.54], 

60.5.2 Mobility: UAVs 

UAVs are increasing in popularity with responders, 
possibly because on demand aerial access is a unique 
capability. Aerial vehicles, particularly helicopters, are 
vulnerable to wind conditions near (or in) structures 
and obstacles such as power lines, trees, and overhang¬ 
ing debris. A UAV at Hurricane Katrina hit a power 
line. 

Novel ideas include a plane with the size of a per¬ 
son’s hand that can fly indoors and planes with fold- 
able wings, making it easier for responders to carry 
them. Quadrotor helicopters appear far more stable 
and easier to pilot; a design that balanced the larger 
size of a quadrotor with an appropriate payload could 
make UAVs more assessable to nonpilots. Another ex¬ 
citing direction is hybrid platforms that can change 
from fixed-wing operations, covering large distances, 
to rotary-wing operations, flying near or inside build¬ 
ings [60.55]. 

60.5.3 Mobility: UMVs 

Surface and underwater vehicles have to contend with 
swift currents and floating or submerged debris, placing 
significant demands on agility and control. As seen at 
the Tohoku Tsunami, AUVs could not be used due to 
the possibility of colliding with flotsam. Teleoperated 
ROVs can function in such conditions but have a risk 
of tethers becoming tangled or caught, as seen at the 
Tohoku Tsunami and the Hurricane Ike response. 
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60.5.4 Communications 

Robots rely on real-time communications for teleopera¬ 
tion and for enabling responders to see what the robot is 
seeing immediately. Ground robots communicate either 
through a tether or via wireless radio. Aerial and surface 
robots are wireless, while underwater rescue robots are 
controlled via a tether. The communication bandwidth 
demands of all modalities are generally high due to the 
use of video imagery, and the tolerance to communica¬ 
tions latency is low due to the control needs. In addition 
to communications between the tactical rescuers and 
their robots, it is difficult to report or transfer critical 
information provided by rescue robots to the strategic 
enterprise. Disasters typically destroy the communica¬ 
tion infrastructure, both telephones and cellular phones, 
and alternatives such as satellite phones become satu¬ 
rated by response agencies. 

Wireless communications with robots remain prob¬ 
lematic. Operations below ground or near structures 
interfere with the physical propagation of radio signals. 
As shown in high-fidelity USAR response exercises, 
ad hoc wireless networks established by responders 
are likely to become quickly saturated, with no way 
of establishing priority over information. At the World 
Trade Center, data from the Solem robot deployed in 
the interior of building WTC4 returned totally black 
frames for 1 min 40 s of the 7 min run before wireless 
communications were totally lost and the robot was 
abandoned [60.36]. In addition, many wireless robots 
use lossy compression algorithms to manage band¬ 
width, which interfere with computer vision techniques, 
and/or connect through insecure links, raising the pos¬ 
sibility that news media might intercept and broadcast 
sensitive video of trapped victims. 

Rescue robots working underground, either for 
USAR or mine rescue, have two alternatives to wireless 
communications: either operate with a tether or deploy 
repeaters to maintain wireless communications. Many 
wireless robots now can be purchased with a fiber-optic 
tether; however, these tethers are fragile and may break 
or tangle, as seen at several mine disasters. The fiber¬ 
optic tether may also tangle with the safety rope used 
to support the robot during vertical drops. Data for the 
World Trade Center deployments indicated that a dedi¬ 
cated person was required to manage the tether but that 
54% of tether management operations were to allow the 
robot to reach a more favorable position or to recover 
the robot after its mission [60.36]. 

At Fukushima, outside the buildings, allocation of 
wireless frequencies and accommodation of communi¬ 
cation methods were extremely difficult, and usage of 
some robot frequently affected the other robots. Inside 
the buildings, wireless communication was sometimes 


unstable, and a few robots could not return. Operators 
always had to pay attention to radio field strength and 
cable tether handling. 

Majority of UGV operations have used tethered 
robots, with the only wireless robot used at the WTC 
disaster being lost and the wireless NZDF robots cre¬ 
ating significant problems with intermittent failures. 
Hybrid communications, in which a robot is primarily 
on a tether and then operates for short distances over 
a local wireless link before reconnecting to the tether, 
appear to be attractive. 

A novel approach is to use other robots as re¬ 
peaters, either stationary or mobile, to facilitate the 
establishment of communications and sensor networks. 
As repeaters for mobile ad hoc networks, robots on the 
land, sea, and air can extend the range and through¬ 
put of wireless networks [60.56]. A recent US De¬ 
fense Advanced Research Projects Agency program, 
LANdroids, developed a set of small robots that acted 
as mobile repeaters for a larger robot entering the 
interior of a structure or tunnel. Aerial vehicles are 
particularly attractive as they can provide larger relay 
ranges while providing a bird’s eye view of the disaster. 
However, a UAV does not always have to move: a teth¬ 
ered blimp or kite can support a sophisticated payload 
with no maintenance or support for days [60.57, 58]. 

60.5.5 Control 

Robot control can be subdivided into platform control, 
which is usually considered by control theory, and ac¬ 
tivity control, which generally falls under the purview 
of artificial intelligence. Rescue robots are challenging 
both for traditional control and for artificial intelligence. 
The high degree of mechanical complexity of all modal¬ 
ities and the demands of the environment present major 
challenges for control theory. In all report events, the 
robot’s activity was handled by teleoperation; a human 
is needed to direct the robot and to perform mis¬ 
sion sensing. The well-documented problems of manual 
teleoperation (Chap. 34) argue for increasing auton¬ 
omy [60.59], both in terms of navigation. 

Navigational autonomy has been the primary focus 
within the disaster robotics community and efforts gen¬ 
erally fall into the following three categories: 

• Simultaneous localization and mapping (SLAM): 
SLAM is needed for the search and reconnaissance 
and mapping tasks. Whereas SLAM in office-like 
environments appears to be almost solved, the appli¬ 
cation of same techniques in harsh and unstructured 
environments turns out to be extraordinarily diffi¬ 
cult or even impossible. One strong limitation of 
current mapping solutions is their inability to pro- 
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vide an accurate 3-D model of the environment at 
high data rate. For USAR such a solution has to 
cope with the reflection properties of debris, smoke 
due to fires, and the spectrum of lightning condi¬ 
tions. The solution also needs to be low cost in terms 
of required onboard CPU (central processing unit) 
computation and payload, due to weight and power 
limitations. Existing solutions rely either on auto¬ 
matically inclined [60.60, 61] or constantly rotating 
2-D (two-dimensional) laser scanners [60.62]. In 
contrary to 2-D maps, which are typically generated 
in real time [60.63], 3-D maps are updated at low 
rates on the order of seconds due to the limited data 
rate of inclined or rotated laser scanners. 

• Exploration, planning, and path execution'. The 
central question in autonomous exploration is: given 
what you know about the world, where should you 
move next to gain as much as many new information 
as possible? [60.64]. Autonomous exploration can 
be subdivided into determining where to go next, 
how to plan the path to get there, and how to exe¬ 
cute that path safely. 

The primary approach to exploration is to use 
a frontier-based exploration algorithm is to gain 
new information by moving to the boundary be¬ 
tween open space and unknown territory, denoted 
as frontier. Frontier-based exploration, either based 
on 2-D [60.64] or 3-D [60.65] map representations, 
is an efficient technique that has been successfully 
deployed together with SLAM techniques for ex¬ 
ploration in NIST-like USAR arenas [60.66]. 

The objective of path planning in harsh environ¬ 
ments is not necessarily to generate the short¬ 
est but the safest path. Wirth and Pellenz intro¬ 
duced an exploration strategy and path planner 
that utilizes occupancy grid maps for combining 
the distance transform and the obstacle transform 
(when planning to several frontier cells at the same 
time [60.67] in order to select the safest alternative 
consisting of target location and path to reach the 
target. 

Most methods for motion planning in USAR are 
still limited to static environments that can effi¬ 
ciently be represented in 2-D. However, since robots 
operate on rough terrain that may suddenly shift it 
is important to consider the shape of the surround¬ 
ing terrain and its potential impact when moving. 
Several researchers have introduced solutions with 
short-term planning look ahead that execute spe¬ 
cific robot behaviors with respect to the current 
situation of the robot. Okada et al. introduced an 
autonomous controller for tracked vehicles that is 
based on continuous 3-D terrain scanning [60.68]. 
Magid et al. introduced a system for keeping the 


robot maximally stable at every step of its path 
while allowing the vehicle to loose balance in 
a controlled manner for facilitating safe climbing 
over debris [60.69]. Sheh et al. developed a method 
for behavioral cloning, a type of learning by im¬ 
itation that produces control rules that clone the 
skills of an expert human operator [60.70]. Dorn- 
hege et al. introduced the concept of behavior maps, 
which link certain robot behaviors on rough ter¬ 
rain such as climbing over stairs and obstacles, to 
structures detected from 3-D point clouds in real¬ 
time [60.71], 

• Object recognition and scene interpretation: The 
search, reconnaissance and mapping, and structural 
inspection tasks for disaster robots would benefit 
from autonomy. They currently rely on humans to 
manually scan video feeds for signs of survivors, 
reconstruct the scene, identify potential hazards, 
and accurately comprehend the integrity of struc¬ 
tures. To this end, Andriluka et al. evaluated various 
state-of-the-art techniques for vision-based victim 
detection from UAVs [60.72]. They concluded that 
by combining multiple weak models the overall 
detection reliability can be increased. Kleiner and 
Kummerle presented an approach for detecting vic¬ 
tims based on several visual features such as body 
motion, skin color, and body heat [60.73]. Hahn 
et al. presented an approach for improving victim 
detection from low-cost thermal sensors by com¬ 
puting heat distributions while exploring an envi¬ 
ronment [60.74]. Birk et al. developed a system for 
recognizing humans from images taken by an in¬ 
frared camera that considers plausible body shapes 
and postures [60.75]. 

60.5.6 Sensors and Sensing 

Sensors, and sensing, pose the greatest mission chal¬ 
lenge; without adequate sensing, a robot may be in an 
area of interest but be unable to navigate or to execute 
the larger mission. The physical attributes of a sensor 
(size, weight, and power demands) impact on whether it 
can be used with a particular robot platform. Currently 
sensors are not interchangeable between platforms; 
standards are needed for footprint sizes, mounting, con¬ 
nections, and display space. 

The functionality of a sensor depends on the modal¬ 
ity and mission [60.16]. The primary sensor missing 
from all robot modalities operating outside of water is 
a miniature range sensor. With a miniature range sen¬ 
sor, the success in localization and mapping seen with 
larger robots would be transferable to rescue robots. 
The sensor payloads for other missions depend on those 
applications; however, two sensor needs for USAR are 
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particularly noteworthy. One is a detector that can per¬ 
ceive victims obscured by rubble; current radars have 
not been reliable in mixed rubble. Another needed sen¬ 
sor is one that can tell if a victim is unconscious but 
alive without touching the victim. A robot may be able 
to see a victim, yet not be able to crawl to the victim and 
make contact or scrape away enough dirt or clothing 
to take a pulse. Stand-off detectors such as millimeter- 
wave radars and gas detectors appear promising but 
have not been validated at this time. 

Smaller, better sensors are not sufficient; improve¬ 
ments in sensing algorithms are also needed. At this 
time, humans are expected to interpret all sensing data 
manually in real time. This is a daunting task for many 
reasons. Human performance is handicapped by phys¬ 
iological factors introduced by sensing through a com¬ 
puter display (also known as computer mediation), the 
location of sensors at viewpoints low to the ground, 
generally restricted fields of view, and fatigue. The 
modality output itself may also be nonintuitive, such 
as ground-penetrating radar. However, autonomous de¬ 
tection and general scene interpretation is considered 
well beyond the capabilities of computer vision. This 
presents a case where neither the human nor the com¬ 
puter can accomplish the perceptual task reliably and 
argues for investigation into human-computer coopera¬ 
tive techniques for perception. Algorithms that enhance 
the image for human inspection, supplement depth per¬ 
ception, or cue interesting areas are within the reach of 
computer vision. 

60.5.7 Power 

The robot modality and mission poses distinct chal¬ 
lenges for power, although only one robot, the NZDF 
robot at the Pike River mine, has caused a failure. 
In general, battery power is preferred over internal 
combustion because of the logistics difficulties in trans¬ 
porting flammable liquids. While the requirements of 
each rescue robot application is largely unknown, a par¬ 
tial understanding is emerging of the power profile. 
For example, the operation tempo of a ground vehi¬ 
cle operating underground is on the order of 3—4 runs, 
each around 20 min in duration, over a 12—14 h shift, 
with the robot kept on hot standby for the major¬ 
ity of the shift. Rotary-wing aerial vehicles for tacti¬ 
cal reconnaissance and structural inspection show an 
operations tempo of 5—8 min per face of a build¬ 
ing, while fixed-wing vehicles are airborne for less 
than 20 min. Other rescue missions, such as wilder¬ 
ness search and rescue, will have different require¬ 
ments but the need for batteries over internal com¬ 
bustion and for determining the power profile is the 
same. 


60.5.8 Manipulation 

Manipulation has been required for mine remediation 
(e.g., DR#1 and McClane Canyon mines) and at the 
Fukushima nuclear accident, but overall manipulation 
has failed in at least one task per mission. Furthermore, 
manipulation is almost always time consuming as all 
reported instances have been teleoperated. Robots with 
manipulators (such as that seen in Fig. 60. 17) extend the 
capabilities of ground vehicles by allowing the robot to 
sample the environment, interact with survivors, move 
light obscurations, and add unique camera viewpoints. 
However, the manipulator extends the volume of the 
robot, impacting navigation. The arm is often at risk 
of being damaged in confined spaces being hitting on 
overhanging rubble. Manipulators also add to the con¬ 
trol and mechanical complexity of the robot. 



Fig.60.17a,b Examples of tracked robots with manipu¬ 
lator arms that have been used for disasters or rescue 
competitions, (a) View from Foster-Miller Talon at the 
WTC of its arm (courtesy of CRASAR) and (b) teleMAX 
by Telerob at the Rescue Robotics Camp (courtesy of 
R. Sheh) 


Part F | 60.5 



















Part F | 60.5 


1596 Part F 


Robots at Work 


60.5.9 Human-Robot Interaction (HRI) 

According to the analysis by Murphy [60.22], over 
50% of the robot failures during a disaster are due 
to human error, which emphasizes the importance of 
human-robot interaction. For the purposes of this chap¬ 
ter, HRI concerns fall into five topics: ergonomics, 
system design, human-robot ratio, situation awareness, 
and training. 

One aspect of HRI is ergonomics. The Fukushima 
nuclear accident highlighted how personal protection 
equipment (e.g., full-face mask and rubbery gloves) 
precluded workers from performing delicate work and 
efficient task execution. 

Another aspect of HRI is the larger system design. 
At the Fukushima nuclear accident, workers had to 
carry the robots to the point of entry; this sometimes in¬ 
creased the total radiation exposure of workers, which 
was clearly not the purpose of the robot. The inability 
to transport certain robots caused them to be excluded 
from the World Trade Center disaster. 

The number of operators per robot has been the sub¬ 
ject of debate, though Murphy and Burke in [60.76] 
offer a formula based on their comprehensive field stud¬ 
ies for computing the baseline human-to-robot ratio 

‘Wumans — hi vehicles T" N payloads T 1 • 

The idea is to then justify why a smaller ratio would not 
increase unacceptable risk to the mission. For example, 
a UGV often does not need a safety officer, while a UAV 
does under most regulations. One way to decrease the 
number of people is to reduce their workload, such as by 
increasing the navigational autonomy from normal op¬ 
erations. However, Murphy and Burke document that if 
the unmanned system encounters a problem and returns 
control to the operator, the operator is unlikely to be 
able to react fast enough to avoid a crash, collision, or 
severing of a tether. This problem has been documented 
in aviation safety with initial problems with auto-pilots, 
where it is generally referred to as the human out of 
the loop control (OOTL) problem. The OOTL problem 
should not be interpreted as discouraging autonomy but 
rather encouraging a more comprehensive considera¬ 
tion of how autonomy will be handled in all conditions, 
not just normal operations. 

Related to the number of operators is how well they 
can maintain situation awareness. Situation awareness 
is defined by Endsley in [60.77] as 

the perception of the elements in the environment 
within a volume of time and space, the comprehen¬ 
sion of their meaning and the projection of their 
status in the near future. 


Drury et al. [60.78] have defined types of situation 
awareness within search and rescue based on an anal¬ 
ysis of RoboCup rescue tasks, while Casper and Mur¬ 
phy [60.79] and Burke et al. [60.80] have examined 
operator situation awareness in technology insertions. 
User interfaces are a key component in facilitating sit¬ 
uation awareness. In rescue robotics, user interfaces 
are generally primitive and work through the opera¬ 
tor’s visual channel to provide robot, task, and situation 
awareness of the state of the robot, task progress, and 
the general operational environment. To highlight the 
importance of user interfaces, one robot at the World 
Trade Center was rejected because of the complexity 
of its interface [60.36]. While the user interface for 
fieldable rescue robots primarily display the video out¬ 
put from the robot, experiences from RoboCup rescue 
suggest that a good interface will both facilitate com¬ 
manding the robot (inputs) and will provide three types 
of information (outputs): 

• The robot’s perspective: camera view(s) from the 
robot’s current position, plus any environmental 
perceptions that enhance the general impression of 
telepresence 

• Sensor and status information: critical information 
about the robot’s internal state and its external sen¬ 
sors 

• If possible, a map: a bird’s eye view of the robot 
situated in the local environment. 

The Fukushima nuclear accident illustrated the im¬ 
portance of training, as training of operators with using 
simulated mockups took over a month. The current 
state of the practice is for responders to receive train¬ 
ing from the robot manufacturer on how to operate and 
maintain the robot, ignoring concepts of operations for 
actual scenarios. Training is a related issue in human- 
robot interaction [60.81]. Rescue workers have limited 
time compared with military operations to learn about 
robots and few opportunities to practice. While a bomb 
squad or special weapons and tactics team for law en¬ 
forcement may be called out several times each month, 
a rescue team may be called out only a few times each 
year. For the near future, rescue workers may not have 
had prior training or experience with a robot prior to 
the disaster and be expected to use prototypes with only 
hasty training. 

60.5.10 Multirobot Team Coordination 

Multirobot teams of robots that cooperate with each 
other or work on the same objective have already been 
used or demonstrated for response and recovery ac¬ 
tivities at Hurricane Wilma (UMV-UAV) [60.24], the 
Tohoku tsunami (UMV-UMV) [60.31], and the To- 
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hoku earthquake (UAV-UGV) [60.30]. These have all 
been heterogeneous teams though homogeneous teams 
have been entered at the RoboCup rescue competi¬ 
tion [60.82]. Only one of the three deployed teams was 
explicitly coordinated, where a Quince robot carried 
a Pelican UAV at the Tohoku earthquake and con¬ 
ducted cooperative mapping of the interior of a dam¬ 
aged building [60.30]. The current research approach 
to teams is focusing on adopting techniques from ar¬ 
tificial intelligence for either centralized or distributed 
coordination. 

Multirobot teams not only offer the possibility to 
field diverse capabilities, they also exhibit increased ro¬ 
bustness due to redundancy, and superior performance 
by parallel task execution [60.83]. This latter aspect 
is as important as the first one, since the time to 
complete a rescue mission is of vital importance. For 
example, one fundamental problem when searching for 
survivors is to efficiently coordinate a team performing 
space exploration. However, the IRS-CRASAR team 
at the Tohoku tsunami response uncovered implicit as¬ 
sumptions about parallel task execution that limited 
performance [60.31]. The coordination of several au¬ 
tonomous units in search and rescue operations still 
represents an open issue. One reason is the fact that 
the higher number of degrees of freedom imposed by 
the robot team requires a larger number of trained hu¬ 
man operators, which in turn makes the deployment 
during a rescue mission more complicated. Certainty, 
the number of required operators also depends on the 
quality of autonomous perception and decision-making 
of each single system. Therefore, in order to reduce 
the human-to-robot ratio in the field, autonomous ca¬ 
pabilities as well as autonomous methods facilitating 
team coordination are needed. There are several exist¬ 
ing techniques for team coordination in USAR, which 
can generally be separated into centralized and decen¬ 
tralized approaches. 

There are three novel types of teams being pursued 
for disaster robotics. One type is swarms. Researchers 
such as [60.84] have discussed the possibility of us¬ 
ing cost-effective, insect-sized robots to penetrate deep 
within a pancake building collapse and then signal 
the presence of a victim. One of the key features of 
swarm approaches is that they can scale up easily. The 
insect swarm scenario leaves hard problems like con¬ 
trol, sensing payloads, localization of the victim, and 
communications to the imagination, but is certainly 
a worthy concept. However, some of the search algo¬ 
rithms used by insects may be adapted to single robots, 
for example, win-shift win-stay sampling exhibited by 
bees may be useful for search [60.85]. A second type of 
team, similar to swarms, is motes, where aerial vehicles 
that drop intelligent sensors called motes. 


A third type of team is a hybrid robot-animal team, 
where search dogs carry roboticized cameras (dog- 
cams) [60.86] or wiring rats with controllers. Attaching 
cameras to search dogs has been explored for several 
years and such a system was used at the World Trade 
Center. The concept does not compete with robots, as 
robots are used to enter places canines cannot. The 
canine team handlers have generally objected to dog- 
cams because the cameras and communications gear 
interfere with the dog’s mobility, pose the hazard of 
snagging, and the dog cannot be readily commanded 
to stop at points of interest beyond the line-of-sight of 
the handler (dogs use visual cues as commands, more 
than audio). However, there is less objection to plac¬ 
ing nodes in a rat’s brain to stimulate and drive the 
rat into a void while carrying a camera or other sen¬ 
sors [60.87]. The motivation for a robot rat stems in 
part from the rat’s mobility and relative low value. 
While the technical feasibility of a robot rat may be 
within reason, assuming advances in wireless commu¬ 
nications, the response community has been lukewarm 
toward the idea [60.88]. Unlike rats, robots can be 
kept in storage for years and can penetrate through 
pockets of fire or areas with no oxygen. A robot rat 
has all of the limitations of dog, including the prob¬ 
lem of a handler becoming too emotionally attached, 
and is likely to scare a trapped survivor just as much 
as the other rats that swarm a disaster. The conven¬ 
tional wisdom is that, if the sensors, wireless, and 
power systems can be miniaturized and operate reliably 
enough to control a rat deep within rubble, those sys¬ 
tems will enable responders and robots to work without 
the rats. 

60.5.11 Other Issues 

In addition to the challenges by the functional subsys¬ 
tems of a robot, three other issues should be considered. 

Robots must be reliable. As noted in the discussions 
of the Pike River mine explosion and the Fukushima 
nuclear accident, failure of a robot can obstruct the mis¬ 
sion execution or cause it to completely fail. A robot 
failing not only means the mission did not get per¬ 
formed, but could prevent other robots from carrying 
out the mission. For example, a robot getting stuck on 
the stairs would pose a navigational hazard to other 
robots or to workers in bulky safety gear. Such risk has 
to be minimized by thorough analysis and preparation 
beforehand. 

Robots must be suitable for the environment. In 
at least two mine disasters, robots could not be used 
because they were not explosion proof - yet needed 
to be used in an explosive atmosphere. At Fukushima, 
robot operators had to be concerned with potential 
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damage of semiconductors by total dose of radiation, 
which could affect not only CPU and image sensors but 
also various components using semiconductors such as 
engine, battery, etc. Monitoring of the total dose using 
a dosimeter was necessary. 

Robots must be decontaminated. Radiation contam¬ 
ination at Fukushima was a serious issue because it 


60.6 Evaluation 

Disaster robotics is still an emerging field and meth¬ 
ods for evaluating a rescue robot or the larger 
mixed human-robot system are still forming. Murphy 
in [60.22] provides an analysis of the performance of 
disaster robots at 34 events. Evaluation is largely cen¬ 
tered around the US government National Institute of 
Standards and Technology (NIST) efforts to standardize 
rescue robots for adoption by US responders via origi¬ 
nally a standard test course and more recently a set of 
standard test methods. Each standard test method is an 
inexpensive, reproducible prop that tests a single capa¬ 
bility; for example, Fig. 60.18 shows a prop designed 
to test a robot’s ability to traverse rough terrain repre¬ 
sented by a wooden step field. The standard test course 
and methods are duplicated via the USARsim computer 
simulation, and at least two sites, NERVE and SWRI, 
have physical testbeds consisting of the NIST standard 
test methods - though other physical testbeds such as 
Disaster City exist. The test course and methods have 
been used since the late 1990s for the RoboCup res¬ 
cue and AAAI mobile robot competitions. This effort is 
being conducted through ASTM and thus, even though 
it is largely driven by the US, will likely result in an 
international standard that may be adopted by other 
countries. 

60.6.1 Computer Simulations 
for Rescue Robotics 

Computer simulations provide a low-cost mechanism to 
explore the larger behavior or a robot or system. Gen¬ 
erally, computer simulations provide high fidelity for 
testing software execution, but their physical fidelity de¬ 
pends on the physics engine. Simulating sensors and the 
complex environments produced by a disaster is dif¬ 
ficult and is rarely accurate enough to test perceptual 
algorithms. At the time of writing, two readily available 
computer simulations exist for exploring the strategic 
and tactical applications of rescue robots within the 
RoboCup rescue framework [60.89], the RoboCup res¬ 
cue simulation project [60.90], and USARSim [60.91]. 
These simulations are well understood, accepted, open- 


caused radiation exposure of workers in maintenance 
and battery exchange. However, UGVs at the World 
Trade Center collapse were exposed to raw sewage and 
body fluids. Decontamination has been discussed in 
standardization procedures but few platforms are built 
to be completely cleaned or to be easily cleaned with¬ 
out exposing the human. 


source, and free; as such, they should be useful for most 
researchers or practitioners interested in ground-based 
rescue robotics. 

The RoboCup rescue simulation project is used in 
the RoboCup rescue simulation league to study agent- 
based approaches to strategic planning for the disaster 
response. The simulator assumes a strong centralized 
response capability that is not necessarily the case for 
all countries or regions; the United States, for exam¬ 
ple, relies on a highly distributed organization that 
obviates many centralized coordination schemes. Al¬ 
though the simulation is focused on strategic decision 
making, particularly dynamic resource allocation, it 
does support the examination of how robot resources 
might be allocated during a disaster and how data 
from a robot might be propagated through a system. It 
permits the simulation of monitoring of disaster dam¬ 
age from reports by humans, distributed sensors, and 
robots and can simulate complex interactions such as 
telemedicine. 

USARSim is a computer simulation developed by 
the University of Pittsburgh for physical robot simula¬ 
tion in disaster situations [60.91]. The simulation repli¬ 
cates the NIST standard test bed for search and rescue 
and permits efficient prototyping and testing of robot 
design and most aspects of control software. It uses Un¬ 
real game engine for handling physics and graphics, and 
virtual robots have capability of sensing (image, laser 
range finder, etc.) and actuation (wheel, motor, etc.) 
with data processing (image recognition, SLAM, etc.) 
in artificial environments. In 2006, the RoboCup res¬ 
cue competition created a simulation league using this 
environment. 

60.6.2 Physical Test Beds 

Physical test beds provide a more realistic venue than 
a computer simulation for evaluating rescue robots, but 
may not be available to researchers, too expensive to 
use or to travel to, or not adequately capture some key 
aspect of a disaster. Physical test beds generally fall into 
three categories: test beds developed for the fire rescue 
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Fig.60.18a-c View of the NIST standard test bed for 
search and rescue used by RoboCup Rescue, (a) View of 
overall test bed, (b) dummy representing a victim, and (c) 
a step field challenging robot mobility (courtesy NIST) 


community, test beds developed for the robotics com¬ 
munity, and the NIST standard test bed for search and 
rescue. 

Fire rescue training test beds occur throughout the 
world and are used to train human firefighters, res¬ 
cue specialists, and canine teams under highly realistic 
conditions. The Texas A&M Engineering Extension 
Service’s Disaster City complex has 52 ac devoted to 



Fig. 60.19 A CMU (Carnegie Mellon University) fixed- 
base snake being tested at a facility in California (courtesy 
H. Choset) 


representative structural collapses, ranging from multi¬ 
story commercial building to wooden housing. Disaster 
City is the site for the NIST Response Robot Evaluation 
Exercises. The NASA Ames Research Center’s Disaster 
Assistance Response Team facility at Moffett Field has 
hosted several events as well. In general, fire rescue test 
beds are constructed from construction and sewer de¬ 
bris, can introduce smoke and some simulants, and pose 
challenging mobility conditions, but vary in terms of fi¬ 
delity. In many test beds, the density of the debris does 
not contain the actual amount of metals in a real col¬ 
lapse. This can lead to optimistic reports of success of 
sensors and wireless communication devices. The test 
beds, being designed for human training, do not repli¬ 
cate the conditions under which a ground robot would 
be used. The terrain is generally on the exterior of the 
rubble and does not exercise the robot in confined or 
vertical spaces. Depending on the size of the facility, 
the test bed may or may not be suitable for evaluating 
UAVs. An example of a fire-training test bed appeared 
earlier in Fig. 60.19. 

Physical testbeds for the robotics community such 
as the New England Robotics Validation and Experi¬ 
mentation (NERVE) Center at the University of Mas¬ 
sachusetts Lowell, and the facilities at the Southwest 
Research Institute (SWRI) in San Antonio, Texas, in¬ 
corporate both general test courses for robots and the 
NIST standard methods. 

Perhaps the most influential physical simulation 
for researchers is the RoboCup rescue physical league 
which uses the NIST standard test bed for search and 
rescue, shown in Fig. 60.18. This competition started in 
2001 [60.92] and has more than 40 team entries every 
year from all over the world. The RoboCup rescue 
physical league scores robot performance in terms of 
mobility, mapping, situation awareness, sensing, shared 
autonomy, etc. A robot or robot team competes in one 
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of three arenas which simulate disaster situations at 
the annual RoboCup world competition. The mission 
of the robot teams is to collect victim information 
(existence, state, and location) by sensor fusion of 
vital signals (heat, shape, color, motion, sound, CO 2 , 
etc.) and report a map of victims in disaster space 
so that responders can efficiently arrive at the victim 
for rescue. In addition to the arenas, the competition 
and test bed contain individual skill test stations, for 
example, in order to test mobility, robots must traverse 
a random step field made of wood. The test bed was 
designed to be portable and reasonably inexpensive 
and several locations around the world have set up 
duplicates. As a result of the constraints of cost and 
portability, the test bed is not fully representative of 
actual disaster physical conditions and does not test the 
operating conditions for the human teams. 

60.6.3 Standards Activity 

Standards for rescue robots and systems are being gen¬ 
erated at the time of writing. The E54.08 subcommittee 
on operational equipment within the E54 Homeland 
Security application committee of ASTM International 
started developing an urban search and rescue (USAR) 
robot performance standard with the National Institute 
of Standards and Technology (NIST) as a US Depart¬ 
ment of Homeland Security (DHS) program from 2005 
to 2010. It plans to cover sensing, mobility, naviga¬ 
tion, planning, integration, and operator control in order 
to ensure that the robots can meet operational require¬ 
ments under the extreme conditions of rescue. The 
standards will consist of performance measures that 
encompass basic functionality, adequacy and appro¬ 
priateness for the task, interoperability, efficiency, and 
sustainability. The components of the robot systems in¬ 
clude platforms, sensors, operator interfaces, software, 
computational models and analyses, communication, 
and information. Development of requirements, guide¬ 


lines, performance metrics, test methods, certification, 
reassessment, and training procedures is planned. 

60.6.4 Evaluation 

Evaluation of rescue robots is difficult not only be¬ 
cause of the diversity of platforms and missions but 
also because each disaster is truly different. In addi¬ 
tion, robots are part of a human-centric system: they 
are operated by humans in order to provide information 
to humans. Evaluation of the performance of a rescue 
robot system at an actual disaster is currently ad hoc. 
No computer or physical simulation for predicting the 
performance of robots and humans in a disaster has 
been validated; indeed, there is little argument that 
simulations are far easier than a real response. The 
difficulties of simulation are exacerbated by the differ¬ 
ences between disasters. For example, the World Trade 
Center was unique in terms of the large amount of 
steel and the density of the collapsed material, while 
earthquakes and hurricanes are different from terrorist 
events. 

Metrics for measuring performance remain a worth¬ 
while quest. Quantitative metrics, such as the number 
of survivors or remains found, do not capture the value 
of a robot in establishing that there are no survivors in 
a particular area. Performance metrics from psychology 
and industrial engineering are only now beginning to 
be applied. These methods require enhanced computer 
and full-scale simulations in order to collect data. Data 
collection on human and overall system performance 
during a disaster has been done through ethnographic 
observations and are now moving to direct observations 
of situation awareness during demonstrations [60.80, 
93]. Direct data collection during a disaster may not 
be possible as methods may interfere with performance 
(and therefore be unreasonable, if not unethical) and 
arouse fears by operators of Big Brother and being held 
liable for any errors in operation. 


60.7 Conclusions and Further Reading 


Rescue robots are making the transition from an in¬ 
teresting idea to an integral part of emergency re¬ 
sponse. Aerial and ground robots have captured most 
of the attention, especially for disaster response, but 
water-based vehicles (both surface and underwater) 
are proving useful as well. Rescue robots present 
challenges in all major subsystems (mobility, com¬ 
munications, control, sensors, and power) as well as 
in human-robot interaction. In terms of size, man- 
portable and man-packable systems are the most pop¬ 


ular because of their reduced logistics burden, but 
the size of the platforms exacerbates the need for 
miniaturized sensors and processors. Wireless com¬ 
munications remains a major problem. While recent 
deployments have relied on polymorphic tracked ve¬ 
hicles, researchers are investigating miniature planes 
and helicopters along with new ground robots designs, 
particularly biomimetic. Research is also exploring al¬ 
ternative concepts of operations and user interfaces. 
Standards are currently under development and this 
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will help accelerate the adoption of rescue robots. The 
annual IEEE (Institute of Electrical and Electronics 
Engineers) International Symposium on Safety, Se¬ 


curity and Rescue Robotics is currently the primary 
conference and clearinghouse for research in rescue 
robotics. 


Video-References 


Assistive mapping during teleoperation 

available from http://handbookofrobotics.org/view-chapter/60/videodetails/140 
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61 . Robot Surveillance and Security 


Wendell H. Chun, Nikolaos Papanikolopoulos 


This chapter introduces the foundation for surveil¬ 
lance and security robots for multiple military and 
civilian applications. The key environmental do¬ 
mains are mobile robots for ground, aerial, surface 
water, and underwater applications. Surveillance 
literally means to watch from above, while surveil¬ 
lance robots are used to monitor the behavior, 
activities, and other changing information that 
are gathered for the general purpose of managing, 
directing, or protecting one's assets or position. 
In a practical sense, the term surveillance is taken 
to mean the act of observation from a distance, 
and security robots are commonly used to protect 
and safeguard a location, some valuable assets, or 
personal against danger, damage, loss, and crime. 
Surveillance is a proactive operation, while security 
robots are a defensive operation. The construction 
of each typeof robot is similarin nature with a mo¬ 
bility component, sensor payload, communication 
system, and an operator control station. 

After introducing the major robot components, 
this chapter focuses on the various applications. 
More specifically, Sect. 61.3 discusses the enabling 
technologies of mobile robot navigation, various 
payload sensors used for surveillance or security 
applications, target detection and tracking algo¬ 
rithms, and the operator's robot control console 
for human-machine interface (HMI). Section 61.4 
presents selected research activities relevant to 
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surveillance and security, includingautomaticdata 
processing of the payload sensors, automatic mon¬ 
itoring of human activities, facial recognition, and 
collaborative automatic target recognition (ATR). 
Finally, Sect. 61.5 discusses future directions in 
robot surveillance and security, giving some con¬ 
clusions and followed by references. 


61.1 Overview 


Surveillance robots and Security robots are tactically 
similar in design, but their specific purposes are very 
different. The surveillance robot uses its payload to 
map or search an area, while the security robot is used 
to protect and safeguard an asset such as the perime¬ 


ter of an airport. In comparing these two categories 
of robots, the difference in operations would be analo¬ 
gous to comparing surveillance to a global problem and 
security to a local problem. Operationally, the surveil¬ 
lance robot collects data with its electronic payload 
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Fig. 61 . 1 a-d Examples of surveillance and security robots, (a) 
Aerial-General Atomics Predator, (b) ground-QinetiQ TALON, (c) 
ground-General Dynamics MDARS-E, and (d) surface water-Elbit 
Stingray 


(also known as the robot’s external sensor) and relays 
the raw data back to its control station or preprocesses 
the data with other information such as geographic in¬ 
formation system (GIS) data and relays that information 
to an exploitation element. Eventually as more auton¬ 
omy is integrated into the vehicle, the robot will be able 
to plan its own paths and point its sensor with mini¬ 
mal human intervention. The quality of surveillance is 
a function of target range, the amount of target clut¬ 
ter, sensor resolution, and its probability of detection of 
a target. Security robot operations consist of a mobile 
robot patrolling a defined area (or perimeter), detect¬ 
ing an intruder or suspicious activities, and initiating 
a response. A security robot can conduct predefined or 
random patrols, and automatically perform surveillance 
tasks such as checking for intruders or assessing the sta¬ 
tus of the perimeter defenses. Input from the operator 
is required only if an intruder is detected or the robot 
encounters a situation it is not programmed to handle. 
Once contact is made, the operator is able to see, hear, 
and talk to the intruder. If the intruder tries to avoid 
the robot, the intruder can track them with its onboard 
sensor and maneuver the robot to follow the suspected 
intruder. Search and Rescue Robotics (Chap. 58) uses 
some aspects of surveillance due to the potentially wide 
area to be searched and the actions of security where 
upon detection, an action is required. 

The basic security robot [61.1] can be as simple 
as a fixed-sensor, mounted up high and able to pan 
and tilt without being able to move around. There are 
concepts to use a low-cost sensor (to sense motion. 


detect vibration, or break a sensor-beam) in order to 
cue a more sophisticated second sensor (e.g., radar or 
video) for confirmation. Depending on the complex¬ 
ity of the security system, the fixed sensor can alert 
a mobile robot, which can otherwise be defeated by just 
waiting for the robot to pass. A combination of fixed 
and mobile assets can be a very robust solution system. 
Again, the nominal robot configurations are still similar 
(Fig. 61.1): robot with sensor payload, an operator sta¬ 
tion, and a communication system that links the robot 
with its control station [61.2-5]. In some cases, there 
may be two distinct control stations, one for control¬ 
ling the robot and one for handling the payload data 
as seen on military drones. A surveillance robot can be 
more data intensive as compared to the security robot 
which often performs in a structured environment and 
has a narrower data load to process. Central to both 
applications are the robot or robots, and the control 
station. Robot surveillance is in greater use today due 
to the war in the Middle East (Iraq and Afghanistan) 
with the proliferation of military drones. The majority 
of unmanned aerial vehicles (UAVs) are controlled via 
teleoperation or by flying automatically through way- 
points. UAVs are now commonplace in the military, 
and slowly making inroads into the commercial sec¬ 
tor with applications in agriculture, forest fire mapping, 
and weather analysis. The next technological step for¬ 
ward for surveillance robots is autonomous operations 
and multi-vehicle coordination (i. e., swarming). Robot 
security is less developed with some on-going research 
programs within the military, and a few commercial 
options available on the market today. Early mobile 
robot companies in the 1980s were formed to address 
these same security concerns such as Cybermotion Inc. 
and Denning Mobile Robotics, Inc. but did not survive 
over time. The next logical progression for robot se¬ 
curity is to bring the costs down to make the systems 
more viable, reduce false-positives to make detection 
reliable, and take unnecessary personnel out of daily 
control through the careful insertion of automatic and 
autonomous technologies. 

There are some fundamental issues that need to 
be addressed in the design and development of the 
surveillance or security robot system. For example, 
each specific applications will dictate if surveillance 
occurs indoors or outdoors, and similarly for indoor 
or outdoor security, such that sensors are different to 
address weather or lighting concerns. There are numer¬ 
ous applications for either case, and will impact the 
type of mobility chosen, and the type of sensors used 
for monitoring and detection. As noted earlier, robot 
surveillance typically requires mobility; while security 
robots can be either stationary or mobile. In the later, 
the environment is known (or classified as being struc- 
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tured) with set vehicle trajectories defined a priori in 
the majority of applications. Some robot security ap¬ 
plications may involve a combination of both fixed and 
mobile robots depending on the size of the facility or 
building. The first issue to contend with is having to 
select between teleoperated and autonomous control of 
the robot. There is a scenario to start with a teleoper- 


61.2 Application Domains 

Surveillance and security robot designs begin by first 
understanding its intended operational environment, 
and simultaneously designing for data collection or var¬ 
ious protection schemas. The typical surveillance robot 
requires a surveillance payload, while a security robot 
integrates a security sensor. This sensor suite is in ad¬ 
dition to a navigation sensor suite required by the robot 
for mobility. There may be an opportunity to use the 
same sensor for both navigation and payload, but this 
option is very isolated due to disparate range and resolu¬ 
tion requirements. Since there are two different sensors, 
some coordination is required between mobility of the 
robot and the desired viewing angle of the payload 
sensor. 

Surveillance and security robots (Fig. 61.2) can 
be found in all the normal environments (e.g., aerial, 
ground, surface water, underwater, or space), for ex¬ 
ample by watching from above from either space or 
from an aerial elevation. Robots patrols are found in 
ground [61.6,7] and aerial applications: 

• Aerial Domain: The majority of long distance aerial 
applications utilize fixed wing vehicles flying at 
a set altitude and monitoring large surface areas. 
Aerial vehicles come in all sizes from full-size plat¬ 
forms that are the size of passenger jetliners to 
small quad-rotors, vertical take-off and landing ve¬ 
hicles, and microflapping wing aircraft as described 
in Chap. 50. Electro-optical (EO), infrared (IR), or 
synthetic aperture radar (SAR) are sensors nomi¬ 
nally used to collect surveillance data. A fixed-wing 
aircraft can also be used for security, but rotorcraft 
platforms (especially a quad-rotor) are the mobil¬ 
ity of choice for its hovering capability as shown in 
ld3M*il>Ril.vEi The weakness in rotorcraft platforms 
is its weight limitations, e.g., sensor payload weight 
and vehicle flight time. Key aerial design constraints 
are Reynolds number, Lift-to-drag (L/D) ratio of the 
wings, Breguet range analysis, vehicle endurance, 
and vehicle aspect-ratios. Communications between 
aircraft and ground control station is direct line-of- 
sight or over-the-horizon using satellites or other 


ated system that can eventually evolve toward a more 
capable automatic or autonomous capability. However, 
experience has shown that this upgrade seldom occurs 
naturally without planning and when the capability does 
mature, it will require large software block upgrades 
and the hardware will normally require upgrading also 
to increase overall capability. 


UAVs when functioning as a communication re¬ 
peater. 

• Ground Domain: Ground vehicles are either 
custom-designed or based on retro-fitting an ex¬ 
isting commercial or military vehicle. They come 
in all sizes from automobile-size to desk-size and 
shoebox-size such as the Dragon Runner as de¬ 
scribed in Chap. 48 and potentially walking as 
described in Chap. 65. An EO, IR, or range finder 
is used to collect surveillance data. For robot secu¬ 
rity, a similar EO, IR, lidar, or range finder sensors 
are often used as can be seen in 

Key ground design constraints are types of vehi¬ 
cle steering modes, type of propulsion (wheeled, 
track, legged, or hybrid) system, range, suspen¬ 
sion, and draw-bar pull performance. The standard 
packet-radio used for communications is bandwidth 
limited, thus ground robots would benefit from and 
automatic RSTA capability (RSTA: reconnaissance, 
surveillance, and target acquisition). 

• Surface Water Domain: Surface vehicles are ei¬ 
ther custom-designed, or based on retro-fitting an 
existing boat such as a rigid hull inflatable boat 
(RHIB). They vary in sizes from a small jet ski to an 
11m RHIB. Surface water robots used for surveil¬ 
lance and security are in its infancy. Potentially, 
a radar can be used to conduct surveillance, data 
and for robot security, a combination of radar and 
EO sensors are envisioned. Key surface water de¬ 
sign constraints are vehicle speed, range, and draft 
and beam dimensions for flotation. Surface navi¬ 
gation requires GPS for localization when visual 
techniques are ineffective and when water features 
are indistinguishable. 

• Underwater Domain: Unmanned underwater ve¬ 
hicles (UUVs) vary in size from man-portable 
lightweight platforms to large diameter vehicles of 
over 10 m in length. The larger vehicles have an ad¬ 
vantage in terms of endurance and sensor payload 
weight capacity as described in Chap. 51. Sonar is 
the payload sensor (sidescan or other) and is used 
to collect surveillance and support navigation data. 


Part F | 61.2 





Part F | 61.3 


1608 Part F 


Robots at Work 



Fig.61.2a-e Wide ranging operational environments: (a) Air-Tacocopter, (b) Ground-Asian Forum for Corrections’ Robo-Guard, 
(c) Underwater-Bluefin 9, (d) Surface water-Rafael Protector, and (e) Space-Ball Aerospace SBSS Satellite (SBSS: space based 
space surveillance) 


The majority of operational UUVs are tethered, but 
untethered vehicles have a larger presence in the 
research world (better known as autonomous under¬ 
water vehicles or AUVs). Key underwater design 
constraints are Reynolds Number, buoyancy, en¬ 
durance, pressure, and being tethered or untethered. 
Secure underwater communications is an issue for 
connecting the robot with its operators. 

• Space Domain : Spacecraft by nature are considered 
robotic. There is a combination of onboard automa¬ 
tion and a mix of commanding from the ground dur¬ 
ing critical events. Surveillance spacecraft range in 
size from microsatellites (50—100 kg) to traditional 
(approximately 15 000 kg) surveillance-size space¬ 
craft, and the typical payload sensors include high- 
definition cameras and SAR. Security satellites are 
used to protect assets on the ground, and having the 
ability to detect and track objects from space is cru¬ 
cial to preventing threats, whether real or accidental. 


Key spacecraft design constraints are weight, power 
usage, thermal cycling, surviving launch loads, and 
operating at cryogenic temperatures. 

As shown, there is a wide range of surveillance 
and security robots in multiple domains. Domains have 
specific individual environmental concerns. A domain 
not discussed is underground, as experienced with min¬ 
ing applications or the surveillance of caves with the 
iRobot 110 FirstLook or other small unmanned ground 
vehicle. It is clear with all of these robots that there is 
a navigational component and a payload/sensor com¬ 
ponent. Coordination between navigating and sensing 
for surveillance and security is of interest to robotic 
researchers. As noted earlier, the dominant robot ap¬ 
plications are aerial surveillance, which covers large 
surface areas by a fast moving aircraft, and ground (and 
surface water) vehicles for security that move slower on 
predefined paths. 


61.3 Enabling Technologies 

Robot surveillance and Security requires sensing, data 
collection, mobility (optional), navigation, communica¬ 
tion, control, computer vision, and having a preplanned 
response (i. e., using deterrence) as a security action. 
Acceptable responses can be as benign as alerting a hu¬ 
man guard, to immobilizing the threat, or potentially 
incorporating a weapon to remove the threat. There are 
many relevant technologies needed for this application. 
In this section, we will discuss the selected enabling 
technologies of: mobile navigation, payload sensors, 
applicable algorithms, and the human-robot interface 
at the operator control console. 

61.3.1 Mobile Navigation 

Teleoperated mobility is a well-developed technology 
as long as the communication bandwidth is available 
between the assets. For fast moving surveillance robots 
such as large aerial platforms [61.8], teleoperation is an 


option but today’s technology is to fly via GPS way- 
points. Waypoints are sets of coordinates that identify 
a point in physical space such as X , Y, Z, or longi¬ 
tude, latitude, and altitude. Navigation systems such as 
GPS (or other radio triangulation systems, i. e., Loran 
or Kaman) are used to maneuver the aircraft to fly¬ 
over or fly-by the waypoints. Fly-by waypoints are used 
when an aircraft should begin a turn to its next course 
prior to reaching the waypoint separating the two route 
segments, and fly-over waypoints are used when the air¬ 
craft must fly over the point prior and start its turn. In 
practice, the aerial robot will fly directly overhead of 
a fly-over waypoint, and can miss a fly-by waypoint 
by a couple of miles due to a practice called cutting 
the corner. The critical phases for all UAVs are during 
its launch and at its recovery, and would benefit from 
emerging sense-and-avoid technologies. For underwa¬ 
ter applications where GPS is not accessible unless the 
platform surfaces periodically, the robot [61.9] navi- 
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Fig. 61.3 Autonomous navigation in outdoor terrain (cour¬ 
tesy of Carnegie Mellon University National Robotic En¬ 
gineering Center) 



Fig.61.4a,b Typical EO sensors: (a) UAV EO gimbal sen¬ 
sor suite (courtesy of forward looking infrared (FLIR) 
systems) and (b) (UGV) EO/IR gimbal sensor suite (cour¬ 
tesy of L3 - Wescam) 

gates through the use of dead reckoning with frequent 
landmark corrections. 

For unknown and unstructured environments, tele¬ 
operation for security robots is the state of practice for 
aerial, ground, and surface water applications. When 
there is a setup period and the path is known a priori, 
as within a structured environment, preplanned paths 
can be incorporated. For a more advanced ground robot, 
autonomous navigation [61.10] is being researched and 
developed (predominantly in the outdoor environment). 
Benchmarks for autonomous navigation (Fig. 61.3) in 
complex, outdoor environments were established on 
the DARPA PerceptOR and LAGR programs (LAGR: 
learning applied to ground robots; PerceptOR: per¬ 
ceptions for off-road robotics). Refer to Chap. 35 for 
techniques that contribute to an autonomous navigation 
capability. 

61.3.2 Payload Sensors 

The end-user for surveillance data are the comman¬ 
ders, decision makers, analysts, targeteers, weaponeers 
(when a response is separate from the UAV), and 


cartographers. The information sought is timeliness, 
synoptic coverage, sensor resolution, sensor accuracy, 
interior details, and the differing elevations. Some basic 
payload definitions are: 

• Real-time: information arrives after the event at the 
speed of electrons. 

• Near Real-time: information arrives between real 
time and 20 min after the event. 

• Accuracy: level of agreement with standard map da¬ 
tum. 

• Precision: fineness of resolution to which measure¬ 
ments can be made. 

• Field of View: angular area observed by a fixed 
(nonscanning) sensor. 

• Field of Regard: angular area observed by a scan¬ 
ning sensor. 

Surveillance sensor options include passive imag¬ 
ing [61.11], active imaging, and nonimaging sensors. 
Film and EO sensors are passive imaging. An exam¬ 
ple of passive imaging on the DARPA UGV Demo II 
program is shown in loifciiiLllliEl In addition to EO, 
IR optical sensors often accompany its daylight coun¬ 
terpart in order to be also able to see in darkness 
(commonly denoted as EO/IR). Active imaging sen¬ 
sors include radar and laser, while nonimaging sensors 
are SIGINT, MASINT, and chemical/biological. Pas¬ 
sive imaging sensors are compared in Table 61.1. 

There are many active imaging sensors (Fig. 61.4) 
such as SAR, a radar that uses its forward motion 
to synthesize the equivalent of a large side-looking 
antenna to produce high-resolution ground mapping. 
Moving Target Indicator (MTI) is a radar mode that 
uses differences in Doppler frequency due to the target’s 
motion to distinguish it from stationary backgrounds 
(i. e., ground clutter). MTI has a wide field of view. 
Foliage penetration is radar using VHF/UHF frequen¬ 
cies and complex algorithms optimized to detect targets 
through foliage (VHF: very high frequency; UHF: ultra 
high frequency). Trees and leaves cover a good portion 
of the world’s terrain. Inverse SAR is a radar mode us¬ 
ing a target’s angular movement (instead of using the 
platforms as for SAR) to detect moving objects, and 
Interferometric SAR is a radar technique using widely 
separated receivers to receive the same signal and com¬ 
bining them to produce three-dimensional (3-D) maps 
with highly accurate terrain relief. High resolution radar 
is radar using short wavelengths (millimeter wave) 
and sophisticated algorithms to produce high resolu¬ 
tion (i. e., measured in inches) imagery. Laser imaging 
radar is a radar using light (laser), rather than radio fre¬ 
quency (RF) energy, for its illumination source. Robot 
researchers recognize LIDAR to refer to the acronym 
Light Detection and Ranging, and typically used in 
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Table 61.1 Passive imaging sensors 


Sensor 

Advantages 

Disadvantages 

Film 

Best resolution 

Easily archived 

Processing time compromises availability 
Dissemination copies degrade with reproduction 

Electro- 

optical 

(visible) 

Easy to data link while in-flight 

High resolution 

Easy to interpret (literal presentation) 

High fidelity reproduction 

Cannot penetrate inclement weather 

Volume of data produced 
(current big data problem) 

Infrared 

(cooled) 

Provides imagery during the dark 

Penetrates some weather or obscurants 

Detects targets otherwise invisible due to range or background 
Provides evidence of recently past activity (IR traces), current 
activity level (engines running), or status (fuel tank levels) 

Effectiveness diminished in warmer seasons, desert 
locales, hotter hours of the day, in Sun’s direction 
Resolution typically 1 NIIRs poorer than that of 
visible 

Infrared 

(uncooled) 

Same operational advantages of cooled IR sensors, but: More 
compact packaging 

Reduced power requirements 

Increased reliability 

Lower costs 

Less sensitive to thermal differences 

Larger pixel size = less resolution 

Spectral 

Provides additional operational information about targets- 
counter CC&D (camouflage, concealment, and deception) 
techniques 

Bandwidth intensive for data relay 


Table 61.2 Active imaging sensors 


Sensor 

Advantages 

Disadvantages 

Synthetic 
aperture radar 
(SAR) 

Sees at greater distances than visible 

Sees through (most) weather and night 

Provides constant perspective imagery 

Power and processing intensive 

Large aperture 

Moving target 
indicator (MTI) 

Detects and tracks moving air/ground objects 

Builds history of traffic flow over set intervals 

A SAR-dependent technique requires further 
processing of SAR returns 

Foliage 

penetration 

(FOPEN) 

Defeats most uses of foliage as camouflage 

Provides limited ground penetration capability 

Resolution limited due to frequencies used 

Inverse SAR 
(ISAR) 

Detects small objects on moving backgrounds 

Examples: boats, periscopes, and icebergs 

Performance varies with aspect angle 

Interferometric 
SAR (IFSAR) 

Achieves locational accuracies of 5—30 m 

Maps large amounts of terrain in a short time 

Requires relative positions of receivers be known 
and maintained precisely 

Requires extensive post-processing of data 

High resolution 
radar (HRR) 

Range limited by frequencies used 

Susceptible to intervening weather 

Laser imaging 
radar (LIDAR) 

Useful for bathymetry, pollution monitoring, and altimetry 
Potential for 2-D range-gated and 3-D imaging 

Limited in range and by intervening weather 


robot navigation. Active imaging sensors are compared 
in Table 61.2. 

Examples of Nonimaging sensors are SIGINT, 
MASINT, and chemical/biological. SIGINT stands for 
Signals Intelligence. Electronic signals are passive and 
the emissions of electronic systems (from radars, nav¬ 
igational aids, and others) can be intercepted and in¬ 
terpreted. The challenge with SIGNIT is frequency 
hopping and spread spectrum. Another facet of SIGNIT 
is the interception of spoken communications signals 
(through radios, etc.). The common challenges with 
communication signals are digitization, encoding, enci¬ 
phering, and/or frequency hopping. Measurement and 
Signatures Intelligence (MASINT) can be either ac¬ 


tive or passive. MASINT adds additional dimensions 
to target data, but requires the intelligent interpretation 
of this data. Finally, chemical/biological agent sensors 
can also be passive or active. There are two types of 
chemical/biological (Chem/bio) sensors: point and re¬ 
mote, and these types of sensors perform based on 
air sampling and can detect when previous chemical 
or biological weapons have been used. Thus, Robots 
have many options for payloads, dominated by weight, 
power consumption, and field of view. EO/IR are the 
most commonly used sensors for surveillance. Its se¬ 
lection is based on resolution, optical efficiency, and 
detector responsiveness. Its resolution is a function of 
its detector array size and the field-of-view of the lens 








Robot Surveillance and Security I 61.3 Enabling Technologies 1611 


used. Figure 61.5 is used to determine the desired reso¬ 
lution. Optical efficiency is dependent on lens diameter, 
lens focal length, and pitch. Finally, detector respon¬ 
siveness is based on the detector’s material, wavelength, 
and temperature. These are all physical parameters. 
A figure of merit used for a better payload is the product 
of its normalized resolution with a normalized optical 
efficiency with normalized detector responsiveness co¬ 
efficient. 

In summary, both EO and SAR sensors are ap¬ 
proaching their limits of immutable laws of physics. The 
challenge today is in data reduction and its exploitation 
in support of surveillance. As a rule of thumb, surveil¬ 
lance sensors are selected based on: 

• VHF/UHF SAR is used for foliage penetration and 
large area coverage. 

• Microwave SAR for high resolution imaging from 
standoff ranges. 

• Microwave ground moving target indicator (GMTI) 
radars for detecting moving targets and large area 
coverage. 

• Electric support measures (ESM), SIGINT, and 
communications intelligence (COMINT) for inter¬ 
cepting, locating, recording, and analyzing radiated 
electromagnetic signals. 

• Film for high resolution target classification/iden¬ 
tification. 

• EO/IR for targeting and target classification/iden¬ 
tification. 

A similar security payload also incorporates EO/IR 
sensors, but with resolution that commensurate with its 
shorter range to targets. As a rule of thumb, security 
sensors are selected as: 

• Radar is used to detect moving targets at a long 
range. 

• Radar for close-in target tracking. 

• EO/IR for close-in target tracking. 

Surveillance data is either stored for later use, or 
transmitted to exploitation centers for processing. Se¬ 
curity data can be processed onboard the robot or 
transmitted to the operator control console for real-time 
observance or for limited post-processing. The key for 
security is in its algorithms with respect to detecting, 
identifying, and tracking a potential target or suspicious 
activities. If weight and processing power is not an is¬ 
sue, multiple overlapping sensors are used on robots 
to reduce false detections and confirm target identifica¬ 
tion [61.12]. This is a layered approach. Performance is 
increased when a wide area surveillance system is used 
for initial detection or cross-cueing. For example, an 


Mapping resolution (GRD) (ft.) NIIRS rating 



Fig. 61.5 Determining sensor resolution for ATR or RSTA (cour¬ 
tesy of L. Newcome) 

MTI sensor can provide a wide area search and collect 
coarse information about potential targets sufficient to 
cue the narrow field of view EO and IR sensors that can 
continue the process of tracking, classifying, and identi¬ 
fying potential targets. Payload sensors collect data, but 
it is the processing of the data into information that can 
be used by others that is the nugget for both the surveil¬ 
lance and security communities. 

61.3.3 Detection and Tracking Algorithms 

An important technology in surveillance reconnais¬ 
sance, surveillance, and target acquisition (RSTA). 

This topic is also closely associated with intelligence, 
surveillance, and reconnaissance (ISR) and automatic 
target recognition (ATR). ATR, RSTA, and ISR are 
age-old problems in the military to identify and clas¬ 
sify activities of interest in the environment [61.13-16]. 

During surveillance, the user has an option to acquire 
a target at a specified location, search an area or sec¬ 
tor for either stationary or moving targets, perform 
a panoramic reconnaissance of an area or sector, ini¬ 
tiate fire control, or manually control its sensors by 
a human operator to acquire surveillance imagery as 
depicted in Fig. 61.6. For example, if the sensor is 
stationary and the target is stationary, than a maximum- 
average-correlation-height approach where a bank of 
linear correlation filters are optimized to respond to 
the presence of a specific target object by producing 
a peak at its corresponding location can be very effec¬ 
tive. If a stationary target search is used, the platform 
is directed to search a specified area for targets using 
stationary target detection and recognition algorithms. 
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Fig. 61.6 UAV sensor coverage 
with aircraft trajectory (courtesy 
of University of Illinois - Urbana 
Champaign) 



These algorithms can detect both stationary and mov¬ 
ing targets, but moving target detection does not make 
use of motion related information that is associated with 
a robot. Moving objects in the scene are the subjects of 
interest in most visual surveillance applications using 
EO/IR sensors. Specifically, the actions of the moving 
objects need to be monitored and appropriate flags need 
to be raised when events of interest occur in the scene. 
The mobile platform needs to be controlled so as to 
aid the surveillance task. Thus, the objectives of the 
surveillance task and the associated real-time computa¬ 
tion constraints involve difficult challenges in building 
highly robust yet computationally cheap algorithms for 
detection, classification, and correspondence. 

Manually, the user can select several options in¬ 
cluding the type of search modes to be used in which 
the sensor is assumed to be either stationary or mov¬ 
ing, and the target is assumed to be either stationary 
or moving. For example, a UAV and satellite are al¬ 
ways in motion, while the UGV/USV/UUV can either 
be either stationary or mobile. The selection of the sta¬ 
tionary target mode causes stationary target detection 
and recognition algorithms to run, while the selection 
of the moving target mode causes moving target detec¬ 
tion and tracking algorithms to be executed. The user 
would benefit from an automatic selection feature. The 
demanding nature of the required detection criteria for 
surveillance and security necessitates the integration of 
complementary technologies which can sense motion, 
characterize patterns, find thermal signatures, and cap¬ 
ture temporal behavior as can be seen in IdaWil'H 'Tfll 

The RSTA or ATR algorithm sequence [61.17] in¬ 
clude reconnaissance, acquire target through search, 
and finally target tracking. A panoramic reconnaissance 
command can direct the robot’s payload to collect a mo¬ 
saic of images over a wide field of regard and send it 
to the operator for manual viewing. There are several 
techniques used to create an image mosaic from a se¬ 


quence of images. The registration between images is 
obtained by minimizing the sum-squared error of image 
intensities at each pixel. Although this technique pro¬ 
duces very accurate registration results, it tends to be 
slow, and typically require operator interaction to ini¬ 
tialize the registration function. The collected mosaic 
may be used to assess trafficability, check out regions 
of interest such as roads and bridges, and manually 
look for enemy targets. To acquire targets, the vehicle 
is directed to look at a particular map location or in 
a particular direction, acquire an image, and perform 
stationary target detection on that image. An operator 
can select several search options in which the robot is 
assumed to be either stationary or moving, and the tar¬ 
get is assumed to be stationary or moving. The search 
area determines whether to search a map region or an 
azimuth-elevation wedge. An azimuth may be specified 
relative to the world (map) or specified in vehicle coor¬ 
dinates. The selection of the stationary target function 
causes stationary target detection and recognition algo¬ 
rithms to run, while selection of a moving target mode 
causes moving target detection and tracking algorithms 
to be executed. The search mode determines whether to 
search an area in a single time or multiple times frames. 
An EO/IR sensor can use a combination of image 
processing algorithms. Probing-type algorithms include 
connected-component analysis or a blob analysis. 

A large body of image processing algorithms has 
been developed to detect targets using techniques in¬ 
cluding histogram manipulation, convolution, morphol¬ 
ogy, over- and under-sampling, quantization, and spec¬ 
tral processing using Fourier transforms and discrete 
cosine transforms (DCTs). These algorithms tend to 
be computationally intensive [61.18,19]. Other pop¬ 
ular ATR algorithms include binary template match¬ 
ing, multispectral imaging, and wavelet transformation 
techniques. Of the many types of image enhancement 
algorithms, a spatial convolution kernel filtering tech- 
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nique produces the most dramatic results. A convolu¬ 
tion kernel generates a new pixel value based on the 
relationship between the value of the pixel of interest, 
and the values of those that surround it. In convolution, 
two functions are overlaid and multiplied by one an¬ 
other. One of the functions is the video frame image 
and the other is a convolution kernel. ATR algorithms 
can be broadly described sequentially as: target detec¬ 
tion, a segmentation function, feature evaluation, and 
classification. Target detection approaches can be either 
image-based or model-based. 

If stationary target search is used, the platform is 
directed to search a specified area for targets using 
stationary target detection and recognition algorithms. 
These algorithms can detect targets using blob extrac¬ 
tion and blob tracking algorithms. When a potential 
target is detected, images are sent to a target recognition 
algorithm for additional processing. Target recognition 
can be accomplished through the use of a hashing al¬ 
gorithm. Various target detections are passed to the 
hashing algorithm for labeling as one of several tar¬ 
get types contained in a database or perception library. 
The hashing algorithm also produces an estimate of 
target pose and when the processing is complete, the 
results are sent to the operator for verification. Moving 
objects are typically detected using background sub¬ 
traction. For this type of search, the platform is directed 
to look at a specified area for targets using a moving tar¬ 
get detection algorithm based on temporal differencing. 
Most surveillance systems use background subtraction 
as an efficient means of motion detection with a sta¬ 
tionary camera. Unfortunately, background subtraction 
techniques are not always robust under camera jitter, 
varying lighting conditions, and moving foliage. Tech¬ 
niques such as affine transformations are used for image 
stabilization. Problems of a periodic nature such as 
jitter and moving foliage induce multi-modal distribu¬ 
tion of pixel intensity values. If the classification of 
the background environment can be calculated, then 
the background can be subtracted from the image to 
enable the use of motion detection and segmentation al¬ 
gorithms. 

One popular method for background model gen¬ 
eration is the incorporation of autoregressive (AR) or 
infinite impulse response (HR) filters. A single AR or 
HR filter is used for each pixel to estimate the domi¬ 
nant mode of the background. The AR filter estimates 
the center and width of the dominant mode of the back¬ 
ground, and it is associated with each filter per pixel 
is a value that approximates the probability that the 
mode represented by the filter is seen by the pixel. 
When an intensity value is seen by the pixel falls within 
the mode of one of the pixel’s filters and the associ¬ 
ated probability is greater than a preset threshold, then 


the pixel is declared as background. When a partic¬ 
ular intensity value is not represented by any of the 
filters for that pixel, a new filter is added with an as¬ 
sociated low probability value. When a filter matches 
intensity value, its probability rating is increased and 
the probabilities of the rest of the filters associated 
with that pixel are decreased. Thus, at each detection 
cycle, the filters corresponding to a pixel adapts to 
better fit the values of its background. This motion 
detection technique is enhanced by the use of feed¬ 
back from higher-level algorithms such as a classifier 
and/or a correspondence algorithm. The classifier has 
the capability to reject spurious detections. This infor¬ 
mation can be used either to create a new filter for the 
pixel or to increase or decrease the probability threshold 
for detection. Information from its correspondence can 
be used to predict future locations of detections. This 
considerably improves the quality of segmentation. An¬ 
other correlation-based tracker function can be added, 
which controls the payload pan/tilt gimbal to keep a de¬ 
tected target centered in the image. A harder problem 
is when the target is moving and the sensor is also 
moving. 

For security robots, camera-based EO/IR sensors 
have long been used for security and observation pur¬ 
poses as depicted in Surveillance cam¬ 

eras are typically fixed at known positions and have 
coverage of a circumscribed area defined by the fields- 
of-view of the cameras. Although some recent vision 
research has addressed autonomous surveillance, in 
most cases, humans perform the sensory processing. 
For an autonomous surveillance example, a vision sys¬ 
tem can provide the geometric shape of the target and 
its angular location relative to a radar sensor, while its 
onboard radar system can measure both the distance to 
the target and its relative velocity radial to the vehicle. 
The target’s perceived geometric shape and distance are 
then used to calculate its cross-section for purposes of 
human target classification. The complementary mix of 
using multiple sensor (camera and radar) technologies 
allows for greater than 99% probability of detection 
with less than 1% nuisance alarm rate (also known as 
false-positives) for fielding a robust security system. 
This is a good example of cross-cueing. Camera control 
can be seen in Once a target is detected, 

it is tracked in the center of the collective field-of-view 
in order to provide verification using successive vision 
and radar-based detections. A track file is maintained on 
each detected target to correlate size, distance, speed, 
and other parameters over an extended period of time. 
This time-based integration approach allows the system 
to significantly reject nuisance alarms while using ex¬ 
tremely high gains during the detection process. The 
classification procedure in a security application maps 
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a given image sequence of a moving object to the most 
likely class label by classifying each image indepen¬ 
dently and then classifying the resulting set of class 
labels. The classifier can also reject classifying an ob¬ 
ject. For example, a classifier on the mobile detection 
assessment and response system (MDARS) program 
has demonstrated class-conditional sequence error rates 
of less than 5% and a false alarm rejection rate of 80% 
in disjoint testing. Unknown objects and novel views of 
known objects are detected by considering the class la¬ 
bel history over an image sequence. Image sequences 
that yield a significant fraction of rejections or cause 
atypical classifier confusion are saved for later human 
interpretation. 

In the last step, the temporal correspondence of 
a moving object plays a very important role in classify¬ 
ing, tracking, and interpreting an object’s actions. Large 
number of targets of varying sizes in a scene preclude 
the use of simple positional correspondence, i. e., cor¬ 
respondence based purely on the positions of moving 
objects. In such situations, other features of the mov¬ 
ing objects, such as different appearance traits, need to 
be considered for robust correspondence. For example, 
the measure of goodness of the features can be chosen 
not only on the object in question, but also on other ob¬ 
jects in the scene. A globally good set of features can 
be estimated a priori, but only a subset of these features 
might be relevant to the correspondence of a partic¬ 
ular object. Carnegie Mellon University (CMU) has 
developed a technique called differential discrimina¬ 
tive diagnosis in which it provides a systematic method 
for estimating the relevance of features and checking 
the temporal consistency of these features for a par¬ 
ticular object. The correspondence task is performed 
with a two-part process. The first part relies on po¬ 
sitional correspondence using linear prediction. This 
part nominates a set of likely candidate matches for 
a reference object. The second part uses appearance- 
based correspondence to find the best match among 
those nominated by linear predictions. A simple lin¬ 
ear classifier is trained based on learning techniques to 
determine whether or not two images are of the same 
object. Differential discriminative diagnosis identifies 
those features that are most relevant to the correspon¬ 
dence problem. Efficient correspondence is achieved by 
enforcing the temporal consistency of the relevances 
determined for a particular object, and this technique 
has demonstrated correspondence among moving ob¬ 
jects with an accuracy of 96%. Target correspondence 
across multiple sensors can be achieved using a max¬ 
imum a posteriori (MAP) estimate of the observation 
sequence to maximize target appearance and spatiotem- 
poral probabilities. A correspondence problem is also 
used in stereo vision. 


Another approach is to perform the correspondence 
step using a multiple robots configuration as in a swarm, 
which is addressed further in the multiple vehicle ATR 
active research area to follow (in Sect. 61.4.4). A more 
complex scenario involves multiple sensors detecting 
several targets simultaneously, and a moving sensor 
payload detecting a moving target [61.20,21]. A mov¬ 
ing target causes the radar signature to shift outside the 
normal return of a radar image. This shift in frequency 
(a Doppler shift) is used to distinguish stationary tar¬ 
gets from moving targets. The detection of a moving 
target is more difficult when the sensor is also in mo¬ 
tion. Augmenting data such as speed and heading of 
the moving target can be used to better profile potential 
targets. Other factors such as size, radar cross-section, 
and environmental clutter are used to make assump¬ 
tions governing the type and intent of the target. MTI 
cross-cueing data has been overlaid in real-time on to 
3-D visualization of the terrain to allow an operator to 
quickly locate a target such that the EO/IR sensor could 
be rapidly slewed to the target of interest. Test results 
have shown that wide field-of-view MTI radar that is 
used to cue a limited narrow field of view EO/IR sensor 
has drastically increased situational awareness on the 
battlefield. The next evolution is to automate the cross- 
cueing process in order to track, classify, and identify 
potential stationary and moving targets [61.22], 

61.3.4 Human-Machine Interface 

Unmanned aerial robots that are used for surveil¬ 
lance come in all types from the smaller, hand 
launched Raven, Puma, and Wasp drones to the mid¬ 
size Aerosonde and ScanEagle to the larger Predator, 
Reaper, and Global Hawk platforms that can stay air¬ 
borne for 24 to greater than 28 h missions [61.23]. 
The Global Hawk architecture is depicted in Fig. 61.7 
with its many communication links. The larger UAVs 
and their ground segment components interact through 
an over-the-horizon (OTH) satellite relay and line-of- 
sight (LOS) links to maintain command and control 
and sensor data dissemination communication paths 
for launch, flight, and recovery. Bandwidth compres¬ 
sion is applied to the sensor data to maximize area 
coverage and data throughput. The communications 
systems are being designed to minimize susceptibility 
to jamming and interception. Dissemination of UAV 
collected intelligence is made through direct downlinks 
to national and theater intelligence centers and other 
exploitation systems, or through established hard com¬ 
munications between UAV ground control stations, and 
the exploitation systems. The UAV ground control sta¬ 
tion has limited capabilities to disseminate collected 
intelligence directly to an exploitation ground system 
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Fig. 61.7 Global Hawk UAV architec¬ 
ture (courtesy of Northrop Grumman 
and Raytheon) 



Fig.61.8a-c Global Hawk ground elements: (a) ground station, (b) operator seats, and (c) UAV display (courtesy of 
Northrop Grumman and Raytheon) 


or battlefield customer. When deployed, a fully oper¬ 
ational system consists of four Predators (with EO/IR 
sensors), a ground control station (GCS) that houses the 
pilots and sensor operators for human-machine inter¬ 
face (HMI), and a primary satellite-link communication 
suite. 

The Predator UAV can run autonomously, executing 
simple missions such as reconnaissance on a software 
program, or it can run under the control of its crew 
(Fig. 61.8). The crew of a single Predator consists of 
one pilot and two sensor operators. The pilot drives 
the aircraft using a standard flight stick and associated 
controls that transmit commands over a C-Band line-of- 
sight data link. When operations are beyond the range 
of the C-Band link, a Ku-Band satellite link is used to 
relay commands and responses between a satellite and 
the aircraft. Onboard, the aircraft receives orders via an 
L-3 Communication satellite data link system. The pi¬ 
lots and crews use the images and radar received from 
the aircraft to make decisions about controlling the air¬ 
craft. The pilot’s main mission is to fly the plane, while 
the sensor operator controls the cameras that bring the 


battlefield into full view to gain intelligence. Unlike 
other manned missions, pilots for large UAVs only con¬ 
trol the plane during landing and takeoff. However, 
sensor operators do much more than watch the feedback 
from their cameras [61.24]. Working together with pi¬ 
lots, they do preflight checks, coordinate with the air 
control tower, and provide feedback to pilots. The intel¬ 
ligence data is streamed back to exploitation elements 
where the data is analyzed by people on the ground or 
with automatic detection algorithms. Not including the 
exploitation element, it currently takes approximately 
82 personnel to operate four of the larger UAVs for 
a 24 h period including technicians and support person¬ 
nel. 

Typically, unmanned ground robots used for secu¬ 
rity [61.25,26] are available in limited options from 
the commercial Vigilus system, Reborg-Q security 
robot, OFRO (by Quadretec Ltd.) and patrolbot, to 
the army’s mobile detection assessment and response 
system (MDARS) and its family of integrated rapid 
response equipment (FIRRE). The MDARS [61.27] 
system consists of ground robotic platforms capable of 
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Fig. 61.9 MDARS operational concept and system components (courtesy of Space and Naval Warfare Systems Center 
(SPAWAR), Pacific) 


preprogrammed autonomous movement, motion detec¬ 
tion, an incident assessment subsystem, and two way 
communications capability between operator and robot. 
Its operational configuration will consist of pair(s) of 
mobile platforms employing a suite of sensors that are 
preprogrammed or remotely controlled by an opera¬ 
tor located at a remote monitoring and control station. 
The actual number of fixed and mobile patrol units for 
a given location will be site specific. The MDARS In¬ 
crement 1 system components are depicted in Fig. 61.9 
with its various interfaces. All such robots and sen¬ 
sors (known collectively as resources) are controlled by 
the multiple resource host architecture (MRHA), with 
minimal human (i. e., security personnel and operators) 
supervision required [61.28]. The individual processors 
are connected via an Ethernet local area network (LAN) 
that supports a peer-to-peer communications protocol 
(expandable to 32 platforms or robots). The user inter¬ 
face provides a big picture representation of secured 
areas and system resources. User intervention is re¬ 
quired only when a platform encounters an exceptional 
condition such as an environmental hazard or a security 
breach. 

The operator control unit is either in monitor mode, 
in which the operator observes and monitors the sta¬ 
tus of the robot and sensors, or the operator is in direct 
control of a single resource such as a single robot in 
a multi-robot system. Moreover, the robot itself carries 
a number of sensory systems, each of which must also 
be controlled individually. For example, MDARS may 


have eight cameras, two radars, and four lights, but only 
a single camera, radar, and light is controllable at any 
given time as shown in Multiple cameras 

can be viewed simultaneously, but the operator cannot 
pan two of them at the same time. A joystick button or 
dialog button can be configured to cycle between the 
various payloads. The control of the resources and each 
of its selected subresources is integrated in such a way 
that a single joystick can be used to perform all relevant 
joystick tasks. As an example, the operator can pan the 
selected camera with the joystick, and then press a but¬ 
ton to put the robot in teleoperation mode and drive the 
robot using the same joystick. The concept of modes is 
used to provide rich control features for each resource. 
This paradigm reduces the complexity of the software 
and simplifies the user interface presented on its dis¬ 
plays, while still giving the operator full control of each 
asset. The MDARs control station allows the operator 
to execute preplanned patrols, random patrols, or user- 
directed patrols as seen in Automation of 

sensor selection would improve operations. 

The controller displays can perform basic image 
processing, which means they take in video at a cer¬ 
tain resolution, and display it in the display’s native 
resolution [61.29]. Video processing is the act of scal¬ 
ing, which is converting an incoming video signal from 
one size or resolution to another in order to work with 
the display panel. This is what is usually referred to as 
video processing, and it is a minor feat compared to 
video enhancement. Video enhancement techniques can 
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Fig.61.10a-c MDARS mission control center, (a) Control station, (b) MDARS operator, and (c) display (courtesy of 


SPAWAR) 


be based on spatial domain methods or frequency do¬ 
main methods [61.30]. Image enhancement algorithms 
offer a wide variety of approaches for modifying im¬ 
ages to achieve visually acceptable images. The choice 
of techniques implemented is a function of the spe¬ 
cific task, image content, observer characteristics, and 
viewing conditions. Mosaic images provide better visu¬ 
alization of a tracking scenario. All security sensors are 
fused together to form a composite threat score on po¬ 
tential targets, and intruder alarms are reported to the 
user on the MDARS console (Fig. 61.10). Also, the 
intruder’s range and bearing are displayed on a map 


window and a digital stream from the camera is shown 
in a video window. For ATR, the operator receives 
notification of a target by the appearance of an icon 
on the map display at the operator interface. Select¬ 
ing an icon brings up a target verification panel and 
a target response panel showing the imagery associ¬ 
ated with the target that was provided by the sensors. 
With experience, most operators became comfortable 
and competent with normal operations, and each ac¬ 
quiring variable levels of proficiency in responding to 
exceptional events such as dealing with intruders or tak¬ 
ing manual control of the robot. 


61.4 Active Research 

In addition to enabling robotic surveillance and secu¬ 
rity technologies, there are many active areas of robot 
research, such as being able to produce high-quality 
imagery from a mobile platform that poses a number 
of challenges. In addition to issues related to camera 
motion and the resulting image perspectives, the qual¬ 
ity of the video imagery can also be compromised by 
poor environmental conditions, data link degradations. 


and bandwidth limitations. Atmospheric factors such as 
poor lighting at dawn, dusk, or night time, the presence 
of shadows, target occlusion and adverse weather, in¬ 
cluding sandstorms and variable clouds, can obscure 
important details. Some identified areas of interest 
include: Sect. 61.4.1 in solving the data processing 
problem due to the large amounts of sensory informa¬ 
tion collected, Sect. 61.4.2 detecting human activity in 



Fig. 61.11 Automatic target recogni¬ 
tion (courtesy of Bilkent University) 
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Fig.61.12a,b Multiple real¬ 
time video stream sensors. 

(a) Gorgon Stare (courtesy 
of Sierra Nevada Corp) and 

(b) ARGUS-IS (courtesy of 
BAE Systems) 


support of surveillance. Sect. 61.4.3 facial recognition 
for security identification purposes, and Sect. 61.4.4 
collaborative ATR by multiple robot in either hetero¬ 
geneous or homogeneous systems. 

61.4.1 Solving the Data Processing Problem 

With all the useful information that UAVs provide 
comes the problem of how to sort through it all and 
find actionable data. This is the Big Data problem for 
the military. One manual solution to the data explo¬ 
sion problem is to tag the data, store it, and retrieve 
it when needed. Unfortunately, this information would 
not be timely. Data fusion involves the use of tech¬ 
niques and software that combine data from multiple 
sources and analyze the data to make it useful for the 
end user [61 .3 1]. Data fusion can involve the combining 
of data (such as UAV video) with a GIS, which adds lo¬ 
cation and time data to the images gathered (Fig. 61.11). 
To accomplish this, the raw data has to be combined 
with metadata, which is information about the data that 
enables the data to be combined. On-going research 
such as with intelligent search is another tool that can 
be used to make UAV data more accessible. Based on 
modern artificial intelligence (AI) techniques such as 
machine learning, depth first searches using iterative 
deepening, or depth first searches based on space com¬ 
plexity, users can quickly personalize a search to make 
relevant data more easily available [61.32]. 

To help users retrieve information, one promising 
method, called natural language processing, discov¬ 
ers user preferences and needs by either extracting 
knowledge from what users are looking for or by in¬ 
teractively generating explanatory requests to focus 
users on the information they are interested in [61.33]. 
The use of natural language to improve the perfor¬ 
mance of the intelligent search engines is one aspect 
of AI. Other related AI technologies for automated 
processing include object recognition and statistical 
machine learning, or application of AI in intelligent 
search is facial recognition. One promising solution 
that could help with the data overload issue is more 
efficient techniques for sensor fusion. For example, 
a multisensor fusion algorithm that can bring together 


information from multiple platforms about a partic¬ 
ular target of interest in a unified display would be 
beneficial to the operator. Researchers will need to 
develop smart technologies that automate the entire pro¬ 
cess of archiving, tagging, retrieving, managing, and 
displaying data and other information gathered by in¬ 
creasingly sophisticated sensors. As an adjacent topical 
area, machine-to-machine interfaces will need to be¬ 
come much more transparent in order to handle the data 
deluge [61.34], 

A future technology called wide area surveillance 
(Fig. 61.12) is based on a new sensor system called 
Gorgon Stare, a video capture technology that uses 
a spherical array of nine to twelve cameras attached to 
an UAV. With so many sensors, Gorgon Stare needs to 
utilize a system of tagging and incorporating metadata 
to be fully effective. Initial reports on the progress of 
Gorgon Stare is that there are deficiencies in its IR per¬ 
formance, having numerous interoperability problems, 
having a lack of stability of the sensors, and encounter¬ 
ing reliability problems. These issues can be overcome 
as the program matures. Unfortunately, the new sensor 
system is still limited by the human’s inability to pro¬ 
cess the information gathered from many cameras, or 
rely on advanced automatic processing techniques to 
fuse the information into the coherent picture. 

61.4.2 Detecting Human Activity in Support 
of Surveillance 

The problem of using computer vision to track and un¬ 
derstand the behavior of human beings is a very impor¬ 
tant tool for surveillance and security operations. At its 
highest level, this problem tries to recognize human be¬ 
havior and understanding intent and motive from visual 
observations [61.35, 36], and [61.37]. This is a difficult 
task, even for humans to perform, and misinterpreta¬ 
tions are common. The problem has several subareas 
of human motion recognition, surveillance, tracking, 
and activity detection. In the area of surveillance, au¬ 
tomated systems to observe pedestrian traffic areas and 
detect dangerous action are becoming important. Many 
such areas currently have stationary surveillance cam¬ 
eras in place. However, all of the image understanding 
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Fig. 61.13 Monitoring individual and group human activi¬ 
ties (courtesy of University of Minnesota) 

and risk detection is left to human security person¬ 
nel. This type of observation task is not well suited to 
humans, as it requires careful concentration over long 
periods of time. Therefore, there is clear motivation to 
develop automated, intelligent, vision-based monitor¬ 
ing systems that can aid a human user in the process of 
risk detection and analysis. Eventually, this capability 
will migrate from stationary cameras to mobile security 
applications. 

Most of these human activity applications require 
an automated recognition of high-level activities, com¬ 
posed of multiple simple (or atomic) actions of persons 
as depicted in Fig 61.13. The goal of human activity 
recognition is to automatically analyze on-going activ¬ 
ities from EO/IR sensors through a sequence of image 
frames [61.38]. There are various types of human activ¬ 
ities and depending on their complexity, human activi¬ 
ties can be categorized into four different levels based 
on: gestures, actions, interactions, and group activi¬ 
ties. Gestures are elementary movements of a person’s 
body part and actions, and is defined as single-person 
activities that may be composed of multiple gestures 
organized temporally. Interactions are human activi¬ 
ties that involve two or more persons and/or objects as 
shown on I<eaM 3EE!EEI, while group activities are the 
activities performed by conceptual groups composed 
of multiple persons and/or objects. Promising events 
for activity monitoring include the detection of loiter¬ 
ing individuals [61.39], and the detection of unusual 
crowd activities [61.40], which can be difficult to dis¬ 
cern from normal crowd activities. It would be useful 
for security robots to detect motion in restricted ar¬ 
eas [61.41], detect perimeter breaches, and detect when 
security cameras are tampered with [61.42]. Also of in¬ 
terest which may or may not involve the human include 
the throwing of an object or the detection of an aban¬ 
doned object. The University of Minnesota has done 


work in detecting abandoned objects as referred to in 

There are several approaches used in the recog¬ 
nition of different levels of activities. Single-layered 
approaches are approaches that represent and recog¬ 
nize human activities directly based on sequences of 
images while hierarchical approaches represent high- 
level human activities by describing them in terms of 
other simpler activities. Single-layered approaches are 
divided into two classifications depending on how hu¬ 
man activities are modeled directly from the video data: 
space-time approaches (a sequence of two-dimensional 
(2-D) images placed in chronological order) and se¬ 
quential approaches (a sequence of observations). Un¬ 
fortunately, the majority of activity monitoring algo¬ 
rithms is far from being real-time, which is an issue 
for surveillance. Robot surveillance is demanded in 
today’s environment where the tracking and monitor¬ 
ing of people is an integral part of everyday activities. 
Monitoring is typically accomplished with multiple 
cameras [61.43] and from multiple vantage points. 
Activities of interest vary widely from throwing an 
object [61.44] to the subtle task of abandoning an 
object such as a backpack or leaving on unknown pack¬ 
age [61.45]. In many instances, it would be important 
to estimate or count the size of a gathering [61.46-48]. 
Activity monitoring suffers from the same issues asso¬ 
ciated with ATR such as signed noise, tracking issues, 
and segmentation issues arising out of the stabilization 
of video images. There are a large number of new and 
promising improvement methods on the horizon, in¬ 
cluding space-time feature-based approaches, manifold 
learning, rigid/nonrigid motion analysis, and hierarchi¬ 
cal approaches. 

61.4.3 Facial Recognition 

for Security Identification 

In order for a security robot to perform its mission to 
protect and safeguard, sometimes it will encounter peo¬ 
ple which have to be discerned as either friend or foe. 
In order to make such a determination, face recognition 
is critical [61.49]. Most of the current face recognition 
algorithms can be categorized into two groups: image 
template-based or geometry feature-based [61.50,51]. 
The template-based methods computes the correlation 
between a face and one or more model templates to es¬ 
timate the face’s identity. For the other approach, the 
geometry feature-based methods analyze explicit local 
facial features, and their geometric relationships. Face 
recognition is a two-step procedure consisting of first 
detecting a face and followed by recognition. Top re¬ 
searchers have argued that the accurate detection of 
human faces in arbitrary scenes is the most impor- 
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Fig. 61.14 Local binary patterns using OpenCV (courtesy of Yale 
University) 


tant process involved, followed by the face recognition 
process. When faces could be located exactly in any 
scene, the recognition step makes the final determina¬ 
tion of classifying the person as being either friend or 
foe. 

Algorithms are used to know how to differentiate 
between a basic face and the rest of the background. If 
the background is controlled (referred to as being highly 
structured and uncluttered), an algorithm to remove the 
background will produce face boundaries. Faces can be 
found by either using color, motion, or a mixture of 
both types of information. The detailed steps include 
acquisition, normalization, and recognition of moving 
faces in dynamic scenes. Color as a cue can be used 
for detection and tracking, and provides a computa¬ 
tionally efficient yet effective method which is robust 
under rotations in depth and during partial occlusions. 
This method can be combined with both motion- and 
appearance-based face detection. The main idea is to 
model a class-conditional density for each person in 
a representation space of relatively low dimensionality. 
Unfortunately, color detection doesn’t work on all kinds 
of skin color, and is still susceptible to varying lighting 
conditions. 

Humans must periodically blink to keep their eyes 
moist, and detecting a blinking pattern in an image se¬ 
quence is a reliable means of detecting the presence 
of a face. Blinking provides a space-time signal which 
is easily detected and unique to faces. Detection by 
blinking is precise, but requires capturing an image pair 
during a blink. A common technique for recognition is 
the correlation with a set of Eigen images. Better results 
have been demonstrated by combining stereo, color, 
and face detection techniques into an integrated ap¬ 
proach [61.52]. Depth estimation using stereo cameras 
is used to eliminate background effects. Skin-hue clas¬ 
sification identifies and tracks likely body parts within 
the silhouette of a user, while face pattern detection dis¬ 


criminates and localizes the face within the identified 
body parts. Other approaches to finding faces include 
the use of neural networks, taking advantage of statisti¬ 
cal cluster information, and model-based face tracking. 
Related work continues using edge-based methods and 
geometric models such as edge-orientation matching 
and using the Hausdorff distance measurement. The 
leading face finding algorithm in unstructured environ¬ 
ments is based on a series of weak classifiers such 
as Haar-like features, representation by integral im¬ 
ages, and a method for combining successively more 
complex classifiers in a cascading structure which can 
increase the speed of detection by focusing attention on 
promising regions of the image. An example of facial 
recognition can be seen in 

Upon detecting faces, facial recognition has numer¬ 
ous distinguishable landmarks, where different peaks 
and valleys make up facial features (Fig. 61.14). Mea¬ 
surements of these features include: 

• Distance between the eyes 

• Width of the nose 

• Depth of the eye sockets 

• The shape of the cheekbones 

• The length of the jaw line. 

The 2-D (or 3-D) image of the face is compared to 
a database of known people’s faces. After detection, the 
image is analyzed to determine the head’s position, size, 
and pose estimation. The sensor measures the curves of 
the face to a submillimeter scale to create a template. 
This face template is translated into a unique code that 
represents the features on the face. This is followed by 
a matching function that compares the image taken with 
a database of known faces. This is followed by a verifi¬ 
cation step that identifies the person in the image. 

Skin biometrics can be used to verify the match. 
Techniques such as Surface Texture Analysis, Local 
Feature Analysis, and Vector Templating are used to 
improve the match. There are some impediments to 
facial recognition such as a lack of resolution in the 
sensor, glare from glasses, obscuration due to long 
hair hiding features of the face, and poor lighting 
conditions. The wider issue is to obtain and main¬ 
tain an up-to-date database for matching. It would be 
simple to keep records on who belong to the facil¬ 
ity, and to declare unknown faces as foes. However, 
there will be instances that the human is not a foe, 
but does not exist in the database either. Another is¬ 
sue would be a scenario in which there are multiple 
human intruders that stretches the capability of the 
robot security system. As a final verifier, the human 
operator can be used to make the final determination 
as friend or foe, visually or through an interrogation 
process. 
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Fig. 61.15 Collaborative ATR (courtesy of NATO) 


61.4.4 Collaborative ATR 

Swarm robots are a new approach to the coordination 
of multi-robot systems that consist of large numbers 
of robots, either homogeneous or possibly heteroge¬ 
neous robots, i. e., team of air and ground platforms as 
depicted in Swarming implies that a col¬ 

lective behavior emerges from the interactions between 
robots or the interactions of robots with its environ¬ 
ment [61.53,54], and [61.55], In a coordinated team 
approach, individual platforms are able to pass target lo¬ 
cation information to one another during a target search 
operation in an attempt to get confirmation of a sus¬ 
pected target. Thus, a robot that detects a target passes 
the target’s location (and a surrounding map-based un¬ 
certainty region) to another robot for confirmation. The 
confirming robot searches the nonoccluded portion of 
the given map region for the detected vehicle and re¬ 
turns the results of its search to the originating vehicle 
where they are compared with the original search re¬ 
sults (Fig. 61.15). A final target’s result is compiled and 
sent to the operator for verification. 

The intent of cooperative target search is to increase 
the accuracy and reliability of the target search process. 
The conditions under which this search is guaranteed to 
be accurate must deal with the following issues: 

• It is unclear under what conditions the originating 
vehicle can be sure that a positive response from the 
confirming vehicle corresponds to the same target 
or to any target at all. 

• If the orientations of either or both robots are off 
significantly from their true orientations, it may be 
that the vehicles are not searching the same map re¬ 
gion or that different targets within the uncertainty 
region are reported as the same target. 

• Cooperative target searches puts stringent demands 
on the accuracy of vehicle orientation estimates. 
This problem can be partially alleviated by having 


reliable, long-range laser rangefinders on all partic¬ 
ipating robots. 

• Cooperative target search also puts stringent de¬ 
mands on the accuracy of target detection and 
recognition algorithms. For example, it is not un¬ 
usual for an unsophisticated target detection algo¬ 
rithm to generate a positive return from a warm rock 
or large animal. 

• The fact that the confirming vehicle may return 
a positive response does not increase the probabil¬ 
ity that the object is actually a valid target. In trials, 
confirmation of nontargets has actually been ob¬ 
served in field testing. 

• It is also difficult to tell whether the confirmation of 
a valid target results from observation of the same 
target or detection of a false target. To alleviate this 
kind of problem, accurate and sophisticated target 
recognition algorithms are required. 

Another concern is when the confirming robot re¬ 
turns a negative response. This lack of confirmation 
may indicate nothing more than that the confirming 
robot is looking in the wrong area or that it simply 
failed to detect a target that is present in its sensor’s 
field-of-view. At this point, it is unclear under what 
conditions the originating vehicle should disregard its 
own initial detection. These issues provide motivation 
for researchers to develop more sophisticated ATR al¬ 
gorithms to obtain higher confidence reports and reduce 
the number of false alarms. These same issues also 
provide motivation for the development of context un¬ 
derstanding algorithms. 

Since cooperative target searches place these strin¬ 
gent demands on the accuracy of vehicle orientation 
estimates, teams of surveillance platforms must solve 
the multi-robot distributed estimation problem [61.56] 
and the related source localization problems [61.57]. 
Each of the robots collects sensor data regarding its 
own motion and shares this information with the rest 
of the team during periodic update cycles. A single 
estimator, possibly in the form of a Kalman filter, pro¬ 
cesses the available positioning information from all 
the members of the team and produces a pose esti¬ 
mate for every one of the robots. The equations for 
this centralized estimator can be written in a decen¬ 
tralized form, therefore allowing this single Kalman 
filter to be decomposed into a number of smaller 
communicating filters. Each of these filters processes 
sensor data collected by its host robot. The exchange 
of information between the individual filters is nec¬ 
essary only when two robots detect each other and 
measure their relative pose. The resulting decentral¬ 
ized estimation schema, which referred to as collective 
localization, constitutes a unique means for fusing mea- 
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surements collected from a variety of sensors with 
minimal communication and processing requirements. 
What complicates the estimation is that the robots are 
moving and the targets (or multiple targets) may also 
be moving. 

Researchers have introduced a simple updating rule 
defining a distributed estimation scheme that balances 
consensus with tracking. It was shown that the probabil¬ 
ity distribution of the vector of estimates obeys a Master 
Equation derived from first and second moment dy¬ 
namics. From this master equation, the evolution of 
its mean estimate and its variance can be computed. 
If the individual robots ignore the discrete state and 
simply do consensus, the estimates converge with the 
probability of one to the average of the initial values 
of the estimates. Moreover, by appropriately weighting 
the discrete state in the update rule, the average can be 
tracked with an arbitrarily small variance at steady state 
conditions. 

Source localization is addressed by incorporating 
a visual gradient-based approach for a robot to adap¬ 
tively search a location-unknown source. While mov¬ 


61.5 Conclusion 

Robots for surveillance typically involve unmanned 
aerial vehicles that can cover large areas and sit 
on regions of interest to provide a persistent pres¬ 
ence [61.60]. UAVs today utilize teleoperation or flies 
via way-points on longer traverses to search selected 
regions. During flight, there are instances in the mis¬ 
sion profile where launch and recovery operations are 
also autonomous. With EO/IR and SAR payloads, the 
data is streamed back to processing elements for post¬ 
processing. This becomes a bottleneck due to the high 
number of platforms in theater and the enormous vol¬ 
ume of data that cannot be processed in real-time. These 
operations are also highly susceptible to weather is¬ 
sues, and impacted by delays and bandwidth limitation 
issues associated with the many communication links 
utilized. Collaborative ATR with swarms of vehicles is 
an emerging capability, but introduces its own set of is¬ 
sues in terms of consistently and correctly identifying 
targets of interest. Finally, there are related efforts in 
ground and unmanned surface water (USV) RSTA, and 
present different environmental-type challenges such as 
handling sensor motion issues due to buoyancy or flying 
indoors (|<£2»M£Il!£l). 

Security robots are a maturing capability, but yet to 
be commonplace. The vision is to have a single opera¬ 
tor commanding multiple stationary and mobile robots 
to deter criminal activities, especially the destructive 


ing, the robot only measures the received image quality 
and adaptively decides the motion direction in each 
time instant. Simulation results indicate that the robot 
can reach the sensor when the initial signal-to-noise ra¬ 
tio (SNR) is as low as OdB and the standard deviation 
of motion errors is 10% of the motion step size. Due 
to the adaptive nature of this solution, this approach is 
robust to measurement noise and motion errors. Since 
the robot moves along the direction of the gradient es¬ 
timated at each time, the approach can be applied to an 
arbitrary signal strength distribution in an area [61.58]. 
One of the scenarios can be in an environment of ob¬ 
stacles so that the robot at the initial location does not 
have a line of sight to the sensor. The other scenario 
to consider is that the source is a cluster of activated 
sensors that send signals at the same time. Thus, the gra¬ 
dient approach can be extended from 2-D to 3-D space 
for applications incorporating aerial vehicles [61.59]. 
A multi-robot system can be applied to both the surveil¬ 
lance problem and the security scenario, but continued 
research is required to make such a system robust and 
practical. 


actions of some people. The overall mission is to de¬ 
tect, assess, and take action in order to deter a potential 
threat. Typical robots include ground platforms, hov¬ 
ering type vehicles, and surface water vehicles of all 
sizes. Key issues include the monitoring of human ac¬ 
tivities, detecting faces, and determining the extent of 
the potential threat. With multiple sensors available to 



Fig. 61.16 Humanoid KUBO as a security guard (courtesy 
of Korea Advanced Institute of Science and Technology 
(KAIST)) 
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the operators, there will be limited opportunities to con¬ 
firm the target or set of targets due to time sensitivities. 
Moreover, sensor fusion can deliver a more complete 
picture of the situation for human or automatic confir¬ 
mation. Affectivity of the system is quantifiable based 
on measuring coverage, having a high recognition rate, 
and demonstrating a low false alarm rate. A future di¬ 
rection is to potentially arm the security robots, but this 
introduces a new series of issues such as liability and 
ethics. Chapter 80 (Roboethics: Social and Ethical Im¬ 
plications) expands on these larger societal issues. 

A future of humanoids (Fig. 61.16) and social 
robots used for surveillance and security is an inevitable 
possibility [61.61]. Humanoids (Chap. 67) provide 
bipedal mobility as a ground robot, and have an inherent 


sensor payload with its head. However, the head sensors 
must be used to both navigate and perform reconnais¬ 
sance. Its surveillance capability is limited by the field 
of view and resolution of its head cameras, but has 
expanded orientation capabilities based on its anthro¬ 
pomorphic neck and torso gimbals. This type of robot 
can take advantage of existing RSTA algorithms such 
as human activity monitoring, face recognition, and 
collaborative ATR for target confirmation as discussed 
previously. However, all these features are dependent on 
having adequate onboard processing capability in order 
to make real-time confirmation and decisions. Who is 
to say a future humanoid security robot cannot pick up 
a remote sensor suite similar to binoculars in order to 
search and detect a potential intruder? 
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Security: Facial recognition 

available from http://handbookofrobotics.org/view-chapter/61/videodetails/553 
Surveillance by a drone 

available from http://handbookofrobotics.org/view-chapter/61/videodetails/554 
Ground security robot 

available from http://handbookofrobotics.org/view-chapter/61/videodetails/677 
People detection from a UAV 

available from http://handbookofrobotics.org/view-chapter/61/videodetails/678 
UGV demo II: Outdoor surveillance robot 

available from http://handbookofrobotics.org/view-chapter/61/videodetails/679 
MDARS I: Indoor security robot 

available from http://handbookofrobotics.org/view-chapter/61/videodetails/680 
Scout robot for outdoor surveillance 

available from http://handbookofrobotics.org/view-chapter/61/videodetails/681 
Detection of abandoned objects 

available from http://handbookofrobotics.org/view-chapter/61/videodetails/682 
Tracking people for security 

available from http://handbookofrobotics.org/view-chapter/61/videodetails/683 
Collaborative robots 

available from http://handbookofrobotics.org/view-chapter/61/videodetails/700 
Multi-robot operator control unit 

available from http://handbookofrobotics.org/view-chapter/61/videodetails/701 
Camera control from gaze 

available from http://handbookofrobotics.org/view-chapter/61/videodetails/702 
Indoor, urban aerial vehicle navigation 

available from http://handbookofrobotics.org/view-chapter/61/videodetails/703 
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62 . Intelligent Vehicles 


Alberto Broggi, Alex Zelinsky, limit Ozgiiner, Christian Laugier 


This chapter describes the emerging robotics appli¬ 
cation field of intelligent vehicles - motor vehicles 
that have autonomous functions and capabilities. 
The chapter is organized as follows. Section 62.1 
provides a motivation for why the development of 
intelligent vehicles is important, a brief history of 
the field, and the potential benefits of the tech¬ 
nology. Section 62.2 describes the technologies 
that enable intelligent vehicles to sense vehicle, 
environment, and driver state, work with digital 
maps and satellite navigation, and communi¬ 
cate with intelligent transportation infrastructure. 
Section 62.3 describes the challenges and solu¬ 
tions associated with road scene understanding - 
a key capability for all intelligent vehicles. Sec¬ 
tion 62.4 describes advanced driver assistance 
systems, which use the robotics and sensing tech¬ 
nologies described earlier to create new safety and 
convenience systems for motor vehicles, such as 
collision avoidance, lane keeping, and parking 
assistance. Section 62.5 describes driver moni¬ 
toring technologies that are being developed to 
mitigate driver fatigue, inattention, and impair¬ 
ment. Section 62.6 describes fully autonomous 
intelligent vehicles systems that have been devel¬ 
oped and deployed. The chapter is concluded in 
Sect. 62.7 with a discussion of future prospects, 
while Sect. 62.8 provides references to further 
reading and additional resources. 
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Robots at Work 



62.1 The Motivation and Approaches to Intelligent Vehicles 


An important field of application of robotics has 
emerged in the last 20—25 years which is centered on 
the automobile, named intelligent vehicles. The auto¬ 
mobile has been one of the most important products 
of the twentieth century. It has generated an enormous 
industry and has given individuals a freedom of move¬ 
ment that has completely changed our way of life. 
Indeed, the automobile has been a key factor in the 
large change in the way our urban societies are struc¬ 
tured. Today, there are more than 800 million vehicles 
on the planet, and this number is expected to double in 
the next 10 years. This challenge has led to the devel¬ 
opment of an active research domain with the ultimate 
goal of automating the typical tasks that humans per¬ 
form while driving. An intelligent vehicle is defined as 
a vehicle enhanced with perception, reasoning, and ac¬ 
tuating devices that enable the automation of driving 
tasks such as safe lane following, obstacle avoidance, 
overtaking slower traffic, following the vehicle ahead, 
assessing and avoiding dangerous situations, and deter¬ 
mining the route. The overall motivation for building 
intelligent vehicles has been to make motoring safer, 
and more convenient and efficient. 

Approaches to this problem revolve about three 
main architectures [62.1], according to the degree of au¬ 
tonomy they aim for (Fig. 62.1): 

a) Driver-centric approaches are built around the idea 
of having a human in the loop supervising the func¬ 
tions of the vehicle. 

b) Network centric-approaches assume that vehicles 
can share information with other vehicles or the in¬ 
frastructure. 


c) Vehicle-centric approaches aim to build fully au¬ 
tonomous intelligent vehicles with no human in the 
loop. 

62.1.1 Brief History 

Futurama was an exhibit at the 1939 World Fair de¬ 
signed by Norman Bel Geddes that presented a pos¬ 
sible model of the world 20 years into the future 
(1959—1960) with automated highways. The exhibit 
was sponsored by the General Motors Corporation and 
was characterized by its vision of automated high¬ 
ways. Technological development in support of this 
vision began in the late 1940s and early 1950s, with 
pioneering efforts by RCA Laboratories and GM Re¬ 
search. By 1960, GM had tested full-scale cars driving 
automatically on test tracks, with lane tracking, lane 
changing and car following functions automated. The 
goal of these efforts was the development of an auto¬ 
mated highway system (AHS). In the 1960s, a key effort 
was research and development under the leadership 
of Fenton and his team at The Ohio State Univer¬ 
sity [62.2], During these years, vehicles were developed 
to test steering and speed controllers. Runs were accom¬ 
plished at the Transportation Research Center (TRC) 
grounds, using active sensing (a live wire imbedded 
in the roadway) and with steering control done with 
analog feedback controllers. In Japan, research on auto¬ 
mated highway systems was started in the early 1960s 
in the Mechanical Engineering Laboratory (MEL) and 
the National Institute of Advanced Industrial Science 
and Technology (AIST). The automated driving system 
in 1960s employed a combination of an inductive cable 


Degree of vehicle 

Degree of human intervention (%) autonomy/intelligence (%) 



Fig. 62.1 Architectures 
for autonomous vehicles 
(after [62.1]) 
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embedded under a roadway surface and a pair of pickup 
coils at the front bumper of a vehicle for lateral control, 
and it was a cooperative system between vehicle and 
infrastructure. In addition to the free agent, a rear-end 
collision avoidance system between two automated ve¬ 
hicles based on roadway vehicle sensors was developed. 
The automated vehicle drove up to 100 km/h on a test 
track in 1967 [62.3], 

The history of intelligent vehicles developed fur¬ 
ther over the ensuing decades. Although the first ideas 
were born in the 1960s, the level of maturity of the 
technology at that time did not allow pursuit of the orig¬ 
inal goal of implementing fully autonomous all-terrain, 
all-weather vehicles. The first documented prototypes 
of such vehicles were fielded by a few groups in the 
military arena in the mid 1980s [62.4-6]. The ini¬ 
tial stimulus that triggered these innovative ideas was 
provided by the military sector, which was eager to 
provide complete automation to its fleet of ground 
vehicles. 

Testing of autonomous vehicles on real roads in 
a real environment was one of the most important 
milestones in the history of intelligent vehicles. In Ger¬ 
many, in the beginning of the 1980s Dickmanns and his 
team equipped a Mercedes-Benz van with cameras and 
other sensors. For safety reasons, initial experiments in 
Bavaria took place on streets without traffic. Since 1986 
the robot car VaMoRs from the same group manages to 
drive all by itself, since 1987 at speeds up to 96 km/h. 
A number of these capabilities were also highlighted 
in the European PROMETHEUS Project, (for exam¬ 
ple, [62.7]) 

This happened in the mid to late 1990s. Figure 62.2 
shows the first motor vehicles that pioneered the devel¬ 
opment of intelligent vehicles. In the summer of 1995, 
the Carnegie Mellon Navlab group ran their No Hands 
Across America experiment [62.8]. They demonstrated 
automated steering, based solely on computer vision. 



Fig.62.2a-d Pictures of pioneering autonomous vehicles 
(a) NAVLAB, (b) UBM, (c) Argo, and (d) Ligier-INRIA 
electric vehicles 


over 98% of the time on a 2800 mile trip across the 
United States. At the same period the EU funded the 
first large and ambitious project for autonomous car 
navigation (the EUREKA Project Prometheus) involv¬ 
ing the major European car constructors and research 
institutes. In the scope of this project, UBM (Bun- 
deswehr Universitat Munich) fielded a vehicle that 
was demonstrated on a 1758km-trip from Munich to 
Copenhagen and back. The vehicle was able to drive au¬ 
tonomously for 95% of the trip. The car suggested and 
executed manoeuvres to pass other cars. Unlike later 
robot cars, this car located itself on the current road and 
followed it until instructed otherwise. It did not local¬ 
ize itself in global coordinates and could drive without 
a global positioning system (GPS) and road maps as 
found in a modern automotive navigation systems. The 
vehicle used dedicated purpose-built computers and 
hardware. 

A different approach was followed by VisLab at 
the University of Parma within the Argo project. The 
passenger car that was designed and developed was 
based on a low-cost approach. An off-the-shelf Pen¬ 
tium 200 MHz personal computer (PC) was used to 
process stereo images obtained from low-cost cameras 
installed in the driving cabin. The vehicle was able 
to follow the lane, locate obstacles, and - when in¬ 
structed - change lane and overtake slower vehicles. 
The main milestone of this project was the successful 
test of the Argo vehicle on a tour of Italy of more than 
2000 km called Millemiglia in Automatico, in which 
the vehicle drove itself for 94% of the total distance. 
Another low-cost approach was developed at INRIA 
in 1995 for its dual mode system, in which a small 
dedicated electric vehicle was able to park itself au¬ 
tonomously using only ultrasonic sensors and a linear 
camera. 

Through the years, many demonstrations have been 
held showing the public the capabilities of autonomous 
vehicles, and by extension, underlining the sensor capa¬ 
bilities at the autonomous driving demonstrations. One 
of the most comprehensive highway-based demonstra¬ 
tions was held in 1997 on 1-15 in San Diego, USA, 
and showed the capabilities of cars, buses, and trucks in 
various automated highway scenarios. The demonstra¬ 
tion (called Demo’97) was organized by the National 
Highway Systems Consortium. The key technologies 
were lane-following using roadway imbedded magnets, 
roadway laid radar-reflective stripes or using lane mark¬ 
ers with vehicle mounted cameras; car following using 
laser or radar, with or without help of inter-vehicle com¬ 
munication. Scenario maintenance was accomplished 
by either total preplanning and timing, or by GPS 
and map-based triggering, or by situation-based trigger¬ 
ing [62.9, 10]. 
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The Demo’97 was followed by a number of demon¬ 
strations around the world, notably in Japan, Hol¬ 
land, France, and the US. In each case, not only 
were state-of-the-art technologies demonstrated, but the 
public was informed about what to expect in future 
cars. 

In November 2000, the demonstration Demo 2000 
Cooperative Driving was organized by the National 
Institute of Advanced Industrial Science and Technol¬ 
ogy in Tsukuba, Japan. Several platooning scenarios 
were accomplished by five automated vehicles in the 
enclosed test field. This platoon of five automated ve¬ 
hicles performed stop-and-go operations, platooning, 
splitting into two platoons demonstrating an exit ramp, 
merging into one platoon from two platoons imitat¬ 
ing highway traffic at a ramp, and passing by the last 
vehicles. Furthermore, obstacle detection, platoon leav¬ 
ing, and joining tasks were accomplished. Autonomy 
was guaranteed by inter-vehicle communication. In ad¬ 
dition, vehicles were equipped by differential global 
positioning system (DGPS) to assure localization by 
the laser radar to detect obstacles and to measure 
the relative position between the vehicles. Vehicles of 
Demo 2000 did not communicate with the infrastruc¬ 
ture i. e., road-to-vehicle communication except DGPS. 
These automated vehicles were autonomously driven 
by exchanging information through inter-vehicle com¬ 
munication [62.11]. 

Current research initiatives are oriented towards the 
development of intelligent vehicles in realistic scenar¬ 
ios. However, until recently, primarily due to legal 
issues, full autonomy had not yet been set as the ul¬ 
timate goal. The automotive industry had set as its 
primary goal the need to equip vehicles with supervised 
systems and - more generally - advanced driving assis¬ 
tance systems (ADAS) instead of automatic pilots. In 
other words, the driver was still in charge of running the 
vehicle, but the driver and vehicle were monitored by an 
electronic system that detects possibly dangerous situa¬ 
tions and reacts by either warning the driver in due time, 
or taking control of the vehicle in order to mitigate the 
consequences of the driver’s inattention. In spite of the 
aforementioned legal issues, the ultimate goal has now 
shifted towards developing self-driving systems and 
a number of automotive manufacturers have announced 
such cars in the market by model year 2020. Concur¬ 
rently, departments of transportation worldwide which 
have been primarily interested in social, economic, or 
environmental objectives aimed at enhancing fuel ef¬ 
ficiency, road network usage, and improving quality of 
life in terms of mobility have now shifted their attention 
to the possibility of mixed traffic on the road, consisting 
of regular vehicles driven by people and autonomous 
vehicles [62.12], 


Different approaches and research goals have 
evolved internationally. The USA has pushed for fully 
autonomous vehicles which could be commercialized, 
Europe has focused on ADAS systems and public 
transportation applications, clean vehicles, and free- 
service autonomous vehicles (Sect. 62.7). This contrast 
is illustrated by the differences in scope and objec¬ 
tives between US funding schemes and competitions 
(e.g., DARPA challenges, Google cars) and European 
projects (e.g., Prometheus, Carsense, Prevent, Inter¬ 
safe). 

The good results obtained by ADAS in the auto¬ 
motive arena in recent years has induced the military 
sector to give a new vigorous push to the original ideas 
of automating its fleet of ground vehicles. The De¬ 
fense Advanced Research Projects Agency (DARPA) 
launched the Grand Challenge in 2003, a race for au¬ 
tonomous vehicles that had to travel for more than 
200 km in unstructured environments. This unprece¬ 
dented challenge attracted a large number of top-level 
research institutes and helped the scientific community 
take a considerable step forward. 

In 2005 the DARPA Grand Challenge required au¬ 
tonomous driving in a rough terrain desert scenario 
with no traffic, obstacle types known in advance, and 
few if any road markers on a course predefined by 
2935 GPS points [62.13]. Five cars (maximum speed 
40 km/h) completed the 211km desert course, with 
Stanford’s Stanley winning the challenge in 6 h 54 min. 
Figure 62.3a-b shows photographs of the top two fin¬ 
ishing vehicles. The reader is referred to [62.14] for the 
technical details of the DARPA Grand Challenge. 

Following the Grand Challenges, the 2007 DARPA 
Urban Challenge was performed in emulated city traffic 
and saw the return to urban vehicles [62.15]. Au¬ 
tonomous ground vehicles competed to win the Urban 
Challenge 2007 prize by accomplishing an approxi¬ 
mately 60 mile urban course area in less than 6 h with 
moving traffic. The autonomous vehicle had to be 
built upon a full-size chassis compared to production 
type vehicles and must have a documented safety re¬ 
port. The autonomy capability demonstrated in forms 
of a complex set of behaviors by obeying all traf¬ 
fic regulations while negotiating with other traffic and 
obstacles and merging into traffic. The urban driving 
environment challenged autonomous driving with some 
physical limits such as narrow lanes, sharp turns, and 
also by the daily difficulties in a daily urban driv¬ 
ing such as congested intersections, obstacles, blocked 
streets, parked vehicles, pedestrians, and moving vehi¬ 
cles. The winner of the Urban Challenge was Tartan 
Racing, a collaborative effort by Carnegie Mellon Uni¬ 
versity and the General Motors Corporation, with their 
vehicle Boss, a heavily modified Chevrolet Tahoe. The 
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second place finisher was the Stanford Racing Team 
with their entry Junior, a 2006 Volkswagen Passat. Fig¬ 
ure 62.3c-d shows photographs of the top two finishing 
vehicles. In May 2011, The Netherlands Organization 
for Applied Scientific Research TNO, together with 
the Dutch High Tech Automotive Systems innovation 
program (HTAS), organized the Grand Cooperative 
Driving Challenge (GCDC) in The Netherlands. The 
2011 GCDC focused on cooperative adaptive cruise 
control (CACC). Nine international teams participated 
in the challenge [62.16]. 

In 2010, the University of Parma’s VisLab In¬ 
tercontinental Autonomous Challenge [62.17-19] fur¬ 
ther pushed the boundaries by having four driverless 
vehicles (Fig. 62. 3e) drive autonomously 15 000km 
from Parma, Italy to Shanghai, China. A more re¬ 
cent example is the Google driverless car (Fig. 62. 3f): 
a fleet of autonomous vehicles which by 2012 had 
driven more than 480000 km autonomously. Google 
has demonstrated fully autonomous driving in dense 
urban California traffic, utilizing vision-based systems 
and Google Street View. This activity, repeated in 
a number of cities generated extensive public interest 
and led to the belief that autonomous vehicles will be 
available for public use in the near future. This has 
been a major factor leading three US states (Califor¬ 
nia, Nevada, and Florida) to approve laws permitting 
driverless cars. A number of car manufacturers have an¬ 
nounced models to come to market by 2020. 


Fig.62.3a-f 2005 Grand Challenge vehicles: (a) Stanley 
(1st), (b) Sandstorm (2nd), 2007 Urban Challenge ve¬ 
hicles, (c) Boss (1st), (d) Junior (2nd), (e) VisLab and 
(f) Google vehicles 


62.1.2 Benefits of Intelligent Vehicles 

Having intelligent vehicles running on our road net¬ 
works would bring a number of social, environmental, 
and economical benefits. An intelligent vehicle able to 
assess the driving scenario and react in case of dan¬ 
ger would allow up to 90% of traffic accidents that are 
caused by human errors to be eliminated, thus saving 
human lives. According to the World Health Organi¬ 
zation an estimated 1.2 million people worldwide are 
killed each year, and about 40 times this number in¬ 
jured, due to traffic accidents. 

At the same time, vehicles able to drive at high 
speeds and very close to each other would decrease fuel 
consumption and polluting emissions; furthermore they 
would also increase road network capacity. Vehicles 
communicating with a ground station could share their 
routes and be instructed to reroute in order to maintain 
a smooth traffic flow. Vehicles that can sense and obey 
speed limits or traffic rules would reduce the possibility 
of misinterpretation and antisocial driving behavior. 

Fully automatic vehicles would also offer a higher 
degree and quality of mobility to a larger population, 
including young, old, or infirm individuals, reducing 
the need even for a driving licence. Finally, the avail¬ 
ability of vehicles that could drive themselves would 
increase the quality of mobility for everyone, turning 
personal vehicles into taxis able to pick up people and 
take them to their final destination in total safety and 
comfort, enabling people to dedicate their driving time 
to their preferred activities. 

However, this full application of intelligent vehicles 
is far from being complete, since unmanned vehicle 
technology is still under development for many other 
applications. The automation of road vehicles is, per¬ 
haps, the most common everyday task that attracts the 
greatest interest from the industry. However, other do¬ 
mains such as agricultural, mining, construction, search 
and rescue, and other dangerous applications in gen¬ 
eral, are looking to autonomous vehicles as a possible 
solution to the issue of the ever-increasing cost of per¬ 
sonnel. If a vehicle could move autonomously on a field 
to seed, or enter a mined field, or even perform dan¬ 
gerous missions, the number of individuals put at risk 
would drastically decrease and at the same time the ef¬ 
ficiency of the vehicle itself would be increased due to 
a 24/7 operational schedule. The key challenge for in¬ 
telligent vehicles is safety; accidents must not occur due 
to automation errors and there is zero tolerance to hu¬ 
man injury and death. 

This chapter focuses on road and traffic applications 
of intelligent vehicles, which are catalyzing the interest 
of the automotive industry, car makers, and providers of 
automotive technologies. 
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62.2 Enabling Technologies 

The basic sensing and actuation technologies for intel¬ 
ligent vehicles are readily available on the market. The 
key challenge to integrating new technologies is heav¬ 
ily dependent on the control strategies associated with 
the sensor data processing and reasoning. Automated 
systems must consider all subsystems in a vehicle, in¬ 
cluding the interaction of the technology with the driver. 
Key drivers for the new technologies in an intelligent 
vehicle are the desired applications. Solutions are de¬ 
veloped on the premise that the system must satisfy the 
requirements of the application with the minimum level 
of technology. 

Intelligent vehicle applications require the follow¬ 
ing: 

• The position, and kinematic and dynamic state of 
the vehicle 

• The state of the environment surrounding the vehi¬ 
cle 

• Access to digital maps and satellite data 

• The state of the driver and occupants 

• Communication with roadside infrastructure or 
other vehicles. 

62.2.1 Vehicle State 

Position localization is a key technology for intelligent 
vehicles. The position of the vehicle must be known if 
the vehicle is to be controlled along a particular tra¬ 
jectory. To control an intelligent vehicle to perform 
applications such as collision avoidance or automatic 
lane changing requires knowledge of the kinematic 
and dynamic states of the vehicle. Standard robotics 
techniques for position, kinematics, and dynamics are 
used in intelligent vehicle applications (Chap. 34). En¬ 
coders are mounted on the steering column and the tail 
shaft. Vehicle heading, speed, and acceleration can be 
computed from the encoders using standard techniques 
(Chap. 24). 

To execute the trajectories in an intelligent vehicle 
a number of techniques are used. In many current ap¬ 
plications of fully automated vehicles, the trajectories 
are fixed with limited choices at intersections. To exe¬ 
cute the trajectories, the vehicles simply follow markers 
such as an antenna (a wire carrying an alternating cur¬ 
rent), magnets, or passive transponders such as radio 
frequency identification (RFID) tags that are imbedded 
in the road or simple painted lines. In more advanced 
navigation systems, the vehicle generates trajectories in 
a map stored in its memory and localizes itself with 
respect to this map in order to execute them. This local¬ 
ization can be done using absolute positioning through 


a combination of global navigation satellite systems 
(GLS), vehicular kinematic states, and inertial naviga¬ 
tion. Alternatively, position localization can be done 
by relative localization using local markers such vi¬ 
sual landmarks. (For specific reference to earlier mobile 
robot landmark navigation see Chap. 46.) 

In order for the vehicle to execute trajectories, it 
needs to control the speed and the steering of the vehicle 
(Fig. 62.4). Modern vehicles now incorporate electronic 
control units to control the acceleration (motor torque 
and possibly gearbox and clutch), the braking (elec¬ 
tric or electrohydraulic brake control), and the steering 
angle (electric power assistance), and it is, therefore, 
quite easy to implement trajectory control of the ve¬ 
hicles [62.20]. The safety of these critical functions 
remains a challenge, and is usually solved through re¬ 
dundancies in the sensors, actuators, and control units. 

62.2.2 Environment State 

Sensing the state of the environment surrounding the 
vehicle is a critical aspect of intelligent vehicle applica¬ 
tions. The most difficult function for intelligent vehicles 
is road scene understanding. This includes locating key 
landmarks: the road, other vehicles, pedestrians, traf¬ 
fic signals, road signs, and other unstructured obstacles. 
A more difficult challenge is speed control following 
the detection of an event in the road scene. Corn- 




turning circle 

Fig. 62.4 Ackerman steering model 
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Lane detection 
Road sign recognition 
Road hazard detection 
Pedestrian detection 
Obstacle detection 
Vehicle detection 
(video, laser, infrared, radar ) 


— Side view monitor (video) 


|— Occupant sensor (video) 

Adjacent lane monitor 
(video) 


Driver state sensor 

- identity 

- fatigue 

- distraction-inattention 
(video) 



Blind-spot sensor 
Parking sensor 
Vehicle detection 
(video, radar, ultrasound) 


Fig. 62.5 Environment state sensing 


mon sensors are infrared [62.21], ultrasound [62.22], 
radar [62.23], laser range finders [62.24], and com¬ 
puter vision, which continually scan the environment, 
as shown in Fig. 62.5. Radar is generally used for 
obstacle detection at a distance, while infrared and 
ultrasound are used for close proximity obstacle de¬ 
tection. Laser ranging and image processing are used 
to more robustly recognize the road scene under vari¬ 
ous weather conditions. Certain road scene conditions 
such as road signs and traffic lights can only be under¬ 
stood using vision sensing. Sensor fusion is commonly 
used in intelligent vehicle applications, due to the com¬ 
plementarity of heterogeneous sensor modalities; this 
can greatly improve perception, for instance, between 
monocular vision and radar/laser sensors. Sensor fusion 
(Fig. 62.5) is critical technology for intelligent vehi¬ 
cles and constitutes a wide and active research field 
for both research teams and automakers (Chap. 35) 
(Fig. 62.6). 

Using such sensors it is possible to map the envi¬ 
ronment surrounding the vehicle and then, using tech¬ 
niques such as simultaneous localization and mapping 
(SLAM) (Chap. 46), generate the complex maneuvers 
needed for parking or for obstacle avoidance. 

62.2.3 Digital Maps and Satellite Data 

Other additional sensing is generally delivered by 
global positioning technologies (for example, GPS) to 
map the vehicle position and refer it to a description 
of the world known a-priori. This mapping procedure 
may also be very precise in some areas (like cross coun¬ 


try roads) and may be particularly noisy in others (like 
downtown areas). To fix issues related to multi-path 
measurements and to fill gaps when the GPS signal is 
missing, inertial measurements units (IMU) are gener¬ 
ally used together with GPS. 

Combining GPS with stored digital maps cre¬ 
ates a wide variety of intelligent vehicle applica¬ 
tions [62.27]. Map data can greatly assist in the problem 
of road scene interpretation, map data can improve 
lane detection quality, help deal with the problems 
when sensors such as cameras do not work, e.g., in 
sunlight, at dusk, or dawn. Digital maps are widely 




Fig.62.6a-e Multi-sensor fusion on a Toyota/INRIA experimen¬ 
tal vehicle (after [62.25,26]). (a) Urban scene as seen from a car, 
(c) occupancy grids, (d) lidar, (e) occupancy grid from a stereo cam¬ 
era, (b) fused data and detected objects 
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used in commercial navigation systems for route guid¬ 
ance [62.28] and are updated frequently. Such systems 
will be improved when real-time updating of the map 
will occur due to the availability of traffic informa¬ 
tion [62.29]. Additional functionalities to be added 
include curve approach warnings [62.30], curve speed 
control [62.3 1], traffic sign information, and speed limit 
information [62.32], as well as predetermining an ap¬ 
propriate heading of the headlights in curved and hilly 
roads. 

62.2.4 Driver State 

Additionally an intelligent vehicle needs to understand 
the driver state, if it is to give appropriate warnings 
or take action. Vision sensors [62.33] can monitor 
a driver’s attentiveness and fatigue by observing the di¬ 
rection of the driver’s gaze and eyelid behavior. In an 
emergency, knowledge of the position of the driver’s 
head as well as all other occupants’ position and posture 
can assist in the safe deployment of airbags. Also, after 
an accident, observing the state of the driver and other 
occupants could be useful for the dispatch of emergency 
services to the accident scene. 

62.2.5 Communication 

Communication technologies enable interesting appli¬ 
cations of intelligent vehicles. Allowing vehicles to 
communicate with each other and with the highway of¬ 
fers the prospect of vast improvements in the safety and 
efficiency of the road system. Applications include in¬ 
tersection collision avoidance, emergency braking, and 
sharing of road and traffic condition information. 

The basic communication modes are distinguished 
as: 

• Vehicle to/from roadside infrastructure 

• Vehicle to vehicle. 

Dedicated short-range communications (DSRC) 
have been set up to from the vehicle to infrastruc¬ 
ture communications to support traveler information, 
commercial applications (toll/parking fee collection, 
in-vehicle advertising), and safety applications (inter¬ 
section collision avoidance, approaching emergency 
vehicle warning, and rollover warning) [62.34]. 

DSRC is a wireless protocol specifically designed 
for intelligent transportation systems. It is a subset of 
RFID technology, working in the 5.9 GHz band (US) 
or 5.8 GHz band (Japan and Europe). It is generally 
implemented with a dedicated protocol, uses short mes¬ 
sages, and works in direct line of sight over short ranges 
(10-1000 m). 


Specific applications are being developed to test 
a broad variety of potential safety and mobility uses of 
the systems including: 

• Warning drivers of unsafe conditions or imminent 
collisions. 

• Warning drivers if they are about to run off the road 
or speed around a curve too fast. 

• Informing system operators of real-time congestion, 
weather conditions, and incidents. 

• Providing operators with information on road ca¬ 
pacity for real-time management, planning, and 
provision of road advisories to drivers. 

For intelligent vehicles safety-related applications 
using DSRC include: 

• Stopped vehicle or obstacle avoidance of obstacles 
on highways. 

• Merging assistance from a slow incoming lane into 
a crowded road. 

• Intersection safety and collision warning systems 
(with or without traffic lights). 

• Establishing and maintaining convoys/platoons. 

In a vehicle-based intersection collision warning 
system, each vehicle approaching the intersection gen¬ 
erates a message. The message includes the vehicle’s 
location, direction of travel, and the intersection loca¬ 
tion. This information is based on the GPS position and 
available map data base. Various strategies to predict 
driver intention are also being studied, and if available 
this information is also broadcast. Any other vehicle ap¬ 
proaching the intersection can calculate the possibility 
of a collision and will generate an appropriate warning 
for its driver. 

There numerous other possible applications. These 
include: 

• Notification of the presence or approach of an emer¬ 
gency vehicle 

• Notification of construction activity 

• Local multi-vehicle sensor fusion to provide better 
global maps and situation awareness 

• Energy saving applications (platooning, energy 
management for HEVs in stop and go traffic). 

The USA, since 1999, and the EU, since 2008, 
have located spectrum in the 5.9 GHz band for wire¬ 
less DSRC systems for vehicular safety and intelligent 
transportation system applications. The standards for 
DSRC in the US are IEEE 802.lip, finalized in 2010, 
and wireless access in vehicular environments (WAVE) 
(IEEE 1609), still in draft status as of luly 2010. The 
802.lip standard is derived from the 802.11A wireless 
LAN standard and thus provides a fairly easy and in¬ 
expensive route to produce communication electronics 
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and hardware. These standards provide network and ap¬ 
plication support, as well as sufficient range and data 
rates, for both V2V and V2I communication applica¬ 
tions. 

In addition to wireless communication standards, 
the development of standardized message sets and data 
definitions in the US has occurred in the SAE J2735 
standards process. 

The WAVE standard supports two types of devices, 
the roadside unit (RSU), and the on-board unit (OBU). 
The RSU would typically be a service provider de¬ 
ployed on the roadside that would provide map data 
like the geometric intersection data (GID) message. If 
it is connected to a traffic light it could also provide 
signal phase and timing (SPaT) messages. Both mes¬ 
sage types are defined in SAE Standard J2735 [62.35]. 
The OBU is responsible for receiving data from RSUs 
and generating and receiving safety messages that re¬ 
lay information about the vehicle’s current state. Many 
investigations of the utility and capabilities of these 

62.3 Road Scene Understanding 

Perception plays a key role in any robotic application. 
In the case of intelligent vehicles, the perception task is 
referred to as road scene understanding. It involves us¬ 
ing different sensors (described in Sect. 62.2) combined 
with automatic reasoning, in order to create a synthetic 
representation of the environment around the vehicle. 
The knowledge base accumulated by this task is then 
used either to issue warnings to the driver in the case 
of advanced driving assistance systems (ADAS) or to 
control vehicle actuators in the case of complete au¬ 
tonomous driving. A complete and precise description 
of the state of the surrounding environment is the key 
factor that allows the reduction of the number of false 
and missed alarms and provides the basis for smooth 
automatic driving. Needless to say, the perception of an 
outdoor environment - even if partially structured - is 
a challenging problem not only due to the intrinsic com¬ 
plexity of the driving environment itself, but also due 
to the impossibility of controlling many environmental 
parameters. Figure 62.7 shows examples of day/night, 
sun/streetlight illumination, temperature, poor visibil¬ 
ity, rain/snow, and different meteorological conditions, 
which in general are impossible to control and have to 
be faced by sensing devices. 

The research community is addressing the issue of 
providing vehicles with robust and precise perception of 
the state of the environment from two different perspec¬ 
tives. One approach is to provide vehicles with ever- 
increasing sensing capabilities and processing power 
aimed at the provision of powerful onboard intelligent 


standards exist [62.36] and large-scale testing is under¬ 
way in the US within the Connected Vehicle program. 

For vehicle-to-infrastructure communications 
[62.37], a communication protocol architecture is 
currently being specified by the ISO-SAE Technical 
Committee 204 Working Group 16 [62.38]. This 
protocol architecture is known as communication 
access for land mobiles (CALM) [62.39] and is 
being implemented in Europe under the cooperative 
vehicle infrastructure systems (CVIS) integrated 
project [62.40]. CALM is based on IPv6 protocols 
developed by the Internet Engineering Task Force 
(IETF) [62.41], particularly the network mobility 
(NEMO) protocol, which allows sessions to be main¬ 
tained between an in-vehicle internet protocol (IP) 
network and the Internet backbone using any type 
of available media (3G, general packet radio service 
(GPRS), Wi-Fi, WiMax, M5, DSRC, satellite, etc.). For 
vehicle-to-vehicle applications, systems that support 
ad-hoc networking are still in early stages. 


systems; [62.42]. An alternative approach is to use road 
infrastructure as an active component capable of com¬ 
municating with all vehicles and sharing information 
on road conditions in real time [62.43]. Indeed, these 



Fig. 62.7 Typical range of road scenes that intelligent ve¬ 
hicles must handle 
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two perspectives can also be merged to provide a mixed 
solution to safely control a vehicle and in dynamic en¬ 
vironments [62.43]. 

The task of road scene understanding may be ad¬ 
dressed differently, depending on the availability of an 
intelligent infrastructure and on other players exhibit¬ 
ing cooperative behavior. The task of understanding the 
state of the environment can be simplified through the 
availability of information coming from other sources, 
thereby limiting the need to perform a robust and 
complete sensing onboard each vehicle. Helpful infor¬ 
mation could come from the infrastructure itself (for 
example, road conditions and geometry, number of 
lanes, visibility, road signs, or even real-time informa¬ 
tion such as traffic-light status or traffic conditions) or 
other players (such as the presence of the vehicle with 
precise position, speed, and direction). The players may 
also carry real-time information gathered by and shared 
with other players. 

Although research is currently focused on both in¬ 
telligent vehicles and intelligent infrastructures, the first 
generation of production intelligent vehicles will have 
to rely primarily on their own sensing capabilities, 
since the availability of information coming from other 
sources such as the infrastructure and other vehicles 
will take a while to be deployed in real-world situa¬ 
tions. In fact, in order to be of practical use, intelligent 
roads must cover a large proportion of a country, and si¬ 
multaneously cooperative intelligent vehicles must also 
be sufficiently widespread. It is important to note that 
the investment in intelligent infrastructures and intel¬ 
ligent vehicles comes from different sources: mainly 
from governmental institutions for the former, and ve¬ 
hicle owners for the latter. 

The information that is owned by the infrastructure 
itself and that could be made available to the vehicles 
includes: 

• Precise geometry of the lane/road 

• Road signs 

• Status of traffic lights. 

On the other hand, the infrastructure can also assess 
and deliver real-time data such as: 

• Road conditions 

• Visibility 

• Traffic conditions. 

Another important piece of information that needs 
to be gathered by intelligent vehicles is the presence of 
other road players, such as: 

• Vehicles 

• Vulnerable road users (pedestrians, motorcycles, bi¬ 
cycles). 


Although it could be assumed that sometime in the 
future all vehicles will be equipped with active systems 
that allow them to be safely avoided by other vehicles, 
it is quite improbable that pedestrians and bicycles will 
have similar equipment; their presence will need to be 
detected using onboard sensors only. The same consid¬ 
eration also applies to obstacles that may unexpectedly 
be found on the road, or to temporary situations such 
as roadwork; if a vehicle needs to cope with the unex¬ 
pected, then it needs to have the capability to assess the 
situation in real time with its own sensors. 

This is why onboard sensing is of paramount im¬ 
portance for future transportation systems; vehicle-to- 
vehicle and vehicle-to-infrastructure communications 
may help and improve the sensing, but a complete sen¬ 
sor suite must also be installed on our future vehicles. 
The main challenges in road environment sensing are 
examined below. 

62.3.1 Road/Lane Tracking 

Many vehicle prototypes have been equipped with lane 
detection and tracking systems, starting from the very 
first implementations in the early 1980s [62.4-6]. In¬ 
deed, in this case computer vision plays a basic role; 
although generally the road can also be detected with 
laser scanners [62.24], the only generic technology 
able to detect lane geometry and lane markings with 
high precision is computer vision. Most lane tracking 
approaches have focused on detecting lane markings 
and exploiting structure in the environment, such as 
the parallelism of the left and right lane markings, 
the invariance of road width, or the widely used flat- 
road assumption. These assumptions were mainly used 
to overcome the problem of having a single camera 
(a choice driven by cost). Some systems use stereo vi¬ 
sion to detect lane markings and are able to work with¬ 
out such constraints. The problem of lane tracking in 
highway situations is basically a solved problem - with 
commercial systems being deployed in passenger and 



Fig. 62.8 Lane detection 
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commercial vehicles. A survey and evaluation of the de¬ 
ployed lane tracking technologies is available [62.44]. 
An example of the typical output of a commercial lane 
tracker is shown in Fig. 62.8. 

However, such systems cannot guarantee that lane 
detection systems will work with 100% reliability; 
these systems typically work with 95—99% reliability. 
Therefore, lane tracking systems are only being used in 
lane departure warning systems, since no failures can 
be tolerated for autonomous driving. Efforts are under¬ 
way to develop algorithms that will tolerate a variety 
of driving conditions, and push the 100% reliability 
boundary [62.44,45], l<Ef'iiWuaa. 

Vision-based lane tracking systems depend on lane 
markings, which are not always present. In these cases 
it is often not enough to track the lane or road’s lim¬ 
its. Instead it is necessary to build a local representation 
of the road’s shape and traversability. This local map 
can be built using SLAM-like techniques (Chap. 46), 
and can classify parts of the map as either static or dy¬ 
namic [62.46]. Alternative approaches aim to classify 
3-D points obtained from laser scanners or stereo cam¬ 
eras as belonging to the road or not [62.47], The output 
of the road detection function is a continuously updated 
representation of the navigable space in front of the ve¬ 
hicle (Fig. 62.9). 

62.3.2 Road Sign Detection 

Another fairly straightforward use of computer vision 
is road sign detection and understanding. Road signage 
is deliberately structured to aid human drivers. Road 
signs use a set of well-defined shapes, colors, and pat¬ 
terns. The signs are placed at consistent heights and 
positions in relation to the road. Therefore, reading road 
signs is an achievable task for computer vision. Detec¬ 
tion is done using a collection of shape and/or color 
detection schemes [62.48,49]. After the detection and 
localization phase, recognition takes place. Normally 
this task is performed by pattern-matching techniques 
such as image cross-correlation, neural networks, or 



Fig. 62.9 Instantaneous representation of the navigable 
space in front of the vehicle (after [62.25]) 


support vector machines, since the possible set of road 
signs is limited and well defined. Figure 62.10 illus¬ 
trates the concept of a speeding warning system based 
on speed sign detection. The challenge for research 
work in this area lies in the robustness of detection 
and the reliability of classification of signs. Most au¬ 
tomotive companies have systems under development, 
e.g., Siemens [62.50]. See video on speed sign detection 
kaXffiEim 

62.3.3 Traffic-Light Detection 

Color and pattern matching are also the key techniques 
used for the detection of traffic lights [62.51]. Although 
the detection of traffic lights is not overly complex, this 
application hides one further aspect that makes vehicu¬ 
lar applications difficult: besides the correct localization 
and recognition of a signal, special care has to be taken 
in checking the signal position and orientation on the 
road/lane since that signal may not be addressed to the 
current vehicle. This is particularly true in downtown 
intersections at which many traffic lights are visible at 
the same time; in this case the vehicle must have the ca¬ 
pability to select the correct traffic signal that must be 
obeyed. Some experiments have been undertaken with 
active traffic signals, able to emit the status of the traffic 
light using radio frequencies [62.52]. This involves ad¬ 
ditional infrastructure; at this stage vision seems to be 
the only simple viable solution. 

62.3.4 Visibility Assessment 

One of the key challenges is the detection of fog. The 
meteorological visibility distance is defined by the In¬ 
ternational Commission on Illumination (CIE) as the 
distance beyond which a black object of an appropri- 



Fig. 62.10 Road sign detection for speed warning applica¬ 
tion 
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ate size is perceived with a contrast of less than 5%. 
Different techniques for measuring this parameter - 
and thus detect foggy conditions - have been imple¬ 
mented [62.53]. Although many of the methods use 
vision, there are also efficient alternatives - generally 
used in fixed locations such as airports and traffic¬ 
monitoring stations - based on the use of multiple 
scattering lidar. The main challenge of using vision to 
estimate visibility is that a moving vehicle generally 
cannot rely on a specific reference point/object/signal 
at a specific distance. 

62.3.5 Vehicle Detection 

The detection of vehicles has been addressed using 
a large variety of sensor technologies, ranging from vi¬ 
sion to lidar, from radar to sonar. 

Despite being different in shape and color, vehi¬ 
cles share the same characteristics and feature a large 
size and reflective material. The position of vehicles 
is predictable once a rough indication of the road/lane 
position is available. Vehicles, in fact, can be suc- 



Fig. 62.11 Vehicle detection - lane and range position 


cessfully detected by many different sensors indepen¬ 
dently [62.55-57]. Figure 62.11 shows a vision-based 
vehicle detection system. 

Nevertheless, although the solution to this prob¬ 
lem seems straightforward, each sensor has its own 
domain of application and its own challenges. Vision 
is generally powerful, but may fail in low visibil¬ 
ity and bad-illumination scenarios (night or tunnels) 
or in heavy traffic conditions when vehicles may oc¬ 
clude each other. Vision in the infrared domain (thermal 
imaging) is able to detect vehicles with a high confi¬ 
dence since vehicle tires and mufflers generally exhibit 
high temperatures and are, therefore, easily detected in 
the image. However, parked vehicles, trailers, and even 
vehicles which have just started to move are colder 
than running vehicles and, therefore, less visible. Li¬ 
dar is generally robust, but has decreased sensitivity in 
adverse weather conditions. Radars, while cheap, can 
suffer from bias in lateral measurements due to the pres¬ 
ence of other nearby reflecting objects. Finally, sonars 
are applicable only for very short distances. The re¬ 
search challenge is to implement multisensor fusion 
robustly. A common approach is to fuse vision with 
radar [62.58]. 

Vehicle detection is often performed by grouping 
together data that are believed to belong to a same 
object, this often implies using some kind of sim¬ 
ilarity metric based in distance and other relevant 
features such as color, and texture, etc. An alternate 
approach is taken by model based techniques, which 
scan scene data trying to match models from a pre¬ 
existing catalog - e.g., geometric descriptions, part 
collections. Finally, grid-based approaches do not di¬ 
rectly model objects themselves, but the evolution of 
the properties in a regular decomposition of the space 
(Fig. 62.12) [62.54], 


Traditional 
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Model 
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Grid 

approach 



Fig. 62.12 Three alternative ap¬ 
proaches to vehicle detection and 
tracking (after [62.54]) 
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Bayesian sensor fusion + detection & tracking 



Fig. 62.13 Grid-based detection and tracking (after [62.59]) 


One of the main problems in tracking is data 
association : finding the correspondence between sen¬ 
sor readings and tracked objects. Tracking approaches 
differ in the way they solve this problem from the 
nearest neighbor, which just assigns the observation to 
the closes object to multi-hypothesis tracking (MHT), 
which maintains a representation of the different as¬ 
sociation hypothesis for several time steps - pruning 
is necessary for tractability. An increasingly popu¬ 
lar alternative is the use of grid-based approaches, 
which do not attempt to explicitly represent objects 
during the iterative grid updates while taking into ac¬ 
count all the possible associations with cells in a given 
neighborhood. This approach leads to postponing as 
long as possible the object detection and data asso¬ 
ciation steps [62.54]. Refer to Fig. 62.13 for an ex¬ 
ample of grid-based tracking. l<sr>M)l'J t'XfUB shows the 
predicitive capabilities of the Bayesian occupancy ap¬ 
proach [62.59]. 

62.3.6 Pedestrian Detection 

The detection of vulnerable road users (pedestrians and 
bicycles) is one of the most difficult tasks for intelligent 
vehicles. The appearance of a pedestrian is challeng¬ 
ing: a pedestrian shape can change greatly within a few 


tens of milliseconds; there are no clear invariants in 
color, texture, or size, and no assumptions can be made 
about posture, speed, or the visibility of parts of the hu¬ 
man body such as the head. Machine learning methods 
have been successfully applied to this problem [62.24, 
42,48, 60]. The focus is on achieving greater reliability 
and reduction in false alarms through improved vision 
algorithms [62.61]. However, the detection of vulner¬ 
able road users is one of the most relevant research 
topics worldwide since a great number of benefits - in¬ 
cluding insurance reductions - may be achieved once 
a fully functional pedestrian detector is available on 
cars. Countermeasures may be activated to reduce the 
consequences of vehicle-pedestrian accidents, such as 
the firing of external airbags or the opening of the 
hood to lessen the impact of a head-on collision. Cur¬ 
rently, with all possible technologies under evaluation, 
no solution seems to offer reliable detection in every 
scenario: radars are not able to detect pedestrians reli¬ 
ably in crowded scenarios, while vision has the many 
drawbacks listed above. Even thermal imaging, which 
- although still very expensive - is generally believed 
to be one of the most promising technologies, fails in 
some situations, such as hot summer days and, in gen¬ 
eral, in high-temperature environments. See video for 
vision based stereo vision detection l<sswti'H'fTf 


62.4 Advanced Driver Assistance 

Given the legal liabilities and technical challenges of 
achieving 100% reliability for autonomous intelligent 
vehicles, it appears likely that motor vehicles will have 


pieces of autonomous functionality added progressively 
and that cars will eventually evolve into autonomous 
robots. The perception techniques described earlier can 
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be used in a variety of ways to make driving safer, 
more efficient, and less demanding. Individual percep¬ 
tion techniques, or combinations of sensing modes, are 
being used to provide warnings to drivers of danger¬ 
ous situations. These warnings are being used to prevent 
collisions in a variety of situations, such as when back¬ 
ing up, when leaving a roadway, into rear ends, on 
lane changing/merging, with pedestrians, and at inter¬ 
sections. 

Developing a collision warning system requires 
many steps beyond building the perception system. For 
a typical example, roadway departure warning, the steps 
followed in a program initiated by the US National 
Highway Traffic Safety Administration include: 

1. Statistical studies. In the United States, crashes in¬ 
volving a single vehicle leaving the roadway are 
relatively rare, but disproportionately dangerous; 
approximately 40% of the 40 000 fatal crashes per 
year in the US are single-vehicle crashes where the 
vehicle leaves the roadway. The first part of the 
study looked at the prevalence of those crashes, and 
determined that this would be a good type of crash 
to prevent. 

2. Causal factors. The second step was to determine 
the causes of those crashes. Most run-off-road 
crashes are due to driver factors, such as exces¬ 
sive speed, inattention, or loss of control. This is an 
important observation, because it means that alert¬ 
ing the driver, or warning of difficult situations, 
could prevent those incidents. For the fraction of 
crashes caused by mechanical failure a warning sys¬ 
tem would not be useful; in this type of crash, 
mechanical failure is involved in less than 5% of the 
crashes. 

3. Opportunities for intervention. This part of the 
study set out to determine whether a warning sys¬ 
tem could be effective, and, if so, how far in advance 
the warning would have to be given. Given typical 
road departure trajectories, typical widths of roads 
and shoulders, and the range of potential steering 
responses, this task generated requirements for how 
accurately the system would have to track vehicle 
trajectory in order to predict a roadway departure. 

4. Human factors. Since the system being designed is 
a warning system, rather than an active control, it is 
crucial to understand what kinds of warning a driver 
(who may be distracted or sleepy) would respond to, 
and how quickly and accurately the response will 
be. Reaction times vary widely across individuals: 
using 1 s for reaction time is a fairly standard esti¬ 
mate. 

5. Simulator studies. A driving simulator is like an 
aircraft flight simulator for cars or trucks, with a va¬ 


riety of simulated roads and conditions. Simulator 
studies were used to test driver response to warn¬ 
ings: directional or nondirectional audio warnings, 
steering wheel shakes, and combinations. A nondi¬ 
rectional audio cue worked best. 

6. System specification development. Based on the pre¬ 
ceding steps, in order for a system to be useful, it 
needs to work at day and night, in almost all weather 
conditions, it needs to measure vehicle speed, lat¬ 
eral position relative to the road, lateral heading, and 
roadway curvature, and predict future vehicle trajec¬ 
tories long enough in advance to trigger a warning 
alarm. 

7. Perception and system development. Given those 
specifications, there are several ways that a percep¬ 
tion system could be built to sense the road and 
the vehicle’s trajectory relative to the road. For this 
particular test, a lane keeping system - the Rapidly 
Adapting Lane Position Handler (RALPH) was de¬ 
veloped and tuned [62.62], 

8. Limited tests. The entire system - sensors, process¬ 
ing, driver interface - was built and tested with 
a limited number of volunteers, and the system 
tuned and validated. 

9. Full-scale operational tests. The system was de¬ 
ployed in test fleets, including long-haul trucks and 
passenger cars. 

62.4.1 Collision Avoidance and Mitigation 

The complete cycle from the idea of using perception 
to prevent crashes, to full system development, took 
over 10 years in this case. The pure robotics part of 
the system is a crucial element, but is only one piece 
of the development needed to make a useful product. 
Some active control has already been assumed by to¬ 
day’s vehicles. Antilock brakes have been on the market 
for many years. Traction control systems which control 
throttle to stop wheel spin are being introduced. Elec¬ 
tronic stability control systems take this the next step 
further, controlling throttle and individual wheel brakes 
to help in cornering performance. So, gradually, people 
are willing to cede some control to very reliable auto¬ 
mated systems. We can expect this trend to continue. 

Each collision warning type has its own list of spe¬ 
cific development challenges, as described below. 

Backup Collisions 

The sensing challenge is to see relatively small ob¬ 
jects, such as fence posts or children’s toys, while not 
picking up false alarms from pavement joints or leaves 
and debris. The sensors used in today’s commercial ve¬ 
hicles are piezoelectric ultrasonic sensors, which are 
inexpensive [62.22]. However, ultrasonic sensors have 
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well-known limitations. The challenge of developing 
low-cost, accurate, reliable sensors remains. 

Rear-End Collisions 

These are among the most difficult collisions to prevent, 
with the most challenging sensing conditions. Rear- 
end collisions often happen at high speeds, requiring 
long-distance sensing of other vehicles (up to 100 m 
at US highway speeds, much longer at the high speeds 
found on some European roads or for the longer brak¬ 
ing distances needed for heavy trucks). That in itself is 
not too demanding a challenge: the sensed objects in 
this case are relatively large and have high metal com¬ 
ponents, so radar and lidar are both feasible sensing 
modes. The biggest range-sensing challenge is sorting 
out true targets (slow or stopped vehicles) from false 
targets (overhead signs or bridges, and side lobes from 
strong reflectors on the side of the road). It is also im¬ 
portant to determine if the sensed vehicle is in the same 
lane as the smart car, or a different lane. Sensing lane 
markings at such a large distance is a very difficult chal¬ 
lenge; merging lane sensing (often done by vision) with 
obstacle sensing (by a different sensor) and registering 
the two to within the resolution of a lane is a daunt¬ 
ing task. This technology remains under development 
through industry and government programs [62.63]. 

Lane-Change/Merge Collisions 
In the simplest case, the countermeasure to this kind 
of collision involves short-range sensing to cover the 
blind spots on the rear corners of a vehicle, where it 
is difficult to see with mirrors. For passenger cars, this 
area is quite small, and can be covered with a single 
sonar or radar. Often, the user interface is a warning 
light placed in the rear-view mirrors [62.64]; this rein¬ 
forces good driver behavior of checking mirrors before 
changing lanes. The sensing challenge for heavy trucks 
or transit buses is the same as for cars, except that the 
area not visible in planar mirrors can be much larger. 
Figure 62.14 shows examples of blind-spot detection 
for cars and heavy vehicles. 

The usual solution is a row of sensors along 
the side of the vehicle, although scanning lidar or 
panoramic vision have been used in experimental ap¬ 
plications [62.65]. The further complication for lane- 
change warnings is in high-speed driving, where it is 
important to look not just adjacently to the vehicle but 
a long way to the rear, to find overtaking vehicles with 
high relative speeds. The first commercial product was 
brought to market in 2007 [62.66]. 

Pedestrian Collisions 

Pedestrians are particularly important to detect, because 
pedestrians are much more vulnerable than people in 



Fig. 62.14 Blind-spot detection 


vehicles; as discussed earlier they are also, unfortu¬ 
nately, relatively difficult to detect and very hard to 
predict. Just detecting a pedestrian is not sufficient. In 
transit operations, for instance, a bus operates close 
to pedestrians much of the time. To do meaningful 
collision warning, it is important to detect the pedes¬ 
trian, detect their current path, look for cues such as 
crosswalks or curb edges that modify the probability 
of the pedestrian’s trajectory, and match all of these 
factors with the predicted trajectory of the vehicle. It 
is crucial to tune the warning system to produce few 
false alarms while not missing real alarms. A partic¬ 
ularly dangerous situation is pedestrians slipping and 
falling underneath a bus: these are very dangerous sit¬ 
uations, but very difficult to detect in time to warn 
the driver. For these reasons, automotive manufactur¬ 
ers have worked on products such as night vision to 
enhance driver perception [62.67], Car manufacturers 
are bringing to market products that use a data-fusion 
approach combined with radar-vision for emergency 
breaking in pedestrian collision situations [62.68] 

Other Obstacle Collisions 

Vehicles have collisions with many things other than 
other vehicles and pedestrians, for example, animals 
(deer, dogs, cats), car parts (tire carcasses, rusted-out 
exhaust systems), cargo that falls off trucks, construc¬ 
tion debris, etc. Warning drivers about these kinds of 
objects on the roadway is a challenging task. A com¬ 
mercial product for safety systems detects large animals 
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Fig. 62.15 Animal detection 


(Fig. 62.15) using vision and radar sensors [62.69]. 
A piece of construction timber on the roadway may 
be large enough to do significant damage to a car, but 
be small enough to be difficult to see, and be invisi¬ 
ble to radar. Some interesting work has been done with 
high-resolution stereo vision [62.70], with polarimetric 
radar [62.71], and with high-resolution scanning laser 
range-finders [62.72]. However, in general, this remains 
a difficult problem. 

Other Actions 

Besides warning the driver, there are other actions that 
an intelligent vehicle can take short of assuming con¬ 
trol. If a collision is inevitable, particularly from the 
side of the vehicle where there is limited crush space, 
the system can brake and deploy airbags even before 
physical contact. Of course, such a system would have 
to be nearly 100% reliable. More simply, if the system 
senses an imminent front collision, it can preload the 
power brakes, saving fractions of a second in brake re¬ 
action time. The driver must still actuate the brakes, 
but the onset of hard braking can be much quicker. 
At 100 km/h, a 0.1 s saving in braking actuation saves 
approximately 3 m of stopping distance, which can be 


the difference between a severe rear-end collision and 
a much lighter crash. Such systems are being introduced 
into the high-end market by all the major automotive 
manufacturers. 

Collision Avoidance 

The next step beyond emergency braking is an auto¬ 
mated system. Such systems have several advantages 
over a human driver: they have much quicker reaction 
times, access to sensors such as individual wheel speeds 
and slips plus external sensors such as radar or lidar, ac¬ 
cess to individual brake controls and other controls, and 
so forth. So, if the system had ideal situational aware¬ 
ness, it could, in many cases, do a better job of avoiding 
a collision than a human could. This is still a very dif¬ 
ficult area for implementation, however. First of all, the 
human has access to higher-level knowledge: the driver 
may be watching the behaviors of other cars, may make 
eye contact with pedestrians or drivers, may be watch¬ 
ing a policeman directing traffic, etc. So it may be that 
a maneuver that, to the system, looks like the best way 
to prevent the collision, is actually the wrong action to 
take. Secondly, in most countries, as soon as the vehi¬ 
cle takes control there is a shift in the liability for any 
resulting crash from the driver to the manufacturer. So 
there is a great reluctance to take active control. An al¬ 
ternative, recently developed approach is to observe the 
environment state with lidar, and monitor the dynamic 
vehicle state to determine whether the accident is un¬ 
avoidable. If the driver can no longer take corrective 
action, that is, brake or turn away safely, then emer¬ 
gency braking occurs [62.24,73]. For now, however, 
active collision avoidance remains a research area, with 
significant questions of reliability, human factors, and 
liability. 

Traffic Situation Awareness 
Traffic situation awareness is a higher level of un¬ 
derstanding that is required for decision making in 
traffic situations. For instance, knowing that a vehi- 



Fig. 62.16 A collision warning system for intersection safety implemented on communicating Renault experimental 
vehicles using the intention/expectation approach (after [62.26] ) 
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cle is traveling in the wrong direction along a street 
is much more informative for estimating the risk of 
the current situation than knowing its velocity. Thus, 
traffic situation awareness is emerging as a new area 
of research effort to understand and reason about the 
future motion of pedestrians and vehicles. The ob¬ 
tained predictions can then be fed to a motion planning 
algorithm, which can use this information to mini¬ 
mize collision probabilities [62.74,75]. Motion mod¬ 
els can be used to infer the maneuver intentions of 
drivers to change lanes [62.76], overtake another vehi¬ 
cle [62.77], make a turn at an intersection [62.78], or 
comply with the traffic laws at an intersection [62.79]. 
l-gtja'II'Jt'KTB shows how an embedded Bayesian per¬ 
ception system have been implemented on an experi¬ 
mental vehicle. 

Intersection Collisions 

Intersection collisions are particularly difficult to pre¬ 
vent because they often involve challenging sensing 
scenarios. Many of these collisions involve occluded vi¬ 
sion, with lines of sight blocked either by large vehicles 
or by adjacent buildings. They also often involve high 
closing rates from oblique angles, making it necessary 
to see a long distance with a very wide field of view. The 
solution that is usually proposed is to add intelligence to 
the infrastructure, either in fixed sensing (such as radars 
looking down each approaching road) or in some kind 
of radio relay that takes data from approaching smart 
cars and passes it to other approaching vehicles. All of 
these solutions are challenging: the large number of in¬ 
tersections makes it difficult to envision any universal 
solution. 

A complementary approach to traffic situation 
awareness is risk estimation, where the objective is 
to determine how dangerous each potential maneu¬ 
ver is for the vehicle. The classic approach is to use 
motion models to predict the possible future trajec¬ 
tories for all the dynamic objects in the scene, and 
then compute the probability of a collision between 
each possible pair of trajectories for each object. Pop¬ 
ular approaches for generating possible future trajec¬ 
tories include rapidly-exploring random trees [62.80], 
Gaussian processes [62.25,81], and stochastic reach¬ 
able states [62.82]. A major limitation of this clas¬ 
sic approach to risk estimation is its computational 
complexity. A recent approach to risk estimation has 
been proposed, which detects conflicts between the in¬ 
tentions of drivers and the expected behavior of the 
vehicle, rather than by detecting collisions between 
possible future trajectories [62.26,83] 

More specifically, dangerous situations are identified 
by comparing what drivers intend to do with what 


drivers are expected to do according to the traffic 
rules. This formulation of risk reflects the fact that 
most accidents are caused by driver error [62.84], and 
matches the intuitive notion that dangerous situations 
are situations where drivers act differently from what 
is expected of them. This approach can in theory be 
applied to any traffic situation, but so far has been 
implemented and tested for scenarios involving cars 
at road intersections only (Fig. 62.16). Significant re¬ 
search projects are underway in the EU for cooperative 
vehicle infrastructure projects such as managing inter¬ 
sections [62.85], 

Currently, automotive manufacturers are dealing 
with traffic situation awareness by specifically focusing 
on discrete driving tasks and typical scenarios. Com¬ 
bining perception with control gives partial automation 
for specific tasks, such as adaptive cruise control, lane 
keeping, assisted parking, and slow driving in stop-and- 
go situations. 

62.4.2 Adaptive Cruise Control 

Adaptive cruise control (ACC) is the logical extension 
of standard cruise control to also include keeping a safe 
distance from the preceding vehicle. If there are no ve¬ 
hicles in front of the smart car, it follows a set speed, 
as with standard cruise control. If a slower-moving ve¬ 
hicle is in front, an ACC-equipped car will sense the 
vehicle using radar or lidar, and slow to maintain a safe 
distance (typically set to a 1.5—2 s following gap). Fig¬ 
ure 62.17 shows an illustration of the ACC concept. The 
sensing challenge for ACC is much easier than the chal¬ 
lenge of rear-end collision countermeasures, since ACC 
systems are only designed to deal with other moving 
vehicles. 

The biggest sensing difficulty for rear-end collision 
countermeasures is separating stopped vehicles on the 
road from objects off the road; for ACC this difficulty 
is bypassed by ignoring all stopped objects. Moving ob¬ 
jects are classified as in-lane or out-of-lane based on 



Fig. 62.17 Advanced cmise control (ACC) 
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a number of heuristics. Often, the smart car’s own steer¬ 
ing radius is used as an estimate of the road curvature 
ahead, in order to determine whether vehicles ahead are 
in the same lane. Since the systems are explicitly sold as 
convenience instead of safety systems, they only need 
to deal with normal situations with relatively low dif¬ 
ferences in velocity, and they leave the more difficult 
situations up to the human driver. The human is still 
alert, controlling the steering, and watching the traffic. 
These systems are being introduced by all the major car 
manufacturers. 

62.4.3 Stop and Go 

Stop-and-go driving assistance (also referred to as low- 
speed ACC) is at the opposite end of the speed spec¬ 
trum, when vehicles creep along in dense traffic. At 
slow speeds, it is easy to track the vehicle ahead, and 
to move when it moves, steer when it steers, and stop 
when it stops. If the traffic accelerates to a modest 
speed, the stop-and-go system disengages, and the hu¬ 
man must assume control of the throttle, brake, and 
steering. Since the speeds are very slow and the dis¬ 
tances are short, many different sensing systems will 
work, such as stereo vision, radar, and lidar [62.86, 
87], 

62.4.4 Parking Assistance 

Parking assistance is also a low-speed aid. In a typi¬ 
cal scenario, the driver initiates the system by pushing 
a button when driving past an empty parking space. The 
system measures the length of the space using odom- 
etry, measures the positions of the cars in front and 
in back using short-range sensors, and infers the posi¬ 
tion of the curb by assuming that the surrounding cars 
are standard-sized cars parked near the curb [62.88]. 
Figure 62.18 illustrates parking assistance and fully au¬ 
tomatic parking [62.88] (l<Et>MHiIiE51). 


Once the system is fully engaged, it takes over 
steering, planning, and executing the ideal parallel park 
steering sequence. In some systems, the human is still 
responsible for throttle and brake, again ensuring that 
the human is alert, watching for encroaching pedestri¬ 
ans or other obstacles. Such a system has been intro¬ 
duced by Toyota [62.89]. 

62.4.5 Lane Keeping 

Lane keeping assistance is the natural extension to road 
departure warning systems. Given a lane tracking sys¬ 
tem, it is straightforward to add control of the steering 
wheel to keep the vehicle centered in its lane [62.4- 
6]. This has a number of uses. Some cities would like 
to run transit buses on narrow roadways, for instance 
the shoulder of highways or through narrow streets in 
old cities. Automated lane keeping systems using me¬ 
chanical guideways are in use in several places [62.90]. 
It is easier and less expensive if such systems can be 
electronic rather than relying on specially installed me¬ 
chanical guides. A specific subcategory is precision 
docking: for a transit bus to pick up a passenger in 
a wheelchair, either the bus must deploy a special ramp 
(which is a slow process), or it must pull up to a level 
dock and leave a very small gap, so that the wheel 
chair can safely roll on or off. Short-range precision 
docking systems use either a downward-looking sen¬ 
sor, looking at painted lines or magnetic markers, or 
a sideways sensor looking at the curb or dock, in order 
to guide the bus to its parking spot. Finally lane-keeping 
assistance is a convenience for driving on highways, 
especially with gusty winds. Honda has released a ve¬ 
hicle equipped with both lane-keeping assistance and 
adaptive cruise control (ACC) [62.91]. The danger with 
such systems is that the driver no longer has an active 
moment-to-moment role, and may lose concentration or 
even fall asleep. These systems are not designed for full 
automation, and still require a driver to handle unusual 
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circumstances. The next stage is to integrate driver state 
monitoring. If the driver is inattentive, then all auto¬ 
matic systems are disengaged. 

62.4.6 Lane Changing 

Lane changing assistance is the next extension in par¬ 
tial autonomy. It combines lane keeping and ACC with 
blind-spot monitoring. If the car can safely overtake 
a vehicle, then a lane change is undertaken and speed 
is unchanged [62.92]. Otherwise, the ACC slows the 
vehicle down. At its simplest, such a driver assistance 
system can advise a driver whether a lane change can 
be safely undertaken [62.93]. In its most advanced form 
the lane change is undertaken completely automatically 


62.5 Driver Monitoring 

There has been an evolution of thought regarding the 
role of the driver in intelligent vehicles. The grand goal 
has been to replace the driver with a fully automated 
system. As discussed earlier, full automation of intel¬ 
ligent vehicles is still some years away due to system 
reliability and legal liability reasons. The next step in 
the development of motor vehicles is partial automa¬ 
tion - where individual autonomous functions such as 
ACC, lane keeping, lane changing etc., are developed. 
Motor vehicle designers have realized that the driver 
cannot be removed from the vehicle and must instead 
be supported by systems onboard a motor vehicle. Over 
92% of motor vehicle accidents are caused by driver 
error [62.94]. It is likely that the next generation of in¬ 
telligent vehicles will work in the following way: 

1. The vehicle will monitor the road scene using the 
advanced driver assistance system technologies dis¬ 
cussed earlier to assess the state of the environment 
and warn the driver of dangerous situations, e.g., 
lane departure warnings. 

2. The vehicle will also monitor the driver using vi¬ 
sion and other physiological sensors to assess the 
state of the driver. If a driver is fatigued, drowsy, 
inattentive, distracted, under the influence of drugs, 
or has an emergency health condition such as heart 
attack, then accidents can occur. The vehicle warns 
the driver of dangerous circumstances, e.g., drowsi¬ 
ness warning. 

3. For legal liability reasons, intelligent vehicles will 
not take control, rather the driver will be alerted 
using visual, audio, or tactile warnings. The ve¬ 
hicle will not perform collision avoidance; rather 



Fig. 62.19 Lane changing - side detection 

by the vehicle [62.4]. Such systems require an ad¬ 
ditional side-facing sensor, typically radar (shown in 
Fig. 62.19) 


collision mitigation will occur through emergency 
braking. 

4. If an accident is unavoidable, the vehicle can au¬ 
tonomously apply emergency braking. To maximize 
the safety of the occupants in the vehicle, seat belt 
restraints are tightened and airbags are safely de¬ 
ployed. 

5. After an accident has occurred, knowledge about 
the state of the driver and the passengers is impor¬ 
tant. If an occupant has been injured, a call to the 
emergency services can be dispatched automatically 
by the vehicle. 

In all the steps described above monitoring the 
driver is critical. For future advanced driver assistance 
systems (ADAS) to work safely the driver should be 
put in the loop, for example, in lane departure warning 
systems, it is not possible to determine whether a vehi¬ 
cle departing from a lane is due to driver intention or 
error. If the state of the driver is being monitored, and 
the system can detect that the driver’s eyes are closed 
or the driver is looking away from the road, then it can 
be inferred that the lane departure was involuntary and 
that a lane departure warning should be given to the 
driver. For ADAS to be accepted by drivers, the systems 
should not give false warnings. If the driver is looking 
directly at the road then lane departure warnings should 
not be given (or a different subtle warning should be 
given). Similarly more sophisticated systems, such as 
lane keeping, should not be engaged unless the driver 
is fully attentive and has his/her hands firmly on the 
steering wheel. The key point is that drivers must be 
fully engaged with and in control of the driving task. 
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This is a most important consideration in the design of 
ADAS for intelligent vehicles. A good overview of the 
challenges associated with putting drivers into ADAS 
systems is given in [62.95] 

Combining perception with control gives partial au¬ 
tomation for specific tasks, such as adaptive cruise 
control, lane keeping, assisted parking, and slow driv¬ 
ing in stop-and-go situations. 

62.5.1 Driver Fatigue, Inattention, 
and Impairment 

By directly monitoring the driver using visual sens¬ 
ing opens up the possibility of developing a new class 
of ADAS applications. It is possible to monitor driver 
state through monitoring signals such as an electro¬ 
cardiogram (ECG), temperature, etc. However, market 
studies by automobile manufacturers have shown that 
people do not like to have wires or gadgets attached; 
driver monitoring must be noncontact and noninva- 
sive. The only solution is to use vision as the sensing 
medium. The technical challenge to develop a computer 
vision system that can automatically detect a driver of 
any age, sex, race, with/without eye or sunglasses, and 
with/without facial hair is enormous. Recently, signifi¬ 
cant progress was made with systems being developed 
that can also detect where a person is looking (gaze di¬ 
rection) [62.33,96]. 

Once the driver’s state (head position, eye gaze, eye 
blink rate) can be measured, then ADAS applications 
can be developed. Figure 62.20 shows the output of 
a commercial driver state detection system. See video 
for driver distraction, inattention and fatigue detection 


Driver Impairment 

Safety authorities estimate that as many as 50% of all 
road accident fatalities are caused by driver impairment 



Fig. 62.20 Driver state detection 


due to alcohol or drugs [62.97]. Recent research has 
shown that driver impairment can be detected by sens¬ 
ing abnormal scanning patterns of eye gaze. It promises 
to open up a new class of ADAS. There have been 
major education and legislative initiatives in many Or¬ 
ganization for Economic Cooperation and Development 
(OECD) countries, resulting in a significant reduction in 
road fatalities. However, the difficult cases (fatigue, dis¬ 
traction, and inattention) have become more prominent. 
ADAS technologies could have a significant impact in 
further reducing the road toll. 

Driver Fatigue 

Safety authorities estimate that 25—30% of all road 
fatalities are caused by driver fatigue [62.98,99]. Re¬ 
search has shown that there are four visible factors 
that indicate the onset of fatigue - prolonged eye clo¬ 
sure, uncontrolled head moments, drooping eyelids, and 
reduced eye-gaze scanning. Systems are under devel¬ 
opment that focus only on eye closure [62.100]; the 
challenge of fusing all four factors together into a ro¬ 
bust algorithm that works for a wide range of drivers 
remains an open research problem. 

Driver Inattention 

Safety authorities estimate that up to 45% of all traffic 
accidents - from minor dents to serious incidents - are 
caused by driver inattention or distraction [62.101]. Re¬ 
search has shown that if the drivers consistently keep 
their eyes on the road, then driving becomes a much 
safer experience [62.102]. All major automotive manu¬ 
facturers are developing ADAS applications that warn 
the driver if they are distracted from the driving task. 
More sophisticated ADAS under development use the 
driver’s gaze direction to check whether safe driving 
practices are being followed, for example, did the driver 
check the side mirror before a lane change was per¬ 
formed? If the driver fails to check the side mirror 
a warning would be issued. Systems of this type have 
been demonstrated [62.103]. 

Driver Workload 

The increase in new electronic systems and gadgets 
that are being installed into today’s motor vehicles is 
also another source of distraction. Questions arise about 
when and under circumstances should a driver change 
a compact disc (CD) or answer a phone call. Research 
is underway into the development of workload sys¬ 
tems. These systems take into account the vehicle state 
(speed, acceleration, braking, gear-change yaw rate, 
etc.) to determine whether information management 
tasks such as answering a phone call, sending a short 
message service (SMS) message are allowed [62.102]. 
The next stage of research is to include state infor- 
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mation about the road scene and the driver. If the car 
is driving on a country road, and the driver is atten¬ 
tive, then distractive tasks could be allowed and man¬ 
aged. Systems for monitoring the scene for monotony 
have been combined with fatigue detection systems - 
and can be combined with information management 
tasks [62.104] 

62.5.2 Driver and Passenger Protection 

The automotive industry is moving towards the use of 
active safety systems such as airbags to complement 
passive systems such as seat belts. ADAS will play 
a critical role in active safety systems. In emergency 
braking or impending rear-end collision situations, seat 
belts can be pretensioned and airbags primed. Drivers 
and passengers could be further protected through the 
development of smart airbags [62.105]. Airbags, while 
considered an essential part of a modern car, can cause 
fatalities if the occupant is too close to the airbag, is 
a child, or is not wearing a seat belt. Smart airbags 
are dependent on the position of the vehicle occu¬ 


pant’s head. The driver state monitoring technologies 
discussed earlier are an essential component of smart 
airbags. 

62.5.3 Emergency Assistance 

The speed at which people are given treatment after 
a serious accident is critically related to the survival rate 
of victims. While it is expected that ADAS will lead 
to a reduction in road accidents, it will not eliminate 
accidents. Therefore, it is important that automated sys¬ 
tems for emergency situations are developed. Systems 
that use GPS, vehicle state information, and mobile 
communication systems have been developed that send 
the world coordinates of the vehicle to emergency au¬ 
thorities after an accident [62.106]. This information 
can be augmented by using driver monitoring tech¬ 
nology to assess the condition of the occupants of 
the vehicle. Such types of system are referred to as 
Telematics are currently being tested by major automo¬ 
tive suppliers, including the development of aftermarket 
products [62.107]. 


62.6 Towards Fully Autonomous Vehicles 


Intelligent vehicles that are fully autonomous can be 
justified for safety, traffic congestion, and environmen¬ 
tal considerations. 

62.6.1 Operating Safely 

The first problem is safety. As stated earlier, automobile 
accidents are one of the main causes of human fatal¬ 
ities - on a staggering scale. This is a catastrophe of 
larger magnitude than all armed conflicts since the be¬ 
ginning of mankind. The most economically advanced 
countries (OECD) have been able to reduce the number 
of fatalities significantly through improved technology 
in vehicles - improved handling, braking, and pas¬ 
sive safety. Transport infrastructure has been greatly 
improved; modern freeways tire ten times safer than 
regular roads [62.108]. However, these improvements 
appear to be reaching a limit in terms of the number of 
deaths per million passengers-kilometers, particularly 
in industrialized countries. 

As discussed earlier, motor vehicles are inherently 
dangerous due to their reliance on human control. Slight 
errors at high speeds can have catastrophic results. As 
discussed earlier the most common cause is driver dis¬ 
traction, which leads to improper reaction times or driv¬ 
ing actions. Human error also frequently occurs while 
handling emergency situations. A large percentage of 
drivers will take improper action in such situations and 


produce an accident that could have been avoided by 
a skilled and attentive driver [62.108]. The best solu¬ 
tion to these problems is to remove the driver from the 
control loop. As discussed earlier, the interim step is to 
assist the driver to warn him/her in the case of potential 
danger (e.g., in the case of excessive speed before a dan¬ 
gerous bend or when a car is present in the blind spot 
while changing lane), or to take over control in emer¬ 
gency situations (e.g., emergency braking in the case of 
impending collision). For legal liability reasons, until it 
can be shown that autonomous systems have high in¬ 
tegrity and reliability, people must be kept in the loop - 
in a supervisory capacity. 

62.6.2 Traffic Congestion 

The success of the automobile also leads to saturation 
of the road infrastructure, particularly in cities. Each 
car needs a certain amount of space in order to oper¬ 
ate safely. The usual width of roads is 3.5 m in order 
to accommodate steering imprecision, while vehicles 
are about 2 m in width. Spacing between vehicles also 
has to be kept at a safe minimum to prevent colli¬ 
sions during deceleration (this principally depends on 
driver reaction time). It is usually recommended that 
the spacing should correspond to a time gap of at least 
1.5 s. This spacing leads to a maximum throughput of 
about 2200 cars per hour per lane, independent of traf- 
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fic speed. This is not high if we consider that a suburban 
train can carry about 60000 passengers per hour on 
an infrastructure of similar dimensions. Furthermore, 
high-density car traffic of greater than 2200 cars leads 
to a breakdown in traffic flow (stop-and-go traffic) and 
to an increased likelihood of accidents, which can dras¬ 
tically decrease the overall capacity of the system. The 
solution to this problem also lies in the removal of the 
driver from the control loop to improve lateral guid¬ 
ance (reduction of the width of lanes) and longitudinal 
control (with possible time gaps of around 0.3 s, inde¬ 
pendent of speed) while maintaining traffic safety. Such 
techniques of automated driving could multiply the 
throughput of road infrastructure by a factor of 5. This 
was demonstrated in particular by the work performed 
in the advanced highway systems (AHS) project in the 
USA, which led to the demonstration of seven-car pla¬ 
toons running at speeds up to 130 km/h on a dedicated 
freeway in San Diego with gaps of about 0.5 s [62.109], 
Another congestion problem is associated with 
parking. Every individual vehicle is only used for 
a small percentage of its total usable life. Most of the 
time, motor vehicles occupy space very unproductively. 
Typically, a car requires about 10 m 2 of space. Usually, 
parking occurs at the curbside - a space that is very 
limited in large cities and cannot accommodate all cars 
of residents and visitors. In parking lots, each car will 
need four times this amount in order to have access to 
each individual slot. If a transportation system based on 
fully automated cars could be developed, people would 
not need to own cars but could rely on a service such as 
the one offered by taxis with vehicles that would come 
on demand and offer a complement to mass transport. 
This is the concept of the cybercar, which is under de¬ 
velopment in Europe [62.1 10]. 

62.6.3 Environmental Factors 

The ever-increasing deployment of passenger and com¬ 
mercial vehicles has led to critical environmental prob¬ 
lems of noise and of pollution in the local community. 
In addition greenhouse-gas emissions have an impact 
at the global environment level. Recently, automotive 
manufacturers have been able to reduce emissions of 
local pollutants drastically; noise in cities is now per¬ 
ceived as the major problem by the inhabitants. At the 
global level, the generation of CO 2 through the use of 
fossil fuels is also considered to be a major problem, 
which will require drastic steps. In the short term, this 
may lead to limiting the use of vehicles that gener¬ 
ate CO 2 above certain levels. In the longer term, this 
will lead the automotive industry to offer vehicles that 
run on various forms of energy and to new forms of 
transportation systems with much higher efficiency. Au¬ 


tomated vehicles running in platoon formations on new 
infrastructures could form such a system. 

62.6.4 The Automobile of the Future 

All three challenges are now at the heart of new poli¬ 
cies being developed in many countries. These policies 
concern safety and emissions features in vehicles with 
a strong push towards advanced safety systems, as was 
recently seen in Europe with the Intelligent Car Initia¬ 
tive [62.111], At the infrastructure level, there is also 
a strong push to implement regulation schemes to limit 
and control the use of road transport and promote alter¬ 
native and more efficient transport means. In the future, 
this means that the use of a private automobile will be 
much more regulated and more integrated with other 
modes of transport. 

The automotive industry might move from an in¬ 
dustry of products to an industry of service where 
anyone can have access to mobility in the most cost 
efficient way. Both mass transportation and individ¬ 
ual transportation would be offered by companies in 
the most cost-efficient way, respecting local regula¬ 
tions. Companies operating in this service industry 
could be transit operators, taxi companies, car rental 
companies, car-sharing companies or even new en¬ 
trants into the transportation business such as mo¬ 
bile phone companies, which are already familiar 
with large customer bases and mobility services. Car¬ 
sharing operators such as those operating in Germany, 
Switzerland, Japan, and the USA are likely candi¬ 
dates [62.112]. Figure 62.21 shows a commercial car¬ 
sharing operation. 

In this context, new types of vehicles such as elec¬ 
tric ones and automated driving are being developed 
because of the decrease in the cost of operation and im¬ 
provements in safety. 

Demonstration projects by Honda and by Toyota 
have already been put in place along these lines. How- 



Fig. 62.21 Car sharing at Toyota 
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ever, this market is still searching for its operators and 
business model, as well as the right products [62.110]. 
Figure 62.22 shows the Honda concept automated car. 

There are two distinct trends in the future of au¬ 
tomated vehicles. One is with advanced driving assis¬ 
tance, which has spread rapidly since the late 1990s 
with numerous techniques appearing recently in high- 
end passenger vehicles and commercial vehicles (buses 
and trucks). This trend was described earlier in this 
chapter. The other trend is associated with the arrival 
of people-movers based on automated guided vehicles 
(small or large) in specific locations and on dedicated 
tracks (protected or not). 

It is forecast that, in the next 10 years, these two 
trends will merge, with individual vehicles having dual¬ 
mode capabilities: manual (with strong control and 
assistance) driving on regular roads and fully automatic 
driving on dedicated zones where no (or few) man¬ 
ual vehicles will be allowed, thereby ensuring smooth 
and safe operation of automated vehicles [62.1 10]. This 
type of vehicle will be perfect for the implementation 
of mobility services with vehicles that can be called on 
demand (perhaps through a mobile phone) when and 
where needed. With the development of such zones, 
new dedicated infrastructures will be built specifically 
for these vehicles to go automatically and at high speed 
from one automated zone to the next. This appears to 
the most realistic path for automated highways to be re¬ 
alized, since it is now considered nearly impossible to 
have a smooth evolution from the actual infrastructure 
with its manual vehicles to one with a majority of fully 
automated ones. This is one of the reasons why the AHS 
project was canceled in the USA despite a very success¬ 
ful demonstration in 1997. 


62.6.5 Automated Vehicle Deployment 

As discussed earlier some automated functions are 
being introduced in production vehicles. Honda and 



Fig. 62.22 Automated electric cars by Honda 


Toyota have introduced a combination of lateral con¬ 
trol (lane keeping assistance using image processing) 
and longitudinal control (adaptive cruise control using 
laser and radar sensors) [62.91]. An intelligent parking 
assist system was introduced recently by Toyota, offer¬ 
ing the ability for the car to be parked without the driver 
using the steering wheel. However, this system does not 
use sensors: the driver has to position his/her vehicle on 
the image of the rear camera [62.89]. However, most 
car manufacturers and some component manufacturers 
are actively working on the sensors to remove this task 
from the driver. 

Fully automated vehicles without any human inter¬ 
vention or supervision are now appearing in the com¬ 
mercial domain. Automated bus rapid transit (ABRT) 
combines the service quality of rail transit with the 
flexibility and cost of buses (as shown in Fig. 62.23). 
Nonautomated bus rapid transit (BRT) is already rec¬ 
ommended by the World Bank in developing coun¬ 
tries as the most efficient mass transport system. 
By reserving a dedicated lane for bus operation and 
adding some light infrastructure to facilitate loading 
and unloading, capacities similar to those of a train 
(60 000 passengers/h/direction) can be obtained, as has 
been demonstrated in South America [62.113]. By 



Fig. 62.23 Toyota ABRT used in IMTS Phileas 
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Fig. 62.24 Cybercars at Schipol airport, Amsterdam 


adding the driving automation on a BRT, the system 
can be made more efficient and safer, as is already the 
case with automated metros. 

The intelligent multimode transit system (IMTS) 
consists of vehicles guided by magnetic markers imbed¬ 
ded in the middle of their dedicated roads. The platoon 
running function (three electronically linked vehicles 
running in file formation at uniform speeds) of IMTS 
consists of precisely controlling the speed of all the ve¬ 
hicles in the platoon to be the same at all times [62.1 14]. 
In the city of Eindhoven in The Netherlands, multiar- 
ticulated buses with several steering axes also run in 


automated mode on a dedicated track using magnetic 
markers [62.115]. These ABRTs usually keep a driver 
in the vehicle, since there is the possibility of reverting 
to manual mode if need be (such as for an unexpected 
obstacle or in situations with many pedestrians); how¬ 
ever, future plans are for full automation. 

ABRT technology can also be found with smaller 
vehicles, now called cybercars, for on-demand door-to- 
door operation [62.110]. These vehicles were put into 
operation for the first time at Schipol airport (Amster¬ 
dam) in December 1997 to move passengers from long¬ 
term parking lots to the airport terminal (Fig. 62.24). 
Since then these cybercars have been further developed, 
financed by the Information Society Technologies (1ST) 
program of the European Commission [62.110]. The 
long-term future of cybercars may lie with the develop¬ 
ment of dual-mode vehicles, with a particular emphasis 
on car-sharing operations [62.112]. Cybercars will have 
an automatic mode for operation in city centers (re¬ 
stricted to this type of vehicle) and a driver-operated 
(with assistance) mode for regular roads. 
illustrates the cybercar concept. The automobile indus¬ 
try is examining the viability of the development of 
such vehicles. 

In order to match the performance of manually 
driven vehicles in urban environments, automated ve¬ 
hicles face major technical challenges. In particular, 
planning a trajectory for a car while avoiding moving 
and static obstacles remains as an important research 
challenge. In 2005, the DARPA Grand Challenge de¬ 
scribed at the beginning of this chapter brought together 
a large number of these researchers to demonstrate the 
feasibility of automated driving techniques. Five ve¬ 
hicles succeeded in completing the difficult course of 
132 miles in the desert in totally autonomous mode. 
The 2007 DARPA challenge focused on multiple vehi¬ 
cles operating simultaneously in an urban environment; 
this challenge and the subsequent Google cars has 
certainly shown that automotive products based on au¬ 
tonomous vehicle techniques are becoming closer to 
market [62.15]. 


62.7 Future Trends and Prospects 

Our cars, trucks, and buses are inevitably getting 
smarter. The future path is not completely known, and 
is likely to vary in different locations, but it is be¬ 
coming widely accepted that there will be no need to 
drive our cars in the remote future. The time frame 
is still undefined, but it appears that autonomous cars 
will populate our highways - perhaps using special 
lanes- in less than a decade. Governments will not 


only have to allow this change, but will have the 
great opportunity to influence its deployment, for ex¬ 
ample by increasing the number of dedicated lanes 
and the benefits that autonomous vehicles will obtain 
as opposed to less safe driving of manually driven 
cars. 

Four different states in the USA have paved the 
way for having autonomous vehicles driven on regular 
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roadways (albeit for testing purposes), moving towards 
legislation that will make autonomous cars legal on ev¬ 
eryday roads. This trend is not only happening in the 
USA, but also in Europe. Indeed, differences in culture 
and in driving habits will bring driverless technology 
into reality in selected locations earlier. Countries in 
which particularly fast and disrespectful driving is toler¬ 
ated will make the introduction of autonomous vehicles 
challenging. Additionally, the safety issues associated 
with intermingling of autonomous and human driven 
cars have become critical and extensive investigations 
are again underway. 

Several trends are clear, however. As sensor- 
equipped vehicles become more common, there will be 
increasing opportunities to build sensor-friendly tech¬ 
nologies into roadways and into vehicles. It is straight¬ 
forward to reduce the amount of radar clutter on road 
sides by replacing large metal signs, which are excellent 
radar reflectors, with nonreflective signs made of plastic 
or composites. At the same time, it will be desirable to 
increase the radar cross-section of small vehicles, such 
as motorcycles, to make them easier for radar-equipped 
cars to detect. Radar-reflecting licence plates, for in¬ 


stance, would make smaller vehicles stand out more 
clearly. 

The next steps may be to deploy systems that 
support the active transmission information from in¬ 
frastructure to vehicle and vehicle to vehicle. Dedicated 
short-range communications (DSRC) is already in wide 
use in smart toll collection stations. It is easy to imagine 
DSRC-equipped vehicles receiving alerts from road¬ 
side sensors of upcoming fog, or ice, or congestion. 
Intelligent vehicles that detect unusual roadway condi¬ 
tions (via radar or lidar, slip detectors, or driver actions 
such as hard braking) could broadcast that information 
to all nearby smart vehicles. As market penetration of 
equipped vehicles increases, the value of such ad-hoc 
communications nets will go up dramatically. 

Vehicle-to-vehicle communications need not be 
only via radio. Light-emitting diode (LED) brake lights 
can be pulsed at kilohertz frequencies, well above the 
bandwidth at which human eyes can detect flicker, but 
easily received by following vehicles. This would be 
a straightforward way for a vehicle to tell other vehi¬ 
cles about the onset of hard braking, or other unusual 
roadway conditions. 


62.8 Conclusions and Further Reading 


Advances in both the technologies and the algorithmic 
aspects of developing autonomous vehicles continue. 
Presently, impressive new systems have been demon¬ 
strated (Bertha [62.116] and PROUD [62.117]). In the 
first case, a Mercedes-Benz S 500 traveled on the 
historical route between Mannheim and Pforzheim in 
Germany in August 2013, while the PROUD experi¬ 
ment involved urban driving in open traffic in the inner 
city of Parma, Italy, and a part of the trip was made with 
nobody behind the steering wheel (l<E?KiiU3!ll£II). 

The general public will become increasingly ac¬ 
customed to intelligent systems - sensors and com¬ 
munications on vehicles. The only showstopper in the 
large-scale deployment of robotics technologies in the 
automobile field is the ability of the industry to de¬ 
liver the technologies with total safety, which focuses 
attention on the problems of failsafe systems and their 
certification. The rail and aerospace industries have 
solved these problems but in a very different environ¬ 
ment. In these industries, the cost of safety for each 
vehicle can be much higher than in a car or a bus, 
and the operational environment is also quite differ¬ 
ent, with professionals operating and maintaining the 
system. Manufacturers will slowly develop experience 
in reliability and cost engineering, and governments 


will gradually work out liability issues. For these rea¬ 
sons, systems that put the driver in the loop of new 
automation technologies will be the current focus of 
development. 

Many issues will need to be carefully addressed, in¬ 
cluding user acceptance, liability, and safety; however, 
it seems quite probable that this technology will lead to 
an unprecedented revolution in the held of transporta¬ 
tion and people mobility. 

For further reading on this topic there are a number 
of sources that can be recommended: 

• Intelligent vehicles survey text [62.9, 10] 

• Intelligent transport systems journals (IEEE Trans¬ 
actions on Intelligent Transportation Systems, IEEE 
Transactions on Vehicular Technology, Transporta¬ 
tion Research PART B: Methodology, IEEE Trans¬ 
actions on Intelligent Vehicles) 

• Annual conferences (Intelligent Vehicles Sympo¬ 
sium, Intelligent Transportation Systems Confer¬ 
ence, Vehicular Technology Annual Conference, 
World Congress on Intelligent Transportation Sys¬ 
tems, Intelligent Vehicle Systems Symposium) 

• Government resources [62.118-121], 

• Magazines (ITS Magazine). 
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PR0UD2013 - Inside VisLab's driverless car 

available from http://handbookofrobotics.org/view-chapter/62/videodetails/178 
VIAC: The VisLab Intercontinental Autonomous Challenge 
available from http://handbookofrobotics.org/view-chapter/62/videodetails/179 
Motion prediction using the Bayesian occupancy filter approach (Inria) 
available from http://handbookofrobotics.org/view-chapter/62/videodetails/420 
Cybercars and the city of tomorrow 

available from http://handbookofrobotics.org/view-chapter/62/videodetails/429 
Bayesian embedded perception in Inria/Toyota instrumented platform 
available from http://handbookofrobotics.org/view-chapter/62/videodetails/566 
Inria/Ligier automated parallel parking demo in an open parking area 
available from http://handbookofrobotics.org/view-chapter/62/videodetails/567 

Collision avoidance at blind intersections using V2V and intention / expectation approach (Inria & Renault) 
available from http://handbookofrobotics.org/view-chapter/62/videodetails/822 
Lane tracking 

available from http://handbookofrobotics.org/view-chapter/62/videodetails/836 
Speed sign detection 

available from http://handbookofrobotics.org/view-chapter/62/videodetails/838 
Pedestrian detection 

available from http://handbookofrobotics.org/view-chapter/62/videodetails/839 
Driver fatigue and inattention 

available from http://handbookofrobotics.org/view-chapter/62/videodetails/839 
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63. Medical Robotics 
and Computer-Integrated Surgery 


Russell H. Taylor, Arianna Menciassi, Gabor Fichtinger, Paolo Fiorini, Paolo Dario 


The growth of medical robotics since the mid- 
1980s has been striking. From a few initial efforts 
in stereotactic brain surgery, orthopaedics, endo¬ 
scopic surgery, microsurgery, and other areas, the 
field has expanded to include commercially mar¬ 
keted, clinically deployed systems, and a robust 
and exponentially expanding research community. 
This chapter will discuss some major themes and 
illustrate them with examples from current and 
past research. Further reading providing a more 
comprehensive review of this rapidly expanding 
field is suggested in Sect. 63.4. 

Medical robots may be classified in many ways: 
by manipulator design (e.g., kinematics, actua¬ 
tion); by level of autonomy (e.g., preprogrammed 
versus teleoperation versus constrained coopera¬ 
tive control), by targeted anatomy or technique 
(e.g., cardiac, intravascular, percutaneous, la¬ 
paroscopic, microsurgical); or intended operating 
environment (e.g., in-scanner, conventional op¬ 
erating room). In this chapter, we have chosen to 
focus on the role of medical robots within the 
context of larger computer-integrated systems 
including presurgical planning, intraoperative 
execution, and postoperative assessment and 
follow-up. 

First, we introduce basic concepts of computer- 
integrated surgery, discuss critical factors affecting 
the eventual deployment and acceptance of 
medical robots, and introduce the basic system 
paradigms of surgical computer-assisted planning, 
execution, monitoring, and assessment (surgical 
CAD/CAM) and surgical assistance. In subsequent 
sections, we provide an overview of the technol¬ 
ogy of medical robot systems and discuss examples 
of our basic system paradigms, with brief addi¬ 
tional discussion topics of remote telesurgery and 
robotic surgical simulators. We conclude with some 
thoughts on future research directions and provide 
suggested further reading. 
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Robots at Work 


63.1 Core Concepts 

A fundamental property of robotic systems is their abil¬ 
ity to couple complex information to physical action 
in order to perform a useful task. This ability to re¬ 
place, supplement, or transcend human performance 
has had a profound influence on many fields of our soci¬ 
ety, including industrial production, exploration, quality 
control, and laboratory processes. Although robots have 
often been first introduced to automate or improve dis¬ 
crete processes such as welding or test probe placement 
or to provide access to environments where humans 
cannot safely go, their greater long-term impact has of¬ 
ten come indirectly as essential enablers of computer 
integration of entire production or service processes. 

63.1.1 Medical Robotics, 
Computer-Integrated Surgery, 
and Closed-Loop Interventions 

Medical robots have a similar potential to fundamen¬ 
tally change surgery and interventional medicine as part 
of a broader, information-intensive environment that 
exploits the complementary strengths of humans and 
computer-based technology. The robots may be thought 
of as information-driven surgical tools that enable hu¬ 
man surgeons to treat individual patients with greater 
safety, improved efficacy, and reduced morbidity than 
would otherwise be possible. Further, the consistency 
and information infrastructure associated with medi¬ 
cal robotic and computer-assisted surgery systems have 
the potential to make computer-integrated surgery as 
important to health care as computer-integrated man¬ 
ufacturing is to industrial production. 

Figure 63.1 illustrates this view of computer- 
integrated surgery (CIS). The process starts with infor¬ 


mation about the patient, which can include medical 
images (computed tomography (CT), magnetic reso¬ 
nance imaging (MRI), positron emission tomography 
(PET), etc.), lab test results, and other information. 
This patient-specific information is combined with sta¬ 
tistical information about human anatomy, physiology, 
and disease to produce a comprehensive computer rep¬ 
resentation of the patient, which can then be used to 
produce an optimized interventional plan. In the operat¬ 
ing room, the preoperative patient model and plan must 
be registered to the actual patient. Typically, this is done 
by identifying corresponding landmarks or structures 
on the preoperative model and the patient, either by 
means of additional imaging (x-ray, ultrasound, video), 
by the use of a tracked pointing device, or by the 
robot itself. If the patient’s anatomy has changed, then 
the model and plan are updated appropriately, and the 
planned procedure is carried out with assistance of the 
robot. As the intervention continues, additional imag¬ 
ing or other sensing is used to monitor the progress 
of the procedure, to update the patient model, and to 
verify that the planned procedure has been success¬ 
fully executed. After the procedure is complete, further 
imaging, modeling, and computer-assisted assessment 
is performed for patient follow-up and to plan subse¬ 
quent interventions, if any should be required. Further, 
all the patient-specific data generated during the plan¬ 
ning, execution, and follow-up phases can be retained. 
These data can subsequently be analyzed statistically to 
improve the rules and methods used to plan future pro¬ 
cedures. 

63.1.2 Factors Affecting the Acceptance 
of Medical Robots 



Fig. 63.1 Fundamental information flow in computer-integrated 
surgery 


Medical robotics is ultimately an application-driven 
research field. Although the development of medical 
robotic systems requires significant innovation and can 
lead to very real, fundamental advances in technology, 
medical robots must provide measurable and signifi¬ 
cant advantages if they are to be widely accepted and 
deployed. The situation is complicated by the fact that 
these advantages are often difficult to measure, can take 
an extended period to assess, and may be of varying 
importance to different groups. Table 63.1 lists some 
of the more important factors that researchers contem¬ 
plating the development of a new medical robot system 
should consider in assessing their proposed approach. 

Broadly, the advantages offered by medical robots 
may be grouped into three areas. The first is the po¬ 
tential of a medical robot to significantly improve sur¬ 
geons’ technical capability to perform procedures by 




































Medical Robotics and Computer-Integrated Surgery I 63.1 Core Concepts 1659 


Table 63.1 Assessment factors for medical robots or computer-integrated surgery systems (after [63.1]) 


Assessment 

factor 

Important 
to whom 

Assessment 

method 

Summary of key leverage 

New treatment 

Clinical 

Clinical and trials 

Transcend human sensory-motor limits (e.g., in 

options 

researchers, pa¬ 
tients 

preclinical 

microsurgery). Enable less invasive procedures 
with real-time image feedback (e.g., fluoro¬ 
scopic or MRI-guided liver or prostate therapy). 
Speed up clinical research through greater con¬ 
sistency and data gathering 

Quality 

Surgeons, patients 

Clinician judg¬ 
ment; revision 
rates 

Significantly improve the quality of surgical 
technique (e.g., in microvascular anastomosis), 
thus improving results and reducing the need 
for revision surgery 

Time and cost 

Surgeons, hospi¬ 
tals, insurers 

Hours, hospital 
charges 

Speed operating room (OR) time for some in¬ 
terventions. Reduce costs from healing time and 
revision surgery. Provide effective intervention 
to treat patient condition 

Less invasiveness 

Surgeons, patients 

Qualitative judg¬ 
ment; recovery 
times 

Provide crucial information and feedback 
needed to reduce the invasiveness of surgical 
procedures, thus reducing infection risk, recov¬ 
ery times, and costs (e.g., percutaneous spine 
surgery) 

Safety 

Surgeons, patients 

Complication and 
revision surgery 
rates 

Reduce surgical complications and errors, 
again lowering costs, improving outcomes and 
shortening hospital stays (e.g., robotic total hip 
replacement (THR), steady-hand brain surgery) 

Real-time feed¬ 
back 

Surgeons 

Qualitative assess¬ 
ment, quantitative 
comparison of 
plan to obser¬ 
vation, revision 
surgery rates 

Integrate preoperative models and intraopera¬ 
tive images to give surgeon timely and accurate 
information about the patient and intervention 
(e.g., fluoroscopic x-rays without surgeon expo¬ 
sure, percutaneous therapy in conventional MRI 
scanners). Assure that the planned intervention 
has in fact been accomplished 

Accuracy or pre¬ 
cision 

Surgeons 

Quantitative com¬ 
parison of plan to 
actual 

Significantly improve the accuracy of therapy 
dose pattern delivery and tissue manipulation 
tasks (e.g., solid organ therapy, microsurgery, 
robotic bone machining) 

Enhanced doc¬ 

Surgeons, clinical 

Databases, 

Exploit CIS systems’ ability to log more varied 

umentation and 
follow-up 

researchers 

anatomical at¬ 
lases, images, and 
clinical observa¬ 
tions 

and detailed information about each surgical 
case than is practical in conventional manual 
surgery. Over time, this ability, coupled with 

CIS systems’ consistency, has the potential 
to significantly improve surgical practice and 


shorten research trials 


exploiting the complementary strengths of humans and 
robots summarized in Table 63.2. Medical robots can 
be constructed to be more precise and geometrically 
accurate than an unaided human. They can operate in 
hostile radiological environments and can provide great 
dexterity for minimally invasive procedures inside the 


patient’s body. These capabilities can both enhance the 
ability of an average surgeon to perform procedures that 
only a few exceptionally gifted surgeons can perform 
unassisted and can also make it possible to perform 
interventions that would otherwise be completely infea¬ 
sible. 
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Table 63.2 Complementary strengths of human surgeons and robots (after [63.1] ) 



Strengths 

Limitations 

Humans 

Excellent judgment 

Excellent hand-eye coordination 

Excellent dexterity (at natural human scale) 

Able to integrate and act on multiple information 

sources 

Easily trained 

Versatile and able to improvise 

Prone to fatigue and inattention 

Limited fine motion control due to tremor 

Limited manipulation ability and dexterity outside natural 
scale 

Cannot see through tissue 

Bulky end-effectors (hands) 

Limited geometric accuracy 

Hard to keep sterile 

Affected by radiation, infection 

Robots 

Excellent geometric accuracy 

Untiring and stable 

Immune to ionizing radiation 

Can be designed to operate at many different scales of 
motion and payload 

Able to integrate multiple sources of numerical and 
sensor data 

Poor judgment 

Hard to adapt to new situations 

Limited dexterity 

Limited hand-eye coordination 

Limited haptic sensing (today) 

Limited ability to integrate and interpret complex information 


A second, closely related capability is the poten¬ 
tial of medical robots to promote surgical safety both 
by improving a surgeon’s technical performance and by 
means of active assists such as no-fly zones or virtual 
fixtures (Sect. 63.2.3) to prevent surgical instruments 
from causing unintentional damage to delicate struc¬ 
tures. Furthermore, the integration of medical robots 
within the information infrastructure of a larger CIS 
system can provide the surgeon with significantly im¬ 
proved monitoring and online decision supports, thus 
further improving safety. 

A third advantage is the inherent ability of med¬ 
ical robots and CIS systems to promote consistency 
while capturing detailed online information for ev¬ 
ery procedure. Consistent execution (e.g., in spacing 
and tensioning of sutures or in placing of compo¬ 
nents in joint reconstructions) is itself an important 
quality factor. If saved and routinely analyzed, the 
flight data recorder information inherently available 
with a medical robot can be used both in morbid¬ 
ity and mortality assessments of serious surgical in¬ 
cidents and, potentially, in statistical analyses exam¬ 
ining many cases to develop better surgical plans. 
Furthermore, such data can provide valuable input 
for surgical simulators, as well as a database for de¬ 
veloping skill assessment and certification tools for 
surgeons. 

63.1.3 Medical Robotics System 

Paradigms: Surgical CAD/CAM 
and Surgical Assistance 

We call the process of computer-assisted planning, 
registration, execution, monitoring, and assessment sur¬ 
gical CAD/CAM, emphasizing the analogy to manu¬ 
facturing CAD/CAM. Just as in manufacturing, robots 


can be critical in this CAD/CAM process by enhanc¬ 
ing the surgeon’s ability to execute surgical plans. The 
specific role played by the robot depends somewhat 
on the application, but current systems tend to exploit 
the geometric accuracy of the robot and/or its ability 
to function concurrently with x-ray or other imaging 
devices. Typical examples include radiation therapy 
delivery robots such as Accuray’s CyberKnife [63.2] 
(Accuray, Inc., Sunnyvale, CA), shaping of bone in 
orthopaedic joint reconstructions (discussed further in 
Sect. 63.3.2) and image-guided placement of therapy 
needles (Sect. 63.3.3). 

Surgery is often highly interactive; many decisions 
are made by the surgeon in the operating room and ex¬ 
ecuted immediately, usually with direct visual or haptic 
feedback. Generally, the goal of surgical robotics is not 
to replace the surgeon so much as to improve his or her 
ability to treat the patient. The robot is thus a computer- 
controlled surgical tool in which control of the robot is 
often shared in one way or another between the human 
surgeon and a computer. We thus often speak of medical 
robots as surgical assistants. 

Broadly, robotic surgical assistants may be broken 
into two subcategories. The first category, surgeon 
extender robots , manipulate surgical instruments under 
the direct control of the surgeon, usually through a tele¬ 
operation or hands-on cooperative control interface. 
The primary value of these systems is that they can 
overcome some of the perception and manipulation 
limitations of the surgeon. Examples include the ability 
to manipulate surgical instruments with superhuman 
precision by eliminating hand tremor, the ability to per¬ 
form highly dexterous tasks inside the patient’s body, or 
the ability to perform surgery on a patient who is phys¬ 
ically remote from the surgeon. Although setup time 
is still a serious concern with most surgeon extender 
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Fig. 63.2 The da Vinci telesurgical 
robot (after [63.3]) extends a sur¬ 
geon’s capabilities by providing the 
immediacy and dexterity of open 
surgery in a minimally invasive surgi¬ 
cal environment (photos courtesy of 
Intuitive Surgical, Sunnyvale) 


systems, the greater ease of manipulation that such sys¬ 
tems offer has the potential to reduce operative times. 
One widely deployed example of a surgeon extender is 
the da Vinci system [63.3] (Intuitive Surgical Systems, 
Sunnyvale, CA) shown in Fig. 63.2. Other examples 
(among many) incude the Sensei catheter system [63.7] 
(Hansen Medical Systems, Mountain View, CA.). the 


Johns Hopkins University (JHU) Steady Hand micro¬ 
surgery robot [63.4-6] shown in Fig. 63.3 and discussed 
in Sect. 63.3, the Rio orthopaedic robot [63.8] (Mako 
Surgical Systems, Ft. Lauderdale, Florida), the DLR 
Miro system [63.9], the Surgica Robotica’s Sergenius 
system (Surgica Robotica, Udine), and Titan Medical’s 
Amadeus System (Titan Medical, Toronto, Canada). 



b) Stereo display Video microscope 


Force and OCT sensing tools Robot 


C v (/handle Ccale./tool J 


-^cmd 

Robot 

interface 


Fig.63.3a-c The Johns Hopkins Steady Hand microsurgical robot (after [63.4-6]) extends a surgeon’s capabilities by 
providing the ability to manipulate surgical instruments with very high precision while still exploiting the surgeon’s 
natural hand-eye coordination, (a) The basic paradigm of hands-on compliant guiding. The commanded velocity of the 
robot is proportional to a scaled difference between the forces exerted by the surgeon on the tool handle and (optionally) 
sensed tool-to-tissue forces. (b) Current laboratory setup, showing the robot, stereo video microscope, stereo display 
with information overlays, display console for optical coherence tomography (OCT) system, and a sensorized tool, 
(c) An earlier recent version of the Steady Hand robot currently being used for experiments in microcannulation of 
100 pm blood vessels 
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A second category, auxiliary surgical support 
robots, generally work alongside the surgeon and per¬ 
form such routine tasks as tissue retraction, limb posi¬ 
tioning, or endoscope holding. One primary advantage 
of such systems is their potential to reduce the num¬ 
ber of people required in the operating room, although 
that advantage can only be achieved if all the tasks 
routinely performed by an assisting individual can be 
automated. Other advantages can include improved task 
performance (e.g., a steadier endoscopic view), safety 
(e.g., elimination of excessive retraction forces), or sim¬ 
ply giving the surgeon a greater feeling of control over 
the procedure. One of the key challenges in these sys¬ 


tems is providing the required assistance without posing 
an undue burden on the surgeon’s attention. A variety 
of control interfaces are common, including joysticks, 
head tracking, voice recognition systems, and visual 
tracking of the surgeon and surgical instruments, for 
example, the Aesop endoscope positioner [63.10] used 
both a foot-actuated joystick and a very effective voice 
recognition system. Again, further examples are dis¬ 
cussed in Sect. 63.3. 

It is important to realize that surgical CAD/CAM 
and surgical assistance are complementary concepts. 
They are not at all incompatible, and many systems 
have aspects of both. 


63.2 Technology 


The mechanical design of a surgical robot depends cru¬ 
cially on its intended application. For example, robots 
with high precision, stiffness and (possibly) limited 
dexterity are often very suitable for orthopaedic bone 
shaping or stereotactic needle placement, and medical 
robots for these applications [63.17-20] frequently have 
high gear ratios and consequently, low back-drivability, 
high stiffness, and low speed. On the other hand, 
robots for complex, minimally invasive surgery (MIS) 



Fig.63.4a-f Dexterity enhancement inside a patient’s body: 

(a) The Intuitive da Vinci Si system and wrist with a typical surgi¬ 
cal instrument (Photos courtesy of Intuitive Surgical) (after [63.3]); 

(b) The end-effectors of the JHU/Columbia snake telesurgical sys¬ 
tem (after [63.11]); (c) Two-handed manipulation system for use 
in endogastric surgery (after [63.12]); (d) five-degree-of-freedom 
3 mm wrist and gripper (after [63.13]) for microsurgery in deep 
and narrow spaces; (e) concentric tube robot (after [63.14,15]); 
(f) Columbia/Vanderbilt high dexterity system for single port ac¬ 
cess surgery (after [63.16]) 


on soft tissues require compactness, dexterity, and re¬ 
sponsiveness. These systems [63.3,21] frequently have 
relatively high speed, low stiffness, and highly back- 
drivable mechanisms. 

63.2.1 Mechanical Design Considerations 

Many early medical robots [63.17,20,22] were essen¬ 
tially modified industrial robots. This approach has 
many advantages, including low cost, high reliability, 
and shortened development times. If suitable modifi¬ 
cations are made to ensure safety and sterility, such 
systems can be very successful clinically [63.18], and 
they can also be invaluable for rapid prototyping and 
research use. 

However, the specialized requirements of surgical 
applications have tended to encourage more special¬ 
ized designs. For example, laparoscopic surgery and 
percutaneous needle placement procedures typically 
involve the passage or manipulation of instruments 
about a common entry point into the patient’s body. 
There are three basic design approaches. The first ap¬ 
proach uses a passive wrist to allow the instrument 
to pivot about the insertion point and has been used 
in the commercial Aesop and Zeus robots [63.21, 
23] as well as several research systems. The second 
approach mechanically constrains the motion of the 
surgical tool to rotate about a remote center of mo¬ 
tion (RCM) distal to the robot’s structure. In surgery, 
the robot is positioned so that the RCM point co¬ 
incides with the entry point into the patient’s body. 
This approach has been used by the commercially 
developed da Vinci robot [63.3], as well as by nu¬ 
merous research groups, using a variety of kinematic 
designs [63.24-26]. Finally, a third approach uses an 
active external wrist [63.9, 17] and thus supports robot- 
assisted interventions that do not require a pivot point, 
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potentially extending robotic surgery to other field of 
surgery. 

The emergence of minimally invasive surgery has 
created a need for robotic systems that can provide 
high degrees of dexterity in very constrained spaces 
inside the patient’s body, and at smaller and smaller 
scales. Figure 63.4 shows several typical examples of 
current approaches. One common response has been to 
develop cable-actuated wrists [63.3]. However, a num¬ 
ber of investigators have investigated other approaches, 
including bending structural elements [63.11], shape- 
memory alloy actuators [63.27,28], microhydraulic 
systems [63.29], and electroactive polymers [63.30], 
Similarly, the problem of providing access to surgical 
sites inside the body has led several groups to develop 
semiautonomously moving robots for epicardial [63.31] 
or endoluminal applications [63.32, 33]. 

Two growing trends MIS are natural orifice trans¬ 
luminal surgery (NOTES) [63.34,35] and single port 
laparoscopy (SPL) [63.36]: the idea is to get access to 
the abdominal cavity by using natural orifices and in¬ 
ternal incisions (in NOTES) or existing human scars 
(e.g., the navel, in SPL). From the mechanical design 
viewpoint, there is the need to develop deployable sur¬ 
gical instruments or accessorised endoscopes and to 
combine flexibility (to reach the target) and stability 
of the platform (to achieve precision). Clashing of in¬ 
struments and difficulty in triangulation are the main 
limitations which companies and research groups try to 
approach [63.37,38]. 

The problem of distal operation, already present 
in MIS, is becoming more dramatic in NOTES and 
SPL and several solutions for helping surgical tasks re¬ 
quiring triangulation have been developed by different 
research groups [63.39]. They are based on magnetic 
fields which can generate an internal force without con¬ 
straining the internal tool to the access port [63.40-42], 

Another significant development in recent years 
has been the emergence and widespread deployment 
of three-dimensional (3-D) printing and other rapid 
prototyping technologies for clinically usable medical 
devices and medical robot components, as well as for 
construction of realistic patient-specific models [63.43, 
44]. This trend has promoted very rapid progress in 
medical robot design and will be increasingly important 
in coming years. 

Although most surgical robots are mounted to the 
surgical table, to the operating room ceiling, or to the 
floor, there has been growing interest in developing 
systems that directly attach to the patient [63.45,46], 
and clinically deployed examples exist [63.47]. The 
main advantage of this approach is that the relative 
position of the robot and patient is unaffected if the 
patient moves. The challenges are that the robot must 


be smaller and that relatively nonintrusive means for 
mounting it must be developed. 

Finally, robotic systems intended for use in spe¬ 
cific imaging environments pose additional design chal¬ 
lenges. First, there is the geometric constraint that the 
robot (or at least its end-effector) must fit within the 
scanner along with the patient. Second, the robot’s me¬ 
chanical structure and actuators must not interfere with 
the image formation process. In the case of x-ray and 
CT, satisfying these constraints is relatively straight¬ 
forward. The constraints for MRI are more challeng¬ 
ing [63.48]. 

63.2.2 Control Paradigms 

Surgical robots assist surgeons in treating patients by 
moving surgical instruments, sensors, or other devices 
in relation to the patient. Generally, these motions are 
controlled by the surgeon in one of three ways: 

• Preprogrammed, semi-autonomous motion: The de¬ 
sired behavior of the robot’s tools is specified inter¬ 
actively by the surgeon, usually based on medical 
images. The computer fills in the details and ob¬ 
tains the surgeon’s concurrence before the robot 
is moved. Examples include the selection of nee¬ 
dle target and insertion points for percutaneous 
therapy and tool cutter paths for orthopaedic bone 
machining. 

• Teleoperator control: The surgeon specifies the de¬ 
sired motions directly through a separate human 
interface device and the robot moves immedi¬ 
ately. Examples include common telesurgery sys¬ 
tems such as the da Vinci [63.3]. Although physical 
master manipulators are the most common input de¬ 
vices, other human interfaces are also used, notably 
voice control [63.21]. 

• Hands-on compliant control: The surgeon grasps 
the surgical tool held by the robot or a control 
handle on the robot’s end-effector. A force sensor 
senses the direction that the surgeon wishes to move 
the tool and the computer moves the robot to com¬ 
ply. Early experiences with Robodoc [63.17] and 
other surgical robots [63.25] showed that surgeons 
found this form of control to be very convenient and 
natural for surgical tasks. Subsequently, a number of 
groups have exploited this idea for precise surgical 
tasks, notably the JHU Steady Hand microsurgical 
robot [63.4] shown in Fig. 63.3, the Rio orthopaedic 
robot [63.8] (Mako Surgical Systems, Ft. Laud¬ 
erdale, Florida) and the Imperial College Acrobot 
orthopaedic system [63.49] shown in Fig. 63.5c,d. 

These control modes are not mutually exclusive and 
are frequently mixed. For example, the Robodoc sys- 
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Fig.63.5a-d Clinically deployed 
robots for orthopaedic surgery. 

(a,b) The Robodoc system (af¬ 
ter [63.17,18]) represents the first 
clinically applied robot for joint 
reconstruction surgery and has been 
used for both primary and revision 
hip replacement surgery as well as 
knee replacement surgery. (c,d) The 
Acrobot system of Davies et al. (af¬ 
ter [63.49]) uses hands-on compliant 
guiding together with a form of virtual 
fixtures to prepare the femur and tibia 
for knee replacement surgery 


tem [63.17,18] uses hands-on control to position the 
robot close to the patient’s femur or knee and pre¬ 
programmed motions for bone machining. Similarly, 
the IBM/JHU LARS robot. [63.25] used both cooper¬ 
ative and telerobotic control modes. The cooperatively 
controlled Acrobot [63.49] uses preprogrammed virtual 
fixtures (Sect. 63.1.3) derived from the implant shape 
and its planned position relative to medical images. 

Each mode has advantages and limitations, de¬ 
pending on the task. Preprogrammed motions permit 
complex paths to be generated from relatively simple 
specifications of the specific task to be performed. They 
are most often encountered in surgical CAD/CAM ap¬ 
plications where the planning uses two- (2-D) or three- 
dimensional (3-D) medical images. However, they can 
also provide useful complex motions combining sen¬ 
sory feedback in teleoperated or hands-on systems. 
Examples might include passing a suture or inserting 
a needle into a vessel after the surgeon has preposi¬ 
tioned the tip. On the other hand, interactive specifica¬ 
tion of motions based on real-time visual appreciation 
of deforming anatomy would be very difficult. 

Teleoperated control provides the greatest versatil¬ 
ity for interactive surgery applications, such as dexter¬ 
ous MIS [63.3,21,26,50] or remote surgery [63.51, 
52]. It permits motions to be scaled, and (in some 
research systems) facilitates haptic feedback between 
master and slave systems. The main drawbacks are 
complexity, cost, and disruption to standard operating 
room work flow associated with having separate master 
and slave robots. 

Hands-on control combines the precision, strength, 
and tremor-free motion of robotic devices with some 
of the immediacy of freehand surgical manipulation. 
These systems tend to be less expensive than telesur- 


gical systems, since there is less hardware, and they 
can be easier to introduce into existing surgical settings. 
They exploit a surgeon’s natural eye-hand coordination 
in an intuitively appealing way, and they can be adapted 
to provide force scaling [63.4,5]. Although direct mo¬ 
tion scaling is not possible, the fact that the tool moves 
in the direction that the surgeon pulls it makes this 
limitation relatively unimportant when working with 
a surgical microscope. The biggest drawbacks are that 
hands-on control is inherently incompatible with any 
degree of remoteness between the surgeon and the sur¬ 
gical tool and that it is not practical to provide hands-on 
control of instruments with distal dexterity. 

Teleoperation and hands-on control are both com¬ 
patible with shared control modes in which the robot 
controller constrains or augments the motions specified 
by the surgeon, as discussed in Sect. 63.2.3. 

63.2.3 Virtual Fixtures 

and Human-Machine Cooperative 
Systems 

Although one goal of both teleoperation and hands-on 
control is often transparency, i. e., the ability to move an 
instrument with the freedom and dexterity he/she might 
expect with a handheld tool, the fact that a computer 
is actually controlling the robot’s motion creates many 
more possibilities. The simplest is a safety barrier or no- 
fly zone, in which the robot’s tool is constrained from 
entering certain portions of its workspace. More so¬ 
phisticated versions include virtual springs, dampers, or 
complex kinematic constraints that help a surgeon align 
a tool, maintain a desired force, or maintain a desired 
anatomical relationship. The Acrobot system shown in 
Fig. 63.5c,d represents a successful clinical applica- 
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tion of the concept, which has many names, of which 
virtual fixtures seems to be the most popular [63.53, 
54]. A number of groups are exploring extensions of 
the concept to active cooperative control, in which the 
surgeon and robot share or trade off control of the 
robot during a surgical task or subtask. As the ability 
of computers to model and follow along surgical tasks 
improves, these modes will become more and more im¬ 
portant in surgical assistant applications. Figure 63.6 
illustrates the overall concept of human-machine co¬ 
operative systems in surgery, and Fig. 63.7 illustrates 
the use of registered anatomical models to generate 
constraint-based virtual fixtures. These approaches are 
equally valid whether the surgeon interacts with the sys¬ 
tem through classical teleoperation or through hands-on 
compliant control. See also Chap. 43. 

Both teleoperation and hands-on control are like¬ 
wise used in human-machine cooperative systems for 
rehabilitation and disability assistance systems. Con¬ 
strained hands-on systems offer special importance for 
rehabilitation applications and for helping people with 
movement disorders. Similarly, teleoperation and intel¬ 
ligent task following and control are likely to be vital 
for further advances in assistive systems for people with 
severe physical disabilities. See Chap. 64 for a further 
discussion of human-machine cooperation in assistive 
systems. 

63.2.4 Safety and Sterility 

Medical robots are safety-critical systems, and safety 
should be considered from the very beginning of the 
design process [63.55,56]. Although there is some 
difference in detail, government regulatory bodies re¬ 
quire a careful and rigorous development process with 
extensive documentation at all stages of design, im¬ 
plementation, testing, manufacturing, and field support. 
Generally, systems should have extensive redundancy 
built into hardware and control software, with multiple 
consistency conditions constantly enforced. The basic 
consideration is that no single point of failure should 
cause the robot to go out of control or to injure a pa¬ 
tient. Although there is some difference of opinion as to 
the best way to make trade-offs, medical manipulators 
are usually equipped with redundant position encoders 
and ways to mechanically limit the speed and/or force 
that the robot can exert. If a consistency check fail¬ 
ure is detected, two common approaches are to freeze 
robot motion or to cause the manipulator to go limp. 
Which is better depends strongly on the particular ap¬ 
plication. 

Sterilizability and biocompatibility are also crucial 
considerations. Again, the details are application depen¬ 
dent. Common sterilization methods include gamma 
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Fig. 63.6 Human-machine cooperative systems (HMCS) in 



surgery 



Fig. 63.7 Human-machine cooperative manipulation using con¬ 
straint-based virtual fixtures, in which patient-specific constraints 
are derived from registered anatomical models (after [63.53]) 


rays (for disposable tools), autoclaving, soaking or gas 
sterilization, and the use of sterile drapes to cover un- 
sterile components. Soaking or gas sterilization are less 
likely to damage robot components, but very rigor¬ 
ous cleaning is required to prevent extraneous foreign 
matter from shielding microbes from the sterilizing 
agent. 

Careful attention to higher levels of application 
protocols is also essential. Just like any other tool, sur¬ 
gical robots must be used correctly by surgeons, and 
careful training is essential for safe practice. Surgeons 
must understand both the capabilities and limitations 
of the robot and of the surgical process as well, since 
safety is a systemic property. This adds new require¬ 
ments to training programs, which must include robotic 
capabilities and nontechnical skills (Sect. 63.3.8). In 
surgical CAD/CAM applications, the surgeon must un¬ 
derstand how the robot will execute the plan and be 
able to verify that the plan is being followed. If the 
surgeon is interactively commanding the robot, it is 
essential that the robot interpret these commands cor- 
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rectly. Similarly, it is essential that the robot’s model 
of its task environment correspond correctly to the ac¬ 
tual environment. The availability of task models is 
necessary to the development of autonomous execu¬ 
tion of robotic gestures as well as the formal ver¬ 
ification of task correctness [63.57], Although care¬ 
ful design and implementation can practically elim¬ 
inate the likelihood of a runaway condition by the 
manipulator, this will do little good if the robot is 
badly registered to the patient images used to con¬ 
trol the procedure. If the robot fails for any reason, 
there must be well-documented and planned procedures 
for recovery (and possibly continuing the procedure 
manually). 

Finally, it is important to remember that a well- 
designed robot system can actually enhance patient 
safety. The robot is not subject to fatigue or momen¬ 
tary lapses of attention. Its motions can be more precise 
and there is less chance that a slip of the scalpel may 
damage some delicate structure. In fact, the system can 
be programmed to provide virtual fixtures (Sect. 63.2.3) 
preventing a tool from entering a forbidden region un¬ 
less the surgeon explicitly overrides the system. 

63.2.5 Imaging and Modelling of Patients 

As the capabilities of medical robots continue to evolve, 
the use of computer systems to model dynamically 
changing patient-specific anatomy will become increas¬ 
ingly important. There is a robust and diverse research 
community addressing a very broad range of research 
topics, including the creation of patient-specific mod¬ 
els from medical images, techniques for updating these 
models based upon real-time image and other sensor 
data, and the use of these models for planning and mon¬ 
itoring of surgical procedures. Some of the pertinent 
research topics include the following: 

• Medical image segmentation and image fusion 
to construct and update patient-specific anatomic 
models 

• Biomechanical property measurement and mod¬ 
elling for analyzing and predicting tissue defor¬ 
mations and functional factors affecting surgical 
planning, control, and rehabilitation 

• Optimization methods for treatment planning and 
interactive control of systems 

• Methods for registering the virtual reality of images 
and computational models to the physical reality of 
an actual patient 

• Methods for characterizing treatment plans and in¬ 
dividual task steps such as suturing, needle inser¬ 
tion, or limb manipulation for purposes of planning, 
monitoring, control, and intelligent assistance 


• Real-time data fusion for such purposes as updating 
models from intraoperative images 

• Methods for human-machine communication, in¬ 
cluding real-time visualization of data models, nat¬ 
ural language understanding, gesture recognition, 
etc. 

• Methods for characterizing uncertainties in data, 
models, and systems and for using this information 
in developing robust planning and control methods. 

An in-depth examination of this research is beyond 
the scope of this article. A more complete discussion 
of these topics may be found in the suggested further 
reading in Sect. 63.4. 

63.2.6 Registration 

Geometric relationships are fundamental in medical 
robotics, especially in surgical CAD/CAM. There is 
an extensive literature on techniques for coregistering 
coordinate systems associated with robots, sensors, im¬ 
ages, and the patient [63.58, 59]. Following [63.59], we 
briefly summarize the main concepts here. Suppose that 
we have coordinates 

v' A = ( x A ,y A ,ZA ) 
v r B = (xb^b.Zb) , 

corresponding to comparable locations in two coor¬ 
dinate systems RefA and Reffi. Then the process of 
registration is simply that of finding a function Tab (■ ■ ■) 
such that 

v B = T ab (v a ). 

Generally, TabO”) is assumed to be a rigid transfor¬ 
mation of the form 

Tab(v ' a ) = R A bv' a +p' AB , 

where R AB represents a rotation and /?ab represents 
a translation, but nonrigid transformations are becom¬ 
ing increasingly common. There are hundreds of meth¬ 
ods for computing T AB (■■■). The most common for 
medical robotics involve finding a set of corresponding 
geometric features r A and r B whose coordinates can be 
determined in both coordinate systems and then finding 
a transformation that minimizes some distance function 
d AB = distance^, T’ab(^a)]- Typical features can in¬ 
clude artificial fiducial objects (pins, implanted spheres, 
rods, etc.) or anatomical features such as point land¬ 
marks, ridge curves, or surfaces. 

One common class of methods is based on the iter¬ 
ated closest-point algorithm of Besl and McKay [63.60], 
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for example, 3-D robot coordinates a, may be found 
for a collection of points known to be on the surface 
of an anatomical structure that can also be found in 
a segmented 3-D image. Given an estimate 7* of the 
transformation between image and robot coordinates, 
the method iteratively finds corresponding points bj on 
the surface that are closest to T^aj and then finds a new 


transformation 

T k + 1 = arg mm - 7a,) 2 . 

j 

The process is repeated until some suitable termination 
condition is reached. 


63.3 Systems, Research Areas, and Applications 


Medical robots are not ends in themselves. As the late 
Hap Paul often remarked, the robot is a surgical tool 
designed to improve the efficacy of a procedure. (Paul 
was the founder of Integrated Surgical Systems. Along 
with William Bargar, he was one of the first people to 
recognize the potential of robots to fundamentally im¬ 
prove the precision of orthopaedic surgery.) 

63.3.1 Nonrobotic Computer-Assisted 
Surgery: Navigation 
and Image Overlay Devices 

In cases where the role of the robot is placing in¬ 
struments on targets determined from medical images, 
surgical navigation is often a superior alternative. In 
surgical navigation [63.61], the positions of instru¬ 
ments relative to the reference markers on the patient 
are tracked using specialized electromechanical, op¬ 
tical, electromagnetic, or sonic digitizers or by more 
general computer vision techniques. After the relation¬ 
ships between key coordinate systems (patient anatomy, 
images, surgical tools, etc.) are determined through 
a registration process (Sect. 63.2.6), a computer work¬ 
station provides graphical feedback to the surgeon to 
assist in performing the planned task, usually by dis¬ 
playing instrument positions relative to medical images, 
as shown in Fig. 63.8a. Although the registration is usu¬ 
ally performed computationally, a simple mechanical 
alignment of an image display with an imaging device 
can be surprisingly effective in some cases. One exam¬ 
ple [63.62] is shown in Fig. 63.8b. 

The main advantages of surgical navigation sys¬ 
tems are their versatility, their relative simplicity, and 
their ability to exploit the surgeon’s natural dexterity 
and haptic sensitivity. They are readily combined with 
passive fixtures and manipulation aids [63.65,66]. The 
main drawbacks, compared to active robots, are those 
associated with human limitations in accuracy, strength, 
ability to work in certain imaging environments, and 
dexterity inside the patient’s body (Table 63.2). 

Because these advantages often outweigh the lim¬ 
itations, surgical navigation systems are achieving 


widespread and increasing acceptance in such fields as 
neurosurgery, otolaryngology, and orthopaedics. Since 
much of the technology of these systems is compatible 
with surgical robots and since technical problems such 
as registration are common among all these systems, we 
may expect to see a growing number of hybrid applica¬ 
tions combining medical robots and navigation. 

63.3.2 Orthopaedic Systems 

Orthopaedic surgery represents a natural surgical 
CAD/CAM application, and both surgical navigation 
systems and medical robots have been applied to or¬ 
thopaedics. Bone is rigid and is easily imaged in CT 
and intraoperative x-rays, and surgeons are accustomed 
to doing at least some preplanning based on these im¬ 
ages. Geometric accuracy in executing surgical plans 
is very important, for example, bones must be shaped 
accurately to ensure proper fit and positioning of com¬ 
ponents in joint replacement surgery. Similarly, os¬ 
teotomies require both accurate cutting and placement 
of bone fragments. Spine surgery often requires screws 
and other hardware to be placed into vertebrae with¬ 
out damage to the spinal cord, nerves, and nearby blood 
vessels. 

The Robodoc system shown in Fig. 63.5a,b repre¬ 
sents the first clinically applied robot for joint recon¬ 
struction surgery [63.17,18]. Since 1992, it has been 
applied successfully to both primary and revision hip 
replacement surgery, as well as knee surgery. Since this 
system exhibits many of the characteristics of surgi¬ 
cal CAD/CAM, we will discuss it is some detail. In 
the surgical CAD phase, the surgeon selects the de¬ 
sired based on preoperative CT images and interactively 
specifies the desired position of the implant compo¬ 
nents. In the surgical CAM phase, surgery proceeds 
normally up to the point where the patient’s bones are 
to be prepared to receive the implant. The robot is 
moved up to the operating table, the patient’s bones 
are attached rigidly to the robot’s base, and the robot is 
registered to the CT images either by use of implanted 
fiducial pins or by use of a 3-D digitizer to match bone 
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Fig.63.8a-d Information en¬ 
hancement for surgical assistance. 

(a) Display from a typical surgical 
navigation system, here the Medtronic 
StealthStation; (b) the JHU image 
overlay system (after [63.62]) uses 
a mirror to align the virtual image 
of a cross-sectional image with the 
corresponding physical position 
in the patient’s body; (c) Sensory 
substitution display of surgical force 
information onto da Vinci surgical 
robot video monitor (after [63.63]); 
(d) Overlay of laparoscopic ultrasound 
onto the da Vinci surgical robot video 
monitor (after [63.64]) 


surfaces to the CT images. After registration, the sur¬ 
geon’s hand guides the robot to an approximate initial 
starting position. Then, the robot autonomously ma¬ 
chines the desired shape with a high-speed rotary cutter 
while the surgeon monitors progress. During cutting, 
the robot monitors cutting forces, bone motion, and 
other safety sensor, and either the robot controller or 
the surgeon can pause execution at any time. If the 
procedure is paused for any reason, there are a num¬ 
ber of error recovery procedures available to permit the 
procedure to be resumed or restarted at one of sev¬ 
eral defined checkpoints. Once the desired shape has 
been machined, surgery proceeds manually in the nor¬ 
mal manner. 

Subsequently, several other robotic systems for 
joint replacement surgery have been introduced or pro¬ 
posed. The references in Sect. 63.4 provide numerous 
examples. Notable hands-on guided systems include 
Rio surgical robot [63.8] (Mako Surgical, Ft. Laud¬ 
erdale, Florida) and the Acrobot [63.49] system for 
knee surgery shown in Figs. 63.5c,d. Similarly, several 
groups have recently proposed small orthopaedic robots 
attaching directly to the patient’s bones [63.45] or com¬ 
pletely freehand systems such as the NavioPFS surgical 
system (Blue Belt Technlogies, Pittsburgh, Pa.) which 
combine surgical navigation with very fast on-off con¬ 
trol of a surgical cutter [63.67, 68]. A recent example of 
a hybrid passive-active robot is [63.69]. 


63.3.3 Percutaneous Needle 
Placement Systems 

Needle placement procedures have become ubiqui¬ 
tous in image-guided intervention, typically performed 
through the skin but also through cavities. These pro¬ 
cedures fit within the broader paradigm of surgical 
CAD/CAM, where the process involves use of pa¬ 
tient images to identify targets within the patient and 
planning needle trajectories; inserting the needles and 
verifying their placement; performing some action such 
as an injection or taking a biopsy sample; and assess¬ 
ing the results. In most cases, an accuracy of 1-2 mm 
is acceptable, which is not easy to achieve freehand, 
because the target is not directly visible, soft tissues 
tend to deform and deflect, and needles often bend. 
The procedures typically rely on some form of intra¬ 
operative imaging (x-ray, CT, MRI, and ultrasound) 
for both guidance and verification. The surgical motion 
sequence typically has three decoupled phases: place 
the needle tip at the entry point, orient the needle by 
pivoting around the needle tip, and insert the needle 
into the body along a straight trajectory. These motions 
are often performed freehand, with varying degrees of 
information feedback for the physician. However, pas¬ 
sive, semiautonomous, and active robotic systems have 
all been introduced. Figure 63.9 shows several clini¬ 
cally deployed systems for needle placement. 
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Fig.63.9a-e Clinically deployed sys¬ 
tems for in-scanner needle placement. 
(a,b) The Neuromate system (af¬ 
ter [63.19]) for stereotactic procedures 
in the brain uses a novel noncontact 
sensing system for robot-to-image 
registration; (c) Johns Hopkins sys¬ 
tem for in-CT needle placement 
(after [63.70,71]); (d,e) Manually 
activated device for in-MRI transrectal 
needle placement into the prostate 
(after [63.72]) 


Freehand Needle Placement Systems 
Freehand needle placement with CT and MRI guidance 
uses skin markers to locate the entry point [63.62], ref¬ 
erence to the scanner’s alignment laser to control needle 
direction, and markers on the needle to control depth. 
With ultrasound, the primary reliance is on surgeon ex¬ 
perience or the use of some sort of needle guide to drive 
the needle to target while it passes in the ultrasound 
plane. Tracked ultrasound snapshots guidance com¬ 
bines orthogonal viewpoints with frozen ultrasound im¬ 
age frames [63.73]. Mechanical needle guides [63.61], 
hand-held navigation guidance [63.74], and optical 
guides have been combined with most imaging modal¬ 
ities. These include laser guidance devices [63.75], 
augmented reality systems [63.76]. Augmented reality 
with 2-D ultrasound [63.77] and CT/MRI slices [63.62] 
(Fig. 63.8b) was developed, where a semi-transparent 
mirror is used together with a flat-panel display to cre¬ 
ate the appearance of a virtual image floating inside the 
body in the correct position and size. 

Passive and Semiautonomous Devices 

for Needle Placement 

Passive, encoded manipulator arms were proposed for 
image-guided needle placement [63.78], where follow¬ 
ing a registration step, the position and orientation 
of a passive needle guide is tracked and the corre¬ 
sponding needle path is displayed on CT or MRI 
images. Semiautonomous systems allow remote, in¬ 
teractive image-guided placement of needles, such as 
transrectal prostate needle placement in MRI environ¬ 
ment [63.72] with an actuated manipulator from outside 


the scanner bore, while the needle driver is tracked in 
MRI with active coils. 

Active Robots for Needle Placement 
Neurosurgery was one of the first clinical applications 
of active robots [63.19,20,22], a natural application 
for surgical CAD/CAM. The entry and target points 
are planned on CT/MRI images, the robot coordinate 
system is registered to the image coordinate system 
(typically with markers affixed to the patient’s head), 
and then the robot positions a needle or drill guide. 
The marker structure may be a conventional stereotac¬ 
tic head frame or, as in the Neuromate system [63.19], 
registration is achieved by simultaneous tracking of the 
robot and markers attached to the patient’s skull. Spatial 
constraints in needle placement led to the development 
of structures achieving remote center of motion (RCM) 
or fulcrum motion [63.24,25]. In these systems, the 
RCM is positioned at the entry point, typically with 
an active Cartesian stage or a passive adjustable arm, 
and the robot sets the needle direction and (sometimes) 
the depth. To speed up imaging, planning, registra¬ 
tion, and execution, the robot can work concurrently 
with imaging devices, such as variants of an RCM- 
based system that was deployed with x-ray [63.24] and 
CT guidance [63.70, 71]. In [63.71], a marker structure 
was incorporated in the needle driver to register the 
robot with a single image slice. MRI has an excellent 
potential for guiding, monitoring, and controlling ther¬ 
apy, invoking intensive research on MRI-compatible 
robotic systems for needle placement [63.79] and other 
more interactive procedures [63.50]. Ultrasound guid- 
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ance offers many unique advantages: is relatively inex¬ 
pensive and compact, provides real-time images, does 
not involve any ionizing radiation, and does not im¬ 
pose significant materials constraints on the robot de¬ 
sign. Several robotic systems have been proposed for 
prostate interventions [63.80] using transrectal ultra¬ 
sound guidance. For other ultrasound-guided needle 
placement applications, examples include experimental 
systems for liver [63.64,81], gallbladder [63.82], and 
breast [63.83]. Figure 63. 8d shows one example of the 
use of information overlay to assist in needle place¬ 
ment in a telesurgical application [63.64]. Whatever 
form of image feedback is available, steering flexible 
needles to hit desired targets while avoiding obstacles 
is a ubiquitous problem, having led to several novel 
approaches [63.84,85] and reviewed most recently 
in [63.86]. Concentric tube robots (also interchange¬ 
ably called active cannulas due to their usefulness in 
medicine), consist of several precurved elastic tubes 
nested within one another. Actuators grasp these tubes 
at their bases and extend them telescopically while also 
applying axial rotations. These movements cause the 
overall device to elongate and bend, providing a nee¬ 
dle-sized device that moves in a manner analogous to 
a tentacle. Chronologically, a precursor to current con¬ 
centric tube robots was an early steerable needle design 
where a curved stylet was used at the tip of a nee¬ 
dle to induce steering [63.87]. These robots fall within 
the broader class of continuously flexible robots called 
continuum robots (see [63.15] for a review.) Over the 
past few years, mechanics-based models of active can¬ 
nulas have rapidly matured due to the simultaneous 
parallel efforts of several research groups [63.14,88]. 
Today, these models provide the basis for an active 
subfield of research on teleoperative control, surgery- 
specific design, and motion planning research. Surgi¬ 
cal applications suggested for concentric tube robots 
and are in various stages of development, ranging from 
purely conceptual to animal studies, which will make 
the next several years exciting for translational clinical 
research. 

63.3.4 Telesurgical Systems 

The concepts of telemedicine, telesurgery, and telep¬ 
resence in surgery date from the 1970s. Since then, 
the potential for telesurgical systems to facilitate ef¬ 
fective interventions in remote or hostile environments 
such as the battlefield, space, or thinly populated ar¬ 
eas has continued to be recognized [63.89], and there 
have been some spectacular demonstrations including 
a transatlantic cholecystectomy [63.51] in 2001, exper¬ 
iments in Italy [63.90] and Japan [63.91] as well as 
more nearly routine use in Canada [63.52], The oper¬ 


ational difficulties due to the intrinsic communication 
delay of long distance tele-surgery affect usability and 
safety and make the regular use of tele-surgery quite 
uncommon. 

ffowever, the primary uses of telesurgical systems 
have been with the surgeon and patient in the same op¬ 
erating room. Teleoperated robots have been used for 
over 15 years in MIS, both as auxiliary surgical support 
systems to hold endoscopes or retractors [63.23,25, 
92-94] and as surgeon extender systems to manipulate 
surgical instruments [63.3, 26]. There has also been re¬ 
cent work to develop telesurgical systems for use within 
imaging environments such as MRI [63.95]. 

A primary challenge for auxiliary support systems is 
to permit the surgeon to command the robot while his or 
her hands are otherwise occupied. Typical approaches 
have included conventional foot switches [63.23], 
instrument-mounted joysticks [63.25], voice con¬ 
trol [63.10,25], and computer vision [63.25,96, 
97], 

A common goal in surgeon extender systems is 
to provide a measure of telepresence to the surgeon, 
specifically, to give the surgeon the sensation of per¬ 
forming open surgery from inside the patient. In early 
work, Green et al. [63.98] developed a successful 
prototype system for telesurgery combining remote 
manipulation, force feedback, stereoscopic imaging, 
ergonomic design, etc. Subsequently, several commer¬ 
cial telesurgical systems have been applied clinically 
for MIS. Of these. Intuitive Surgical’s da Vinci [63.3] 
has been the most successful, with over 2500 systems 
deployed as of 2013. Experience with these systems 
has demonstrated that a high-dexterity wrist is often 
critical for surgeon acceptance. Although originally 
targeted at cardiac surgery, as well as more general in¬ 
terventions, the most successful clinical applications to 
date are radical prostatectomies, where significant im¬ 
provements in outcomes have been reported [63.99], 
and hysterectomy, where the clinical benefits com¬ 
pared to conventional laparoscopy are still being stud¬ 
ied [63.100]. 

One emerging area for research exploits the in¬ 
herent ability of telesurgical systems to act as flight 
data recorders during surgical procedures. Several au¬ 
thors [63.101-104] have begun analyzing such data 
for such purposes as measuring surgical skill, learn¬ 
ing surgical gestures and motions, and providing data 
for surgical simulators. Another emerging area for re¬ 
search [63.105] focuses on semi-automation of surgi¬ 
cal gestures between the surgeon and the robot, often 
based on learned models. Other research [63.106-108] 
exploits augmented reality methods to enhance the in¬ 
formation available to the surgeon during telesurgical 
procedures. 
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63.3.5 Microsurgery Systems 

Although microsurgery is not a consistently defined 
term, it generally indicates procedures performed on 
very small, delicate structures, such as those found 
in the eye, brain, spinal cord, small blood vessels, 
nerves, or the like. Microsurgical procedures are com¬ 
monly performed under direct or video visualization, 
using some form of magnification (e.g., microscope, 
surgical loupes, high-magnification endoscope). The 
surgeon typically has little or no tactile appreciation 
of the forces being exerted by the surgical instruments 
and physiological hand tremor can be a significant 
factor limiting surgical performance. Robotic systems 
can help overcome these human sensory-motor limi¬ 
tations, and efforts to develop specialized systems for 
applications such as ophthalmology [63.6,109-115]) 
and otology [63.116-118], Several groups have also 
experimented with magnetic manipulation for these ap¬ 
plications [63.1 13,119]. There have been several efforts 
to compare microsurgical anastamosis procedures using 
laparoscopic telesurgical systems to conventional mi¬ 
crosurgery. Schiff et al. [63.120] among others reported 
significant reductions in tremor with either robot and 
significantly improved technical quality and operative 
times compared to conventional microsurgery. As the 
nubber of da Vinci systems has proliferated, such appli¬ 
cations are increasingly common [63. 121]. A number of 
groups have implemented telesurgery systems specif¬ 
ically for microsurgery [63.13,122-124] [63.95,109]. 
These systems are in various stages of development, 
from laboratory prototype to preliminary clinical exper¬ 
imentation. 

Not all microsurgical robots are teleoperated. For 
example, the cooperatively controlled JHU Steady 
Hand robots [63.4,5] [63.6,115] shown in Fig. 63.3 
are being developed for retinal, head-and-neck, neuro¬ 
surgery, and other microsurgical applications. A mod¬ 
ified version of this system has also been used for 
microinjections into single mouse embryos [63.125]. 

There have also been efforts to develop completely 
hand-held instruments that actively cancel physiolog¬ 
ical tremor, for example. Riviere et al. [63.126-128] 
have developed an ophthalmic instrument using op¬ 
tical sensors to sense handle motion and adaptive 
filtering to estimate the tremulous component of in¬ 
strument motion. A micromanipulator built into the 
instrument deflects the tip with an equal but opposite 
motion, compensating the tremor. Simple mechanical 
devices [63.129] for reducing tremor in specific tasks 
have also been developed. 

An additional type of hand-held microsurgical 
and micro-therapeutic devices is reported in [63.130], 
which describes an active microendoscope for neuroen¬ 


doscopy and therapy of the spinal cord able to safely 
navigate in the subarachnoid space and to avoid danger¬ 
ous contact with the internal delicate structures thanks 
to a system based on hydrojets. Hydrojets come from 
the lateral surface of the catheter and, appropriately 
tuned and oriented, allow the tip of the endoscope 
to proceed without touching the spinal cord internal 
walls. The shared control system of the neuroendo¬ 
scope, based on processing, segmentation, and analysis 
of the endoscopic images, assists the safe advancement 
of the tool in real time [63.131]. 

63.3.6 Endoluminal Robots 

The term endoluminal surgery was first coined by 
Cuschieri et al. [63.132] as a major component of en¬ 
doscopic surgery. Endoluminal procedures consist of 
bringing a set of advanced therapeutic and surgical tools 
to the area of interest by navigating in the lumina (i. e., 
the tube-like structures) of the human body, such as the 
gastrointestinal (GI) tract, the urinary tract, the circu¬ 
latory system, etc. Currently, most endoluminal robots 
are designed for gastrointestinal applications, although 
there has been some initial work for other areas. There 
are several advantages (from a robotics research per¬ 
spective) of working in the GI tract. The GI tract is not 
sterile and relatively large in diameter. Further it can be 
punctured intentionally to reach other abdominal cavi¬ 
ties in NOTES approaches. 

Traditionally, catheters and flexible endoscopes for 
endoluminal procedures have been inserted and ma¬ 
nipulated manually from outside the body with the 
assistance of one or more visualization systems (e.g., 
direct endoscopic video, x-ray fluoroscopy, ultrasound). 
One major challenge is limited dexterity making it dif¬ 
ficult to reach the desired target. Typically, flexible 
endoscopes have a bendable tip that can be steered by 
means of cable drives, and catheters may have only 
a fixed bend on a guide wire. There is also the inher¬ 
ent difficulty of pushing a rope, which some companies 
are trying to address by using external magnetic field 
(e.g., [63.133]). Once the target site is reached, these 
limitations become even more significant. Very simple 
instruments can be inserted through working channels 
or slid over guide wires, but dexterity is severely lim¬ 
ited and there is no force feedback beyond what can be 
felt through the long, flexible instrument shaft. 

These limitations have led a number of researchers 
to explore integration of more degrees of freedom in 
the catheter/endoscope body, as well as the design 
of intelligent tips with higher dexterity and sensing 
capabilities. Early work by Ikuta et al. led to the 
development of a five-segment, 13 mm-diameter sig- 
moidscope using shape-memory alloy (SMA) actuators. 
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Subsequently, Ikuta et al. developed 3 mm-diameter ac¬ 
tive endovascular devices using hydraulic actuators 
incorporating a novel band pass valve fabricated using 
micro-stereolithographic techniques [63.29]. 

Several examples exist of instrumented catheter tips 
with force sensors [63.134] that allow the right branch 
of the circulatory systems to be found by estimating the 
force generated between the tip and the vessel walls. 
Basically, these sensorized endoluminal devices belong 
to the larger group of micro-electromechanical systems 
(MEMS)-instrumented surgical devices and the same 
sensing technologies can be also exploited for micro¬ 
surgery. A survey article by Rebello [63.135] provides 
an excellent overview of sensorized catheters and other 
MEMS-based devices in endoluminal and microsurgi- 
cal applications. 

Another approach to endoluminal robots is repre¬ 
sented by systems that move under their own power 
through the body, rather than being pushed. Early work 
on such systems is well summarized in [63.136]. In 
1995 Burdick et al. developed an inchworm-like mech¬ 
anism for use in the colon. This device combined 
a central extensor for propulsion and inflatable bal¬ 
loons for friction enhancement with the slippery colon 
tissue. A more advanced inchworm design for a semi- 
autonomous robotic colonoscope was developed by 
Dario et al. [63.33] (Fig. 63.10). This device consists 
of a central silicone elongator, two clamping systems 
based on suction and gentle mechanical grasping of 
the colon tissue, and a silicone steerable tip integrating 
a complementary metal-oxide-semiconductor (CMOS) 
camera and a light-emitting diode (LED)-based illu¬ 
mination system. Thanks to its intrinsic flexibility, the 
robotic colonoscope applies forces on colon tissues 
that are ten times lower than those produced by tra¬ 
ditional colonoscopies. This system is now clinically 
tested [63.137], and similar devices combining flexi¬ 
bility and painless operation have been proposed by 
some companies [63.138,139]. Although the applica¬ 
tion is not endoluminal, the HeartLander system of 
Riviere et al. [63.31] shown in Fig. 63.1 la,b shares 
many of the characteristics of these systems. It uses an 
inchworm-like gait to traverse the surface of the heart 
and to perform simple operation. An instructive paper 
on the combination between flexibility and stiffness of 
endoscopic devices for allowing painless manoeuvra¬ 
bility and stable anchoring when performing surgical 
tasks has been recently published [63.140]. It presents 
a number of design criteria and solutions for transform¬ 
ing endoscopic devices from diagnostic tools to stable 
surgical paltforms. 

The natural evolution of GI endoscopic devices 
is represented by endoscopic capsules [63.141] which 
keep the promise to make endoscopy of the GI tract 
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Fig.63.10a,b Medical robot for colonoscopy (af¬ 
ter [63.33]): (a) the gait cycle of the robot, consisting of: 
(1) proximal clamping, (2) elongation, (3) distal clamping, 
and (4) retraction; (b) a recent working prototype used for 
clinical trials 


a screening method highly tolerated by patients. For 
transforming simple CMOS swallowable cameras with 
illumination and transmission functionalities into use¬ 
ful diagnostic devices, several research groups have 
explored variegated solutions for active capsule loco¬ 
motions. An example of legged locomotion capsules for 
GI application is illustrated in Fig. 63.1 lc and described 
in [63.28,142]. In order to overcome dramatic powering 
problems, magnetic capsules propelled by permanent 
magnets [63.143] or by modified MRI fields [63.144] 
have been proposed. An earlier application of electro¬ 
magnetic manipulation of an object within the body was 
the video tumor fighter of Ritter et al. [63.145]. 

63.3.7 Sensorized Instruments 
and Haptic Feedback 

Surgical procedures almost always involve some form 
of physical interaction between surgical instruments 
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Fig.63.11a-d Mobility inside the body. (a,b) HeartLander de¬ 
vice for crawling across the surface of the heart (after [63.31]). 
(c) Legged capsule for gastrointestinal diagnosis and therapy (af¬ 
ter [63.28, 148]), (d) magnetic capsule for exploration of the GI 
tract, showing left capsule and components and right magnetic 
dragging platform based on a permanent magnet driven by a robotic 
manipulator (after [63.143]) 


and patient anatomy, and surgeons are accustomed 
to use their haptic appreciation of tool-to-tissue in¬ 
teraction forces in performing surgical procedures. In 
situations such as where the clumsiness of instrumen¬ 
tation or human sensory-motor limitations limit the 
surgeon’s ability to feel these forces, surgeons have 
been trained to rely on visual or other cues to com¬ 
pensate. Apart from the need for haptic feedback for 
diagnostic procedures and tissue identification, it has 
been demonstrated that reduced tactile and force infor¬ 
mation can result in the application of unnecessarily 
large forces on the patient during surgery, with pos¬ 
sible harm to the patient [63.146]. The quality of 
haptic feedback available in currently deployed sur¬ 
gical robots is poor or nonexistent. Current research 
addresses the limitations firstly by integrating force 
and other sensors into surgical end-effectors and sec¬ 
ondly by developing improved methods for processing 
and conveying the sensed information to the surgeon. 
However, the debate of the usefulness of force feed¬ 
back in robot-assisted surgery is still open and is made 
more complex by the many scientific and technical 
difficulties due to the intrinsic presence of (possibly 
variable) communication time delay in the robotic sys¬ 
tem [63.147], 

Although the most obvious way to display haptic 
information in a telesurgical system is directly through 
the master hand controllers, this method has several 
limitations, including friction and limited bandwidth in 
the hand controllers. Although these issues may be ad¬ 
dressed through specialized design (which may raise 
costs and require other compromises) and improved 
control, there has been considerable interest in sensory 
substitution schemes [63.149-151] in which force or 
other sensor information is displayed visually, aurally, 
or by other means. Figure 63.8c shows one example of 
sensory substitution for warning when a da Vinci robot 
is about to break a suture [63.63]. 

Starting in the 1990s, several groups [63.151-153] 
have sensorized surgical instruments for microsurgery 
and MIS by equipping them with force sensors. Gen¬ 
erally, these efforts relied on sensory substitution to 
display force data, either for freehand or telesurgi¬ 
cal application. For example, Poulose et al. [63.152] 
demonstrated that a force sensing instrument used to¬ 
gether with an IBM/JHU LARS robot [63.25] could 
significantly reduce both average retraction force and 
variability of retraction force during Nissen fundoplica- 
tion. The first surgical robot with force feedback used in 
in vivo experiments was the JPL-NASA RAMS system, 
which was tested on animal models for vascular anas¬ 
tomosis [63.154] and carotid arteriotomies [63.155]. 
Rosen et al. [63.153] developed a force-controlled tele- 
operated endoscopic grasper equipped with position 


sensors and actuated by direct-current (DC) motors 
whose output torque is sensed and fed back through 
motors integrated in a grasping handle. A similar ap¬ 
proach was used by Menciassi et al. [63.156] for 
a microsurgery gripper equipped with semiconductor 
strain gauges and a PHANTOM (SensAble Technolo¬ 
gies, Inc.) haptic interface. More recent work at Johns 
Hopkins has focused on incorporation of optical fiber 
force [63.157-160] and OCT [63.161-163] sensors into 
microsurgical tools. Both video and auditory sensory 
substitution [63.164], as well as direct feedback to 
robotic devices have been used to help surgeons con¬ 
trol tool-tissue interaction forces and distance. 

Several researchers [63.165] have focused on spe¬ 
cialized fingers and display devices for palpation tasks 
requiring delicate tactile feedback (e.g., for detecting 
hidden blood vessels or cancerous tissues beneath nor¬ 
mal tissues). There has also been work to integrate non- 
haptic sensors into surgical instruments, for example, 
Fischer et al. have developed instruments measuring 
both force and tissue oxygenation levels [63.166]. This 
information can be used for such purposes as assessing 
tissue viability, distinguishing between different tissue 
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types, and controlling retraction so as not to cause is¬ 
chemic tissue damage. 

Finally, it is important to note that sensorized surgi¬ 
cal tools have important application beyond their direct 
use in surgical procedures, for example, one use is in 
biomechanical studies to measure organ and tissue me¬ 
chanical properties to improve surgical simulators. 

63.3.8 Surgical Simulators 

and Telerobotic Systems for Training 

Medical education is undergoing significant changes. 
The traditional paradigm for surgical technical train¬ 
ing is often summarized as see one, do one, teach one. 
This method can be effective in open surgery, where 
the surgical trainee directly observes the expert surgeon 
hands, sees the instrument motion, and follows the or¬ 
gan manipulation. However, in endoscopic surgery it is 
difficult to observe the surgeon’s hand movements (out¬ 
side the body) and the surgical tool actions (inside the 
body and only visible on a video monitor). In addi¬ 
tion, endoscopic surgery requires different skills than 
open surgery, such as spatial orientation, interpretation 
of 2-D images in 3-D, and manipulating instruments 
through entry portals. These considerations led to in¬ 
troduction of surgical simulation systems of varying 
degrees of complexity and realism for endoscopic and 
other minimally invasive procedures. Nowadays, train¬ 
ing simulators have achieved widespread acceptance 
in the field of anaesthesia, intensive care, flexible en¬ 
doscopy, surgery, interventional radiology, and other 
fields. The use of simulators for training is so com¬ 
mon that working groups have been set up in order 
to evaluate these training systems based on shared 
guidelines [63.167] and many teaching hospitals have 
extensive simulation training centers. Simulators are 
being validated on their basic parameters (face, content 
and construct) [63.168], whereas results on concurrent 
and predictive validity of simulation training are still 
not available. 

Survey [63.169] divides training simulators into 
three groups, depending on the type of technology used: 
mechanical, hybrid, and virtual reality. 

Mechanical simulators are boxes where objects or 
organs are placed and manipulated using surgical in¬ 
struments. The manipulation is observed through a la¬ 
paroscope and a screen. The simulated surgical task 
is observed by an experienced surgeon, who gives 
feedback to the trainee. Generally, there are no registra¬ 
tion processes and the simulator must be reassembled 
after any training session. The LapTrainer with SimuVi- 
sion (Simulab Inc., Seattle, USA) is a training system 
with a simulated laparoscope that consists of a boom- 
mounted digital camera in an open box trainer. 


A hybrid simulator uses a box with objects or or¬ 
gans as a mechanical simulator, but in addition the 
performance of the trainee are monitored by a com¬ 
puter which gives guidance for the task execution and 
an objective feedback based on preplanned metrics. 
Thanks to this feedback, the assistance and judgement 
of an experienced surgeon are not strictly necessary, for 
example, the P 10 MIS (Haptica Inc., Boston, USA) is 
a typical hybrid simulator for training basic skills such 
as suturing and knot tying. Surgical instruments are 
passed through dedicated ports and the trainee receives 
the same haptic feedback as in real surgery during ma¬ 
nipulation in the box. In addition, ProMIS analyzes the 
trainee’s performance by tracking the instrument posi¬ 
tion in 3-D and by measuring the execution time, path 
length, and smoothness of task execution. Another re¬ 
cent hybrid simulator is the Perk Tutor open-source 
platform for ultrasound-guided percutaneous needle in¬ 
sertion training [63.170], which has been successfully 
applied in teaching of ultrasound-guided facet joint in¬ 
jections [63.171]. Specific to robotic surgery is the 
RoSS system [63.172], which simulates the operator 
console of the da Vinci robotic system. Intuitive Surgi¬ 
cal currently markets the dv-Trainer [63.173] simulator 
for the da Vinci system, comprising the surgeon console 
from a da Vinci Si system and a computer back end to 
simulate the patient-side manipulators, and its efficacy 
is being evaluated [63.174]. 

Finally, virtual-reality training systems combine vi¬ 
sualization and haptic interfaces to enable surgeons to 
interact efficiently and naturally with real-time com¬ 
putational models of patient anatomy [63.175]. The 
development of these systems is inherently a multidis¬ 
ciplinary effort, including real-time computer graph¬ 
ics, the development of high-bandwidth haptic devices, 
real-time biomechanical modeling of organs, tool- 
tissue interactions, expertise in training assessment, and 
human-machine information exchange, etc. [63.176]. 
Research in these areas is closely related to and syn¬ 
ergistic with comparable developments in technology 
and systems for performing real interventions, for ex¬ 
ample, modeling of organ motion in response to forces 
is necessary to improve the accuracy of needle place¬ 
ment procedures. Haptic feedback devices must meet 
similar requirements whether the forces displayed are 
simulated or measured directly in telesurgery, and so 
on. Finally, as noted earlier, sensorized instruments 
and real-time imaging are critical sources of data 
needed to create realistic biomechanical models. The 
use of the computer graphical processor (GPU) al¬ 
lows the simulation of organs at rates in excess of 
10kHz [63.177], which enables good haptic feedback. 
Similarly, new physics-based graphical libraries sup¬ 
port the development of low cost virtual reality simu- 
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lators that correct render the interaction of instruments 
and anatomical parts, and thus can support large train¬ 
ing classes [63.178]. 

The variety of interface devices and virtual reality 
laparoscopic simulators is quite wide and increasing 
numbers of systems are becoming commercially avail¬ 
able. The Phantom interface is used in conjunction with 
virtual simulators to provide users with realistic haptic 
feedback (SensAble Technologies Inc., Woburn, MA, 
USA). The Xitact LS500 laparoscopy simulator (Xi- 
tact S.A., Lausanne, Switzerland) is a modular virtual- 
reality simulation platform with software for training 
and assessing performance in laparoscopic tasks. It is an 
open system including all or some of these subsystems: 
laparoscopic tools, mechanical abdomen, a personal 
computer (PC) providing the virtual-reality scenario, 
a haptic interface, a force feedback system and a track¬ 
ing system for the tools. Several other systems for 
virtual reality simulation exist that exploit the hardware 
from Xitact or Immersion Medical, Inc. (Gaithers¬ 
burg, MD, USA) and that are dedicated to specific 
surgical tasks: Lapmentor [63.179], the Surgical Edu¬ 
cation platform [63.180], LapSim [63.181], Procedicus 
MIST [63.182], EndoTower [63.183], the Reachin la¬ 
paroscopic trainer [63.184], Simendo [63.185], and 
the Vest system [63.186]. Specifically for training eye 
surgeons, the surgical simulator EYESi [63.187] uses 
advanced computer technology and virtual reality to 
simulate the feel of real eye surgery, making it possi¬ 
ble for surgeons at all levels to acquire new skills and 
perfect their techniques in preparation for surgery on 
the human eye. Training for dexterity skills in robotic 
surgery is supported by the new virtual simulator XRON 
developed at the University of Verona, which interfaces 
physics-based simulation with several commercial joy¬ 
sticks and provides evaluation metrics of interactive 
tasks [63.188]. 


Making a comparison between these different cate¬ 
gories of simulators is not trivial. Basically, box trainers 
and hybrid simulators require some experienced techni¬ 
cians for the set up and some organizational logistics 
due to legal and ethical factors related to the storage 
of freshly explanted organs. The most evident advan¬ 
tage of these simulators is that the tactile response from 
the manipulated objects is the same as in real surgery 
and complicated models of organs and tissue-tool inter¬ 
action are not required. On the other hand, completely 
virtual-reality trainers are potentially very flexible, can 
take advantage of powerful graphical engines, but they 
are limited by the availability of realistic calibration 
data for the anatomical and biomechanical models. Al¬ 
though demonstrations exist of the ability of simulators 
to record, objectively score, and hone the psychomo¬ 
tor skills of novice surgeons [63.189], the debate about 
their value is still open because no multicentre trails 
has yet been conducted to determine their efficacy. 
Furthermore, it has been shown that simulation train¬ 
ing must be included in structured curricula in robotic 
surgery, where trainees are also introduced to basic con¬ 
cepts of robotics and, in particular, to the so called 
non technical skills, e.g., organization, leadership and 
communication, which significantly affect the surgical 
outcome [63.190]. 

63.3.9 Other Applications 
and Research Areas 

The research areas described above illustrate major 
themes within medical robotics, but they are by no 
means exhaustive. Many important application areas 
such as otolaryngology [63.191-194] and radiation 
therapy have necessarily been excluded for space rea¬ 
sons. For a fuller exploration, readers should consult the 
further reading suggestions in Sect. 63.4. 


63.4 Conclusion and Future Directions 


Medical robotics (and the larger field of computer- 
integrated interventional medicine) has great potential 
to revolutionize clinical practice by: 

• Exploiting technology to transcend human limita¬ 
tions in treating patients 

• Improving the safety, consistency, and overall qual¬ 
ity of interventions 

• Improving the efficiency and cost-effectiveness of 
robot-assisted patient care 

• Improving training through the use of validated 
simulators, quantitative data capture and skill as¬ 


sessment methods, and the capture and playback of 
clinical cases 

• Promoting more effective use of information at all 
levels, both in treating individual patients and in im¬ 
proving treatment processes. 

From being the stuff of late-night comedy and sci¬ 
ence fiction 20 years ago, the field has reached a critical 
threshold, with clinically useful systems and commer¬ 
cial successes. The scope and number of research pro¬ 
grams has grown exponentially in the past 8 years, and 
this chapter is by no means a comprehensive survey 
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of the field. In fact, such a survey would be practi¬ 
cally impossible in less than 100 pages. Many important 
emerging systems and applications are necessarily left 
out. Interested readers are urged to refer to the further 
reading section for more complete treatments. In partic¬ 
ular, the survey articles and books in listed at the end of 
this section collectively contain somewhat fuller bibli¬ 
ographies than space permits here. 

In the future, we can expect to see continued re¬ 
search in all aspects of technology and system devel¬ 
opment, with increasing emphasis on clinical applica¬ 
tions. As this work proceeds, it is important that re¬ 
searchers remember several basic principles. The first, 
and arguably most important, principle is that medi¬ 
cal robotics is fundamentally a team activity, involving 
academic researchers, clinicians, and industry. Each of 

Video-References 


these groups has unique expertise, and success comes 
from effective, highly interactive partnerships drawing 
upon this expertise. Building these teams takes a long¬ 
term commitment, and the successes in recent years 
are largely the pay-off from investments in creating 
these teams. Second, it is important to work on prob¬ 
lems with well-defined clinical and technical goals, 
in which the criteria for measuring success are ulti¬ 
mately related to real advantages in treating patients. 
In working toward these goals, it is important to have 
measurable and meaningful milestones and to empha¬ 
size rapid iteration with clinician involvement at all 
stages. Finally, it is essential that all team members 
recognize the level of commitment that is required 
to achieve success and that they enjoy what they are 
doing. 
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H.F. Machiel Van der Loos, David J. Reinkensmeyer, Eugenio Guglielmelli 


The field of rehabilitation robotics considers 
robotic systems that 1) provide therapy for persons 
seekingto recovertheir physical, social, communi¬ 
cation, or cognitive function, and/or that 2) assist 
persons who have a chronic disability to accom¬ 
plish activities of daily living. This chapter will 
discuss these two main domains and provide de¬ 
scriptions of the major achievements of the field 
over its short history and chart out the challenges 
to come. Specifically, after providing background 
information on demographics (Sect. 64.1.2) and 
history (Sect. 64.1.3) of the field, Sect. 64.2 de¬ 
scribes physical therapy and exercise training 
robots, and Sect. 64.3 describes robotic aids for 
peoplewith disabilities. Section 64.4 then presents 
recent advances in smart prostheses and orthoses 
that are related to rehabilitation robotics. Finally, 
Sect. 64.5 provides an overview of recent work in 
diagnosis and monitoringfor rehabilitation as well 
as other health-care issues. The reader is referred 
to Chap. 73 for cognitive rehabilitation robotics and 
to Chap. 65 for robotic smart home technologies, 
which are often considered assistive technologies 
for persons with disabilities. At the conclusion of 
the present chapter, the reader will be familiar 
with the history of rehabilitation robotics and its 
primary accomplishments, and will understand 
the challenges the field may face in the future as 
it seeks to improve health care and the well being 
of persons with disabilities. 


64.1 Overview . 1686 

64.1.1 Taxonomy 

of Rehabilitation Robotics . 1686 

64.1.2 World Demographics . 1687 

64.1.3 Short History 

of Rehabilitation Robotics . 1688 


64.2 Rehabilitation Therapy 

and Training Robots . 1692 

64.2.1 Grand Challenges 

and Roadblocks . 1692 

64.2.2 Movement Therapy 

After Neurologic Injury . 1694 

64.2.3 Robotic Therapy 

for the Upper Extremity . 1695 

64.2.4 Robotic Therapy for Walking . 1699 

64.3 Aids for People with Disabilities . 1703 

64.3.1 Grand Challenges 

and Enabling Technologies . 1703 

64.3.2 Types and Examples 

of Assistive Rehabilitation 

Robots . 1705 

64.4 Smart Prostheses and Orthoses . 1711 

64.4.1 Grand Challenges 

and Roadblocks . 1711 

64.4.2 Targeted Re-Innervation . 1711 

64.4.3 Neural Interfaces 

for Limb Prosthetic Devices . 1712 

64.4.4 Advances in Neural Stimulation .. 1713 

64.4.5 Embedded Intelligence . 1713 

64.5 Augmentation for Diagnosis 

and Monitoring . 1713 

64.5.1 Grand Challenges 

and Enabling Technologies . 1714 

64.5.2 Smart Homes for Health Care 

Monitoring and Care . 1714 

64.5.3 Home-Based 
Rehabilitation Monitoring 

and Therapy Systems . 1714 

64.5.4 Wearable Monitoring Devices . 1715 

64.6 Safety, Ethics, Access and Economics . 1715 

64.7 Conclusions and Further Readings . 1716 

Video-References . 1717 

References . 1718 


Part F | 64 
































Part F | 64.1 


1686 Part F 


Robots at Work 


64.1 Overview 

In this chapter, we discuss an application of robotics 
that will likely touch many of us in an acutely per¬ 
sonal way at some point in our lives. When, through 
injury or disease, we become unable to interact physi¬ 
cally with our immediate environment as we desire to 
achieve our personal goals, or when one of our fam¬ 
ily members, friends, or neighbors is in this situation, 
technology-based solutions will likely be a major com¬ 
ponent in the treatment interventions that therapists 
prescribe to assist us in re-leaming how to complete 
our activities of daily living (ADL), or to assist us in 
actually doing them if we are unable to recover lost 
function. While human therapists and attendants can 
indeed shoulder the types of assistance required, the 
projected short-term demographics of many countries, 
including China, Japan, and the Scandinavian countries, 
show a growing shortage of working-age adults. Age- 
related disabilities will soon substantially impact the 
service sector job market, put many older and disabled 
people at risk, and increase the need for institutional¬ 
ization when there is no viable home-based solution. 
National programs to develop personal robots, robotic 
therapy, smart prostheses, smart beds, smart homes, 
and tele-rehabilitation services have accelerated in the 
past 20 years and will need to continue apace with the 
ever increasing ability of health care to allow people 
to live longer through the repression of disease and 
improvement in surgical and medication interventions. 
Rehabilitation robotics, although only a 50 year old dis¬ 
cipline [64.1-3], is projected to have a fast-growing 
future in the coming decades. Within the past 10 years, 
the field has seen significant strides in the commercial¬ 
ization of rehabilitation robotics due to an increasing 
acceptance of the validity of this approach by clinical 
care providers, as well as the cost reductions in sensors 
and actuators. We have also witnessed the expansion 
of applications from the social robotics research do¬ 
main into rehabilitation, increasing the range of persons 
whose impairments can be targeted by robotic technolo¬ 
gies in the years to come. 

64.1.1 Taxonomy 

of Rehabilitation Robotics 

The field of rehabilitation robotics is generally di¬ 
vided between the categories of therapy and assistance 
robots, although some devices can serve both purposes. 
In addition, rehabilitation robotics includes aspects 
of artificial limb (prosthetics) development, functional 
neural stimulation (FNS) and technology for diagno¬ 
sis and monitoring of people during activities of daily 
living. 


Therapy robots generally have at least two main 
users simultaneously, the person with a disability who 
is receiving the therapy and the therapist who sets up 
and monitors the interaction with the robot. As this type 
of therapy is moving into the home, a third user group 
has also become prominent: the disabled person’s care¬ 
givers and family. 

The Types of therapies that have benefited from 
robotic assistance are upper- and lower-extremity 
movement therapy, communication-enabling for chil¬ 
dren with autism and exploration-enabling (education) 
for children with cerebral palsy (CP) or other develop¬ 
mental disabilities (Chap. 73). A robot may be a good 
alternative to a physical or occupational therapist for the 
actual hands-on intervention for several reasons: 

1. Once properly set up by a therapist - an essential 
role - an automated exercise machine can con¬ 
sistently apply therapy over long periods of time 
without tiring. 

2. The robot’s sensors can measure the work per¬ 
formed by the patient and quantify, to an extent not 
detectable by standard clinical scales, any recovery 
of function that may have occurred, which may be 
highly motivating for a person to continue in the 
therapy. 

3. The robot may be able to engage the patient in 
types of therapy exercises that a therapist cannot do, 
such as computer game-based therapy or magnify¬ 
ing movement errors to provoke adaptation [64.4, 

5]. 

One way to classify robotic solutions for applica¬ 
tion to rehabilitation therapy is by how they contact the 
patient [64.6]. 

Operational Therapy Robots 

(or End-Effector-Based Therapy Robots) 

For these machines, the trajectories of the robot end- 
effector and that of the human end-effector, e.g., the 
hand or the foot, are physically coupled in operational 
space. In joint space, the trajectories of the robot joints 
and of the human joints can significantly differ. Main 
advantages of operational therapy robots are that: (a) 
they can be designed by using off-the-shelf components 
or robots; (b) they can be easily programmed in Carte¬ 
sian space by nonexpert users. Main limitations are that: 
(a) they are not able to assist each single human joint 
independently; (b) patients using these robots are sup¬ 
posed to feature a minimum level of residual motor 
synergies in order to coordinate their own multijoint 
movement, producing the configurations of the affected 
limb required by the therapy exercise. 
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Wearable Therapy Robots 

(or Exoskeleton-Based Therapy Robots) 

For these machines, a larger portion of the human 
body (typically the whole affected limb) is in contin¬ 
uous physical contact with the robot. In most cases, 
a biomimetic exoskeleton kinematic structure is se¬ 
lected. In this case, not only are the trajectories of the 
robot and human end-effectors the same in operational 
space, but the trajectories of the robot joints approxi¬ 
mate those of the human joints in joint space. The main 
advantage of wearable therapy robots is the possibility 
of sensing the configuration and assisting each human 
joint independently. The main drawbacks of these sys¬ 
tems are additional design considerations needed to 
avoid misalignments between robot and human joints, 
and to minimize the invasiveness for the patient in 
terms of weight, dimensions, and overall wearability 
(Sect. 64.2.4). 

Non-Contact Therapy Robots 
(or Socially Assistive Robotics) 

Robotic devices are also being developed for rehabili¬ 
tation therapy that do not physically contact the patient, 
but instead monitor and coach the patient during ther¬ 
apy [64.7], A key concept of this approach is that 
humans are wired to respond to embodied agents in 
ways that are important for motivation in rehabilita¬ 
tion therapy (see Chap. 73 for a detailed discussion of 
socially assistive robotics). One of the earliest robotic 
therapy devices, developed by Erlandson, can be clas¬ 
sified as a socially assistive robot, as it provided reach¬ 
ing targets for patients rather than physically assisting 
movement [64.8] (see later). The main advantage of 
noncontact therapy robots is that of being intrinsically 
safe, since they are not supposed to physically interact 
with the patient, although this could limit significantly 
the scope of their clinical application. 

From a design perspective, it is important to under¬ 
stand that therapy robots are tools typically intended for 
temporary use (i. e., the duration of the therapy at home 
or at the clinic), and are designed to maximize the ob¬ 
jective clinical effectiveness of the therapy as well as 
the outcome and the efficiency of the entire clinical pro¬ 
cess. 

Assistive robots, instead, are solutions for promot¬ 
ing independent living of disabled and elderly citizens. 
They need to be designed to be usable in a lifelong per¬ 
spective in real-life scenarios, and thus, designers need 
to take into account (at a much more serious level than 
in the case of therapy robots) the end-user subjective 
preferences, and human factors, in general, in order to 
maximize their overall, long-term acceptability [64.6], 
Whereas limited usability, some level of discomfort, or 
bad aesthetics might be tolerated by the patients expe¬ 


riencing the application of a therapeutic tool used in 
a gym or at home, these same factors are unacceptable 
for disabled or elderly persons who are supposed to de¬ 
pend upon an assistive robot for activities of daily living 
in a variety of social contexts for the rest of their lives. 

Assistive robots can be grouped on the basis of their 
focus on manipulation, mobility, or cognition. Manipu¬ 
lation aids can further be classified into fixed-platform 
and portable-platform and mobile-autonomous types. 
Fixed-platform robots perform functions in the kitchen, 
on the desktop, or by the bed. Portable types are ma¬ 
nipulator arms attached to an electric wheelchair to hold 
and move objects and to interact with other devices and 
equipment, as in opening a door. Mobile-autonomous 
robots can be controlled by voice or other means to do 
manipulation and other errands in the home or work¬ 
place. Mobility aids are divided into electric wheelchairs 
with navigation systems and mobile robots that act as 
smart, motorized walkers, allowing people with mobility 
impairment to lean on them for fall-prevention and sta¬ 
bility. The third main type, cognitive aids, assists people 
who have dementia, autism, or other disorders that affect 
communication and physical well-being. 

The fields of prosthetics, orthotics, and functional 
electrical stimulation (FES) are closely allied with reha¬ 
bilitation robotics. Prostheses are artificial hands, arms, 
legs and feet that are worn by the user to replace ampu¬ 
tated limbs. Prostheses are increasingly incorporating 
robotic features [64.9, 10]. Robotic orthoses are ac¬ 
tuated braces that can assist a person in walking or 
moving the arms or hands. FES systems seek to re¬ 
animate limb movements of people who are weak or 
paralyzed by electrically stimulating nerve and muscle. 
FES control systems are analogous to robotic control 
systems, except that the actuators being controlled are 
human muscles. Another related field is technology for 
monitoring and diagnosing health care issues as a per¬ 
son performs activities of daily living. 

This chapter is organized according to this taxon¬ 
omy. After providing background information on de¬ 
mographics (Sect. 64.1.2) and history (Sect. 64.1.3) of 
the field. Sect. 64.2 describes rehabilitation therapy and 
training robots, and Sect. 64.3 describes robotic aids 
for people with disabilities. Section 64.4 then reviews 
recent advances in smart prostheses and orthoses that 
are related to rehabilitation robotics. Finally, Sect. 64.5 
provides an overview of recent work in diagnosis and 
monitoring for rehabilitation as well as other health¬ 
care issues. 

64.1.2 World Demographics 

Different areas of rehabilitation robotics focus on dif¬ 
ferent user populations, but a linking characteristic 
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of these populations is that they have a disability. 
Disability is defined in the US with the Disabilities 
Act as A physical or mental impairment that sub¬ 
stantially limits one or more of the major life ac¬ 
tivities. Worldwide disability, based on survey results 
from the WHO ranges from about 10 to 40% of the 
population, depending on gender, age, wealth, and 
residence, with an overall prevalence of about 16% 
(Table 64.1). Prevalence is the proportion of a pop¬ 
ulation who have (or had) a specific characteristic in 
a given time period. In medicine, typically this char¬ 
acteristic is an illness or impairment. Prevalence is 
usually expressed as a percentage (e.g., 5%,), or as 
the number of cases per 10000 or 100000 people, de¬ 
pending on how common the illness or risk factor is 
in the population. Across all other parameters, there 
is approximately a 4 : 1 disparity in disability rates in 
the elderly population (>60 years) over working-class 
adults. In addition, lower birth rates and life-extending 
health care are the dominant factors contributing to 
the aging of the population and concomitant rise in 
disability overall. Other factors, such as China’s popu¬ 
lation control policies of the 1970s, have created a lack 
of working-age adults to support the world economy. 
These and other factors make it clear that rehabilita¬ 
tion robotics developers will be faced with users who, 
as a demographic group, have generally lower lev¬ 
els of sensory and motor capability, and may have 
impaired cognition as well. The urgency of making 
advances in this field is increasing in line with the 
demographics. 

64.1.3 Short History 

of Rehabilitation Robotics 

The history of rehabilitation robotics is almost as old 
as that of robotics itself, although emanating from 
very different sources. Several book chapters and pa¬ 
pers have been written on the history of rehabilitation 


robotics in more detail than this section [64.1, 19,20], 
and numerous papers in the proceedings of the IEEE In¬ 
ternational Conference on Rehabilitation Robotics also 
provide more grounding for historical perspective. The 
chronology below pays particular attention to early 
work and to projects with notable clinical and/or com¬ 
mercial impact. 

Early robotics, starting in the late 1950s, focused 
on large manipulators to replace workers in factories 
for dirty, dangerous, and undesirable tasks. The earliest 
rehabilitation robots came from the field of prosthetics 
and orthotics (P&O). Both the Case Western University 
Arm (1960s) and the Rancho Los Amigos Golden Arm 
(early 1970s) (reviewed in [64.19]) were adaptations 
of replacement mechanical arms meant as powered or¬ 
thoses [64. 1], The user drove the Golden Arm with a set 
of tongue-operated switches, joint-by-joint, an arduous 
means of endpoint control. In the mid-1970s, the De¬ 
partment of Veterans Affairs began funding a group 
at the Applied Physics Lab under the guidance of 
Seamone and Schmeisser to computerize an orthosis 
mounted on a workstation to perform activities of daily 
living (ADL) tasks such as feeding a person and turning 
pages [64.21]. For the first time, a rehabilitation robot 
had a command-type interface, not just a joint-by-joint 
motion controller. 

The 1970s also saw the French Spartacus system 
being developed, guided by the vision of Vertut, for 
use by people with high-level spinal cord injury (SCI) 
as well as children with CP [64.22]. This system did 
not emerge from the P&O field but was developed by 
the French Atomic Energy Commission (CEA), which 
used large tele-manipulators for nuclear fuel rod han¬ 
dling. One of these was adapted so that people with 
movement impairment could control it using a joystick 
for pick-and-place tasks. A decade later, one of the re¬ 
searchers on the Spartacus project, Kwee, began the 
Dutch MANUS Project [64.23] a dedicated effort to 
develop the first wheelchair-mounted manipulator de- 


Table 64.1 Prevalence of disability and aging in selected countries (after [64.18]) 


Country 

Number people with 
disabilities 

Percentage of population 
with disabilities 

Number of elderly people 

Percentage of population 
that is elderly 

France 

5 146 000 

8.3 

12151000 

19.6 

USA 

52 591000 

20.0 

35 000000 

12.4 

Great Britain 

4453 000 

7.3 

12 200000 

29.5 

Netherlands 

1 432 000 

9.5 

2118 808 

13.4 

Spain 

3 528 220 

8.9 

6936000 

17.6 

Japan 

5 136000 

4.3 

44982 000 

35.7 

Korea 

3 195 000 

7.1 

16300000 

36.0 

Italy 

2 609 000 

4.8 [64.11] 

12 302 000 

20.3 [64.12] 

Germany 

7 101 682 

8.7 

16 844 300 

21.0 [64.13] 

China 

84 600 000 

6.5 [64.14] 

122 880000 

9.1 [64.15] 

India 

21000 000 

1.8 [64.16] 

67117 826 

5.6 [64.17] 
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signed expressly as a rehabilitation robot, not adapted 
from a design from another field. 

In between, several other major programs were be¬ 
gun. In 1978, at Stanford University, and then with 
decades-long funding from the US Department of Vet¬ 
erans Affairs, Leifer started the Vocational Assistant 
Robot program, culminating in several clinically tested 
versions of the Desktop robot, DeVAR [64.3, 24, 25], 
the mobile version, Mo VAR [64.26], and finally a Pro¬ 
fessional version, Pro VAR, developed by Van der Loos 
et al., which had the advanced ability for the user to 
program tasks in an easy-to-use browser-type environ¬ 
ment [64.27]. Although DeVAR briefly made it onto 
the market in the early 1990s, multisite user testing 
revealed this industrial-arm-based approach to assis¬ 
tive robotics was still too costly for the low level of 
functionality achieved, even with ProVAR’s advanced 
interface. 

In the mid-1980s, from observations on the unsuit¬ 
ability of existing industrial, educational and orthosis- 
derived manipulators for rehabilitation applications, 
Tim Jones at Universal Machine Intelligence (later Ox¬ 
ford Intelligent Machines: OxIM) in the UK began an 
intensive effort to provide the rehabilitation robotics 
community with its first workhorse system especially 
designed from the ground up for human service tasks. 
Over 10 years, a series of systems, starting with the 
RTX model, were used in numerous research labs and 
clinics around the world. The most extensive effort 
to use the OxIM arm was in France, and a suite of 
research projects, funded by the French government 
and the European Research Commission, starting as 
RAID, then as MASTER [64.28], developed and clin¬ 
ically tested workstation-based assistive systems based 
on the RTX and subsequent OxIM arms. When OxIM 
ceased building its arms, the French company Afma 
Robotics [64.29] took over efforts to commercialize 
the MASTER system, although it has also stopped 
production. 

The UK witnessed the first commercially avail¬ 
able feeding robot, Handy-I, an inexpensive and well- 
received device first designed by Topping and then 
commercialized by Rehabilitation Robotics, Ltd. in the 
1990s [64.30]. Primarily for people with CP to achieve 
a measure of independence in feeding themselves, task 
environments later also included face washing and ap¬ 
plying cosmetics, areas of high demand identified by its 
users. 

The history of mobile manipulator applications be¬ 
gan in the 1980s with adaptations of educational and 
industrial robots, and achieved a boost with the funding 
of the US National Institute on Disability and Reha¬ 
bilitation Research (NIDRR) for a Rehabilitation En¬ 
gineering Research Center on Rehabilitation Robotics 


(RERC) at the AI duPont Hospital in Delaware from 
1993 to 1997. With its ability to fund dozens of research 
projects in parallel, it also formed a partnership with 
a local company. Applied Resources, Corp. (ARC), 
which developed and marketed several rehabilitation 
technology products. One of the RERC researchers, 
Mahoney, moved to ARC and was instrumental in 
extending the company’s repertoire to the RAPTOR 
wheelchair-mounted arm [64.31]. 

In Europe, the most significant mobile manipula¬ 
tor project was the MANUS Project [64.23] mentioned 
earlier. With much of the work done under the direc¬ 
tion of Kwee at the iRV (Rehabilitation R&D Center) 
in the Netherlands, the project culminated in a robot 
specifically designed for wheelchair mounting, with 
user control through a joystick and feedback by a small 
display on the arm itself. This project led to numer¬ 
ous follow-on research projects, and, most significantly, 
to the commercialization of the system by Exact Dy¬ 
namics, BV, in the Netherlands. The company’s current 
product, called the iARM, is provided on physician pre¬ 
scription by the Dutch Government to qualified people 
with a disability, such as CP or tetraplegia from an SCI. 

Realizing a potential growth in this market niche, 
the Canadian company, Kinova Robotics, in 2009 began 
commercializing a competing product, called the Jaco 
Arm [64.32], with a different design approach. Using 
carbon fiber segments and lightweight actuator and con¬ 
trol components, the payload specifications and control 
options are approximately the same as for the iARM, 
but achieved at a lower arm weight. 

Autonomous navigation systems on electric 
wheelchairs began in the 1980s also, benefiting initially 
from the development by Polaroid Corp. of range 
finders for its cameras using ultrasonic sensors. They 
were inexpensive, and small enough, at 30 mm in 
diameter, that dozens of them could be placed around 
the periphery of a wheelchair to aid medium-range 
navigation (f» 10—500 cm). In the 1990s and early 
2000s, with the advent of vision-based servoing and 
laser-range scanners, algorithms for faster, smarter, 
less-error prone navigation and obstacle avoidance 
dominated research advances in this sector. In Korea, 
for example, Bien et al., at the KAIST Human Welfare 
Robotics Center began developing the KARES line 
of wheelchair-based navigation systems in the late 
1990s [64.33] and the NavChair project at the Univer¬ 
sity of Michigan was the start of a development line 
that led to the commercialized Hephaestus system at 
the University of Pittsburgh [64.34, 35]. 

Therapy robots had a later start than assistive robots. 
Research on upper extremity therapy robots started 
in the mid-1980s with early exercise devices, such 
as the BioDex [64.36] a first step in programmable. 
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force-controlled, albeit single-axis devices. The first 
multiaxis concept was published by Khalili and Zom- 
lefer [64.37], and the first tested system by Erland- 
son et al. at Wayne State University emerged in the 
mid-1980s as well [64.8]. The RTX manipulator had 
a touch-sensitive pad as an end-effector, presenting 
targets at different locations for patients with upper 
extremity weakness (e.g., following a stroke) to hit af¬ 
ter the screen gave a visual signal. Software logged 
response times, thereby providing a score that was tal¬ 
lied and compared to previous sessions. Later robots 
used advanced force-based control, which required sig¬ 
nificantly more computer power. The early-1990s saw 
the start of the MIT-MANUS Project with Hogan 
and Krebs, simple robotic devices designed for bi¬ 
manual therapy after stroke designed by Lum et al. 
at U.C. Berkeley [64.38,39], followed a few years 
later by the Palo Alto VA MIME (mirror image mo¬ 
tion enabler) Project [64.40] and its derivative, Driver’s 
SEAT [64.41], with Burgar, Van der Loos and Lum, as 
well as the Rehabilitation Institute of Chicago ARM 
Guide Project with Reinkensmeyer et al. [64.42]. Each 
had a different philosophy on upper extremity stroke 
therapy and each was able to demonstrate clinical effec¬ 
tiveness in a different way. Several of these programs, 
now two decades later, have made significant technical 
advances, and all the investigators are still active in the 
field. Current work is described in subsequent sections 
of this chapter. 

Research on lower extremity therapy robots be¬ 
gan with the work of Scherb in 1919 who pro¬ 
posed an approach called mechanotherapy [64.43]. He 
also developed a prototype, depicted in Fig. 64.1, of 
a cable-driven machine for assisting the motion of 
the lower limbs for bedridden patients. In 1976, the 
use of a master/slave exoskeleton system for lower 
limb therapy was introduced by the French physi¬ 
cians Bel and Rabischong [64.44], who developed 



Fig. 64.1 The meridian. A universal machine for applying 
passive motion to any joint (after [64.43]) 


a pneumatically actuated wearable robot to be used 
for assisted therapy of paraplegic patients (Fig. 64.2). 
The robot movements were remotely controlled by 
a therapist who wore a master device capable of sens¬ 
ing and recording the configuration of her/his lower 
limbs. 

About 20 years later, the Lokomat project for the 
development of a stand-alone, programmable wearable 
robot for lower extremity was launched and originated 
the establishment of the Hocoma company by Colombo 
and Hostettler. The first prototype of the Lokomat sys¬ 
tem [64.45] was developed in close cooperation with 
the Balgrist University Hospital in Zurich and initially 
applied to patients who had had spinal cord injuries 
(Fig. 64.3b). During the same time, Hesse and Uhlen- 
broch developed the Gait Trainer [64.46], (Fig. 64.3a), 
which allowed training without the use of a treadmill 
by controlling the center of mass (COM) in both in the 
vertical and horizontal directions. 

In the last decade the growth in rehabilitation 
robotics has been characterized by the development of 
new devices (as explained above) as well as by new 
investigation methods aimed at understanding brain 
changes induced by rehabilitation. A recent clinical 
study [64.50] confirms that high-intensity, repetitive, 
and task-oriented training yields a faster learning and 
recovery time. Robots are also a key-enabling tech¬ 
nology for labor-intensive and patient-tailored training 
as well as for accurately imposing spatial and tem¬ 
poral constraints within interactive scenarios so as to 
augment patient involvement [64.51]. A recent inter¬ 
national online survey, distributed through professional 
organizations to therapists, has set out a set of impor¬ 
tant robotic rehabilitation device requirements [64.52], 
such as variety of arm movements, biofeedback to 
patients, adjustable resistance, and need for virtual 
activities. Robots offer the advantage of providing ob- 



Fig. 64.2 This machine developed by Professor Ra¬ 
bischong allows the patient in rehabilitation to maintain her 
balance while inciting her muscles to move (after [64.47]) 
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Fig.64.3a-e Gait-training robotic systems in use in clinics; (a) the gait trainer GT-I, developed by Hesse’s group and 
commercialized by Reha-Stim (Germany), (b) the Lokomat, developed by Colombo and colleagues and commercial¬ 
ized by Hocoma AG (Switzerland), (c) AutoAmbulator, developed by the HealthSouth Corp. (USA), (d) LokoStation 
with Lokohelp (Darkov Spa, Czech Republic; after [64.48]), (e) G-EO Evolution (Panos Th. Skoutas S.A., Greece; 
after [64.49]) 


jective measures of user performance, thus enabling 
the development of quantitative kinematic and dynamic 
metrics [64.53-56]. The analysis of brain reorgani¬ 
zation and associated clinical outcomes induced by 
robot-aided therapy in chronic stroke patients have 
shown that motor performance improvement is corre¬ 
lated with changes in inter-hemispheric connectivity 
between primary somatosensory areas: for those pa¬ 
tients the connectivity returns to nearly physiological 
levels [64.57]. 

Protocols based on monitoring patient performance, 
strength, endurance, and emotional state are being in¬ 
vestigated. Anticipating patient movement intentions 
and modeling internal states lead to adaptive patient- 


robot interaction control. This approach paves the way 
to a new generation of robotic therapy devices using 
bio-cooperative controllers, where psychophysiologi- 
cal and biomechanical information is used as feedback 
for updating robot control [64.58-60]. First exam¬ 
ples of biocooperative controllers can be found in 
the robotic platforms developed within the European 
projects ECHORD/MAAT [64.56] (multimodal inter¬ 
faces to improve therapeutic outcomes in robot-assisted 
rehabilitation) (Fig. 64.4) and MIMICS (multimodal 
immersive motion rehabilitation with interactive cog¬ 
nitive systems). Other bio-cooperative controllers have 
been developed by Riener and Munih [64.58], who de¬ 
veloped new control strategies applied to lower limb 
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Fig. 64.4 Set up of the MAAT platform (after [64.61]) 



Fig. 64.5 PARO, an advanced interactive robot developed 
by AIST (after [64.62]) 

rehabilitation protocols that promote active participa¬ 
tion of patients during training. 

Cognitive robotics had started in the early 1980s to 
aid children with communication disorders and phys¬ 
ical impairment to achieve a measure of control of 
their physical space. Using mostly educational manip¬ 
ulators, several demonstration systems were developed. 


In the early 2000s, Latham of Anthrotronix, Inc. com¬ 
mercialized small robot systems to enable children with 
physical disabilities to play games with simple inter¬ 
faces. A bit later, small mobile robots were used in 
clinics by Dautenhahn and Werry [64.63] with chil¬ 
dren who had autism, since robots have such simple 
interfaces that they appear not to be that challeng¬ 
ing as communication with other humans. The early 
2000s also saw the advent of pet robots, such as the 
Paro seal robot developed by Shibata et al. [64.64] 
(Fig. 64.5), as companions for both children and the 
elderly who are confined to clinics and have limited 
real companionship. This topic is further elaborated in 
Chap. 73. 

The applications for robotics continues to increase 
in number as advances in materials, control software, 
higher robustness, and the diminishing size of sen¬ 
sors and actuators allow designers to attempt new ways 
of using mechatronics technology to further the well¬ 
being of people with disabilities. 


64.2 Rehabilitation Therapy and Training Robots 


The human neuromuscular system exhibits use- 
dependent plasticity, which is to say that use alters the 
properties of muscles and neurons, including the pattern 
of their connectivity, and thus their function [64.65- 
67]. For a detailed description of patient populations 
expected to benefit from robot-assisted therapy and 
the most common impairments to be treated, which 
include incoordination, abnormal synergies, spasticity 
and weakness, please see [64.50, 68-70]. 


64.2.1 Grand Challenges and Roadblocks 

The process of neurorehabilitation seeks to exploit 
use-dependent plasticity in order to help people to re¬ 
learn how to move following neuromuscular injuries or 
diseases. Neurorehabilitation is typically provided by 
skilled therapists, including physical, occupational, and 
speech therapists. This process is time-consuming, in¬ 
volving daily, intensive movement practice over many 
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weeks. It is also labor-intensive, requiring hands-on 
assistance from therapists. For some tasks, such as 
teaching a person with poor balance and weak legs to 
walk, this hands-on assistance requires that the ther¬ 
apists have substantial strength and agility to provide 
safe and effective interventions. 

Because neurorehabilitation is time and labor in¬ 
tensive, in recent years healthcare players have put 
limits on the amount of therapy that they will reim¬ 
burse in an effort to contain spiraling healthcare costs. 
Ironically, at the same time, there has been increasing 
scientific evidence that more therapy can in many cases 
increase movement recovery via use-dependent plastic¬ 
ity [64.71]. As robotics and rehabilitation researchers 
began to recognize beginning in the late 1980s, neurore¬ 
habilitation is a logical target for automation because of 
its labor-intensive, mechanical nature, and because the 
amount of recovery is linked with the amount of repe¬ 
tition. Robots could deliver at least the repetitive parts 
of movement therapy at lower cost than human thera¬ 
pists, allowing patients to receive more therapy, recover 
more function, remain independent longer, and reduce 
downstream healthcare expenses. 

The grand challenge for automating movement ther¬ 
apy is determining how to optimize use-dependent 
plasticity. That is, researchers in this field must deter¬ 
mine what the robot should do in cooperation with the 
patient’s own movement attempts in order to maximally 
improve movement ability, all the while engaging the 
patient. Meeting this challenge involves solving two 
key problems: determining appropriate movement tasks 
(what movements should patients practice and what 
feedback should they receive about their performance), 
and determining an appropriate pattern of mechani¬ 
cal input to the patient during these movement tasks 
(in other words, at what magnitude, how often, and 
in which directions, should the robot apply forces to 
the patient’s limb to provoke increased plasticity). The 
prescription of movement tasks and mechanical input 
fundamentally constrains the mechanical and control 
design of the robotic therapy device. 

There are three main roadblocks to achieving the 
grand challenge. The first is a scientific roadblock: 
neither the optimal movement tasks nor the optimal 
mechanical inputs are known. The scientific basis for 
neurorehabilitation remains ill-defined, with competing 
schools of thought. The number of large, randomized, 
controlled trials that have rigorously compared differ¬ 
ent therapy techniques is still small, in part because 
these trials are expensive and difficult to control well. 
Therefore, the first problem that a robotics engineer will 
encounter when setting out to build a robotic therapy 
device is that there is still substantial uncertainty as to 
what exactly the device should do. 


This uncertainty corresponds to an opportunity to 
use robotic therapy devices as scientific tools them¬ 
selves. Robotic therapy devices have the potential to 
help identify what exactly provokes plasticity dur¬ 
ing movement rehabilitation, because they can provide 
well-controlled patterns of therapy. They can also si¬ 
multaneously measure the results of that therapy. Better 
control over therapy delivery and improved quantitative 
assessment of patient improvement are two desirable 
features for clinical trials that have often been lacking in 
the past. Recent work with robotic movement training 
devices is leading, for example, to the characterization 
of computations that underlie motor adaptation, and 
then to strategies for enhancing adaptation based on 
optimization approaches [64.5,72,73]. Early work at 
UBC on the characterization of human balance in quiet 
standing will lead to the ability to use robotics clinically 
to develop new therapies to prevent falls for stroke sur¬ 
vivors and others with balance impairment [64.74,75]. 

The second roadblock is a technological one: 
robotic therapy devices often have as their goal to as¬ 
sist in therapy of many-body degrees of freedom (e.g., 
the arms and torso for reaching, or the pelvis and legs 
for walking). The devices also require a wide dynamic 
bandwidth such that they can, for example, impose a de¬ 
sired movement on a patient who is paralyzed, but also 
fade-to-nothing as the patient recovers. Further, mak¬ 
ing the devices lightweight enough to be wearable is 
desirable, so that the patient can participate in reha¬ 
bilitation in a natural setting (for example, by walking 
over ground or working at a counter in a kitchen), or 
even throughout the course of normal activities of daily 
living. The development of high degree-of-freedom, 
wearable, high-bandwidth robotic exoskeletons is an 
unsolved problem in robotics, although much progress 
has been made toward this goal in the last 10 years, 
as described below. Still, no device at present comes 
close to matching the flexibility of a human therapist, 
in terms of assisting in moving different body degrees 
of freedom in a variety of settings (e.g., walking, reach¬ 
ing, grasping, neck movement), or the intelligence of 
a human therapist, in terms of providing different forms 
of mechanical input (e.g., stretching, assisting, resist¬ 
ing, perturbing) based on a real-time assessment of the 
patient’s response. 

The third roadblock is maintaining the motivation 
and engagement of the patient through the tens of 
thousands of repetitions required to achieve meaning¬ 
ful increases in function [64.50] that will carry over to 
performing actual ADLs. Rehabilitation can be com¬ 
pared to the exercise required by an athlete to compete 
at an elite competition level. By necessity, therapy at 
this intensity will have to move largely to the home set¬ 
ting to be economically viable, and therefore motivation 
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becomes the key ingredient of success. Current work fo¬ 
cuses on developing adaptive therapy through robotics 
and feedback, embedding therapy in computer games, 
music, and sport, and applying motor-learning theo¬ 
ries, for example, the Challenge Point [64.76] model 
of keeping people at a desirable difficulty level, to 
maximize effectiveness and minimize frustration over 
the long term [64.76]. Meeting the grand challenge 
of robotic therapy therefore will require substantial, 
inter-related advances in clinical neuroscience, robot 
engineering, and kinesiology. 

64.2.2 Movement Therapy After Neurologic 
Injury 

Modem, evidence-based medicine relies on objective 
evaluation and quantitative comparative analysis of the 
impact of different therapeutic approaches. Robotics 
technology has the potential to boost evidence-based 
neurorehabilitation: therapy robots provide precise and 
sensitive tools for assessing and modeling human be¬ 
havior, well beyond the capability of a human observer. 
This is of paramount importance for enabling appro¬ 
priate initial diagnosis, early adoption of corrective 
clinical strategies, and for identifying verifiable mile¬ 
stones as well as prognostic indicators of the recovery 
process. 

Bosecker et al. [64.77] tried to evaluate clinical 
scores during upper limb therapy by means of robot- 
based metrics; Zollo et al. [64.54,78] evaluated robotic 
therapy exploiting robotic outcome; thus providing 
quantitative measure of biomechanical and motion 
planning features of arm motor control following upper 
limb rehabilitation. A recent study carried out by Krebs 
et al. showed that robotic measurements of arm move¬ 
ments after upper limb, robot-aided therapy may estab¬ 
lish biomarkers for motor recovery [64.79]. Similarly, 
Domingo and Lam [64.80] tried to apply robot-based 
assessment to the lower limb by using the Lokomat sys¬ 
tem and customized software. 

Another important advantage of therapy robots is 
the possibility that a single operator can effectively 
supervise multiple patients, locally or even remotely, 
i. e., in a telerehabilitation scenario [64.81]. Therapy 
robots can potentially improve patients’ access to reha¬ 
bilitation by providing the opportunity to increase the 
duration and the frequency of their therapy experience, 
with limitations mainly depending on clinical consider¬ 
ations rather than on other organizational or economic 
constraints. 

At present, much of the activity in physical ther¬ 
apy and training robots has been focused on retraining 
movement ability for individuals who have had a stroke 
or SCI, or are affected by CP. The main reasons 


for this emphasis are that there are a relatively large 
number of patients with these conditions, the rehabil¬ 
itation costs associated with them tire high, and because 
these patients can sometimes experience large gains 
with intensive rehabilitation because of use-dependent 
plasticity. 

A stroke refers to an obstruction or breakage of 
a blood vessel supplying oxygen and nutrients to the 
brain. Approximately 800000 people suffer a stroke 
each year in the United States, and about 80% of these 
people experience acute movement deficits [64.82]. In 
Europe, 1 100000 people suffered a stroke in 2000, 
and they are expected to increase up to 1 500 000 in 
2025. There are over 3 000000 survivors of stroke 
in the United States [64.83], with over half of these 
individuals experiencing persistent, disabling, move¬ 
ment impairments. Data are similar to those of Europe, 
where it has been estimated that between half and 
two-thirds of people survive stroke. Of these, half do 
not recover fully, and a quarter need assistance in 
daily living. While most of the survivors of stroke 
are elderly persons, motor impairment from a stroke 
affects 3000 of the 10000 children born with CP 
each year [64.83]. While this is a small proportion 
of all stroke survivors, the motor function impair¬ 
ment will persist throughout the person’s entire life, 
so the impact on independence is disproportionately 
large. 

The number of people who have experienced and 
survived a stroke is expected to increase substantially 
in the United States and other industrialized countries 
in the next two decades, mostly because age is a risk 
factor for stroke and the mean age of people in indus¬ 
trialized countries is rapidly increasing due to the baby 
boom of the 1950s, but also because improved acute 
treatments will allow more people to survive a stroke, 
albeit perhaps with impairment. 

The number of people who experience an SCI in 
the United States each year is relatively smaller - about 
15 000, with about 200000 people alive who have sur¬ 
vived an SCI - but the consequences can be even more 
costly than stroke [64.82]. In Europe, there are about 
11 000 new cases of SCI per year, and about 330 000 
have survived an SCI [64.84]. The most common causes 
of SCI are automobile accidents and falls. These acci¬ 
dents crush the spinal column and contuse the spinal 
cord, damaging, or destroying neurons within the spinal 
cord. Lower limb robotic therapy has demonstrated 
promising results when applied to SCI patients with 
incomplete lesions [64.85] although large, systematic, 
and randomized controlled trials are still missing. It is 
important to note that SCI patients are typically younger 
than stroke patients, which may affect their familiarity 
with and acceptance of technical aids. 
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64.2.3 Robotic Therapy 

for the Upper Extremity 

This section first describes early, clinically tested upper 
limb therapy robot systems (Fig. 64.6a-e), then more 
recent systems. 

MIT-MANUS 

The first robotic therapy device to undergo extensive 
clinical testing, and, now, to achieve some commercial 
success, is the MIT-MANUS, sold as the InMotion2 by 
Interactive Motion, Inc. [64.86]. MIT-MANUS is a pla¬ 
nar two-joint arm that makes use of the SCARA con¬ 
figuration, allowing two large, mechanically grounded 
motors to drive a lightweight linkage. The patient sits 
across from the device, with the weaker hand attached 
to the end-effector, and the arm supported on a table 
with a low-friction support. By virtue of the use of the 
SCARA configuration, the MIT-MANUS is perhaps the 
simplest possible mechanical design that allows planar 
movements while also allowing a large range of forces 


to be applied to the arm without requiring force feed¬ 
back control. 

MIT-MANUS assists the patient in moving the arm 
across the table-top as the patient plays simple video 
games, such as moving a cursor into a target that 
changes locations on a computer screen. Assistance is 
achieved using a position controller with an adjustable 
impedance. Additional modules have been developed 
for the device for allowing vertical motion [64.87], 
wrist motion [64.88], and hand grasp [64.89]. Software 
has been developed for providing graded resistance as 
well as assistance to movement [64.90], and for varying 
the firmness and timing of assistance based on real-time 
measurements of the patient’s performance on the video 
games [64.91]. 

MIT-MANUS has undergone extensive clinical test¬ 
ing in several studies, summarized as follows. The first 
clinical test of the device compared the motor recov¬ 
ery of acute stroke patients who received an additional 
dose of robot therapy on top of their conventional 
therapy, to that of a control group, who received con- 



Fig.64.6a-e Arm-therapy robotic systems that have undergone extensive clinical testing; (a) MIT-MANUS, developed 
by Hogan, Krebs, and colleagues at the Massachusetts Institute of Technology (USA), (b) MIME, developed at the 
Department of Veterans Affairs in Palo Alto in collaboration with Stanford University (USA), (c) GENTLE/s developed 
in the EU, (d) ARM-Guide, developed at the Rehabilitation Institute of Chicago and the University of California, Irvine 
(USA), and (e) Bi-Manu-Track, developed by Reha-Stim (Germany) 
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ventional therapy and a brief, sham exposure to the 
robot [64.92]. The robot group patients received the ad¬ 
ditional robotic therapy for an hour each day, five days 
per week, for several weeks. The robot group recov¬ 
ered more arm movement ability than the control group 
according to clinical scales, without any increase in ad¬ 
verse effects such as shoulder pain. The improvements 
might subjectively be characterized as small but some¬ 
what meaningful to the patient. The improvements were 
sustained at a three-year follow-up. 

This first study with MIT-MANUS demonstrated 
that acute stroke patients who received more therapy re¬ 
cover better, and that this extra therapy can be delivered 
by a robotic device. It did not answer the question as to 
whether the robotic features of the robotic device were 
necessary. In other words, it may have been that pa¬ 
tients would have also improved their movement ability 
if they had practiced additional movements with MIT- 
MANUS with the motor’s off (thus making it equivalent 
to a computer mouse), simply by virtue of the increased 
dose of movement practice stimulating use-dependent 
plasticity. Thus, while this study indicated the promise 
of robots for rehabilitation therapy, it did not close the 
gap of knowledge as to how external mechanical forces 
provoke use-dependent plasticity. 

Subsequent studies with MIT-MANUS have con¬ 
firmed that robotic therapy can also benefit chronic 
stroke patients [64.93]. The device has been used 
to analyze different types of therapies, for exam¬ 
ple, to compare assisting movement versus resisting 
movement in chronic stroke subjects, but with in¬ 
conclusive results: both types of therapies produced 
benefits [64.90]. The device has also been used to 
compare assistive robot therapy with another techno¬ 
logical approach to rehabilitation - electrical stim¬ 
ulation of finger and wrist muscles [64.94]. Again, 
significant benefits were found for both therapies, and 
those benefits were specific to the movements prac¬ 
ticed, but the benefits were not significantly different 
between therapies. We note that the lack of a sig¬ 
nificant difference in these studies may simply be 
due to the limited number of patients who partici¬ 
pated in these studies (i. e., inadequate study power), 
rather than a close similarity of the effectiveness of the 
therapies. 

As mentioned above, the MIT-MANUS device was 
recently tested in a multisite clinical trial funded by 
the Department of Veterans Affairs in the United 
States [64.50]. This study compared clinical outcomes 
in chronic stroke survivors who were randomized to 
three groups: usual care, robot-assisted therapy with 
arm, wrist, and hand modules, and one-on-one therapy 
with a rehabilitation therapist that was dose matched to 
the robotic therapy in terms of the number of move¬ 


ments achieved per therapy session. The robot-assisted 
therapy group improved their movement ability more 
than the usual care, and about the same as the dose- 
matched group. This is an important finding for the 
field, as it demonstrated with the highest scientific rigor 
that robotic therapy was about as effective as an intense, 
therapist-delivered therapy. 

A cost-benefit analysis of this study suggested that 
although both robotic and intense, therapist-based ther¬ 
apies were more expensive to deliver than usual care, it 
reduced long-term follow-up costs, so that the total cost 
of care of the patients was the same over the duration of 
the study [64.95]. Thus, as the cost of robotic therapy 
devices decreases, it should be possible to provide pa¬ 
tients with improved outcomes while reducing the cost 
of therapy - another important finding. 

MIME 

Another early system to undergo clinical testing was 
the MIME (mirror-image movement enhancer) system, 
which used a Puma-560 robot arm to assist in move¬ 
ment of the patient’s arm [64.96]. The device is attached 
to the hand through a customized splint and a connec¬ 
tor that is designed to break away if interaction forces 
become too large. Compared to MIT-MANUS, the de¬ 
vice allows more naturalistic motion of the arm because 
of its six degrees of freedom (DOF), but must rely on 
force feedback so that the patient can drive the robot 
arm. Four control modes were developed for MIME. 
In the passive mode, the patient relaxes and the robot 
moves the arm through a desired pattern. In the ac¬ 
tive assist mode, the patient initiates a reach toward 
a target, indicated by physical cones on a table top, 
which then triggers a smooth movement of the robot 
toward the target. In the active-constrained mode, the 
device acts as a sort of virtual ratchet, allowing move¬ 
ment toward the target, but preventing the patient from 
moving away from the target. Finally, in the mirror- 
image mode, the motion of the patient’s less impaired 
arm is measured with a digitizing linkage, and the 
impaired arm is controlled to follow along in a mirror- 
symmetric path. The initial clinical test of MIME found 
that chronic stroke patients who received therapy with 
the device improved their movement ability about as 
much as patients who received conventional table-top 
exercises with an occupational therapist [64.96]. The 
robot group even surpassed the gains from human- 
delivered therapy for the outcome measures of reaching 
the range of motion and strength at key joints of the 
arm. A follow-on study attempted to elucidate which of 
the control modes or what combination of MIME ex¬ 
ercises caused the gains, but was inconclusive [64.97]. 
A multisite randomized control of MIME again funded 
by the Department of Veterans Affairs compared the 



Rehabilitation and Health Care Robotics I 64.2 Rehabilitation Therapy and Training Robots 1697 


effect of a high-dose (30 h) of additional therapy with 
MIME, to a lower dose (15 h), to 15 h of additional con¬ 
ventional therapy in 54 acute stroke patients [64.98]. 
Gains in the primary outcome measure, the Fugl-Meyer 
assessment, were not significantly different between the 
groups at the 6-month follow-up, although there the ac¬ 
tual dose of robotic therapy patients received predicted 
their recovery. 

ARM Guide 

The question of the effect of robot forces on movement 
recovery was also left unresolved by a study with an¬ 
other device, the ARM Guide, which is a trombone-like 
device that can be oriented then locked in different di¬ 
rections, and can assist people in reaching in a straight 
line. Chronic stroke patients who received assistance 
during reaching with the robot improved their move¬ 
ment ability [64.99]. However, they improved about 
as much as a control group that simply practiced 
a matched number of reaches without assistance from 
the robot. This suggests that movement effort by the 
patient is a key factor for recovery, although the small 
sample size of this study limited the ability to resolve 
the size of the difference between guided and unguided 
therapies. 

Bi-Manu-Tracl< 

Perhaps the most striking clinical results generated 
so far have come from one of the simplest devices 
built. Similar to a design proposed previously by Lum 
et al. [64.39], the Bi-Manu-Track uses two motors, one 
for each hand, to allow bimanual wrist-flexion exten¬ 
sion [64.100]. The device can also assist in forearm 
pronation/supination if it is tilted downward and the 
handles are changed. In an extensive clinical test of 
the device, 22 subacute patients (i. e., 4—6 weeks af¬ 
ter stroke) practiced 800 movements with the device 
for 20 minutes per day, five days per week for six 
weeks [64.100]. For half the movements, the device 
drove both arms, and for the other half, the patient’s 
stronger arm drove the motion of the more impaired 
arm. A control group received a matched duration 
of electrical stimulation (ES) of their wrist extensor 
muscles, with the stimulation triggered by voluntary ac¬ 
tivation of their muscles when possible, as measured 
by electromyography (EMG). The number of move¬ 
ments performed with EMG-triggered ES was 60-80 
per session. The robot-trained group improved by 15 
points more on the Fugl-Meyer scale, a standard clini¬ 
cal scale of movement ability with a range from 0 to 66 
points in upper extremity function. It assigns a score 
of 0 (cannot complete), 1 (completes partially), or 2 
(completes normally) for 33 test movements, such as 
lifting the arm without flexing the elbow. For compar¬ 


ison, reported gains in Fugl-Meyer score after therapy 
with the MIT-MANUS and MIME devices ranged from 
0 to 5points [64.101]. 

Other Early Devices to Undergo Clinical Testing 
Other early devices to undergo clinical testing are as 
follows. The GENTFE/s system uses a commercial 
robot, the HapticMaster, to assist in patient movement 
as the patient plays video games. The HapticMas¬ 
ter allows four degrees of freedom of movement and 
achieves a high bandwidth of force control using force 
feedback. Chronic stroke patients who exercised with 
GENTFE/s improved their movement ability [64.102, 
103], The Rutgers ARM robotic device uses low- 
friction pneumatic cylinders to help extend or flex the 
fingers, and has been shown to improve hand move¬ 
ment ability of chronic stroke subjects [64.104]. Simple 
force-feedback controlled devices, including a 1-DOF 
wrist manipulator and a 2-DOF elbow-shoulder ma¬ 
nipulator, were also recently shown to improve move¬ 
ment ability of chronic stroke subjects who exercised 
with the devices [64.105]. A passive exoskeleton, the 
T-WREX arm orthosis, provides support to the arm 
against gravity using elastic bands, while still allow¬ 
ing a large range of motion of the arm [64.106]. By 
incorporating a simple hand-grasp sensor, this device 
allows substantially weakened patients to practice sim¬ 
ple virtual reality exercises that simulate functional 
tasks such as shopping and cooking. Chronic stroke 
patients who practiced exercising with this nonrobotic 
device recovered significant amounts of movement abil¬ 
ity, comparable with the Fugl-Meyer gains seen with 
MIT-MANUS and MIME. A randomized controlled 
trial of T-WREX in 28 chronic stroke subjects com¬ 
pared therapy with the device to conventional, table-top 
therapy [64.107] and found that the device was slightly 
more effective in improving patient’s motor ability at 
the six-month follow-up according to the primary out¬ 
come measure, the Fugl-Meyer score, and that patients 
strongly preferred exercising with the device. T-WREX 
has been commercialized as the Armeo Spring upper 
extremity training device by Hocoma, and was in use 
in over 500 clinics as of early 2014, with published 
reports studying its use for individuals with multiple 
sclerosis [64.108], SCI [64.109], and proximal humeral 
fracture [64.110]. NeReBot is a 3-DOF wire-based 
robot that can slowly move a stroke patient’s arm in spa¬ 
tial paths. Acute stroke patients who received additional 
movement therapy beyond their conventional rehabil¬ 
itation therapy with NeReBot recovered significantly 
more movement ability than patients who received just 
conventional rehabilitation therapy [64.111]. RehaRob 
uses an industrial robot arm to mobilize patients’ arms 
along arbitrary trajectories following stroke [64.1 12]. 
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Further Research and Developments 

on Robotic Therapy for the Upper Extremity 
One recent trend in the field is to develop devices 
that can provide therapy for a larger number of de¬ 
grees of freedom of the arm. For example, at the high 
end of cost and complexity are the ARMin [64.113], 
Pneu-WREX [64.114], and BONES [64.115], which 
are exoskeletons that accommodate nearly naturalis¬ 
tic movement of the arm while still achieving a wide 
range of force control. A system that couples a im¬ 
mersive virtual reality display with a haptic robot 
arm is described in [64.116]. A wearable exoskeleton 
driven by pneumatic muscles is described in [64.117]. 
Several other exoskeletal machines have been pro¬ 
posed to jointly train arm and wrist, such as the 
CADEN-7, an anthropometric 7-DOF powered ex¬ 
oskeleton system [64.118]; the SUEFEL-7, which 
exploits EMG signals to adjust impedance param¬ 
eters [64.119]; a lightweight exoskeleton described 
in [64.120], and the highly redundant 9-DOFs machine 
introduced in [64.121]. 

Another area of active development in robotic ther¬ 
apy devices for the upper extremity is to devices that 
can be used at home or in rank-and-filed clinics. For 
example, at the lower end of cost/complexity are de¬ 
vices that use force feedback joysticks and steering 
wheels with a view toward implementation in the 
home [64.114—125]. In 2008, a planar machine for up¬ 
per limb rehabilitation designed for delivering at-home 
neurorehabilitation, namely CBM MOTUS, was devel¬ 
oped and patented [64.126] (Fig. 64.7). Such a device 
has low inertia and highly isotropic behavior. Recently, 
a passive module has been developed to be installed 
on the CBM-MOTUS, able to further reduce robot 
perceived inertia when the machine is moved by the pa¬ 
tient, while being highly rigid when the machine assists 
the patient’s movements. 

As far as wearable robotic systems for the upper 
limb are concerned, Hocoma engineered and com¬ 


mercialized the ARMEO power system, and also the 
ArmeoSpring and ArmeoBoom machines [64.108]. 

A third area in robotic therapy devices for the 
upper extremity that has received increased attention 
is to develop devices for hand rehabilitation, since 
hand movement ability is essential for functional re¬ 
covery. Examples of recent, novel robotic devices 
for the hand are the Haptic Knob for grasping and 
wrist pronation/supination [64.128]; the HandCARE, 
a cable-actuated rehabilitation system [64.129]; a de¬ 
vice for repeating controlled passive movements of 
paralyzed fingers given in [64.130]; the exoskeletal ma¬ 
chines HWARD [64.131], and HEXORR [64.132] to 
help open and close the hand; the 18-DOFs highly 
redundant Gifu Haptic Interface [64.133], able to pro¬ 
duce adduction-abduction and flexion-extension finger 
movements. Other devices are presented in [64.134— 
138], and a review of robotic therapy of the hand 
is given in [64.139]. One robotic therapy system for 
the hand incorporates the idea of using visual feed¬ 
back distortion to enhance motivation of patients during 
movement therapy [64.140]. A few commercial ma¬ 
chines are available, such as the Reha-Digit, Amadeo, 
and ManovoSpring. 

Based on the availability of modular hand, wrist, 
and shoulder-elbow therapeutic robotic devices, some 
recent studies tried to tackle the fundamental question 
of whether proximal or distal treatment differentially 
affect the recovery of arm/hand function. For more 
information of these preliminary studies, see refer¬ 
ences [64.100, 141, 142]. 

An important recent development in robotic ther¬ 
apy devices is the development of devices that can be 
used in conjunction with the instrumentation needed to 
measure neurophysiological signals, such as functional 
magnetic resonance imaging (fMRI)-compatible sys¬ 
tems [64.143, 144] or, more generally, brain-imaging 
(Bl)-compatible robotic systems that can be used in 
conjunction not only with fMRI but also with magne- 



Fig.64.7a,b CBM Motus robot for upper limb rehabilitation (after [64.127]) 
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toencephalography (MEG), transcranial magnetic stim¬ 
ulation (TMS), repetitive TMS (rTMS), transcranial 
direct current stimulation (tDCS), near infrared spec¬ 
troscopy (NIRS) and other brain imaging and stimula¬ 
tion equipment. Such devices are notable because they 
will allow a systematic scientific study of neural recov¬ 
ery during robotic therapy. Developing these devices 
requires dealing with the problems of electromagnetic 
compatibility and interaction. 

Recently, a few fMRI-compatible devices for hand 
rehabilitation have been designed to acquire functional 
imaging while performing therapeutic exercises, such 
as the exoskeletal machine in [64.145]; the pneu¬ 
matic actuated 2-DOF device to help wrist prona¬ 
tion/supination and hand open/closure in [64.146]; ma¬ 
chines driven by electro-rheological fluids, able to pro¬ 
vide a variable resistance to patient motion [64.147]; 
the planar device to help move fingers described 
in [64.148]. A passive fMRI-compatible manipulan- 
dum morphologically similar to MIT-MANUS has 
been developed and tested to acquire functional imag¬ 
ing in [64.149]. Another study proposed the appli¬ 
cations of a brain-computer interface (BCI) therapy 
approach on a cohort of stroke patients, [64.150]; 
a magnetoencephalography-based BCI system is used 
which in turn raised or lowered a screen cursor in the 
direction of a target. Results suggest that volitional con¬ 
trol of neuromagnetic activity features recorded over 
central scalp regions can be achieved with BCI training 
after stroke. 

Finally, the primary paradigm that has been tested 
so far with upper extremity robotics is to assist patients 
in moving, a strategy which may in some cases have the 
unintended affect of causing patients to slack [64.151]. 
In general, the field is still relatively undeveloped in 
its ability to identifying the most appropriate forms of 
robotic intervention given the nature of the impairment 
and the patient. For example, an approach opposite to 
physical assistance, which is using robotic force fields 
to amplify the kinematic errors of stroke patients during 
reaching, may provoke novel forms of the adaptation of 
those patterns [64.4, 152]. A major emphasis of the field 
in the next 10 years will be improving the mechanis¬ 
tic understanding of how robotic interaction influences 
brain plasticity. 

64.2.4 Robotic Therapy for Walking 

Scientific evidence that gait training improves the re¬ 
covery of mobility after neurologic injury started to 
accumulate in the 1980s through studies with cats. 
Cats with SCI can be trained to step with their hind 
limbs on a treadmill with the partial support of body 
weight and assistance of leg movements [64.153, 154]. 


Following the animal studies, various laboratories de¬ 
veloped a rehabilitation approach in which the patient 
steps on a treadmill with the body weight partially 
supported by an overhead harness and assistance from 
therapists [64. 155-158]. Depending on the patient’s im¬ 
pairment level, from one to three therapists are needed 
for BWSTT (body-weight supported treadmill train¬ 
ing), with one therapist assisting in stabilizing and 
moving the pelvis, while two additional therapists sit 
next to the treadmill and assist the patient’s legs in 
swing and stance. This type of training is based on the 
principle of generating normative, locomotor-like sen¬ 
sory input that promotes the functional reorganization 
and recovery of the injured neural circuitry [64.159]. 
In the 1990s, several independent studies indicated that 
BWSTT improves stepping in people with SCI or hemi¬ 
plegia after stroke [64.155-157]. 

Gait training is particularly labor-intensive and 
strenuous for therapists, so it is an important target for 
automation. The efforts of roboticists have been es¬ 
pecially focused on BWSTT rather than over-ground 
gait training because BWSTT is done on a station¬ 
ary setup in a well-defined manner and thus can be 
more easily automated than over-ground gait train¬ 
ing. Randomized, controlled clinical trials have shown 
that BWSTT is comparable in effectiveness to con¬ 
ventional physical therapy for various gait-impairing 
diseases [64.160-166]. These trials support the efforts 
toward the automation of BWSTT, as the working 
conditions of physical therapists will improve if the 
robots do much of the physical work, which, in the 
case of BWSTT, actually leads to occasional back in¬ 
juries to therapists. Usually, only one therapist is needed 
in robot-assisted training, for the tasks of helping the 
patient into and out of the robot and monitoring the ther¬ 
apy. In the case of SCI patients, a small randomized, 
controlled trial [64.161] reported that robotic-assisted 
BWSTT with a first-generation robot required signif¬ 
icantly less labor than both conventional overground 
training and therapist-assisted BWSTT, with no signif¬ 
icant difference found in effectiveness. 

Gait-Training Robots in Current Clinical Use 
Some gait-training robot systems are commercially 
available and are used for therapy in several clin¬ 
ics worldwide, such as the gait trainer GT-I [64.46], 
the Fokomat [64.45] the ReoAmbulator, the Foko- 
Help [64.167], and the G-EO System [64.168] 
(Fig. 64.3a-e). 

Of these robots, the GT-I (commercialized by Reha- 
Stim) is the one that departs most from therapist- 
assisted BWSTT, since it interacts with the patient’s 
lower limbs through two footplates rather than act¬ 
ing on the shank as human therapists do. It also ap- 
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pears to depart more from natural walking because 
the footplate principle substantially alters the sensory 
cues of the foot impact with the ground or treadmill 
band. The GT-I footplates are driven by a singly ac¬ 
tuated mechanism that moves the foot along a fixed 
gait-like trajectory with a doubled crank and rocker sys¬ 
tem [64.46]. The stride length can be adjusted between 
sessions by changing gears. The body weight is un¬ 
loaded as needed by an overhead harness. The torso 
is moved sagittally in a phase-dependent manner by 
ropes attached to the harness and connected by another 
crank to the foot crank. The GT-I is currently installed 
in dozens of clinics, mainly in Europe. One random¬ 
ized, controlled study has been reported that tested the 
GT-I with 30 subacute stroke patients [64.169]. The 
robot group improved their overground walking abil¬ 
ity more than the control group, although differences 
were not significant at a 6-month follow-up. A total of 
80% of the patients said they preferred training with the 
robot rather than the therapists because training with 
the robot was less demanding and more comfortable. 
The other 20% of patients stated that swinging of the 
paretic limb seemed less natural and thus less effective 
when training with the robot. Robot-assisted training re¬ 
quired an average of one therapist per patient, while 
therapist-assisted training required two therapists per 
patient on average. A follow-up, randomized controlled 
study comparing conventional training plus robotic 
training with the GTI, to a time-matched amount of con¬ 
ventional training alone with subacute stroke patients, 
found that the group that received some robotic train¬ 
ing recovery walking ability to a great extent [64.170]. 
More recent clinical tests with the GT I are reported 
in [64.171], 

The Lokomat (commercialized by Hocoma) is 
a robotic exoskeleton worn by the patients during 
treadmill walking [64.45]. Four motorized joints (two 
per leg) move hip and knee. The actuators consist 
of ball screws connected to dc motors. The legs are 
driven in a gait-like pattern along a fixed position- 
controlled trajectory. The device attaches to the thighs 
and shanks through padded straps. A parallelogram 
mechanism allows the vertical translation of the pa¬ 
tient’s torso, restricting lateral translation. The patient’s 
body weight is actively unloaded as needed through 
an overhead harness. The Lokomat is currently be¬ 
ing used in over 100 clinics worldwide. In 2005, Wirz 
et al. [64.85] reported preliminary results of robot- 
assisted BWSTT with the Lokomat in 20 chronic 
incomplete SCI patients. The improvements in over¬ 
ground walking speed and endurance were statistically 
significant: approximately 50% gain on average in 
the 16 patients who were ambulatory before training. 
There were no significant changes in the requirement 


of walking aids, orthoses, or external physical assis¬ 
tance. The improvements appear to be comparable 
to those achieved by similar SCI patients who re¬ 
ceived therapist-assisted BWSTT [64.161,172]). For 
the case of stroke patients, however, therapy with the 
Lokomat was beneficial but about half as effective as 
treadmill-based or therapist-based training in improv¬ 
ing overground gait velocity and endurance [64.173, 
174], 

The ReoAmbulator (commercialized by Motorika 
and marketed in the United States as AutoAmbulator) 
consists of two robotic arms that assist patients to step 
on a treadmill with their body weight supported as 
needed. The interface to the patient’s legs is through 
straps at the thigh and ankle. The ReoAmbulator is 
currently being used in at least 57 HealthSouth reha¬ 
bilitation centers, all of them in the United States, but 
little data have been published concerning its use. 

The LokoHelp (commercialized by LokoHelp 
Group) assists users’ feet motion along physiological 
trajectories while walking on a treadmill, also providing 
body weight support. Clinical trials [64.167, 171, 175] 
show that therapeutic outcomes are similar to manual 
training with reduced therapist effort. 

The G-EO system (commercialized by Reha Tech¬ 
nology) is also based on the footplate principle as for 
the GT I system. Studies with stroke survivors [64.176], 
patients with SCI [64.177] and with Parkinson’s dis¬ 
ease [64.178] have recently shown the value of this 
particular device. 

Further Research and Developments 
on Robotic Therapy for Walking 
Several groups worldwide are working toward improv¬ 
ing gait-training robotic technologies. A great deal of 
effort has been made to incorporate and investigate the 
ability to assist as needed [64.72, 179-183], that is, 
the ability of the robot to let the patients contribute to 
the locomotor efforts as much as they are able. This 
is likely essential for maximizing locomotor plastic¬ 
ity [64.184]. Some effort has also been directed toward 
adding more active DOFs, particularly for torso manip¬ 
ulation [64.182,185]. These robotic tools are needed 
not only for their potential clinical use in therapy, but 
for studying what aspects of the assistance are impor¬ 
tant for effective gait training and how best to control 
and implement them with robotic devices. 

The team responsible for the GT-1 has developed 
the Haptic Walker [64.179], which maintains the per¬ 
manent foot/machine contact but allows the footplates 
to move along 3-DOF trajectories. In addition, it incor¬ 
porates force feedback and compliance control, as well 
as haptic simulation of ground conditions (e.g., stair 
climbing). 
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An advanced version of the Lokomat integrates 
force sensors and automatic adaptation of gait patterns 
to allow for a reduction of the interaction effort between 
patient persons and machine [64. 1 80]. It has been tested 
on unimpaired persons and SCI patients, who were able 
to influence the gait trajectories toward a more desired 
motion by means of their own motor activity [64.180, 
186], 

PAM is a 5-DOF robot for torso manipulation, and 
POGO is a leg robot with 2-DOF per leg. PAM’s and 
POGO’s actuators are pneumatic, which cost less than 
electric motors and have higher power-to-weight ra¬ 
tios [64.185]. The robots’ ability to control forces and 
yield to patients and/or therapists has been tested with 
unimpaired and SCI participants [64.187]. Of particular 
note here is the development of an adaptive synchro¬ 
nization algorithm that allows these compliant robots to 
provide assistance at the right time as the participants 
varies the timing or size of steps. 

Based on the string-puppet principle, the String- 
Man achieves weight bearing and compliant 6-DOF 
torso manipulation by means of seven wires and a force 
sensor on each wire [64.182]. In addition, a control 
scheme has been designed for the String-Man to control 
both the zero-moment-point location and the ground re¬ 
action force with help of foot force sensors. 

Veneman et al. [64.189] developed the LOPES ex¬ 
oskeleton (lower extremity powered exoskeleton) visi¬ 
ble in Fig. 64.8. In this system, two horizontal pelvis 
translations are actuated, while the vertical motion is 
left free and weight is compensated. Furthermore, the 
LOPES have three actuated rotational joints per leg: two 
at the hip and one at the knee. The actuation system 
for hip and knee flexion/extension [64.183] combines 
Bowden cables with series elastic actuation. The Bow¬ 


den cables allow the motors to be mounted remotely in 
a fixed position, thus reducing the mass to be moved 
on the exoskeleton links. The spring element connect¬ 
ing the Bowden cables with the joint allows the closing 
of a torque feedback control loop with a position sensor 
that measures the spring elongation, a concept inspired 
from the series elastic actuators (SEAs) described by 
Pratt and coworkers [64.190]. 

Veneman et al.’s experimental results on LOPES 
SEAs show that adequate torque control bandwidth 
was achieved by the prototype of their Bowden-cable- 
based actuation design [64.183], so that the robot 
can execute both a stiff, position-dominated robot-in- 
charge mode and a compliant, low-impedance patient- 
in-charge mode. Some results on the testing of the 
LOPES exoskeleton are presented in [64.192-194]. 

The FET European project Evolving Morphologies 
for Human-Robot Symbiotic Interaction (EVRYON) 
investigated a novel approach for the design of wearable 
robots in which robot morphology and control are co¬ 
evolved in a physics-based simulation environment to 
achieve a symbiotic interaction, with useful behaviors 
(walking patterns) emerging from the dynamic interac¬ 
tion between the robot and the human body. To narrow 
the search space, an atlas of topologies of robot archi¬ 
tectures assisting hip and knee flexion/extension was 
produced [64.195], showing that only ten topologies 
are capable of providing independent assistance to hip 
and knee if the number of robot links is not higher 
than 4 and only revolute joints are considered. The 
kinematic structures (morphologies) descending from 
such topologies do not require the alignment of robot 
and human joints (nonanthropomorphism), thus possi¬ 
bly shortening calibration time and limiting wearability 
issues. Among the abovementioned ten topologies, only 



Fig.64.8a,b LOPES (lower-extremity 
powered exoskeleton) device for gait 
training and assessment of motor 
function in stroke survivors (af¬ 
ter [64.188]) 
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Fig.64.9a,b EVRYON/LENAR. 

(a) Wearable robot to assist hip and 

(b) knee flexion/extension through 
series elastic actuators (after [64.191]) 


one allows us to mechanically (intrinsically) minimize 
unwanted shear forces while keeping encumbrance low. 
A morphology belonging to this topology class was 
optimized to minimize reaction forces on the human 
body [64.196] and to maximize wearability and back- 
drivability [64.197]. Based on this study, the lower ex¬ 
tremity nonanthropomorphic robot (LENAR) [64.191] 
was developed (Fig. 64.9), incorporating custom-made 
series-elastic actuators for a robust interaction con¬ 
trol [64.198], 

A different approach to gait training was taken with 
the KineAssist device [64.199]. KineAssist (Fig. 64.10) 
is a motorized mobile platform that follows the patient 
and therapist as they move overground and incorporates 
a smart brace that compliantly supports the patient’s 
trunk and pelvis. This smart support is designed to al¬ 
low the therapist to adjust its stiffness from fully rigid 
down to fully compliant. Within a safety zone, the fully 
compliant mode allows patients to challenge the limits 
of their stability. A compliant virtual wall catches the 
patients when they lose balance. The location of this 
virtual wall is also adjustable. The body weight can be 
unloaded as needed. The main advantage of this system 
is the possibility for the therapist to work in close con¬ 
tact with the patient while cooperating with the robotic 
system, which deals with the crucial, basic task to keep 
the patient stable and safe. From this research plat¬ 
form, HDT Robotics began commercialization of the 
KineAssist-MX. Actuation and sensors allow interac¬ 
tive force-held environments so that a wide variety of 


challenging mobility experiences can be delivered to 
the user. 

Other efforts include Ferris and cowork¬ 
ers [64.202], who are developing foot, ankle, knee, and 
hip orthoses actuated by artificial pneumatic muscles 
that may possibly be used to assist in gait training. 
The Rutgers Ankle is a 6-DOF pneumatic system 
based on a Stewart Platform that allows exercise of the 
ankle [64.203]. Also in the United States, Agrawal’s 
group proposes the use of gravity-balancing leg or¬ 
thoses for people with gait impairments to practice 
walking [64.204]. Their designs allow the orthoses 
to passively support the gravity torque required at 
the patient’s joints. This approach would have the 
advantage of being safer than powerful robots for 
clinical use. They have also extended their design 


Fig. 64.10 KINE 
ASSIST for 
unobtrusive sup¬ 
port to patients, 
allowing them 
to walk on their 
own, or with 
variable levels 
of support (af¬ 
ter [64.200]) 
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Fig. 64.11 Active 
leg exoskele¬ 
ton (ALEX) 
to supplement 
traditional re¬ 
habilitation 
therapy (af¬ 
ter [64.201]) 


to include actuators with reduced torque require¬ 
ments [64.205]. A robot has been used to provide 
graded body weight support as a patient who cannot 
bear full body weight because of a medical problem 


walks in a circle [64.206]. Banala et al. developed the 
treadmill-based rehabilitation robot ALEX [64.207], 
as seen in Fig. 64.11. In this system, hip and knee 
joints are actuated in the sagittal plane while hip ab¬ 
duction/adduction and ankle motion are spring-loaded. 
More details and further examples of robotic systems 
for walking therapy are reported in [64.208]. 

Other Robotic Movement Therapy Approaches 
As reviewed earlier, most of the work to date in robotic 
therapy devices has focused on robots that are attached 
to patients to assist them in practicing reaching or walk¬ 
ing exercises. Other early proposals for using robots 
for movement therapy included using two planar robot 
arms to carefully control continuous passive motion 
of the knee following joint surgery [64.37], and using 
a multiaxis robot arm to place targets for patients do¬ 
ing reaching exercises [64.209]. An emerging approach 
toward robotic movement therapy is to provide the ther¬ 
apy at a distance, in a form of telerehabilitation, in 
order to improve accessibility to the therapy [64.81, 
123,210]. Non-contacting, socially assistive robots, as 
reviewed in Chap. 73, may play an important role in 
motivating and monitoring therapy. 


64.3 Aids for People with Disabilities 


Enabling technologies assist people with disabilities to 
achieve a quality of life on a par with able-bodied in¬ 
dividuals through increased functional independence. 
The main issue with most such technologies is that dis¬ 
ability has a highly individualized impact: a solution for 
one person will not work for someone else, even if their 
disabilities appear clinically similar. 

64.3.1 Grand Challenges 

and Enabling Technologies 

The more a disability impacts function, the more costly 
the technical intervention tends to be, since the con¬ 
sumer market cannot benefit from economies of scale 
if each solution must be individualized. As an extreme 
example, an electric wheelchair with individualized 
padding, motorized recliner, and customized joystick 
control costs as much as a mass-produced mid-sized 
automobile, but has a fraction of the electronics, ro¬ 
bustness, and functions. A grand challenge for assistive, 
enabling technologies is to find a means to make mass- 
personalization possible, as it has been in the automo¬ 
tive industry, for example. One component is designer 
focus. If we can re-badge assistive technology as de¬ 
sign for well-being products, the change in focus from 


fixing people to improving their quality of life will 
have the effect of mainstreaming disability itself so that 
manufacturers of consumer equipment tend to develop 
products that can explicitly accommodate a much wider 
range of functional abilities and therefore provide ben¬ 
efit to a larger, overall less-able, consumer base. As the 
average age of the baby boomers climbs into retirement 
years with significant disposable income, this segment 
will compel the market into providing better solutions 
to their well-being needs. 

Another grand challenge is robotic autonomy. Espe¬ 
cially for persons with reduced communication, phys¬ 
ical and/or cognitive abilities, a rehabilitation robot 
will need to have sensory (e.g., vision, auditory) and 
motor capabilities, combined with its own software 
processing capabilities (also termed artificial intelli¬ 
gence), that make it a sufficiently safe and capa¬ 
ble system to coexist with and benefit humans. This 
challenge will to some extent be dependent on con¬ 
tinuing increases in computer-processing power, and 
also specifically dependent on the algorithmic devel¬ 
opments that issue from the community of robotics 
researchers. 

For instance, several advanced navigation assis¬ 
tive tools for blind and visually impaired persons have 
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been developed by exploiting knowledge and tech¬ 
nologies directly derived from research on autonomous 
robot navigation; as a sample, Borenstein and Ulrich in 
1997 developed the GuideCane [64.21 1], an intelligent 
cane, based on ultrasound proximity sensor technology, 
which was designed to help blind or visually im¬ 
paired travellers to navigate safely and quickly among 
obstacles and other hazards faced by blind pedestri¬ 
ans [64.212]. 

Research on robotic aids (namely physically assis¬ 
tive robots or, also, contact assistive robots) has so 
far primarily targeted persons with mobility and ma¬ 
nipulation limitations, rather than children and adults 
with cognitive impairments [64.213,214]. However, 
increases in the prevalence of cognitive impairments re¬ 
lated to aging will make the latter focus increasingly 
important. Socially assistive robots, also named con¬ 
tactless assistive robots, are emerging assistive systems 
that focus on helping human users through social rather 
than physical interaction (see Chap. 73 for more de¬ 
tails on socially assistive robots). Research has been 
limited to the mobility focus due to the difficulty of 
designing and developing intrinsically safe robots that 
can coexist with people and exhibit a certain amount of 
autonomy while performing useful work. Robots, there¬ 
fore, today rely on user vigilance and explicit control to 
be safe. If the user does not have the cognitive capac¬ 
ity to evaluate a robot’s safety situation or the ability 
to communicate efficiently, then the positive value of 
a function-enhancing robot is nullified by the harm that 
it could inflict on the user or bystanders. Coupled with 
the fact that the design of interfaces to personal robots is 
still in its infancy, a challenge for robotic aid developers 
is a significant improvement in intrinsic safety without 
a decrease in function (strength, speed, etc.) from what 
is typical today in industrial robotics. 

To address some of these challenges, the US gov¬ 
ernment, through NSF, NIH and other federal agencies, 
in 2011 issued a call for a $ 50 million per year, 5- 
year program called the National Robotics Initiative 
(NRI) [64.215]. The realization of co-robots acting in 
the direct support of individuals and groups. A substan¬ 
tial amount of this funding is focused on healthcare of 
the future. 

Disabilities and Functional Limitations Served 
by Robotic Aids 

Assistive robots have been designed for people who 
have become severely disabled as a result of, for ex¬ 
ample, muscular dystrophy or a high-level SCI, for 
children who have CP, and more generally for any¬ 
one who lacks the ability to manipulate household 
objects. A market research study conducted 10 years 
ago, specifically for rehabilitation robotics clients, con¬ 


servatively projected a US market of 100000 peo¬ 
ple [64.216]. With the incidence of disability increasing 
exponentially, and the niches that robots can fill in re¬ 
habilitation applications multiplying with advances in 
robotics and rehabilitation science, it is clear that the 
market for rehabilitation robotics can only continue to 
increase. 

Human-Robot Interface Design 
for Assistive Robots 

A fundamental difference between using industrial and 
assistive robots is the interface required to command, 
control, and ultimately benefit from them. An industrial 
robot commonly has a combination of a manual con¬ 
troller and a programming language interface to allow 
an operator to teach a robot where to go and to enter 
the specific motion, grasping, tool changing, and error- 
recovery steps it must follow repeatedly in its factory 
automation scenario. An assistive robot, on the other 
hand, typically has three main differences and chal¬ 
lenges: 

1. The operator is not by definition a roboticist or en¬ 
gineer, so the interface must make accessible all the 
functions of the robot to allow its user to complete 
the required tasks. 

2. The user of a rehabilitation robot is, by defini¬ 
tion, a person with a disability, which means that 
physical, sensory, communication, and/or cognitive 
limitations in accessing the commands and controls 
of a robot need to be handled on a systems level 
by the designers of robots and their interfaces, with 
critical attention to universal design principles, and 

3. All rehabilitation robots require individualization 
of the interface to each user by the engineering 
and therapy professionals in charge of prescription 
and fitting, since disabilities vary considerably in 
how they restrict adaptability to standard configu¬ 
rations [64.217]. 

Interfaces of assistive robots consist of the software 
and hardware components conceived to enable a per¬ 
son with a disability to interact with an assistive device, 
thus tapping into residual communication capabilities 
of each user. For example, many people with tetraple¬ 
gia retain the ability to move a hand, arm, foot, or the 
head in a repeatable, even if range-limited way, and 
possibly even in two axes, such as forward/backward 
and left/right. With a proper placement of pushbut¬ 
tons, a joystick, or noncontact position measurement 
device, a rehabilitation engineer and therapist can de¬ 
velop a custom solution for each of their clients with 
disabilities to control a wheelchair computer and robot. 
In addition, adaptive hardware and software for control¬ 
ling a computer, such as head-position cursor control, 
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eye-trackers, speech recognition systems, trackballs 
and special keyboards, can be used to provide access 
to computer-based robot functionality. 

Even more important than for able-bodied com¬ 
puter and robot users, redundancy in input modality 
is important for persons with disabilities to prevent 
a system from becoming inoperable due to a simple in¬ 
terface malfunction or calibration problem. Providing 
two means of creating a mouse-click action (for exam¬ 
ple, a separate button placed next to a cursor-control 
trackball, as well as dwell time on a software button 
on-screen), even if one is inherently slower than the 
other, allows continued and uninterrupted use of the 
computer without outside assistance even if one of the 
two fails. 

For therapy robots, physical interfaces resemble 
those for physical and occupational therapy equip¬ 
ment, in general, and have a commonality with sports 
equipment interfaces as well, with adjustable hook- 
and-loop-type straps, heat-formable plastic cuffs, soft 
rubber, foam-based materials, and durable coverings for 
abrasion resistance and long wear. After a session or 
two for fitting and adaptation, a person using a therapy 
robot can often use the same interface for a long period 
of time. 

In summary, the keys to interface design are cus¬ 
tomizability, individualization, functional redundancy, 
adaptability, and patience in getting the interface to 
a comfort and functional level appropriate for the ef¬ 
fective use of the robot. 

64.3.2 Types and Examples 

of Assistive Rehabilitation Robots 

As mentioned in the Introduction, assistive robots can 
be divided into three main categories: manipulation 
aids, mobility aids, and cognitive aids. Each can be sub¬ 
divided as follows. Manipulation aids are commonly 
divided into fixed, portable, and mobile subtypes. Mo¬ 
bility aids are divided into electric wheelchairs with 
autonomous navigation features and smart walkers. 
Cognitive aids are divided into communication aids 
such as pet robots and autonomous caretaker robots. 
These categories are introduced below, and representa¬ 
tive systems that have undergone scientific user studies 
or are commercial products are presented (Fig. 64.12a- 
c). Other examples are mentioned in Sect. 64.1.3. 

Manipulation Aids: Fixed-Base 
Common robots of this type are ADL and voca¬ 
tional manipulation aids and kitchen robots. In the 
United States, the professional vocational assistive 
robot (ProVAR) is a research prototype based ini¬ 
tially on a PUMA-260 robot arm mounted on a 1 m- 



IC(Milru*r 


Fig.64.12a-c Workstation-type robots: (a) AfMaster, developed 
by the French Muscular Dystrophy Association, (b) ProVAR, de¬ 
veloped at the VA Palo Alto Rehabilitation R&D Center, and 
(c) Handy-1, developed by RehabRobotics, Ltd. (UK) 


transverse overhead track that allows the robot to ma¬ 
nipulate objects and operate devices on side shelves and 
the table-top, bringing objects (like a drink of water 
or throat lozenge) to the robot’s operator. The inter¬ 
face is via a JAVA/VRML plug-in to a common Internet 
browser, delivering high-level control to disabled office 
workers in a conventional pull-down menu and a con¬ 
trol screen interface [64.27,218]. This system and its 
predecessor DeVAR have been field tested by over 50 
subjects at 5 rehabilitation clinics to assess feasibil¬ 
ity and acceptability [64.219,220]. At a cost of over 
US $ 100000 currently, there are currently no plans for 
eventual product introduction. 

In the EU, following a development path parallel to 
ProVAR’s, there is the AfMASTER/RAID workstation, 
whose concept, instead of being built into a worksta¬ 
tion, includes a 2 m x 3 m robot work area in the user’s 
office to store objects and place appliances, next to the 
user’s own office space. The system has been developed 
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over a 20-year span and was briefly in limited produc¬ 
tion [64.29] but is no longer offered for sale. 

The kitchen robot, Giving-A-Hand, developed at the 
Scuola Superiore Sant’Anna in Pisa, Italy, is a low- 
degree-of-freedom device for mounting on the front rail 
of a kitchen counter and able to move food contain¬ 
ers to and from appliances, such as refrigerators and 
ovens [64.221], With an integrated control system, it 
can also make use of the internal controls of the devices 
to, for example, set cooking times and open doors. 

The UK-developed Handy-1 is a domestic robot 
with 3-DOF designed for one-switch operation by per¬ 
sons with CP [64.30]. Originally designed to allow 
a person to eat a meal one bite at a time, its application 
areas have been extended to face hygiene and cosmet¬ 
ics. A commercial product selling for about US$6000, 
it has been a commercial success due to its simplicity 
and application focus. An even simpler feeding robot, 
the UK’s electric Neater Eater [64.20] is for sale world¬ 
wide at about US$ 5000, and is designed for eating only. 

In [64.222], an overview of manipulation robotic 
aids is provided. They are classified through five cri¬ 
teria, based on robotic arm usage scenarios and sur¬ 
veys. In particular, new assistive manipulators have 
recently been developed that address interaction safety 
as a priority criterion in their design. Take for in¬ 
stance JACO [64.223], iARM [64.224] and RA- 
PUDA [64.225] as examples of robot arms achieving 
safety by limiting the performance of the robotic arm 
in terms of arm-movement speed and acceleration in 
space, end-effector force and maximum possible pay- 
load. Examples of robots addressing safety through 
backdrivable joints (as in the WAM Arm [64.226]) 
or through active impedance control (as in KARES 
II, WAM Arm, Elumotion RT2 [64.227], DLR LWR- 
III [64.228]] can also be mentioned. 

While a robot conventionally connotes a stand¬ 
alone system with some automation features, a smart 
bed and a smart home can legitimately be termed robots 
since they sense and act with motors under the shared 
control of its human users and its real-time software 
programming. Smart beds, such as SleepSmart mea¬ 
sures body position and temperature, as well as trends 
and anomalies over the course of a night. Restless¬ 
ness can be measured, and bed geometry (tilt of bed 
segments) and ambient conditions (light, temperature, 
sounds) can be adjusted according to presets and pref¬ 
erences [64.229]. 

Smart homes, such as the AwareHome domotic en¬ 
vironment at Georgia Tech, NL-iRV, and the University 
of Tokyo [64.230], provide integrated climate, secu¬ 
rity, lighting, entertainment and transport assistance, 
which is enabling especially to persons with severe 
functional disabilities. Coupled with health care-related 


functionality (following section), these robotic homes 
can allow a person with a cognitive or physical dis¬ 
ability to control many ADL functions and live safely 
through monitoring. 

Manipulation Aids: 

Wheelchair Manipulator Arm Systems 
A need for electric wheelchair users is the manipula¬ 
tion of objects while navigating a home or a public 
place such as a restaurant or grocery store. The assis¬ 
tive robot service manipulator (ARM) (Exact Dynam¬ 
ics, Netherlands) - previously known as MANUS - 
is a commercial robot arm that can be attached to an 
existing wheelchair to the side of the lap tray and con¬ 
trolled by the wheelchair’s own joystick or a number 
pad [64.23, 231] (Fig. 64.13). The robot has undergone 
numerous user studies with persons who have muscular 
dystrophy, a high-level SCI or CP. Worldwide, this is 
currently the only commercial rehabilitation robot arm 
that can be prescribed by a physician and that is reim¬ 
bursed by a government health care system. 

Weston [64.232] and Bridgit [64.233] are two 
wheelchair manipulator arm systems addressing the is¬ 
sue of interaction safety in their design, as for some 
of the manipulation aids with a fixed base presented 
earlier. Weston uses low-power motors in order to stat¬ 
ically and intrinsically limiting the arm’s acceleration, 
force and payload. Bridgit is a manipulator arm placed 



Fig. 64.13 Wheelchair manipulator robot MANUS devel¬ 
oped at the Rehabilitation R&D Center, Hoensbroek, and 
marketed by Exact Dynamics (The Netherlands) 





Rehabilitation and Health Care Robotics I 64.3 Aids for People with Disabilities 


on a wheelchair on a rail system. The robot moves over 
the rail system resulting in optimal positioning for each 
task. The rail system also allows for easy docking of the 
robot either on the back or front side of the wheelchair 
when not in use. 

Manipulation Aids: 

Mobile Autonomous Systems 
The most commonly thought-of form of a robot is 
that of an autonomous, mobile system with arms, 
having sensory-motor functionality similar to that of 
a human being, while serving people in performing 
menial physical tasks. This Handbook’s chapter on Hu¬ 
manoid Robots explores the domain as well (Chap. 67). 
Since locomotion is a key requirement for Humanoid 
robotics, other robots with wheeled bases have been de¬ 
veloped before the first walking robots were invented 
to explore more applied domains with more short-term 
usefulness. In film, robots such as Star Wars’ R2D2 
have made this form factor commonly known around 
the world. More recently, real robots such as the Help- 
Mate [64.234] have been employed in US hospitals as 
fetch-and-carry robot orderlies, using floor maps and 
short-range ultrasonic sensors for navigation and ob¬ 
stacle avoidance. The Italian MovAid research robot 
platform [64.235] (in Fig. 64. 14) adds manipulation and 
vision to these capabilities to navigate in home-like 
environments to provide object manipulation and de¬ 
vice operation to individual users. The European project 
Robot-Era [64.236] is following up these developments 
with a specific target on the needs of aging popula¬ 
tion. The German Care-O-bot [64.237] has explored 
advanced navigation and sensing in a wheeled robot that 
can also be used as a physical support to people requir¬ 
ing mobility and stability assistance. It has also doubled 
as a mobile kiosk, moving around a trade show floor and 
delivering information to attendees. In [64.238] a case 
study of a personal robot based on the PR2 Humanoid 
robot (Willow Garage, Menlo Park) is presented. The 
approach pursued consists of developing a diverse suite 
of open-source software tools that blend the capabilities 
of the user and the robot in order to enable the assistive 
mobile manipulators to move in real homes and work 
with people with disabilities. 

Mobility Aids: Wheelchair Navigation Systems 
A critical function for people who use electric 
wheelchairs for their mobility impairment and who in 
addition have communication or cognitive disability is 
semiautonomous navigation assistance (Fig. 64.15). 

Add-ons to commercial wheelchairs have been de¬ 
veloped by numerous research groups for this ser¬ 
vice. Such wheelchairs are typically referred to as 
smart or intelligent wheelchairs. As proposed by Simp- 



Fig. 64.14 MovAid (after [64.239]) 


son [64.240], smart wheelchairs can be classified 
by form factor: early smart wheelchairs were mo¬ 
bile robots with added seats. The vast majority of 
smart wheelchairs developed until 2005 were based 
on heavily modified, commercially available, powered 
wheelchairs. Only a smaller fraction of them were 
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Fig.64.15a,b Wheelchair navigation aids: (a) Wheelesley and 
(b) Hepahaestus 


equipped with add-on units that could be attached to 
and removed from the underlying powered wheelchair. 

The NavChair [64.241] was one of the first to 
demonstrate robust wall-following, door passage even 
with narrow doorways, and speed adaptation to peo¬ 
ple walking in front of the wheelchair, all using only 
short-range ultrasonic and other sensors, but not vision. 
The Hephaestus [64.35] is a next-generation system 
made specifically as a commercial accessory for a va¬ 
riety of wheelchair brands, tapping into the joystick 
controller and power system. The Wheelseley [64.242] 
and KARES [64.33] robots have explored similar func¬ 
tionality using a vision system for scene analysis and 
way-finding. 

More recently, the Can Wheel project team de¬ 
veloped an intelligent wheelchair system called 
NOAH [64.243,244], The system has three main ca¬ 
pabilities: collision avoidance, infer the user’s goal 
location/activity and provide automated reminders, pro¬ 
vide navigation assistance using prompts. The rationale 


for proposing such solutions is to enhance mobility and 
to help improving the quality of life of older adults with 
cognitive impairments, while simultaneously reducing 
the burden on caregivers. 

In 2013, How et al. [64.245] proposed a new in¬ 
telligent wheelchair system (IWS) with anticollision 
and navigation features. User trials showed the IWS’s 
potential to improve powered wheelchair safety and 
subjective usability. 

The IntellWheels [64.246] project proposes a mod¬ 
ular platform based on a multiagent system paradigm 
for the development of intelligent wheelchairs based 
on commercial products. Within this project, promis¬ 
ing results have been achieved on the development of 
adapted control methods for CP users of an intelligent 
wheelchair. Experiments demonstrated that users felt 
that they had better control over the wheelchair move¬ 
ment when using shared control rather than manual or 
automatic control modalities [64.247], 

The iBOT [64.248-250] is a powered wheelchair 
for persons with mobility impairment developed by Ka- 
men, in a partnership between DEKA and Johnson & 
Johnson’s Independence Technology division. Research 
on iBOT was discontinued in 2009. The iBOT features 
self-balancing technology, which allows users to go up 
and down staircases, to navigate on uneven terrain and 
to stand at an eye level with people walking nearby. 

Mobility Aids: Walking Assistance Systems 
A third type of mobile robot for stability assistance has 
the peculiarity that it is underactuated and has similar¬ 
ity with the co-bot concept, in that the wheels are not 
driven, but are actively steered and braked (Fig. 64.16). 
The concept of collaborative co-bots was originally 
introduced by Colgate et al. for robots operating in di- 



Fig.64.16a-c Human assistance robots: (a) Care-O-bot, developed by the Fraunhofer Research Institute (Germany), (b) 
Helpmate by Transitions Research Inc., USA. (c) Pam-Aid (aka Guido), developed in the UK 
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rect physical interaction with a human factory worker, 
handling a shared payload. They are a marked depar¬ 
ture from autonomous industrial robots, which must be 
isolated from people for safety reasons [64.251,252]. 
Co-bots interact with people by producing software- 
defined virtual surfaces which constrain and guide the 
motion of the shared payload, but add little or no 
power [64.253]. Today, cobots are being prototyped in 
the rehabilitative and assistive context, e.g., for bed 
to chair/wheelchair transfer or table-top upper limb, 
stroke rehabilitation at home. For example, the Pam- 
Aid [64.254] looks like a closed-front walker on wheels 
and has bicycle-type handlebars. The person walking 
behind the device turns the handlebars, causing the 
wheels to turn in the correct direction. If the ultrasonic 
sensors detect an obstacle in front of it, the brakes pre¬ 
vent the user and device from colliding with it. The 
Care-O-bot (see earlier) designed originally as a mo¬ 
bile autonomous robot approximately human size, has 
a similar set of handlebars, similar to the Pam-Aid 
it, so it can be used as a smart walker. The larger 
mass of the Care-O-bot, however, requires it to be 
motorized. 

Mobility Aids: Exoskeletons 
Exoskeletons for walking assistance are similar to 
robots used in treadmill-based environments for reha¬ 
bilitation. These systems are portable and autonomous 
and intended to be used in daily life scenarios. Several 
review papers have been recently published [64.255- 
257] on this topic. 

In the framework of NISTs Advanced Technology 
Program, Ekso Bionics (Berkeley, USA) developed the 
Ekso device [64.258] (Fig. 64.17). This robot has been 
developed for people with lower extremity weakness 
or paralysis due to neurological disease or injury (e.g., 
spinal cord injuries, multiple sclerosis, Guillain-Barre 
syndrome) and it has an almost anthropomorphic struc¬ 
ture with hip and knee joints actuated in the sagittal 
plane. The ankle joints are not actuated but are com¬ 
pliant in the sagittal plane and locked out in the other 
DOFs. Testing of the device has included paraplegic 
persons with complete or incomplete paralysis [64.259] 
and chronic stroke patients [64.260]. 

The Re Walk was developed by Argo Medical Tech¬ 
nologies [64.261], It is actuated by DC (direct current) 
motors at the hip and knee joints in the sagittal plane, 
while the ankle joint is not actuated. The system is 
designed with a remote controller that can be used to 
change the motion mode (e.g., ground walking, climb¬ 
ing stairs). A posture sensor on the torso detects the 
upper body movement of the user and estimates mo¬ 
tion intention. The wearer also has to use crutches for 
stability and safety reasons. The system is undergoing 



Fig. 64.17 Ekso Bionics exoskeleton for paraplegics (af¬ 
ter [64.258]) 


clinical trials among other research centers at Moss- 
Rehab (Philadelphia, USA) and at the Centro Protesi 
INAIL di Vigorso di Budrio (Bologna, Italy) on para¬ 
plegic subjects. The device is now available on the 
market in two versions: the Re Walk-Rehabilitation for 
institutional use and the ReWalk-Personal developed 
for daily use (Fig. 64.18). 

Sankai’s group at the University of Tsukuba (Japan) 
developed an exoskeleton both for performance aug¬ 
mentation and for rehabilitation and assistance [64.263, 
264]. The current version, HAL-5, powers the flex¬ 
ion/extension of hip and knee via DC motors while 
ankle dorsi/plantar flexion DOF is passive. The HAL-5 
system (Fig. 64.19) integrates a number of sensors: 
skin-surface EMG electrodes placed below the hip and 
above the knee on both the anterior (front) and poste¬ 
rior (back) sides of the wearer’s body, potentiometers 
for joints angles measurement, ground reaction force 
sensors, a gyroscope and accelerometer mounted on 
the backpack for torso posture estimation. HAL-5 is 
currently commercialized by the spinoff company Cy¬ 
berdyne (Tsukuba, Japan). To date, it appears that no 
peer-reviewed, quantitative results have been published 
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Fig. 64.18 ReWalk computerized exoskeleton (ReWalk 
Robotics, Inc., USA; after [64.262]) 


on the effectiveness of the exoskeleton for the improve¬ 
ment of walking functions. 

The Vanderbilt powered orthosis [64.265] is a pow¬ 
ered lower limb exoskeleton intended to provide gait 
assistance to individuals suffering from spinal cord 
injury by providing assistive torques in the sagittal 
plane for both the hip and knee joints. It includes nei¬ 
ther a portion that is worn over the shoulders, nor 
a portion that is worn under the shoes. Each joint is 
powered by a brushless DC motor. The orthotic is 
intended to be worn together with a standard ankle- 
foot orthosis, which provides support at the ankle and 
prevents foot drop during swing. In order to demon¬ 
strate its ability to assist walking, the orthosis was 
experimentally tested on paraplegic subjects. Experi¬ 
mental results indicate that the orthosis is capable of 
providing a repeatable gait with knee and hip joint 
amplitudes that are similar to those observed during 
non-SCI walking. 



Fig. 64.19 Robot suit HAL-5 designed by lapanese 
robotics firm Cyberdene 


REX, produced by REX Bionics (Auckland, New 
Zealand), is an anthropomorphic lower body robot de¬ 
signed for sit-to-stand, stair climbing and overground 
walking, without the use of crutches. The system does 
not use sensors to detect the intention of the user but 
rather it uses a joystick as its interface. The system has 
been tested with healthy subjects, and for sit-to-stand of 
wheelchair users [64.266]. 

Cognitive Aids 

There has recently been increased interest in using 
robots as motivational and educational agents during 
rehabilitation therapy. This approach typically involves 
small, pet-like, toy-like, approachable devices that do 
not physically interact with the patient, but exist pri¬ 
marily to engage the patient in an affective way that 
promotes personal health, growth, and interaction. For 
more information, please see Chap. 73. 
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64.4 Smart Prostheses and Orthoses 

In 2005, the Defense Sciences Office (DSO) of the 
US governmental research agency DARPA launched 
a program to revolutionize prosthetics in a four-year 
timeframe. According to the agency website, this pro¬ 
gram will: 

deliver a prosthetic arm for clinical trials that is 
far more advanced than any currently available. 
This device will enable many degrees of freedom 
for grasping and other hand functions, and will be 
rugged and resilient to environmental factors. In 4 
years, DSO will deliver a prosthetic for clinical trials 
that has function almost identical to a natural limb in 
terms of motor control and dexterity, sensory feed¬ 
back (including proprioception), weight and envi¬ 
ronmental resilience. The four-year device will be di¬ 
rectly controlled by neural signals. The results of this 
program will allow upper limb amputees to have as 
normal a life as possible despite their severe injuries. 

64.4.1 Grand Challenges and Roadblocks 

This program announcement lays down the grand chal¬ 
lenge for prosthetics research in an ambitious time- 
frame: develop an artificial limb that has function and 
durability at least as good as a natural limb. There are 
several roadblocks to meeting this challenge. First, pro¬ 
viding an intuitive way for individuals to control and 
coordinate multiple joints of a robotic limb is challeng¬ 
ing. Second, robots do not yet match the human arm in 
terms of the combination of range of force, weight, and 
duration of use with a portable power source. Third, hu¬ 
man limbs are rich with tactile and movement sensors. 
Installing artificial sensors on a robotic limb, and then 
returning information from those sensors in a way that 
is usable by the user is challenging. Thus, solving the 
grand challenge will require better sensory-motor in¬ 
terfaces for prosthetic limbs, as well as lighter, stronger 
actuators, and better power sources. 

Substantial progress has recently been made in im¬ 
proving sensory-motor interfaces for prosthetic limbs, 
and this progress is the focus of this section. For the 
current state of robotic actuators that could be used in 
prosthetic devices, the reader is referred to Chap. 77 
on Neurorobotics. For an overview of the design of 
conventional prosthetic hands and arms, the reader is 
referred to [64.267], 

64.4.2 Targeted Re-Innervation 

Standard prosthetic arms and hands are commonly con¬ 
trolled with a cable drive or by electromyogram (EMG) 


signals from residual muscles. For example, to open and 
close an artificial hand, one common technique is to 
place a Bowden cable around the shoulders in a harness, 
and connect the cable directly to the artificial hand. The 
user can then shrug the shoulders to move the cable 
and open and close the hand. Alternately, electrodes 
can be placed on a muscle in the residual limb or on 
the user’s back, for example, and then used to control 
a motor on the artificial hand. The cable technique has 
the advantages of simplicity, and of having the prop¬ 
erty of extended physiological proprioception (EPP), 
which refers to the fact that the grip force is mechan¬ 
ically transmitted back to the user’s shoulder muscle 
force sensors so that the user can gauge the strength 
of the grasp. Because of their simplicity and EPP, ca¬ 
ble drives (or body powered prostheses) have been 
more popular than myoelectric (or externally powered) 
prostheses. However, the body-powered technique is 
amenable to controlling only one degree of freedom 
at a time, although chin switches, for example, can be 
used to switch between degrees of freedom in a some¬ 
what cumbersome way. The myoelectric approach can 
be used to control multiple degrees of freedom, but such 
control is nonintuitive and cumbersome. Also, multi¬ 
ple control sites for reading out EMG are not available 
for people who have lost their entire arm. Thus, pros¬ 
thetic control systems are typically limited to one or 
two degrees of freedom, while functional arm and hand 
movement benefits from at least four degrees of free¬ 
dom (three to position the hand, and one to open and 
close it). 

Kuiken et al. [64.268] recently developed a novel 
approach to improving control of a multijointed pros¬ 
thetic arm. In this targeted re-innervation technique, 
they re-routed the nerves that previously innervated the 
lost limb to a spared muscle, and then read out the 
user’s intent to move the limb using electromyography 
at the spared muscle. They demonstrated this technique 
in a bilateral shoulder disarticulation amputee who had 
lost both of his arms in an electrical power accident. 
They took the residual brachial plexus nerve for the left 
arm, which normally innervates the left elbow, wrist, 
and hand, and moved it to the pectoralis muscle. The 
subject could still contract his pectoralis muscle, but 
this muscle was no longer useful to him since it used to 
attach to his now-missing humerus. A surgeon dissected 
portions of the nerve associated with different muscles 
in the elbow, wrist, and hand, and innervated three bun¬ 
dles of the pectoralis muscle. After three months, the 
nerve re-innervated the bundles so that the patient could 
cause the bundles to twitch by trying to bend his miss¬ 
ing elbow, for example. Surface EMG electrodes were 
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placed over the bundles. Then, when the user willed to 
open his hand, for example, a pectoralis muscle bundle 
contracted, and this contraction was detectable with the 
EMG electrodes. The EMG signal was in turn used to 
control the hand motor of the prosthetic arm. The net 
result was that the user could will his different (miss¬ 
ing) anatomical joints to move, and the corresponding 
joints on the robotic arm would move. He could simul¬ 
taneously operate two joints, such as the elbow and the 
hand. The user became able to do tasks that he was 
not able to do before with his conventional myoelec¬ 
tric controlled arm, such as feeding himself, shaving 
and throwing a ball. A secondary remarkable finding 
was that the sensory neurons in the re-routed nerves 
re-innervated sensors, so that now when the person’s 
chest is touched, the person perceives it as a touch to his 
missing limb. This sensory re-innervation could possi¬ 
bly be made into an interface to provide tactile sensation 
from the artificial limb. These findings were recently 
confirmed in another person who received targeted re¬ 
innervation [64.269]. 

64.4.3 Neural Interfaces 

for Limb Prosthetic Devices 

Neural interfaces provide an interesting and challeng¬ 
ing solution to retrieve the natural way of interfacing 
the human nervous system to prosthetic artifacts. They 
are systems capable of recording either invasively or 
noninvasively the electrical activity of peripheral nerves 
as well as of the brain cortex. It has also recently been 
shown that the direct electrical stimulation of a residual 
peripheral nerve can provide usable information regard¬ 
ing force to a person with an amputation [64.270], thus 
paving the way for bidirectional neural interfaces capa¬ 
ble of restoring both efferent and afferent information 
flow to/from the prosthesis. Recently, thin film intra- 
fascicular electrodes implantable in peripheral nerves 
have been developed [64.271], and successfully val¬ 
idated in 2008 in Italy at Campus Bio-Medico Uni¬ 
versity of Rome on a human amputee [64.272] within 
the LifeHand project, a cluster of European and Ital¬ 
ian research actions focused on neural interfaces for 
prosthetics (Fig. 64.20). In 2013, a second round of 
experiments in Italy demonstrated the possibility of de¬ 
livering physiologically appropriate (near-natural) sen¬ 
sory information to an amputee during the real-time 
decoding of different grasping tasks to control a dex¬ 
terous hand prosthesis by stimulating the median and 
ulnar nerve fascicles using transversal multichannel in- 
trafascicular electrodes, according to the information 
provided by the artificial sensors from a hand pros¬ 
thesis [64.273]. The results also demonstrate that the 
subject was able to identify the stiffness and shape of 



Fig. 64.20 LifeHand aims to create a completely im¬ 
plantable prosthesis system, richly sensorized and cron- 
trolled through the patient’s nervous system, with a dex¬ 
terity comparable to a natural limb (courtesy of Campus 
Bio-Medico University Rome, Italy) 

three different objects by exploiting different charac¬ 
teristics of the elicited tactile sensations. These results 
are in line with earlier studies which outlined the im¬ 
portance of restoring tactile feedback on a prosthetic 
device, such as the one proposed by Meek et al. in 
1989 [64.274], 

Today the above mentioned approach is trying to be 
extended on lower limb prosthetics as well. Herr et al. 
a MIT pioneered a new class of bio-hybrid smart pros- 
theses and exoskeletons [64.275] aiming at improving 
the quality of life of people with physical challenges. 
Some of these devices are now commercialized by 
a spin-off company, BiOM. For instance, a computer- 
controlled prosthesis called the Rheo Knee [64.275] is 
outfitted with a microprocessor that continually senses 
the joint’s position and the loads applied to the limb. 
A powered ankle-foot prosthesis emulates the action 
of a biological leg to create a natural gait, allowing 
amputees to walk with normal levels of speed and 
metabolism as if their legs were biological [64.275]. 
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There has also recently been progress in decoding 
movement-related signals in real-time directly from the 
brain (see the cover story and related articles in Nature 
of 13 July 2006 [64.276]. The first of several recent hu¬ 
man volunteers, a person with tetraplegia due to SCI, 
has received a BrainGate electrode array implant, and 
has been able to control the movement of a cursor on 
a computer screen [64.277]. Noninvasive systems op¬ 
erate by recording brain activity from the outside of 
the skull via well-known clinical noninvasive diagnostic 
devices [64.278] such as electro-encephalogram (EEG). 
Individuals have been demonstrated to be able to learn 
to control the amplitude of the EEG signal as a function 
of time, or the amplitude of specific frequency com¬ 
ponents of the EEG signal, with a moderate amount 
of practice (several hours to several days). The level 
of control is sufficient to operate a typing program on 
a computer, or to control the movement of a cursor to 
multiple targets. 

In summary, given the significant progress observed 
in the last decade, it appears that future control sys¬ 
tems for smart prosthetics and orthoses will have the 
option to rely on direct interfaces to the brain, which 
should allow control of multiple joints through thought 
alone. The initial work on both targeted re-innervation 
and brain-machine interfaces to the PNS or to the CNS 
has allowed three to 4-DOF of control in a naturalistic 
manner and elicitation of some sensory feedback, which 
is an advance over conventional prosthetic control tech¬ 
niques. 

64.4.4 Advances in Neural Stimulation 

Functional neural stimulation techniques (FNSs) seek 
to electrically stimulate the residual nervous system 
to re-animate the limbs. FNS for standing, walking, 
reaching, and grasping has been demonstrated, but 
these techniques have met with limited commercial 
success because of a combination of factors, includ¬ 
ing the ease of use of systems that use surface elec¬ 
trodes, duration of use before fatigue, risk from im¬ 
plantation and complexity of the associated control 
problems. 


Two research lines are being pursued to help move 
the FNS field forward. The first one focuses on hard¬ 
ware innovation. A good example is the BION, an 
injectable stimulator the size of a very large grain of 
cooked rice [64.279], which can be inserted without 
surgery (using a large-gauge needle) and is robust and 
resistant to infections. The second research line aims 
at stimulating the control circuits in the nervous system 
rather than individual muscles. For example, it has been 
shown that locomotor-like movements can be eliciting 
in multiple muscles of the cat hind limb by stimulating 
regions of the spinal cord directly [64.280]. 

64.4.5 Embedded Intelligence 

Recent robotics-related advances for prosthetic legs 
have included embedding microprocessors and pas¬ 
sive braking systems into artificial knees, so that the 
knees can, for example, be made relatively stiff during 
the stance phase of gait, and free to move during the 
swing phase of gait [64.281]. The first microprocessor 
knee introduced was the Ottobock C-Leg (Germany), 
introduced in 1999. The C-Leg uses a servomotor to 
adjust valves to hydraulic pistons. The rechargeable 
battery lasts about 24 hours. The pattern of resistance 
throughout the gait cycle can be adjusted for each user. 
A dramatic example of the benefit of the C-Leg is the 
story of a man who made it down from the 70th floor 
of the World Trade Center on 9/11/2001 with only 
minor bruising to his residual stump [64.282]. Other 
microprocessor-controlled knees are the endolite adap¬ 
tive prosthesis, which uses pneumatic and hydraulic 
valves, the Rheo Knee (Ossiir Hf, Iceland), which uses 
magneto-rheologic fluid to vary the knee impedance 
and the Intelligent Prosthesis. 

The first powered knee that can generate power, 
rather than just dissipate energy, is currently being com¬ 
mercialized by Ossur as the Power Knee. The system 
combines an electromechanical power source that will 
be controlled with input from sensors on the sound leg 
shoe. Initial reports suggest that this is the first knee that 
allows the user to walk up stairs with a step-over-step 
pattern. 


64.5 Augmentation for Diagnosis and Monitoring 


A critical aspect of rehabilitation is health maintenance 
with age-related or degenerative functional decline and 
after a medical intervention. In-home diagnostic equip¬ 
ment, devices worn on or in the body for vital signs 
monitoring, tele-health services, and institution-based 
monitoring automation are all examples of systems be¬ 


ing developed to improve the quality of life for both per¬ 
sons at risk and their caregivers. Institutional systems 
of this nature, more properly part of the field of clinical 
engineering, are incorporating more robotic, networked 
and autonomous devices to take more accurate diag¬ 
nostics, provide better information to physicians and 
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provide faster alerts. Key enabling technologies in this 
field are advanced materials and nanotechnologies. 

64.5.1 Grand Challenges 

and Enabling Technologies 

For all devices that are worn on the body, the inter¬ 
face must be skin compatible. A grand challenge for 
this field in the near-term is the better incorporation 
of active and sensing elements with textiles. Several 
prototype sensor shirts show promise, but rehabilita¬ 
tion will have a much richer toolset for diagnosis and 
monitoring with advances in this area. Nanotechnol¬ 
ogy, an enabling technology for the longer term, has the 
promise to miniaturize virtually everything mechatronic 
that is currently macroscopic. Injected devices, such as 
nano-robotic drug dispensers and clot-busters will aid 
in rehabilitation. 

64.5.2 Smart Homes for Health Care 
Monitoring and Care 

A special class of fixed-station rehabilitation robots is 
an automation system designed to provide a safe envi¬ 
ronment to assist and monitor a person with a disability 
living at home or in an institutionalized setting such as 
an assisted living facility or nursing home. 

Smart Nursing Home Automation 
An assisted living, hospice, or nursing care institu¬ 
tional facility will include residents who have mild 
to severe cognitive impairment in addition to physical 
disabilities. The facility may have zones to separate res¬ 
idents who have different levels of dependency since 
the architectural, monitoring and personnel needs are 
different. To better serve residents and guests, to opti¬ 
mize function and to minimize cost, only the areas for 
persons with high dependency have a 24 h staffed vital 
signs monitoring and alert capability, for example. Fa¬ 
cility care is highly staff-intensive, though automation 
through diagnostic vital signs monitoring, electronic 
surveillance, and patient tracking continue to improve 
safety and efficiency. Robotics and automation are be¬ 
ginning to find applications in the physical tasks asso¬ 
ciated with patient care, therapy and oversight. Some 
examples are described below. 

Examples of the State of the Art 
Wandering, especially at night, is a significant prob¬ 
lem for institutional facilities with ambulatory residents 
who are cognitively impaired. Simple architectural 
modifications include painting the hallway in front of 
doors black to make them look like deep holes. An au¬ 
tomated voice system triggered by a motion detector to 


say Go back to bed is effective, but not fool-proof, ei¬ 
ther. Resident detection systems based on ID badges 
with embedded RFID chips that can be sensed in 
a hallway work, but only if the resident is wearing it. 
A robotic sentry system, including mobile platforms to 
aid in solving this problem, especially at night, has not 
yet been developed. 

A serious rehabilitation issue in institutionalized 
facilities is the transfer of residents from bed to 
wheelchair and other surfaces. A number of manual, 
electric, and robotic devices have been developed to 
assist the nursing staff to safely transfer residents and 
patients who may be significantly heaver than they are. 
This remains an unmet clinical need, though not for lack 
of innovative attempts [64.283, 284]. 

64.5.3 Home-Based Rehabilitation 

Monitoring and Therapy Systems 

Numerous smart homes have been developed for non¬ 
rehabilitation as well as assistive purposes [64.285], 
These systems have as their goal the safety of people 
with disabilities living in the home and communication 
with caregivers outside of the home. Caregivers can be 
live-in family or attendants who, even when they are not 
home, need to be kept informed on the status of the dis¬ 
abled person, as well as clinicians who need to be sent 
regular vital signs and other medical/therapy reports. 
In-home systems typically feature the same principal 
elements: 

1. Sensors to monitor both ambient and as people- and 
object-specific parameters (e.g., person location, 
stovetop operation); actuators to modify ambient 
conditions (heat, lighting, sound system, etc.) and 
operate devices (doors, refrigerator, etc.). 

2. A means to network all the sensors and actuators 
for uni- or bi-directional communication with the 
host computer. This network can be wireless (e.g., 
802.11 g), wired (e.g., coaxial cable), or dependent 
on an existing network (e.g., signals superimposed 
on current carried by the electrical mains wiring or 
phone wiring). 

3. A host computer that allows all sensor states to be 
displayed and actuators to be operated from one or 
more locations in the home by the inhabitant(s) us¬ 
ing common computer I/O devices. Higher order 
functions are built upon this basic capability. 

4. An external network to allow communication with 
the Internet via phone, cable, satellite or other 
means. This capability allows for remote monitor¬ 
ing and operation, sending of alarms and discus¬ 
sions with rehabilitation professionals at medical 
centers. 
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The host computer software may also have higher 
order features, for example timers for repetitive ac¬ 
tuation of lights and monitoring for anomalous sen¬ 
sor readings (e.g., call security when the smoke de¬ 
tector activates, alert inhabitants with in-home alarm 
when stove top power is on and no pot is on the 
stove). More advanced features that involve multi¬ 
input and multioutput control and adaptive, predictive, 
context-aware operation [64.286] are areas of active re¬ 
search, and especially important to the rehabilitation 
community. 

As previously mentioned in this chapter, devices 
for home-based robot-assisted therapy have been re¬ 
cently developed, and home-based therapy delivered by 
robotic devices is expected to have a significant growth 
in the short-medium term. 

CBM Motus [64.126] and T-WREX, [64.287,288], 
a passive exoskeleton for upper limb rehabilitation now 
sold by Hocoma AG as ArmeoSpring, are just two ex¬ 
amples of systems for home-based therapy described 
above (Sect. 64.2.3). Another example is the Hand 
Mentor, which is a device which uses video games and 
robotics to cognitively involve the patient in his/her 
rehabilitation [64.289,290]. The Hand Mentor can be 
used in the clinic, or taken at home and incorporated 
into patients’ daily therapy sessions to lengthen short¬ 
ened tissues, facilitate hand opening and closing, and 
reduce spasticity. It is mainly conceived for stroke pa¬ 
tients’ rehabilitation and is commercialized by Kinetic 
Muscles, Inc. 


64.5.4 Wearable Monitoring Devices 

One component of an automated rehabilitation environ¬ 
ment is the subsystem that a person wears to be able to 
measure, analyze, and communicate physiological sig¬ 
nals to an external computer wirelessly. Systems such 
as the LifeShirt (VivoMetrics, Inc., Ventura) [64.291] 
have been and are being developed for front-line sol¬ 
diers and rescue operation personnel whose health may 
become imperiled when out of touch with and unable 
to communicate verbally with their base command. For 
rehabilitative purposes, for example, the Intel Proactive 
Health Initiative [64.292] is an example of a system that 
use on-person position and motion sensing to detect po¬ 
tentially dangerous or undesirable situations. 

The use of a robot to deliver therapy already en¬ 
ables measuring information on the patient state by 
exploiting robot sensors and other wearable units, if 
needed [64.293]. Such capability can be exploited for 
patient monitoring. A recent work proposed a method 
for reconstructing the human arm kinematics by resort¬ 
ing to an inverse kinematics technique for redundant 
robot arms [64.294]. 

The most significant obstacles to the widespread 
adoption of these technologies in the short term are cost, 
false-positive alarms, inconvenience and encumbrance. 
Advances in micro-electronics, nanotechnologies, soft¬ 
ware algorithms, and networking capabilities will con¬ 
tinue to drive the research and consumer acceptance of 
this technology sector. 


64.6 Safety, Ethics, Access and Economics 


Rehabilitation robots interact closely with humans, of¬ 
ten sharing the same workspace and sometimes phys¬ 
ically attaching to humans, as in the case of robotic 
movement training devices and prosthetic limbs. Fur¬ 
thermore, the devices are by necessity powerful enough 
to manipulate the environment or the user’s own limbs, 
which means that they are also often powerful enough 
to injure the user or another person nearby by colliding 
with them or moving their limbs inappropriately. Safety 
is clearly of paramount importance. 

A common strategy for ensuring safety is to incor¬ 
porate multiple, redundant safety features. A device can 
be designed to be mechanically incapable of moving 
itself or the user’s limbs in such a way as to cause in¬ 
jury. Limits can be placed on the range, strength and 
speed of actuators so that they can accomplish the de¬ 
sired task but no more. Break-away attachments can be 
used to attach to users’ limbs. Covers can protect the 
user from pinch points in the device. Redundant sen¬ 


sors can be included, so that if one sensor malfunctions 
another sensor can identify the malfunction and help 
safely shut down the machine. Watchdog timers can 
monitor the health of the control computer. Software 
checks can limit forces, motions, speeds, and user ad¬ 
justments to control parameters, as well as check for 
sensor health and other dangerous situations. Control 
strategies can be designed so that the device is mechan¬ 
ically compliant, reducing the risk of forcing a limb 
into an undesirable configuration, or of a high-impact 
collision. A manual override switch can be incorpo¬ 
rated so that the user can shutdown the system. Finally, 
the user can be instructed on how to safely operate 
the device and avoid dangerous situations. Safety ul¬ 
timately depends, however, on careful and rigorous 
failure mode analysis and remediation by the system 
designers. 

From a system’s perspective, when all else fails ac¬ 
tively to protect the user, it must be the design itself 
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that makes the robot inherently unable to injure its user. 
Part of the solution is in reducing the weight, rounding 
the surface characteristics, and making appropriate ma¬ 
terials choices. The goal of inherent safety, however, is 
often at odds with high performance and adequate pay- 
load for real-life tasks. Recently, several approaches to 
designing personal robots - in other words - the class 
to which assistive rehabilitation robots belong - have 
sought to address both goals by dividing the two tasks 
of compensating for gravity (arm plus payload) and 
moving the payload around in space [64.295]. The so¬ 
lution is to provide two actuators per joint on the joints 
that support the arm segments and payload against grav¬ 
ity: one slow, gear-reduced motor and energy storing 
device such as a large spring or compressed air vol¬ 
ume, and one small, backdrivable motor that provides 
the power needed to move objects around quickly and 
precisely. Most robot manipulators have approximately 
a 1 : 10 (or worse) payload-to-weight ratio. A system 
with a dual, parallel actuator system that requires only 
the small, fast actuator to be carried in the arm, leav¬ 
ing the slow energy-storage system on the base and not 
contributing to the inertia of the arm itself, can lead to 
a 1 : 1 payload-to-weight ratio more in line with a hu¬ 
man’s own arm characteristics, and thereby provides 
a safe yet high-performance solution. An added bene¬ 
fit from this type of arrangement is that the movements 
of the arm will tend to be more human-like, providing 
a measure of confidence to the user that the robot is per¬ 
forming properly and moving in a safe way. 

Strategies for improving safety have been proposed 
and methods to assess safety have been developed and 
adapted for rehabilitation robotics [64.296, 297] based 
on accepted risk analysis methods. While industrial 
robots have benefited from ISO user safety regulations 
since 1992 (ISO 10218), the fundamental issue of hu¬ 
man proximity to robots for the personal, service, and 
rehabilitation sectors has prevented that standard from 


being applicable to rehabilitation robots. Currently, 
a draft of a new safety requirements for personal care 
robots standard, ISO 13482, is under development. Cur¬ 
rently, the existing industrial standards, augmented with 
provisions from medical equipment standards and but¬ 
tressed by engineering best practices and adherence to 
professional codes of ethics by designers, have guided 
rehabilitation robotics designers. Clearly, as products 
appear on the market and the expected rapid expansion 
of this sector happens, better regulations and standards 
must be developed. 

Beyond safety, there are other ethical concerns that 
will emerge as robotics technology becomes more in¬ 
telligent with advances in cognitive software, more in¬ 
vasive with nanotechnologies and better integrated with 
human systems through bioengineering advances. Ethi- 
cists and roboticists are starting to deal with these is¬ 
sues [64.298, 299], which to date have been the purview 
of only futurists and science fiction writers. Chap. 80 
deals with these issues in detail. 

An economic advantage has not yet been demon¬ 
strated in a decisive way for most rehabilitation 
robotics. For example, the therapeutic benefits con¬ 
ferred by robotic therapy devices, and the assistive 
benefits conferred by wheelchair-mounted robots rel¬ 
ative to the devices’ cost, have not yet been large 
enough to cause widespread adoption. Improvements in 
their efficacy and reductions in their cost will increase 
their usage. For example, a robotic therapy device that 
helps people learn to walk after a stroke, in a way 
that is decisively better than other training techniques, 
would become widely used very quickly. Likewise, 
a wheelchair-mounted robot that gives a disabled per¬ 
son a substantial and efficient increase in autonomy at 
a reasonable cost would also quickly become widely 
used. An example of a robotics technology that has 
achieved an attractive cost-benefit ratio and thus is com¬ 
mercially successful is the powered wheelchair. 


64.7 Conclusions and Further Readings 


Rehabilitation robotics is a dynamic application area 
because its grand challenges are at the forefront of 
both robotics and biology research. The ongoing ma¬ 
jor themes of the field can be summarized as the 
development of robotic therapy devices, smart prosthe- 
ses, orthoses, functional aids and nurses that match or 
exceed the capabilities of their human counterparts. Re¬ 
habilitation robotics is also a highly motivating field 
because the technology developed will directly help 
people who are limited in major life activities. The field 
will continue to grow because of the dramatic aging of 


the populations of industrialized countries that is just 
beginning. 

The grand challenges of rehabilitation robotics are 
grounded in the distinguishing features of the field: 
functional involvement with humans, a physical user 
interface and behavior that is intelligent, adaptive, and 
safe. These characteristics require high levels of re¬ 
dundancy, sensory-motor capability, adaptability, and 
multilevel software architecture. The grand challenges 
therefore span the domains of electromechanical de¬ 
sign, software design, and, due to the applied and in- 
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nately human-focused nature of rehabilitation robotics, 
all aspects of user interface design, including physical, 
communication, learning, emotional, and motivational 
factors. The first products in this field have come on 
the market in only the last 15 years; worldwide de¬ 
mographic trends will provide the force to accelerate 
product development in the future. 

For further investigation on rehabilitation robotics, 
there are three major sources of published information: 

1. Books on personal, service and rehabilitation 
robotics, such as: [64.300-302] 


2. Review articles in journals and periodicals, such 
as [64.303-307] and 

3. Articles that deal with individual topics, such as 
those in the reference list below, and conferences 
such as ICORR, RESNA, RO-MAN, BIOROB, and 
AAATE, which are also represented in the reference 
list. 

Cutting-edge research will be reported on the web¬ 
sites of investigators at academic, government and cor¬ 
porate research labs, and it is recommended to start at 
the sites of the researchers cited in this chapter. 
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The WREX exoskeleton 

available from http://handbookofrobotics.org/view-chapter/64/videodetails/499 
MIT Manus robotic therapy robot and other robots from the MIT group 
available from http://handbookofrobotics.org/view-chapter/64/videodetails/496 
MANUS assistive robot 

available from http://handbookofrobotics.org/view-chapter/64/videodetails/500 
The ArmeoSpring therapy exoskeleton 

available from http://handbookofrobotics.org/view-chapter/64/videodetails/502 
Lokomat 

available from http://handbookofrobotics.org/view-chapter/64/videodetaiis/503 
Gait Trainer GT1 

available from http://handbookofrobotics.org/view-chapter/64/videodetails/504 
Kineassist 

available from http://handbookofrobotics.org/view-chapter/64/videodetails/505 
Ekso 

available from http://handbookofrobotics.org/view-chapter/64/videodetails/507 
ReWalk 

available from http://handbookofrobotics.org/view-chapter/64/videodetails/508 
HAL 

available from http://handbookofrobotics.org/view-chapter/64/videodetaiis/509 
Indego 

available from http://handbookofrobotics.org/view-chapter/64/videodetails/510 
REX 

available from http://handbookofrobotics.org/view-chapter/64/videodetails/511 
Targetted reinnervation and the DEKA arm 

available from http://handbookofrobotics.org/view-chapter/64/videodetails/513 
PAM 

available from http://handbookofrobotics.org/view-chapter/64/videodetails/515 
The Arm Guide 

available from http://handbookofrobotics.org/view-chapter/64/videodetaiis/494 
ARMin plus HandSOME robotic therapy system 

available from http://handbookofrobotics.org/view-chapter/64/videodetails/497 
BONES and SUE exoskeletons for robotic therapy 

available from http://handbookofrobotics.org/view-chapter/64/videodetails/498 
The MIME rehabilitation therapy robot 

available from http://handbookofrobotics.org/view-chapter/64/videodetails/495 
Handsome exoskeleton 

available from http://handbookofrobotics.org/view-chapter/64/videodetaiis/568 
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65. Domestic Robotics 


Erwin Prassler, Mario E. Munich, Paolo Pirjanian, Kazuhiro Kosuge 


When the first edition of this book was published 
domestic robots were spoken of as a dream that 
was slowly becoming reality. At that time, in 2008, 
we looked back on more than twenty years of 
research and development in domestic robotics, 
especially in cleaning robotics. Although every¬ 
body expected cleaning to be the killer app for 
domestic robotics in the first half of these twenty 
years nothing big really happened. About ten years 
before the first edition of this book appeared, all 
of a sudden things started moving. Several small, 
but also some larger enterprises announced that 
they would soon launch domestic cleaning robots. 

The robotics community was anxiously awaiting 
these first cleaning robots and so were consumers. 

The big burst, however, was yet to come. The price 
tag of those cleaning robots was far beyond what 
people were willing to pay for a vacuum cleaner. 

It took another four years until, in 2002, a small 
and inexpensive device, which was not even called 
a cleaning robot, brought the first breakthrough: 

Roomba. Sales of the Roomba quickly passed the 
first million robots and increased rapidly. While for 
the first years after Roomba's release, the big play¬ 
ers remained on the sidelines, possibly to revise 
their own designs and, in particulartheir business 
models and price tags, some other small players 
followed quickly and came out with their own 
products. We reported about theses devices and 
their creators in the first edition. Since then the 
momentum in the field of domestics robotics has 
steadily increased. Nowadays most big appliance 
manufacturers have domestic cleaning robots in 
their portfolio. We are not only seeing more and 
more domestic cleaning robots and lawn mow¬ 
ers on the market, but we are also seeing new 
types of domestic robots, window cleaners, plant 
watering robots, tele-presence robots, domestic 
surveillance robots, and robotic sports devices. 65.3 


Some of these new types of domestic robots are still 
prototypes or concept studies. Others have already 
crossed the threshold to becoming commercial 
products. 

For the second edition of this chapter, we have 
decided to not only enumerate the devices that 
have emerged and survived in the past five years, 
but also to take a look back at how it all began, 
contrasting this retrospection with the burst of 
progress in the past five years in domestic cleaning 
robotics. We will not describe and discuss in detail 
every single cleaning robot that has seen the light 
of the day, but select those that are representative 
for the evolution of the technology as well as the 
market. We will also reserve some space for new 
types of mobile domestic robots, which will be the 
success stories or failures for the next edition of 
this chapter. Further we will look into nonmobile 
domestic robots, also called smart appliances, and 
examine their fate. Last but not least, we will 
look at the recent developments in the area of 
intelligent homes that surround and, at times, 
also control the mobile domestic robots and smart 
appliances described in the preceding sections. 
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65.1 Mobile Domestic Robotics 

The first mention of a domestic cleaning robot dates 
back to 1985. The device, nicknamed Robby, was devel¬ 
oped by Hitachi starting in 1983 and officially carried 
the name HCR-00 (Fig. 65.1). Robby was equipped 
with a gyroscope to keep track of its position and a ro¬ 
tating sonar scanner for obstacle detection. 

65.1.1 Domestic Floor Cleaning 

Very similar to today’s cleaning robots described fur¬ 
ther below, Robby already mapped its environment and 
used the map for path planning. HRC-00 remained 
a prototype like its successors HRC-01 to HRC-03. 

First Generation 

of Domestic Cleaning Robotics (1985-1999) 

Also in 1985, the Swedish appliance manufacturer 
Electrolux started the development of a concept vacuum 
cleaner Stardust. The device was equipped with eight 
fixed sonar sensors and one rotating sensors for obsta¬ 
cle detection. To maintain its orientation Stardust used 
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Fig. 65.1 First generation of domestic floor cleaning robots (1985- 
1999) 


an infrared sensor that tracked an infrared light bulb 
mounted to the ceiling. In 1988 Stardust was presented 
to the public at Domotechnica in Cologne, Germany, 
one of the largest fairs worldwide for domestic appli¬ 
ances [65.1]. 

About five years later, between 1989 and 1991, 
Panasonic undertook an effort to develop a domestic 
cleaning robot that led to two prototypes, one of them, 
Brownie, is shown in Fig. 65.1. Brownie was a battery- 
powered vacuum cleaner with a diameter of 40 cm, and 
at 18 kg, a super heavy weight compared to today’s 
domestic cleaning robots. It was equipped with a gyro¬ 
scope, a ring of sonar sensors, and a dust sensor. Likely, 
it was limitations in battery technology for driving such 
a heavy robot, as well as the total cost, which prevented 
Brownie from become anything more than a prototype. 

Six more years went by before Minolta presented 
its Minolta Cleaning Robot and the first European 
player, Electrolux, its Robot Vacuum Cleaner, later 
called Trilobite, which was presented to the public in 
BBC’s TV show Tomorrows World in 1996. 

The devices had already become significantly 
smaller and lighter in these six years. The Minolta robot 
had dimensions of 321 mm x 320mm x 170mm and 
weighted about 8 kg. It was powered by nickel metal 
hybrid battery and used sonar and tactile sensors for 
obstacle detection, a cliff-sensor to discover staircases, 
and a gyroscope to track its position and orientation. 
Electrolux’ robot also had a sophisticated sonar system, 
which let it follow contours and even return to a homing 
position. It already had the size and shape of later floor 
cleaning robots. 

Five more domestic cleaning robots made their first 
appearance during this time, which we call the in¬ 
fancy of domestic cleaning robotics. Two of them only 
reached the proof of concept level: AutoCleaner from 
InMach Intelligent Machines - the German startup later 
developed low-cost navigation systems for the profes¬ 
sional cleaning robot Robo40, which became commer¬ 
cially available in 2007 - and Koala form EPFL-Lami. 
AutoCleaner was the first domestic wet-cleaner proof 
of concept, which cleaned the floor using a rotating 
microfiber towel that was pulled through a water tank. 
Koala used a suction spout to reach corners. Two more 
robots reached at least the status of industrial proto¬ 
types, but were never commercialized: Dyson’s DC06 
and Cye from Probotics. DC06 was a unique domestic 
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cleaning robot. It differed not only in terms of the an¬ 
nounced list price of approximately US$ 4000, but also 
in many other aspects: Dyson claimed that it had three 
onboard CPUs, more than 50 sensors for obstacle avoid¬ 
ance, cliff-detection, localization and more. On top of 
that came the unique Dual Cyclone cleaning technol¬ 
ogy. The DC06 never made it to the market. Neither did 
Cye. Cye was a small mobile robot that could pull and 
push a (semi-)regular vacuum cleaner. It was announced 
as the first personal robot which could not only vacuum 
the floor but also serve coffee or deliver mail. It is un¬ 
clear if Cye ever delivered coffee or mail or vacuumed 
a real living room. It disappeared as quickly as it ap¬ 
peared. The last out of the nine first robots that we call 
first generation was the Karcher RoboCleaner. Its suc¬ 
cessor RC3000 was amongst the first cleaning robots 
that had a docking station at which the robot could not 
only recharge its battery but also unload the dust, which 
it had collected. We will come back to the Karcher 
RC3000 in the next section. 

The first generation domestic floor cleaning robots 
shown in Fig. 65.1 were the ones presented to the pub¬ 
lic. However, they were by far not the only ones that 
were developed in the period between 1985 and 1999. 
By browsing over the patents in the field of cleaning 
robots during that period it becomes obvious that many 
of the big appliance and electronic device manufactur¬ 
ers worldwide performed research and development on 
domestic cleaning robots. Besides the ones mentioned 
above, one can find such big names as Nintendo, Mat¬ 


sushita, Sanyo, Samsung, Honda, Procter & Gamble, 
Electrolux, Philips, Henkel and more. So, although only 
a few players dared to go public, domestic cleaning was 
already on the radar of many international enterprises. 
What kept them away from taking the next steps is not 
hard to guess. It was the risk of getting into a rather 
conservative and tight market - the cleaning business is 
extremely conservative - with a semimature technology 
carrying a price tag that looked quite differently than the 
one on well-established technology, such as traditional 
vacuum cleaners. 

The Second Generation 
of Domestic Cleaning Robotics (2000-2008) 
Figure 65.2 shows the second generation of domes¬ 
tic cleaning robots. What made the difference between 
the first and the second generation? The amount of ex¬ 
perimentation regarding the overall design decreased, 
with the majority of manufacturers settling on the disc 
shape. The number of sensors decreased, as well as 
their level of sophistication. Sophisticated and expen¬ 
sive sensors such as sonar largely disappeared. Most 
designs were limited to very few, simple, and inexpen¬ 
sive sensors such as bumpers, simple one-dimensional 
range sensors, typically based on infrared (IR), and 
cliff sensors. This was most likely a rather painful 
lesson learned from the designs in the first genera¬ 
tion. What roboticists considered the state of the art in 
mobile robotics: environmental sensing, map building, 
range sensors with high quality angular and range res- 
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olution, and combined localization and mapping, were 
far too expensive to be built into a domestic cleaning 
robot. Those robots had to compete with regular vac¬ 
uum cleaners in a price range of a few hundred dollars. 
This competition imposed very harsh cost limits for the 
robotic components that could be built into a domestic 
cleaning robot: 50 to 100 USD and often less. These 
cost constraints had a significant impact on a very fun¬ 
damental expectation of what cleaning robots should 
achieve: systematic coverage of the area to be cleaned. 
Cleaning was understood to be synonymous with cover¬ 
ing an area with a systematic and intuitive motion while 
applying some type of cleaning operation. Most of the 
patents filed for the first generation of domestic clean¬ 
ing robots mentioned above described cleaning robots 
that were supposed to cover their workspace in a sys¬ 
tematic fashion. 

Systematic coverage, however, is impossible with¬ 
out absolute positioning and without decent knowledge 
of the environment. The second generation of domes¬ 
tic cleaning robots had to provide a solution for this 
dilemma of systematic coverage with cheap and sim¬ 
ple sensors, which must not cost more than several tens 
of dollars. 

The solution to this dilemma was already proposed 
in some early prototypes. The idea was to waive the 
requirement of systematic coverage, which would also 
involve intuitive motion patterns, and instead produce 
semisyslematic coverage and semi-intuitive motion pat¬ 
terns. This semisystematic coverage was achieved by 
a combination of random motion, hard-coded motion 
patterns such as spiral- or meander-shaped motions, 
which - considered alone - reflects some system¬ 
atic coverage, and some other systematic and intuitive 
motion patterns, for example, following contours of 
walls or other objects in the workspace. The theory of 
stochastic process states that the mean squared distance 
of a particle that performs a random walk with respect 
to the origin of its motion increases proportionally with 
time. That implies that the particle moving in confined 
space will cover that space in a finite amount of time. 

First-generation domestic cleaning robots like the 
Electrolux Robotics Vacuum Cleaner (later Trilobite) or 
the Karcher Robot Cleaner (later Karcher RC 3000) al¬ 
ready used random motion combined with hard-coded 
motion patterns to achieve a certain degree of cover¬ 
age. But apparently the insight that the attribute robotic 
in front of vacuum cleaner was not enough of a sales 
argument to justify a price three to five times higher 
than that of a regular vacuum cleaner was not so easy to 
digest. Some of the second-generation cleaning robots 
were offered for a price on the order of 1500 USD or 
even more. In the end these robots shared the same fate: 
they become shelf warmers. 


This is also due to the fact that in 2002 a robot was 
launched which was to herald a breakthrough in domes¬ 
tic cleaning robotics: Roomba. Its creator, iRobot Inc., 
had learned the lesson that others were still struggling 
with: if you want to sell a domestic appliance, better 
sell it for a price that is known for domestic appliances. 
For the sake of fairness one has to mention that the tra¬ 
ditional vacuum cleaner manufacturers had to design 
the robots so they matched the quality and performance 
level that could be expected from the brand name. They 
could not take the risk of making their product too poor. 
Therefore it easily became over specified compared to 
what the market required at the time. New manufactur¬ 
ers such as iRobot in contrast had no brand name to 
defend [65.1]. 

If it is a totally new device, which may not only 
cause excitement but also concerns and reservations, 
sell it for less and not for more. When Roomba en¬ 
tered the market it was sold for 199 USD. That was 
a price, which did not cause customers to think about 
whether they really needed it, whether the quality was 
good enough, or whether the device would get into ev¬ 
ery comer of a room or underneath every couch or bed. 
The creators of Roomba were also smart enough not to 
call it a robot. This that prevented many customers from 
developing wrong expectation of what robots could or 
should do. 

The robot technology that was built into the 
Roomba was everything but new or revolutionary. 
Roomba used a suspended front-shield for contact¬ 
sensing, a low-cost infrared range-sensor for contour¬ 
following, a cliff-sensor to prevent it from falling down 
stair-cases, and it could detect photoelectric barriers, 
so-called virtual walls, which kept it from leaving 
a room or a certain part of its workspace. Roomba 
further used the combination of random motion and 
hardcoded area-covering motion patterns to achieve 
a certain level of coverage. All this technology was 
known before Roomba. 

Still Roomba can be seen as a milestone not only 
in domestic robotics but also in robotics at large. Why 
is that? It was the first time in robotics history that 
robotics was no longer synonymous with high-tech, 
high-price. Roomba showed that automation of every¬ 
day service tasks, such as domestic cleaning, could be 
achieved with a moderate effort in terms of hardware 
and at a decent price, given that one accepts some grace¬ 
ful degradation in the overall performance. There is no 
such thing as a free lunch, not even in robotics and 
this degradation in the performance is the price to be 
paid. What turned Roomba into a milestone was the 
cost-effective design, where the limitations of low-cost 
hardware - in Roomba’s case the sensors - were bal¬ 
anced out by smart heuristics for problem-solving. This 
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Table 65.1 Technical specification of a selection of second-generation domestic cleaning robots 



Manufacturer 

iRobot 

Sharper Image 

Electrolux 

Karcher 

Yujin 

Model 

Roomba 

eVac 

Trilobite 2.0 

RC 3000 

iclebo 

Drive 

Differential 

Differential 

Differential 

Differential 

Differential 

Sensors 

Suspended front shield 
as contact sensor, IR 
range sensor, four 

IR cliff sensors, dust 

sensor 

Touch sensor, cliff 
sensor, IR for wall¬ 
following 

180° sonar sensors 
(1 transmitter, 

8 receivers), infrared 
cliff sensor, mag¬ 
netic stripe detector, 
suspended front 
shield as contact 

sensor 

Suspended front 
shield as contact 
sensor, four IR cliff 

sensor 

Main brush, side 
brush anti-bacterial 

filter 

Mapping 
and localization 

n/a 

n/a 

n/a 

n/a 

n/a 

N avigation/coverage 

Random motion with 
bang and bounce, 
contour following, 
spiral motion 

Random motion 
with bang and 
bounce, spot clean¬ 
ing with star pattern 

Wall-following, 
random motion with 

obstacle avoidance 

Random motion 
with bang and 
bounce, spot clean¬ 
ing with see-saw 
motion pattern 

Heuristic pat-terns: 
random, circular, 
zig-zag, wall 
following 

Cleaning technology 

Side brush, two 
counter-rotating 
brushes, suction pump 

Rotating brushes, 
vacuum pump 

Rotating brush, 
suction pump 

Rotating brush, 
suction pump 

Main brush, side 
brush 

Run time (min) 

60-90 

15-45 

60 

20-60 

150 

Performance 

- 

- 

28 m 2 /h 

15 m 2 /h 

- 

Docking/recharging 

station 

Yes 

No 

Yes 

Yes 

no 

Size (0/h) (cm)/(cm) 

35/8,25 

32/14 

35/13 

28/10 

35/9 

Weight (kg) 

2,7 

3 

5 

2 

4 

Year of launch 

2002 

2004 

2004 

2003 

2005 

Price range 

250 USD 

100 USD 

1300 EUR 

1350 EUR 

530 USD 


design of a commercially viable product, at a price 
point competitive with nonrobotized solutions, turned 
Roomba into the first successful domestic cleaning 
robot, and the most frequently sold robot in the past 
50 years. 

In Table 65.1 we give an overview of the techni¬ 
cal specifications of some of these second generation 
robots. A more complete overview is included in 065- 
bib43 

The idea to develop and commercialize a domestic 
cleaning robot at a price level of 199 USD, comparable 
to the price of traditional domestic appliances, was very 
appealing to consumers. However, it was not an overly 
profitable business model, even if Roomba was manu¬ 
factured in a low-wage country at a cost of significantly 
less than 100 USD. Extreme cost pressure, however, of¬ 
ten compromises the quality of a product. So a decision 
to adhere to this low price of below 200 USD may also 
have been a decision against the quality of the prod¬ 


uct. Today nearly a dozen models of Roomba are on 
the market, which differ mostly in terms of their clean¬ 
ing technology and extra features. Their prices are in 
a range from 250 to 900 USD. 

The Third Generation 

of Domestic Cleaning Robotics (2009-2012) 

The second generation of domestic cleaning robots also 
provided their developers and distributors with a num¬ 
ber of painful and partly contradictory insights: 

• A domestic cleaning robot sold at a price that is 
significantly beyond the price of comparable non- 
robotic devices, runs a high risk of failing, because 
many customers may not be willing to pay extra 
money for a cleaning robot just because it is a robot. 

• A domestic cleaning robot sold at the same price 
level as an inexpensive comparable appliance, runs 
a high risk of failing, because the low price may 
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compromise the quality and functionality of the 
product, and many customers may not be willing 
to pay even a rather low price for a product that is 
known to be of poor quality. 

• For most customers - traditional, as well as 
technophiles - efficient cleaning requires systematic 
and efficient coverage of the workspace. A domes¬ 
tic cleaning robot that uses sophisticated sensors to 
achieve systematic and efficient coverage may eas¬ 
ily end up with a price tag that is significantly above 
the price of a comparable nonrobotic device and 
hence runs a high risk of failure. 

• A domestic cleaning robot, which refrains from 
using expensive sensors to achieve systematic cov¬ 
erage but instead uses low-cost sensors for collision 
avoidance, fall protection, and confinement as well 
as using random motion combined with some hard¬ 
coded motion patterns to achieve a certain level 
of coverage, runs a risk of not satisfying those 
customers who expect systematic cleaning and cov¬ 
erage. 

From a developers point of view these lessons sound 
as if customers expect not less than to square the circle. 
The truth is that customers do not care about squares 
and circles. They expect value for money. 

With the insights above one could classify potential 
customers of domestic cleaning robots into three cate¬ 
gories: 

• Customers who only care about the price and not 
so much about quality and functionality. The crite¬ 
rion for this class of customers was obvious: reduce 
the price as much as possible without ignoring that 
there may be a bottom line for what customers 
might expect in terms of quality and functionality. 

• Customers who care about the price and qual¬ 
ity, but are willing to adjust their expectations of 
functionality and efficiency. The criterion for such 
customers was to reduce the price but only to a cer¬ 
tain extent, which does not compromise the quality 
of the product too much. Expectations of such 
customers in terms of efficiency (i. e., systematic 
coverage) can be compensated by auxiliary equip¬ 
ment such as automatic charging stations. 

• Customers, who care about price, quality, and ef¬ 
ficiency. Apparently these customers are the most 
demanding. None of the second-generation domes¬ 
tic cleaning robots has really managed to satisfy 
them. The criterion to serve them would be to 
develop a low-cost navigation system based on low- 
cost sensors that provides systematic coverage. 

As satisfying the last group of customers, which 
is also likely the largest of the three groups, seemed 


nearly impossible, the third generation of domestic 
cleaning robots have diverged somewhat. Some man¬ 
ufacturers have focused on products that serve the 
first category of customers, some serve the second 
group and some even made an effort to square the 
circle. 

May they be successful in the end or not, what 
cannot be overlooked is the explosion of the number 
of manufacturers and distributors of domestic cleaning 
robots in the past five years. At the end of the 2012, 
twenty-seven years after the first mention of a domes¬ 
tic cleaning robot to the public, Amazon alone listed 
more than 131 results under the key word robot vacuum 
cleaner, with 14 manufacturers and suppliers. The busi¬ 
ness to business (B2B) portal www.made-in-china.com 
lists more than 71 companies for robot vacuum cleaner 
and a total of 875 products. Some of these 875 robot 
vacuum cleaners look surprisingly similar to the prod¬ 
ucts sold over B2C platforms under very-well known 
brand names. 

In Fig. 65.3 we show some of the products, which 
primarily intend to serve customers of the first cate¬ 
gory and therefore stayed under a price of 200 EUR. 
Figure 65.3a shows a model series of six cleaning 
robots made by XRobot of the Chinese manufacturer 
Shenzhen Silver Star Intelligent Electronics Ltd. In 
B2B trading these robots cost between 64 and 102 
USD per piece at a minimum order of 500 pieces. 
Most models of this series are also sold under the 
brand name of European and American enterprises. Fig¬ 
ure 65.3b shows another series of still rather cheap 
robotic vacuum cleaners. The attentive reader may no¬ 
tice that the robots in the left and the right figure 
are not totally unique. Some just have different names 
and different prices. This is not entirely unintended. 
It just illustrates that in domestic cleaning robotics 
the value creation chain is no longer limited to the 
developers and manufacturers, just as in every other 
business. 

Apart from a few minor details, all robots in 
Fig. 65.3 use very similar, though not to say the 
same, technology. They use very few and very cheap 
(contact, cliff, sometimes dirt) sensors and random 
motion combined with preprogrammed motion pat¬ 
terns. The vacuum technology consists of a rotating 
brush, sometimes combined with a small fan. Not sur¬ 
prisingly, robotic vacuum cleaners like the ones in 
Fig. 65.3 are often called Roomba-clones. The collec¬ 
tion of robots in Fig. 65.3 is neither representative nor 
comprehensive. 

Roomba has undoubtedly written robot history. It is 
the merit of Roomba and its developers that domestic 
cleaning robots are no longer considered as gadgets but 
as real appliances. Roomba has opened the door for all 
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O S' 

XRobot M-288 XRobot M-388 

$ 64/pc $ 64/pc 



# 

XRobot M-488 XRobot M-588 

$ 64/pc $ 64/pc 


XRobot M-788 
$ 102/pc 



XRobot XR201 
$ 99/pc 



Clatronic BSR 1282, 
€58 



Deebot D54, € 199 



Dirt Devil Libero, 
€ 101 


Vileda 137173, 
€ 129 



Philips Easy Star, Klarstein Cleanfriend, 

€ 177 € 135 


Fig. 65.3 Third generation 
of domestic floor cleaning 
robots: low-cost, low-tech 



Fig. 65.4 The seventh generation of 
Roombas 


the cleaning robots mentioned above, be they Roomba- 
clones or not. And Roomba is the by far the best-selling 
robot ever. 

In Fig. 65.4 we show some of the grandchildren and 
great-grandchildren of the very first Roomba, which 
started in 2002. The Roomba has gone through sev¬ 
eral facelifts in these ten years. The latest series, the 
Roomba 700 is the sixth generation of Roombas. It has 
matured quite a bit in many respects. It has matured 
in terms of handling, cleaning technology, obstacle 
avoidance and navigation technology. At the same time 
Roomba is delivered in more than a dozen versions, 
ranging from 250 to 900 USD. These versions differ in 
the design, in the sophistication of the user interface, in 
the navigation and coverage strategies, and in the details 
of the cleaning technology. 

According to iRobot more than 6 million units have 
been sold in the ten years of its existence. Roomba has 
certainly satisfied quite a number of customers. It is 
built with good quality, and comes with a self-charging 


home base and other auxiliary equipment. What has not 
changed, however, is the basic strategy for covering the 
workspace. The sixth generation just like the first uses 
random motion with some precoded motion patterns 
and heuristics. 

This raises one rather fundamental question. Will 
Roomba’s success story continue? Will it maintain 
its market position? Or has Roomba possibly reached 
or even passed its summit? These questions will not 
be answered before the third edition of this book, 
as any statement in this direction would be pure 
speculation. 

In any case, there are a few recent domestic cleaning 
robots, which demonstrate that covering a work space in 
a systematic fashion does not necessarily require expen¬ 
sive sensing, that would push the price for a domestic 
cleaning robot to a level which customers would not be 
willing to pay. 

Four different key technologies enable the devices 
shown in Table 65.2 to localize themselves and navigate 
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Table 65.2 Systematic cleaning at low-cost. The information in this table was partly collected from the websites and the technical 
documentation of the manufactures and partly from public websites such as www.botroom.com,staubsaugerroboter-test.org, 
www.robotreviews.com,www.geek.com,gizmodo.com, www.engadget.com 



Manufacturer 

LG 

Samsung 

Neato Robotics 

Iclebo 

Evolution 

Robotics/iRobot 

Distributor 



Vorwerk 

Philips 

Dirt Devil 

EVO/iRobot 

Model 

Hom-Bot 3.0 

Navibot SR 8895 

Silencio 

Neato XV-21 

Philips FC9910/01 
HomeRun/ iclebo 

smart 

Mint 4200 

CPU 

32 bit 

2 CPUs 



ARM7 

Drive 

Differential 

Differential 

Differential 

Differential 

Differential 

Sensors 

Camera (ceiling), 
camera (floor), sonar 
& infrared for colli¬ 
sion avoidance cliff 

sensors, gyroscope, 

acceleration 

Range sensors cam¬ 
era, cliff sensors, 
collision sensors, 
gyro 

1 -D laser range 
finder, cliff sensor, 
gyroscope, accelera¬ 
tion, 

Camera (ceiling), 
cliff sensor, infra 
red for collision 
detection, gyroscope. 

Northstar Indoor 
GPS, cliff sensors, 
bumper, gyro 

Mapping and local¬ 
ization 

SLAM 

SLAM based on 
ceiling pictures at 

30 fps (Visionary 
Mapping System) 

SLAM based on 
laser range finder 
(onboar d Room 
Positioning System 
(RPS)) 

SLAM based on 
ceiling pictures 

SLAM with North- 
star, map-building 

N avigation/coverage 

Systematic 

Systematic 

Systematic 

Systematic 

Systematic 

Cleaning technology 

Brush roller, 2 side 
brushes 

Brush roller, 2 side 
brushes 

Bristled brush 

Brush roller, 2 side 
brushes, micro-fiber 
mop 

Dry or wet (or pre¬ 
moistened) 
microfiber cloths 

Run time (min) 

75 

90 

90 

70 

180 

Performance 

- 

80m 2 /h 

- 

40m 2 /h 

- 

Docking/recharging 

station 

Yes 

Yes 

Yes 

Yes 

No 

Size (0/h) (cm)/(cm) 

36/9 

35/8 

34x 34/10 

35/10 

31x 11/29 

Weight (kg) 

3,2 

3,2 

5 

4 

2 

Year of launch 

2011 

2011 

2012 

2011 

2010 

Price range 

US$ 799.00 
(Amazon.com) 

EUR 349.00 
(Amazon.de) 

US$ 350.00 
(Amazon.com) 

EUR 499.00 
(Amazon.de) 

US$ 174.00 
(Amazon.com) 


in an unknown domestic workspace and systematically 
cover that space: 

• Visual odometry, with a camera pointing to the ceil¬ 
ing, combined with simultaneous localization and 
mapping (SLAM) used by LG Homebot, Samsung 
Navibot SR 8xxx, iclebo smart and Philips Home- 
run, 

• Localization, using infra read patterns projected to 
the ceiling, and mapping based on contact informa¬ 
tion used by Mint, 

• Proprioceptive motion estimation, using inexpen¬ 
sive inertial measurement units (including inexpen¬ 
sive gyroscopes and accelerometers) 


• Simultaneous localization and mapping (SLAM), 
using, for example a one-dimensional laser 
rangefinder like Neato XV-xx. 

These technologies by themselves are not entirely 
new. As a matter of fact methods such as simultaneous 
localization and mapping, visual odometry, and virtual 
landmark based navigation are not state of the technolo¬ 
gies form a scientific point of view. 

The true achievement of the developers of the above 
systems, which cannot be assessed high enough, is that 
they managed to reduce the cost of these technologies 
and at the same time make them robust enough for 24/7 
operation. These key technologies will be described in 






Domestic Robotics I 65.1 Mobile Domestic Robotics 


more detail in Sect. 65.1.2 Enabling Technologies. A 
review of domestic vacuum cleaning robots and some 
criteria to evaluate their performance are presented in 
|<©»M2H2Ea and kaXESSEl. 

65.1.2 Domestic Window Cleaning 

Window cleaning does not seem to be significantly 
more pleasant housework as floor cleaning. Notwith¬ 
standing, robotic window cleaners have not experi¬ 
enced similar attention or progress as floor cleaners. As 
a matter of fact, today there are only three commer¬ 
cial robotic window cleaners on the market, shown in 
Fig. 65.5. www.made-in-china.com does not even list 
one entry for window cleaning robot, although two of 
the three commercial products are Chinese brands. Why 
is that so, given the fact that domestic floor cleaning 
has become a billion-dollar business? The answer to 
this question has two parts: an economic and a technical 
one. 

The economic one is that windows in private homes 
are cleaned far less often than floors. Customers may 
hesitate to buy expensive equipment for a task that 
needs to be done once a month or even less. So the mar¬ 
ket for domestic window cleaners is presumably much 
smaller than that for floor cleaners. 

The technical one refers to the technical hurdles 
that have to be overcome. The technical problems that 
robotic window cleaners have to face are: 

• Adhesion to a vertical, fragile surface, which pos¬ 
sibly needs to be moistened in order to be cleaned, 
and related, the power supply necessary to produce 
that adhesion. 

• Requirements for cleaning performance; people 
may tolerate if the floor is not 100% clean, but 
nobody would buy a window cleaner that leaves 
streaks on the window. 

While floor cleaners do not bother about gravity and 
falling down unless they are near staircases or ledges, 
gravity is an essential problem for window-cleaning 
robots, and the solutions are usually not very cheap. 
Special mechanisms have to be designed for secure mo¬ 
tion. Typically special tether mechanisms prevent the 
robots from falling. Special locomotion mechanisms 
have to create enough adhesion force to hold a robot 
attached to a flat, vertical, damageable surface such as 
glass and at the same time move the body up and down 
and sideways. These mechanisms have to be small and 
light and create enough adhesion forces as well as low 
energy and resource consumption. 

The two robotic window cleaners shown in 
Fig. 65.6, RACOON and QUIRL, are research proto¬ 
types, which were developed by Fraunhofer Institute for 


Manufacturing Engineering and Automation (IPA) in 
Germany. In these prototypes the adhesion problem was 
solved by means of suction cups. RACOON used cater¬ 
pillar drives that were equipped with passive suction 
cups. Passive means that the system does not actively 
create a vacuum in the cup. Rather a small valve aerates 
or seals the suction cup depending on the position of the 
cup along the drive. Drives with passive suction cups 
have the advantage of moderate energy consumption. 
They have, however, one severe disadvantage. They 
tend to lose their adhesion after a while. The reason for 
this is the torque that acts on the center of gravity of the 
system. Due to this torque there is a traction force act¬ 
ing on the upper cups while at the same time pressure is 
exerted on the lower cups. Without any attractive force 
acting on the upper cups the adhesion there gets weaker 
and eventually the system falls. Therefore passive suc¬ 
tion cups are rarely practical. 

An apparent solution to this problem is the use of 
active suction pumps, which generate a vacuum un¬ 
der the upper suction cups. This solution prevents the 
system from falling. However, supplying the vacuum 
to the cups makes the system significantly more com¬ 
plex, heavier, and larger. Researchers at Fraunhofer 
IPA [65.2] have therefore invented a smart solution, 
which gets by with passive cups, but gets around the 
problem of decreasing adhesion. The solution uses 
a spacer at the rear of the vehicle. This spacer neu¬ 
tralizes the torque around the center of gravity which 
is typical for a systems with passive suctions cups. 
The spacer causes a traction force which acts on the 
lower suction cups. This traction force creates a torque 
around the spacer, which counteracts the torque around 
the center of gravity and also causes a pressure on the 
upper suction cubs. RACOON was presented at the 
Hannover Fair in 2002 and got quite some attention. But 
it never became a product. Neither became its successor 
QUIRE. 

In QUIRE, the number of components, the weight, 
and the size of the system were significantly optimized. 
The main functions cleaning, holding, and moving were 
unified in one single component. QUIRE consisted of 




Fig. 65.6 (a) RACOON and (b) QUIRL, two early research proto 
types of domestic window cleaning robots 
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Fig.65.5a-c Commercial robotics 
window cleaners (a) Windoro, (b) 
Winbot, (c) Hobot 


two vacuum cups that were attached to a common frame 
which were driven by two separate motors, which ro¬ 
tated independently of each other. The overall motion 
of QUIRL could be controlled by selecting the veloci¬ 
ties and rotational directions of the vacuum cups. If the 
motor of one vacuum cup was turned off and the cup did 
not rotate, QUIRL rotated around this fixed cup. If both 
motors and cups rotated in the same direction this led to 
an overall rotation of QUIRL about its vertical axis. If 
both drives rotated in the opposite direction at exactly 
the same velocity then QUIRL made a linear motion. If 
both drives rotated in the opposite direction but their ve¬ 
locities were not identical then the translational motion 
was superimposed by a rotational motion and QUIRL 
moved along a curved trajectory. In order to clean the 
surface some cleaning mechanism or tool needed to be 
fixed to the vacuum cups. By attaching, for example, 
specific cleaning towels in the cups the abrasion effect 
was increased and a very good cleaning performance 
could be achieved. 

The field of robotic window cleaning has not made 
as much progress as domestic floor cleaning, but it has 
still made progress. In Fig. 65.5 we show three com¬ 
mercially available window cleaners - Windoro WCR 
1001, Winbot 7, and Hobot 168 - available today. 

In 2010 the South-Korean enterprise Ilshim Global 
Co. Ltd. introduced its commercial window cleaner 
Windoro (WCR). Windoro consists of two modules, 
a navigator module and a cleaning module that are held 
together by two strong neodymium permanent magnets, 
whose distance can be adjusted. The navigator mod¬ 
ule and the cleaning module are placed on the inner 
and outer side of the window, respectively, like a sand¬ 
wich with the glass pane in between. The two modules 
operate as a tandem. The navigator module has a differ¬ 
ential drive system with two wide-based rubber wheels. 
Also the cleaning unit moves on two rubber wheels, 
which serve as spacers for the four spinning cleaning 
pads. When the navigator module moves, the cleaning 
module on the other side of the window moves with it. 
The linkage by permanent magnets obviously has one 
rather fundamental advantage. As long as the window 
does not exceed a certain thickness, namely 28 mm, 
the permanent magnets hold the robot safely attached 


to the window. Windoro cannot fall, even if its bat¬ 
tery runs out of power. For that reason Windoro does 
not need any safety mechanism like a safety rope. Un¬ 
fortunately there is a price to pay for this: it is very 
difficult and unhandy if not impossible to use Win¬ 
doro if you cannot open the window that you want to 
clean. 

Windoro first explores the width and height of the 
window before it then starts to clean the window from 
the top to the bottom in a zig-zag motion, with a veloc¬ 
ity of 8 cm/s. While the robot moves over the window 
the cleaning module sprays a cleaning solution onto 
the window surface. With four spinning microfiber pads 
the cleaning module removes dirt and sprayed solution 
from the surface. 

The second commercial window-cleaning robot is 
Winbot, a product of the Chinese company Ecovacs, 
which has established its own brand. Winbot was first 
introduced in 2011. The most recent version Winbot 
7 was presented at the electronics fair CES (Con¬ 
sumer Electronics Show) 2013 in Las Vegas. Ecovacs 
also manufactures and distributes the robotic vacuum 
cleaner family Deebot. 

Winbot uses two suction rings and a vacuum pump 
for the adhesion at the window. The outer ring also 
serves as a safety mechanism. If the air pressure in 
the ring increases, that means Winbot has reached the 
edge of a window plane, it backs up, turns around and 
moves in a different direction. A second suction cup 
serves as a safety anchor for Winbot while it moves. It 
is connected to Winbot by a safety rope and catches the 
robot if it falls. Winbot can move at a velocity of ap¬ 
proximately 15 cm/s. Its drive system consists of two 
differentially driven anti-slip rubber tracks. 

After it has been turned on, Winbot, like Windoro, 
first explores the height and width of the window, then 
calculates a zig-zag path that covers the window area 
and finally executes this path. Winbot has no active 
cleaning technology such as spinning pads. It uses two 
micro-fiber towels that are attached to two plates at the 
front and at the rear of the robot. The micro-fiber towel 
in the front has to be moistened before Winbot starts 
moving. It resolves and removes the dirt. A rubber blade 
behind the front towel removes the remaining moisture. 
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Aquabot turbo, Aquabot 



Aquabot Alpha 
Aqutron robotic systems 



TigerShark, Hayward 



Vero pool cleaner, 
iRobot 



Dolphin Dynamic Plus, Maytronic 



SwimBot Pro Pool 
Cleaning Robot, Swimob 



Dirt Devil, Rampage 
Robotic pool cleaner 



Polaris 9400, 
Zodiac Pool Systems 


Fig. 65.7 Robotics pool cleaners 


The micro-fiber towel in the rear of the robot finally 
dries the window. 

The third, and most recent, system, Hobot 168, is 
a product of the Taiwanese company Hobot Technology 
Inc. It was first presented at IFA 2012 (Internationale 
Funk Ausstellung) in Berlin and launched in summer 
2013. It looks surprisingly similar to Quirl in Fig. 65.5 
and apparently uses a similar adhesion and locomotion 
mechanism, namely two rotating vacuum cubs, to move 
across the window. 

The preceding paragraphs read as if the times are 
over where windows had to be cleaned by hand. Re¬ 
grettably this is not the case. Although all three devices 
reasonably solved the adhesion problem their overall 
performance is modest. Several tests by housewives 
and magazines came to the same sobering conclusion: 
There is a lot of noise and very little cleaning. As 
a matter of fact, although the navigation problem on 
a vertical rectangular surface, which is free of obstacles 
apart from the window frame, seems to be a solv¬ 
able one, Hobot 168 and Winbot showed a rather poor 
performance in terms of systematic coverage. After 
a semisuccessful effort to explore the width and height 
of the window, Winbot, in one of the tests, moved more 
or less erratically for several minutes, before it gave 
up somewhere along the road. Several of the currently 
available commercial window cleaning robots are re¬ 
viewed in laa jmUHjgHl , 
and ksMZEEiiaa. 

65.1.3 Pool Cleaning 

While robotic floor cleaner and window cleaner were 
still struggling to get rid of the image of only being 
the crazy ideas of engineers and researchers at the be¬ 


ginning of the millennium, pool-cleaning robots were 
already well-established products. This may be due to 
the fact that the challenge of cleaning a rectangular pool 
is rather modest and so is the robotic technology used in 
robotic pool cleaners. It may also be due to the fact that 
pool owners belong to a class of customers who did not 
create the same price pressure as the ordinary house¬ 
wife. 

First patents on self-propelled pool cleaning devices 
date back to 1965, three years after the first indus¬ 
trial robot was installed. Ferdinand Chauvier, a South 
African engineer, could possibly be considered the 
father of automated pool cleaning. He developed sev¬ 
eral generations of devices for pool cleaning before 
he finally marketed Kreepy Krauly, the first automated 
pool cleaner in 1974. Kreepy Krauly was not only 
the first one of its kind but also the very first domes¬ 
tic service robot ever, 15 years before Joe Engelberg 
published his book Robots in Service at MIT Press in 
1989 and coined the term service robot. Since then 
the technology of pool cleaning robots hasn’t changed 
much. 

As one can see in Fig. 65.7, most pool cleaners have 
track drives, which are operated differentially. Typi¬ 
cally the motors that drive the tracks are also connected 
to the front and rear scrubbing brushes, which clean 
the pool surface, while the robot moves. While tracked 
drive system were rather common for the early gener¬ 
ations of pool cleaners, newer pool cleaners also use 
wheel drives, for example, the Polaris 9400 from Zodiac 
Pool Systems. A wheel drive can be advantageous be¬ 
cause the space between the bottom of the robot and the 
ground allows a better water flow and a higher through¬ 
put. Also Zodic claims that Polaris 9400 has higher 
maneuverability. 
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Fig. 65.8 Example of the navigation and cleaning strategy 
of a robotic pool cleaner 


Since most sensors, which are used in mobile robots 
or in aerial robots do not work under water, the sensor 
modalities that are used by pool cleaners are manage¬ 
able. Unfortunately the manufacturers do not disclose 
much about the sensor technology and technology in 
general used in their system. So we need to speculate 
a bit about how the behaviors shown in the commercials 
are internally implemented. 

Most pool cleaners claim to be capable of avoiding 
obstacles. When they sense a collision, the back up for 
a certain distance, reverse the direction of motion and 
then continue, possibly on a track parallel to the one 
the led to the obstacle. They can recognize the walls of 
a pool, which form the borders of their workspace. They 
can even climb up the walls of a pool, float along the 
perimeter and then submerge again. They can explore 
the length and width of a pool. They can drive a certain 
distance before they change direction. 

All these behaviors require a combination of several 
of the following sensors: odometry to measure traveled 
distance, an inclinometer to sense if the robot starts 
moving upward, e.g., when it keeps pushing against 
a wall and the front wheels start moving in a vertical 
direction, contact sensors to detect when the robot col¬ 
lides with an obstacle - be it a wall of the pool or a real 
obstacle on the ground of the pool - sensor to measure 
the motor current , which can be used in addition or in¬ 
stead of a collision sensor to detect if the robot pushes 
against an object, and possibly an inertial measurement 
unit to correct the heading when driving. Polaris and 
possibly other systems as well use an accelerometer to 
constantly determine its position in the pool. A sensor 
modality, which might be used underwater for obsta¬ 
cle detection and avoidance is laser, with wavelengths 
in the lower nano-meter range (e.g., 405 nm blue laser). 
But it is unknown if this principle has been considered 
for pool-cleaners. 

In terms of navigation and coverage pool cleaners 
follow similar strategies to those floor cleaners, which 
we roughly classified as Roomba-clones. Earlier pool 


cleaners used random motion pattern. The newer ones 
shown in Fig. 65.7 use certain heuristics and strategies 
to perform some form of localization and exploration of 
the pool. The Maytronic’s Dolphin, for example, first 
explores the length and the width of a pool. After be¬ 
ing dropped into the pool and floating to the bottom of 
the floor it crosses the pool until it hits the first wall. 
With the support of a thruster it climbs up the wall until 
it reaches the surface. There it hovers to the side be¬ 
fore it submerges again and moves back to the ground 
of the pool. Next it moves to the opposite side of the 
pool, climbs up the wall, hovers to the side and glides 
back to the ground. While moving on the ground from 
one wall to the other Dolphin measures the distance 
from wall to wall. Once it glides back from the sec¬ 
ond wall it moves halfway back toward the other wall. 
It then makes a 90° turn and repeats the exploration for 
the second set of opposite walls. After the exploration 
is completed. Dolphin knows the length and width of 
the pool and plans a pattern of parallel and orthogonal 
tracks as shown in Fig. 65.8 that in the end covers the 
entire pool. 

Removing dirt and debris from the ground and the 
walls of a pool, requires loosening the dirt - if it is not 
loose already - and to soak it into a container, other¬ 
wise the dirt would only be circulated in the pool. The 
container is typically a jet pipe with a filter at one end 
that holds the dirt back while the water flows through 
the pipe back into the pool. Soaking in water and debris 
from the floor into a filter and pumping it back into the 
pool requires significant suction power. This is the rea¬ 
son why pool cleaners typically have an external power 
supply. This motor serves as a pump and as a thruster at 
the same time. Both effects together allow pool cleaners 
to easily climb up vertical walls. Using inflation at the 
bottom of the robot, and a water jet ejected at the top, 
enough traction power is created that both tracked and 
wheeled pool cleaners can drive up vertical surfaces. 

As mentioned earlier, the cleaning technology, be¬ 
sides the water pump and the filter to keep back the 
debris, consists of a system of counter-rotating rubber 
brushes which brush the debris underneath the pool 
cleaner, where it ends up in the intake socket of the 
water pump. |o>Jfc4iiLIil£n and loMEEilBil compare 
a selection of home pool cleaning robots. 

65.1.4 Lawn Mowing 

Together with robotic pool cleaners and domestic floor 
cleaners, robotic lawnmowers today count as regular 
everyday products. People no longer consider them 
mystical pieces of technology, which have their own life 
and which at times become so autonomous that the user 
no longer knows what they are after. 
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Fig. 65.9 Example for virtual fence 
and coverage strategy (random 
motion) of robotics lawnmowers 


It is not surprising that robotic lawnmowers have 
a lot in common with domestic floor cleaners. They 
have to cover a workspace of a certain size with as lit¬ 
tle interaction with the owner as possible. They have to 
perform a certain operation to the surface such as clean¬ 
ing the floor or mowing the lawn. They must not collide 
with any obstacles and if they do, they should at least 
not cause any damage. They must not get stuck any¬ 
where in the environment and they should not leave the 
workspace without authorization. 

The challenge for lawnmowers, much the same for 
domestic floor cleaners, is the systematic coverage of 
the workspace, which in turn requires precise position¬ 
ing and mapping of the workspace. Given the fact that 
the price pressure is not as back-breaking as for robotic 
floor cleaners - there are not that many robotic lawn- 
mowers which costs less than 1000 EUR - why not 
invest a little more in sensing and especially in position 
sensing and obtain a decent solution for the localization 
and coverage problem? 

The answer to this question is: things are not that 
simple. Lawnmowers operate outdoors and none of the 
solution developed for the floor-cleaners in Fig. 65.2 
will work. Regular GPS (global positioning system) has 
an accuracy of several meters and has a tendency to 
deliver erratic readings, which would lead to equally 
erratic motions of the lawnmowers. As a matter of fact 
Automower 220 from Husquana uses GPS, but only as 
an antitheft protection device. Differential GPS, which 
would provide accuracy below one meter, would be 
too expensive for an affordable robotics lawnmower. In 
a nutshell, absolute positioning or SLAM, which would 
be necessary to cover a large outdoor area like a garden 
in a systematic fashion, is not practical. 

In order to cover their workspace robotics lawn- 
mowers use similar strategies to the Roomba-like floor 
cleaners. They refer to heuristics, which do not provide 
an optimal performance but still show a decent result. 
Using sensors such as gyroscopes, digital compasses, or 
inertial measurement units, robotic lawnmowers follow 
a certain heading and cover the workspace as much as 
possible by parallel tracks. They move along a straight 
line until they hit an obstacle or reach the border of 
the workspace. There they back up to become clear 


from the obstacle, make a U-turn by 180° and drive 
back the way they came. Another heuristic that is fre¬ 
quently applied by robotic lawnmowers is the random 
motion shown also in Fig. 65.9. The mower moves 
along a straight line until it hits an obstacle or reaches 
the border of the workspace but then it does not just 
reverse but chooses a new direction randomly. 

Since robotic lawnmowers move in open space, 
there is a danger that they leave their workspace and 
travel to areas where they are not supposed to be. For 
floor cleaners so-called virtual walls or fences solve 
this problem. Virtual walls are realized by infra read 
light beams emitted by some projectors, which can be 
placed in the workspace. The robots can sense these 
infrared light beams and consider them as obstacles, 
which evoke the typical obstacle avoidance behaviors. 
Lawnmowers use a similar technique, which is based 
on induction rather than light. To mark the border of the 
robot’s workspace the owner has to place a wire around 
the area, which the robot must not leave (Fig. 65.9). 
This wire is connected to a low-voltage alternating cur¬ 
rent source. When the robot approaches the wire an 
inductivity sensor senses the current in the wire and 
causes the robot to reverse. Today most lawn mowers 
use a more sophisticated so-called true in/out sys¬ 
tems, where the position is permanently tracked, not 
only when the robot approaches or passes the virtual 
fence [65.3]. 

The very first robotic lawnmowers had to be 
recharged manually. To avoid an all too frequent in¬ 
volvement of the human into the operation, Husq- 
varna - one of the pioneers in robotic lawn mowing - 
equipped its first mower with solar panels and called 
it SolarMower. SolarMower was released in 1995, and 
was one of the first robotics lawnmowers. Nowadays, 
all lawnmowers come with a base station where they 
can recharge their batteries without human intervention. 
Also solar panels are coming back as power source, 
however only as auxiliary power to increase the per¬ 
formance and runtime between two charges and not as 
main power supply. 

Given the fact, that robotic lawnmowers do have 
poor positioning capabilities, it is somewhat tricky to 
guide them back to the docking station once the battery 
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Automower 220, 
Husquama 



Robomow, 
Friendly Robotics 


Ambrogio, Zucchetti 


Tango, John Deere 



Fig. 65.10 Robotic lawn- 
mower 



gets low and needs to be recharged. Some of the lawn- 
mowers shown in Fig. 65.10 use special wires which 
radiate from the position of the docking station. This 
way a robot only needs to follow such a radiating wire 
to return to the docking station on the fastest way. An¬ 
other strategy is to follow the wire at the workspace 
border. That will eventually lead the robot back to the 
docking station. 

An exception is the Bosch Indego, which is 
equipped with mapping and localization capabilities. 
This allows Indego to plan a path that leads to a point 
close to the docking station. Then the robot can follow 
a wire for the docking maneuver [65.1]. 

To comply with the safety regulations and to avoid 
any injuries of humans or animals the cutting mecha¬ 
nism has to be very lightweight and designed such that 
it is guaranteed to stop if the device tilts or is lifted. 
The cutting mechanism of robotic lawnmowers typi¬ 
cally consists of a rotating disc with three razor-like 
blades, which automatically retract into their mountings 
if the robot is stopped unscheduled, for example, if it 
hits an obstacle or is lifted. 

Naturally the lightweight design limits the thickness 
of the grass that can be cut. Also it is important for 
the proper functioning of robotic lawnmowers that the 
lawn, which is to be cut, is not too high. This in turn re¬ 
quires a rather regular if not continuous operation of the 
lawnmowers. With regular use of the lawnmowers the 
grass cuttings are short enough to quickly decompose 
into nutritious compost, so there is no need to remove 
the cuttings after the lawn is mowed. 

Figure 65.10 shows some of the better-known 
robotic lawnmowers on the market today. Besides the 
pioneers in robotic lawn mowing Husqvama, Friendly 
Robotics and Zuchetti also a number of new players 
have entered the market, most noticeably the Ger¬ 
man automotive supplier Bosch and the Japanese car 
manufacturer Honda, which has set a milestone in non¬ 
industrial robotics with its humanoid robot Asimo. 


We certainly do not claim that the collection in 
Fig. 65.10 is complete. Like for domestic floor cleaning, 
Chinese B2B portals such as www.made-in-china.com 
or www.alibaba.com list around a hundred products 
under the category of robotic lawnmowers and 35 sup¬ 
pliers; the market development for robotic lawnmowers 
in the past five years was not quite as overwhelming as 
for domestic floor cleaners but was still remarkable. In 
Table 65.3 we show the technical specifications of a se¬ 
lection of lawnmowers from Fig. 65.10. 

65.1.5 Sports Robotics 

A subdomain of domestic robotics that was not included 
in the first edition of this handbook, because it virtually 
did not exist, or it was not visible at that time, is sports 
robotics. What is a sports robot? Since an official defini¬ 
tion of this term does not yet exist - at least we haven’t 
found one - we take the liberty and provide such a defi¬ 
nition here. We define a sports robot as a robotic device , 
which either supports the human user in their physical 
exercises as a coach or a companion , or acts as an oppo¬ 
nent in a game. An important aspect of this definition is 
the physical exercise of the human, which is supported 
or challenged. 

We would like to emphasize that the above def¬ 
inition does not include any form of entertainment 
robots, which play games such as soccer against each 
other but do not involve any human activity other than 
watching. 

In Fig. 65.11 we show three examples of robotic 
baseball players. As the name says the Headless Bats¬ 
man can act as a batter at least for exercising. It rather 
successfully hits baseballs thrown at it in many ways. 
The kinematic structure consists of two arms, one leg 
and no head, as its developer, an industrial designer 
from Robocross, likes to call it, and is made from auto 
parts, steel pipes, and pneumatic hoses. The pneumatic 
hoses are parts of its pneumatic actuation by an air 
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Headless Batsman Pitching Machine 


PhillieBot - a robotic pitcher 


Fig. 65.11 Robotic baseball players 


Table 65.3 Technical specification of a selection of robotic lawnmowers 



Manufacturer 

Husqvama 

Friendly Robotics 

Zuchetti 

Bosch 

Honda 

Model 

Automower 220 AC 

Robomow RL1000 

Ambroggio 

Indego 

Miimo 

CPU 




ARM9 (32 bit) + 
PowerPC (32 bit) 


Drive 

Differential 

Differential 

Differential 4 driving 
wheels 

Differential 

Differential 

Sensors 

Lift sensor colli¬ 
sion/contact sensor 

Rain sensor, lift sen¬ 
sor, collision/contact 

sensor 

Grass sensors, safety 
sensors on handles, 
lift sensor, sensor for 
collision sensor and 

rollover 

Lift sensor, colli¬ 
sion/contact sensor, 
gyro, tilt sensor 

3 x 360 degree bump 
sensors, 2 lift sensors 

N avigation/coverage 

Random 

Random 


Logicut navigation 
system: parallel 

Random, parallel or 
mixed 


tracks in parcels; lo¬ 
calization, mapping, 






path planning 


Cutting technology 

3 pivoting razor 
blades 

3 razor blades 

Rotating disc 

3 pivoting blades 

3 blades, which 
bend on hitting hare 
objects 

Virtual fence and 
guides 

Dual guide wires to 
return to the base 

station 

Boundary wire 

Boundary wire 

Boundary wire 

Boundary wire 

Performance (m 2 ) 

1800 (m 2 ) 

2000 (m 2 ) 

400 (m 2 ) 

1000 (nr) 

3000 (m 2 ) 

Ground speed 



18 m/min 

27 m/min 


Motor power (W) 


3x150 


2 X 20 + 80 

2x25 + 56 

Battery type and 
capacity 

NiMH 30 W 


Li-Ion, 6.9Ah 

32 V Li-Ion, 3Ah 

Li-Ion 

Run time 

45 min 

2 h 30 min 

3 h 30 min 

50 min 


Docking/recharging 

station 

Automatic docking 
and recharching 


No 

Automatic docking 
and recharching 

Automatic docking 
and recharching 

Theft protection 

Pin code lock 

Personal password 


Alarm & Pin code 

Pin code lock 

Size (LxWxH) 

71x55x30 

87,5x65x31 




Weight (kg) 

10 

25 

7,9 

11.1 


Year of launch 




2012 


Price range 

~$ 2400,00 

$ 1900,00 


1850,00 

~$ 2600,00 


compressor. The Headless Batsman is fully controlled three buttons: one to control the robot’s hip, a sec- 
by a human operator via a remote control, which has ond one that actuates the arms and a third one to lift 


1743 


Part F | 65.1 



















Part F | 65.1 


1744 Part F 


Robots at Work 



JoggoBot - a jogging companion RUFUS - a robotic running coach 


Fig. 65.12 (a) JoggoBot and (b) RUFUS two robotic sports com¬ 
panions 

and drop the inside shoulder, changing the trajectory of 
its swing. Similar mechanisms like Headless Batsman 
were developed at Hiroshima University and Tokyo 
University. 

The counterpart of Headless Batsman is a pitch¬ 
ing machine, which is operated with the same remote 
control. The barrel of the machine is sawn-off fire ex¬ 
tinguisher whose other end is directly connected to 
a one-inch port poppet valve, which in turn is con¬ 
nected to an 8 bar air pressure tank. The air pressure 
tank is supplied from a screw compressor. The charging 
mechanism consists of a double rod actuator with a ring 
welded horizontally on to the end. Mounted above the 
ring is a magazine that holds up to 10 balls, which di¬ 
rectly drop into the ring. For a pitch the actuator moves 
the ring with the ball over the edge of the barrel, where 
it falls on to a small guide. From there the balls rolls 
backward into the barrel. The actuated ring is covered 
by a steel strip, which holds back the other balls in the 
magazine until the actuated ring returns to its initial po¬ 
sition. Since both mechanisms are remotely controlled, 
the human, who controls them, can actually sit in a chair 
next to the playground and let the robots play against 
each other. Such a use would clearly violate the above 
definition of a sports robot but does not seem to be the 
one that is primarily intended. 

Another robot that can throw baseballs is Philliebot 
developed in University of Pennsylva¬ 
nia’s GRASP laboratory. Philliebot was developed in 
a couple of weeks using only spare parts in the GRASP 
lab. It uses a Segway as mobile base, a Barret arm, and 
pneumatically actuated wrist to create the necessary dy¬ 
namics for the pitch. When the button is pressed the 
arm moves to the back of the robot and then acceler¬ 
ates its motion toward the target of its pitch. When the 
arm reaches the highest point of motion the pneumatic 
wrist cylinder delivers a burst of compressed carbon 
dioxide to snap the wrist forward and release the ball. 
What remains is the question of why use robotic equip¬ 
ment worth several tens of thousands of dollars to throw 


a baseball, given that pitching machines have existed for 
many years. According to the developers, the fact that 
Philliebot is mobile and its software can be tweaked to 
vary pitch velocity and trajectory was enough to justify 
the experimentation. 

The two sports robots shown in Fig. 65.12 do not 
really seduce their users to sit in a chair and relax. They 
both serve as so-called robotic running coaches or run¬ 
ning companions. 

Researchers at the Royal Melbourne Institute of 
Technology in Australia have redesigned a commer¬ 
cially available Parrot AR Drone quadrocopter and 
turned it into an autonomous, flying running partner 
for joggers, called Joggobot [65.4]. Joggobot uses an 
integrated, front-mounted camera to detect and track 
a special patter printed on the T-shirt worn by the jog¬ 
ger. Joggobot takes off when the camera registers the 
pattern and rises to about the same height as the pattern 
on the t-shirt. An internal sensor determines Joggobot’s 
altitude. Joggobot can be set into a companion mode, in 
which at flies at a steady pace at a relative distance of 
about three meters to the jogger or in a coach mode in 
which it flies at a slightly more challenging speed. 

There are two features of Joggobot that make the de¬ 
vice somewhat limited: first the capacity of the battery 
limits the flight time to 20 min, which in turns limits 
the time for exercise; for a short run, this is certainly 
ok, but for serious training this is insufficient. Second, 
Joggobot can only fly in a straight line, to let Joggobot 
follow an arbitrary path the jogger needs to remotely 
control Joggobot’s flight path. 

A slightly different concept of a jogging companion 
is pursued in RUFUS (l‘S3>itiiili!l£IM ), which is devel¬ 
oped by runfun (www.runfun.com), a German startup 
company. RUFUS is an electrically driven, automati¬ 
cally guided, ground vehicle that supports and guides 
a runner during his/her training. RUFUS plays the role 
of a personal running coach. It fulfills a similar function 
to a treadmill, which exposes its user to a varying strain 
by varying its velocity and inclination and thereby im¬ 
proves the fitness, endurance, and resilience of the 
cardio-vascular system of the user. Unlike a treadmill, 
RUFUS is not a stationary device, however. It drives 
ahead of the runner like a pace maker in a marathon or 
a fake rabbit in a dog race, and sets the runner’s speed. 
RUFUS’s velocity is set either manually or automati¬ 
cally via a training program. 

If operated in manual mode the velocity is either set 
directly as a velocity set point or indirectly as a heart¬ 
beat set point. If the training guidance is based on the 
heartbeat, then RUFUS controls its velocity such that 
the runner exercises optimally and continuously within 
a certain heartbeat interval under a moderate stain of the 
cardiovascular system. 
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Double Robotics, USA 





Beam RPD, $ 16 000 
Suitable Technologies, USA 
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Vgo, $6000 

Vgo Communications, USA 



RP-VITA Robot, $ 4700 per month 
InTouch Health/iRobot, USA 
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QB Avatar, $ 9700 
Anybots, USA 



Giraff, price unknown 
Giraff Technologies, Sweden 


Fig. 65.13 Tele-presence robots and robotic avatars 


This has a twofold use: on the one hand this prevents 
users from overstressing themselves through overam- 
bitious and intensive training modules, possibly from 
even injuring him or herself. Such a protection func¬ 
tion is beneficial for unfit or untrained runners. On 
the other hand RUFUS facilitates optimal training ef¬ 
fect and progress through a careful guidance of the 
training. 

The training effect can be further improved if RU¬ 
FUS is operated in the program mode instead of the 
manual mode. In this mode RUFUS executes complete 
training modules, for example pyramid speed interval 
workouts, which are customized to the user. Such train¬ 
ing modules are typically elaborated on by physiother¬ 
apists or sports physicians. They can be downloaded 
to the RUFUS embedded PC like an app from an app 
store. 

RUFUS has a major advantage over JoggoBot. It 
has a battery capacity that allows it driving for about 
six hours on a flat road without recharging. 

65.1.6 Tele-Presence 

In a world in which not only large, but even small and 
medium-sized enterprises operate globally, in which 
families are scattered over continents, in which ubiqui¬ 
tous presence seems to become an essential requirement 
for professional progress, and in which professional ser¬ 
vices are more and more delivered over the internet, 


tele-presence has become a fast growing market in the 
past years. 

Robotics adds a very important aspect to plain tele¬ 
vision by turning it into tele-presence : embodiment and 
remotely controlled motion. As tele-presence is nothing 
but the combination of tele-vision and tele-operation 
using a robotic device, which are often called tele¬ 
presence robots or robotic avatars. 

Tele-presence robots offer a whole spectrum of 
services and applications ranging from plain mobile 
video-conferencing systems to tele-surveillance, tele- 
diagnosing, and tele-care, to tele-teaching and tele¬ 
commuting. The term tele-commuting was coined by 
Scott Hassan, a Google developer of the early days, 
nowadays entrepreneur and investor, and founder of 
WillowGarage and Suitable Technologies, the manufac¬ 
turer of Beam-RPD (see also Table 65.4). 

A tele-presence robot typically consists of a mobile 
robotic platform that: 

• Can be tele-operated through some user-interface, 

• Carries a camera, which often can be actuated sep¬ 
arately (via a pan-tilt unit) and allow the operator to 
actively explore the remote environment, and 

• Carries a display, which allows those at the remote 
site to see the operator of the tele-presence system 
and communicate and interact with it. 

Figure 65.13 shows a collection of such tele¬ 
presence robots. The devices range from a price of 
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Table 65.4 Technical specification of a selection of tele-presence robots 



V 

I • 

■ 

. r 

1 

■r 


KT J 

*Q> 


Manufacturer 

Double Robotics 

Vgo Communications 

Gostai 

Suitable Technologies 

Model 

Double 

VGo 

Jazz Connect 

Beam RPD 

Height 

120-150 cm 

120 cm 

100.5 cm 

5 feet, 2 inch 

Weight 

7 kg 

9 kg 

8 kg 

45 kg 

Screen size 

9, 1 " (iPad) 

6" 

5" 

17" 

Camera/field 

of view 



High resolution/ wide 
angle, up to 640 x 480 
pixels at 25 fps 

Two wide-angle HD 

cameras 

Video 

conferencing 

Open-tok 

VGo video conf. 

2-way audio and video for 
remote discussions 

- 

Network 

WiFi 

WiFi / 4G / LTE 

WiFi 

WiFi (two dual-band 
radios) / 4G 

Remote Control 

iPad App 

VGo App 

Intuitive control interface 

on internet browser 

Beam software client, 
mouse, keyboard, or Xbox 
controller 

Navigation 

Tele-operated 

Tele-operated 

Tele-operated with obstacle 
detection 

Tele-operated 

Sensors 

Gyroscope, accelerometer 

Obstacle and cliff detection 

sensors 

12 ultrasonic sensors, 4 IR 
receivers (for base dock¬ 
ing), telemetric laser for 
autonomous navigation 


Drive 

Differential (10" wheels) 

Differential 

Differential 

Differential 

Battery 

Lithium ion 

- 

- 

- 

Run time (h) 

8 

12 

5 

8 

Docking station 

- 

Yes auto-docking 

Yes auto-docking 

Yes 


some US$ 1500 for TeleMe from MantaroBot to more 
than ten-fold that amount for Beam RPD from Suitable 
Technologies. The system RP-VITA (Remote Presence 
Virtual + Independent Telemedicine Assistant) emerg¬ 
ing from a cooperation between InTouch Health and 
iRobot is available only for lease, at a monthly fee 
of $ 4700. Giraff is the result of a European research 
project lead by Giraff Technologies funded by the Eu¬ 
ropean Commission and is not commercially available 
yet. 

Not all tele-presence systems shown in Fig. 65.13 
can be classified as domestic robots. A system that 
clearly stands out and is by no means a domestic robot 
is RP-VITA by InTouch Health and iRobot. RP-VITA 
is a remote healthcare system. RP-VITA shall enable 
doctors to command any clinical, patient or care team 
management process remotely. RP-VITA has a full- 
fledged autonomous navigation system that allows the 
personnel to focus on the patient care task rather than 
on remote navigation. This feature has been awarded 
clearance by the US Food and Drug Association (FDA). 


RP-VITA further provides access to important clinical 
data to support physicians, nurses and other care per¬ 
sonnel in their diagnosis and other medical workflow. 
For example RP-VITA connects with diagnostic de¬ 
vices such as ultrasound and comes equipped with the 
latest electronic stethoscope. So RP-VITA is in a class 
of its own, which may also justify the higher price. 

Apart from RP-VITA, the tele-presence robots 
shown in Fig. 65.13, can all be classified as semiprofes¬ 
sional or domestic service robots. The functionalities 
and services they offer do not necessarily vary on the 
same scale as their prices. This can be seen by a com¬ 
parison of two of the above systems: Double and Beam 
RPD (see also Table 65.4). 

Double is not much more than a mobile iPad 
equipped with the video-conferencing system Opentok. 
The mobile base uses a Segway-like dual-wheeled drive 
system that can balance a pole, which holds the iPad. 
When Double stands still, two retractable kickstands 
are deployed and allow the system to put the control 
system in an idle mode and save energy. Double can 
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be remotely controlled and driven around a remote site 
through an app installed on a second iPad that enables 
communication with all known Doubles over the web. 
The height of the iPad holder can be remotely adjusted 
to enable communication at eye-level. Double Robotics 
list a number of potential services and applications for 
which Double could be used: Companies with sites 
at various locations can us Double to improve com¬ 
munication and collaboration between remote teams. 
Families can use Double to communicate with family 
members living abroad. Museums and art galleries can 
use Doubles to offer remote tours through their exhibi¬ 
tions. 

Beam RPD uses two HD cameras with custom 
wide-angle lenses instead of the plain iPad camera. This 
gives Beam RPD peripheral vision that is comparable 
to a human’s field of view. A digital zoom lets the op- 


65.2 Enabling Technologies 

The mass consumer market is very price-sensitive, so 
the price of the robot is key for the success of the 
product among consumers. Certain guidelines used in 
the consumer electronics market are relevant for the 
domestic robot market to provide a rough estimate of 
cost of the robot. Let’s say that you want to develop 
a floor-care robot that would retail at $ 300, the rule of 
thumb is that your bill of materials (BOM) should be 
between 1/3—1/5 of the retail price. In other words, 
your BOM should be within $ 60-$ 100! And the BOM 
must include all mechanical parts, electrical parts, bat¬ 
tery, processor, memory, motors, assembly, packaging, 
user manuals, etc. 

Given the extreme cost constraints outlined above, 
this chapter focuses on enabling technologies that are 
viable, from the cost point of view, to be included 
in a mobile domestic robot with a price lower than 
$ 1000 (or ideally below $ 500). These technologies 
are required to have a reliability level in line with the 
expected life time (and warranty) of the product; other¬ 
wise, no matter how good the technology is, if it stops 
working in an unreasonable period of time, the robot 
will be returned to the retailer. Special emphasis should 
be placed in the ease of manufacture of the technology. 
Difficult to manufacture components create delays in 
the production line, decreasing the yield of the product 
and eventually increasing the overall cost of production, 
leading to either eroding profit margins or a rise in the 
retail price. 

Mobile robots need to sense and understand the 
environment in which they operate. The first key en¬ 
abler is the capability of detecting obstacles and hazards 


erator further focus on details in the remote site. Beam 
uses an array of six microphones and audio processing 
algorithms, background noise reduction and echo can¬ 
cellation. This equipment gives Beam an audio-quality, 
which obviously goes far beyond that of an iPad. Beam 
uses a 17-inch screen mounted at a height of 1.58 m 
that allows the display of a human face at its natu¬ 
ral size and height. Another feature that goes beyond 
iPad standards is the WiFi connectivity. To provide re¬ 
liable and seamless WiFi connectivity. Beam uses two 
dual-band radios and proprietary roaming algorithms. 
Altogether it is obvious that Beam RPD is far more 
than a movable iPad. It is left to the customers to 
decide whether this is worth a price which is an or¬ 
der of magnitude higher. l<s>M2ESE!3i, 

and I^ESJEESiESi introduce several of 
the tele-presence robots available on the market today. 


to safely and accurate navigate around them. Sec¬ 
tion 65.2.1 describes the different available technolo¬ 
gies for obstacle and hazard detection. The second key 
enabler is the ability to localize and create a map of the 
environment to intelligently plan actions and motions 
that allow the robot achieve its goal. Section 65.2.2 
presents the technologies available for localization and 
mapping using a number of low-cost, yet powerful sen¬ 
sors. Section 65.2.3 discusses alternative approaches 
to coverage of the space implemented in commercial 
products. 

65.2.1 Sensing and Obstacle Avoidance 

Domestic robots aim to take care of tedious chores, 
interacting with a household that includes owners, chil¬ 
dren, babies, pets, and stationary objects such chairs, 
tables, walls, etc. Domestic robots must be safe in order 
to gain acceptance in our daily life: it is not tolera¬ 
ble to have a robot falling down the stairs or hurting 
a household member. Thus, robots must be equipped 
with drop/cliff sensors and proximity sensor that ensure 
proper operation while still satisfying the mentioned 
cost constraints. 

Cliff Sensors 

A number of solutions are present in current robots 
in the market. Off-the-shelf solutions are IR sensors 
from Sharp that consists of an emitter (light emitting 
diode LED) and a receiver (photodetector or position- 
sensitive-device PSD) that provides an output propor¬ 
tional to the distance to the object. The Roomba uses 
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Fig.65.14a,b Laser Distance Sensor, 
(a) Prototype and (b) occupancy map 
generated with the sensor 


a custom IR cliff sensor based on a similar principle 
that trades generic distance measurement with sensor 
cost. The Mint robot from Evolution Robotics employs 
a factory-calibrated mechanical hammer that triggers 
upon cliff detection. Solid-state sensors are usually 
more reliable that mechanical sensors, since they do not 
have moving parts, but have the drawback of a response 
dependent on the reflectivity of the surface in the IR 
spectrum and a dead-band in the response. 

Contact and Proximity Sensors 
Mechanical switches, called bump sensors, are com¬ 
monly used for detecting when the robot gets in contact 
with obstacles. Bump sensors are cost-effective solu¬ 
tions providing the ability to stop the robot without 
damaging the obstacle. Touching obstacles is not de¬ 
sired unless performed very gently to ensure that the 
robot goes under curtains and bed skirts. IR and sonar 
sensors are frequently employed as touchless alterna¬ 
tives to the bump sensors by measuring the distance 
to obstacles. Both types of sensors are composed of 
an emitter and a receptor with an output proportional 
to the measured distance. IR sensors are usually more 
focused than sonars and less sensitive to multiple re¬ 
flections on walls and other obstacles, but might lose 
thin obstacles such as chair legs. This type of sensors 
provides a point-wise measurement of distance to ob¬ 
stacles, so a robot needs a number of these sensors to 
obtain a dense representation of the obstacles in the 
environment. The information on obstacles and haz¬ 
ards (cliffs) is collected in occupancy grid maps and 
used for decision-making in systematic cleaning robots. 
A number of cost-effective dense distance measuring 
sensors have recently appeared in the market and will 
be discussed in the next sections. These dense distance 
measuring sensors have a cost on the order of tens 
of dollars, while the point distance measuring sensors 
have cost only a few dollars, so most of the domestic 


robots currently available on the market have yet to in¬ 
corporate dense distance measuring sensors. The only 
exception is the Neato XV-21 that uses a low-cost laser 
distance measuring system. 

Laser Distance Sensor 

A low-cost Laser Distance Sensor was developed by 
Konolige and colleagues [65.5] using a laser point beam 
and a global shutter CMOS imaging sensor separated by 
a small baseline. The system operates by triangulation 
and achieves full 360 planar scan by rotating the optical 
assembly on a full circle. The sensor has a range of 0.2 
to 6 m with an error < 3 cm at 6 m and an angular reso¬ 
lution of 1 degree, providing 4000 readings per second 
(up to 10 Hz) with a small size (approximate width of 
10 cm shown in Fig. 65.14a) and low power (< 2 W). 
The sensor is eye safe and provides measurements that 
enable laser-based SLAM as shown by Fig. 65.14b. 

Structured-Light Distance Sensors 
Structured-light distance sensors consist of an emitter 
that projects a known pattern on the environment and 
a receptor that computes depth based on the deforma¬ 
tion of the received pattern. The Kinect [65.6] interface 
to the Xbox game system uses a structured-light sensor 
from PrimeSense [65.7] thus showcasing the readiness 
of this sensor for consumer applications. The emitter 
consists of a laser with optics that projects a known pat¬ 
tern (Speckles [65.8,9]) in near-IR light and a comple¬ 
mentary metal-oxide-semiconductor (CMOS) IR cam¬ 
era that observes the pattern to estimate depth using tri¬ 
angulation. The emitter and the camera are calibrated 
during manufacturing assuming a rigid configuration. 
The speckles can be further shaped into ellipsis using 
optics with different focal lengths in x and y so that 
the orientation of the observed ellipsis is proportional to 
depth. Speckles of different sizes are used to obtain dif¬ 
ferent depth accuracy depending on size. Figure 65.15a 
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Scene projected 
by invisible IR light 


Fig.65.15a-c Primesense sensor: (a,b) Speckles pattern projected 
by the laser emitter, (c) Block diagram of the Kinect sensor 


shows the speckles pattern projected by the sensor and 
an image taken in the dark by photographer Audrey Pen- 
ven showing the IR speckles. Figure 65.15b shows the 
components of the Kinect sensor and the corresponding 
RGB and depth images of a scene. 

Time of Flight (TOF) Distance Sensors 
Time of flight (TOF) sensors consist of a light source 
(usually a laser) that emits a continuous waveform and 
a special imaging sensor that measures the phase shift 
of the received signal in each pixel. The depth at each 
pixel is proportional to the phase shift. TOF sensors 
have been in the market for quite some time, but the 
need for allocating a large portion of the sensor to the 
decoding electronics has made it challenging to produce 
low-cost sensors at reasonable resolution. Some of the 
companies offering TOF sensor have been Mesa Imag¬ 
ing AG that produces the SwissRanger [65.10] sensors, 
Softkinetics [65.11], PMDVision. 

Stereo Vision 

Stereo vision is a well-known computer vision solution 
to the extraction of three-dimensional (3-D) depth maps 
in areas with sufficient texture to find image correspon¬ 
dences. As opposed to structured light or TOF sensors, 
stereo vision systems are totally passive, but require 
a calibrated stereo rig, and their performance depends 
on the level of external illumination and on the amount 
of texture present in images. 

The selection of the optimal dense mapping sensor 
depends on the application. The laser distance sensor 
provides reasonable information for laser-based SLAM 
and obstacle detection and avoidance; however, it only 
provides range information on a plane as opposed to the 
dense 3-D range offered by the structured light, TOF, or 
stereo systems. The structured light sensor uses a sim¬ 
ple imaging sensor but requires additional computation 
to estimate depth in each pixel while the TOF sensor 
computes depth in each pixel at the expense of a sensor 
with lower fill factor. The stereo system does not require 
additional lighting, but requires an additional camera 
and a computation module to extract depth. Other pa¬ 
rameters to consider are the maximum and minimum 
range that the sensor provides to ensure that it fits the 
requirements of the application in terms of mapping and 
obstacle detection. 

65.2.2 Localization and Mapping 

A robot that knows its location and understands its 
surroundings is able to plan intelligent maneuvers to 
achieve its goals. Localization and mapping are ba¬ 
sic primitives that enable smart and efficient behavior. 
Early successful robots like the Roomba chose to sac¬ 


rifice localization and mapping in order to achieve an 
appealing retail price since the localization and map¬ 
ping technology was either too expensive or not existent 
at the time (many of the dense sensors presented in the 
previous section were developed after the Roomba). In 
recent years, a number of low-cost, yet powerful simul¬ 
taneous localization and mapping (SLAM) technolo¬ 
gies have been developed and integrated in floor-care 
products. These technologies are described in the fol¬ 
lowing sections. 

Vector Field SLAM 

The Mint robot by Evolution Robotics uses active bea¬ 
cons for localization. The Northstar beacon projects two 
IR spots onto the ceiling that are modulated to sim¬ 
plify the detection of the spots with the sensor. The 
spots are invisible to the human eye and thus do not 
produce visual clutter. While placing the beacon can 
still be regarded as a modification to the environment, 
surveys among our customers suggest that the vast ma¬ 
jority largely accepts setting up the beacon prior to 
running the robot [65.12]. The Northstar sensor uses 
3 photodiodes to compute the direction to the spots 
from the measured current through the photodiodes. 
The photodiodes can be sampled at high frequency to 
detect the modulated spot frequencies for data asso¬ 
ciation. The sensor is quite inexpensive and suitable 
for cost-sensitive applications. However, the sensor suf¬ 
fers from multipath since light not only reaches the 
sensor directly but also through reflections from walls 
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and other furniture, making the position computed by 
the sensor inadequate to be directly used for local¬ 
ization and mapping. The sensor is augmented with 
a localization method that learns the light distribu¬ 
tion in the room through a simultaneous localization 
and mapping (SLAM) approach [65.13]. Figure 65.16 
shows the Mint robot cleaning the environment depict¬ 
ing the operation of the Northstar localization system 
and the problem of multipath. The figure also shows 
the Mint robot and the Northstar cube and the Northstar 
sensor. 

In vector field SLAM the spatial variation of con¬ 
tinuous signals is learned and simultaneously used for 
localizing the robot by fusing information from dead¬ 
reckoning (odometry and gyro) and Northstar. In the 
following, this method is introduced and tailored toward 
measurements obtained from Northstar. The signal field 
is represented as a regular grid of fixed node positions 
bi = (bj^, b it y) T , i = 1... N, where each node W; e R l 
holds the expected Northstar positions of both spots 
when placing the robot at fc, and pointing it in a fixed 
orientation 8q = 0. Vector field and robot pose are then 
estimated through the application of SLAM. 

Let the robot path be a time series of poses xo . ■ ■ xj, 
x t E SE(2), i. e., the set of rigid transformations in the 
horizontal plane, and let xo = (0,0, 0) T . At each time 
step t=l...T the robot receives a motion input u t 



Fig.65.16a-c Mint robot, (a) Normal operation of the 
Mint robot using Northstar (the yellow path indicates mul¬ 
tipath). (b) Mint robot and Northstar cube, (c) Northstar 
sensor 


with covariance R t and a measurement z t = fed, z y \, 
z x 2 , Zyi) 1 of the two Northstar spot positions with co- 
variance Q,. The spot positions are also each affected 
by the rotational variability denoted c = ( c x , cy) T . The 
rotational variability models errors in measuring the di¬ 
rection to the spots caused by not having the sensor 
perfectly level. 

The SLAM problem is solved with the ESEIF- 
SLAM that is constant time and requires memory linear 
in the size of the area explored by the robot. The method 
has been implemented in the processor of Mint, an 
ARM 7 processor with 64kByte RAM [65.13]. Vec¬ 
tor field SLAM has been extended to address covering 
larger areas by using more Northstar beacons [65.15] 
and to provide a solution for the re-localization of the 
robot after it has been kidnapped or paused and re¬ 
sumed [65.16]. 

Figure 65.18 presents the map obtained by the robot 
after a cleaning run in a home environment of 125 m 2 
covered by three Northstar beacons marked with the 
pink and green disks (units are in meters). The robot 
navigated in the home by following a cleaning strategy 
based on systematically covering sectors of the envi¬ 
ronment. As long as at least one beacon is visible to 
the robot, the strategy moves the robot onto a neigh¬ 
boring region until no space is left to clean. At the end 
the robot follows along the perimeter of detected obsta- 



Fig. 65.17 Vector field SLAM with Northstar 



Fig. 65.18 Robot map obtained with vector field SLAM in 
a 125 m 2 home environment 
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Fig. 65.19 (a) cv-SLAM (after [65.14]). (b) Features extracted by the vision front-end. Map obtained with the EKF 


cles for a thorough cleaning around walls and furniture. 
As the robot moves through the environment it creates 
an occupancy grid map using the position information 
from localization. Each visited cell is classified into one 
of the following categories: obstacles (drawn in black), 
floor changes (drawn in brown), hazards (drawn in red), 
and free space (drawn in shades of blue that indicate 
visibility of the spots, the lighter the blue, the better the 
visibility). 

Visual SLAM 

Visual localization and mapping is attractive for a vari¬ 
ety of applications due to the rich input and the low cost 
and footprint of visual sensors. The difficulties lie in ro¬ 
bustly extracting a critical subset of information from 
the high-rate visual data stream and processing it effi¬ 
ciently to yield useful output. Despite steady increases 
in the computational power of most platforms, such 
challenges are nonetheless exacerbated by the limited 
processing and storage provided by low-cost, embed¬ 
ded systems appropriate for low-power applications or 
consumer products. Many state-of-the-art approaches to 
visual SLAM depend on a per-frame processing rate 
sufficiently high, relative to the speed of camera motion, 
to permit strong temporal assumptions on the image se¬ 
quence. This section reviews two approaches to visual 
SLAM focused on domestic robots. Ceiling vision sim¬ 
ilar to the work of Jeong and Lee [65.14] can be found 
in the Navibot from Samsung, the Roboking from LG, 
and the Iclebo/Homerun from Yujin Robotics/Philips. 
Furthermore, visual SLAM systems have been devel¬ 
oped for low-cost and embedded systems [65.17, 18]. 

cv-SLAM 

The ceiling vision SLAM [65.14] system is composed 
of a camera facing upward, looking toward the ceiling. 
The system extracts corner features using a Harris de¬ 
tector and matches features using correlation. Feature 
matching achieves invariance to view point changes by 
training a set of multiview descriptors of the features 
as matching proceeds during the run. The main orien- 
tation(s) of the feature are further used as descriptors 


of the feature to ensure a two-dimensional (2-D) ro¬ 
tation invariant feature matching that can be used for 
relocalization. The localization and mapping backend 
of the system is based on an extended Kalman filter 
(EKF) that fuses visual and dead-reckoning informa¬ 
tion (odometry and gyro) and that tracks the pose of 
the robot in a two-and-a-half-dimensional (2.5-D) ( x , 
y, 9) and the 3-D position of the features (landmarks 
of the map). The orientation(s) of the features are also 
represented in 3-D and tracked with the EKF to pre¬ 
dict changes in the feature patch due to motion of the 
robot (as the robot moves, features on the ceiling will 
undergo a rotation while features on the walls will 
experience a shearing transformation in addition to ro¬ 
tation). 

Figure 65.19a shows corner points and their esti¬ 
mated orientations in views with rotation only. Fig¬ 
ure 65.19b presents snapshots from a sequential map¬ 
building experiment on a corridor. 

vSLAM 

The vSLAM [65.17,18] system from Evolution 
Robotics is designed for a low-cost robotic platform 
equipped with simple odometry and a single camera. 
Figure 65.20 shows the block diagram of the system. 
Operating primarily as a view recognition engine, the 
visual measurement front-end requires only occasional, 
weak assumptions on processing rate, and intrinsically 
provides robust loop closing when previously mapped 
areas are revisited. The visual measurements and odom¬ 
etry are fused in the back-end in a graph representation 
and optimized incrementally. The SLAM graph com¬ 
plexity is bounded during operation using variable elim¬ 
ination and constraint pruning, with heuristic schedules 
in order to keep optimization and storage costs com¬ 
mensurate with explored area, rather than with time of 
exploration, all while causing minimal loss in mapping 
and localization accuracy. The system has been evalu¬ 
ated on real datasets with planar ground-truth reference, 
showing that the system operates successfully even at 
frame rates below 2 Hz, running on an ARM9 proces¬ 
sor with 64 MB of RAM [65.18]. 
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Fig. 65.20 vSLAM block 
diagram 


View recognition engines [65.17-22] have proven 
attractive components for SLAM systems because they 
permit robust and flexible loop closing. Instead of 
making correspondences between individual features 
or measurements as in the cv-SLAM system, view 
recognition engines typically match constellations of 
features or entire images, without requiring track¬ 
ing. vSLAM [65.17] creates views by first matching 
SIFT [65.23] features on pairs of incoming images, and 
second computing structure and motions estimates us¬ 
ing bundle adjustment. The SIFT features are stored 
in a local database for each view and on a global 
database for the complete map. View recognition is 
performed by a feature lookup in the global database 
that provides a set of candidate view matches. Feature 
lookup in the local view database followed by robust 
outlier rejection and a local bundle adjustment com¬ 
pletes the view recognition and visual pose estimation 
process. 


Commonly used graph SLAM methods for agglom¬ 
erating sensor information often incur computation and 
storage costs that grow with time, rather than with 
space explored. For a robot operating for extended 
periods within a limited spatial area - typical of prac¬ 
tical applications - this is an undesirable trade-off. 
vSLAM [65.18] instead applies probabilistically sound 
graph reduction methods that limit the complexity of 
the graph to a linear factor of the complexity of the ex¬ 
plored space. Past poses of the robot that are not used 
for view recognition can be marginalized out of the esti¬ 
mation, and their incident constraints are collapsed back 
into the graph. 

Figure 65.21 presents the results of running the 
vSLAM system in two sequences, one collected in 
a regular household (right) (same house as the one 
shown in Fig. 65.18 for vector field SLAM) and the 
other collected in a large warehouse environment (left). 
The first set of plots show the ground truth path of 




Fig. 65.21 vSLAM results on a warehouse environment and in a household environment 
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Fig. 65.22 Map created with the Neato XV-11 robot 
(www. youtube.com/watch?v=zodC8EFvh7g) 

the robot and the floor plan of the environments. The 
second set of plots shows the trajectory of the robot 
estimated with vSLAM. The warehouse environment 
extends over an area of 24x12 m and the house over 
an area of 20x9 m. The estimated trajectory has a root 
mean square (RMS) error of 44 cm in the warehouse 
case and of 28 cm in the household case. 

Laser-Based SLAM 

The previous sections presented a number of low-cost 
solutions to the SLAM problem in which the data asso¬ 
ciation was (almost) perfect. In the case of Vector-Field 
SLAM, the modulation of the spots of the Northstar 
cube ensures a unique identification of the spot. In the 
case of visual SLAM, the visual front-end incorporates 
a number of checks to ensure that misrecognitions are 
very rare. Laser-based SLAM has the characteristic that 
the measurements acquired with the laser are not unique 
since it is possible to obtain similar measurements in 
a variety of places in a household (just aiming the laser 
to a wall will provide measurements that are indis¬ 
tinguishable from a measurement to a different wall). 
In addition to the data association problem, laser dis¬ 
tance measurement sensors, such as the one presented 
in Sect. 65.2.1, provide both range and bearing to the 
landmarks. Both Northstar and cameras give only bear¬ 
ing measurements to the spots or features. 

The laser-based SLAM literature is quite exten¬ 
sive [65.25]. The estimation back-end could be an 


extended Kalman filter (EKF), a Particle Filter, or 
a GraphSLAM system. Several data association algo¬ 
rithms are available, some of them tire proactive (or 
even greedy) [65.26,27] and some are lazy [65.28] in 
assigning correspondences between measurements and 
landmarks. The selection of the algorithm to implement 
in the domestic robot would be guided by the trade-off 
between computational resources and the performance 
requirements. Figure 65.22 presents a map obtained 
with the Neato XV-11 robot. 

65.2.3 Navigation and Coverage 

The Trilobite robot vacuum cleaner by Electrolux was 
one of the pioneer robots to be commercialized. The 
trilobite was equipped with a sophisticated custom 
sonar sensor system that allowed the robot to navigate 
without touching obstacles (or touching them very gen¬ 
tly). The coverage strategy consisted of two stages: first, 
explore the perimeter of the room to estimate the area 
to clean and second, cover the room with wall-to-wall 
diagonal passes. The perimeter exploration stage as¬ 
sumes that the robot would eventually traverse through 
the complete boundary of the area to clean and return to 
the charging station (Fig. 65.23a). 

As mentioned previously, the cost constraints forced 
early robots like the Roomba to eschew advanced fea¬ 
tures like localization and mapping in order to offer 
a product at a price point accepted by consumers. Nev¬ 
ertheless, the navigation strategy implemented in the 
Roomba was quite effective in covering the space, es¬ 
pecially when running in single rooms or in presence of 
large amount of clutter. The Roomba uses a spiral pat¬ 
tern in open area to optimally cover the space without 
localization until it hits an obstacle (Fig. 65.23b). Then 
it selects random orientation and continues moving in 
straight line until reaching the next obstacle where it 
selects another random orientation to continue with the 
same behavior. The spiral can be triggered when the 
robot travels for a certain distance without having found 
any obstacles. Many of the other random robots present 
in the market have a strategy that follows similar prin¬ 
ciples to the Roomba one. 



Cleaning 


Fig.65.23a,b Coverage 
strategies. Trilobite strategy. 
Roomba strategy [65.24] 
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Fig.65.24a-c Long exposure im¬ 
ages [65.29] showing the coverage 
strategies of the (a) Roomba, (b) 
Neato, and (c) Mint 



Fig.65.25a,b Coverage strategies for 
lawnmowers. (a) Tango [65.30] by 
John Deere and (b) Indego [65.31] by 
Bosch 


The systematic robots on the other hand take advan¬ 
tage of the localization system to plan efficient cleaning 
paths to maximize coverage in the least amount of 
time, pause for charging on the docking station and 
resume cleaning from the last cleaned spot, and in¬ 
telligently navigate from room to room. Mint takes 
the strategy of focusing first on open area cleaning, 
covering the space in parallel straight passes to tra¬ 
verse as much open space as fast as possible, and 
then performing a final cleaning on the perimeter and 
around the obstacles. This perimeter cleaning step 
enables Mint to uncover portals to new areas that 
were not encountered during open area cleaning. Other 
robots like the Navibot from Samsung, the Robok- 
ing from LG and the iClebo from Yujin/Philips use 
a similar strategy without the final perimeter clean¬ 
ing. The Neato XV-11, on the other hand, performs 
first an exploration and cleaning of the perimeter of 
the environment in order to create a good localiza¬ 
tion map and then completes the coverage of the open 
area. Figure 65.24 shows long exposure pictures of 
different robots: Roomba, Neato, Mint sweeping and 


65.3 Smart Homes 

Several attempts have been made in the literature to de¬ 
fine the term smart home, for example, in [65.32] the 
term is defined as the the latest expression of the various 
ways in which technology in the home has developed. 
In [65.33], the notion of a smart home is defined more 
explicitly as: 


Mint mopping. These long exposure pictures show 
the trajectory of the robots during normal cleaning 
operation. 

The lawnmower robots use similar coverage strate¬ 
gies to the ones of vacuum cleaner robots. One element 
common to all lawnmower robots is the usage of an 
embedded wire to define the perimeter of the lawn. 
Within the boundaries defined by the wire, some robots 
like the Automower from Husqvama or the Robomow 
from Friendly Robotics use a complete random cov¬ 
erage strategy. The Tango from John Deere combines 
straight motions in random directions with spiral mo¬ 
tions (a-la-Roomba). The Indego from Bosch is the only 
robot that attempts to systematically cover the space 
by first exploring the perimeter to estimate the size of 
the lawn and then traversing the interior with parallel 
straight passes. Systematic coverage is enabled by fus¬ 
ing sensory information from wheel odometry and an 
inertial measurement unit (IMU) by means of proba¬ 
bilistic reasoning with prior knowledge. Figure 65.25 
depicts the coverage strategies of the Tango and the 
Indego. 


a residence equipped with computing and informa¬ 
tion technology which anticipates and responds to 
the needs of the occupants, working to promote their 
comfort, convenience, security and entertainment 
through management of technology within the home 
and connections to the world beyond. 
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Smart homes typically comprise elements such as net¬ 
work of sensors and actuators, and also entire robotic 
systems. 

In 1999, a company called e2 Home was established 
by Ericsson and Electrolux to explore the possibili¬ 
ties of smart homes [65.34]. E2 Home built several 
houses with smart home devices using IT technologies. 
Through the exploration, several issues relating to smart 
homes from business point of view have been unveiled, 
which include the difhculties relating to the complexity 
of the system, starting up the new business model, han¬ 
dling of intellectual property rights (IPR) for the third 
parties contents, and the consumer’s needs. The com¬ 
pany was liquidated in 2004. 

Current smart home technology includes video 
monitoring, motion detectors, fall detectors, pressure 
mats, environment control, health monitoring such as 
blood pressure, pulse rate, body temperature, weight, 
and human computer interaction (HCI) technology, for 
example, to recognize gestures. Smart homes also of¬ 
ten refer to houses connected to a smart grid, which is 
defined as [65.35]: 

a developing network of new technologies, equip¬ 
ment, and controls working together to respond 
immediately to our 21st century demand for elec¬ 
tricity. 

In this case, the smart home is defined as [65.35]: 

a residence with the capability of efficiently control¬ 
ling generated solar energy and power consumption 
making it ideal for vehicle power supply and man¬ 
agement. 

The development of smart homes requires a num¬ 
ber of technical questions and challenges to be ad¬ 
dressed [65.32]: how to convert current home structures 
and architectures into smart homes, how to standard¬ 
ize smart home components, for example, sensor net¬ 
works, how to keep the equipment cost at a reasonable 
level, and how to deal with security and privacy is¬ 
sues. 

In the following sections we will describe a num¬ 
ber of prominent smart home developments, the Aware 
Home at Georgia Institute of Technology, the Gator 
Tech Smart House at the University of Florida, and 
the sensorized environment for life (SELF) at AIST 
(the Japan National Institute of Advanced Industrial 
Science and Technology) (Figs. 65.26, 65.27). These 
developments show that current smart home technol¬ 
ogy goes significantly beyond existing home automa¬ 
tion. Accounting for the significant increase in the 
elderly population in the near future, many of to¬ 
day’s smart home developments pay special attention 
to improving the quality of life of elderly people. The 


descriptions below will also address the question on 
how to integrate robotics systems into smart home 
concepts. 

65.3.1 Gator Tech Smart House 

The Gator Tech Smart House (Fig. 65.26) was built at 
the University of Florida [65.36]. It addresses the needs 
of elderly people to live independently and maintain 
dignity and quality of life at older ages. The house is 
equipped with many smart devices such as smart floors, 
tracking the motion of the occupants of the house, smart 
blinds, automatically adjusting ambient light, smart 
display, smart cameras, smart phones that can act as 
a remote control to other appliances, location tracking, 
smart leak detectors, and smart beds. The exterior of 
the house has a smart mailbox which alerts residence 
if mail is delivered and a smart front door which can 
sense home owners using an radio frequency identifica¬ 
tion (RFID) tag, allowing keyless entry to home owners. 

The kitchen of the Gator Tech Smart House in¬ 
cludes a smart microwave, which uses RFID on food 
packages. This allows the microwave to adjust the 
settings for cooking the meal. It also informs the res¬ 
ident about the readiness of the meal. The kitchen 
further comprises a smart refrigerator that monitors 
food availability and consumption, and detects expired 
food items. The smart refrigerator can create shopping 
lists automatically and has an integrated meal prepa¬ 
ration advisor based on items in the refrigerator and 
pantry. 

The implementation of such a complex system has 
raised a number of technical issues and questions, in¬ 
cluding the development of the smart devices, data 
handling of networks of sensors, and interconnecting 
smart devices to other devices in the environment. 
These questions led to some new research tracks on 
smart houses, primarily grouped into pervasive comput¬ 
ing and mobile computing network research. 



Fig. 65.26 Georgia Institute of Technology: Aware Home 
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a) Conventional computer Living-space computer with a function 

for monitoring human respiratory system 



Embedded in room Washstand display 

Pressure sensor bed 


b) Conventional monitor for human respiratory system 


Embedded in room 


Ceiling dome microphone 
Wall camera 



Fig. 65.27 AIST, the sensorized environment for life 



Realized 
in room 


c) Developed room 



65.3.2 Aware Home 

Aware Home is a living laboratory for research in 
ubiquitous computing for everyday activities. This 
project is conducted at Georgia Institute of Tech¬ 
nology [65.37]. The major objective of the Aware 
Home project is to build an environment that is ca¬ 
pable of being aware and keeping track of the states 
and activities of its inhabitants. Aware Home creates 
a partnership between the resident and the surround¬ 
ing sensing and computing technologies. This opens 
several fields of research, not only from the technol¬ 
ogy point of view, but also in terms of the social 
aspects of the inhabitants. The main research agenda 
of Aware Home spans human-centered and technology- 
centered research, software engineering, and social 
implications. 

Technology and application-centered research fo¬ 
cuses on sensor networks, distributed computing, con¬ 
text awareness and ubiquitous sensing, individual in¬ 
teraction with the home, smart floors, and finding lost 
objects. Research on context awareness is inspired by 
the fact that humans communicate with each other 
very success-fully by referring to what is called shared 


context. For communication between humans and com¬ 
puter systems, this shared context must be made explic¬ 
itly. Sensor systems, which facilitate the extraction of 
context, need to be developed. 

This human-centered research focuses on support 
for the elderly and other social issues. A key concept in 
sup-porting the elderly is aging in place. Aware Home 
is designed to support the elderly and allow them to 
be independent instead of moving to elderly care fa¬ 
cilities. Supporting the elderly leads to the study on 
cognitive support such as reminding them when to take 
medication, guiding them when they lose their way, and 
locating lost items. 

65.3.3 SELF - Sensorized Environment 
for LiFe 

SELF stands for Sensorized Environment for 
LiFe [65.1]. The objectives of SELF are to de¬ 
velop a network of sensors which are embedded in 
the environment, information gathering using the 
networked sensors, storing and analyzing the infor¬ 
mation, and the reporting of useful information to 
assist and maintain good health. The basic advantages 
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of SELF, due to the embedded nature of the sensors 
are: 

1. No limit on size, weight, or power source, 

2. It does not disturb the human, 

3. It does not impose physical restrictions, and 

4. Sensors are rarely broken since they are fixed to the 
environment. 

SELF can be viewed as a system that monitors a per¬ 
son’s behavior or activity and represents the data ob¬ 
jectively in an approach known as self-externalization. 
SELF is motivated by the fact that humans sometimes 
cannot notice a change in their condition which affects 
their health without a medical doctor. Therefore, the use 
of network sensors to monitor human behavior and re- 

Video-References 


port useful information that greatly affects the health 
status will further improve quality of life. 

The SELF study considers behavior as a means of 
communication, and sensors embedded into the envi¬ 
ronment as one way to observer a person’s behavior. 
The SELF implementation consists of a bed with a sen¬ 
sor, a ceiling with microphones, a washstand with 
a display, etc. The bed with sensors can determine the 
time the subject sleeps and wakes up, their posture 
during sleeping, and their breathing pattern. The mi¬ 
crophones attached to the ceiling can detect snoring or 
normal breathing sounds. Based on the monitored data, 
the washstand display is used as an output device to pro¬ 
vide the subject’s health status and thus create feedback 
to the subject. 
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Robotic Vacuum Cleaners Reviewed by Click - Spring 2014 
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66. Robotics Competitions 
and Challenges 


Daniele Nardi, Jonathan Roberts, Manuela Veloso, Luke Fletcher 


This chapter explores the use of competitions to 
accelerate robotics research and promote science, 
technology, engineering, and mathematics (STEM) 
education. We argue that the field of robotics 
is particularly well suited to innovation through 
competitions. Two broad categories of robot 
competition are used to frame the discussion: 
human-inspired competitions and task-based 
challenges. Human-inspired robot competitions, 
of which the majority are sports contests, quickly 
move through platform development to focus on 
problem solving and test through game play. Task- 
based challenges attempt to attract participants 
by presenting a high aim for a robotic system. The 
contest can then be tuned, as required, to main¬ 
tain motivation and ensure that the progress is 
made. Three case studies of robot competitions 
are presented, namely robot soccer, the UAV chal¬ 
lenge, and the DARPA (Defense Advanced Research 
Projects Agency) grand challenges. The case studies 
serve to explore from the point of view of organiz¬ 
ers and participants, the benefits and limitations 
of competitions, and what makes a good robot 
competition. 

This chapter ends with some concluding 
remarks on the natural convergence of human- 
inspired competitions and task-based challenges 


The idea of promoting science and technology through 
competitions became pervasive at the end of the 20th 
century [66.1]. Since then, competitions have been pro¬ 
moted in several fields, taking a broad range of formats, 
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in the promotion of STEM education, research, and 
vocations. 


goals, and target audiences. The main aims underlying 
this trend are to create a motivating framework for sci¬ 
ence and technology, and to support the development of 
skills, ideas, and technical solutions. 
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66.1 Introduction 

These objectives fit particularly well with the field of 
robotics, since the design and implementation of robots 
is a great challenge for a competition. Robots are phys¬ 
ical artifacts, whose design and implementation require 
creativity, technical skills, and scientific knowledge. 
Arguably, Robotics is nowadays the field where com¬ 
petitions have the strongest impact. 

The competitions that aim at the design and imple¬ 
mentation of robots cover the whole spectrum of the 
educational chain, from young pupils of the elementary 
schools to senior researchers, such as the authors of this 
chapter. Moreover, there is a large world of hobby robot 
competitions, that we will not specifically address in 
this chapter. The focus of this chapter are those initia¬ 
tives that are specifically targeting autonomous robotics 
research and education. 

Another ingredient that makes robotic competitions 
very successful is that robotics requires a great deal of 
integration. Consequently, building a winning team re¬ 
quires a substantial cooperation among team members 
with very different skills and backgrounds. This cre¬ 
ates an ideal playground to address teamwork within 
educational programs, and an engineering challenge for 
higher level students and researchers. The level of per- 

66.2 Overview 

The history of robot competition begins with the Micro¬ 
mouse contests. In 1977, the IEEE Spectrum magazine 
announced their intention to hold an Amazing Micro¬ 
mouse Competition. The contest took the form of a time 
trial where a small wheeled robot mouse raced to com¬ 
plete a maze in the shortest possible time. The first 
event to be held was in New York in 1979 and saw 
over 6000 entries. However, the field rapidly shrank to 
15 competing mice. This first contest of these early au¬ 
tonomous robots was won by a simple wall-following 
robot. While a valid victory, the rules for future events 
were swiftly changed to move the goal point from the 
edge of the maze to the center, ensuring that the path 
search aspect of the problem was emphasized [66.2], 
The micromouse contest then became very popular 
across Europe, then later Japan and then back to the 
USA by the mid-1980s. Micromouse events continue to 
held to this day, including contests from India and South 
Korea. The contest which began as a struggle to develop 
a device that could reliably traverse corridors without 
jamming, is now a highly optimized contest where pre¬ 
cision machines complete the maze in under 5 s [66.3]. 

Seeing the success of the micromouse competition 
and the power to inspire science and technology ed- 


formance that robots achieve in the competitions is 
often compelling, as compared with the prototypes that 
are developed within conventional research projects. 
In this respect, competitions provide an outstanding 
testbed for scientific and technical solutions. 

Robotic competitions can have great appeal for 
a general audience; thus, robotic competitions are also 
a very effective tool to bridge the gap between citizens 
and the progress of science and technology. 

There are several features that can be used for 
the classification and presentation of competitions. We 
have chosen to distinguish between competitions where 
the contest mirrors a traditional human contest and 
competitions where robots are challenged to perform 
some task. The task whose achievement, once mature, 
would extend the possible uses of robots in our society. 

Next we present an overview including past robot 
competitions and considerations that apply across all 
robot contests. We then focus on human-inspired con¬ 
tests with a case study on robot soccer. Later we review 
task-orientated contests with case studies on the UAV 
challenge and the DARPA grand challenges. The chap¬ 
ter ends with concluding remarks on lessons learned 
thus far in robot competitions. 


ucation, Dean Kamen conceived the For Inspiration 
and Recognition of Science and Technology or FIRST 
robot competition. In the FIRST robot competitions 
high school teams would compete to build and program 
a robot to complete a challenge. The original challenge 
in 1992, titled Maize Craze, involved herding balls into 
a home goal. Since then the challenge, from stacking 
containers to throwing flying discs, has been changed 
year to year while keeping the basic structure of the 
contest the same. The contest, like the Micromouse con¬ 
tests, have expanded internationally and continue to this 
day. In 2014, the competition is projected to involve 
68 000 students [66.4], 

Meanwhile, in the early 1990s, the advances and 
limitations of game playing computers, and the need to 
evaluate competing algorithms in mobile robotics con¬ 
verged toward the concept of a new competition for 
academic robotics: robot soccer [66.5-7]. We will con¬ 
tinue this discussion in Sect. 66.3. 

In 1995, restlessness at the pace of innovative 
change since the space race of the 1960s sparked 
aerospace entrepreneur Peter Diamandis to propose the 
X-Prize. Ansari X-Prize was a $10 million prize to 
demonstrate a functional privately developed space pas- 



Robotics Competitions and Challenges 


66.2 Overview 1761 


senger plane [66.8]. Rekindling the idea of racing to 
achieve an ambitious task was captivating and caused 
ripples through the scientific community [66.1]. The 
inspiration spread to other fields and funding bodies re¬ 
sulting in many initiatives such as the UAV challenge 
and the DARPA grand challenge which we discuss in 
Sect. 66.4. 

Robotics is well suited to innovation through com¬ 
petition. As this handbook testifies the field of robotics 
is broad and covers research in mechanical design, 
electronics, control theory, signal processing, software 
engineering, and machine learning to name a few. In 
addition to the breadth of the field the nature of the 
research also requires one or more agents interacting 
with the physical environment, thereby limiting the use 
of techniques common to other technical fields to share 
and validate their research. 

Standard techniques such as: standardized tests 
(which are commonplace in science and engineering), 
shared test data sets (such as those used in image 
processing and machine learning communities), and 
reference implementations (used in computer science 
algorithm evaluation) are invaluable in robotics but can¬ 
not cover the full scope of a robot system operating in 
the world. Indeed, many components used in robotic 
systems can be tested in isolation with these same tech¬ 
niques; however, as the components of a robot are 
integrated into a system of systems interacting with the 
physical world the limits of the above methods becomes 
evident. In terms of test methodology: unit and inte¬ 
gration level testing are well served; however, system, 
dynamic and, in particular, acceptance testing of inno¬ 
vation across the field was missing. An answer came in 
a resurgence back to the roots of robotics and in favor 
of robot competitions. 

The range of robot scientific competitions is very 
broad, and in this section we look at them from differ¬ 
ent perspectives, aiming at providing an overview of the 
settings that have been developed so far. To this end, in 
the following we address a number of aspects that are 
relevant to the design and implementation of a robot 
competition. 

66.2.1 Aims and Targeted Participants 

As suggested in the introduction, the competitions that 
are addressed in this chapter are those focussed on 
research and education. These two aspects are often 
deeply inter-related. On one hand, there is always an ed¬ 
ucational component in any research involving a team 
with different backgrounds and different levels of ex¬ 
pertise. Moreover, if one looks at competitions as an ed¬ 
ucational tool, they represent one of the most advanced 
forms of education; consequently, they specifically at¬ 


tract teachers and mentors that are involved in research 
on education. Although research and education are re¬ 
lated to one another, the balance between research and 
education is naturally significantly different across the 
numerous robot competitions. 

66.2.2 Competition Challenges 

Each competition relies on a challenge and there are 
several ways of categorizing them. While competitions 
are nowadays mostly among robots and not robot-vs- 
human, a large group of competitions take inspiration 
from human competitions, ranging from soccer, to the 
Olympic Games. These competitions have an explicit 
or implicit underlying assumption of comparing robot 
performance with human performance. The other kind 
of challenge aims at a proposing a specific problem to 
be addressed using machines the hope is that by rac¬ 
ing to achieve the aim, new capabilities can be matured 
faster into future robots. 

66.2.3 Types of Robots 

Another key element for classification of competitions 
is the robot type. Any kind of robot is conceivable for 
use in competition. Robot choice is driven by the com¬ 
petition format and targeted participants. The use of 
commercially available robots and robot kits enables 
less mechanically focused participants to move forward 
quickly. The use of homogeneous sets of robots enables 
competitions to be fought on algorithmic innovations 
over robot manufacturing skill. An unrestricted cate¬ 
gory of robot type can be useful for some task-based 
challenges, where a novel physical design may be the 
key in simplifying the problem and achieving the ob¬ 
jective (such as in rescue robotics). 

66.2.4 Competition Scenarios 

The scenario is largely determined by the challenge ad¬ 
dressed, be it a competition among soccer robots or 
among service robots operating in a home environment; 
consequently, there is a broad variety. In some cases, 
the scenario is fixed and known before the competition, 
such as in robot soccer; in other cases, only a gen¬ 
eral description is provided beforehand and the actual 
competition scenario is not known in advance as, for 
example, in search and rescue competitions. The com¬ 
petition scenario has deep implications not only on the 
setup of competition venues, but also on the participat¬ 
ing teams. Practice environments for teams in lead up 
to competition must be representative, while not pro¬ 
hibitive to construct. From Lego robot soccer to the 
DARPA Robot challenge simple readily available hard- 
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ware store materials have been the material of choice 
for contest environments. 

66.2.5 Safety 

In some cases, the competition raises safety issues for 
the participants. This is the case for example of UAV 
competitions, and proper measures need to be taken in 
order to ensure a satisfactory level of safety. Moreover, 
the organizers must carefully address regulations for de¬ 
ployment of unmanned vehicles and/or use restricted 
access areas for the competition. 

66.2.6 Qualification 

Competitions with a large number of participants, such 
as education programs, can require regional and quali¬ 
fying rounds similar to traditional sports leagues. When 
contests require a substantial effort and expense to par¬ 
ticipate funding agencies may make grants available 
to teams which succeed at a qualifying round to con¬ 
tribute to the cost to compete. Diverse selection criteria 
for satisfactory progress may well be considered in this 
case including justification of sound financial backing 
to complete the competition. 

66.2.7 Evaluation 

Evaluation is one of the key features of competi¬ 
tions both in terms of education (students must deliver 
a working solution) and in terms of research, where the 
integration of components is not a straightforward en¬ 
gineering practice. However, competitions also provide 
a great opportunity for developing benchmarks and new 
test methods: performance evaluation is an open issue 
in robotics and competitions are providing significant 
insights into the development of reliable and repeatable 
measures and testbeds. 

66.2.8 Organization 

Competitions are typically originated by a perceived 
need by a related scientific interest group, such as 


a conference (e.g., AAAI, ICRA), a research fund¬ 
ing agency (e.g., DARPA, ESA) or an entrepreneurial 
private company. Once established an ongoing com¬ 
petition founded by the first two groups will often 
be ran through a dedicated not-for-profit foundation. 
However, increasingly, education is a large business 
and competitions are becoming a significant activity 
therein; moreover, companies that help running compe¬ 
titions as projects are now on the market and production 
companies are now looking at competitions as viable 
tools for beyond state-of-the-art product development 
(not restricted to robotics) and for promoting their 
products. 

66.2.9 Sponsors 

The organization of a competition requires a substan¬ 
tial effort and budget. There are, however, several 
sources for acquiring the necessary resources. A first 
category of sponsors are schools that undertake compe¬ 
titions as part of their educational activity. The other 
natural sponsor are the funding agencies that institu¬ 
tionally provide support to research in robotics. The 
involvement of funding agencies can be many fold, 
ranging from the whole creation of a competition to 
the support of specific event, in particular at the lo¬ 
cal level. Another kind of sponsorship comes from 
public institutions that may regard competitions as 
a mean to drive citizens’ attention on scientific and 
technological development. Robot manufacturers are 
also interested in sponsoring competitions that pro¬ 
mote the use of their products. Besides promoting 
specific competitions, companies are also generally 
interested in supporting competitions as a form of 
advertisement. This is obviously related to the kind 
of visibility competitions can reach on the media. 
Moreover, companies may sponsor teams of young 
students as a form of recruitment and supporting 
education. 

As the earlier discussion shows, a successful robotic 
competition not only requires to put forward a grand 
challenge or target specific societal needs, but it also 
requires a careful design and implementation. 


66.3 Competitions Inspired by Human Competitions 


In this section, we focus on competitions among robots 
that take inspiration from human competitions. The 
challenges that have been driving a lot of research in 
Artificial Intelligence, such as chess, go, checkers, are 
inspired by human competitions/games, but they do 
not address the interaction with physical world. On 


the other hand, robotic competitions that are inspired 
by human competitions are typically connected with 
sports. A variety of sports have been addressed, from 
Olympic games [66.9, 10], to robot soccer [66.11, 12], 
to other competitions, such as sailing [66.13] or rac¬ 
ing [66.14]. 
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The distinguishing feature of this category of com¬ 
petitions is the grand challenge not only of emulating 
human performance, but to do this in the context of 
a human competition. Consequently, the robot is in 
principle humanoid, although several other platforms 
have been used, creating games for robots on wheels, 
given that, at this stage, it is a much simpler and ac¬ 
cessible locomotion. It is worth noticing that a big push 
to this kind of competitions was given by companies 
such as Sony or Honda, which developed humanoid 
platforms also as showcases of their research and de¬ 
velopment activities. 

One additional challenge that characterizes the 
human-inspired competitions is that several of them 
involve multiple robots. This feature has several signif¬ 
icant consequences, not only in terms of the require¬ 
ments on the teamwork among the robots, but also on 
the teamwork of the development team. 

The scenario of these competitions is mostly out¬ 
door; however, due to both technological and practical 
reasons often they take place indoor. On the one hand, 
platforms that are waterproof, work on grass or uneven 
terrain are unavailable or much more difficult to real¬ 
ize; on the other hand, practical considerations suggest 
to keep participation affordable and accessible. 

One important feature of human-inspired compe¬ 
titions is that they aim also at providing some enter¬ 
tainment for the general public. Therefore, a suitable 
balance between technical difficulties and level of per¬ 
formance is needed. It is worth noticing how the general 
public is more keen to accepting the limitations of 
robots in playing soccer than in accomplishing a mis¬ 
sion in Fukushima plant. Consequently, even games 
with slow and somewhat disappointing performance 
can generate participation, interest, and even strong 
emotions in the audience. 

Human-inspired competitions span over the whole 
spectrum of participants, including both research ori¬ 
ented and more educational targets. In all cases, the type 
of competition creates an interest both in young genera¬ 
tions that are naturally attracted by their ludic character, 
and in more mature researchers that aim toward the sci¬ 
entific challenges that they embody. A consequence of 
the intuitive appeal exerted by human-inspired com¬ 
petitions is that they easily attract large numbers of 
participants. Indeed, a frequent pattern of participation 
is to start with games and then move to other kinds 
of competitions that target socially or business relevant 
goals. In some cases, this endeavor is encouraged by 
the presence in the same event of both types of compe¬ 
titions. 

The evaluation of the performance, in the case of 
human-inspired competitions, is very clear and amounts 
to winning the game. There are, however, examples of 


specific capabilities associated with the game that are 
individually evaluated. 

Given the reference provided by human competi¬ 
tions, human-inspired robot competitions typically have 
a permanent nature and a structure with local qualify¬ 
ing events. Consequently, there is an organization body 
managing the competition across the years and the main 
ones are nonprofit organizations (NPO)s. There are, 
however, commercial opportunities and companies that 
are interested in the deployment and sale of their robots. 
At present, these companies are mostly sponsoring the 
events, and often provide special offers for their prod¬ 
ucts to the teams that enter the competition. 

As mentioned earlier, human-inspired competitions 
have a strong appeal to the general public. Conse¬ 
quently, public organization often endorse and support 
them within their initiatives aiming at public awareness 
and promotion of science and technology. According to 
a common pattern, this may also attract large companies 
that invest some of their marketing budget in supporting 
socially and scientifically targeted events. 

The ultimate consequence of the above sketched 
scenario is that the human-inspired competitions have 
a large impact on the general public. We have seen re¬ 
ports on human-inspired competitions in the media not 
only in the sections that typically cover science and 
technology, but also in the general news, and some¬ 
times, even in connection with sport reports. 

Besides this somewhat indirect effect, there are two 
major direct impacts of human-inspired competitions: 
on education and research. The renovation and advance¬ 
ment of the school system is embracing competitions 
for two major reasons: they motivate pupils and they 
teach key skills that are typically not addressed by con¬ 
ventional curricular activities. Human-inspired robotic 
competition perfectly fit both purposes. As for research, 
the scientific challenges, that have been put forward, 
whether or not they will turn out to be too ambitious, 
have gathered the attention of significant number of re¬ 
searchers, have driven a large body of research and have 
given rise to a number of side-effect developments: they 
are therefore fulfilling a role that is nowadays essential in 
the overall effort for advancing science and technology. 

Robot soccer has a significant role within this 
category of competitions. The idea of robot soccer 
was discussed by several researchers in the early 
1990s [66.5,6]. Shortly afterward. Robot soccer com¬ 
petitions have been designed, in several venues, includ¬ 
ing Federation of International Robot-soccer Associa¬ 
tion (FIRA) [66.1 1, 15] and RoboCup [66.7, 12, 16]. 

In the next section, we address in more detail the 
RoboCup competitions. RoboCup remarkably started 
with the statement of a challenge, by Hiroaki Kitano 
and the initial RoboCup cofounders [66.7], namely 
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Table 66.1 Five subleagues of the RoboCup soccer games 


League 

Soccer players 

Perception 

Reasoning 

Actuation 

Communication 

Simulation 
(Fig. 66.1) 

(11) Software agents, 2-D 
(two-dimensional) and 3-D 
(three-dimensional) biped 

Server engine 

Client 
per team 

Physical motion and 
perception 

Explicit 

Small-size 
(Fig. 66.2) 

(5) Custom-built wheeled 
robots 

Centralized 

overhead camera 

Centralized 

offboard 

Fast ball/robot speed, 
kicking, dribbling, 
passes, goal shots 

Dedicated Radio 

Middle-size 
(Fig. 66.2) 

(5) Custom-built wheeled 
robots 

Onboard, 

omnidirectional 

Large-space 

control 

Kicking, goal defense 

Dedicated WiFi 

Standard platform 
(SPL) (Fig. 66.3) 

(5) Purchased legged robots 

Onboard, 
directional range 

Distributed 

Legged, kicking, goal 
defense 

Dedicated WiFi 

Humanoid 
(Fig. 66.4) 

(3,4,5) Custom-built, or 
purchased, biped multisized 
robots 

Onboard, 
directional range 

Distributed 

Legged, kicking, goal 
defense 

Dedicated WiFi 


that By the year 2050, there would be a robot soccer 
team that would be able to beat the human World Cup 
team. With such long-term goal, RoboCup is an ongo¬ 
ing competition, that aims at research, educational, and 
engineering challenges through a set of testbeds that 
implement soccer games among robots. The competi¬ 
tion keeps evolving every year, by yearly evaluating the 
performances of the robots and generating roadmaps to¬ 
ward its goal. 

66.3.1 Case Study: Robot Soccer 

The first organized RoboCup competitions took place 
in 1997 [66.16], and significant progress is noted every 
year. Interestingly, starting in 2007, and yearly there¬ 
after, a team of five human players (volunteers from 
the RoboCup Board of Trustees) plays a demonstra¬ 
tion game with the best robot team of the middle-size 
league, which we introduce below. Furthermore, each 
RoboCup competition event includes a symposium, 
which highlights research and development contribu¬ 
tions relevant to the RoboCup competitions, and also to 
general robotics problems. As of 2014, the robots can 
play the basics of the game, with great speed and accu¬ 
racy, but are clearly largely noncompetitive with human 
players. There is still a long way to go for RoboCup to 
reach its goal, but RoboCup continues to achieve annu¬ 
ally increasing success, scope, and wide participation. 

RoboCup soccer games are structured in five sub¬ 
leagues, where robots have different sizes and different 
capabilities, as overviewed (Table 66.1). 

Many scientific challenges are addressed through 
different league settings, including teamwork and 
strategic decision making, in particular with systems 
of multiple robots in adversarial environments. To this 
end, the setup of each competition has evolved along 
the years from the initial proposal [66.7]. The sim¬ 
ulation, small-size, and middle-size leagues were the 


initial leagues in 1997. The standard platform league, 
started as a demonstration in 1998 and as a com¬ 
petition in 1999, was played with the Sony AIBO 
quadruped robots until 2007, and since 2008 with the 
Aldebaran NAO robots. Finally, the Flumanoid Robot 
League started in 2000 with team-built biped robots 
that could barely stand on one leg to kick, probably to 
the most compelling derivative in improvement, leading 
into 2012 to biped robots of a variety of sizes that play 
complete games. 

We now briefly describe the individual goals and 
set up of the soccer leagues, aiming at discussing the 
technical challenges and solutions devised. A sum¬ 
mary of the complete technical contributions is found 
in the RoboCup symposia proceedings [66.16-32] 

Simulation 

The RoboCup simulation league was developed from 
the very beginning of RoboCup, as an 11-player two- 
team simulation server environment, the SoccerServer, 
conceived and implemented by Noda [66.33]. The sim¬ 
ulation league consists of a client-server architecture 
in which each of the two client teams is given, as an 
input by the SoccerServer, the state of the game (play¬ 
ers state and ball position), and generates actions for 
the simulated robots, which are passed to the server 
for simulation of the new state of the world. Given the 
controlled realistic simulated environment, the goal of 
the league is to enable the researchers to abstract from 
the complications of real hardware and focus on team¬ 
work architectures and coordination, strategic decision 
making, and learning in complete teams of 11 players. 
Initially, the league was a 2-D simulation, and since 
2009, and following its roadmap to the RoboCup goal, 
it was extended to include a 3-D competition, leading to 
the current two subleagues: the 2-D and 3-D Simulation 
Leagues (Fig. 66.1). 
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Fig.66.1a,b Simulation 
leagues: (a) 2-D and (b) 3-D 



Fig.66.2a,b RoboCup soccer 
leagues: (a) small-size robots and (b) 
middle-size robots 


In the 2-D simulation subleague, the SoccerServer 
simplifies the environment by representing robots as 
basic circular elements, though with orientation and 
different capabilities in terms of perception and actu¬ 
ation. The client teams focus on effective team com¬ 
position and on the strategic aspects of the game. In 
the 3-D Simulation subleague, the players are sim¬ 
ulated biped humanoids. The teams need to address 
low-level control of such humanoid robots, including 
the creation of their basic behaviors, such as biped 
walking, running, kicking, turning, and standing up. 
In addition, the teams design and implement multi¬ 
agent higher level behaviors for realistic humanoid 
robot teams. The 3-D simulation league is currently 
tightly connected to the standard platform league, as 
the simulated humanoid robots are based on the char¬ 
acteristics of the Aldebaran NAO humanoid robots, 
as used in the RoboCup standard platform league 
(see in the following). The goal is to directly transfer 
techniques and approaches from simulation into real 
robots. Both the 2-D and 3-D servers are open software 
projects. 

Small Size 

The RoboCup small-size league (SSL) has been in place 
since the initial RoboCup in 1997. The league is set 
up as a field of predefined dimensions for teams of 
custom soccer robots of predefined small dimensions. 
Initially the teams were of three robots, and later have 
been increased to five robots. The initial playing field 
was a green ping-pong table, which was selected prag¬ 
matically by the founders in 1996, in order to make the 
specifications of the newly created RoboCup competi¬ 


tion accessible to any research team in the world. A few 
years later, the field became green carpeted with white 
line markings, and its size has incrementally grown to 
the recently increased size of 6 m x 9 m. Since its be¬ 
ginning, the game is played with an orange golf ball 
and an overhead vision system. Robots are custom built 
according to the specifications of the league, currently 
to fit within a 180 mm diameter circle, no higher than 
15 cm (Fig. 66.2). 

The teams are allowed to use at least one camera 
overhead of the field. Robots have colored marks on 
their tops to enable visual identification and tracking. 
Off-board computation by each team uses the processed 
images to plan and then wirelessly communicate the 
team actions as the motion of the small robots on the 
field. The RoboCup SSL league offers challenges and 
significant contributions in: hardware design, colored- 
based real-time image processing (of the colored ball 
and colored marker identifiers of the robots), and in 
team strategy planning. 

The RoboCup SSL robot hardware has evolved 
from the early simple and brittle robots in 1997 to the 
sophisticated, fast, robust, kicking, and dribbling, om¬ 
nidirectional robot platforms of the current days. The 
hardware of the RoboCup SSL clearly pioneers small 
robot design, in contrast with all other much larger 
research robots in other applications. The league also 
significantly evolved in the vision processing and the 
playing strategy. Although the global view of the com¬ 
plete playing field is a clear advantage in capturing 
a complete view of the position of all the players and 
ball, it proved to be quite challenging to process in real 
time at 60 Hz, 10 fast moving robots, and a small orange 
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golf ball. As the league evolved, and some teams suc¬ 
ceeded in devising vision processing algorithms, since 
2010 such solutions became a shared vision system 
as part of the league setup that can be used by any 
team [66.34]. This SSL vision is available to all re¬ 
searchers and can be set to be used in other applications 
that may need global overhead perception. In terms of 
strategy, the teams aimed at passing and positioning 
well on the field since the beginning of the league. Both 
the capabilities of individual robots and teamwork be¬ 
came increasingly sophisticated and effective over the 
years, with the ability to dribble, to flat or chip kick, 
to perfectly navigate collision free at high speeds in 
a space crowded with opponents as dynamic obstacles, 
and to coordinate the team with effective defense, ac¬ 
curate passes, and precise aim at narrow openings of 
the goals. shows an example of the multi 

robot teamwork by the CMU Dragons. 

The ongoing research directions aim at learning 
the opponent strategy and responding to it. Through 
the common vision systems, all games are logged 
and now available for researchers to challenge learn¬ 
ing algorithms with the complex positional, fast game 
data. Furthermore, the league aims at playing games of 
eleven players on a much larger field, also in ad-hoc 
teams, where the players are elements of different and 
previously unknown research teams. 

Middle Size 

The RoboCup middle-size league (MSL) started in the 
first RoboCup event in 1997. The league is set on 
a much larger field than the SSL one, initially of the 
size of 3 by three ping-pong tables. Currently, two MSL 
teams of six wheeled robots play onal8mxl2m green 
carpeted field, with an official FIFA winter ball (or¬ 
ange). 

MSL clearly differs from the SSL in its lack of 
centralized overhead perception and offboard compu¬ 
tation. Each MSL robot is onboard fully autonomous, 
equipped with onboard sensors, computation, and actu¬ 
ation. Through wireless communication the robots can 
establish interteam cooperation and receive all referee 
commands. These two SSL and MSL leagues, were 
designed to offer frameworks for different robotics re¬ 
search interests in perception, planning, and mechanical 
robot design. 

The MSL robots have evolved from a variety of 
initial experimental designs to the very effective, fast 
omnidirectional robots with creative kicking and de¬ 
fending actuation devices, in particular also in the 
goalie robots. At some point, robots had omnidirec¬ 
tional cameras that allowed each robot to compute the 
position of all the players on the field, for avoiding 
obstacles in its motion planning. Wireless communica¬ 


tion allows for sharing state information and supports 
the implementation of cooperation strategies. Currently, 
teamwork accomplishments are based on the effective 
selection of robots to roles and attacking passes. In par¬ 
ticular, the recent rules of the game force the robot to 
pass the ball in the opponents middle field before shoot¬ 
ing to the goal. In this way, cooperation and passes have 
become a key feature for a successful team. The MSL 
goalie robots have been of special note, as teams build 
special hardware for them to allow for kicking and to 
effectively defend the long and well-aimed shots from 
the attackers. The MSL is compelling for the largest 
field size in RoboCup and robots of dimensions and 
speed that also allow humans to test play with the 
robots. Since RoboCup 2007, a team of volunteer mem¬ 
bers of the RoboCup Board of Trustees always plays 
a game against and with the MSL winning team. Such 
demonstrations are well appreciated by the competition 
audience and participants and provide evidence to the 
progress of the performances of the robots. 

Standard Platform 

The standard platform league started with the goal of 
providing a common hardware platform, so that the 
teams can focus their development on the algorithms 
and compare the performance of their solutions on the 
same hardware. Thus, in the standard platform league, 
all teams use identical robots, which happened to be 
legged robots. In the standard platform league all the 
challenges of the middle-size league are present (i. e., 
the robots operate onboard fully autonomously), but on 
a platform that requires a sophisticated motion control. 
The motion of the robot also affects perception and 
causes a greater uncertainty in the estimation of the en¬ 
vironment and, thus, in basic capabilities such as ball 
tracking and localization of the robot in the field. The 
field evolved over the years: it is currently 6 mx 9 m and 
the game is played by two teams of five players with an 
orange ball. 

The league started with the Sony AIBO (Fig. 66.3), 
in its early stages of development in 1998. After 
the first year demonstration, the league attracted sev¬ 
eral researchers, and underwent a substantial develop¬ 
ment both in terms of the platform, that evolved in 
three major revisions after the initial prototype, and 
in terms of the rules and requirements for the field. 
The AIBO robots had a very high number of ac¬ 
tuators (16—18) to support 4-legged locomotion and 
body pose. At each revision, robustness of mechanical 
design, sensors, and computational power, steadily im¬ 
proved, thus, allowing to increase the size of the field 
(from 3 m x 4 m, to 4 m x 6 m) and to remove artificial 
landmarks, side barriers and strict requirements on the 
lighting. 





Robotics Competitions and Challenges I 66.3 Competitions Inspired by Human Competitions 1767 



Fig. 66 . 3 a,b Standard platform 
league: (a) AIBOs and (b) NAOs 


Both the extremely advanced features of the plat¬ 
form and the availability of software developed by the 
RoboCup teams made the AIBO a very popular plat¬ 
form in research laboratories to test the performance of 
new legged locomotion techniques, perception localiza¬ 
tion and teamwork. As the performance of the robots 
improved over the years, not only the rules have evolved 
to meet new technical and scientific challenges, but also 
the games became very attractive for the general audi¬ 
ence. 

When Sony announced the discontinuation of the 
production of the AIBOs, RoboCup issued a Call for 
Platform that ended up with the choice of the current 
standard platform: the humanoid NAO by Aldebaran 
Robotics (Fig. 66.3), which has been used since 2008. 
The set up of the field did not change substantially; 
however, the biped platform required not only new loco¬ 
motion, but also redesign of all the basic components. 
Again the platform evolved over the years, with cor¬ 
responding changes of the field and rules. The current 
version of the NAO robot is 58 cm tall humanoid robot, 
with 25 degrees of freedom. The processing is pro¬ 
vided by an Intel ATOM 1.6 GHz CPU (located in the 
head) that runs a Linux kernel and supports Aldebaran’s 
proprietary middleware (NAOqi) and a second CPU 
(located in the torso). The platform is equipped with 
various communication devices, including voice syn¬ 
thesizer, LED lights, and two speakers, two cameras 
(one pointing toward the feet to control the kicking), 
tactile sensors and sonar rangefinders, Ethernet and Wi¬ 
Fi. The capabilities of the current platform allowed to 
enlarge the field to the current size (to 6 m x 9 m) and 
to remove from it all artificial features. Each game is 
played by five robots, plus an additional coaching robot 
outside the field. A section of the competition has re¬ 
cently started, where teams are formed by robots that 
are designed by different participants. 

The adoption of a common platform has several ad¬ 
vantages. First of all, the hardware is chosen so that the 
teams have a powerful, yet affordable robot, and speed 
up the development by sharing the same infrastructure. 


Moreover, a standard hardware platform system makes 
a systematic comparison of different technical solutions 
more accurate. Finally, a common platform supports 
significantly community development that allows to 
reach the critical mass needed to develop complex 
robotic systems. 

Humanoid 

In the humanoid league, autonomous robots with 
a human-like body plan and human-like body senses 
play soccer against each other. The robots are designed 
and built by the competing teams, and include some of 
the best autonomous humanoid robots in the world. 

The humanoid league was established in RoboCup 
2002, Fukuoka. The later start, as compared with the 
other leagues is connected with the development of 
humanoid robots. The first demonstration of the early 
humanoid robots took place in RoboCup 2000 and 
Honda ASIMO first showed the ability to kick the ball, 
as a demonstration, in RoboCup 2002. Since then, the 
league started, initially addressing basic skills such as 
walking and kicking: the soccer game was limited to 
penalty kicks. In the first years of the league, very dif¬ 
ferent robots were competing, but soon a separation 
in subleagues was gradually introduced, based on the 
size of the robots and on the constraints on the struc¬ 
ture of the robot aiming to enforce dynamic walking. 
In 2005 the first two against two games were started in 
the smallest robot subleague. Since 2008, the game had 
three players in the kid-size and the league made a sig¬ 
nificant progress in terms of the number of participating 
teams and the capability of the hardware platforms. In 
the current setting, the robots of the humanoid league 
are divided into three size classes (Fig. 66.4): Kid- 
Size (30—60 cm height), TeenSize (90—120 cm), and 
AdultSize (130cm and taller). In the KidSize soccer 
competition, teams have three players, in the TeenSize 
two players (keeper and forward) and in AdultSize soc¬ 
cer, a striker robot plays against a goal keeper robot 
in a dribble and kick competition. The size of the 
field is 4 m x 6 m and the ball is of different size for 
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Fig.66.4a-c Humanoid Robots: (a) KidSize, (b) TeenSize, (c) AdultSize 


the different subleagues. Unlike humanoid robots out¬ 
side the Humanoid League, the task of perception and 
world modeling is not simplified by using nonhuman¬ 
like range sensors. 

The challenges addressed by the Humanoid League 
embrace those arising in the realization of fully au¬ 
tonomous humanoid robots in the standard platform 
league; however, the overall design, including the hard¬ 
ware, makes it possible to aim at a richer range of 
solutions. More specifically, key issues addressed in 
the Humanoid League include dynamic walking, run¬ 
ning, kicking the ball while maintaining balance, visual 
perception of the ball, other players, and the field, self¬ 
localization, and team play. 

Conclusion 

RoboCup is recognized as a major forum for research 
and education through competitions. The main inter¬ 
national yearly event gathers about 200 teams, 1500 
participants (not including Junior), and tenths of thou¬ 
sands visitors. Regional events are also held worldwide, 
some of them of a size comparable to that of the 
main one. RoboCup is not only soccer, since it aims 
at fostering the development of new solutions also in 
applications that have a direct impact in our society: 
RoboCup Rescue, where robots compete to search and 
rescue in disaster scenarios, RoboCup @ Home, where 
robots perform tasks that help people in daily life, Lo¬ 
gistic League and the RoboCup @ Work, where robots 
operate in industrial settings. Moreover, RoboCup pro¬ 
vides a link to young generations through RoboCup 
Junior, a set of competitions specifically designed for 
the introduction of children to robotics and organized 
within the major event. 

RoboCup has a profound impact on research. Re¬ 
searchers enter RoboCup competitions to validate their 
newly designed approaches to the broad spectrum of 
problems that arise from the grand goal of building 
teams of autonomous robots. Algorithms and robots 
are put to test in adversarial environments not pre¬ 
viously seen, to show effective performance and ad¬ 


vance the scientific and engineering state of the art in 
Robotics. There is a countless number of publications 
that present, in the most qualified Artificial Intelligence 
and Robotics venues, scientific contributions arising 
from the technical challenges tackled in RoboCup and 
validated through RoboCup games and platforms. The 
approaches and solutions developed in RoboCup have 
impacted many research areas. Particularly significant 
examples are the fields of humanoid robotics, multi¬ 
agents and multirobots, and performance metrics for 
Robotics. 

RoboCup has brought significant contributions in 
education. The students, who form the teams, are the 
driving force of the effort to design and implement 
the components of rather complex integrated systems. 
Learning about cutting-edge solutions and experiment¬ 
ing them is a very effective way to develop creativ¬ 
ity and design skills. Moreover, engineering within 
a team, in the framework of a competition, is a unique 
experience that goes far beyond the standard prac¬ 
tice of academic curricula. In addition, RoboCup or¬ 
ganized several Summer Schools, with a substantial 
hands-on component that has also been subsequently 
adopted in many other events. While some of the 
above features are not unique to RoboCup, the blend¬ 
ing of ingredients and the environment provided by 
RoboCup have proven extremely rich, attractive and 
successful. 

A few examples of particularly significant impacts 
that RoboCup had within our society are hinted below. 
Aldebaran Robotics NAO robot won RoboCup 2007 
Open Call for a standard platform and afterward it has 
become popular through the standard platform league. 
Today NAO is deployed worldwide by research labora¬ 
tories in robotics, becoming a commercial success. Kiva 
Systems are revolutionizing automation by using hun¬ 
dreds of mobile robots for storing, moving, and sorting 
inventory. In March 2012, Kiva Systems was acquired 
by Amazon. RoboCup and robot soccer were credited 
by Kiva’s CEO for the success of the company. The 
search and rescue robot Quince, during the rescue oper- 
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ations in the Fukushima-Daiichi Nuclear Power Plant, 
could go up to the second to fifth floors in the nuclear 
reactor buildings. The robot was designed and evalu¬ 


66.4 Task-Oriented Competitions 

In this section, we consider competitions that take their 
inspiration from common, but important tasks, cur¬ 
rently undertaken by people. Competitions in this class 
tend to be focussed of dirty, dangerous and dull activ¬ 
ities. Robotics of this kind is known as Field Robotics 
with robots typically having to operate in unstructured 
or semistructured environments, which often change 
over time. 

The robots in these competitions are normally vehi¬ 
cles, such as ground vehicles (often cars), submersibles, 
or unmanned aerial vehicles. There is even a current 
high-profile competition to send a robot to the Moon, 
the Google Lunar X-Prize. 

Unlike the competitions inspired by humans, task- 
oriented competitions are not driven by spectator ex¬ 
perience. Spectators are welcomed at some of the 
competition events but not at all. Some are closed to 
competitors, organizers, and media only. 

Task oriented competitions are held to drive inno¬ 
vation, raise the profile of a particular area of robotics, 
or to overcome nontechnical hurdles such as regulation 
and insurance which would otherwise stifle individ¬ 
ual research projects. A competition is a more efficient 
method of addressing these issues than a collection of 
disparate research projects across the world. This type 
of competition also attracts professionals in robotics 
or soon to-be professionals such as university students. 
They are hence a useful way to increase interest in in¬ 
dustry participation and you will see major corporations 
with interests in the relevant technology and applica¬ 
tions areas supporting or sponsoring a competition. 

Unlike robot competitions inspired by sporting con¬ 
tests, task oriented competitions often happen as one- 
off events. 

Task-based competitions in which the exact manner 
by which the task is to be accomplished is unknown, 
or significantly beyond the state of field, are often re¬ 
ferred to as a Challenge. Challenges often have large 
cash prizes for the team that accomplishes the task re¬ 
flecting the substantial investment of effort envisaged to 
claim the prize by the prize authority. 

Ultimately even substantial cash prizes are almost 
always surpassed by the accumulated cost of the win¬ 
ning team’s labour, equipment, donations by sponsors 
and overheads. Instead most participants are driven by 
prestige of conquering the challenge, doing the im¬ 


ated through RoboCup Rescue competitions, showing 
excellent mobility in comparison with the other com¬ 
peting robots. 


possible and claiming the prize, or at least attempting 
to. 

In recent years, the success of the robot compe¬ 
tition format has lead to a number of contests in¬ 
cluding the MAGIC Competition [66.35], RoboCup 
Rescue [66.36], ELROB [66.37], and the AUVSI com¬ 
petitions [66.38]. 

A push to robot competitions to task-oriented robot 
challenges has also been given within the framework of 
scientific conferences such as AAAI and ICRA. Within 
this context, service robotics has also been a source 
of inspiration, addressing tasks such as serving drinks 
or preparing sushi. RoboCup@Home combines multi¬ 
ple challenges facing service robotics into a structured 
competition framework [66.39]. 

In the following sections, we examine at two repre¬ 
sentative task oriented competitions, the UAV challenge 
and the DARPA Challenge. While the robots differ sig¬ 
nificantly the drivers and approach to competition has 
many parallels. 

66.4.1 Case Study: The UAV Challenge 

Flying Robot Competitions 
Competitions that encourage students to develop their 
skills with flying robots in exciting and challenging 
ways have been developed by universities and other 
research and education organizations since the early 
1990s. One of the first, longest running and most suc¬ 
cessful flying robotics competitions is the International 
Aerial Robotics Competition (IARC) [66.40]. It was 
first held in 1991 at Georgia Institute of Technology. 
The aim of the competition is to push forward the 
state of the art in autonomy in flying robots. Teams are 
given challenges based on real-world scenarios, typi¬ 
cally mock ups of disaster response or covert military 
style operations. A new challenge is announced once 
the current scenario is completed. Many scenarios take 
a number of years to be successfully demonstrated. 
Since 1991, IARC has seen six different competition 
scenarios. Nearly all of the scenarios involve inter¬ 
action with ground elements meaning that the aerial 
robot must avoid obstacles and pick something up. This 
means that the successful platforms of IARC are nearly 
always rotary wing vehicles such as helicopters, ducted 
fan VTOL vehicles or in more recent year, multiro- 
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tor platforms. I ARC is open to university-based teams 
from around the world. The past three IARC missions 
have involved the aerial robots having to navigate inside 
buildings as well as free-flying outdoors. 

Another long running event is the International Mi¬ 
cro Air Vehicles (IMAV) competition, which has been 
running since 1997 [66.41]. This competition focusses 
on very small unmanned aircraft operating either in¬ 
doors, or in outdoor areas around an airfield or built 
environment. The IMAV events consist of a number of 
subevents, each aiming to push technology to solve spe¬ 
cific research challenges, such as autonomy, perception, 
and flight dynamics. As reported in [66.41], the level of 
autonomy demanded to complete events has been in¬ 
creasing in recent years. 

The Singapore Amazing Flying Machine Competi¬ 
tion (SAFMC) is organized by Singapore’s DSO Na¬ 
tional Laboratories and Science Centre Singapore and 
was first run in 2009 [66.42,43]. It is typical of the type 
of competition event that is comprised of a number of 
different categories, from a basic radio-controlled flying 
competition to a category encouraging university level 
teams to design and build automated aircraft to navigate 
complex indoor obstacle course. 

The UAV Challenge Outback Rescue is an inter¬ 
national flying robot challenge that was first run in 
2007 [66.44]. It is different to other competitions in 
that the aircraft have to be flown a considerable distance 
from the take-off and landing location. The competition 
revolves around a simulated scenario of a lost walker in 
the Australian outback with the start of the search area 
being 4 km from the take-off location and the furthest 
point of the search area being 6.5 km away. These dis¬ 
tances require autonomous operation of the aircraft and 
reliable communications back to the ground stations - 
all things that stretch the capability of university and 
amateur teams [66.45, 46]. 

In 2013, the Wildlife Conservation UAV Challenge 
was launched [66.47]. This competition has a similar 
format to that of the UAV Challenge Outback Rescue, 
but is focussed on a scenario to find poachers of endan¬ 
gered animals in Africa. The aim of this challenge is 
to drive the cost down, and the reliability up, of small 
robot aircraft. The scoring system hence rewards teams 
heavily for low-cost designs, long duration capability 
and autonomy. 

There are numerous other flying robot competi¬ 
tions run each year, including the USA-European micro 
UAV flight competition [66.48], the AUVSI Seafarer 
Annual Student UAS Competition [66.49], the Inter¬ 
national Universities MINI UAV Competition [66.50], 
the Chinese Robotics Competition which has a fly¬ 
ing robot category [66.51], and the Annual Search 
and Rescue Cyber Physical Systems Challenge, where 


teams build ground vehicles and the organizers provide 
a video feed from a UAV hovering above the indoor 
search area [66.52, 53], A final category of flying robot 
competitions are the type held solely for participation 
within single universities and which are typically part of 
core course work in robotics or extension course work. 
A representative example of this type of internal univer¬ 
sity competition is given by Schochmann [66.54]. 

Motivation for Flying Robot Competitions 
The stated aim of the longest running flying robot con¬ 
test, International Aerial Robotics Competition, is to 
drive innovation [66.40], particularly in the area of au¬ 
tonomy. Many of the flying robot competitions and 
challenges of the past decades have focussed on this 
technology development or innovated uses of existing 
technology. However, not all flying robotics competi¬ 
tions and challenges are created for this reason alone, 
with some being linked to the education of high-school 
or university students who are studying robotics, while 
others are conceived to overcome nontechnologically 
based problems. There are a number of key issues that 
are potential inhibitors to the adoption of flying robots 
in the civilian context. In the mid-2000s, the general 
public considered flying robots as a military-only tool, 
being expensive and dangerous. This was largely due to 
a number of popular movies at the time depicting such 
things and high media interest in the use of unmanned 
aircraft by the USA in the Middle East conflicts of the 
time. In the 2010s, the public was concerned with pri¬ 
vacy issues surrounding the use of drones (as they have 
become known in the media) and the fear that small 
unmanned aircraft could be misused by criminals or ter¬ 
rorists [66.55]. 

A second significant issue is that of airspace reg¬ 
ulation. Flying robots will share the sky with manned 
aircraft. The airspace inhabited by manned aviation is 
highly regulation across the world. Before flying robots 
can be routinely used for civilian missions, both the 
airspace regulator of a particular country, and the gen¬ 
eral public in that country must be comfortable that 
flying robots will be as safe as current conventionally 
piloted aircraft. This issue is the single most significant 
hurdle to the adoption of unmanned aircraft in civilian 
airspace. A third issue, strongly related to the regulation 
and safety issue, revolves around the appropriate train¬ 
ing, certification, and insurance of unmanned aircraft. 
Insurance companies work on precedence and past ac¬ 
cident rates in order to determine risk and hence set 
insurance premiums. In a new industry, such as that of 
unmanned aircraft, with little data on which to basis risk 
probabilities it is difficult for insurers to cover opera¬ 
tors. Finally, as the unmanned aircraft industry grows, 
it will need people with the correct skills. These new 
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Fig. 66.5 (a) The aim of the Search 
and Rescue Challenge is to locate 
and drop a water bottle to a walker, 
Outback Joe. The mannequin wears 
bright clothes to help the teams find 
him using computer vision, (b) The 
course showing the Mission Boundary 
( yellow ) and the Search Area {green). 
The airfield is at the top of the picture 


Engineers, operators, technicians, and researchers will 
need to be trained in unmanned aircraft-specific areas 
of expertise. 

The UAV Challenge Outback Rescue competition 
was created in Australia in 2007 [66.44] to attempt 
to simultaneously address all these major inhibitors to 
civilian use of flying robots. These broad aims meant 
that the targeted participants were from a wide cross- 
section of the community and included, the general 
public, high-school students, university students, radio- 
controlled aircraft hobbyists, the Australian aviation 
regulator (CASA), government, research institutes and 
universities, aerospace industry, and media. 

Competition Formats and Challenges 
The majority of flying robot competitions and chal¬ 
lenges consist of multiple subevents or competitions, 
usually aimed at either different age groups of competi¬ 
tors or different skill levels. The level of complexity 
and the difficulty of the specific tasks set are set so 
that younger, less experienced students will almost 
always succeed to some degree and are hence compet¬ 
ing against others like a sporting event. On the other 
hand, the events aimed at older or more experienced 
entrants are often deliberately conceived to be impos¬ 
sible to complete the first year that a team attempts 
it. These events are often referred to as Challenges 
while the easier events, where a team is expected to fin¬ 
ish, are known as Competitions. A case in point is the 
UAV Challenge Outback Rescue that comprises a dif¬ 
ficult Challenge, that is open to teams from around the 
world, and a high-school competition, that is far sim¬ 
pler, but still challenging, for high-school aged students. 
The two events share a common theme to encourage 
the high-school students to consider progressing to the 
Challenge in later years. The scenario for the UAV 
Challenge is that of locating a lost walker and dropping 
him a water bottle. Such a scenario is appealing to the 
general public, the media and the government. 

The UAV Challenge Outback Rescue’s major event 
is known as the Search and Rescue Challenge and was 


unique in the world at the time it began in 2007 in that 
the competition took place over a large area of farmland 
and public roads. The lost walker was represented by 
a mannequin and was placed in a farmer’s field approx¬ 
imately 4—6.5 km from a public airport in the regional 
town of Kingaroy in Queensland, Australia. The aim 
of the Search and Rescue Challenge was for a team to 
use their flying robot to find the lost walker (Fig. 66.5) 
and then drop him a water bottle. If a team managed 
those tasks, returned their aircraft safely to the airport 
and the water bottle landed intact less than 100 m from 
the bushwalker, they would win $50000. The Search 
and Rescue scenario was created to be as realistic as 
possible with competitors given a briefing document at 
the start of the day outlining where the lost walker was 
last seen and what he was wearing. Teams were under 
time pressure and given time limits on setup, mission 
time and pack up time. All of these restrictions were 
designed to make the scenario as close to a real rescue 
as it could be. Teams were evaluated on their mission 
performance and on their documentation meaning that 
even if no team completed the mission, their could still 
be a points winner, who would be awarded the title of 
First Place and receive a smaller cash prize. This is typ¬ 
ical of Challenge-type events where a number of years 
may go by without a team completing the main chal¬ 
lenge. Another typical feature of this event it is that it is 
not set up as a spectator event because the nature of the 
Challenge makes it almost impossible to follow if you 
are not directly involved in the action. 

The high-school competition of the UAV Challenge 
Outback Rescue is known as the Airborne Delivery 
Challenge as takes place on the airport site and within 
the airport boundary. This competition is therefore set 
up to be a spectator friendly event. This scenario re¬ 
quired high-school students to pilot their unmanned 
aircraft over the same walker, who is located in sight of 
the general public close to the air-side fence. Teams are 
required to drop a package to the walker, but the pilot 
cannot select when to make the drop. Instead, a second 
team member located in a tent and with no line of sight 
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Fig. 66.6 The Airborne Delivery Challenge course showing the hurdles and the flying zone 


to the bushwalker has to make the drop (Fig. 66.6). The 
aim of this scenario is to force teams to use some tech¬ 
nology to help assess when to drop the package. Many 
teams are successful and a points system determines the 
winner. 

Qualification and Precompetition Testing 
Some flying robot competitions and challenges have 
a qualification element to them which means that teams 
must demonstrate a certain level of competence prior 
to the actual event. This requirement implies that many 
of these competitions can be considered to be much 
longer in duration than the relatively short events that 
take place on the competition days at the end. The UAV 
Challenge Outback Rescue for example, takes place 
over a long time period leading up to the event. Since 
2007, the Search and Rescue Challenge has been held 
both over a single year and as a 2-year long campaign. 
In both forms of the event, the high-school and the inter¬ 
national open event, teams must provide documentation 
to the organizers to demonstrate their competence, their 
progress, and their treatment of safety. Safety is the 
priority issue for the organizers and the teams at fly¬ 
ing robot competitions and challenges. In the case of 
true UAV Challenge, the unmanned aircraft are oper¬ 
ated over farmland, over public roads and close to the 
general public who come to spectate (Fig. 66.5). 

Teams first register for the UAV Challenge and 
are then asked to provide documentation, known as 
Deliverables. The first deliverable must show a basic 
system design and an analysis of risk. The second de¬ 
liverable must show the final design and adherence 
to the mandatory safety features of the UAV system. 
For the 2-year long version of the Search and Res¬ 
cue Challenge, a third deliverable was added where 
teams were asked to show documented evidence of at 
least 5 h of autonomous flight of their unmanned air¬ 
craft. This final set of documentation was introduced 


to ensure that only teams that were capable of actu¬ 
ally flying for a sustained period of time in the search 
range qualified for the event. By its nature, the qual¬ 
ification process significantly reduces the size of the 
field as a competition or challenge progresses. For ex¬ 
ample, in 2011—2012, 68 teams registered to take part 
in the Search and Rescue Challenge, with 61 submit¬ 
ting a the first deliverable document. Of those, 53 teams 
from ten countries, qualified to continue. At the second 
deliverable stage, 23 teams qualified to continue to the 
final deliverable phase where nine teams qualified and 
were invited to the actual flying event. Between that 
announcement and the event, four of the nine teams 
pulled out due to crashes during testing and other is¬ 
sues leaving just five teams to compete during the event 
(Fig. 66.7,66.8). 



Ph 


Fig. 66.7 The number of teams declines as the UAV 
Challenge progresses. In 2011—2012, 68 teams registered 
and throughout the 18 months of the competition teams 
dropped away leaving only 5 at the event 
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Fig. 66 . 8 a,b (a) First place team from the 2012 Search and 
Airborne Delivery Challenge, MUROC Hawks 


It is common at flying robot events to test teams 
prior to the actual competition flights. In the UAV 
Challenge Search and Rescue event, the final phase of 
qualification involves the teams being tested or scru- 
tineered. The scrutineering process aims to test both 
the safety systems of the unmanned aircraft, the sup¬ 
porting systems and the team themselves, including 
any safety pilots. The scrutineering is undertaken by 
industry professionals and consists of a static scruti¬ 
neering test where the team is interviewed, the aircraft 
and systems inspected and the flight termination tested. 
Teams that pass scrutineering are then invited to fly 
their aircraft within the boundary of the airport where 
the ability of the team to fly autonomous is tested. 
Only teams that pass this final hurdle are invited to 
launch a search and rescue mission and compete for the 
grand prize. In 2011—2012 Search and Rescue Chal¬ 
lenge, four teams passed the final scrutineering tests 
meaning that just over 5% of teams that originally reg¬ 
istered qualified for the final mission. In comparison in 
the 2013—2014 event, this qualification rate increased 
to 17%. 

Regulations and Safety 

Many flying robot competitions are carried out indoors. 
As well as guaranteeing that bad weather will not cause 
problems for the event, a major reason to hold a com¬ 
petition indoors is that in most countries, this means 
that the official airspace regulator does not need to 
be involved in any way. Unmanned aircraft flown in¬ 
doors pose no threat to manned aviation flying above. 
However, a number of high-profile flying robot com¬ 
petitions do take place outdoors, including IARC and 
the UAV Challenge Outback Rescue. The latter event 
must be undertaken within the specific regulations for 
the operation of unmanned aircraft in Australia. These 
regulations were launched in 2002 by Australia’s civil¬ 
ian air regulator, the Civil Aviation Safety Authority 
(CASA) and are known as CASR Part 101. The reg¬ 



Rescue Challenge, Canberra UAV. (b) Winners of the 2012 


ulations outline the operating rules for three defined 
classes of unmanned aircraft, micro UAS under 100 g, 
small UAS 100 g to 150 kg and large UAS 150 kg and 
above. The rules of the UAV Challenge dictate that the 
competing team’s cannot enter an aircraft in the large 
UAS class. Normal commercial operation of small UAS 
would require the operating business to obtain an Oper¬ 
ators Certificate from CASA. However, the competition 
is classified Sport and Recreation by CASA and hence 
an Operator Certificate is not required. CASA treats the 
UAV Challenge organizers as the body accountable for 
the safe conduct of the event and adherence to the rel¬ 
evant regulations. The UAV Challenge organizers have 
developed the event rules to specifically meet the re¬ 
quirements of the regulations and to hence ensure that 
if a team is following the rules, they are in turn not in 
breach of the regulations. Further to this, the organizers 
have also developed a detailed Policy and Procedures 
manual that outlines how the UAV Challenge will be 
conducted to ensure safety and adherence to the reg¬ 
ulations by the organizers themselves. Developing the 
manual and the supporting rules has been a major activ¬ 
ity of the organizing committees over the years that the 
event has been run and is a significant output from the 
whole UAV Challenge activity. 

What makes the UAV Challenge different to many 
other unmanned aircraft competitions is that because 
the event takes place largely beyond visual range of 
the team’s ground controllers, and an aircraft could be 
up to 7 km from the ground station, there is a higher 
potential for a so-called fly-away than there is dur¬ 
ing the more common style of events where the flying 
takes place in close proximity to the ground station. 
A fly-away is where the aircraft flies out of the des¬ 
ignated competition area and cannot be commanded 
to return. Such an incident could be seen as a major 
setback to the developing unmanned aircraft industry. 
In the first 2 years of the UAV Challenge, the mitiga¬ 
tion for fly-away was to mandate the use of a flight 
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Fig.66.9a,b DARPA grand chal¬ 
lenge vehicle ancestry. Pioneering 
autonomous vehicles of 1980s, 1990s. 
(a) 1985: UBM's Varnors. (b) 1995: 
CMU’s Navlab 5 used for the No 
Hands Across America trial 


termination mode whereby any aircraft that crossed 
the defined competition boundary or lost contact with 
the ground station had to automatically perform a fast 
dive into the ground. All aircrafts were tested during 
the scrutineering sessions to ensure that this behavior 
was built into the systems. Since, 2010, the require¬ 
ments have kept step with the more common industry 
approach to lost communications and teams are now 
allowed to deploy aircraft that will automatically re¬ 
turn to a predetermined location in the competition area 
should communications be lost between the ground sta¬ 
tion and the aircraft. The requirement to automatically 
terminate a flight on competition boundary crossing re¬ 
mains and is still the last line of defence against a fly 
away. Many teams have lost aircraft during the testing 
and development of their flight termination systems and 
this requirement is still one of the most challenging of 
the UAV Challenge. 

Organization and Sponsorship 
The nature of flying robot competitions and challenges 
often require large teams of organizers with comple¬ 
mentary skills. For example, the UAV Challenge Out¬ 
back Rescue is organized and run by two committees 
whose members typically come from the organizations 
that provide support for the event. The first committee 
is the Steering Committee and runs all nontechnical as¬ 
pects of the UAV Challenge event such as equipment 
hire, securing sponsorship, liaising with local govern¬ 
ment and carrying out event communications and media 
activities. This committee is also responsible for the 
event finances. The second committee is the Technical 
Committee. This committee is responsible for develop¬ 
ing and releasing the rules for the UAV Challenge. The 
committee also liaises with air regulator and works with 
the UAV Challenge Steering Committee on the event 
organization. At the event, the Technical Committee 
is responsible for the airside part of the UAV Chal¬ 
lenge which includes selection of the technical team 
that runs all technical issues relating to the event, scru¬ 
tineering of the teams, the supervision and scheduling 
of the actual competition flights and the coordina¬ 
tion of aerodrome activities during the event times. 
The Technical Committee meets regularly throughout 
the year with initial meetings held to formulate the 


rules and incorporate lessons learned from the previous 
year. The Technical Committee reports to the Steering 
Committee. 

Each time the UAV Challenge is run, sponsors are 
sought to cover the costs of the prize money and the 
hire of facilities. The organizations that come together 
to run the UAV Challenge provide the time of their 
staff as an in-kind contribution. The total budget of 
a UAV Challenge is approximately $125 000 plus the 
staff time of the organizers. It is hence a considerable 
undertaking and requires a significant number of spon¬ 
sors to break even. The UAV Challenge has been well 
supported over the years, attracting the sponsorship of 
major aviation companies such as Boeing and Lock¬ 
heed Martin as well as unmanned aircraft companies 
such as Insitu. The safety regulator, CASA, has also 
sponsored the event. 

66.4.2 Case Study: The DARPA Challenges 

In 2001, the United States Congress challenged the 
Department of Defence with the following man¬ 
date [66.56]: 

It shall be a goal of the Armed Forces to achieve the 
fielding of unmanned, remotely controlled technol¬ 
ogy such that by 2015, one-third of the operational 
ground combat vehicles of the Armed Forces are un¬ 
manned. 

In response, a little over a year later, the Defense 
Advanced Research Projects Agency (DARPA), the pri¬ 
mary research and development organization for the 
United States Department of Defense, announced that 
there would be a race. 

The race, to be titled the DARPA grand challenge 
was to drive a robot car 300 miles from Los Angeles 
to Las Vegas in the fastest time to claim the $1-million 
dollar prize [66.57]. 

The aim of the DARPA grand challenge was to rein¬ 
vigorate and extend autonomous vehicle technologies 
from the 1980s and 1990s to future defense vehicles. 
Initiatives such as the Universitat der Bundeswehr Mu¬ 
nich (UBM)’s Vamp/Vamors vehicles, University of 
Parma and Carnegie Mellon University’s NavLab’s re¬ 
search program (shown in Fig. 66.9) had demonstrated 
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Fig. 66.10 A sample route network 
definition file (RNDF) provided by 
DARPA for the 2007 Challenge 
(overlaid on aerial imagery) 


impressive feats of road vehicle autonomy. Trials across 
thousands of highway kilometres had demonstrated 
lane keeping, obstacle avoidance and even automatic 
overtaking (passing) [66.58-60]. 

The problem was, close to a decade later, the 
capability of current technology full-sized, fully au¬ 
tonomous vehicles was unknown. The impressive trials 
of the late 1980s and 1990s concentrated on highway 
driving where the primary task was computer vision- 
based lane keeping. There were indoor and outdoor 
robots navigating through challenging obstacle field 
style environments [66.61,62]. There were also teleop- 
erated vehicles already in use by the defense forces. The 
question was, given computation power tracking with 
Moore’s law over the intervening years and many other 
advances (such as the use of LIDAR), what would robot 
car be capable of? 

DARPA aimed to [66.63]: 


• Demonstrate an autonomous vehicle able to travel 
over rugged terrain at militarily relevant speeds and 
distances. 

• Accelerate autonomous ground vehicle technology 
development in the areas of sensors, navigation, 
control algorithms, hardware systems, and systems 
integration. 

• Attract and energize a wide community of partici¬ 
pants not previously associated with Department of 
Defense programs or projects to bring fresh insights 
to the autonomous vehicle problem. 

Potential participants were drawn to the challenge. 
Many participants could also see the potential for re¬ 
lated technologies in the automotive industry and for 
road safety [66.64]. 

Teams would develop a vehicle that could follow 
a simple route across a course representing typical ter- 
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rain encountered by defense vehicles. The route map 
would be a set of paths specified by GPS way-points. 
The vehicle would be then given a list of destinations in 
the map and instructed to drive to each location. A typ¬ 
ical route map is shown in Fig. 66.10 overlaid on the 
satellite image. 

For this first competition, as for those that fol¬ 
lowed, the rules stated that the robot must be a fully 
autonomous and unmanned ground vehicle. Each vehi¬ 
cle was fitted with a wireless remote E-stop to bring 
the vehicle to a halt. On the base vehicle was added 
sensing, processing, and control. Some vehicles were 
built from the ground up. Most teams fit actuators to the 
manual controls of an existing vehicle, some built upon 
disability aid kits fitted to the vehicle. A couple teams 
were able to tie into existing by-wire capabilities of the 
vehicle, a notable example being Victor Tango’s 2007 
challenge entry Odin [66.65]. In addition to GPS, vehi¬ 
cles needed to have sensing for obstacle avoidance, in 
which LIDAR sensors dominated. As the vehicles were 
to be completely autonomous, all sensor processing, 
and control had to be done on-board so a certain amount 
of computer power was incorporated. Then the practi¬ 
calities would necessitate other devices such as power 
generators and additional air conditioning, if required. 
The general anatomy of a DARPA grand challenge ve¬ 
hicle is illustrated at Fig. 66.11. This vehicle, from the 
2007 DARPA Urban Challenge, was at the upper end 
of the spectrum for additional equipment with LIDAR, 
video cameras and automotive radar, as well as a gen¬ 
erator and supplementary air conditioning required for 
a blade server computer housed in the rear compart¬ 
ment [66.66]. 

2004 DARPA Grand Challenge 
DARPA hired Score international, an off-road race or¬ 
ganizer, to develop the course [66.67]. A 142-mile 
course was laid out through the Mojave Desert. The 
competition was structured as a time trial to complete 
the course in the shortest time under 10 h. One hundred 
and six teams applied. 

In March 2004, 25 teams turned up for the com¬ 
petition. In the days leading up to the race a qualifier, 
inspection and demonstration (QID) round was con¬ 
ducted. This event was primarily to verify that the 
vehicles were fit to compete and capable of operating 
safely. By race day, the 15th of March 2004, a field of 
15 vehicles had qualified and were ready to race. 

As stated earlier, a primary goal of the competi¬ 
tion was to gauge the state the art in the field. Steep 
elevation, blind turns, and sheer drops adjacent to the 
course were included to challenge the vehicles. While 
the degree of difficulty was low by human off-road 
driving standards, the course was designed to be in¬ 


teresting and make for a good competition [66.67], 
Expecting something akin to human driven competi¬ 
tion, DARPA even advised the media to set up at the 
finishing line where the action was expected to take 
place [66.68]. 

However, the course and the autonomous driving 
problem proved too difficult for the robot vehicles. No 
vehicle made it beyond the first seven miles of the 
course. After a promising start the vehicles soon be¬ 
came stuck or had to be disabled by a safety official 
after deviating from the course. One vehicle’s wheel 
caught on fire, another flipped around a bend another 
began circling around in the starting chute [66.69]. 

Inexperience was the chief cause of failure on the 
2004 contest. The new nature of the challenge and the 
stretch of mobile robotics to a full-sized off-road vehi¬ 
cle introduced too many complications for the robots 
systems. 

The challenge stood unbeaten, the prize money re¬ 
mained unclaimed. The real world complications and 
the size of the challenge were now apparent. Also ap¬ 
parent, however, was the curiosity sparked within the 
research community and the general public. 

Three months after the first competition, DARPA 
announced the 2005 DARPA Grand Challenge - a re¬ 
run of the event, this time for a $2-million dollar prize. 

2005 DARPA Grand Challenge 
With a slightly longer lead time, DARPA held a partici¬ 
pants’ conference and conducted site visits to each team 
to gauge progress. At the participants’ conference team 
representatives and DARPA officials discussed the draft 
race rules, clarifying rule interpretations and technical 
specifications. Out of 195 entrants, DARPA officials 
flew out to attend 118 site visits. 



Fig. 66.11 Anatomy of a DARPA grand challenge vehicle. 
2007 DARPA Urban Challenge entrant MIT’s Talos was 
on the upper end of the additional equipment spectrum 
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From the site visits 43 teams were selected to par¬ 
ticipate in the national qualifying event (NQE). During 
the qualifying event, which was twice as long as 2004, 
the teams had to complete a miniature version of the 
course which was created at the California Speedway 
(Fontana, California, Fig. 66.12). 

By race day, 23 robots were selected as starters in 
the race. The vehicles queued up to start at 5-min inter¬ 
vals. DARPA officials paused slow-moving vehicles to 
permit faster robots to pass. 

The difference between the 2004 and 2005 com¬ 
petitions was stark. Eighteen months on from the first 
competition, five robots drove 132 miles across the Mo¬ 
jave Desert to complete the course [66.63]. 

The race winner Stanley, from the Stanford Rac¬ 
ing team, claimed the prize - completing the course 
in 6h, 53 min and 58 s (Fig. 66.13) [66.70]. Second 
and third places were taken by robots from Carnegie 
Mellon University (CMU) teams: Red Team and Red 
Team Too [66.63]. Kat-5 from Team Gray came forth, 
a remarkable achievement from a team founded by 
Louisianan insurance executives. And last, but not 
least, the 16-ton TerraMax truck by the Ochkoch 
Truck Corporation which completed the course the fol¬ 
lowing day after spending the night paused on the 
course. 



Fig. 66.12 Miniature version of the course which was cre¬ 
ated at the California Speedway 



Fig. 66.13 Winner of the 2005 grand challenge Stanford 
Racing’s robot Stanley 


It was a trail blazing event which captivated the 
public with news images of unoccupied race vehicles 
kicking up dust as they drove across the Mojave desert 
sand. 

During the qualifiers and the race some common 
weaknesses were also identified. The vehicles required 
GPS for navigation as all the course information were 
given GPS coordinates, however the vehicles also ap¬ 
peared to be overly reliant on a strong, correct, GPS sig¬ 
nal. During the race vehicles drove under high voltage 
power lines which caused GPS loss. Vehicles deviated 
from the course or even crashed due to loss or multipath 
signals. Several robots were also able to make signifi¬ 
cant progress driving blind after sensor failures meant 
that local obstacle avoidance was no longer functioning 
correctly. 

In 2006, DARPA announced the next grand chal¬ 
lenge, the DARPA Urban Challenge. The contest this 
time required fully autonomous vehicles to drive on an 
urban street network. 

2007 DARPA Urban Challenge 
In the DARPA Urban Challenge, robot vehicles were 
to follow the Californian driver’s handbook. Unlike the 
previous challenge where vehicles were not intention¬ 
ally presented with moving obstacles (other robots were 
paused when they became too close to each other) the 
aim of the contest was to test the vehicles’ ability to 
drive in traffic. This time vehicles were expected to keep 
to lanes, queue in traffic, park, merge, pass, and even 
observe intersection precedence (i. e., yield to cars al¬ 
ready at an intersection) [66.71]. 

In a new element in the 2007 Challenge, DARPA 
introduced seed funding of up to $1-million dollars for 
per team for 11 teams based on a competitive applica¬ 
tion process to bootstrap the cost of development of the 
increasingly complex robot platform. The seed funding 
attracted new entrants to the competition in line with 
DARPA’s aim to attract a wide community of partici¬ 
pants [66.72]. 

On November 3, 2007, the urban challenge event 
(xUCE) was held in Victorville, California. This event 
set loose, simultaneously, for the first time, 11 full-size 
autonomous vehicles on a closed course (Fig. 66.14). 

The UCE was held on a closed course within the 
decommissioned George air-force base. The course was 
predominantly the street network of the residential zone 
of the former base with several graded dirt roads added 
for the competition. The contest was cast as a race 
against time to complete three missions. The missions 
were different from each team but were designed to re¬ 
quire each team to drive 60 miles to finish the race. 
Penalties for erroneous or dangerous behavior were also 
converted into time penalties. 
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Most roads were paved with a single lane in each 
direction as typical for a suburban street. A couple 
of roads had two lanes of traffic in each direction to 
simulate an arterial road or highway. One road, in the 
southeastern comer of the network, was a raised dirt 
road constructed especially for the event. 

This time all 11 qualifying robots were allowed to 
interact in the UCE course simultaneously, with ad¬ 
ditional traffic vehicles driven by professional drivers. 
For the third time, like anxious parents, the researchers 
manned the fences helpless as the robots, whose de¬ 
velopment had consumed the past 12 months of their 
lives, one by one, drove away. There were some tense 
moments and even a fender bender [66.66,73]. How¬ 
ever, powering through the mission objectives with 
a driving style described as a soccer mom with some 
place to be [66.74], CMU’s Tartan racing’s robot Boss 
(Fig. 66.15), crossed the finish line [66.73] to claim 
the prize. 2005 winning team Stanford Racing came 
second with Junior while team Victor Tango’s robot 
Odin secured third place. Six robots completed the 
race [66.72], 

Review 

DARPA endorses the use of competitions to cause 
paradigm shifting advances in science and technology. 
As stated DARPA’s 2007 Fiscal year report [66.72]: 



Fig. 66.14 DARPA starters 



Fig. 66.15 Tartan racing's robot Boss crossing the finish¬ 
ing line 


The program achieved its program goals and stim¬ 
ulated interest in the programs and projects of 
interest to the DoD Science and Technology (S&T) 
community. It was successful in attracting con¬ 
siderable joint investment by the participants and 
their sponsors, effectively leveraging Government 
investment in the program. The technical challenge 
was carefully defined and staged to bring coher¬ 
ence to the community and increase the chance 
for cross-fertilization among competing groups. The 
solicitation and qualification process was success¬ 
ful in attracting a large pool of strong teams with 
participation from the defense industry, automo¬ 
tive industry, academia, as well as a number of 
smaller organizations. This investment in expand¬ 
ing the community will continue to pay dividends as 
DoD benefits from a strengthened commercial sec¬ 
tor autonomous vehicle technical community. The 
program has been successful in attracting many 
young people to work on S&T problems in areas af¬ 
fecting national security, and benefits are expected 
to accrue for many years as this group enters the 
workforce. 

Although DARPA’s approach may raise the question of 
whether research initiated by the competitions should 
be supported beyond competition, DARPA’s view is 
that its role is to kick start ground breaking research not 
bring it to completion [66.1]. They encourage research 
teams and commercial organizations to team up to bring 
the technology to market for defense vehicles [66.72]. 

Velodyne LIDAR, a dominant sensing technology 
in the 2007 grand challenge, was developed with road 
safety potential in mind by Team DAD from the 2005 
DARPA grand challenge [66.75]. The invention is 
a good example of the technological pull DARPA was 
wanting to achieve with the competition. 

Another direct outcome of the competitions is the 
Google self-driving car project, which is based on 
the research and researchers from grand challenge 
teams [66.76]. Many vehicle manufacturers, such as 
Ford, General Motors, Volkswagon, and Toyota among 
others, have either entered the competition, partnered 
with teams or employ competition alumni to con¬ 
tinue to develop and apply related technologies to road 
safety [66.64, 70, 72], 

The DARPA Robotics Challenge 
Buoyed by the success of the grand challenges, DARPA 
now hopes to accelerate the development of humanoid 
rescue robotics. In April, 2012 DARPA proposed the 
DARPA robotics challenge. The Challenge is to develop 
an adaptable ground robot capable of completing tasks 
in a dangerous and degraded environments, using hand 
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Fig. 66.16 DARPA robotics challenge conceptual illustra¬ 
tion 

tools to driving vehicles for disaster relief operations (as 
illustrated in Fig. 66.16 [66.77], 

Similar to the first grand challenge, an ambitious 
goal has been set to inspire and stretch the state of the 
art. 

Though it is important to note that the new robotics 
challenge is a separate research program from the 
previous grand challenges, some best practices and 
lessons learned from the previous Challenges have been 
adopted. Entrants are divided into four tracks. Tracks A 
and D compete to develop hardware and software of 
a complete robotic system. Tracks B and C compete to 
develop software first to run on a common simulated 
robot. Tracks B and C will then compete in a vir¬ 
tual competition. In 2014, successful Track B and C 
teams will be able to develop and deploy the software 
to a common robot platform provided by DARPA. The 
entrants can also apply for seed funding (similar to the 
), which differentiates teams between Tracks A and B, 
who have DARPA seed funding, and C and D who are 
self-funded [66.77]. 

The idea of using a common platform across teams 
and a simulator-based challenge addresses the obser¬ 
vation that successful teams in the 2007 challenge had 
arrived upon similar sensors and sensor configurations. 
So a future challenge could remove the significant 
hardware development burden by allowing teams to 
compete on a common platform. The motivation is to 
once again broaden field of participants [66.77]. 

The new robotics challenge also introduces access 
to a remote human supervisor. Previous challenges were 
strictly fully autonomous. A criticism had been that due 
to the extreme lack of contextual information available 


to most robotic systems, simple ambiguities that arise 
can impede a robot unnecessarily if a supervisor can 
correct the problem. The lack of supervisory control 
meant that in previous challenges vehicles would ap¬ 
pear overly cautious. 

For example, many vehicles in the 2004 Challenge 
were incapacitated by trivial problems such as the robot 
misjudging which side of a road boundary they were 
actually on. Full autonomy should be the ultimate aim; 
however, allowing limited supervisory contact would 
have permitted the 2004 contest to continue allowing 
more trial time for teams. 

Conclusion 

DARPA never aimed to create the ultimate autonomous 
vehicle. The true aim of the competition was to advance 
the science and technology research and education 
underpinning the original Congressional mandate on 
autonomous vehicles. 

Of course, the exercise has come with consider¬ 
able cost. From the 2007 Fiscal report [66.72] the 
true dollar cost to DARPA of the competition came 
in at around 25 million dollars. Three and a half mil¬ 
lion dollars were paid directly to the winning teams 
as prize money. A further $ 11 million was paid in 
Track A grants to teams, with the remaining $ 10.5 
million was used to host the participants’ confer¬ 
ence, site visits, qualifying round, and stage the final 
event. 

Notwithstanding these costs and the necessarily 
limited nature of these competitions - which create an 
artificial environment to focus a broad group of experi¬ 
menters on a common goal, for a given period of time - 
they have brought clear benefits for DARPA, and the 
field of robotics. 

Regardless of the percentage of autonomous vehi¬ 
cles fielded by the Department of Defense in 2015, 
the experience shared by participants from: research 
institutions, car manufacturers, defense contractors, 
technology companies, schools, a loud speaker com¬ 
pany [66.75] and even insurance executives [66.78] to 
attempt the challenge of creating a full size, fully au¬ 
tonomous, self-driving car has gathered the field for an 
even greater race. The race between companies such as 
Google, Mercedes Benz, Toyota and their technology 
suppliers to bring these vehicles to market not only for 
defense vehicles but for all road vehicles. 

An outcome beyond the dreams of the forward 
thinkers in the 2001 Congress. 
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66.5 Conclusion and Further Reading 


In this chapter, we have discussed several types of robot 
competitions highlighting their impact on research and 
education. Here we summarize the manifold impacts of 
competitions as well as the major issues that need to be 
addressed to design and run them. 

Robotics competitions with ambitious aims can pro¬ 
vide significant breakthroughs in the development of 
science and technology. 

To this end, competition models based on one time 
challenges have the advantage of being able to head¬ 
line a fresh new goal. The drawback for the organizer 
and participants is the significant effort to build sys¬ 
tems and infrastructure that may be too specialized to 
be used after the competition for subsequent challenges. 
On the other hand, challenges that are developed across 
the years can benefit from a continuous development, 
as long as they suitably build on yearly achievements, 
without loss of focus on the long-term roadmaps. The 
two approaches as outlined in the case studies demon¬ 
strate the merit of both as well as the unquestionable 
impact on science and technology. 

Robots are complex systems composed of many 
mechanical, electrical, software, and algorithmic com¬ 
ponents and as such are hard to test and compare. In 
isolation a test can be derived to evaluate some com¬ 
ponents or subcomponents. For example, one can test 
the degrees of freedom of a mechanical design, or the 
bandwidth and latency of a data bus. Image processing 
and mapping algorithm development have made great 
strides using shared datasets to compare results. The 
performance of a robot system as a whole would of¬ 
ten fall back to a qualitative assessment presented as 
a video accompanied by the encouraging words of the 
researcher. Competitions provide the motivation and the 
forum for robotics researchers to undertake the signif¬ 
icant effort (in time, compatibility, and transportation) 
to develop and validate innovative solutions on complex 
systems under controlled conditions. 

Robot competitions have a positive impact on 
robotics research groups in that the team work require¬ 
ment within and between groups means that a better 
standard of core software is achieved across the group. 
Good tools in terms of software libraries and best prac¬ 
tice by the more experienced software engineers in 
a group are often adopted across the team enabling al¬ 
gorithms to make it onto robot hardware sooner. To this 
end, it is critical that competitions support the sharing 
of solutions and tools, by developing community in¬ 
frastructure and resources. This kind of gently enforced 
openness keeps competitions competitive, fosters in¬ 
teractivity between teams and encourages diversity of 
participants by minimising the barriers to entry. 


Robot competitions can play a tremendous role in 
education. They are attractive to students and teach¬ 
ers, while providing an interdisciplinary framework to 
support the development of several types of technical 
skills, including teamwork. Maintaining a good balance 
between competing and learning becomes a major con¬ 
cern when the focus of the competition is education. In 
international competitions with younger students, dif¬ 
ferences in culture and language need to be carefully 
taken into consideration to maintain fairness and inclu¬ 
siveness. 

Last, but not least, robot competitions have a sig¬ 
nificant impact on public opinion and the public under¬ 
standing of the science and technology. Simple distinc¬ 
tions such as autonomous versus teleoperated robotics 
are often mistaken or overlooked by the news media 
and the public. Human-inspired competitions have at¬ 
tracted many spectators and the attention of the media, 
and they certainly contribute to make scientific progress 
more accessible, through the language of sports. Grand 
challenges, such as the DARPA Challenges have been 
extensively covered by the media, thus, contributing, al¬ 
though from a different perspective to disseminate the 
awareness on the progress of robotics in the society. 

The aforementioned outcomes do not come with¬ 
out a significant effort. A successful competition must 
clearly define a target participation and suitably sup¬ 
port it, by making the entry level accessible to a large 
number of participants. The technical challenge needs 
to be appropriately compelling and attractive; more¬ 
over, the competition setting must ensure that winning 
is achieved through innovation and scientific progress. 
This is achieved by designing the requirements on 
the robots and on the competition scenario as well as 
through a well-designed performance assessment. 

The organization of competitions also comes with 
formidable challenges: logistics for transporting robots 
and teams, venue and infrastructure for the competition 
event, safety of participants and spectators. The fi¬ 
nancing and administration, often underestimated, also 
requires substantial preplanning, know-how, and re¬ 
sources. 

This notwithstanding, we expect the use of compe¬ 
titions to motivate, educate, and foster research inno¬ 
vation only to increase in coming years as competition 
has proven to be a particularly useful tool for robotics 
research. 

66.5.1 Further Reading 

Research results of robot competitions are often ac¬ 
cumulated into a journal special issue. The reader is 
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encouraged to explore the articles as they are often ap¬ 
proachable and can include interesting details of events 
during the contests. For the DARPA grand challenge the 
reader may refer to the Journal of Field Robotics Vol¬ 
ume 23 (2006) Issues 8 and 9 or the Springer Tracts 
in Advanced Robotics Volume 36. For the DARPA the 
reader may refer to the Journal of Field Robotics Vol¬ 


ume 25 (2008) Issues 8 and 9 or the Springer Tracts in 
Advanced Robotics Volume 56. In future, similar spe¬ 
cial issues are likely to be available on the DARPA 
robotics challenge. 

US Innovation Programs [66.79] is interesting re¬ 
view on the broader discussion of competitions in 
funding science and technology research. 
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Brief history of RoboCup robot soccer 

available from http://handbookofrobotics.org/view-chapter/66/videodetails/385 
|<s>)Kii>ll 2 liiiEa Multirobot teamwork in the CMDragons RoboCup SSL team 

available from http://handbookofrobotics.org/view-chapter/66/videodetails/387 
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Robots and Humans covers some of the most re¬ 
cent advances concerned with human-robot interaction, 
ranging from designing biologically inspired robots, to 
programming and safety issues for human-robot inter¬ 
action, and ethical issue brought forth by robotics. Our 
field’s future vision for technology is the leap from per¬ 
sonal computers to personal robots, in a world where 
robots exist pervasively and work side by side with 
humans. Over the past 50 years we have made great 
strides in robotics, as the previous parts of this Hand¬ 
book show. However, there are still new capabilities that 
need to be developed and existing capabilities that need 
to be improved to create a world in which robots and 
humans work together. Robot bodies should be easily 
integrated into our living environments. Robots should 
be safe to be around. Robots should take commands 
from human users easily. Robots should be functionally 
capable. Robots should engage humans to help mitigate 
error states and task uncertainties. Meeting these chal¬ 
lenges will bring robots closer to our vision of pervasive 
robotics. 

The topics in Part G are essential for creating 
robots that operate in human-centered environments. 
The chapters cover organically human-centered and 
life-like robots and include hardware design, control 
(of locomotion, manipulation, and interaction), per¬ 
ception, user interfaces, and social and ethical im¬ 
plications for robotics. As such, Part G builds on 
all of the Handbook’s previous parts. The connec¬ 
tion with the chapters in Robot Foundations (Part A) 
and Robot Structures (Part B), particularly Mecha¬ 
nisms and Actuation (Chap. 4), Sensing and Estimation 
(Chap. 5), Motion Planning (Chap. 7), Robotics Sys¬ 
tems Architectures and Programming (Chap. 12), AI 
Reasoning Methods for Robotics (Chap. 14), Robot 
Hands (Chap. 19), Limbed Systems (Chap. 17), Mod¬ 
ular Robots (Chap. 22) and Wheeled Robots (Chap. 24) 
are essential. There is also an important dependency 
on Sensing and Perception (Part C), Manipulation and 
Interfaces (Part D), and Moving in the Environment 
(Part E) as these chapters cover some of the basic al¬ 
gorithms and technologies required by robots operating 
in human-centered environments. 

Today’s approach to computation has progressed 
naturally from desktop computing, to mobile com¬ 
puting, to pervasive computing, ultimately leading to 
computation for interaction with the physical world. 
In other words, Part G of the Handbook on Robotics 
presents a snapshot of the field’s advances for creating 
machines in our own image that are smart and obedient. 
With this overview of Part G, we now provide a brief 
synopsis of each chapter. 


Chapter 67, Humanoids, provides a concise 
overview of the state of the art in creating human-like 
machines, starting with a historical and philosophical 
discussion on the human example and understanding 
intelligence. These machines are capable of bipedal 
locomotion and manipulation. In addition, humanoid 
robots can take advantage of their entire body (e.g., lift¬ 
ing objects while moving the body to compensate for 
the load). Control issues specific to humanoids include 
static and dynamic stability during a mobile manipula¬ 
tion action. Humanoid robots interact with humans via 
perception systems and should be capable of interpret¬ 
ing behavior and mood as well as via speech systems 
for interpreting human language. 

Chapter 68, Human Motion Reconstruction, de¬ 
scribes the most recent algorithms that rely on data from 
motion capture systems to synthesize human-like mo¬ 
tion models for machines. The chapter introduces two 
models for information capture: optical motion capture 
and mechanical motion capture. The skeleton model is 
computed on the captured data using kinematics and 
dynamics models. Musculoskeletal models can also be 
computed for motion analysis involving muscles. The 
chapter discussed model identification techniques, seg¬ 
mentation of human motion, classification of human 
motion, as well as temporal sequencing and linguistic 
classification of human motion. The chapter concludes 
with a survey of techniques for motion reconstruction 
for robots. 

Chapter 69, Physical Human-Robot Interaction, 
discusses what happens when humans and robots share 
the same workspace and come into contact with one 
another. Safety can be jeopardized by mechanical fail¬ 
ures, operator errors, or software problems. Design¬ 
ing and controlling robots with safety guarantees is 
a very important precondition for physical human- 
robot interaction (pHRI). The chapter discusses safety 
standards for pHRI and surveys several approaches to 
safety. 

Chapter 70, Human Robot Augmentation, intro¬ 
duces wearable robotics systems that enhance the capa¬ 
bilities of the humans wearing them, while allowing the 
humans to control them to enhance their performance 
in a task-specific way. There are three types of devices 
that deliver human augmentation: 

i) Prostheses, for the functional replacement of human 

limbs 

ii) Powered orthoses, which function is to actively op¬ 
erate in parallel with unhealthy human joints 

iii) Robotic exoskeletons that operate in parallel with 

healthy human limbs. 
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The chapter describes prostheses, powered orthoses 
and exoskeletons for upper limb, lower limbs and whole 
body structures. State-of-the-art devices together with 
their functionalities and main robotics components are 
presented for each class of wearable system. Critical de¬ 
sign issues and open research aspects are reported. 

Chapter 71, Cognitive Human Robot Interaction, 
considers human(s), robot(s), and their joint actions as 
a cognitive system. This chapter surveys the state of the 
art in modeling, algorithms, and design guidelines to 
enable the design of systems where humans and robots 
can interact. The chapter describes the representations 
and actions that allow robots to participate in joint ac¬ 
tivities with people and the models of joint activity 
for human-robot interaction. The chapter also describes 
recent research on developing a deeper understand¬ 
ing of human expectations and cognitive responses to 
robot actions. This chapter draws on research questions 
and advances from a wide range of fields including 
computer science, cognitive science, linguistics, and 
robotics. 

Chapter 72, Social Robotics, discusses the creation 
of robots designed to interact with people in a social 
and emotional way for applications in education, enter¬ 
tainment, health care, etc. These robots’ embodiment is 
life-like and they are capable of multi-modal commu¬ 
nication. The chapter uses a case study of the Kismet 
robot to illustrate the idea of a machine capable of 
projecting emotion. The chapter surveys some basic ap¬ 
proaches to giving robots sociocognitive skills such as 
shared attention, emotional empathy, and mental per¬ 
spective tasking. 

Chapter 73, Socially Assistive Robotics, covers the 
design of human-robot systems that use verbal and 
non-verbal expression and communication to interact 
and assist users through social rather than physical 
interaction. Socially assistive robotics (SAR) focuses 
on the challenges of providing motivation, coaching, 
training, and rehabilitation through non-physical inter¬ 
action; such systems have been validated in hands-off 
stroke rehabilitation, social skill training of children 
with autism, and eldercare, among others. This chapter 
reviews the critical societal issues that have motivated 
research into socially assistive robotics, describes the 
reason why physical robots rather than virtual agents 
are essential to this, highlights the major research issues 
within this area, describes the primary application do¬ 
mains and populations where SAR research has shown 
an impact, and closes with some of the ethical and 
safety issues. 

Chapter 74, Learning from Humans, covers im¬ 
portant topics about how we approach teaching robots 


what to do by providing either good or bad exam¬ 
ples. This is an intuitive approach to robot control and 
learning of tasks with the potential of opening robot 
applications to non-expert users. The chapter starts by 
discussing what it means for a robot to learn a new 
skill and surveys several learning approaches for this. 
Biologically oriented learning approaches such as con¬ 
ceptual models of imitation learning and neural models 
of imitation learning are also introduced. The chapter 
concludes with several open issues in robot program¬ 
ming by demonstration. 

Chapter 75, Biologically Inspired Robots, starts 
with a discussion of the difference between bio-in¬ 
spired and biomimetic robots. Bio-inspired robots seek 
to reproduce a natural phenomenon but not necessar¬ 
ily the underlying means. Biomimetic robots reproduce 
both the natural phenomenon and the means. The chap¬ 
ter starts with a survey of bio-inspired morphologies. 
A survey of bio-inspired sensors from vision to au¬ 
dio, touch, smell, and taste is then presented. Bio¬ 
inspired actuators include locomotion topics such as 
crawling, legged locomotion, climbing swimming, fly¬ 
ing, and manipulation. The chapter also examines bio¬ 
inspiration in the context of control and planning. This 
includes behavior-based architectures, learning robots, 
evolving robots, and developing robots. Finally, the 
chapter examines how to make machines self-sustain¬ 
ing from an energy point of view. 

Chapter 76, Evolutionary Robotics, discusses 
a method for creating robots that is inspired by the 
Darwinian principle of selective reproduction of the 
fittest captured by evolutionary algorithms. The chap¬ 
ter introduces the evolutionary computation method and 
presents several case studies. Because the evolutionary 
method requires intensive computation, it is often run 
in simulation and the final result is transferred to the 
robot platform. However, in the recent past there were 
several successes with running evolutionary algorithms 
in real time on physical platforms to achieve complex 
behaviors such as learning to walk. 

Chapter 77, Neurobotics: From Vision to Action, 
looks at how neuroethology, the study of brain mech¬ 
anisms for animal behavior, inspires the creation of 
a central nervous system for a machine. The chap¬ 
ter presents several case studies that sample this bio¬ 
inspired design of robot controllers structurally and 
functionally: the optic flow in bees and robots, visually 
guided behaviors in frogs and robots, and navigation in 
rats and robots. The chapter then examines the role of 
the cerebellum in motion control and the role of mir¬ 
ror systems. The chapter concludes by observing the 
brain has evolved to serve as action-oriented percep- 
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tion, which is a useful guiding principle for building 
robot controllers. 

Chapter 78, Perceptual Robotics, discusses prin¬ 
ciples derived from high-level cognitive processing in 
vision in the human brain that have led to results in 
robotics and computer vision. The chapter is focused 
on the technical realization of perceptual functions in 
robots by examining relationships between biological 
perception and robotics systems. Object recognition is 
used as an exemplary task and issues such as represen¬ 
tation, structural description models, neural representa¬ 
tions, as well as recognition and learning algorithms are 
discussed. Next, example-based movement representa¬ 
tions are examined to establish the connection between 
the functionality of the visual cortex and a class of com¬ 
puter vision algorithms. 

Chapter 79, Robotics for Education, provides an 
overview of the key ingredients that make success¬ 
ful education robots possible. It begins with a survey 
of robot tournaments, which are the most prevalent 
mechanism for educating young students in robotics. 
Robot tournaments have already impacted tens of thou¬ 
sands of students across diverse geographic and age 
group boundaries. Next, the chapter technology stan¬ 


dards required for effective use of robots in education. 
Educational robot devices consist of both hardware 
(pre-assembled or as kits or components) and software 
(both as source code and programming environments). 
The chapter surveys the physical robot platforms that 
have achieved notable success, and the software that 
enabled the interaction with young students (that is, 
the low-level controllers that interface those platforms 
to high-level computation and the top-level program¬ 
ming environments.) Finally, the chapter describes how 
conventional analytical tools may be used to evaluate 
unconventional educational programs that use robots. 

Chapter 80, Roboethics: Social and Ethical Impli¬ 
cations, discusses the social and ethical implications in 
a society where robots coexist with humans. Starting 
with a philosophical introspection into the dangers of 
unlimited use of technology, the chapter argues for the 
need for a new ethics of robots. Many other factors, 
including the cultural differences in societies ready to 
accept robots, define this need. The chapter surveys the 
code of ethics, privacy, accuracy, intellectual property, 
and access, which could be adopted by robotics. A tax¬ 
onomy of robot types, along with socio-ethical issues 
for each robot type is then presented. 
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67. Humanoids 



Paul Fitzpatrick, Kensuke Harada, Charles C. Kemp, Yoshio Matsumoto, Kazuhito Yokoi, Eiichi Yoshida 


Humanoid robots selectively immitate aspects 
of human form and behavior. Humanoids come 
in a variety of shapes and sizes, from complete 
human-size legged robots to isolated robotic 
heads with human-like sensing and expression. 
This chapter highlights significant humanoid plat¬ 
forms and achievements, and discusses some of 
the underlying goals behind this area of robotics. 
Humanoids tend to require the integration of many 
of the methods covered in detail within other 
chapters of this handbook, so this chapter focuses 
on distinctive aspects of humanoid robotics with 
liberal cross-referencing. 

This chapter examines what motivates re¬ 
searchers to pursue humanoid robotics, and 
provides a taste of the evolution of this field 
over time. It summarizes work on legged hu¬ 
manoid locomotion, whole-body activities, and 
approaches to human-robot communication. It 
concludes with a brief discussion of factors that 
may influence the future of humanoid robots. 
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67.1 Why Humanoids? 

Throughout history, the human body and mind have 
inspired artists, engineers, and scientists. The field of 
humanoid robotics focuses on the creation of robots that 
are directly inspired by human capabilities (Chap. 75). 
These robots usually share similar kinematics to hu¬ 


mans, as well as similar sensing and behavior. The mo¬ 
tivations that have driven the development of humanoid 
robots vary widely. For example, people have developed 
humanoid robots to serve as general-purpose mechan¬ 
ical workers, entertainers, and test-beds for theories 
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from neuroscience and experimental psychology [67. 1- 

3]. 

Notably, while this chapter focuses on robots that 
have been explicitly designated as humanoid robots by 
their creators, the lines between these robots and others 
can be blurry. Many robots share characteristics with 
humans, or have been inspired by humans. 

67.1.1 The Human Example 

On a daily basis, humans perform important tasks that 
are well beyond the capabilities of current robots. More¬ 
over, humans are generalists with the ability to perform 
a wide variety of distinct tasks. Roboticists would like 
to create robots with comparable versatility and skill. 
Considering the physical and computational mecha¬ 
nisms that enable a person to perform a task is a com¬ 
mon approach to automating it. Exactly what to borrow 
from the human example is controversial. The literal¬ 
minded approach of creating humanoid robots may not 
be the best way to achieve some human-like capabilities 
(Chap. 65). For example, dishwashing machines bear 
little similarity to the manual dishwashing they replace. 

67.1.2 The Pleasing Mirror 

Humans are humanity’s favorite subject. A quick look 
at popular magazines, videos, and books should be 
enough to convince any alien observer that humanity 
is obsessed with itself. The nature of this obsession is 
not fully understood, but aspects of it have influenced 
the field of humanoid robotics. 

Humans are social animals that generally like to ob¬ 
serve and interact with one another [67.4]. Moreover, 
people are highly attuned to human characteristics, such 
as the sound of human voices and the appearance of 
human faces and body motion [67.5-7]. Infants show 
preferences for these types of stimuli at a young age, 
and adults appear to use specialized mental resources 
when interpreting these stimuli. By mimicking human 
characteristics, humanoid robots can engage these same 
preferences and mental resources. 

Humanity’s interest in itself has been reflected in 
media as diverse as cave paintings, sculpture, mechani¬ 
cal toys, photographs, and computer animation. Artists 
have consistently attempted to portray people with the 
latest tools at their disposal. Robotics serves as a pow¬ 
erful new medium that enables the creation of artifacts 
that operate within the real world and exhibit both hu¬ 
man form and behavior [67.8]. 

Popular works of fiction have frequently in¬ 
cluded influential portrayals of humanoid robots and 
human-made humanoid creatures. For example, Karel 
Capek’s science fiction play Rossum’s Universal 


Robots (R.U.R.) from 1920 centers around the story of 
artificial people created in a factory [67.9]. Many other 
works have included explicit representations of hu¬ 
manoid robots, such as the robot Maria in Fritz Lang’s 
1927 film Metropolis [67.10], and the thoughtful por¬ 
trayal of humanoid robotics by Isaac Asimov in works 
such as The Caves of Steel from 1954 [67.11], The long 
history of humanoid robots in science fiction has influ¬ 
enced generations of researchers, as well as the general 
public, and serves as further evidence that people are 
drawn to the idea of humanoid robots. 

67.1.3 Understanding Intelligence 

Many researchers in the humanoid robotics commu¬ 
nity see humanoid robots as a tool with which to better 
understand humans [67.3, 12]. Humanoid robots offer 
an avenue to test understanding through construction 
(synthesis), and thereby complement the analysis pro¬ 
vided by researchers in disciplines such as cognitive 
science. 

Researchers have sought to better immitate human 
intelligence using humanoid robotics [67.13]. Develop¬ 
mental psychologists, linguists, and others have found 
strong links between the human body and human cog¬ 
nition [67.14], By being embodied in a manner similar 
to humans, and situated within human environments, 
humanoid robots may be able to exploit similar mech¬ 
anisms for artificial intelligence (AI). Researchers are 
also attempting to find methods that will enable robots 
to develop autonomously in a manner akin to human in¬ 
fants [67.15]. Some of these researchers use humanoid 
robots that can physically explore the world in a manner 
similar to humans [67.16]. 

67.1.4 Human Environments 

People inhabit environments that accommodate human 
form and human behavior [67.17, 18]. Many important 
everyday objects fit in a person’s hand and are light 
enough to be transported conveniently by a person. Hu¬ 
man tools match human dexterity. Doors tend to be 
a convenient size for people to walk through. Tables 
and desks are at a height that is well matched to the 
human body and senses. Humanoid robots can poten¬ 
tially take advantage of these same accommodations, 
thereby simplifying tasks and avoiding the need to al¬ 
ter the environment for the robot [67.19]. For example, 
humanoid robots and people could potentially collabo¬ 
rate with one another in the same space using the same 
tools [67.20]. Humanoid robots can also interface with 
machinery that does not include drive-by-wire controls, 
as shown by the teleoperated robot in the cockpit of 
a backhoe in Fig. 67.1 [67.21], 
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Robots with legs and human-like behavior could po¬ 
tentially traverse the same environments that humans 
traverse, such as rugged outdoor environments and the 
industrial plant shown in Fig. 67.2, which has stairs and 
handrails designed for human use [67.22]. In addition to 
mobility advantages, legs have the potential to help in 
other ways. For example, legs could enable a humanoid 
robot to change its posture in order to lean into some¬ 
thing, pull with the weight of its body, or crawl under 
an obstacle [67.23,24]. 

The Fukushima Daiichi nuclear disaster, which oc¬ 
curred in March 2011 in Japan, is a compelling example 
scenario for robots. The disaster resulted in human 
environments that were unsafe for humans. Robots 
capable of performing diverse tasks in these environ- 



Fig. 67.1 The humanoid robot HRP-1S (HRP: humanoid 
robot project) driving a backhoe (courtesy Kawasaki 
Heavy Industries, Tokyu Construction and National In¬ 
stitute of Advanced Industrial Science and Technology 
(Japan) (AIST)). The robot can be teleoperated by a human 
operator to control the backhoe remotely. The same robot 
could potentially interface with many different unmodified 
machines 



Fig. 67.2 HRP-1 operating in a mockup of an industrial 
plant (courtesy Mitsubishi Heavy Industries) 


ments via remote control would have been valuable. 
Future humanoid robots may be able to access simi¬ 
lar environments using narrow passageways, ladders, 
and other environmental features designed for peo¬ 
ple (Fig. 67.3). Likewise, they may be able to re¬ 
motely perform tasks involving control panels, valves, 
and tools designed for people. This type of scenario 



Fig. 67.3 An image of supposed disaster-response sce¬ 
nario DARPA Robotics Challenge 



Fig. 67.4 The humanoid robot HRP-2 dancing with a human (af¬ 
ter [67.25]). The human is a master of a traditional Japanese dance 
whose dancing was recorded by a motion-capture system, and trans¬ 
formed for use by the robot 



Fig. 67.5 Actroid (courtesy Kokoro), an android designed 
for entertainment, telepresence, and media roles 
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Fig. 67.6 Atlas robot provided as a platform for the 
DARPA Robotics Challenge (courtesy Boston Dynamics) 


has inspired the DARPA (Defense Advanced Research 
Projects Agency) Robotics Challenge in which robots 
will compete by performing related tasks. Notably, 
DARPA plans for some teams to compete using Atlas 
humanoid robots from Boston Dynamics Fig. 67.6. 

67.1.5 Human Interaction 

People are accustomed to working with other people. 
Many types of communication rely on human form and 
behavior. Some types of natural gestures and expres¬ 
sion involve subtle movements in the hands and face 
(Chap. 72). People can interpret eye gaze and facial 
expressions without training. Humanoid robots can po¬ 
tentially simplify and enhance human-robot interaction 
by taking advantage of the communication channels 
that already exist between people. 

Similarly, people already have the ability to perform 
many desirable tasks. This task knowledge may be more 
readily transferred to humanoid robots than to a robot 



Fig. 67.7 Petman is a humanoid robot that Boston Dynam¬ 
ics developed to test chemical protection clothing for the 
US military (after [67.26]) 


with a drastically different body. This is especially true 
of cultural actions centered around the human form 
(Fig. 67.4). 

67.1.6 Entertainment, Culture, 
and Surrogates 


Humanoid robots are inherently appropriate for some 
applications. For example, robots that resemble hu¬ 
mans could play roles in entertainment, such as 
theater, theme parks, and companionship for adults 
(Fig. 67.5). Realism in form and function could 
make humanoid robots preferable to wax figures and 
animatronics. 

A humanoid robot could serve as an avatar for 
telepresence, model clothing, test ergonomics, or serve 
other surrogate roles that fundamentally depend on the 
robot’s similarity to a human. For example, Boston Dy¬ 
namics developed the humanoid robot Petman to test 
clothing that is intended to protect military person¬ 
nel from chemical agents (Fig. 67.7). Robotic pros- 
theses and cosmeses also have a close relationship 
to humanoid robotics, since they seek to directly re¬ 
place parts of the human body in function and form 
(Chap. 64). 


67.2 History 

There is a long history of mechanical systems with 
human form that perform human-like movements. For 
example, Al-Jazari designed a humanoid automaton 
in the 13th century [67.27], Leonardo da Vinci de¬ 
signed a humanoid automaton in the late 15th cen¬ 
tury [67.28], and in Japan there is a tradition of cre¬ 
ating mechanical dolls called Karakuri ningyo that 
dates back to at least the 18th century [67.29]. In 
the 20th century, animatronics became an attraction 


at theme parks. For example, in 1967 Disneyland 
opened its Pirate’s of the Caribbean ride [67.30], which 
featured animatronic pirates that play back human¬ 
like movements synchronized with audio. Although 
programmable, these humanoid animatronic systems 
moved in a fixed open-loop fashion without sensing 
their environments. 

In the second half of the 20th century, advances 
in digital computing enabled researchers to incorporate 
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Fig. 67.8 (a) WABOT-1 (1973) and (b) WABOT-2 (1984; 
courtesy Humanoid Robotics Institute, Waseda University) 


significant computation into their robots for sensing, 
intelligence, control, and actuation. Many roboticists 
developed isolated systems for sensing, locomotion, 
and manipulation that were inspired by human capa¬ 
bilities. However, the first humanoid robot to integrate 
all of these functions and capture widespread attention 
was Waseda robot (WABOT-1), developed by Ichiro 
Kato et al. at Waseda University in Japan in 1973 
(Fig. 67.8). 

The WABOT robots integrated functions that have 
been under constant elaboration since: visual object 
recognition, speech generation, speech recognition, bi¬ 
manual object manipulation, and bipedal walking. For 
instance, WABOT-2’s ability to play a piano, publicized 
at the Tsukuba Science Expo in 1985, stimulated signif¬ 
icant public interest. 

In 1986, Honda began a confidential project to 
create a humanoid biped. Honda grew interested in hu¬ 
manoids, perhaps seeing in them devices of complexity 
comparable to cars with the potential to become high- 
volume consumer products one day. In 1996, Honda 
unveiled the Honda Humanoid P2, the result of this con¬ 
fidential project. P2 was the first full-scale humanoid 
capable of stable bipedal walking with onboard power 
and processing. Successive designs reduced its weight 
and improved its performance (Fig. 67.9). Compared 
to humanoids built by academic laboratories and small 
manufacturers, the Honda humanoids were a leap for¬ 
ward in sturdiness, using specially cast lightweight 
high-rigidity mechanical links, and harmonic drives 
with high torque capacity. 

In parallel with these developments, the decade- 
long Cog project began in 1993 at the MIT Artificial 
Intelligence laboratory in the USA with the intention of 



Fig. 67.9 (a) Honda P2 (180cm tall, 210kg), (b) P3 (160cm, 
130 kg), and (c) advanced step in innovative mobility (glossnoidx- 
ASIMOadvanced step in innovative mobility) (120 cm, 43 kg) (af¬ 
ter [67.31]; courtesy Honda) 



Fig. 67.10 The humanoid robot Cog used neural oscillators 
in conjunction with compliant torque-controlled arms to 
perform a variety of everyday tasks with human tools, such 
as crank turning, hammering, sawing, and playing a snare 
drum (after [67.32]; courtesy Sam Ogden) 

creating a humanoid robot that would, learn to think 
by building on its bodily experiences to accomplish 
progressively more abstract tasks [67.13]. This project 
gave rise to an upper-body humanoid robot whose 
design was heavily inspired by the biological and cog¬ 
nitive sciences (Fig. 67.10). Since the inception of the 
Cog project, researchers across the world have initiated 
many humanoid robotics projects and formed commu¬ 
nities devoted to developmental robotics, autonomous 
mental development (AMD [67.33]), and epigenetic 
robotics [67.34]. 

As of the early 21st century, a large number of com¬ 
panies and academic researchers have become involved 
with humanoid robots and created new humanoid robots 
with distinctive characteristics. 
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67.3 What to Immitate? 

Humanoid robots come in a variety of shapes and sizes 
that immitate different aspects of human form and be¬ 
havior (Fig. 67.11). As discussed, the motivations that 
have driven the development of humanoid robots vary 
widely. These diverse motivations have led to a variety 
of humanoid robots that selectively emphasize some hu¬ 
man characteristics, while deviating from others. 

67.3.1 Body Parts 

One of the most noticeable axes of variation in hu¬ 
manoid robots is the presence or absence of body parts. 
Some humanoid robots have focused solely on the head 
and face, others have a head with two arms mounted 
to a stationary torso, or a torso with wheels (for exam¬ 
ple, Fig. 67.12). Some humanoid robots even combine 
a highly-articulate face with arms, legs, and a torso. 

67.3.2 Mechanics 

Humanoid robots immitate various mechanical aspects 
of the human body, such as its kinematics, dynamics, 
geometry, material properties, and actuation. As such, 
humanoid robotics is closely related to the field of hu¬ 
man biomechanics. 

Humanoid robots often consist of rigid links with 
kinematics that approximate the kinematics of the hu¬ 
man musculoskeletal system. Even a rigid-link model 
of human kinematics can have a very high number of 
degrees of freedom (DOF). A humanoid robot typi¬ 
cally imitates degrees of freedom that are pertinent to 
its intended use. For example, humanoid robots rarely 
attempt to immitate the human shoulder’s ability to 
translate or the flexibility of the human spine [67.35, 
36] 



Fig. 67.11 Kismet is an example of a humanoid head for 
social interaction 


The human hand serves to further illustrate these 
issues. Modern humanoid robots frequently have two 
arms, each with seven degrees of freedom, but their 
hands vary considerably (Chap. 19). The human hand 
is highly complex with over 20 DOFs (i. e., approxi¬ 
mately four DOFs per finger and a five-DOF thumb) 
in a very compact space with a compliant exterior, 
dense tactile sensing, and low distal mass. Researchers 
have approximated the human hand with varying levels 
of accuracy, including the anatomically correct testbed 
(ACT) hand, the 20-DOF Shadow Hand, the 12-DOF 
DLR-Hand-II (DLR: Deutsches Zentrum fiir Luft- und 
Raumfahrt), the 11-DOF Robonaut hand, and the 2- 
DOF Cog hand [67.37^-1]. The ACT hand represents 
the high-fidelity end of the spectrum, since it approxi¬ 
mates the bone structure, inertial properties, kinematics, 
and actuation of the human hand. 

Actuation is another property of humanoid robots 
that varies considerably. Human actuation consists 
of a complex, highly-redundant system of variable¬ 
stiffness muscles. In contrast, many humanoid robots 
use a stiff, position-controlled actuator at each joint. 
There are early exceptions, such as the use of series 
elastic actuators in Cog’s arms [67.32,42], and various 
forms of compliant actuation have now become com¬ 
mon in the arms of humanoid robots. 

67.3.3 Sensors 

Humanoid robots have made use of a variety of sensors 
including cameras, three-dimensional (3-D) cameras, 
laser rangefinders, microphone arrays, lavalier micro¬ 
phones, and pressure sensors. Some researchers choose 
to immitate human sensing by selecting sensors with 



Fig. 67.12 The NASA (National Aeronautics and Space 
Administration) Robonaut consists of an upper body 
placed on a wheeled mobile base 
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clear human analogs and mounting these sensors on the 
humanoid robot in a manner that mimics the placement 
of human sensory organs. As discussed in Sect. 67.6, 
this is perhaps most evident in the use of cameras. Two 
to four cameras have often been mounted within the 
heads of humanoid robots with configurations similar 
to human eyes. 

The justifications for this bias towards human¬ 
like sensing include the impact of sensing on natural 
human-robot interaction, the proven ability of the hu¬ 
man senses to support human behavior, and aesthetics. 
For example, with respect to human-robot interaction, 
nonexperts can sometimes interpret the functioning and 
implications of a human-like sensor, such as a cam¬ 
era, more easily. Similarly, if a robot senses infrared 
or ultraviolet radiation, the robot can see a different 
world than the human. With respect to behavior, place¬ 
ment of sensors on the head of the robot allows the 
robot to sense the world from a vantage point that 
is similar to that of a human, which can be valu¬ 


67.4 Locomotion 

Bipedal walking is a key research topic in humanoid 
robotics (see also Chap. 48, Legged Robots , for a review 
of this topic in the context of locomotion in general). 
Legged locomotion is a challenging area of robotics 
research, and bipedal humanoid locomotion is espe¬ 
cially challenging. Some small humanoid robots are 
able to achieve statically stable gaits by having large 
feet and a low center of mass, but large humanoids with 
a human-like weight distribution and body dimensions 
typically need to balance dynamically when walking 
bipedally. 

67.4.1 Bipedal Locomotion 

Currently the dominant methods for bipedal legged 
locomotion with humanoids make use of the zero- 
moment point (ZMP) criterion to ensure that the robot 
does not fall over [67.43], As discussed in detail in 
Chap. 48, control of the robot’s body such that the 
ZMP sits within the support polygon of the robot’s foot 
ensures that the foot remains planted on the ground, 
assuming that friction is high enough to avoid slip¬ 
ping. The ZMP can be used to plan walking patterns 
that make the robot dynamically stable while walk¬ 
ing. Conventionally, biped locomotion had been offline 
generated by solving an ordinally differential equa¬ 
tion with respect to the motion of the COG (center 
of gravity) given a desired trajectory of the ZMP. 


able for finding objects that are sitting on a desk or 
table. 

Prominent humanoid robots have added additional 
sensors without human analogs. For example, Kismet 
used a camera mounted in its forehead to augment the 
two cameras in its servoed eyes, which simplified com¬ 
mon tasks such as tracking faces. Similarly, versions of 
ASIMO have used a camera mounted on its lower torso 
that looks down at the floor in order to simplify obstacle 
detection and navigation during locomotion. 

67.3.4 Other Characteristics 

Other common forms of variation include the size of the 
robot, the extent to which the robot attempts to appear 
like a human, and the activities the robot performs. The 
remainder of this chapter provides examples from three 
active areas of humanoid robotics research: locomotion, 
whole-body activities, and morphological communica¬ 
tion. 


Recently, several extensions have been done for the 
ZMP based biped gait generation as shown in the 
following: 

Realtime Walking Pattern Generation 
By solving the ordinally differential equation in re¬ 
altime, biped gait is generated in realtim [67.44,45]. 
Since the realtime walking pattern generator enables 
us to change the landing positions of foot in realtime, 
it is used in various situations; in [67.46], the land¬ 
ing position of the foot changes in accordance with 
the hand reaction force as will be described more con¬ 
cretely in the subsection of manipulation. As shown 
in Fig. 67.13 [67.47], the walking pattern is gener¬ 
ated in realtime in accordance with the amount of 
external disturbance applied to the robot. In this case, 
after the torso of a robot is pushed by a human, 
biped gait for a few steps is generated in realtime to 
recover the balance. In [67.48], the humanoid robot 
ASIMO walks in the environment with moving ob¬ 
stacles. By using the estimation of the object motion, 
the walking pattern of the robot is generated in real¬ 
time. Fig. 67.14 [67.49] shows the biped locomotion 
on uneven terrain. In this experiment, the shape of the 
environment is measured by a laser range sensor. Ac¬ 
cording to the shape information of the environment, 
the landing position on uneven terrain is calculated in 
realtime. 
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Fig. 67.13 Experiment of push recovery 


Running 

By additionally considering the flight phase to 
the ZMP based walking pattern generator, run¬ 
ning motion of biped robot is generate [67.50-52], 
Fig. 67.15 [67.52] shows an example of running mo¬ 
tion by a biped humanoid robot. Running motion 


Fig. 67.14 Experiment on biped gait on uneven terrain 


of a human-sized humanoid robot with 7 km/h is 
realized. 

Extension of ZMP Based Method 
Althogh the ZMP is a two dimentional information de¬ 
fined for the interaction between a robot and the ground 
surface, a robot applies 6 dimensional force/moment 
onto the ground. Hence, just by regulating the position 
of the ZMP, it is impossible to control all dimension of 
the interaction force/moment. More concretely speak¬ 
ing, the robot may slip on the ground surface or may 
loose contact with the ground. Research on biped lo¬ 
comotion considering full 6 dimensional force/moment 
has been don [67.53, 54], 

Human-Like Walking Motion 
The biped locomotion generated just by using the ZMP 
may not be a human-like one. Challenge has been done 
to generate a human-like biped gait of a humanoid 
robo [67.55]. Fig. 67.16 [67.55] showws a biped loco¬ 
motion where single toe support, knee stretching and 
human like swing leg trajectory are applied. It is com¬ 
pared with the human walking motion where the model 
belongs to Walking Studio Rei. 

Force/Moment Controller 

Bipedal walking needs to be robust to unexpected dis¬ 
turbances encountered during the execution of planned 
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Fig. 67.16 Human like walking motion compared with human motion 


walking patterns. In these situations, walking can some¬ 
times be stabilized with feedback control and appro¬ 
priate sensin [67.56]. Many humanoid robots, such as 
Honda’s ASIMO, make use of accelerometers, gyro¬ 
scopes, and six-axis force/torque sensors to provide 
feedback to the robot during locomotion. 

Force/torque sensors have long been applied to ma¬ 
nipulators for the implementation of force control, but 
force/torque sensors with sufficient robustness to han¬ 
dle foot impact for a full-size humanoid robot are 
relatively new. When the foot of the robot touches 
down, the foot receives an impact which can disturb 
its walking. This impact can be rather large, especially 
when the robot is walking quickly. Some feet now in¬ 
corporate a spring and damper mechanism as shown in 
Fig. 67.17 in order to mitigate these problems. 



Fig. 67.17 Example of a humanoid foot structure for legged loco¬ 
motion that uses compliance and force/torque sensing 


that walk with high efficiency in a human-like way by 
exploiting natural dynamics (Fig. 67.18 [67.57]). 


Passive-Gait-Based Approach 
Alternative to the ZMP-based approach, researchers 
have begun to use the principles of bipedal passive- 
dynamic walkers to develop powered bipedal walkers 


67.4.2 Other Various Locomotion Styles 

Most humanoid robots have two legs and two arms. 
Here, in addition to the legs, the arms can be used to 
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Fig. 67.18 These robots from Delft, MIT and Cornell (left 
to right) are designed to exploit their natural dynamics 
when walking (after [67.57]; courtesy Steven H. Collins) 

enhance the mobility of a humanoid robot. For exam¬ 
ple, when a robot walks while grasping a handrail, the 
contact could potentially increase the stability of the 
robot. Attempt [67.58] have been done to generate the 
motion of a humanoid robot by using the arms in addi¬ 
tion to the legs to enhance its mobility of it. As shown 
in Fig. 67.19 [67.58], a humanoid robot walking on 
uneven terrain sometimes uses to increase the robot’s 
stability. 

A human-scale robot should expect to fall from time 
to time in realistic conditions. A humanoid robot may 
fall down due to a large disturbance even if the motion 
is planned carefully and a sophisticated feedback con¬ 
troller is applied to the robot. In this event, the robot 
could be damaged significantly during a fall, and could 


also damage the environment or injure people who are 
nearby. An important area of research is how to control 
the robot’s fall in order to gracefully recover or min¬ 
imize damage. The Sony QRIO (Quest for cuRIOsity) 
can control its falling motions in order to reduce the im¬ 
pact of touch down [67.50], although it is of a relatively 
small size (which simplifies the problem). Fujiwara 
et al. developed a falling motion controller for a human- 
size humanoid robot that is falling backwards [67.59]. 
Figure 67.20 shows an example of a controlled falling 
motion. The general problem is still very much an ac¬ 
tive area of research. Similarly, there is also the issue of 
getting back up again [67.60] (Fig. 67.21). 

67.4.3 Localization and Navigation 
Among Obstacles 

In order for a humanoid robot to walk in unmod¬ 
eled environments, localization and obstacle detection 
are essential. Wheeled robots encounter similar is¬ 
sues while navigating, but full bipedal humanoids have 
more-specialized requirements. For example, bipedal 
humanoids have the ability to control contact with the 
world through their highly articulate legs. 

Artificial landmarks can simplify localization. As 
shown in Fig. 67.22, Flonda’s ASIMO uses a camera 
mounted on its lower torso that looks down at the floor 
to find artificial markers for position correction [67.61], 
Accurate positioning is important for long-distance 
navigation and stair climbing, since slippage usually 
occurs while walking and accumulated positional and 
directional errors can lead to severe failures. 

Obstacle avoidance is also an important function for 
locomotion. Disparity images generated by stereo vi- 
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Fig. 67.20 Example of controlled falling-down motion 


Fig. 67.21 The humanoid robot HRP-2P getting up from 
a lying-down position ► 


sion have been utilized for this purpose. For example, 
the plane segment finder [67.62] developed by Okcidci 
et al. helps detect traversable areas. Figure 67.23 shows 
the result of detecting clear areas of the floor plane ap¬ 
propriate for gait generation. 

Humanoids require a great deal of computation due 
to the need for sophisticated sensing and control. Cus¬ 
tomized computational hardware may help mitigate this 
problem. For example, Sony’s humanoid robot QRIO is 
equipped with a field-programmable gate array (FPGA) 
to generate disparity maps in real time from the stereo 
cameras. This real-time vision system has been used to 
detect floor areas, stair steps, and obstacles for naviga¬ 
tion [67.63, 64]. 

67.4.4 Generating Motions 
when in Contact 
with an Object 

Many approaches to whole-body motion generation as¬ 
sume that the robot is only in contact with the ground. 
When a humanoid robot’s hands make contact with the 
environment, it can no longer maintain balance using 
the conventional ZMP property defined by the center 
of pressure of the supporting feet [67.65]. This leads 
to significant challenges for whole-body activities, es¬ 
pecially since the properties of the environment with 
which the robot is making contact may not be known in 
advance. 

Harada et al. have introduced generalized ZMP 
(GZMP) as a method of handling some of these is¬ 
sues, such as the hand reaction forces generated from 
contact with the environment [67.66]. Researchers 
have developed methods that directly make use of 
the six-dimensional force/torque acting on the robot 
at the hands, which can be sensed with conventional 
force/torque sensors placed at the wrists [67.67]. Re¬ 
searchers have also developed specialized methods for 
generating stable robot motion while an object is being 
manipulated [67.46, 65, 68-70]. 




Fig. 67.22 ASIMO and artificial landmarks on the floor 




Fig. 67.23 Plane segment finder for detecting traversable floor 
area 
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t= 18 s r = 21s 


Fig. 67.24 (a-h) Lifting an object while moving the waist 
to compensate for the load (after [67.71]) 

Carrying an Object 

In a manner analogous to the previously described 
methods, coarse motions that do not consider the hand 
reaction forces can be modified [67.65, 68, 71, 72]. Fig¬ 
ure 67.24 shows an experimental result of carrying an 
object that weighs 8 kg [67.71]. Based on measure¬ 
ments of the hand reaction force, the position of the 
waist is modified to compensate for the load and main¬ 
tain stability. 

Pushing an Object 

As another example of using force sensing to adapt 
behavior, consider the problem of pushing a large ob¬ 
ject placed on the floor. For such a task, if the gait 



t = 12 s t = 14 s 


t = 18 s / = 22 s 


t = 32 s / = 42 s 

Fig. 67.25 (a-h) Example of pushing manipulation and 
cooperation (after [67.46]) 

pattern is determined before the robot actually moves, 
the robot may not stay balanced if the weight of the 
object or the friction coefficient between the object 
and the floor is different from the predicted values. 
To address this problem, the gait pattern can be adap¬ 
tively changed depending on the output of a force 
sensor at the end of the arms in order to handle 
changes in the object’s weight and the friction coeffi¬ 
cient [67.46]. 

Figure 67.25 shows an experimental result for this 
approach [67.46]. In the experiment, the table weighs 
about 10 kg. Even though the motion of the table is dis¬ 
turbed externally during the experiment, the robot stays 
balanced by adaptively changing its gait pattern based 
on the measured forces. 
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67.5 Whole-Body Activities 

The two previous sections have focused on humanoid 
locomotion and manipulation separately. This section 
outlines whole-body motions that require coordinated 
control of arms and legs to perform various tasks, 
such as carrying a bulky objects, climbing a lad¬ 
der, or going through narrow spaces with contact 
supports on the environment. Humanoid motion is 
characterized by their redundancy and underactuation 
(Chaps. 10 and 17). Unlike fixed industrial robots, it 
has a floating base (usually it is set at the pelvis) 
that can only be controlled through the leg locomo¬ 
tion or multiple contact motion involving arms and 
legs. It is therefore essential how to define the de¬ 
sired task and to generate the motion that achieves it. 
Since humanoids have a redundant structure, a gen¬ 
eral approach is first to generate coarse motion, and 
then to transform it into a whole-body coordinated 
joint trajectory that is executed by a controller main¬ 
taining the stability through sensor feedback, as il¬ 
lustrated in Fig. 67.26. The first half of this section 
addresses the first two components of this general 
approach. 

The latter half of this section deals with issues re¬ 
lated to various complex whole-body motion that the 
basic conversion methods addressed in Sect. 67.5.2 
could not always resolve. Resolution of various con¬ 
current tasks, including those expressed as inequality 
and dynamic constraints is presented in Sect. 67.5.3. 
This framework can be applied to a reaching task 
while keeping the visibility of the object, as well as 
footstep planning. Finally, motion generation including 
multiple contacts is introduced as an advanced topic 
in Sect. 67.5.4. A wide variety of its application is 
expected to extend the activity fields of humanoids 
in cluttered environments where the humanoid should 
maintain its balance by supporting its body on non- 
coplanar contact points. 



Fig. 67.26 Overview of motion-generation stages for 
a balancing robot 


67.5.1 Coarse Whole-Body Motion 

There are several ways to generate coarse humanoid 
motion: 

1. Using motion capture system 

2. Using graphical user interface (GUI) 

3. Using automated motion planning 

4. Using abstract task specification. 

Using Motion Capture System 
As humanoid robot has a human-like structure, a natural 
and common way of motion generation is using mea¬ 
sured human motion. Motion retargeting from recorded 
human motion to digital characters is a well-studied 
area in computer animation. Typically a human subject 
performs actions while wearing easily detected mark¬ 
ers on his or her body. The motion of these markers 
is recorded by cameras placed in the room, and soft¬ 
ware then infers the 3-D positions of these markers over 
time. A number of studies have been reported to convert 
the capture motions to humanoid whole-body motions 
through learning [67.73] and optimization [67.74,75]. 
Figure 67.27 shows an example: the captured mo¬ 
tions of a woman performing a Japanese traditional 
dance [67.76] that the performance in Fig. 67.4 is based 
on. Kinematic similarity allows using the captured 
motion as a reference for a humanoid’s whole-body mo¬ 
tion, by computing the corresponding joint angles from 
forming virtual links with several markers. However, 
due to dynamic differences, such as mass distributions 
and torque generation, captured motions are generally 
not stable nor feasible when applied to a humanoid. It 
is therefore necessary to adapt them to humanoid body 
as explained in Sect. 67.5.2. 

Using GUI 

Tools such as those used in character animation for 
computer graphics can also be used to design move¬ 
ments for humanoid robots. If the designer were forced 
to control each of the many degrees of freedom inde¬ 
pendently or to takes care of the balance, the process 
would be tedious and inefficient. One solution that en¬ 
ables non robotics expert to design robot motion is 
key-pose based approach on GUI, which allows the de¬ 
signer to define the key-poses of the desired motion with 
the help of inverse kinematics of the end effectors. The 
interface take care of the interpolation and dynamic bal¬ 
ance compensation so that the input motion is feasible 
for the humanoid as explained later in Sect. 67.5.2. Fig¬ 
ure 67.28 illustrates the overview of a GUI interface 
developed as Choreonoid for this purpose [67.77,78]. 
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Fig. 67.27 A sequence of captured motion of dancing (after [67.76]) 


Using Automated Motion Planning 
The previous two methods are based on mainly joint 
angles from a motion capture system or a GUI. If the 
purpose of the robot motion is going from one con¬ 
figuration to another without collisions and how the 
humanoid moves does not really matter, automated mo¬ 
tion planning can provide efficient solutions (Chaps. 36 
and 47). 

Fast path-planning techniques such as rapidly- 
exploring random trees (RRT) can compute basic 
collision-free postures with static balance [67.79, 80] or 
walking paths [67.81] automatically with a simplified 
model. Figure 67.29 illustrates a humanoid carrying an 
object whose lower body is modeled by a bounding box. 
Given geometric models of the humanoid and the envi¬ 
ronment, initial and goal configurations, the planning 
system automatically searches for a path free of colli¬ 
sions at both upper and lower bodies. This coarse path 
can be converted into dynamically stable whole-body 
motion a walking pattern generator including upper- 
body motion compensation as described in Sect. 67.5.2. 
For further reading on humanoid motion planning, the 


readers are referred to a book dedicated to this sub¬ 
ject [67.82], 

Using Abstract Task Specifications 
Tasks for a humanoid to execute are not always speci¬ 
fied in the joint space, but often in the workspace. For 
instance, if the humanoid wants to grasp an object on 
the table or floor, this task is expressed as the hand 
position and orientation in Cartesian space [67.83-87]. 
Another example is teleoperation: it is easier for the 
operator to guide an operational point, such as the end- 
effector or the head of the humanoid, rather than to 
give a whole-body joint configuration [67.88]. These 
tasks are represented in an abstract way by a smaller 
number of DOF than the redundant structure of hu¬ 
manoid. Figure 67.30 shows a motion of bimanual 
manipulation based on abstract representation of mo¬ 
tion as a sequence of attractor points acting in the task 
space [67.89]. 

On the other hand, other constraints such as balanc¬ 
ing or joint limits should also be taken into account 
to generate a whole-body motion to achieve the task. 



Fig. 67.28 A motion chore¬ 
ography tool working within 
the Choreonoid framework. 
In this tool, whole body 
motions of biped humanoid 
robots can be created with 
key-frame editing simi¬ 
lar to computer graphics 
(CG) character animations 
(after [67.78]) 
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Fig. 67.29 Humanoid modeled by rectangle box with 
a bar. In the first stage the geometric and kinematic path 
planner generates collision-free path for the 9-DOF sys¬ 
tem including robot waist on the plane (3-DOF) and object 
(6-DOF) 

A mechanism is therefore necessary that derives a hu¬ 
manoid positioning as well as a whole-body target 
posture from the abstract goal specification. Whereas 
the previously introduced GUI including automatic 
balancing is useful for cases with large free space, 
cluttered environments require integration of search 
techniques to compute a valid whole-body posture from 
the task [67.90,91]. Motion generation for multiple 
tasks and constraints is later discussed in Sect. 67.5.3. 

67.5.2 Generating Dynamically Stable 
Motions 

The methods presented in Sect. 67.5.1 can be useful 
when generating coarse motions for a humanoid robot, 
such as dance performance, manipulating or walking. 
However, for some of these methods the motions gen¬ 
erated will not take into account the dynamic stability 
of the robot, and may result in the robot falling over. 
This subsection presents some approaches for con¬ 


verting coarse motions to dynamically-stable motions, 
dynamic balancing algorithms and task-balance func¬ 
tional decomposition. In the former approach, all the 
joints including upper body are involved for whole- 
body balancing based on the reference motion, like 
kicking motion that needs upper body motion compen¬ 
sation. The latter uses mainly lower body for balancing 
or walking while upper body takes care of the desired 
tasks such as manipulation. 

Dynamic Balancing 

A framework called autobalancer is one of the pioneer¬ 
ing studies for the dynamic whole-body balancing for 
humanoids. It all joint angles at every sample in time 
by solving a quadratic programming (QP) optimization 
problem in order to convert a given motion to a balanced 
one [67.92], This method can be effective for a motion 
in which static balancing is dominant, such as when 
the humanoid is standing. The autobalancer calculates 
a whole-body motion first by fixing the center of gravity 
(COG) on the vertical axis which passes through a point 
in the support polygon of the humanoid. Then it keeps 
inertia moments around the COG at acceptable values 
in order to satisfy the balancing conditions. This tech¬ 
nique was combined with a fast sampling-based motion 
planner to derive a dynamically motion by exploring 
configurations with balance constraints [67.79]. 

Resolved momentum control (RMC) [67.93] is 
a framework for whole-body control based on the linear 
and angular momentum of the entire robot. The robot is 
regarded as a single rigid body whose linear and an¬ 
gular momentum is to be controlled. At each point in 
time, this framework uses least squares to find joint ve¬ 
locities that will achieve the desired linear and angular 
momentum of the robot. Elements of the momentum 
can also be left unspecified as free variables, which 
is often done in practice with elements of the angular 
momentum. In addition to elements of the momentum, 
resolved momentum control requires that desired ve¬ 
locities for the feet be specified. This method has been 
applied to teleoperation [67.88] or stable reaching or 
kicking motions [67.93]. Other methods like dynamics 



Fig. 67.30 Bimanual manipulation by humanoid robot ASIMO based on a motion representation using attractor dynam¬ 
ics in task space (after [67.89]) 
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Fig.67.31a-g A 3-D collision-free motion for bar-carrying task by humanoid robot HRP-2 from starting (a) an intial 
position to final configuration (goal position) (b) using whole-body motion (after [67.81]). The robot rotates the bar 
horizontally to make the bar go through a gap between poles whose distance is shorter than the of the bar (c-e). By 
making use of the concave part of the carried object (f) for 3-D collision avoidance, it arrives at the goal configuration 
with another avoidance motion (g) ItaMEBSEM) 
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Fig. 67.32 Waist trajectory adjustment, which is automatically pro¬ 
cessed immediately after every time key poses are modified. As 
a result, horizontal waist positions of key poses are slightly modi¬ 
fied and a dynamically balanced motion is obtained (after [67.77]) 


filter that makes reference motions dynamically feasible 
by a humanoid [67.94], upper-body motion compensa¬ 
tion for Waseda bipedal humanoid (WABIAN) [67.95] 
have also been proposed as this type of approach. 

Task-Balance Functional Decomposition 
In this approach, dynamic stability is maintained dur¬ 
ing whole-body motion based on pattern generation or 
a balance compensation by the legs to maintain the 
ZMP inside the foot support area, while upper body is 
in charge of specified tasks such motions as manipula¬ 
tion or designed movement. 

An iterative two-stage motion planning method has 
been proposed for a humanoid to perform manipulation 
and locomotion at the same time [67.81]. At the first 
stage, the motion planner generates the upper-body mo¬ 
tion with a walking path of the bounding box of the 
lower body as in Fig. 67.29. The second stage over¬ 
lays the desired upper-body motion on the dynamically 
stable walking motions generated by a dynamic walk¬ 
ing pattern generator based on preview control of ZMP 
for a linear inverted pendulum model [67.96] (Chap. 48 
and Sect. 67.4). This upper-body motion during walk¬ 
ing induces errors in resulting ZMP from the reference, 
which may make the humanoid instable. By apply¬ 
ing the preview control again to this ZMP error the 
necessary compensation motion can be computed as 
a horizontal offset on the waist position. If the resulting 































Humanoids I 67.5 Whole-Body Activities 1805 


whole-body motion is not collision-free, the planning 
process goes back to the first stage to reshape and 
this procedure is repeated until a valid motion is ob¬ 
tained. Figure 67.31, l<s>JKM£EIl and l<gf'll'Wlaail 
show the resulting collision-free manipulation motion 
of a bar-shaped object in an environment populated with 
obstacles. 

Lower-body motion compensation for dynamic 
balance is also used for GUI-based motion design¬ 
ing [67.77]. The GUI accepts a sequence of key-poses 
for the desired motion as the input and interpolates be¬ 
tween them to compute the whole-body motion of the 
humanoid. Since the generated motion is not dynam¬ 
ically stable in general, both the key poses and the 
interpolated motion are adjusted to be dynamically bal¬ 
anced by applying the waist trajectory adjustment in 
such a way that the trajectory of the ZMP from body 
motion is always inside the foot support area. The ad¬ 
justment only modifies the horizontal waist position of 
the key poses and the interpolated ones, for the adjusted 
motion to be as close as the original one (Fig. 67.32). 
The adjustment is automatically and immediately done 
every time a user has finished an edit operation so that 
the user can see the resulting motion. 

67.5.3 Generating Whole-Body Motions 
with Various Tasks 

The main purpose of the methods in previous section 
was to make the given coarse motion dynamically sta¬ 
ble. One can think of a case the task is only given in an 
abstract manner, for instance reaching the end-effector 
in specified position and orientation in workspace, or 
aligning a camera axis in a direction. This section takes 
a step forward in order to generate automatically the 
motion to achieve the specified tasks by taking into ac¬ 
count such constraints as balance, foot positions or joint 
limits at the same time. Generalized inverse kinematics 
technique with task priority and its extension is utilized 
as a key tool for local whole-body motion generation 
(Chap. 10). 

The main particularities of humanoid robot from the 
viewpoint of inverse kinematics are the following: ne¬ 
cessity of dynamic balancing, changing fixed root joint 
and floating base frame. Those issues should be dealt 
with appropriately depending on the task of the robot. 
Some extensions for more complex tasks including in¬ 
equality constraints, footstep planning and dynamics 
are also mentioned at the last part of this section. 

Dynamic Balancing and Walking 
Some examples are shown the whole-body motion 
generation based on task priority generalized inverse 
kinematics. As shown in Chap. 10, this framework ac¬ 


complishes first the task with the highest priority and 
then tries to achieve those with lower priority at the best 
in the null space of the higher-priority tasks. 

Tasks are specified locally as a velocity in 
workspace, such as hand velocity to reach the target. 
The balance constraint can therefore be expressed as 
the velocity of the center of mass (COM). The ZMP- 
based pattern generator has the advantage that it outputs 
the velocity of the COM of dynamically stable walking 
motion from the reference ZMP trajectory, which can 
be easily integrated into this inverse kinematics frame¬ 
work by using COM Jacobian [67.97]. Figure 67.33 
shows the whole-body reaching motion including a step 
to take a ball localized by a vision system [67.85]. 
The high priority is assigned to COM and foot mo¬ 
tion to avoid falling in this example. As can be seen, 
the legs are used not only for stepping but also bend¬ 
ing to reach a lower position in a manner coordinated 
with the upper body. The left arm moves backwards 
as the result of balancing task. Whole-body motions 
for manipulation of daily-life tools [67.84] or object 
pushing/lifting [67.98-100], and also for self-collision 
avoidance [67.101] have been implemented also based 
on a similar framework. 

Another example is given in Fig. 67.34, 
I4S&M2H5E3 and l^ajEESjEH where the humanoid 



Fig. 67.33 (a-f) A whole-body grasping motion generated 
through task-priority generalized inverse kinematics (af¬ 
ter [67.85]). Upper and lower bodies coordinate to achieve 
the desired grasping task while making a step and main¬ 
taining the balance 
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Fig.67.34a-f Experiments of whole-body pivoting manipulation. 
Starting from the initial position (a) with obstacle at right-hand 
side, the humanoid robot manipulates the object backwards away 
from the wall (b). After switching motion direction to forward 
(c), the robot continues to manipulate the object to the goal po¬ 
sition by avoiding the obstacle (the hanger) (d-f) 



Fig. 67.35 Footstep planning modeled as a whole-body inverse 
kinematic problem (after [67.102]) (l<£3>*lhHltU, laaiMlhlililll) 

execute pivoting manipulation to carry a bulky object 
without lifting [67.86]. In this case, a coarse path of 
the object towards its goal position is fist planned 
to compute the trajectory of the hands that perform 
the manipulation. Then foot positions are determined 
along the object path, from which the COM trajectory 
is derived using the dynamic walking pattern generator. 
Those tasks are provided to the inverse kinematics 
to generate the coordinated arm and leg motion for 
this complex manipulation. In addition to the dynamic 
balance, the change of the root joint is also considered 
to compute the whole-body motion when the support 
leg changes during walking. The same framework has 
been applied to a motion for catching a moving object 


during walking where the task of visual tracking by 
the end-effector is integrated with dynamic walking 
motion [67.103]. 

Floating base frame of a humanoid sometimes 
brings difficulties in determining the whole-body con¬ 
figuration from a specific abstract task specified in 
workspace. In the methods mentioned above, the goal 
configuration is derived as a result of repeated compu¬ 
tation of local generalized inverse kinematics. However, 
there are often cases where goal whole-body config¬ 
urations is first needed to be used popular motion 
planning techniques searching in configuration space 
such as sampling-based planning. Some methods that 
derive the goal configuration based on inverse kinemat¬ 
ics can be useful for this purpose [67.90,91,104] or 
a precomputed reachability map that characterizes the 
capacity of reaching in discretized workspace around 
the robot [67.105, 106]. 

Extensions for Complex Tasks 
The whole-body motion generation with tasks can be 
extended to cope with more complex tasks such as 
stepping and those expressed as inequality or dynamic 
constraints. 

One extension particular to humanoid is incor¬ 
porating stepping in the framework of whole-body 
generalized inverse kinematics. Kanoun et al. [67.102] 
introduced an augmented robot structure by introduc¬ 
ing virtual planar links attached to a foot that represents 
footsteps as illustrated in Fig. 67.35, I^EESETl 
and l-s&M'll’Jt'ITUI. This modeling makes it possible to 
solve the footstep planning as a problem of inverse 
kinematics, and also to determine the final whole- 
body configuration. After planning the footsteps, the 
dynamically stable whole-body motion including walk¬ 
ing can be computed by using the method presented 
earlier. 

Task-priority generalized inverse kinematics for re¬ 
dundant robots in Sect. 10.3 usually models tasks as 
equalities so that the operational points can achieve the 
desired velocity. However, tasks are sometimes given as 
inequalities: keeping the hand out of some area to avoid 
collisions or robot view obstruction, respecting joint 
limits, or maintaining the COM inside the foot support 
area. Inequality tasks have usually been transformed 
into more restrictive equality constraints through po¬ 
tential fields. A method for extending the task-priority 
inverse kinematics for those inequality tasks is pro¬ 
posed to remove this limitation based on a sequence of 
QP optimization [67.87]. This method searches for the 
optimal sets for the sequence of QPs by minimizing the 
error to the desired equality tasks in such a way that 
inequality ones can also be satisfied at a desired prior¬ 
ity. This method allows the humanoid to perform such 
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Fig. 67.36 Framework of contact 
motion planner composed of contact 
best-first-planner (BFP) and posture 
generator (after [67.107]) 



Fig. 67.37 A sequence of planned 
whole-body motion including multiple 
contacts on a irregular terrain 
(after [67.108]) 


a task of reaching its arm towards an object on the floor 
without obstructing its view. 

This approach based on cascaded QP can be gen¬ 
eralized to generate whole-body motions including 
dynamic equality and inequality tasks [67.109]. In 
addition to inverse kinematics considered so far, in¬ 
verse dynamics is also integrated to the task-priority 
whole-body motion generation framework. By using 
this method, the dynamic balance can be addressed 
directly without converting the dynamic ZMP con¬ 
straint into COM velocity via a pattern generator. This 
approach assumes a torque-controlled humanoid as op¬ 
posed to position controlled ones that are often the case 
for platform currently used. However, dynamic whole- 
body motion generation with various tasks is being 
actively studied owing to not only recent progress of 
robot hardware [67.110] but also increasing interests 
on more complex tasks including multiple contacts pre¬ 
sented in the next section. 

67.5.4 Generating Motions 

Including Multiple Contacts 

The whole-body motions presented so far basically sup¬ 
poses only the contacts between the humanoid’s feet 
and the floor. Looking at our daily life however, con¬ 
tacts other than with feet occurs often, for example pass 


through narrow spaces or to support the body when 
reaching a distant place on the desk. Since a humanoid 
is high affinity to environments designed for humans, 
its application fields could be expanded by exploiting 
the contacts as much as possible rather than by avoid¬ 
ing them as is often the case in motion planning. This 
section addresses planning and control of whole-body 
motions with multiple contacts that have been inten¬ 
sively studied in recent years. The role of planner is to 
derive a global sequence of configuration with multiple 
contacts to reach the goal, whereas the controller gen¬ 
erates dynamically stable motions to transit from one 
contact state to another. 

Motion Planning for Multiple Contacts 
Multicontact nongaited have been proposed that are ap¬ 
plicable to legged robots including humanoids [67.107, 
108,1 12]. By defining a stance as a finite set of contacts 
between the robot and the environment, the planner 
generates a sequence of stances that can reach the 
goal. During the planning, possible transitions from 
a stance are explored by sampling another stance with 
a feasible and stable robot configuration, as shown in 
Fig. 67.36 [67.107]. Figure 67.37 shows a resultant lo¬ 
comotion planned using this planning method [67.108]. 
A more generalized framework is proposed to deal 
with multirobot and multiobject systems [67.1 13]. This 
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Fig. 67.38 Dynamic multicontact 
motion generated through global 
trajectory optimization (after [67. Ill], 

l^zMMSEll) 


generalization allows for a common description and 
treatment of locomotion and manipulation problems, 
either for a single robot or for multiple collaborating 
robots. 

The output of the planner is a sequence of stati¬ 
cally stable contact stances that can be executed through 
quasi-static motions. In order to generate fast but dy¬ 
namically stable motions, a global optimization ap¬ 
proach has been proposed [67.111]. This method pa¬ 
rameterizes joint trajectory by B-Spline function to 
convert the infinite trajectory problem to semi-infinite 
one so that optimization technique can be applied. The 
whole-body motion is generated through nonlinear op¬ 
timization to minimize square torque and execution 
time, by taking into account such constraints as joint 
torque limits and dynamic multiple contact stability. As 
a result, the generated motion is much faster than quasi¬ 
static ones. An example of resultant motion is shown in 
Fig. 67.38 and Trajectory optimization 

approach can also be applied for lifting of a very heavy 
object by generating a weight-lifting motion by always 
respecting physical constraints [67.114]. 

Even though the feasibility of multicontact motions 
like in Fig. 67.38 have been validated with experiments, 
the optimization process is time-consuming and cannot 
cope with errors or disturbances during execution. It is 
therefore necessary to build a controller that ensures ex¬ 
ecution of multicontact whole-body motions. 

Stability Measure for Motion 
with Multiple Contacts 

Before discussing controller, it is worth mentioning 
first dynamic stability measure for non coplanar con¬ 
tacts and its usage. Although the ZMP is well-known 
dynamic stability criteria, it can only be applied to 
contacts on a flat plane. Stability margin for mobile 
robots [67.118] is only applicable for static gait for 
legged robots. A generalized version of ZMP (GZMP) 



Fig. 67.39 Walking on a rough terrain using a support of 
arm on a handrail (after [67.1 15]) 


has therefore been proposed [67.119] by extending the 
ZMP by considering interaction forces other than floor- 
feet contacts. The stable region for multiple contacts 
can be obtained by considering the infinitesimal dis¬ 
placement and the moment about the edges of the 
convex hull of the supporting points. Hirukawa et al. 
proposed another criterion called contact wrench sum 
(CWS), which is the sum of the gravity and the inertia 
wrench applied to the COG of the robot. The humanoid 
is stable if it is inside the polyhedral convex cone of the 
contact wrench between the feet of a robot and its envi¬ 
ronment [67.1 15, 120]. Based on this criterion, walking 



Fig. 67.40 Real-time simulation of a multicontact behav¬ 
ior with user-enabled interactive control of the robot’s right 
hand. A virtual linkage model is overlaid capturing the in¬ 
ternal force behaviors acting between supporting bodies 
(after [67.116]) 


a) b) c) d) 



Fig.67.41a-d A digital figure climing a ladder by using whole- 
body controller with task priority with multiple contacts (af¬ 
ter [67.117]) 
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Fig. 67.42 Snapshots from multicontact planning and control of HRP-2 ingress in a car (after [67.121]). From the initial 
posture, supported on the steering handle and the seat by an arm, the robot finally succeeds in entering the car 


on a rough terrain by supporting a handrail has been 
performed (Fig. 67.39). 

Controlling Whole-Body 

with Multiple Contacts 

The execution of the multicontact motions presented 
earlier by a humanoid requires the control based on 
sensor feedback to absorb unexpected disturbance or 
modeling errors. 

Khatib et al. extended their framework of opera¬ 
tional space approach that enables a humanoid robot to 
perform motions that simultaneously meet prioritized 
objectives in existence of multiple contacts [67.116, 
117]. A torque-based approach for the control of inter¬ 
nal forces is suggested and integrated into the frame¬ 
work for whole-body prioritized multitasking, thus 
enabling the unified control of COM maneuvers, oper¬ 
ational tasks, and internal-force behavior (Fig. 67.40). 
Figure 67.41 shows an example of multicontact be¬ 
havior on a real-time simulator. Hyon et al. proposed 
another approach of passivity-based controller that can 


adapt to unknown external forces applied to arbitrary 
contact points without sensing the contact forces by us¬ 
ing a torque-controlled robot [67.122]. 

Another real-time controller based on linear QP 
optimization has been proposed for whole-body mo¬ 
tion with multiple contacts [67.121, 123]. The motion is 
constrained by the free-floating whole-body dynamics 
of the humanoid robot without using a reduced model 
such as inverted pendulum since the motions we aim 
at are more general than bipedal walking. The opti¬ 
mization process also incorporates such constraints as 
nonsliding condition, actuation torque limits, contact 
forces within friction cones, and avoidance of undesir¬ 
able self-collisions and collisions with the environment. 
Taking a target contact stance and also necessary sen¬ 
sory information as its input, the controller can compute 
feedback control commands to execute the whole-body 
motion with multiple contacts in real time, on less than 
100 Hz control loop. Figure 67.42 shows a simulation 
result of complex motion of a humanoid entering into 
a car based on the control method. 


67.6 Morphological Communication 

Humans evaluate each others’ state through body pos¬ 
ture and movement. It is quite natural to extend this 
form of communication to include robots that share our 
morphology. 

67.6.1 Expressive Morphology and Behavior 

Humanoids can communicate with people through ex¬ 
pressive morphology and behavior. As with people, 
humanoid robots integrate communicative and noncom- 
municative functionality. For example, the arms and 
hands of a robot can reach and grasp, but also point 
and gesture. Heads for humanoid robots are an espe¬ 
cially important example of these overlapping roles, 
and have had an important impact on humanoid robotics 
and robotics in general [67.124]. 

The head of a humanoid robot has two main func¬ 
tions: 


• To orient directional sensors as needed for the pur¬ 
poses of perception, while leaving the main body 
free to meet other constraints such as maintaining 
balance and gait. Cameras and sometimes micro¬ 
phones are usefully oriented in this way. 

• To strike expressive poses, along with the rest of 
the body. Even if a robot head is not intended to be 
expressive, it will be interpreted as being so by hu¬ 
mans - particularly as a cue to the robot’s presumed 
locus of visual attention. It is also possible to de¬ 
liberately engineer an approximate face that can be 
an important line of communication with humans 
(Chap. 72). 

Locus of Attention 

Eyes can be one of the most expressive components of 
a humanoid robot. For humans, eye movements are both 
expressive and important for sensing. Humanoid robots 
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have the option to factor these two roles by moving 
eyes that are only for display, and using sensors placed 
elsewhere. Most humanoid robots, however, use head- 
mounted servoed cameras that play both expressive and 
sensory roles. These mechanisms exhibit different de¬ 
grees of biological realism, for example, the Kismet 
head captured many of the expressive components of 
human eye movements, while having a nonhuman-like 
camera arrangement that simplified some forms of per¬ 
ception (Fig. 67.43). 

Many humanoid robots use biologically inspired, 
foveated vision systems, which provide a wide field 
of view with low detail, combined with a narrow field 
of view with high detail (Fig. 67.44). With appropri¬ 
ate control strategies to fixate the narrow field of view 
on task-salient regions detected in the wide field of 
view, these robots achieve a practical compromise be¬ 
tween resolution and field of view. Additionally, the 
configuration of the eyes communicates the robot’s lo¬ 
cus of attention in an intuitive way. Many systems use 
four cameras, with a narrow- and wide-angle camera 
for each of the robot’s eyes, but some researchers have 
also used special-purpose space-variant cameras mod¬ 
eled after the space-variant receptor densities in the 
human eye [67.125]. 

The eye movements of some humanoids are mod¬ 
eled explicitly after human eye movements. An exam¬ 
ple of a model of this kind is shown in Fig. 67.45. These 
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Fig. 67.43 On Kismet, foveal vision was implemented 
using cameras in the eyes, and peripheral vision used unob¬ 
trusive cameras on the head (after [67.124]). This achieved 
good expression of locus of attention, while simplifying 
the process of differentiating egomotion from motion of 
objects (since the head moved less frequently and more 
slowly than the eyes). This is an example of a partial de¬ 
coupling of expressive and functional concerns, showing 
that many different levels of humanoid fidelity are possible 


bio-inspired approaches to active vision typically have 
four types of visual behavior: 

Saccades. These are high-velocity movements to fix¬ 
ate a new target or catch up with a fast-moving target. 
From a control point of view, these movements are bal¬ 
listic (at least in humans) - once initiated, they continue 
without responding to changing stimuli. 

Smooth Pursuit. These are movements to continu¬ 
ously track a moving target. They apply at low veloc¬ 
ities. These movements respond constantly to visual 
feedback about the target’s location. A fast-moving tar¬ 
get may also trigger small saccades. 

VOR and OKR. The vestibulo-ocular reflex and optoki¬ 
netic response work to stabilize the direction of gaze in 
the presence of movement of the head and body, using 
inertial and visual information respectively. 

Verge nee. This movement drives the relative angle of 
the two eyes so that the same target is centered in both. 
This only applies to two-eyed systems that have this 
freedom of motion. For conventional stereo algorithms, 



Fig.67.44a-d The heads of humanoid robots come in 
many forms. A popular arrangement is to have two cam¬ 
eras per eye, as a crude approximation of foveal and 
peripheral vision in humans, (a) Biomimetic oculomo¬ 
tor control investigated on DB (after [67.126]). (b) Cog’s 
head (after [67.127]). (c) The double-camera arrangement 
can be arranged in a less-double-barreled appearance (af¬ 
ter [67. 128]; see Ude et al. [67. 128] for more examples and 
an analysis) ATR; Humanoid head developed by ATR and 
SARCOS. (d) The Infanoid robot (after [67.129]) 
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Target state Target velocity 



Fig. 67.45 A biomimetic control 
model (after [67.130]), that integrates 
saccading, smooth pursuit, the 
vestibular-ocular reflex (VOR), and 
the optokinetic response (OKR). 
Smooth pursuit and VOR/OKR 
commands are summed, with periodic 
corrections to errors in position made 
by saccades 


vergence is a disadvantage, since the algorithms are 
simplest when the cameras remain parallel. Other al¬ 
gorithms are possible, but it is currently quite common 
not to use vergence. 

67.6.2 Interpreting Human Expression 

The interpretation of human expression is essential 
for many forms of natural human communication that 
could be valuable for humanoid robots. 

Posture and Expression 

The recognition and interpretation of the location and 
pose of humans is important, since humanoids are often 
expected to work in human environments. Algorithms 
for the following functions have been incorporated in 
various humanoids: 

• Person finding 

• Person identification 


• Gesture recognition 

• Face pose estimation. 

ASIMO has used these functions to perform a proto¬ 
typical reception task as shown in Fig. 67.46. The robot 
can find and identify a person, then recognize gestures 
such as bye-bye, come here, and stop, which are utilized 
for performing reception tasks. In general, such func- 



Fig. 67.46 ASIMO recognizing a pointing gesture during 
a reception task 
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tions on a humanoid are not yet robust, and are active 
areas of research. 

Speech Recognition 

Speech is a natural, hands-free mode of communica¬ 
tion between humans, and potentially between robots 
and humans. Speech recognition is a popular inter¬ 
face utilized for commanding a humanoid, and many 
off-the-shelf packages are now available. However the 
use of microphones embedded in the robot is prob¬ 
lematic, because general-purpose speech recognition 
software is usually optimized for utterances captured 
by a microphone that is close to the speaker. In order 
to achieve sufficient recognition performance in natural 
interaction situations between a humanoid and a hu¬ 
man new methods for speech recognition are being 
investigated. These methods compensate for sources 
of noise, such as the robot’s motors and air flow in 
the environment, by using multiple microphones and 
multimodal cues [67.131]. However, at the time of writ¬ 
ing researchers often circumvent these issues by using 
a headset, lavalier, or handheld microphones. 

Auditory Scene Analysis 

In order to attain more-sophisticated human-robot in¬ 
teraction, researchers have been developing methods 
for computational auditory scene analysis on a hu¬ 
manoid robot. The objective of this research is to under¬ 
stand an arbitrary sound mixture including nonspeech 
sounds and voiced speech, obtained by microphones 
embedded in the robot. Beyond speech recognition, this 
also involves sound-source separation and localization. 

As for sound recognition, sound categories such as 
coughing, laughing, beating by hand, adult’s voice, and 
child’s voice have been shown to be recognizable using 
maximum-likelihood estimation with Gaussian mixture 
models. This function has been utilized during interac¬ 
tions between the HRP-2 and a human [67.132]. 

Multimodal Perception 

Sound-source separation can be achieved by beam 
forming. In order to perform beamforming effectively, 
sound-source localization is essential. Vision can be uti¬ 
lized for finding the talker within the field of view. Hara 
et al. used a camera and an eight-channel microphone 
array embedded in the head of HRP-2, and succeeded 
in speech recognition in the presence of multiple sound 
sources by using sound source separation [67.133]. Fig¬ 
ure 67.47 shows a scenario in which speech recognition 
is taking place with television (TV) sound playing in 
the background. 

When integrated with speech recognition, vision 
can also help resolve the ambiguities of speech. For in¬ 
stance, the ambiguity of demonstrative pronouns such 


as this or that can sometimes be resolved by recog¬ 
nizing pointing gestures. Similarly, the face and gaze 
direction can be used to realize communication via 
eye contact, so that the humanoid only replies when 
a human is looking at it and talking to it [67.132]. Mul¬ 
timodal interaction with these functions has also been 
demonstrated by HRP-2, as shown in Fig. 67.48. 

67.6.3 Physical Interaction 

and Developmental Robotics 

Humanoid robots typically use methods for percep¬ 
tion and interaction that are established in fields such 
as computer vision and dialogue systems. There is 
also an emerging research field called developmental 
robotics or epigenetic robotics in which human-like per¬ 
ception and interaction abilities are obtained through 
physical interaction with the real environments includ¬ 
ing humans [67.134—136], In developmental robotics, 
researchers aim at studying the developmental mecha¬ 
nisms, architectures and constraints that allow life-long 
and open-ended learning of new skills and new knowl¬ 
edge in embodied machines. Much of the research in 
this field utilizes humanoid robots, such as the iCub 
shown in Fig. 67.36. As in human children, learning 
is expected to be cumulative and of progressively in- 



Fig. 67.47 HRP-2 recognizing speech with background 
noises (TV sound) 



Fig. 67.48 HRP-2 recognizing face and gaze direction for 
communication via eye contact 
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creasing complexity, and to result from self-exploration 
of the world in combination with social interaction. 
The typical methodological approach consists in start¬ 
ing from theories of human development elaborated in 
fields such as developmental psychology, cognitive sci¬ 
ence, neuroscience, developmental and evolutionary bi¬ 


ology, and linguistics, then to formalize and implement 
them in robots. The experimentation of those models 
in robots allows researchers to confront them with real¬ 
ity, and as a consequence developmental robotics also 
provides feedback and novel hypothesis on theories of 
human development. 


67.7 Conclusions and Further Reading 

Because of the integrative nature of humanoid robotics, n 
this chapter has avoided details and formalisms and lib- r 
erally cross-referenced other chapters within the hand- s 
book that can provide the reader with deeper coverage 
of many of the areas of robotics on which humanoid r 
robots depend. Additionally, this chapter references a 
work within the humanoid robotics community and re- a 
lated communities. f 

Humanoid robotics is an enormous endeavor, c 

The emulation of human-level abilities in a human- t] 

like robot serves as a grand challenge for robotics, li 

with significant cultural ramifications. The motiva- ii 

tions for humanoid robotics are as deep as they h 

are diverse. From the earliest cave drawings, human- u 

ity has sought to represent itself. Robotics is one \ 

of the most recent mediums for this ongoing fas- c 

cination. Besides this deep societal motivation, hu- fi 


manoid robots offer unique opportunities for human- 
robot interaction, and integration into human-centric 
settings. 

Over the last decade, the number of humanoid 
robots developed for research has grown dramatically, 
as has the research community. Humanoid robots have 
already gained a foothold in the marketplace as robots 
for entertainment and research (e.g., the Robo-One 
competition and the NAO from Aldebaran). Given 
the special properties of humanoid robots, they seem 
likely to further increase in number as their capabil¬ 
ities improve and their costs go down. Robots with 
human characteristics, and technologies related to hu¬ 
manoid robotics, also appear destined to proliferate. 
Will human-scale, legged robots with human form be¬ 
come commonplace, as so often imagined by science 
fiction? Only time will tell. 
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3-D collision-free motion combining locomotion and manipulation by humanoid robot HRP-2 
available from http://handbookofrobotics.org/view-chapter/67/videodetails/594 
Whole-body pivoting manipulation 

available from http://handbookofrobotics.org/view-chapter/67/videodetails/595 
Footstep planning modeled as a whole-body inverse kinematic problem 
available from http://handbookofrobotics.org/view-chapter/67/videodetails/596 
Dynamic multicontact motion 

available from http://handbookofrobotics.org/view-chapter/67/videodetails/597 

3-D collision-free motion combining locomotion and manipulation by humanoid robot HRP-2 (experi¬ 
ment) 

available from http://handbookofrobotics.org/view-chapter/67/videodetails/598 
Regrasp planning for pivoting manipulation by a humanoid robot 
available from http://handbookofrobotics.org/view-chapter/67/videodetails/599 
Footstep planning modeled as a whoie-body inverse kinematic problem (experiment) 
available from http://handbookofrobotics.org/view-chapter/67/videodetails/600 
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68. Human Motion Reconstruction 


Katsu Yamane, Wataru Takano 



This chapter presents a set of techniques for re¬ 
constructing and understanding human motions 
measured using current motion capture technolo¬ 
gies. We first review modeling and computation 
techniques for obtaining motion and force in¬ 
formation from human motion data (Sect. 68.2). 
Here we show that kinematics and dynamics algo¬ 
rithms for articulated rigid bodies can be applied 
to human motion data processing, with help 
from models based on knowledge in anatomy 
and physiology. We then describe methods for 
analyzing human motions so that robots can 
segment and categorize different behaviors and 
use them as the basis for human motion un¬ 
derstanding and communication (Sect. 68.3). 
These methods are based on statistical tech¬ 
niques widely used in linguistics. The two fields 
share the common goal of converting continu¬ 
ous and noisy signal to discrete symbols, and 
therefore it is natural to apply similar techniques. 
Finally, we introduce some application examples 
of human motion and models ranging from sim¬ 
ulated human control to humanoid robot motion 
synthesis. 
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68.1 Overview 

Human motions are valuable resources for robots as 
they implicitly represent human strategies for control 
and planning. They are particularly useful when the 
control objective is difficult to formulate as a cost 
function, such as conveying expressions or moving in 
a particular style. Many researchers have used human 
motions for teaching new tasks to robots [68.1] and gen¬ 
erating robot motions with human-like styles [68.2]. 

In order to use human motions for robotics applica¬ 
tions, we first have to reconstruct the motion from raw 
measurement from a motion capture system. For this 


purpose, we can apply many techniques and algorithms 
developed in robotics such as kinematics, dynamics, pa¬ 
rameter identification, and statistical models. In some 
applications such as robot learning, reconstructed mo¬ 
tion data represented as sequences of joint angles are 
often not useful. In this case, the motion must be inter¬ 
preted as higher level representation such as a sequence 
of behaviors. 

This chapter reviews some of the algorithms for 
reconstructing and interpreting human motions for 
robotics applications. The next section describes the 
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kinematic and dynamic computations for human mo¬ 
tion reconstruction. As these algorithms were originally 
developed for robot manipulators, we need substantial 
adaptation to human body due to its complexity. In 
Sect. 68.3, we present an overview of statistical ap¬ 
proaches for understanding human motions as discrete 
behaviors, and use them for communication between 


humans and robots. The technique presented here has 
a close relationship to natural language processing that 
deals with the similar problem of converting continuous 
signal (sound) to discrete symbols (words). Finally, we 
review a few examples of using human motions to gen¬ 
erate human-like motions on virtual humans as well as 
humanoid robots. 


68.2 Models and Computations 

Motion capture refers to measuring motions in the 
Cartesian space. The subjects are usually humans but 
animals or robots may also be used. Many motion 
capture technologies are commercially available with 
varying costs and capabilities. We will give a brief 
overview of the most popular motion capture systems: 
optical and mechanical. 

68.2.1 Motion Capture Technology 

Optical Motion Capture 

Optical motion capture systems are popular thanks to 
the minimum disruption to motions, although high-end 
systems can be expensive because of high-resolution, 
high-speed cameras and elaborate software. Further¬ 
more, offline data cleanup is often necessary to handle 
occasional occlusion and mislabeling. These systems 
are widely used in many fields including biomechanics, 
entertainment, and robotics. 

Figure 68.1 shows the principle of optical motion 
capture. Multiple markers (white circles) are attached 
to the subjects’ body (gray) as landmarks to be de¬ 
tected by the cameras surrounding the capture area. The 
three-dimensional (3-D) marker positions are obtained 
by computing the intersection of lines connecting the 
cameras’ optical center and the markers on the image 
plane. 

Human whole-body motion capture typically re¬ 
quires more than 40 markers. The markers are usu- 



Fig. 68.1 Principle of optical motion capture 


ally spherical and retro-reflective, which means that 
they reflect the incoming light directly back to the 
light source. The subject experiences very little dis¬ 
ruption during motions because the markers are light 
and small, although they can be a problem in motions 
that involve physical contacts with other subjects or the 
environment. 

The capture area is surrounded by three or more 
cameras with near-ultrared light source. Because the 
markers are retro-reflective, they can be easily detected 
in the camera images as bright disks by simple image 
processing with little effect from the lighting condi¬ 
tion. The internal and external camera parameters are 
identified through a system-specific calibration process 
before each motion capture session. 

Human motions are reconstructed in three steps. 
First, the markers are detected from the camera images. 
Then the Cartesian positions of the markers can be com¬ 
puted by using the marker positions from multiple cam¬ 
eras. Finally, joint angle data are computed by perform¬ 
ing inverse kinematics computation (|43>a!M&U£a). 
This step will be described in Sect. 68.2.2. 

The complexity of the software is due to the three- 
dimensional (3-D) reconstruction process where it is 
difficult to identify the correspondence between the 
markers in multiple images because all markers look 
identical. Furthermore, some of the markers may be lost 
due to occlusion and fast motions. In practice, careful 
marker and camera placement is required to obtain con¬ 
sistent measurements. 

Mechanical Motion Capture 
Another family of motion capture technology is me¬ 
chanical, where a mechanical device is attached to the 
subject’s body to directly measure the joint angles. Ear¬ 
lier systems used mechanical linkages consisting of 
rigid links and joint angle sensors, which were bulky 
and therefore significantly restricted the subjects’ mo¬ 
bility. However, more recent systems use flexible suites 
with embedded bend sensors that do not cause as much 
restriction. The suites are equipped with additional in¬ 
ertial sensors to measure the global position. 
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An advantage of this technology is that it can di¬ 
rectly measure the joint angles, and therefore is more 
robust than optical systems. It also allows unbounded 
capture area, possibly in outdoor environments. On the 
other hand, the inertial sensors used for measuring the 
global motion of the body may suffer from drifts that re¬ 
sult in inaccurate global position measurements. Drifts 
are usually corrected assuming that at least one of the 
foot segments is fixed to the environment, but this is not 
the case in motions involving flights. 

Future of Motion Capture 

Development of markerless, realtime, and low-cost mo¬ 
tion capture systems is an active research area in both 
academia and industry. In fact, several commercial 
products with some of those features are becoming 
available. 

For example, the OpenStage system [68.3] from 
Organic Motion realizes realtime markerless motion 
capture. Kinect [68.4] has been developed for gaming 
interface but it can potentially be used as a low-cost, 
realtime motion capture system. Some researchers are 
working on even more challenging motion capture 
techniques, such as markerless motion capture using 
monocular vision [68.5]. 

68.2.2 Skeleton Model and Computation 

Once motion capture data are obtained, we can ana¬ 
lyze the data by computing the joint angles and torques, 
provided that kinematics and dynamics models of the 
subject are available. While a number of algorithms 
have been developed for motion analysis and simula¬ 
tion of robots, applying these algorithms to the human 
body is not straightforward. 

The human body is much more complex than most 
mechanical systems. It consists of a number of ele¬ 
ments such as bones, muscles, organs, and skin, most 
of which are deformable and difficult to model them¬ 
selves. Many joints are also difficult to model and 
simulate because of the complex constraints. For exam¬ 
ple, the center of rotation of the shoulder joint moves 
along with the joint movement. The knee and elbow 
joints are constrained by rolling contacts and ligaments 
between bones. 

In practice, therefore, it is unrealistic to model all 
the details due to computational complexity, and the 
human body is typically approximated by articulated 
rigid body models familiar to the robotics community. 
We can then apply the kinematics and dynamics al¬ 
gorithms for mechanical systems as described in this 
subsection. Figure 68.2 shows an example of detailed 
human skeleton model [68.6], which consists of 155 de¬ 
grees of freedom (DOF) excluding the fingers. 



Fig. 68.2 Detailed human skeleton model 

Even if we use simplified skeleton models, there 
is still an issue of obtaining accurate model param¬ 
eters. As mentioned above, many joints do not have 
a fixed axis as in mechanical joints, which means that 
the kinematic parameters such as link length may not 
be constant as in mechanical systems. It is also difficult 
to estimate the inertial parameters because we cannot 
directly measure the individual link parameters, and be¬ 
cause the human body is flexible and some of the soft 
tissue would move between links. We will discuss the 
parameter identification issue in Sect. 68.2.4. 

Kinematics 

Inverse and forward kinematics computations are used 
to analyze the kinematic quantities of measured mo¬ 
tions such as positions, velocities, and accelerations of 
the links. 

Inverse kinematics computation is critical to data 
obtained by optical motion capture because we have 
to convert the marker positions to skeleton joint an¬ 
gles. Typical inverse kinematics problem in robotics 
(Chap. 2) is to compute the joint angles given the 
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end-effector position and/or orientation. Inverse kine¬ 
matics for motion capture data processing needs 
a slightly different problem setting because many mark¬ 
ers are attached throughout the body, often resulting 
in an over-constrained system. Furthermore, measured 
marker positions would not perfectly fit an articulated 
rigid body model because human kinematics param¬ 
eters may change during the motion, as discussed 
above. 

A simple method for computing the joint angles, 
often used as the default method in commercial opti¬ 
cal motion capture systems, is to first compute the joint 
centers by interpolating the positions of a few strate¬ 
gically placed markers, and then computing the joint 
angles from the joint centers. The advantage of this 
method is that it is easily customizable by, for example, 
simple scripts and therefore suitable for commercial 
systems. 

Another method is to apply numerical optimiza¬ 
tion. For example, the cost function can be the sum of 
squared distances between the marker positions from 
the measurement and those computed by the kinematics 
model. Another approach is to drive the model toward 
the measured marker positions [68.7]. In either case, 
the resulting joint angle will evenly distribute the error 
caused by measurement error and model inaccuracy to 
all markers. 

Figure 68.3 shows an example of inverse kinematics 
using a skeleton model. 

Forward kinematics is performed to obtain the 
Cartesian positions, velocities, and accelerations of the 
links from the corresponding joint variables. These 
quantities are often used for the dynamics computation 
later. 

Dynamics 

Similarly to kinematics computation, we can consider 
both inverse and forward dynamics. 



Fig. 68.3 Inverse kinematics computation example 


Inverse dynamics computes the joint torques and 
possibly contact forces required to perform the mea¬ 
sured motion. Any algorithm for inverse dynamics 
computation of articulated rigid bodies, such as the 
Newton-Euler formulation [68.8], can be applied. 
Chapter 3 gives the details on dynamics algorithms for 
general kinematic chains. 

A major difference from manipulator inverse dy¬ 
namics is that the human body is a floating-base 
system where the root joint can move freely in the six¬ 
dimensional (6-D) space but is not actuated. Generic 
inverse dynamics algorithms for manipulators give the 
6-D force and torque for the root joint but they are 
not actually available. The equivalent force must be 
supplied instead by the contact forces, which have to 
be explicitly considered because they affect the joint 
torques. 

The equation of motion of an n-DOF articulated 
rigid body system is written as 

Tg = M (6)6 +c(9,0)+g(0), (68.1) 

where r G eR" is the generalized force, 6 e R" is the 
generalized coordinate, M e R" x " is the joint-space 
inertia matrix, cel' represents the centrifugal and 
Coriolis forces, and g e R n is the gravitational force. 
In human body models subject to m contact constraints, 
Tg is computed from active joint torques r e R' !— 6 and 
contact forces f c e R m as 

t g = S T r + Jj/ C , (68.2) 

where J c e R mX " is the Jacobian matrix of the con¬ 
tact constraints and S T e 6) j s ^e ma t r i x that 

converts the active joint torques to generalized forces. 
Without losing generality, we assume that the root joint 
corresponds to the first six elements of the generalized 
coordinate. Then S T takes the form 


S T = 


^6x (n —6) 
I(n—6 )x (/i—6) 


(68.3) 


where O* and I* are the zero and identity matrices of 
the sizes given by the subscripts. 

The number of constraints m changes depending on 
the contact condition. If, for example, both feet are in 
flat contact with the ground, m would be 12 because the 
position and orientation of each foot are constrained. 

Equations (68.2) and (68.3) indicate that only / c 
contributes to the first six elements of tg- This property 
corresponds to the well-known fact that the whole-body 
momentum can be changed only by applying external 
forces such as contact forces. We can utilize this fact 
to compute/ c independently of the joint torques [68.6]. 
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We can then compute the joint torques using the bottom 
(n — 6) equations of (68.2). 

Forward dynamics is used for dynamics simulation. 
The simulation in this context usually involves a set 
of controllers that attempts to make the human model 
track the measured motion. This computation is often 
performed to simulate the effects of external distur¬ 
bances and physical disabilities. However, developing 
controller for human body simulation is very difficult 
because the root joint is not actuated and the contact 
forces are subject to unilateral constraints. Although 
this topic is extensively discussed in humanoid robotics, 
graphics, and biomechanics, there is no consensus on 
how to build general controllers for human models. 

68.2.3 Musculoskeletal Model 
and Computation 

Motion analysis based on skeleton models is not suf¬ 
ficient for applications that require anatomical infor¬ 
mation such as muscle lengths and tensions. In biome¬ 
chanics, for example, researchers are often interested 
in computing the muscle tensions required to perform 
a motion. Also in robotics, many muscle-like actua¬ 
tors are developed and used to design more human-like 
robotic systems. 

Musculoskeletal models are used for motion analy¬ 
sis involving muscles. In addition to a skeleton model 
discussed in the previous section, a musculoskeletal 
model includes a musculotendon network model that 
describes where the muscles are attached to and how 





Fig.68.4a,b Examples of muscle models; (a) simple wire 
connecting two bones, (b) muscle with via point 


they are routed around the bones. Similarly to skeleton 
models, musculotendon network models include a num¬ 
ber of simplifications due to the complexity of the real 
human body. A muscle is usually modeled as a line seg¬ 
ment that produces a uniform tensile force. Interactions 
of muscles with bones and skin are not usually modeled 
except for some of the major muscle-bone interactions. 

Apart from these simplifications, the force interac¬ 
tion between the muscles and bones are described as 
accurately as possible. A simple muscle-tendon pair 
connects two bones and produces a torque at the joint 
between them (Fig. 68.4a). More complex muscles may 
pass through one or more points fixed on the bone 
surface (Fig. 68.4b), or may have branches and there¬ 
fore connects three or more bones. Wide muscles, such 
as pennate muscles, are modeled by several parallel 
muscles. 

Figure 68.5 shows a detailed musculoskeletal model 
consisting of 955 muscles [68.6], 

Kinematics computation on a musculoskeletal 
model gives the muscle lengths. The length and its 
velocity may then be used for various analysis. For ex¬ 
ample, we can use Hill-Stroeve muscle model [68.9, 
10] to compute the maximum muscle tension. 

The active joint torques r are now produced by 
muscle tensions f m . Applying d’Alembert’s principle, 
we can derive the relationship between r and f m as 

T = fj m , (68.4) 

where J m is the Jacobian matrix of the muscle lengths 
with respect to the generalized coordinates. 



Fig. 68.5 A detailed musculoskeletal model 
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a) Normalized force t>) Normalized force 




Fig.68.6a ,b Relationship 
between maximum muscle 
tension and muscle fiber 
length, (a) Length-tension 
relationship, (b) velocity- 
tension relationship 


The inverse dynamics computation for muscu¬ 
loskeletal models is summarized as follows: 

1. Compute the joint angles, velocities, and accelera¬ 
tions of the motion capture data. 

2. Compute the joint torques r of the skeleton model. 

3. Map the joint torques to muscle tensions using 
(68.4) 

Because most musculoskeletal models contain more 
muscles than the skeleton DOF, the final step does not 
have a unique solution. The muscle tensions also have 
to be constrained to be tensile. A common approach for 
this problem is to apply numerical optimization tech¬ 
niques [68.6, 1 1], However, actual muscle tensions may 
not be optimal because of the phenomena called co¬ 
contraction, where antagonistic muscles are activated at 
the same time. Such effect can be incorporated by mea¬ 
suring the muscle activity through electromyography 
(EMG) and consider the data in optimization [68. 12]. 

Another method for computing muscle tensions 
is to utilize physiological muscle models. The most 
commonly used model was originally proposed by 
Hill [68.9] and later formulated by Stroeve [68.10], 
Muscle tension / is characterized by three compo¬ 
nents: muscle activity a, muscle length /, and change of 
muscle length l as follows: 

f = a F\(l)F v (l)F mm , ( 68 . 5 ) 

where F max is the muscle-specific maximum voluntary 
force, Fi(*) and F v (*) are the functions that repre¬ 
sent length-tension and velocity-tension relationship 
respectively (Fig. 68.6). The muscle activity a changes 
according to the following differential equation, 

u — ci 

a = ~y~ , ( 68 . 6 ) 

where T is a time constant and u is the normalized in¬ 
put from the motor neuron which can be measured by 
EMG. 


68.2.4 Model Identification 

Obtaining accurate human model parameters is difficult 
but very important for accurate motion and force esti¬ 
mation. For robots, we can build a model based on CAD 
(computer-aided design) models or even measurements 
of the individual link mass, but both are impossible for 
humans. On the other hand, personalized models are 
important to make the analysis results useful for indi¬ 
viduals. 

Identification of Kinematic Parameters 
In human motion analysis, the set of joints constructing 
a human model is usually given. Kinematic parame¬ 
ters include distance between joints and direction of the 
joint axes. 

We can apply some of the kinematic parameter 
identification techniques for robots [68.13] using mo¬ 
tion capture to provide the data necessary for identifi¬ 
cation. Another approach is to use a human dimension 
database [68.14] and use a few measurements from the 
subject to estimate the rest. 

Identification of Inertial Parameters 
Conventional methods for manipulator inertia parame¬ 
ter identification (Chap. 6) use joint torque data, and 
therefore cannot be applied to the human body. 

However, Ayusawa et al. [68.15] proved that contact 
force data give enough information to estimate the same 
set of parameters for floating-base models. As shown in 
Chap. 6, the generalized force tq,- required to perform 
a given motion described by the generalized coordi¬ 
nates 6 i, velocities 9 and accelerations 6 t is written 
as a linear equation of the identifiable inertial parame¬ 
ters p as 

I, (0i, 9i,h)p = r Gi . ( 68 . 7 ) 

We can obtain a number of these samples from motion 
capture data. In manipulators with joint torque sensors, 
all elements of I and Tg are available and therefore it is 
trivial to solve (68.7). 
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As shown in (68.2), the generalized forces for hu¬ 
man body models are produced by active joint torques 
and contact forces. From (68.3), (68.7) can be divided 
into the first six rows and the remaining (n — 6) rows as 

l fi (0 i ,e i ,0 i )p = j1 fi fci, ( 68 . 8 ) 

0j, 6i)p = r,+iljci ■ ( 68 . 9 ) 


For human body, joint torque data T; are not avail¬ 
able. However, we can obtain the contact force data 
from force sensors embedded in the floor. In this 
case, we have all the information in (68.8). Ayusawa 
et al. [68.15] proved that this equation is enough for 
identifying the same set of parameters as using the full 
set of (68.7). 


68.3 Reconstruction for Understanding 


Human motion data can be used for motion and task 
learning for humanoid robots, as represented by imita¬ 
tion learning or learning by demonstration [68.1]. 

68.3.1 Segmentation of Human Motion 

Human motion data are converted to a sequence of con¬ 
figurations, such as joint angles or link positions over 
the whole of the robot’s body, and the sequence is en¬ 
coded into a set of model parameters as the motion 
primitive. This encoding should follow the motion seg¬ 
mentation; a chunk of motion should be obtained by 
finding the frames where the motion starts and termi¬ 
nates in the seamless motion data. Manually selected 
motion segments are intuitive, reliable, and easily as¬ 
signed their relevant motion labels. However, manual 
segmentation is limited by its lack of scalability for the 
large numbers of human motions required for training. 
Human motion should be segmented so that a humanoid 
robot can observe human actions and incrementally 
memorize the motion primitives by segmenting and en¬ 
coding unknown motions. 

In the supervised segmentation method, the rela¬ 
tion between each motion frame and its boundary score 
from the training dataset is learned, and the bound¬ 
ary in the motion observation is identified. For the 
training dataset, human subjects have to detect the mo¬ 
tion boundaries. Supervised segmentation has the same 
drawback in scalability to large motion datasets as man¬ 
ual segmentation. 

There are studies of unsupervised segmentation 
based on various methods: stopped moment, motion 
clustering, and motion prediction. Motion segmenta¬ 
tion based on the stopped moments is very simple. This 
segmentation assumes that a motion pattern transits 
to another motion via the stopped moment. The sim¬ 
plest strategy only has to find a pose when either joint 
stops as a boundary candidate. The extensive approach 
approximates a temporal sequence of the boundary can¬ 
didates by a periodic function as a reference for the 
motion boundaries. These methods may work for rhyth¬ 
mic movements, such as dance, but it is not clear 


whether they would be useful for segmenting daily hu¬ 
man actions [68.16, 17]. 

Stochastic graphical models, typified by hidden 
Markov models (HMM), are used to model the stream 
of motion data [68.18]. An optimal node in the graphi¬ 
cal model that is the most likely to generate the current 
motion observation can be calculated. The observations 
classified into the same node consequently are con¬ 
sidered to be the same motion. In other words, the 
transition between two nodes corresponds to the bound¬ 
ary of the motions [68.19]. 

In natural language processing, word segmentation 
is critical, especially for speech recognition, in which 
words must be selected from a stream of phonemes. 
Predictability strategy hypothesizes that it is difficult 
to predict the character at the beginning of the word. 
A recurrent network based on this strategy has been 
applied to the segmentation of words [68.20]. The 
network learns the sequence of characters, and conse¬ 
quently predicts the character from its preceding one. 
This predictor detects a boundary where the error be¬ 
tween the predicted and actual character becomes large, 
and subsequently selects words. Similar to this ap¬ 
proach, during motion learning motion is converted to 
a sequence of short movements, each of which is rep¬ 
resented by a binary vector equivalent to a word, and 
this sequence is trained as the correlation matrix. This 
matrix can be easily used to predict the short move¬ 
ment by multiplying this matrix by the vector of the 
preceding movement [68.21]. Movements with a large 
prediction uncertainty can be detected as the motion 
boundary. 

68.3.2 Categorization of Human Motion 

Human motion data is encoded into a set of parame¬ 
ters in the dynamical systems [68.22, 23] or stochastic 
models [68.24,25]. The dynamical system represent¬ 
ing state transitions by nonlinear differential equations 
is a popular way to encode human motions and sub¬ 
sequently synthesize robot motion adapted to various 
environments. The dynamics in the human motions is 
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defined as a vector field, and each motion pattern is 
expressed by a closed curve in this field, which is con¬ 
structed so as to attract robot’s motion to the reference 
motion [68.26]. This attractor field will stabilize the 
robot’s motion even when the robot is perturbed by 
the external environment. A popular approach based on 
the dynamical system is a framework of the dynamic 
movement primitives (DMPs) [68.27]. This framework 
learns a single trajectory by using a spring-damping 
model including a forcing term which makes it easy to 
control a robot motion reaching a goal configuration. 
This framework has been extended to learn multi¬ 
ple human motions such that the robot can perform 
various motions and demonstrate the highly scalable na¬ 
ture [68.28]. These dynamical systems mainly focus on 
synthesize and control robot motions adaptively to the 
environment. While they can predict motions and clas¬ 
sify the observation into its specific motion category, 
but they are not directly used for motion classification. 
Another popular approach based on the dynamical sys¬ 
tem is a recurrent neural network (RNN). The RNN 
generates the human posture following the input cur¬ 
rent one, and is optimized such that error between the 
generated posture and measured one can be minimized. 
The resultant RNN can be useful for the motion predic¬ 
tor. The RNN has been extended to a modified structure 
with a layer of the small number of parametric biases, 
into which the dynamics in the human motion can be 
encoded [68.29]. This structure has an advantage of not 
only generating motions from perception, but also rec¬ 
ognize the perception as a category defined by these 
biases. Due to the complexity of the model structure, it 
is difficult to tune the parameters from a large number 
of training data. 

Stochastic framework, typified by HMMs, are 
widely used for categorization of human motions. 
A compact notation A = {77,A,B} represents an 
HMM. FI is a set of initial node probabilities II = to-}, 
A is a transition matrix A = {aA whose entries atj is the 
probability of transitioning from the i-th node to the /'-th 
node, and B is a set of output distributions B = {b,(0)}. 
Figure 68.7 shows the overview of the HMM. Human 
motion is expressed by a sequence of joint angles or link 
positions, x = {0 1 , 62 , ■ ■ ■ • Or}- The parameters of the 
HMM are optimized by Baum-Welch algorithm, which 
is one of expectation maximization algorithms, such 
that the likelihood P(jc|A) of the human motion x being 
generated by its relevant HMM A is maximized [68.30]. 
The HMM represents spatiotemporal features of the hu¬ 
man motions, and is hereafter referred to as motion 
primitive. 

Human motion x can be classified into a motion 
category represented by the motion primitive. The clas¬ 
sification finds an HMM A R which is the most likely to 



Fig. 68.7 The overview of an HMM. An HMM is the 
graphical model. A node extracts the spatio feature and an 
edge extract the temporal feature 


generate the human motion as follows 

Ar = argmaxR(x|A) . (68.10) 

The likelihood P(x|A) can be efficiently computed in 
the recursive manner by the forward algorithm 

«/(/) = y^oq-iO) aijbj(0t) , (68.11) 

i 

P(jc|A) = ^ aj-(i) , (68.12) 

i 

where a,(j) is the probability of generating a sequence 
{ 61 , 02 , ■ ■ ■, 0,} and staying at the j -th node at the 
time t. 

68.3.3 Temporal Sequence 
of Human Motion 

The representations of not only motion segments but 
also the temporal sequence of them are important for 
the robots to accurately understand human actions. Mo¬ 
tion graph is the popular approach to represent the 
transition between human postures [68.31]. It is possi¬ 
ble to transfer between two postures that are similar to 
one another. This framework creates the edges among 
the human postures based on their spatial similarities. 

The motion segmentation and motion catego¬ 
rization can be extended to the representation of 
temporal sequence of human motions. Human mo¬ 
tion is expressed by a series of motion segments 
{... ,x(k— l),x(A),x(A-|- 1),...} by the segmentation 
technique, and the series of motion segments is con¬ 
verted to a temporal series of motion primitives 
{..., X(k— 1), X(k), k(k+ 1),..., } by the motion cat¬ 
egorization technique. The transition structure of the 
motion primitives can be extracted by deriving a motion 
A-gram model from the series of motion primitives. The 
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motion A-gram model assumes that the motion prim¬ 
itive depends only on its N— 1 pieces of the motion 
primitives. The simplest structure is the motion bigram 
(2-gram) model. The probability of transitioning from 
the motion A; to the motion Xj can be estimated as 


P(Xi\Xj) 


C[X(k— 1) = Xj, X(k) = A,-] 

E k c[X(k) = Xj] 

( 68 . 13 ) 


where C(A) is the frequency of the motion A appearing 
in the observation of human motions, and this transition 
probability maximizes the probability of the observa¬ 
tion being generated by the motion bigram model. 

This transition structure can be used for several ap¬ 
plications such as editing animation character, detecting 
abnormal actions, or predicting human actions [68.32], 
In the case of the action prediction, current human mo¬ 
tion is classified into the specific motion A(0). The 
sequence of the motions A = (A(l), A(2), A(A')}, which 
is the most likely to be generated by the motion bigram 
model, can be searched for by Dijkstra’s algorithm 


Ay = argmaxPfAO^OlA^— 1)] 

x P[X(K— 1)|A(A'— 2)] ■ ■ • P[A(1)|A(0)]. 

( 68 . 14 ) 


Another integration of the human motions with 
language is stochastical approach. Human action 
is represented as a sequence of motion primitives 
{A(1), A(2),..., X(k)}. A sequence of verbs co( 1), 
co(2),... ,a>(l) is also manually assigned to the same 
human action, where co(i) is the verb assigned to the 
manually segmented motion at the i-th position. The 
problem of extracting the references between the hu¬ 
man motions and the verbs can be considered as a trans¬ 
lation problem since the human motion is expressed by 
a sequence of symbols (Fig. 68.8). The IBM transla¬ 
tion model has been widely used in the natural language 
processing [68.35], and can be applied to integrating 
the human motions with their relevant verbs [68.36]. 
This translation model consists of translation probabil¬ 
ity f(A|&)) of the motion A being generated by the verb 
co, and alignment probability a(i\j, l, k) that the i-th po¬ 
sition in a sequence of the verbs can align to the y-th 
position in a sequence of the motions. These parame¬ 
ters are optimized such that the following probability 
that the sequence of verbs generates the sequence of 
motions is maximized by the expectation-maximization 
(EM) algorithm. 


0 = p (A(l), A(2),.... A(k)|®(l),ffl(2),...,«(/)). 

( 68 . 15 ) 


The selected sequence of motions is considered 
to be motions predicted from the current motion 

(IdalKIIMEi*). 

68.3.4 Linguistic Interpretation 
of Human Motion 


Additionally, the verb bigram model extracts the tran¬ 
sition between two verbs as the verb transition prob¬ 
ability P ( cOj | cOj) from the sequences of verbs attached 
to the human motions. A sequence of verbs co = 
(a)(1),..., co(l)} which is the most likely to describe 
the human motion is 


Integration of the human motions with the natural 
language leads to not only understanding the human 
motions in the form of linguistic expressions but also 
generating robot’s motions from the queries of language 
commands. A framework of RNN with parametric bias 
has been applied to the integration of the motions and 
language. One RNN represents the dynamics in the 
motions, and the other RNN represents the dynam¬ 
ics of language, more specifically sequences of words. 
The parametric biases shared by these two RNNs join 
together RNNs [68.33]. Given sentences, the paramet¬ 
ric biases for the sentences are estimated in the RNN 
for the language, the RNN for the motion generates 
motions using the parametric biases derived from the 
language RNN. The robot can generate the motions cor¬ 
responding to the sentences. This conversion from the 
language to the motions is reversed, and it allows for 
the conversion from motions to their descriptive sen¬ 
tences [68.34]. 


coji = argmaxP(<w|A) = argmaxP(A|w)P(<w) . 

CO CO 

( 68 . 16 ) 


Verb sequence 
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A sequence of motion primitives 


Fig. 68.8 Mapping between sequences of verbs and mo¬ 
tion primitives 
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Fig. 68.9 Modeling of the behavioral interaction 


,P(A|«) can be computed using the translation proba¬ 
bility and alignment probability, and P(co) can be com¬ 
puted using the verb transition probability. The human 
motion is interpreted as the resultant verb sequence <n K . 
This framework has been extended to handling not only 
verbs but also nouns, adverbs such that human motion 
can be converted to the descriptive sentences [68.37] 

(i<£t>xnn3Ma). 


68.3.5 Theory of Communication 

HMMs has been extended to hierarchical HMMs 
(HHMM) or coupled HMMs (CHMM) to represent 
complicated spatiotemporal features. Statistical model 
is embedded into each node of the HMM in HHMM 
while commonly used HMM has an Gaussian distri¬ 
bution function in the node [68.38], CHMM, in which 
multiple HMMs interact, is a method for representing 
mutual dependence between two such as a human and 
a robot, or a motion and a manipulated object. These ap¬ 
proaches may be applied to the interaction that requires 
the complex representation [68.39]. However, the large 
number of parameters in these frameworks limits their 
application, and it is difficult to use these frameworks 
for human-robot interaction. 

One efficient approach to using the representation 
of human motions as HMM is to design hierarchy for 
motion primitives and interaction primitives. HMMs 
representing human whole body motions can be use¬ 
ful to recognize motion observation. The HMMs can be 
reused to generate human-like motions for humanoid 
robot. A node is sampled according to the node tran¬ 
sition probabilities, and a configuration is sampled 
according to the output probabilities on the selected 
node. The generation of the robot motion by the sam¬ 
pling manner can be done in the same framework. It 
is fundamental for the humanoid robot not only to rec¬ 


ognize human motions but also to generate human-like 
motions, and this framework has been applied to the 
interaction between a human and a humanoid robot as 
shown in Fig. 68.9. 

The behavioral interaction is defined by the 
exchange of the motion primitives between the 
two [68.37]. Observation of two interacting humans is 
converted to a sequence of pairs of motion primitives 
by recognizing their motions using the HMMs. HMMs 
in the higher level represent these interactions. These 
HMMs are more abstract analogy of representation of 
the motions, and useful for recognition and generation 
of the interaction patterns. A hypothesis is then intro¬ 
duced for the simple commutation, more specifically 
the communication is established by recognizing the re¬ 
lationship of the two and subsequently maintaining it. 

The two recognition output of the human and robot 
using the HMMs in the lower level becomes the input 



Fig. 68.10 Human-robot interaction realized by a symbol¬ 
ically hierarchical communication model 
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of the HMMs in the higher level, and the interaction can 
be recognized as a higher HMM. The current node can 
also be estimated in the selected higher HMM. The gen¬ 
eration output from this node are a pair of two motion 
primitives for the human and robot, and they become 
the input of the lower HMMs. One motion primitive 
for the human becomes the prediction of human behav¬ 
ior, and another motion primitive for the robot becomes 
the command for the motion generation. The recog¬ 
nition and generation by the higher HMMs imply the 

68.4 Reconstruction for Robots 

This section reviews some examples of synthesizing 
human-like motions of robots or virtual humans based 
on human motions and/or physiological model. 

68.4.1 Reconstruction for Virtual Humans 

Instead of using inverse kinematics as described 
in Sect. 68.2.2, Demerican et al. [68.42] developed 
a method for reconstructing human motions by directly 
tracking marker trajectories with a physically simu¬ 
lated skeleton model. Figure 68.11 shows the set of 
markers tracked as well as the skeleton model used 
in the simulation. Examples of measured and simu¬ 
lated marker trajectories are shown in Fig. 68.12, which 
demonstrates the smoothness of the reconstructed hu¬ 
man motions. 

68.4.2 Controller Optimization 
from Physiological Model 

Recorded human motion is an outcome of the human 
motor control. Instead of using the outcome, i. e., the 


estimated state of the interaction and the control strat¬ 
egy for the interaction, respectively. The importance 
for the naturally drifting interaction is to represent the 
process of the control strategy based on the estimated 
interaction state as a shortcut between the recognition 
and generation [68.40]. The experiment of interaction 
between the human and robot in a virtual world demon¬ 
strated its validity as shown in Fig. 68.10, and is further 
applied to the human-robot interaction with compliant 
physical contacts [68.41]. 


motion, Wang et al. [68.44] optimized controllers based 
on physiological human model that includes eight ma¬ 
jor leg muscles, including both uniarticular (drive single 
joint) and biarticular (drive two joints) (Fig. 68.13). The 
method optimizes the controller parameters based on 
simulation considering the physiological muscle prop¬ 
erties [68.9]. The results demonstrate good match with 
human measurements in terms of both joint trajectories 
and torques. 

68.4.3 Reconstruction for Physical Robots 

Tracking human motions with a physical humanoid 
robot is not as straightforward as it may appear 
even if the robot has human-like body topology, due 
to various differences between the human and robot 
bodies. 

First, there is always some difference in the kine¬ 
matic properties such as dimensions and proportion. In 
manipulation, for example, simply copying the joint an¬ 
gles will not result in the same hand positions if the 
upper and lower arm lengths are different. 



Marker subsets 
for direct control 


Fig.68.11a,b Marker set and skeletal 
model used in (after [68.42]). 

(a) Motion capture data, (b) Skeleton 
model in simulation and active 
interfaces (SAI) (after [68.43]) 
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Right arm joint angles 



Left arm joint angles 



Time (s) 


Fig.68.12a,b Joint angles computed from direct marker tracking (after [68.42]). (a) Right arm joint angles, (b) Left arm 
joint angles 




Fig.68.13a-c The musculoskeletal model used in [68.44] 
(a) skeleton and muscles (red lines), (b) five uniarticular 
muscles, and (c) three biarticular muscles 

The kinematic difference comes not only from dif¬ 
ferent dimensions, but also from flexibility of the hu¬ 
man body that is difficult to replicate with mechanical 
systems. For example, the human spine has much more 
degrees of freedom than the torso of typical humanoid 
robots. 

Secondly, robot hardware is generally much more 
restrictive in terms of joint motion range as well as 
velocity, acceleration, and torque limits. As a result, 


robots cannot always perform the same range of mo¬ 
tion as humans. We would therefore have to modify the 
original motions such that they are within the hardware 
limits. 

Finally, the inertial properties such as mass, inertia, 
and center of mass locations are different. In motions 
that involve balancing, the different inertial properties 
may make the motion unbalanced and cause a fall. 

While motion capture data have been frequently 
applied to virtual human characters in simulated envi¬ 
ronments [68.45—47], much less work has been done 
on directly controlling humanoid robots with human 
motion capture data. Nakaoka et al. [68.2] divided cap¬ 
tured dancing motion into multiple tasks and optimized 
each task to be feasible for the robot dynamics. Miura 
et al. [68.48] computed feasbile center of mass trajec¬ 
tory based on the center of pressure trajectory deter¬ 
mined from the footsteps in the motion data, and then 
computed the joint motions by inverse kinematics. 

Ott et al. [68.49] and Yamane and Hodgins [68.50, 
5 1], on the other hand, combined online balance control 
and tracking control to address the problem of dif¬ 
ferent dynamics. Figure 68.14 depicts some snapshots 
from the hardware experiment of tracking a human mo¬ 
tion capture sequence with a floating-base humanoid 
robot [68.51] 
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Fig. 68.14 Snapshots from the 
hardware experiment 


Video-References 












Example of optical motion capture data converted to joint angle data 
available from http://handbookofrobotics.org/view-chapter/65/videodetails/762 
Example of muscle tensions computed from motion capture data 
available from http://handbookofrobotics.org/view-chapter/65/videodetails/763 
The Crystal Ball: Predicting future motions 

available from http://handbookofrobotics.org/view-chapter/65/videodetails/764 
Human motion mapped to a humanoid robot 

available from http://handbookofrobotics.org/view-chapter/65/videodetails/765 
Converting human motion to sentences 

available from http://handbookofrobotics.org/view-chapter/65/videodetails/766 
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69. Physical Human-Robot Interaction 


Sami Haddadin, Elizabeth Croft 


Over the last two decades, the foundations for 
physical human-robot interaction (pH Rl) have 
evolved from successful developments in mecha- 
tronics, control, and planning, leading toward 
safer lightweight robot designs and interaction 
control schemes that advance beyond the current 
capacities of existing high-payload and high- 
precision position-controlled industrial robots. 
Based on their ability to sense physical inter¬ 
action, render compliant behavior along the 
robot structure, plan motions that respect hu¬ 
man preferences, and generate interaction plans 
for collaboration and coaction with humans, these 
novel robots have opened up novel and unfore¬ 
seen application domains, and have advanced the 
field of human safety in robotics. 

This chapter gives an overview on the state of 
the art in pHRI. First, the advances in human safety 
are outlined, addressing topics in human injury 
analysis in robotics and safety standards for pHRI. 
Then, the foundations of human-friendly robot 
design, including the development of lightweight 
and intrinsically flexible force/torque-controlled 
machines together with the required perception 
abilities for interaction are introduced. Subse¬ 
quently, motion-planning techniques for human 
environments, including the domains of biome- 
chanically safe, risk-metric-based, human-aware 
planning are covered. Finally, the rather recent 
problem of interaction planning is summarized, 
including the issues of collaborative action plan¬ 
ning, the definition of the interaction planning 
problem, and an introduction to robot reflexes 
and reactive control architecture for pHRI. 
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69.1 Classification 

Robotics is currently undergoing a fundamental 
paradigm shift, both in research and real-world applica¬ 
tions. Classically, it was dominated for the last decades 
by possibly dangerous position-controlled rigid robots 
carrying out typical automation tasks, such as position¬ 
ing and path tracking in various applications. Recently, 
a new generation of mechatronic robots has appeared 
on the landscape, including novel concepts in general 
robot design within the soft-robotics context. This trend 
brings us closer to the long-term goal of safe, seamless 
physical human-robot interaction (pHRI) in the real do¬ 
mestic and professional world (Fig. 69.1). 

Recent advances in physical human-robot interac¬ 
tion have shown the potential and feasibility of robot 
systems for active and safe workspace sharing and col¬ 
laboration with humans. The fundamental breakthrough 
was the human-centered design of robot mechanics and 
control (soft-robotics), which also induced the novel 
research stream of intrinsically elastic robots (series 
elastic actuators (SEA) or its generalization variable 
impedance actuators (VIA)). By considering the phys¬ 
ical contact of the human and the robot in the design 
phase, possible injuries due to unintentional contacts 
can be considerably mitigated. Furthermore, taking into 
account the human’s intention and preferences will en¬ 


able the realization of human-friendly motions and in¬ 
teraction behavior. Some of the most advanced systems 
that were developed are now entering into industrial 
markets. These technologies serve both industrial and 
service-oriented domains. Possible future applications 
of these novel devices developed for close interaction 
with humans are depicted in Fig. 69.2. They range from 
industrial coworkers and mobile servants over robots 
in the professional service sector, assistive devices for 
physically challenged individuals, to service robots for 
the support of general household activities. All of these 
applications share the common requirement of close, 
safe, and dependable physical interaction between hu¬ 
man and robot in a shared workspace. Therefore, such 
robots need to be carefully designed for human friendli¬ 
ness. That is, they have to be able to safely sense, reason, 
learn, and act in a partially unknown world inhabited 
by humans. In turn, this set of requirements necessitates 
the design of novel solutions in various theoretical and 
technological developments. In contrast to the classi¬ 
cal modular view on robotics technology, and the role 
humans play in this, a fundamental paradigm shift in 
robot development has to be pursued. While encompass¬ 
ing safety issues based on biomechanical human injury 
analysis as well as on the kinesiologic biomechanics 



Fig. 69.1 The current paradigm shift in robotics induced by new target domains and robots toward the vision of close 
human-robot coexistence (courtesy of Keller und Rnappich Augsburg (KUKA), Deutsches Zentrum fur Luft- und Raum- 
fahrt (DLR), ABB. Rethink Robotics) 
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Fig.69.2a-f Application examples for pHRI, ranging from shop floor logistics and manipulation (a,b), over professional 
service robots and assisitve devices for the disabled (c,d), to service robots in domestic applications (e,f) 


of human movements, human-friendly hardware design 
and interaction control strategies, learning, perceptive, 
and cognitive key components have to be developed 
and validated. These need to enable robots to track, un¬ 
derstand, and predict human motions in real time in 
a weakly structured dynamic environment. Apart from 
developing the capabilities for interactive autonomy in¬ 
cluding self-improvement, human safety and physical 
interaction have to be embedded at the cognitive de¬ 
cisional level as well. This will enable the robots to 
react or physically interact with humans in a safe and 
autonomous way. Biomechanical knowledge, neurome¬ 
chanical insights, and biologically motivated variable 
compliance actuators can be used to design manipula¬ 
tion/interaction systems of varying complexity close to 
human properties and performance. Further fundamen¬ 
tal insights into novel designs of VIAs for an improved 
torque/mass ratio and energy efficiency with new con¬ 
trol methods to exploit the stiffness and damping prop¬ 
erties are required. 

Planning and adapting motions and tasks of such 
complex systems in real-time require new concepts, 
including tight coupling of control, planning, and learn¬ 
ing, which will lead to reactive behaviors capable 
of self-improvement. Moreover, self-explaining inter¬ 
action and communication frameworks need to be 
developed to enhance the system usability and inter- 
pretability for humans. These should, for example, 
communicate whether a situation is safe or dangerous 
using not only verbal, but also nonverbal communi¬ 


cation cues, such as gestures and emotional feedback. 
Finally, the dependability of all system components and 
algorithms is a major issue, the systematic treatment 
of which is of particular importance for subsequent in¬ 
dustrial commercialization of the technology and also 
for the commercial domestic use of robots in every¬ 
day environments. Thus, the foreseeable breakthrough 
of the next generation of robotic systems in flexible au¬ 
tomation in both small and medium enterprises (SMEs) 
and global market companies depends primarily on 
the pHRI development over the next years. Robotic 
assistance in manual processes that advantageously 
partner human and robot workers has an enormous un¬ 
explored potential to amplify productively in processes 
that previously could not be automated due to techno¬ 
logical, cost, or efficiency reasons. Furthermore, very 
promising application domains of the technology are 
in the professional service sector (e.g., hospital sup¬ 
port systems) and in the logistics domain (food logistics 
and quality inspection), which are so far to a large 
extent still purely manual work places. As the nat¬ 
ural next step, systems capable of pHRI will enter 
the home sector as home assistants, elderly care as¬ 
sist, and assistive devices (Its&WlnR'tW 

IdaMMSEEI) for physically challenged 

people 

l<s>M3E2ISl, First, rather basic tasks, 

such as fetch-and-carry or environment manipulation, 
will be solved followed by applications with increasing 
complexity. 
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Fig. 69.3 Classification scheme for pHRI, by proximity of the interaction and agency (available autonomy) of the robot 


69.1.1 Classification of Interaction 

pHRI falls into the category of proximate interaction 
where humans and robots are colocated, as opposed 
to remote or teleoperated interaction (Chap. 43) [69.1], 
Beyond proximity, the nature of the physical interaction 
can be understood in the context of the tasks and roles 
undertaken by the robot and human actors in a pHRI 
scenario. In all of these scenarios, a key feature is the 
available autonomy, or agency, of the robot partner for 
performing its portion of the task. This agency separates 
pHRI from Cobotic devices [69.2] and other passive 
robotic lift assists that require, by design, input from 
the operator. 

Most work in pHRI can be generally classified 
across three broad categories of interaction: support¬ 
ive, collaborative, and cooperative. Ordered in this 
way we note that these interactions are marked by 
increasing frequency and necessity of physical con¬ 
tact with the robot and level of proximity to the user 
(Fig. 69.3). Further categories include touch-based, per¬ 
sonally responsive robots, for example, Paro [69.3] 
and the Haptic Creature [69.4], and wearable robots 
(Chap. 70). 


In supportive interactions, we group interactions 
where the robot is not integral to the central perfor¬ 
mance of a task, but instead provides the human with 
the tools, materials, and information to optimize the 
human’s task performance or objectives, for example, 
museum tour guide robots, shopping assistant robots for 
aiding seniors [69.5], and homecare robots (Chaps. 65 
and 73). In this context, pHRI is typically concerned 
with safety, that is, preventing and mitigating the effect 
of unexpected contacts or collisions, and performing 
appropriate proxemic behavior. When required, phys¬ 
ical interaction is infrequent and transitory in nature - 
typically limited to handoffs or other infrequent trans¬ 
actional exchanges. To support safety, as well as these 
limited physical interactions, well-structured human- 
robot communication (Chap. 71) is essential. For ex¬ 
ample, recent work [69.6-11] has demonstrated the 
importance of bi-lateral gesture cues in performing 
turn-taking, information sharing, close proximity activ¬ 
ities, and precontact handover operations. 

In collaborative interactions the 

human and robot both work on the task, with the labor 
divided between the robot and human, each separately 
completing the parts of the task best suited to their 
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abilities, but more frequently interacting through turn¬ 
taking and part/tool passing [69.12,13] (l<s>]Klhll!liiJi), 
or haptically enabled mode switching where contact is 
used to switch the robot’s interactive behavior [69.14] 

) In these scenarios, the 
human completes task elements requiring human dex¬ 
terity or decision making, while the robot completes el¬ 
ements not well suited to direct human involvement, for 
example, repetitive or high-force applications, chem¬ 
ical deposition, or precision placement. In both sup¬ 
portive and collaborative interactions, physical space 
is often shared but planned physical interactions, al¬ 
though much more frequent, are still transactional in 
nature. 


69.2 Human Safety 

Providing safety in pHRI is a multifaceted challenge 
and requires an analysis on various levels of abstrac¬ 
tion. pHRI aims at the coexistence of humans and 
robots in a common workspace and at extending their 
communication modes by physical means. This spatial 
proximity leads to a variety of potential threats, deter¬ 
mined by the current state of the system of interest, 
which consists of the human(s), the robot(s), and their 
surrounding environment. Understanding the respective 
threats, in particular regarding potential human injury 
originating from physical robot-human contacts, and 
embedding the insights accordingly into safety stan¬ 
dards/regulations is one of the major challenges of 


Cooperative interactions refer to the extension of 
cooperative manipulation (see also Chaps. 39 and 70) to 
include force interactions with humans. This type of in¬ 
teraction is differentiated from Cobots in that the robot 
operates as an independent agent, rather than a passive 
assist. That is, the human and the robot work in direct 
physical contact, or indirect contact through a com¬ 
mon object, with continuous and cooperative shared 
control of the task. Cooperative interactions encompass 
tasks, such as cooperative lifting and carrying [69.15- 
17] (l<Et>2tihlilIifJF l<Et>MiliLi>AHl), kinesthetic teach¬ 
ing [69.18] (l<g &m'H.H.l*« ) coordinated material han¬ 
dling (e.g., managing long and flexible objects), and 
rehabilitation therapy (Chap. 64). 


nowadays robotics (note that this chapter does not cover 
functional safety or robot dependability). 

69.2.1 Human Injury in Robotics 

Impact Scenarios 

In order to quantify human injury that may occur 
in the context of pHRI, one needs to understand 
how mechanical contacts may cause injury in princi¬ 
ple. Figure 69.4 depicts relevant robot-human impact 
scenarios. These may involve unconstrained impacts, 
clamping in the robot structure, constrained impacts, 
partially constrained impacts, and resulting secondary 



Fig. 69.4 Robot-human 
impact scenario classes. Un¬ 
constrained and constrained 
impacts are considered the 
two main scenarios 
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impacts [69.19]. Apart from such situational defini¬ 
tions, the most urgent question is how to quantify the 
human injury level that might occur due to a colli¬ 
sion between human and robot. The understanding of 
human injury has been treated in the fields of injury 
biomechanics and forensics for several decades and the 
respective studies served for the early work on human 
injury in robotics. In fact, various injury measures from 
biomechanics and forensics were applied to human in¬ 
jury analysis in robotics [69.19-25]. An overview on 
the most important existing injury classification met¬ 
rics and biomechanical injury measures can be found 
in [69.26] (I^MSHHEEi). The most important results 
from the biomechanics, forensics, and robotics litera¬ 
ture are briefly reviewed now. 

Overview Biomechanics Literature 
In order to derive the injury characteristics of different 
body parts for direct collisions with an impactor, which 
is the most relevant case for robotics, countless experi¬ 
ments and publications have been produced over the last 


50 years. The investigated impactors used in robotics 
and biomechanics experiments vary significantly in size 
and shape. However, from the test setups, one can 
identify and cluster principal geometric primitives. The 
main primitives and their parameters are depicted in 
Fig. 69.5. The z-axis of the coordinate frame associated 
with each primitive defines the direction of impact u. 

Numerous relevant impact experiments with ca¬ 
davers, volunteers, crash test dummies, and biological 
tissue for the head, neck, and chest were generated 
(Tables 69.1-69.4). There for all selected experimen¬ 
tal campaigns, the collision scenario, impacted body 
part, impact parameters according to Fig. 69.5, subject, 
and impact velocity are listed. For describing the colli¬ 
sion scenario, we use the following abbreviations: D: 
dynamic, QS: quasi-static, U: unconstrained, C: con¬ 
strained, PC: partially constrained. A collision experi¬ 
ment denoted by DU is, thus, dynamic unconstrained, 
while quasi-static constrained impacts are labeled QSC 
(Sect. 69.2.1, Synopsis). The respective impactor type 
and parameters are listed for comparison. 



Fig.69.5a-e Typical impactor primitives with according parameters, (a) Sphere, (b) edge, (c) cuboid, (d) flat circular, (e) sharp 
tools 
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Table 69.1 Overview of selected impact experiments from biomechanics and robotics literature. Body part: Head 


Impactor type Impactor parameters 

Flat circular 

Collision case Subject Mass (kg) Velocity (m/s) References 

Maxilla, zygoma, frontal, 
temporo-parietal, mandible 

14.3 mm radius 

dynamic constrained 
(DC) 

Cadaver 

1.08-3.82 

2.99-5.97 

[69.27,28] 

Temporo-Parietal 

12.7 mm radius 

DC 

Cadaver 

10.6 

2.7 

[69.29] 

Nose 

14.3 mm radius 

DC 

Cadaver 

3.2 

1.56-3.16 

[69.30] 

Frontal 

35 mm radius 

DU 

Cadaver 

14.3 

3.37-6.99 

[69.31] 

Edge 

Nose 

12.5 mm radius 

DU 

Cadaver 

32,64 

2.77-6.83 

[69.32] 

Maxilla, zygoma, frontal 

10 mm radius 

DC 

Cadaver 

14.5 

2.4—4.2 

[69.33] 

Frontal 

12.7 mm radius 

dynamic partially 
constrained (DPC) 

Cadaver 

00 (human falling 
on impactor) 

2.23-3.14 

[69.34] 

Cuboid 

Temporo-parietal 

50 mm length, 

100 mm width 

DC 

Cadaver 

12 

4.3 

[69.29] 

Frontal 

Size not specified, padded 

DPC 

Cadaver 

5.31-5.97 

3.56-9.6 

[69.35] 

Frontal 

size not specified 

DPC 

Cadaver 

00 (human falling 
on impactor) 

2.23-3.87 

[69.34] 

Sphere 

Frontal 

120 mm radius 

DU, QSC, DPC 

Hybrid III 
dummy 

4, 67, 1980 

0.2—4.2 

[69.36,37] 

Frontal 

203.2, 76.2 mm radius 

DPC 

Cadaver 

00 (human falling 
on impactor) 

2.87-3.5 

[69.34] 


Table 69.2 Overview of selected impact experiments from biomechanics and robotics literature. Body part: Torso 

Impactor type Impactor parameters Collision case Subject Mass (kg) Velocity (m/s) References 

Flat circular 

Thorax 76.2 mm radius, 12.77 mm 

edge radius 

Thorax 76 mm radius, rubber padded 

Thorax 76.2 mm radius, 12.77 mm 

edge radius 

Abdomen 12.7 mm radius 


Sphere 


Thorax 

120 mm radius 

DU, QSC 

Hybrid III dummy 

4, 67, 1980 

0.2—4.2 

[69.36,37] 

Abdomen 

5, 12.5 mm radius 

DC 

Pig tissue 

2-10 

0.5—4.0 

[69.25] 

Edge 


Abdomen 

45° angle, 200 mm length, DC 

Pig tissue 

2-10 

0.5—4.0 

[69.25] 


0.2 mm edge radius 






DU, DC 

Cadaver 

1.6-23.6 

4.34-14.5 

[69.38,39] 

DU 

Volunteer 

10 

2.4—4.6 

[69.40] 

DU 

Cadaver 

19.27 

4.0-10.6 

[69.41] 

DU 

Cadaver 

32,64 

4.9-13.0 

[69.42] 


Table 69.3 Overview of selected impact experiments from biomechanics and robotics literature. Body part: Upper extremities 


Impactor type 

Impactor parameters 

Collision case Subject Mass (kg) 

Velocity (m/s) 

References 

Edge 





Forearm 

12.5 mm radius, angle 0° 

DC 

Cadaver 

9.48 

3.63 

[69.43] 

Forearm 

size not specified 

DC 

Cadaver 

9.75 

2.44, 4.23 

[69.44] 

Shoulder, upper arm, 

forearm 5 mm edge radius, 30° angle 

DC 

Volunteer 

4.16,8.65 

0.45-1.25 


Flat circular 

Forearm, hand 

size not specified 

QSC 

Cadaver 

oo (velocity 
control) 

25 mm/min 

[69.45] 


Table 69.4 Overview of selected impact experiments from biomechanics and robotics literature. Body part: Lower extremities 


Impactor type 

Impactor parameters 

Collision case Subject 

Mass (kg) 

Velocity (m/s) 

References 

Sharp 

Fig. 69.5 

DC Pig tissue, volunteer 

4 

0.16-0.8 

[69.24] 
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Next, some essential characteristics of human-robot 
impacts are elaborated for a more general understand¬ 
ing of the underlying dynamics. 


Robot-Human Impacts 

Let us assume that of a serial chain rigid robot consist¬ 
ing of n joints, there is at most a single link involved in 
a collision. Let 


x c = 


( V A = ( Jc.linl?) \ ( 
\ w c/ \Jc,ang((7)/ 




be the stacked (screw) vector of linear velocity at the 
contact point and the angular velocity of the associ¬ 
ated robot link, with an associated (geometric) contact 
Jacobian i c (q) that is a function of the joint angle q. Ac¬ 
cordingly, the Cartesian collision wrench is denoted by 


7 


ext — 



6l 6 . 


( 69 . 2 ) 


Robot Collision Modeling. When such a collision oc¬ 
curs, the robot dynamics becomes 

M(q)q + C(q,q)q + g(q) + r F = r + T ext , (69.3) 

where M(<jr) e R" x " is the symmetric and positive def¬ 
inite joint space inertia matrix, C (q. q)q e R" is the 
centripetal and Coriolis vector, and g(q) e R" is the 
gravity vector; r e R' ! is the motor torque, and tf e 1” 
is the dissipative friction torque; r ex t G R" is the typi¬ 
cally unknown external joint torque given by 

r ex t = J](q) 7 en- ( 69 . 4 ) 


The effective mass m u of a robot acting in the 
instantaneous collision direction u, which has to be con¬ 
sistent to 3 c (q), can be deduced from M (q) via the 
Cartesian kinetic energy matrix A (q). This is defined as 

A (9) = [J c ( 9 )M(g) — 1 J C ( 9) T ] —1 , ( 69 . 5 ) 

where the inverse of A (q) is based on the decomposi¬ 
tion of the kinetic energy matrix 


Mqr 1 


A v (q) 1 A vco (q) 
A vcoiq ) 1 A a}(q)~ l 


( 69 . 6 ) 


with Kvcoiq) = J c ,iin(?)M(qr) 1 J c , ang (?) T . Finally, m u 
is found to be 


m u = [u T \ v (q) 1 m] 1 


( 69 . 7 ) 


It should be noted that the Jacobian has to be the 
center-of-mass-Jacobian. Otherwise, the entire inverse 
of the Cartesian inertia tensor has to be used, and not 
just its translational component block. More details 
can be found in [69.46]. We assume the local impact 
curvature in the (/-direction to be denoted by c„. 


Characteristic Robot-Human Impact Force Profile. 
A physical collision between robot and human is typ¬ 
ically characterized by a distinct force profile that is 
composed by two consecutive phases (note that for un¬ 
constrained soft-tissue collisions these two phases can 
simplify into a single Phase I impact) (Fig. 69.6): 

1. Phase I is characterized by a very short impact, gov¬ 
erned by the robot- and human-reflected dynamics. 

2. Phase II is characterized by a quasistatic contact 
event. Without clamping, this is a pushing force, 
whereas if the human is clamped it is a crushing 
force. 

Phase I can be treated from a pure impact physics, 
almost open-loop point of view, that is, it is determined 
by the reflected inertia, velocity, and impact curvature 
c„ of the robot together with the characteristics of the 
respective body part that is being struck. The maximum 
contact force is denoted F\. 

Phase II, on the other hand, has to be further sub¬ 
divided into either clamping or no clamping incident. 
In the case of no clamping, the maximum force is Fha, 
whereas for clamping, the maximum force is Fun • In 
particular. Phase II is highly robot control and design 
dependent and is especially important in the case of 
clamping: 

• Phase IIA: No clamping. Typically, for free impacts 
at robot velocities > 0.3 m/s, Fu a is significantly 
smaller than F\. Otherwise, F\ is smaller than Fha 
and is governed by the robot actuator torques (ac¬ 
tive quasistatic pushing) and the reaction of the 
human body that is mainly governed by its reflected 
impedance. 

• Phase IIB: Clamping. In the case of clamping, the 
final maximum force Fnn is limited by the maxi¬ 
mum motor torques r max of the robot via jF ext = 
jJ # T max , where jJ # is the contact Jacobian pseu¬ 
doinverse. If the robot is powerful enough to gener¬ 
ate active contact forces that penetrate or break hu¬ 
man tissue/structure, the contact force is, of course, 
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Frontal collision force (kN) Chest collision force (kN) 
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Fig.69.7a,b Mass-velocity dependency for (a) human head and (b) chest contact force. A mass-spring-mass model is 
used for collisions against the head, where the head mass Mh is 4.5 kg and the approximate contact stiffness of the frontal 
bone Kh = 1000N/mm (after [69.33]). For the chest, the model proposed in [69.47] is used 


limited by the human maximum tissue resistance. 
Please note that singularities need careful treat¬ 
ment, which, however, goes beyond the scope of the 
chapter. 


other impact locations, such as contacts with the chest 
(Fig. 69.7b). 

If the robot mass is significantly larger than the hu¬ 
man head mass, that is, m u Mh, (69.8) reduces to 


Next, the influence of robot mass and velocity for 
the unconstrained impact are described. This analysis is 
particularly important to understand Phase I. 

Influence of Robot Mass and Velocity. Assume 
a simple mass-spring-mass model for the impact be¬ 
tween human and robot. Mh is the reflected inertia of 
the human. Ku is the contact stiffness, which is in the 
case of a rigid robot mainly the effective stiffness of the 
human contact area, jt® is the relative impact velocity 
between the robot and human. Solving the correspond¬ 
ing differential equation leads to the maximum contact 
force 

JeT= (69.8) 
V m u + Mh 

Assume a simplifying decoupling of the head from 
the torso, which holds for the short duration of the 
impact. For the post-impact phase, neck stiffness and 
body inertia have to be considered, which compli¬ 
cates the analysis considerably. The dependency of 
frontal bone contact force on the robot mass and 
velocity is depicted in Fig. 69.7a. It can be ob¬ 
served that collision force (which is a well-known 
bone fracture indicator) generally increases with ve¬ 
locity. For increasing mass, however, a saturation ef¬ 
fect takes place. After a certain robot mass has been 
reached (rn u % 20 kg in Fig. 69.7), additional weight 
has only negligible influence on collision force. This 
inertial saturation effect can also be observed for 


^ Mh) = \! ^hM h x° . (69.9) 

This shows that for a robot with significantly larger re¬ 
flected inertia than the human head, only the contact 
stiffness, the impact velocity, and the mass of the hu¬ 
man head are relevant but not the robot mass. 

The behavior of human tissue during collisions is 
complex. Consequently, surrogates cannot reveal the 
entire diversity. Accordingly, the conduction of human 
voluntary experiments is necessary to fully understand 
human injury and pain dynamics in robotics. 

Human-Robot Impact Voluntary Testing 
The following experimental test was the first systematic 
analysis in this direction. The voluntary experiments 
were conducted with a healthy young adult in the 
year 2011. The collision experiments were performed 
with the KUKA/DLR lightweight robot (LWR) and 
the following approaches to injury and pain analysis 
were carried out: injury severity analysis according to 
AO (Arbeitsgemeinschaft fur Ostheosynthesefragen), 
biomechanical analysis, pain, and imaging meth¬ 
ods. The setup and experiment steps are depicted in 
Fig. 69.8. The robotic system allows to conduct con¬ 
trolled robot-human collisions in order to analyze input 
parameters and their effect on output parameters, such 
as pain and injury. Measured impact characteristics and 
quantities included impact force, impact area, tissue 
displacement, tissue stiffness, stress, impact velocity, 
kinetic energy, and energy density. The reflected inertia 


Part G | 69.2 













Part G | 69.2 


1844 Part G 


Robots and Humans 


a) Drive to contact point 

1 

Mark contact area 

Follow-up examination j 

Drive upward and wait 

i 


t 


MRI-examination 





Fig. 69.8 (a) Flow chart depicting the basic experimental steps, (b) Collision trajectory with subject 


was kept constant at m u = 3.75 kg for every test. The 
used impactor for the resulting Table 69.5 was a sphere 
with a radius of 12.5 mm. 

The injury was defined using the AO-classifica- 
tion [69.48] directly after each test series. Each impact 
series was carried out at the same location on the 
human body at increasing impact velocity until the par¬ 
ticipant initiated a controlled system stop during the 
experiment. The impact areas were then imaged with 
a magnetic resonance imaging (MRI) after a time in¬ 
terval of about 4—5 h. The remaining tissue did not 
show any pathological signs. Compared to an equiv¬ 
alent drop test in [69.25] with abdominal pig tissue 
(large sphere, 4.2 kg, 2.5 m/s), the voluntary experi¬ 
ments provide similar results in terms of injury severity. 
The maximum velocity of 2.55 m/s is at the border 
of inducing a contusion. Where there were no marks 
immediately after impact, a mild contusion formed at 
day 1. For the pain tolerance at a visual analog scale 
(VAS) of 6/10, an impact force of F = 272.2 N was 
measured. The energy density appears to have the most 
significant correlation to pain. 

Synopsis 

An overview of the potential injury threats depending 
on the current state of the robot and the human, a clas¬ 
sification of these mechanisms, governing factors of the 
particular process and possible injuries are depicted in 


Fig. 69.9. Physical contact can be divided into two fun¬ 
damental subclasses: quasi-static and dynamic loading. 
Fundamental differences in injury severity and mech¬ 
anisms are observed as well if a human is (partially) 
constrained or not, leading to the second subdivision. 
For the quasi-static case, it is differentiated between 
near-singular and nonsingular clamping as already out¬ 
lined. The last differentiation separates injuries caused 
by blunt contact from the ones induced by tools or sharp 
surface elements. 

Each class of injury is characterized by possi¬ 
ble injuries (PI), worst-case factors (WCFs), and their 
worst-case range (WCR). WCFs are the main contribu¬ 
tors to the worst case, such as maximum joint torque, 
the distance to singularity or the robot speed. The 
worst-case range indicates the maximum possible in¬ 
jury depending on the worst-case factors. In addition to 
the classification of injury mechanisms for each such 
class, suggestions for injury measures (IMs) are given 
as well. They are specific injury measures which are 
appropriate, useful for the classification and measure¬ 
ment of injury potentially occurring during the physical 
human-robot interaction. Please note that the list of 
injury measures is not necessarily complete, but these 
ones are certainly suitable to be applied to a more gran¬ 
ular robotics injury analysis. This does not mean that 
criteria, such as the well-known head injury criterion 
(HIC), do not provide general insights; they are just not 
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Table 69.5 Impact data for the lateral surface of the right upper arm 


Impact 

Max. 
impact 
force (N) 

Impact 

area 

(mm 2 ) 

Displace¬ 

ment 

(m) 

Tissue 

stiffness 

(N/m) 

Stress a 
(N/mm 2 ) 

Impact 

velocity 

(m/s) 

Kinetic 

energy 

(j> 

Energy 

density 

(J/mm 2 ) 

AO 

VAS 

1 

9.5 

966 

0.03 

316.7 

0.001 

0.2 

0.08 

0.0001 

IC1MT1NV1 

0 

2 

19 

966 

0.037 

513.5 

0.002 

0.44 

0.36 

0.0007 

IC1MT1NV1 

0 

3 

38.1 

966 

0.044 

865.9 

0.039 

0.65 

0.80 

0.0016 

IC1MT1NV1 

0 

4 

59.6 

966 

0.055 

1083.6 

0.062 

0.88 

1.45 

0.003 

IC1MT1NV1 

0 

5 

81.4 

966 

0.058 

1403.4 

0.084 

1.11 

2.31 

0.005 

IC1MT1NV1 

1 

6 

103.5 

966 

0.060 

1725 

0.107 

1.34 

3.37 

0.007 

IC1MT1NV1 

1.5 

7 

128.1 

966 

0.064 

2001.6 

0.133 

1.55 

4.50 

0.009 

IC1MT1NV1 

2 

8 

154.1 

966 

0.069 

2233.3 

0.16 

1.76 

5.81 

0.012 

IC1MT1NV1 

3 

9 

186.4 

966 

0.069 

2701.4 

0.193 

2.03 

7.73 

0.016 

IC1MT1NV1 

3 

10 

224.5 

966 

0.069 

3253.6 

0.253 

2.24 

9.41 

0.019 

IC1MT1NV1 

4 

11 

272.2 

966 

0.077 

3535.1 

0.282 

2.55 

12.2 

0.025 

IC1MT1NV1 

6 


necessarily optimal to understand injury on a more dif¬ 
ferentiated lower-injury scale. 

For example ® in Fig. 69.9 represents blunt clamp¬ 
ing in the near-singular configuration (Fig. 69.9). Even 
for low-inertia robots, this situation could become dan¬ 
gerous and is, therefore, a possible serious threat with 
almost any robot on a fixed base within a (partially) 
confined workspace. Possible injuries are fractures and 
secondary injuries, for example, caused by penetrat¬ 
ing bone structures or an injured neck if the trunk is 
clamped but the head is free. This would mean that 
the robot pushes the head further, while the trunk re¬ 
mains in its position. Another possible threat is shearing 
off a locally clamped human along an edge. Appro¬ 
priate indices are, for example, the contact force and 
the compression criterion (CC) [69.49]. ® in Fig. 69.9 
represents the clamped blunt impact in nonsingular 
configuration. The injury potential is defined by the 
maximum actuation torque r max and can range from 
no injury to severe injury or even death for high-inertia 
(and torque) robots. The robot stiffness does not con¬ 
tribute to the worst case since a robot without collision 
detection would simply increase the motor torque to 
follow the desired trajectory. Therefore, robot stiffness 
only contributes to the detection mechanism by en¬ 
larging the detection time. Also, the contact force and 
CC are well suited to predict occurring injury. ® in 
Fig. 69.9 denotes the unconstrained impact which was 
the first injury mechanism investigated in the robotics 
literature. This process is governed by the impact veloc¬ 
ity and (up to a saturation value) by the robot mass. As 
shown in [69.22], even a robot of arbitrary mass cannot 
severely injure a human head by means of impact- 
related criteria from the automobile industry like the 
head injury criterion (HIC). Flowever, fractures, for ex¬ 
ample, of facial bones are likely to occur but not all 
would be classified as a serious injury. Laceration by 


means of crushes and gashes are worth evaluating, espe¬ 
cially with respect to service robotics. The contact force 
and CC are well-suited severity criteria for this class. In 
order to evaluate lacerations the energy density has to 
be considered. 

The preceding overview is intended as a worst-case 
analysis for the described contact cases. The next step is 
to ask which actions can be taken against each particular 
threat [69.19]. At this point, however, it shall be noted 
that instead of quantifying injury in terms of a mea¬ 
surable injury criterion, injury evaluation by a medical 
expert, for example, via the AO-classification can al¬ 
ways be applied and would presumably result in a more 
exhaustive and precise judgement. 

69.2.2 Safety Standards for Human-Robot 
Interaction 

Robotics standardization made significant progress to 
establish the underlying regulations for co-working 
cells in the real world. Safety for industrial robots is 
addressed in a variety of general standards [69.50— 
52]. The most important industrial robotics standards 
is the International Organization for Standardization 
(ISO) 10218. It was established in the recognition of 
the particular hazards that industrial robots and indus¬ 
trial robot systems may pose. The machinery concerned 
and the extent to which hazards, hazardous situations, 
and events are covered are indicated in the scope of 
ISO 10218. In recognition of the variable nature of 
hazards with different uses of industrial robots, ISO 
10218 is divided into two parts. It provides a de¬ 
tailed analysis of mechanical hazards, such as impacts, 
crushing, shearing, entanglement, drawing-in or tap¬ 
ping, cutting or severing, and contact of persons with 
live parts (direct contact) [69.53]. In particular, the 
introduction of collaborative robots has been a ma- 
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Fig. 69.9 Safety tree showing possible injury (PI), major worst-case factors (WCF) and the possible worst-case range 
(WCR). * indicates still ongoing topics of research. Additionally, relevant injury criteria are given for the head, chest, 
and soft-tissue injuries 


jor acknowledgment to the advances made in robotics 
research in pHRI over the last decade. The recent up¬ 
dates to ISO 10218 (safety requirements for industrial 
robots) led to the development of the new technical 
specification (TS) 15066. It is regarded as a comple¬ 
mentary information that concretizes the content of 
ISO 10218. Generally, ISO/TS 15066 provides guid¬ 
ance for collaborative robot operation where a robot and 
a person share the same workspace. It considers collab¬ 


orative modes and requirements, such as minimum sep¬ 
aration distances, safety-rated monitored stops, speed 
and separation monitoring, and power and force lim¬ 
iting. In collaborative operations, the integrity of the 
safety-related control system is of major importance, 
particularly when process parameters, such as speed 
and force, are being controlled. A comprehensive risk 
assessment is required to assess not only the robot 
system itself, but also the environment in which it is 
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placed, that is, in the workplace. A key process in the 
elimination of hazards and reduction of risks is the 
design of the collaborative robot system and the as¬ 
sociated cell layout. Various considerations about the 
access and clearance of the collaborative workspace are 
provided. During the design of a robotic system, the 
maximum space and the restrictions of the collabora¬ 
tive robot system have to be considered. Furthermore, 
the need for clearances around obstacles and the acces¬ 
sibility for operators should influence the design. The 
intended contact(s) between parts of the robot system 
and an operator plays a major role toward a possi¬ 
bly intrinsically safe design. In order to identify the 
risks resulting from the collaborative action, an appro¬ 
priate set of collision incidents that can occur during 
the collaborative work activities and foreseeable mis¬ 
use has to be determined. This has to include affected 
body regions and the involved collision areas of the 
robot. The limit values that may not be exceeded dur¬ 
ing the collision incident depend on the affected body 
regions. The geometry of the involved areas of the 
robot and the biomechanical properties of the affected 
body regions influence the forces occurring during 
the collision incident. Therefore, the ISO/TS 15066 
describes injury severity criteria that consist of maxi¬ 


69.3 Human-Friendly Robot Design 

Designing robots for interaction has become a chal¬ 
lenging subdomain in pHRI, leading to novel devices 
that have one thing in common: active and/or pas¬ 
sive compliance together with lightweight design being 
the central design paradigms. A number of research- 
focused robots have been designed specifically for 
pHRI. The most important design guidelines, represen¬ 
tatives, and modeling basics are outlined in this section. 
Apart from robots that are designed to act as general 
purpose co-workers, large robot assists with balanced, 
inertia reducing, cable-driven gantry systems can be 
used to create large payload robots that may be oper¬ 
ated next to human workers [69.56]. 

69.3.1 Lightweight Design 

In the process of making robots inherently suitable for 
close physical interaction with humans or only partially 
known environments, a design paradigm shift mov¬ 
ing away from heavy, stiff, and rigid designs toward 
lightweight and highly integrated mechatronics designs 
has taken place. Low inertia and high (active) compli¬ 
ance have become desirable features, as has the use of 


mum allowable limit values on individual body regions. 
These limit values are established to prevent the occur¬ 
rence of skin/tissue penetrations that are accompanied 
by bleeding wounds, fractures, or other skeletal dam¬ 
age [69.54], 

In addition to the industrial standardization efforts 
in the pHRI domain, the ISO 13482 [69.55] is the first 
nonindustrial robot safety standard that allows/regulates 
close pHRI. This international standard specifies re¬ 
quirements and guidelines for the inherent safe design, 
protective measures, and information for the use of so 
called personal care robots. It focuses on three types 
of personal care robots (mobile servant robots, physi¬ 
cal assistant robots, and person carrier robots). These 
robots typically perform tasks to improve the quality 
of life of intended users irrespective of age or capabil¬ 
ity. The standard describes hazards associated with the 
use of these robots and provides requirements to elimi¬ 
nate or reduce the risks associated with these hazards to 
an acceptable level. Significant hazards are presented 
and this standard describes how they are to be dealt 
with for each personal care robot type. Robotic de¬ 
vices used in personal care applications are also covered 
by this standard and are to be treated as personal care 
robot. 


redundant sensing principles on the proprioceptive level 
(position, velocity, and torque). 

General Characteristics 

Generally, two major design approaches for light¬ 
weight robots have proven successful over the last 
years [69.57], namely the mechatronics approach and 
the tendon-based approach, respectively. Their com¬ 
monalities are as listed: 

• Lightweight structures'. Lightweight, high-strength 
metals, or composite materials for the robot links. 
Moreover, the design of the entire system (con¬ 
trollers, power supply) is optimized for weight re¬ 
duction to enable mobility. 

• Low power consumption: This is mainly achieved 
by small moving inertias and accordingly designed 
motors. 

Typically, mechatronic robots integrate electronics 
into the joint structure for allowing highly modular 
units. Such a design enables the assembly of different 
kinematics with increasing complexity, while keeping 
the respective joint philosophy. In terms of actuation, 
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Fig. 69.10 (a) Barrett arm (after [69.58]), (b) Mitsubishi PA10 arm, (c) DLR lightweight robot III (after [69.59]), 
(d) KUKA LBR iiwa (after [69.60]) (courtesy of Barret Technology Inc., DLR, KUKA) 


the combination of high power/torque motors with high 
transmission ratio gears is usually applied. From the 
proprioceptive sensing side, these systems are typically 
equipped with additional sensors, such as joint torque, 
force, and current sensing in addition to basic motor 
side position and current sensing only (Sect. 69.3.3). 

Tendon-based robots have three major character¬ 
istics. First, they are typically equipped with remote 
direct drives. This reduces the overall weight of mov¬ 
ing parts for a fixed base manipulator as the actuators 
are located in the robot base. For placing the actuators 
remotely from the base, cable-pulley systems are usu¬ 
ally applied. Finally, low reduction ratios are used for 
keeping the system backdrivable. In turn, larger motors 
have to be selected, which adds additional total weight. 

Another interesting class of compliant actuators is 
of note that implements compliance quite differently. 
They use rheological fluids such that they can alter their 
characteristic properties under the influence of either 
a magnetic or an electric field. A clutch between link 
and motor is operated in such a way that the output 
torque can be controlled. These actuators were also mo¬ 


tivated and discussed from a human-safety perspective 
in [69.61]. 

Lightweight Robotic Systems 
The most prominent robots that fall into the category 
of lightweight robots are depicted in Figure 69.10. The 
Barrett arm is a classic example of a tendon-based de¬ 
sign, where the actuators are placed in the manipulator 
base and the joints are backdrivable due to the low 
reduction ratio. The Mitsubishi PA10 arm was a com¬ 
mercially available lightweight redundant arm, with 
a weight of 38 kg and a payload of 10 kg. The fully 
torque-controlled KUKA LBR iiwa is based on the DLR 
lightweight robot technology [69.60]. Its third genera¬ 
tion DLR LWR-III weighs 13.5 kg and is able to handle 
loads up to 15 kg, so an approximate unitary payload- 
to-weight ratio is achieved. The robot is equipped with 
joint torque sensors in each joint and has redundant po¬ 
sition measurement (on motor and link side) [69.62], 

Apart from single-arm robots as described above, 
various lightweight designs were also successfully inte¬ 
grated into research and commercial humanoid systems 



Fig. 69.11 (a) NASA Robonaut 2, (b) DLR Rollin' Justin, (c) Rethink Robotics Baxter and (d) Boston Dynamics Atlas 
(courtesy of NASA, DLR, Rethink Robotics Inc., Boston Dynamics) 
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(Fig. 69.11). The NASA (National Aeronautics and 
Space Administration) Robonaut 2 [69.63], equipped 
with SEA, was originally designed for teleoperation 
and exploration in space [69.64]. The fully torque- 
controlled humanoid robot TORO [69.65], based on 
the DLR lightweight robot technology, has its ori¬ 
gins in the upper body bimanual system Justin [69.66] 
(|4a>3KIHIDS3l). Rethink Robotics Baxter is a commer¬ 
cial two-arm upper body system for pick and place tasks 
that is equipped with SEA for torque measurement pur¬ 
poses. The hydraulically actuated humanoid. Atlas from 
Boston Dynamics, is one of the systems that entered the 
DARPA (Defense Advanced Research Projects Agency) 
Robotics Challenge. 

Modeling Lightweight Robots 
For a robot with n viscoelastic joints, the so-called re¬ 
duced model of Spong [69.67] has become the standard 
way of modeling lightweight robots. When including 
also the presence of joint torques due to contact forces 
(on the link dynamics), we shall consider the following 
dynamic model of robots with viscoelastic joints: 

M(q)q + C(q.q)q+g(q) = Tj + T ex t, (69.10) 

B0 + tj = t. (69.11) 

The generalized coordinates are doubled since there will 
be a dynamic displacement S = 0 —q between the mo¬ 
tor positions 6 el”, as reflected through the gear ra¬ 
tios, and the link positions q. The matrix B = diag(/i, ) e 
M" Xn is the diagonal, positive definite motor inertia 
matrix. We define the elastic joint torque transmitted 
through the joints and coupling (69.10) and (69.11) 

Tj = K 1 (0-q) + B J (6-q), (69.12) 

where Kj = diagj^j,) e R' ,x " is the diagonal, positive 
definite joint stiffness matrix, and Dj = diag(Dj jl ) e 
R" Xn is the diagonal, positive semidefinite joint damp¬ 
ing matrix. The quantity tj in (69.12) is also the 
output of joint torque-sensing devices, when available. 
In many practical cases, the mechanical design of the 
transmission/reduction elements is such that one can 
neglect the joint damping, that is, Dj ~ 0. Joint elas¬ 
ticity has long been addressed for lightweight robot 
systems, however, more as an undesired consequence 
that the control has to handle [69.68]. This requires ad¬ 
vanced control techniques in order to obtain accurate, 
performant motion. For a complete overview on the 
properties of robots with joint elasticity effects, please 
refer to Chap. II. 


69.3.2 Intrinsically Flexible Design 

Recently, the class of intrinsically flexible actuators and 
robots have become increasingly popular. Inspired by 
the flexible properties in biological muscles, compli¬ 
ant joints are designed with the aim of imitating human 
or animal motions during various tasks. The main idea 
of intrinsically flexible actuation is to come closer to 
human capabilities in terms of speed and shock absorp¬ 
tion. This is not realizable with todays rigid industrial 
robots, when assuming approximately the same torque 
range or weight as humans have. By storing and releas¬ 
ing energy in the joints, one aims for improvement in 
tasks, such as running or throwing [69.69-72] where 
humans still clearly outperform robots. In Chap. 21 
and [69.73], a more in depth review of intrinsically flex¬ 
ible actuation is presented. In this chapter, we review 
only the relevant insights for placing the technology in 
the pHRI context. 

General Characteristics 

Roughly speaking, intrinsically flexible actuators can 
be divided into two categories: 

• Actuators with fixed mechanical impedance, where 
the effective joint impedance is altered via active 
control. The most well-known example is the series 
elastic actuator (SEA) [69.74], whose acronym be¬ 
came a generic term for this class of actuators. 

• Actuators where the impedance can be adjusted 
by altering mechanical joint properties, such as 
stiffness and damping. There exist various cate¬ 
gories like variable stiffness actuators (VSAs) that 
allow stiffness changes or variable impedance ac- 



Fig. 69.12 Human-robot collision in operational space, 
which is defined by the reflected flexible dynamics of the 
robot and the local contact stiffness/mass properties of the 
human head 
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tuators (VIAs) that enable more general impedance 

changes, including damping adjustment. 

The original motivation for introducing VS A and 
VIA, for example, in [69.20], was to make robots safer 
during unforeseen collisions due to dynamic decoupling 
of the motor and link-side inertia from each other. This 
effect reduces the collision danger by alleviating the 
impacting robot inertia. In [69.20], it is, for example, 
shown that HIC could be reduced by introducing elastic¬ 
ity in the joint. This idea was generalized and systemat¬ 
ically analyzed in [69.75, 76], and is summarized here. 

In order to simplify the human robot collision anal¬ 
ysis basic models in operational space coordinates were 
used for this analysis (Fig. 69.12). For this, operational 
space coordinates and reflected inertias/stiffnesses 
along an arbitrary instantaneous impact direction u are 
considered [69.46]. The masses Me, m u , B x e are 
the reflected human, link, and motor mass. Ka,Kj tX e 
R“*~ are the reflected (human) contact stiffness and 
joint/structural elasticity. The projected human, motor, 
and link impact position are denoted by xu,xg, and 
x q e R, respectively, xg = p u (T(0)) andx q = p„(T(q)) 
are defined via the respective forward kinematics maps 
projected in the M-direction. 

As pointed out in [69.20], a robot with quite low 
reflected link inertia m u = 0.1 kg is able to reduce the 
impact forces significantly if a contact stiffness of K H = 
5kN/m is assumed. Similar to the work in [69.21], it 
was shown that a decrease in joint stiffness can signif¬ 
icantly reduce the impact characteristics and, thus, is 
a powerful countermeasure against large contact forces. 
In [69.77], it was deduced that for the case of a 2- 
DOF (degree of freedom) planar intrinsically compliant 
robot, already slightly touching a rigid wall with its 
second link, the compliant mechanism can limit the 
maximum static force/torque effectively if the motor 
torque is slowly increased. It is, of course, unquestion¬ 
able that joint elasticity decouples the motor from the 
link. However, as was indicated in [69.22], a reduction 
in joint stiffness cannot reduce the impact characteris¬ 
tics during very rigid, fast, and blunt crash-test dummy 
impacts for a lightweight system, such as the LWR-III 
(which is basically an SEAs type robot from a mod¬ 
eling perspective). This was proven by measuring the 
decoupling of motor and link inertia via the integrated 
joint torque sensors and the additionally recorded exter¬ 
nal contact force. 

Figure 69.13 depicts the experimental evidence for 
a collision at 1 m/s between a DLR LWR-III and a sta¬ 
tionary unconstrained Hybrid III dummy (Hill) head 
on the frontal area [69.22]. The contact force/ ext was 
measured with a high-speed force sensor/ ext ,f s and for 
consistency check with a triaxial accelerometer / extjas . 


As one can see, the simulation and experimental sig¬ 
nals show very good consistency. Clearly, the projected 
elastic joint force /j reacts delayed to the collision, 
thus proving the desired decoupling property already 
for such an intrinsically very stiff robot. For the im¬ 
pact simulation, a reflected stiffness of 6.72 xl0 4 N/m 
is chosen, which represents a realistic joint stiffness and 
structural elasticity value. The reflected motor inertia is 
B x = 13 kg and the reflected link side inertia m u = 2 kg. 
The dummy head mass is Mu = 4.5 kg and the contact 
stiffness is Ku = 3.2 xlO 5 N/m. 

This result showed that already the compliances of 
the built in gear and the joint torque sensor is suffi¬ 
cient to decouple the motor from the link, making it 
entirely unnecessary to further reduce joint stiffness for 
the given robot for this purpose. There are two main as¬ 
pects, which have to be considered to fully understand 
this result. On the one hand, the contact stiffness of the 
crash-test dummy used is significantly larger than the 
reflected elasticity of the DLR LWR-III; however, it is 
also realistic for the human frontal area [69.27]. Fur¬ 
thermore, the reflected motor and link inertia for the 
DLR LWR-III are the same order of magnitude as for 
the human head mass. This is quite reasonable for a full- 
scale lightweight robot arm. 

Two further benefits of inherent compliance are 
as follows. First, the robot itself is able to resem¬ 
ble compliant behavior without the inherent need for 
high-performance force/torque feedback. Please note 
that it is not possible to display arbitrary Cartesian 
stiffness behavior for a robot with diagonal stiffness 
matrix only [69.79]. Second, the joint elasticity acts in 
combination with the link inertia as a mechanical low- 



Fig. 69.13 Impact experiment and simulation of a LWR- 
III with a Hybrid III Dummy. The experimental contact 
force is measured for consistency check with a high-speed 
acceleration (as) and force sensor (fs) simultaneously 
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Fig.69.14a-c Passively flexible anthropomorphic robots: compliant humanoid platform (COMAN, (a)) from IIT (af¬ 
ter [69.78]), Valkyrie (b) from NASA, and hand arm system (HASY, (c)) from DLR (courtesy of IIT, DARPA, 
DLR) 


pass filter, thus protecting the drive train from impact 
shocks; that is, it makes the system inherently more ro¬ 
bust [69.71,80], 

Another interesting use of elasticities in the drive 
train of a manipulator relates to the fact that it can serve 
as a storage mechanism for potential energy, and be 
used to increase a robot’s link speed beyond the max¬ 
imum motor velocity [69.70,71,80-82], This ability 
strongly influences the safety of intrinsically elastic sys¬ 
tems, as a robot’s impact speed determines its inherent 
collision danger to a large extent. However, we refer 
to [69.75] for further details on this matter, where the 
effect of a robot’s elastic speed increase on potential 
danger in terms of HIC is derived. 

Intrinsically Flexible Robotic Systems 
Figure 69.14 shows some of the most prominent recent 
anthropomorphic/humanoid devices that employ pas¬ 
sively compliant actuators. The COMAN system of IIT 
(l<et«ii]»iw) is a full-scale SEA robot, as is NASA’s 
Valkyrie. Both system’s ability to measure torques re¬ 
sults from the ability to flex around their joint axes and 
then relate this elastic deflection to the joint torque. The 
DLR hand-arm system [69.72] is fully equipped with 
VS A in every joint, while having approximately human 
size, strength, and dexterity. 

Modeling Passively Flexible Robots 
From a formal modeling point of view, lightweight 
robots and robots with fixed mechanical impedance are 


very similar. The main difference lies in the respec¬ 
tive torque sensor measurement principle. The joint 
elasticity in lightweight robots is rather high and al¬ 
lows only for small deflections between the motor and 
link position. The stiffness in compliant actuators, how¬ 
ever, is intentional and typically at least an order of 
magnitude lower, thus enabling significantly larger de¬ 
flections. Consider the elastic joint model (69.10) and 
(69.11) from Sect. 69.3.1, which under certain mild 
conditions also holds for intrinsically elastic robots. The 
main difference is that for lightweight robots, Kjj is 
large compared to SEA-like systems. For many VSA 
systems, the transmitted torque can be expressed as 

tj =f(8.a), (69.13) 

where a denotes a stiffness adjustment control in¬ 
put (typically the position of a second motor). 
The torque-displacement curve /'(. a ) may exhibit 
different characteristics, such as being strictly in¬ 
creasing df(S,a)/dS > 0 and convex d 2 f(S,a)/dS 2 > 
0. More details on implementations can be found 
in [69.73]. 

69.3.3 Perception for Interaction 

The recent advances in pHRI benefit greatly from previ¬ 
ous achievements in contact and noncontact sensors and 
sensing techniques. In the current chapter, we shortly 
review the essential sensing techniques that had strong 
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Fig. 69.15 (a) Furry robot skin utilizing piezo-resistive fabric and conductive sensors provides a low cost design concept 
with high tactile gesture recognitions rates (after [69.83]). (b) The hand of TWENDY-ONE with integrated distributed 
pressure sensors (courtesy of Karon MacLean, UBC, Shigeki Sugano Lab., Waseda University) 


impact on pHRI and focus on the underlying main con¬ 
cepts. Chapters 28 and 32 give a full view and can be 
consulted for further details. 

69.3.4 Proprioceptive Force/ 

Torque Sensing 

Many robot arms have the option to include a six- 
axis force/torque sensor at the wrist, enabling not only 
force control-based manipulation, but also symbolic 
haptic interaction [69.84, 85], possibly also based on 
joint torque sensing. More recently, torque sensors 
have been integrated into the joints of commercial 
robots, for example, into the KUKA LWR [69.86]. 
This allows for contact sensing along the entire robot 
structure with measurements of contact magnitude, 
direction, and knowledge of which link was con¬ 
tacted [69.87]. 

Essentially, two major sensing principles for joint 
torque measurement exist: either by directly measuring 
the torque which is typically done via strain gauges, 
or by measuring it implicitly via the deflections 5 
between link and motor position. For this, it is as¬ 
sumed that S directly relates to torque typically by 
a constant multiplier, the joint stiffness Kjj. For the 
latter, both link- and motor-side position sensors need 
to be mounted. The first works that aimed at devel¬ 
oping and minimizing torque-sensing devices can be 
found in [69.88-90]. The torque-sensing principle has 
also a strong relation to the design classification in 
Sect. 69.3. 


69.3.5 Tactile Perception 

Several tactile skins have been developed over the years 
such as, for example, [69.91-94]. In the latter work, for 
example, the human skin served as a design metaphor. 
Therein, a solution of the goal conflict between the 
desired high sensitivity and the required mechanical ro¬ 
bustness is in the focus. Furthermore, beyond providing 
the required sensory capability, the mechanical defor¬ 
mation and damping properties necessary for the opera¬ 
tion of the robotic system are considered as well. Using 
whole-arm tactile sensing across the manipulator sur¬ 
face explicitly allows the sensing of multiple contacts; 
for example, in [69.95], the authors operate a robot in 
a cluttered environment in order to successfully reach 
a goal that is occluded by obstacles using a model pre¬ 
dictive controller. Other recent examples of localized 
sensors include touch pads placed at key points on the 
robot [69.96], and force-sensitive resistors distributed 
over the robot [69.97]. The latter example is used in 
a novel haptic robot designed to both read and dis¬ 
play emotion mediated through cues sensed primarily 
through physical interaction. These methods have the 
advantage of using reliable and fairly precise sensors, 
but are capable of measuring contact only at discrete 
locations. For more extensive contact sensing, several 
researchers have developed various types of robot skin; 
that is a distributed sensor network that covers extensive 
areas and measures diverse qualities of contact events, 
including location, magnitude, orientation, and temper¬ 
ature. Hard robot skins or shells have been used to 
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Fig. 69.16 Low cost 3-D-RGB-D cameras: (a) Microsoft Kinect (Wikipedia Commons), (b) Asus Xtion (courtesy Asus) 


detect collisions [69.98], while soft robot skins that can 
conform to different robot shapes have been used for 
more complex, often social or affective, human-robot 
interaction, for example, [69.99]. While robot skin can 
add significant sensing capability to robots, it may also 
add to the complexity and cost, and thus compromises 
the robustness of the robot. However, recently a furry 
creature skin (Fig. 69.15) was created [69.83] using 
a combination of low-resolution piezo-resistive fabric, 
and conductive fur sensors to create a low-cost robot 
covering with 90% interaction gesture (stroking, pat¬ 
ting, poking, hitting) recognition rates when trained 
on an individual and 68—80% recognition rates when 
trained on groups (I^MSEESEIl). A detailed review of 
tactile sensing technology for human-robot interaction 
can be found in [69.100]. Other examples can be found 
in Chap 28. 


69.3.6 Visual Perception 

Visual (noncontact) sensing is very important for 
preparing for pHRI. Tracking and planning for the 
location of a human partner and predicting, for exam¬ 
ple, the motion of the partner’s hand is an important 
precursor to a successful handover operations [69.12]. 
While well-known marker-based systems, such as Vi- 
con and Optotrak systems, provide very high resolu¬ 
tion tracking systems, they are impractical for day- 
to-day use. The development of low-cost 3-D-RG- 
B-D (three-dimensional red green blue depth) cameras 
(Fig. 69.16) permits the 3-D tracking of full body 
models in large and partially occluded spaces, with on¬ 
going improvements toward robustness in body pose 
tracking and for tracking of hand gestures [69.101, 
102], 


69.4 Control for Physical Interaction 

For soft and safe pHRI, the question arises how to gen¬ 
tly handle physical contact in robotics from a control 
point of view. As impedance control [69.103] became 
the most popular interaction control paradigm in the 
pHRI world, this particular scheme will be one fo¬ 
cus of this section. Its generalization to multipriority 
impedance control laws allows the realization of sophis¬ 
ticated robot compliance with multiple objectives via 
active control. A major advantage of impedance con¬ 
trol is that discontinuities like contact-noncontact do 
not create stability problems as they occur, for exam¬ 
ple, with hybrid force control [69.104]. Its extension to 
impedance and feed-forward learning and adaptation, 
for which the first works can be found in [69.105, 106], 
is discussed after introducing the concept of multiprior¬ 
ity impedance control. Apart from nominal interaction 
control, a robot sharing its workspace with humans 
and physically interacting with its environment should 
be able to quickly detect collisions and safely react 
to them. In the absence of external sensing, relative 


motions between robot and environment/human are un¬ 
predictable and unexpected collisions may occur at 
any location along the robot arm. The state-of-the-art 
schemes for collision detection and reflex reaction are 
introduced. Finally, the last part of the section deals 
with the shared manipulation problem, one of the stan¬ 
dard application examples of pHRI. 

69.4.1 Interaction Control 

Originally developed for robust and compliant object 
manipulation, impedance and the related admittance 
control (I^M'jl'KT-il'l) form a paradigm to treat robotic 
systems from an energetic point of view such that 
motion and force can be controlled in a unified man¬ 
ner. They offer the advantage over hybrid force-motion 
controllers to provide a framework independent from 
kinematic work space constraints. These control types 
popularized by [69.103] are also especially advanta¬ 
geous in terms of uncertainties and disturbances in 
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unknown environments due to their inherently robust 
nature [69.107]. The terms impedance and admittance, 
are derived from electrical system theory where they de¬ 
scribe the relationship between voltage and current as 
input/output pairs. To generalize impedance and admit¬ 
tance such pairs can be defined domain-independently 
as effort and flow variables. For robotics, the me¬ 
chanical analogies, that is, mechanical impedance and 
admittance are of particular interest. 

More details on the conceptual basics of impedance 
and the dual admittance control can, for example, be 
found in [69.108,109]. Furthermore, Chap. 9 gives 
a more thorough basis on force control strategies in gen¬ 
eral and impedance control in particular. 

The mostly used version of impedance control is 
to impose a second-order dynamics of a mass-spring- 
damper system (so-called target impedance [69. 103]) on 
the closed-loop equations. Typically, the control objec¬ 
tive is expressed in operational space coordinates x as 

M v x + D r x + K v x = _y ext , (69.14) 

where x := x — jcj is the position error and x ( | is called 
equilibrium position. M. v denotes the desired inertia, 
while D A and K, are the according closed-loop damping 
and stiffness matrices in operational space. jF ext denotes 
the external wrench acting at the end-effector of the 
robot. Assuming rigid body dynamics, the control law 
to obtain the aforementioned behavior is 

rc* =g(q) + J(«r) T [A(?)id + /<-(i.^)] 

+ (69.15) 

In order to fully implement this scheme, a wrist force 
torque sensor is necessary for the inertia shaping part. 
In [69.110], a modified impedance controller was de¬ 
signed that uses angle/axis representations for the ro¬ 
tational components of the operational space. For its 
derivation, energy contributions with physical interpre¬ 
tation are considered and the end-effector orientation 
displacement representation is chosen to be in terms of 
a unit quaternion to avoid singularities. 

For redundant robots, it is typically desired to also 
control the nullspace behavior in order to embed other 
control objectives r Nii into a stacked hierarchy of tasks 
(Chaps. 17 and 36). For the case of a single nullspace 
controller r N , this torque has to be projected via the 
nullspace projector matrix N (<7 ) into the nullspace of 
the task, leading to the overall control law 

Tc = T C * + N(q) T r N . (69.16) 

The nullspace projection matrix can be chosen in dif¬ 
ferent ways. The simplest case is N (q) = \ — j(q) # i(q), 


where J(^) # denotes the Moore-Penrose pseudoinverse. 
Alternatively, one may chose the dynamically consis¬ 
tent generalized pseudoinverse 

J( 9 ) # = M(< 7 )- 1 J( 9 ) t A( 9 ). (69.17) 

In particular, in the pHRI domain, a multitude of dif¬ 
ferent subtasks r Wi , are meaningful to be executed 
simultaneously. These may, for example, involve: 

• Safety (collision anticipation and avoidance, self¬ 
collision avoidance, ...) 

• Physical constraints (joint limits, geometric task 

constraints) 

• Task execution (tracking control, ...) 

• Posture primitives (in particular for humanoids). 

To realize consistent behaviors, task hierarchies are 
constructed such that certain tasks are prioritized over 
others [69.111]. In [69.112], a hierarchy is realized 
by null space projection techniques, which also pre¬ 
vents discontinuities concerning unilateral constraints 
by smoothing out transitions. 

Extensions to the basic schemes for flexible joint 
dynamics [69.113-115] and for the SEA case [69.116] 
were developed as well. Furthermore, Cartesian imped¬ 
ance control has been applied to grasping and multiple- 
arm robotic systems in [69.117]. For a more in depth 
treatment of multipriority impedance control, please 
also refer to Chap. 67. 

69.4.2 Learning and Adaptation 

Close physical interaction between human and robot 
is a complex evolving process with high uncertainty 
and is hard to be modeled explicitly. Therefore, sev¬ 
eral learning and adaptation approaches were proposed 
to enhance a robot’s capability and account for the 
inherently present uncertainty and unpredictability. In 
this sense, the extension of impedance control toward 
adaptive controllers that are able to learn and/or adapt 
the controller’s impedance and feed forward torques is 
a challenging and rather recent research problem. Ob¬ 
viously, also the learning of the desired trajectories, 
for example, in terms of motion patterns became an 
important problem in this context. Generally, also col¬ 
laborative tasks (Sect. 69.4.4) require learning either the 
impedance properties and/or certain trajectories such 
that the robot gains the ability to adapt its behavior to 
the human counterpart, or even guides it to guarantee 
successful collaboration. An important aspect of learn¬ 
ing physical interactions is how to choose the right task 
coordinates and (meta-)parameters. This is essential to 
reformulate the otherwise high-dimensional problem 
spaces in a tractable form. 
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Since impedance control has already shown to be 
a valuable technique in cluttered and complex manip¬ 
ulation tasks, recent research focuses on the adaption 
of the impedance characteristics to further improve the 
robot capabilities during interaction. Iterative learning 
control techniques belong to the first methods that 
have been investigated to tackle difficult manipulation 
problems [69.118,119]. However, they do not only 
require the exact same repetitive motion of the manip¬ 
ulator, but also to account for unforeseen changes in 
the environment due to force inconsistencies which are 
particularly present in pHRI. Other early approaches 
include, for example, the use of neural networks in 
impedance control to counteract disturbances and envi¬ 
ronmental uncertainties [69.120]. An approach to adapt 
force, trajectory, and impedance simultaneously has 
been presented in [69.121] as a biomimetic controller. 
It is constructed based on studies in neuroscience that 
show that humans adapt feedfoward and feedback 
forces as well as their impedance in order to learn 
unstable tasks in daily life. According to [69.122], the 
principles of motor learning are: 

1. Motor commands to perform a desired action are 
composed of both feedforward commands, defined 
as the component of the motor command learned by 
repeating an activity, and feedback commands. 

2. Learning is performed in muscle space. 

3. Feedforward increases with the muscle stretch in 
previous trial. 

4. Feedforward also increases with antagonist muscle 
stretch. 

5. Feedforward decreases when the error is small. 

Mathematically, these principles can be expressed 
as follows [69.105]. Generally, the central nervous sys¬ 
tem tends to minimize the motion error as well as the 
metabolic cost such that no extra effort will be spent on 
the learned impedance and feedforward torques. This 
can be expressed by minimizing the joint-level cost 
function 

t 

V(t)=\ I 4>\a)Q- l 4>(a)da 

t—T 

+ i<F(0 T M(<jr)e(f). (69.18) 

Therein, Q is a positive definite weighting matrix cor¬ 
responding to the learning rates; 0 is defined as the 
difference between the instantaneous value 

0 = [vec(K fy (f)) T , vec(D ? (f)) T , r ff (t) 1 ]' (69.19) 

and the required optimal value 0 * , that is, 0 = 0 — 
0* . The quantities e, K ? , D ? , and Tff denote the track¬ 


ing control error [69.123] and the values to be learned, 
namely, the closed loop joint space stiffness, damping, 
and feed forward motor torques. For the joint stiffness 
adaption law, this leads for example to 

8K,(f) = K,(f)-K,(f-r) 

= Qk, [f0M0 T - y0) K ?0)]. (69.20) 

where y(t) > 0 is a constant forgetting factor and Q K , 
which is contained in Q, is a symmetric positive definite 
matrix corresponding to the learning rate of the stiff¬ 
ness. T is a time constant to denote task: periodicity. In 
an analog manner, adaption for feedforward torque and 
damping can be deduced, which results in the following 
control law, 

t C ( 0 = r ffM “ K 9 (f>(0 - D q (t)e(t) 

— L(f)e(?) + T r (f) (69.21) 

e = e(t) + Ke(t) , (69.22) 

where T r (f) compensates for robot/arm dynamics and 
bounded noise. The term L(f)e(f) ensures a certain sta¬ 
bility margin. The overall control law flow chart is 
depicted in Fig. 69.17. A stability proof of this approach 
can be found in [69.105]. An open problem with this 
method that remains to be unsolved is how to automat¬ 
ically select the meta parameters, such as the forgetting 
factor y. A similar approach was developed in [69.124], 
where impedance adaptation laws were tested on a two- 
robot collaboration task. The robots lift together a beam 
after a task model was learned. A different approach 
based on the perturbations of the robot imposed kines- 
thetically by a human teacher such that the stiffness of 
the robot is adapted is investigated in [69.125]. Finally, 
the so-called teleimpedance paradigm [69.126] aims to 
remotely controlling the robot via the human arm refer¬ 
ence position and impedance. 

Learning task trajectories has been a rather well- 
studied field for several years now. For example, 
in [69.127], the authors examined the task of learning 
collaborative lifting of an object using hidden Markov 
models (HMM) and Gaussian mixture regression. Task 
demonstration was done via a haptic interface that con¬ 
trolled the robot hand. HMMs are also used in [69.128] 
in order to learn semantic task structures during a joint 
task execution. A semantic label of recognized task seg¬ 
ments is acquired from a human partner by using speech 
recognition. Tasks in pHRI where transitions from con¬ 
tact to noncontact situations arise have been examined 
in [69.129]. Therein, marker data is used to obtain mo¬ 
tion primitives. These are then updated with contact 
time information, comprising a low-level layer of learn¬ 
ing. In addition, there is a high-level layer, which selects 
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Fig. 69.17 Control diagram 
of the biomimetic adaptive 
impedance controller from 
(after [69.105]) 


a motion primitive and an end-effector reference po¬ 
sition. Again, both layers are encoded via HMMs to 
provide a suitable abstraction, which is a key when 
dealing with high-dimensional spaces. 

In [69.17], dimensionality reduction for learning 
was employed to realize assistive tasks, such as helping 
a robot to stand-up or walk. This principal compo¬ 
nent analysis (PCA)-based reduction lets the learning 
take place in a low-dimensional joint manifold instead 
of the original high-dimensional space. It reduces the 
complexity of the learning domain for multidegree-of- 
freedom systems such as humanoids. A recent approach 
to learning trajectories based on human preferences was 
proposed in [69.130], where machine-learning tech¬ 
niques were applied to derive the optimal behavior 
from human action ratings during runtime. Figure 69.18 
shows how the robot learns to handle a knife safer 
by receiving low rewards from the human if the dan¬ 
gerous tool is moved in undesired orientations and/or 
trajectories. 

69.4.3 Collision Handling 

One of the core problems in pHRI is the handling of 
collisions between robots and humans, with the pri¬ 
mary motivation of limiting possible human injury due 
to physical contacts. Various monitoring signals can be 


used to gather context independent information about 
the event. 

The collision detection phase, whose binary output 
denotes whether a robot collision occurred or not, is 
characterized by the transmission of contact wrenches, 
often for very short impact durations. The occurrence 
of a collision, which may happen anywhere along the 
robot structure, shall be detected as fast as possible. 
A major practical problem is the selection of a threshold 
on the monitoring signals, so as to avoid false positives 
and achieve high sensitivity at the same time. A heuris¬ 
tic approach is to monitor the measured currents in 
robot electrical drives, looking for fast transients pos¬ 
sibly caused by a collision [69.131, 132], Another pro¬ 
posed scheme compares the actual commanded torques 
(or motor currents) with the nominal model-based con¬ 
trol law (i. e., the instantaneous torque expected in the 
absence of collision), with any difference being at¬ 
tributed to a collision [69.133]. This idea has been 
refined by considering the use of an adaptive compli¬ 
ance control [69. 134, 135]. However, tuning of collision 
detection thresholds in these schemes is difficult be¬ 
cause of the highly varying dynamic characteristics of 
the control torques. 

Knowing which robot part (e.g., which link of a se¬ 
rial manipulator) is involved in the collision is an 
important information that can be exploited for robot 



Fig. 69.18 The robot Baxter learns to safely handle a knife ([69.130]; courtesy of Saxena’s Robot Learning Lab) 
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reaction. Collision isolation aims at localizing the con¬ 
tact point x c , or at least which link i c out of the n-body 
robot collided. One way to obtain both collision detec¬ 
tion and isolation is to use sensitive skins [69.91-94]. 
However, it would obviously be more practical and 
reliable to detect and possibly isolate a collision with¬ 
out the need of additional tactile sensors. On the other 
hand, the previously mentioned monitoring signals used 
in [69.131-135] are in general not able to achieve reli¬ 
able collision isolation (even when robot dynamics is 
perfectly known). In fact, they either rely on computa¬ 
tions based only on the nominal desired trajectory, or 
compute joint accelerations by inverting the mass ma¬ 
trix and thus spreading the dynamic effects of collision 
on a single link, or use acceleration estimates for torque 
prediction and comparison, which inherently introduces 
noise (due to double numerical differentiation of posi¬ 
tion data) and intrinsic delays. The common drawback 
of these methods is that the effect of a collision on a link 
propagates to other link variables or joint commands 
due to robot dynamic couplings, thus affecting the iso¬ 
lation property. 

Other relevant quantities about a collision that are 
deduced during the collision identification phase are the 
directional information and the intensity of the general¬ 
ized collision force, either in terms of the acting Carte¬ 
sian wrench T emit) at the contact, or of the resulting 
joint torque t ext (?) during the entire physical interaction 
event. This information characterizes (in some cases, 
completely) the collision event. The first method that 
simultaneously achieved collision detection, isolation, 
and identification was proposed in [69.136]. The basic 
idea was to view collisions as faulty behaviors of the 
robot actuating system, while the detector design took 
advantage of the decoupling property of the robot gen¬ 
eralized momentump = M (q)q [69.137, 138]. 

During the collision reaction phase, the robot 
should react purposefully in response to a collision 
event, that is, taking into account available contextual 
information. Because of the fast dynamics and high un¬ 
certainty of the problem, the robot reaction should be 
embedded in the lowest control level. For instance, the 
simplest reaction to a collision is to stop the robot. How¬ 
ever, this may possibly lead to inconvenient situations, 
where the robot is unnaturally constraining or blocking 
the human [69.19]. To define better reaction strategies, 
information from collision isolation, identification and 
classification phases should be used. Some examples of 
successful collision reaction strategies have been given 
in [69.87, 139]. 

Collision Detection and Identification 
A recent overview on standard techniques to esti¬ 
mate r ext can be found in [69.26]. In this chapter, 


we focus on the main method, namely the monitoring 
method based on the observation of the generalized mo¬ 
mentum that was introduced in [69.136]. The scheme, 
which is regarded as the standard algorithm, was mo¬ 
tivated by the desire of avoiding the inversion of the 
robot inertia matrix, decoupling the estimation result, 
and also eliminating the need of an estimate of joint ac¬ 
celerations. The according disturbance observer-based 
estimator for the rigid case is defined as 

r(t) =K 0 jp(?)- f [T-/J(^)+r]d.y-p(0)j , 

(69.23) 

with p = M(q)q, P(q,q) = g(q) + C(q. q)q-M(q)q, 
and Ko = diag(ko,;) > 0 being the diagonal gain ma¬ 
trix of the observer. Under ideal conditions, M = M 
and [i = (I, the dynamic relation between the external 
torque T ex t and r is 

f = Ko(r ex t —r) . (69.24) 

In other words, r is a stable, linear, decoupled, first- 
order estimation of the external collision torque T ext . 
Large values of koj give small time constants Tqj = 
l/koj in the transient response of that component of 
r, which is associated with the same component of the 
external joint torque r ex t- In the limit, we obtain 

K 0 — 5 - 00 => r ss Text ■ (69.25) 

Collision Reflex Reactions 
After a collision has been detected, suitable collision re¬ 
flex reaction is needed. Four basic context-independent 
joint-level collision reflexes are discussed next. They 
lead to significantly different reflex behavior after 
a contact was detected. In the third and fourth schemes, 
the directional information on contact torques provided 
by suitable identification schemes such as (69.23) may 
be used to safely drive the robot away from the collision 
location. 

Robot Stop. The most obvious strategy to react to 
a collision is to stop the robot. This behavior can, for ex¬ 
ample, be obtained by setting q,\ = q(t c ), where t c is the 
instant of collision detection or by simply engaging the 
robot’s brakes. More elaborate braking strategies can be 
found in [69.140]. 

Torque Control with Gravity Compensation. One 
may also react to a collision by switching the con¬ 
trollers. Typically, prior to the collision incident, the 
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robot moves along a desired trajectory with a position 
reference-based controller (e.g., position or impedance 
control). After detection, the control mode is switched 
to a compliance-based controller that ignores the pre¬ 
vious task trajectory. A particularly useful variant is to 
switch to torque control mode with gravity compensa¬ 
tion r = g(q) Note that this strategy 

does not explicitly take into account any information 
about Text- 

Torque Reflex. This strategy extends the torque con¬ 
trol-based strategy by explicitly incorporating the esti¬ 
mation or measurement of r ex t into the motor torque r 
via 

r =g(q) + (I-K r )r ext , (69.26) 

where K r = diag{£ r ,;} > 1. It can be shown that, under 
sufficiently accurate estimates or measurements, such 
a law is equivalent to scaling of the robot dynamics by 
K~*. The closed-loop dynamics become 

KJT'Mfa) q + K~ 1 C(q,q)q+r ext = 0 , (69.27) 

M'( ? ) 

where M(#) > M'(^r) holds component-wise. 

Admittance Reflex. Reference trajectory modifica¬ 
tion via an admittance-type strategy that uses the mea¬ 
surement or estimation of T ex t can easily, for example, 
be realized via 

T 

< 7 d (0 = ~ J K a r ext dr, (69.28) 

tc 

where K a = diag{£ ai } > 1. With this scheme that re¬ 
quires no control switching the robot quickly drives 
away from the external torque source and decreases the 
contact forces till they decay to zero. 

69.4.4 Shared Manipulation Control 

Collaborative carrying, particularly of a long, large, 
heavy or flexible object, is a common scenario in pHRI 
research (Fig. 69.19). As discussed earlier, Cobots rep¬ 
resent the parallel case where passive robotic devices 
control the path along which a shared load will be trans¬ 
ported, but give their operator full control of load mo¬ 
tion along that path (Chaps. 39 and 70; 

Most shared manipulation schemes utilize some form 
of impedance control [69.142]. In an early work, the 
authors of [69.143,144] proposed an impedance con¬ 
troller with speed-dependent damping coefficients. The 


authors of [69.145] employed a similar approach (with 
fixed virtual impedance) to control the horizontal move¬ 
ment of their Mobile Robot Helper’s mobile base in 
response to applied forces of a human user on the other 
end of a cooperatively carried load (loiKililil&tM). The 
authors of [69.144] also described a lifting controller 
employing a pair of cascaded second-order virtual ad¬ 
mittance controllers. Due to the raising/lowering of 
a cooperatively carried load the applied torques on 
an admittance-controlled, high-stiffness wrist generate 
a wrist deflection that translated (via a fixed gain) 
into a virtual vertical force. This force raised or low¬ 
ered the robot end effector via a second admittance 
controller. In [69.141], an admittance controller was 
presented that was tuned for human preference, which 
is typically slightly underdamped. In [69.146], an ad¬ 
mittance strategy was used to translate user force input 
into robot steering commands within a constrained 
trajectory. 

Ideally, in cooperative manipulation, robotic and 
human partners will naturally take turns with leading 
and following roles depending on the state of a shared 
task. A switching model for haptically linked human- 
robot pairs that allows the robot to continuously vary its 
behavior from completely following to completely lead¬ 
ing was introduced in [69.16]. In [69.147], the authors 
presented a mathematical treatment of the cooperative 
load manipulation problem to allow one or more robots 
to carry a load with a human user along a desired trajec¬ 
tory. They also incorporated variable leading behavior 
in the robots, enabling them to just steer the load (fol¬ 
lowing) or completely control the load’s axial motion 
(leading). 

More recently, Evrard’s homotopy approach was 
applied to load lifting, enabling the robot assistant to 
vary its behavior between leading and following based 
on its confidence in its predictions of the human user’s 



Fig. 69.19 Collaborative lifting experiment with a long 
object (after [69.141]) 
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Fig. 69.20 

Generalized 
schematic for 
collaborative task 
control 


intentions [69.15] (l^s&JEEEEEI). These are monitored 
through the kinematics of the shared load. Recently, 
an admittance control law that guarantees the stability 
of the robot during constrained motion and also pro¬ 
vides a quite intuitive human interaction was designed 
in [69.148]. The admittance law uses the time deriva¬ 
tive of the contact force between the human and the 
robot to estimate human intent and an online estimate 
of the interaction stiffness resulting in very accurate 
shared-control of the robot system. In [69.149], the au¬ 
thors developed and user tested an interactive controller 


for a collaborative carrying task, utilizing a strategy to 
model effort sharing between the robot and the user. 
A generalized schematic for admittance-based collabo¬ 
rative task control is depicted in Fig. 69.20. On the mo¬ 
tion planning side, minimum jerk trajectories [69.150] 
have been used to generate robot trajectories that are 
well matched to the motion of the human partner, and 
require the human to use less energy in the interaction, 
for example, [69.151]. Results from [69.152] indicate 
that a simpler quintic trajectory was also suitable for 
this purpose. 


69.5 Motion Planning for Human Environments 


The definition and quantification of injury, pain, or gen¬ 
eral risk are essential to express what safe behavior 
really means. The according insights can also be ap¬ 
plied to generate safer robot motions such that injury 
and risk prevention are explicitly taken into account at 
this level. Two main branches of motion planning algo¬ 
rithms for danger reduction were developed, which are 
described hereafter. 

69.5.1 Biomechanically Safe 
Motion Planning 

As mentioned in the beginning of the chapter, hu¬ 
man collision safety has to be ensured from an injury 
biomechanics perspective. Safety could, for example, 
be defined as ensuring that only mild contusions may 
occur in worst case scenarios. The natural question that 
arises is now how to formally respect such a metric, 
that is, how should a robot be controlled so that an un¬ 
foreseen contact remains subcritical according to the 
underlying injury biomechanics or pain data. For this 
the authors of [69.25] gave the schema, termed safe 
motion unit (SMU), to link basic data to intrinsically 
safe robot velocity. The basic idea is to represent any 
data or general insight in a (possibly purely data driven) 
functional relation, linking impact properties to human 


injury or pain. With this, it is possible to calculate the 
instantaneous safe robot velocity given the robot’s in¬ 
ertial properties, surface curvature, and possible human 
body parts it might collide with 

(mass, velocity, geometry, body part) 

—> observed injury/pain . (69.29) 

For the injury/pain metric, multiple international clas¬ 
sifications exist from which one may choose from. 
In [69.25], for example, the AO-classification was used. 
Thereafter, a careful set of experiments, or if possible 
simulations, can be generated to deduce the relation 
of interest. Each experiment should be designed such 
that the impacting mass, its velocity, and curvature are 
known in addition to any supplementary sensory read¬ 
ings of interest. Furthermore, the observed, measured, 
or calculated injury and/or pain level of the involved 
body part has to be quantified according to the se¬ 
lected scale. To restrict the amount of experiments to 
a feasible number, the impactor geometry is assumed 
to be composed of different basic geometric primitives 
(spheres, cuboids, corners). Thus, a finite set of so- 
called safety curves can be constructed, which represent 
fitted threshold curves for a given injury indicator in 
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Fig. 69.21 A unified view: From collision experiment/simulation to cumulative safety cun’es for biomechanically safe 
robot velocity 


the mass-velocity space parameterized by the body part 
(Fig. 69.21). 

This idea can now be applied to a robot for safe ve¬ 
locity generation (Fig. 69.22). The concept is to scale 
any desired velocity command q d by a to a biomechan¬ 


ically safe value q d . The scalar a is computed from 
evaluating the instantaneous robot inertial and local sur¬ 
face properties with respect to their injury potential. 
For this, the effective mass m poi in the respective in¬ 
directions of the relevant points of interest (POIs) are 


Dynamic/kinematic parameters, 
incl. end-effector & loadparameters 


Environment model 
Robot model, incl. all POIs 


Effective masses 


Fig. 69.22 Pipeline for 
creating biomechanically safe 
velocities based on the SMU 
algorithm (after [69.25]) 
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evaluated via (69.7). Each POI is associated with a sur¬ 
face geometry primitive that is then used to link the 
respective POI to some underlying biomechanical in¬ 
jury, pain, or risk safety cutve. 

69.5.2 Risk-Metric-Based Motion Planning 

As discussed in Sect. 69.4.3, there are numerous tech¬ 
niques for detecting impacts or potential collisions 
during pHRI. Collision monitoring methods can be used 
to ensure that forces and torques, or more simply the en¬ 
ergy, of the robot system are limited during a collision 
event. To ensure safe and human-friendly interaction in 
unstructured environments, additional safety measures, 
utilizing system control, and planning are required. 

Since physical interaction itself is a collision, a key 
problem of human-safe planning and control methods 
is to identify when human safety is actually threatened. 
In [69.154], a danger evaluation method was developed 
using the potential impact force as an evaluation mea¬ 
sure. In this work, several danger indices were proposed 
based on the design properties of the robot. This dan¬ 
ger index can be defined as a product of factors which 
affect the potential impact force between the robot and 
the human, such as distance, relative velocity, robot iner¬ 
tia, and robot stiffness. The authors applied this index as 
an objective function for improved mechanical design, 
control, and motion planning. A danger index based 
on estimated impact forces for a potential collision be¬ 
tween a robot and a human was computed in [69.155] 
and used as an input to a real-time trajectory generation 
system, which balances the goal seeking with potential 
danger. In [69.153], the same authors combined their 
system with a vision based and physiological monitor¬ 


ing system, which allows the robot to respond to the 
user’s real-time position and attentiveness. During the 
interaction, the user is monitored to asses his level of ap¬ 
proval of the robot’s actions while the trajectory planner 
monitors safety factors, such as robot velocity and user 
intent. Finally, the safety control module provides a real¬ 
time response to short-term horizon factors evaluated by 
a safety measure estimation module (Fig. 69.23). 

For mobile service robots, the authors of [69.156] 
proposed a collision avoidance controller based on es¬ 
timates of user behavior, using a social force model 
to determine whether the user intends to avoid the 
collision or not so that the robot can respond accord¬ 
ingly. In [69.157], the authors introduced a method 
using so-called kinetostatic danger fields as a metric 
for the danger which the current posture and velocity 
of a robot pose to objects in its environment. The work 
presented in [69.121] shows an algorithm which maxi¬ 
mizes the productivity of an industrial robotic manipu¬ 
lator while guaranteeing a safe human-robot distance. 
In addition to the relative human-robot distance, it takes 
also dynamic and control characteristics into account. 
In [69.158], the authors proposed a method based on an 
intuitive physical interpretation, namely, an impedance¬ 
like second-order motion generation. It is designed to 
provide safe motion in complex environments, taking 
into account both proximity to objects, and external 
forces. A method that achieves collision avoidance for 
a redundant robot while permitting the end-effector task 
to continue is presented in [69.159]. Using data from 
a 3-D-RGB-D camera, this method is purely based on 
computations of distances between the robot body and 
dynamic obstacles in the workspace (e.g., the user) that 
are processed through a risk function to adjust the joint 



Fig. 69.23 Schematic of motion planning system with user intent and safety monitoring input (after [69.153]) 
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Fig. 69.24 Human-aware motion planner of Laboratoire 
d’analyse et d’architectures des systemes (LAAS) (af¬ 
ter [69.160]; courtesy of Rachid Alami) 


velocities. The motion task of the end effector is modi¬ 
fied by an artificial potential field-type method. 


69.6 Interaction Planning 

In order to profit from the collaboration of human and 
robot by combining the flexibility, knowledge and sen¬ 
sory skills of a human with the efficiency, strength, 
endurance, and accuracy of a robot, according inter¬ 
action planners need to be designed to plan their joint 
actions for a common goal. In the interaction planning 
domain, the central question is how to plan robot human 
joint actions and reactions in a certain interaction pro¬ 
cess, involving also unexpected environmental changes, 
one of the most basic ones being the human entering 
the robot’s workspace. The definition of the interaction 
planning problem and the integration of reflex reac¬ 
tion schemes that potentially lead to an abrupt deviation 
from nominal course have to be elaborated. On an ar¬ 
chitectural level, the incorporation of reactivity has to 
be systematically represented. 

69.6.1 Collaborative Action Planning 

Approaches to subproblems of the full interaction 
planning problem were addressed in the literature. 
In [69.163], for example, the system SHARY for 
human-aware task planning was introduced. It produces 
social plans of a task by implementing communication 
schemes to negotiate the task solution with the human 
partner. 

In [69.164], dynamic neural fields (DNFs) were 
used to build a decision-making system for interac¬ 


69.5.3 Human-Aware Motion Planning 

Applying and extending classical motion planning tech¬ 
niques to the problem of human-robot interaction was 
originally done in [69.160]. The authors developed 
a human aware mobile robot motion planner, which 
incorporates humans accessibility, their vision field, 
and their preferences in terms of relative human-robot 
placement (Fig. 69.24). Human dynamics were inte¬ 
grated into the algorithm. 

Extensions of the original work toward a motion 
planner for pHRI scenarios were elaborated in [69. 161]. 
Therein, certain constraints, such as distance, visibil¬ 
ity, and comfort are taken into account to generate safer 
motions and were demonstrated within a handover sce¬ 
nario. The algorithm has been extended to cluttered 
environments in [69.162], where a randomized cost- 
based exploration method provides an initial path that 
is relevant with respect to pHRI and workspace con¬ 
straints. 


tion in cooperative human-robot tasks. The main idea 
is to let the robot imitate human behavior in order 
to make the cooperation between robot and human 
appear rather intuitive. A structural overview of the 
system is depicted in Fig. 69.25. A vision system ob¬ 
serves the scenery and recognizes task-related objects 
and gestures of the human coactor. Object positions 
are stored in the object memory layer. The action ob¬ 
servation layer decides which type of known action 
was executed by the coactor. The expected outcome 
of this action is then simulated by the action simula¬ 
tion layer and thereafter passed to the intention layer. 
Here, the intention of the coactor is determined. The 
knowledge about the coactor’s intention and the ex¬ 
pected outcome of his actions is used by the common 
subgoal layer. It contains prior knowledge about the 
task structure and the subgoals to be achieved for task 
completion. The common subgoal layer is responsi¬ 
ble for enabling actions that lead to the fulfillment 
of a currently achievable subtask only. Based on the 
information coming from the intention layer and the 
object memory layer, the action execution layer de¬ 
cides to perform an action, which supports the human 
in the intended subtask or leads to the completion of 
an independent subtask and does not interfere with the 
current human intention. Each layer of the system is 
formalized by one or more DNFs. The activity m,(x, t) 
at time t of a neuron x is described by the differential 
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Fig. 69.25 Structural overview of the decision-making system from (after [69.164]; courtesy of Estella Bicho) 


equation 

8m, ■ (x, t) 

ri —j-= - Ui(x , t) + Si(x, t) 

01 

+f no-mu, 

(69.30) 

where r,- > 0 and /z, > 0 denote the timescale and the 
resting level of the field dynamics, respectively. 5) is 
the summed input to a local population. The output 
function f is chosen to be a sigmoid function and 
the interaction strength w,( x—xf) is a Gaussian curve, 
which depends only on the distance between x and x' 
(Fig. 69.26). 

In [69.129], a mimetic communication model for 
pHRI is introduced, where the according motion prim¬ 
itives are taught by human demonstration via marker 
control. These are subsequently encoded into hidden 
Markov models (HMMs) and allow the robot to exe¬ 
cute them and even to recognize the motion primitives 
executed by the human partner. Based on the motion 
primitives, interaction primitives are then learned as 
chains of actions and reactions. The flow of interac¬ 
tion primitive learning is depicted in Fig. 69.27a. First, 
the robot executes a motion primitive, followed by ob¬ 


serving the human’s respective reaction in order to 
learn the correct interaction pattern sequence. After this 
step, the motion primitives are updated and the pro¬ 
cess is repeated. The underlying scheme of interaction 
is shown in Fig. 69.27b. The learned motion primi¬ 
tives are encoded as continuous HMMs (CHMMs) and 
form the middle layer of the system. They are then 
used to recognize the motion primitive executed by 



Fig. 69.26 (a) Nonlinear threshold output function that maps activ¬ 
ity to a sigmoidal with threshold uo . (b) Synaptic weight function 
for modeling the interaction strength between any two neurons x 
and*' in (69.30) (after [69.164]; courtesy of Estella Bicho) 
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Fig. 69.27 (a) Learning of interaction primitives, (b) Scheme of interaction, (c) Adaption of robot motion via virtual spring 
(after [69.129]; courtesy of Dongheui Lee) 


the human and to generate the motion of the robot 
according to the interaction primitives encoded as dis¬ 
crete HMMs (DHMMs) at the top. The robot motion 
(more precisely its behavior in the context of the au¬ 
thors’ work) may then be modified to adapt to human 
motion in the real world (expressed by the thin hori¬ 
zontal arrow). The bold horizontal arrow represents the 
assumed adaption of the human to the robot behav¬ 
ior. The adaptation strategy of the robot movement is 
depicted in Fig. 69.27c. For this, an impedance con¬ 
troller is used in combination with a virtual spring 
connected to the robot hand to attract the device into 
the correct position. Figure 69.28 depicts an applica¬ 
tion of the interaction scheme, where the robot high- 
or low-fives a human coactor with one or two hands, 
respectively. 

69.6.2 Interaction Planning Problem 

In order to formalize the interaction planning problem, 
one needs to be able to describe the entire scenario, 
the system state of the robot, the state of the human(s) 
(possibly including future behavior prediction of both), 
the environment state including all relevant objects and 
the overall abstract task state. This information about 
the world state comprises the concept of an interaction 


world , which builds the basis for formulating the gen¬ 
eral interaction planning problem. 

Definitions 

The set of world states W5 is defined as 

WS = RS x HS" x OBS " x TS . (69.31) 

where RS.HS.OBS.TS denote the set of the robot 
states, n humans, m obstacles, and an overall task state, 
respectively. 

The set of robot states 

RS = Sx RA" x IR (69.32) 

contains information about the internal state s e S of 
the robot, the robot awareness ra e RA , and the inter¬ 
action state ir e IR. The interaction state of the robot 
may, for example, take the values autonomous, collab¬ 
orative, or cooperative. The robot awareness indicates 
its ability to predict human actions. The set of internal 
states S = S ac x 5b x S p contains information about the 
set of actions carried out by the robot S ac . Furthermore, 
the behavior set Sb, which includes, for example, con¬ 
troller choices and respective parameterizations, reflex 
reactions, and type or parameterizations of trajectory 



Fig. 69.28 pHRI using the mimetic communication model (after [69.129]; ItStjM'il'lfrXfU; courtesy of Dongheui Lee) 
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planners. Such a reflex reaction which can be thought 
of as an analogy to human reflexes, intends to bring 
the robot to a safe state or even a reflex cascade R 
(Sect. 69.6.3). Finally, the physical state sp e S p con¬ 
tains, for example, the robot’s instantaneous position, 
velocity, and momentum. 

The set of human states 

HS = PSH x PA x HA x 1H x D (69.33) 

combines the sets of physical state PSH (position, 
velocity, etc.), personal attributes PA (fitness, age, ex¬ 
perience, etc.), human awareness HA (the ability of the 
human to predict the robots actions), interaction states 
of the human IH (for example, waiting for robot, work 
with robot or work without robot), and the distances D 
between human and robot. In its most simple nontriv¬ 
ial case, D could, for example, consist of the elements 
in workspace, in perception, out of perception, and lost 
perception. 

The set of task states 

TS = Ax TC x IS (69.34) 

contains A, TC, IS that denote the sets of possible ac¬ 
tions, task criticality, and expected interaction between 
human and robot, respectively. The task criticality spec¬ 
ifies the effect of potential task failure on human safety. 
A task consists of the meaningful composition of ac¬ 
tions and/or more complex skills such as grasp object 
or simply move to pose according to a particular cre¬ 
ation process. 

The Interaction Planning Problem 
The interaction planning problem then denotes the 
problem of selecting a suitable, and if possible opti¬ 
mal, robot action and associated behavior in every time 
step based on the information contained in the world 
state, the history of the process HIS, and available (dy¬ 
namic) knowledge stored in a knowledge base KB (for 
example, safety knowledge, object properties, world 
grounding rules, ...). Formally, this can be expressed 
by the select action mapping 

sa : W5 x HIS x KB S ac x S b . (69.35) 

Optimality could, for example, be expressed in the 
following reinforcement learning sense, where the se¬ 
lected behavior shall lead to a maximum reward. This 
is obtained as follows: 

1. A task evaluator evaluates the suitability of a partic¬ 
ular action/behavior for fulfilling the desired task. 

2. A safety evaluator evaluates the overall safety of 
the situation by taking into account different biome¬ 
chanical injury criteria, the expected interaction 


between human and robot, human-robot-related 
geometric quantities, such as minimal distance, as 
well as the task criticality. 

3. Finally, additional rewards may be given by the hu¬ 
man in order to capture human-friendly behavior. 

4. The overall reward is then a suitable combination of 
the basic rewards. 

69.6.3 Robot Reflexes 

Human reflexes are involuntary reactive body move¬ 
ments in response to a perceived input, the so-called 
stimulus, that is, one does not even have to think about 
what to do. In humans, reflexes aim for body protec¬ 
tion. Based on built-in heuristics, they automatically 
protect the human body from injury. However, the exact 
separation in terms of hybrid system-like theory is not 
yet clear. Thus, the line between discrete reflex states 
and according systemic responses on both electrical and 
mechanical levels is somewhat blurred. In robotics, on 
the other hand, it is possible to separate these levels and 
extend the concept of basic nominal robot actions by 
reflex reactions [69.9, 85, 139]. These concepts system¬ 
atically incorporate the possibilities one gains from the 
already described collision detection and reflex reaction 
schemes (Sect. 69.4.3). Other work in similar directions 
for developing human-like withdrawal reflexes can be 
found in [69.165]. 

In contrast to the classical planning and execution 
pipeline, the activation of reflexes that are able to over¬ 
ride the nominal task plan due to environmental or 
internal conditions that do not comply with nominal 
task behavior have to be considered. In contrast to hu¬ 
mans, these reflexes do not necessarily have to be fixed 
(or slowly time-varying over learning cycles). On the 
contrary, they may be tailored to every nominal ac¬ 
tion such that local context dependency (in particular 
instantaneous sensory input) may be taken into ac¬ 
count. After instantaneous reactions are executed, the 
main problem is to decide what to do next. Either lo¬ 
cal re-entry into the previous plan or entire re-planning 
have to be executed. Figure 69.29 depicts the over¬ 
all concept also in the context of task and interaction 
planning. 

Formally, a robot reflex is associated with a suit¬ 
able activation signal. Typically, this represents either 
the indication of a certain stimulus or a fault. Stimuli are 
general perception inputs, whereas faults are detected 
either by processed stimuli (observation of external 
torques, proximity information, etc.) or general system 
malfunctions as, e.g., communication collapse or run¬ 
time violations. Even rather complex reflex patterns can 
be activated that may be represented as directed reflex 
graphs, i.e., a decisional component in the inner most 
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Fig. 69.29 Conceptual sketch of robot reflexes;/] and /2 indicate the update rates, or alternatively the system bandwith 
of the nominal planning and execution as well as the reflex reaction level. Typically, the former acts at a much lower 
frequency than the latter one (after [69.85,139]) 


control loop of the system. A still relatively open re¬ 
search problem is how to resolve potential failures after 
reflexes were triggered by planning suitable re-entering 
and continuation of the task. 

69.6.4 Reactive Control Architecture 

Obviously, the diverse methods and complex require¬ 
ments in the pHRI domain and more specifically from 
interaction control, reflex planning, human-centered 
motion planning, and interaction planning make new 
architectural concepts necessary. The central require¬ 
ments to an according control architecture become 
rather different from the classical ones (Chap. 12) due 
to the demands for very responsive behavior not only 
on control but also on planning level. Figure 69.30 
depicts such a reactive control framework [69.166] 
flaam'il' H'HH i One of the main questions to address 
is how to adapt dynamically, safe, and task consistent, 
while keeping the overall plan and the respective con¬ 
text in mind. The control framework is composed of 
three layers of abstraction operating at different time 
scales: 


1. On the highest level of abstraction a global task 
planning module builds global task plans that con¬ 
tain temporo-logical concatenations of robot skills, 
which are derived from any suitable task planning 
language. Typically, this module runs either offline, 
in particular when building an entire nominal task 
plan that contains interaction schemata, or at very 
slow update rates of several seconds or minutes, de¬ 
pending on the complexity and novelty of the respec¬ 
tive task. Skills are topologically invariant capabil¬ 
ity structures, which instantiations can be modified 
in terms of skill parameter vectors. The most basic 
skill is the atomic robot action that represents the 
formal interface to the robot real-time control core 
and mechatronics. More complex skills can, e.g., be 
grasp an object or hand over an object. Each skill 
itself is aware of the current task state and which 
atomic action is currently being executed. This in¬ 
formation is then sent to the dynamic planning layer. 

2. The second layer of abstraction, the dynamic plan¬ 
ning level is capable of executing and/or modifying 
plans dynamically, which means that it shall run at 
least in the range of % 1 — 10 Hz. It is responsible 
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Fig. 69.30 Reactive and human-friendly control frame¬ 
work for acting in partially known environments and 
interaction ► 

for selecting the best action under the premise of the 
current task state, human state and behavior, as well 
as the environmental state. In a learning and adap¬ 
tion unit the global task knowledge is translated 
into the respective dynamic planning domain lan¬ 
guage, i. e., global knowledge and plans are encoded 
into a dynamic level such that dynamic adaptation 
can make use of global context and the respective 
mission. The according reactive planning unit is 
then able to (re-)plan the robot’s desired nominal 
actions such that safe, yet task consistent actions, 
are executed if possible. In particular, instantaneous 
perception may cause alterations of the original 
global plans. A major task on this abstraction level 
is to change the preplanned course of action safely 
and task consistently. However, finding its way back 
into the task is at least equally challenging and re¬ 
quires careful treatment (re-entry planning). 

3. The lowest architectural layer is the low-level (real¬ 
time) control layer. Typically, this is subdivided into 
multiple hierarchical layers, involving also the re¬ 
flex machine (Fig. 69.29). However, for sake of 
clarity, we consider it to be a single consistent 
representation that is accessible via a desired ac¬ 
tion/behavior complex (ad. ^d) that is sent to the 
system for execution. As the desired behavior may 
alter due to reflex behaviors in case of acciden¬ 
tal events such as unforeseen collisions, the control 


Global task planner 



layer feeds back the currently active action/behavior 
pair (a, b ), such that the dynamic planning layer is 
able to react accordingly. 

Fundamental to the aforementioned architecture is 
the capacity to observe human actions on various lev¬ 
els of abstraction. In this sense, a human observer that 
gathers all relevant information and knowledge about 
human agents is essential. In particular, it provides 
human-related information that can be of further use. 


69.7 Conclusions and Challenges 

pHRI has become a central discipline in robotics over 
the last decade. This is due to the significant progress 
made in the fields of mechatronics, interaction control, 
motion planning, and 3-D sensing toward highly inte¬ 
grated and sensorized lightweight systems that are able 
to physically interact with their surrounding. In direct 
consequence the learning, planning, and execution of 
safe and legible interactions has become a widely taken 
research direction. Clearly, the rise of a new generation 
of commercial robots capable of physical interaction 
has also contributed to the large interest in the field. 
The robotics research and industrial community expects 
these systems to open up new markets and to push 
robotics further toward domestic applications that may 
also involve even more complex and possibly mobile 
manipulators. 


However, despite this recent success in research 
and also in the commercialization of assistance robots, 
there are many open research questions that need to 
be tackled before this class of systems can become 
a commodity not only in early adopter industrial appli¬ 
cations but also on a broader scale: Continuing the road 
toward safe robotics by tightly coupling injury biome¬ 
chanics and safe interaction control with lightweight 
and compliant robot design will further push the bound¬ 
aries and build the foundation of pHRI. On the other 
hand, learning interaction controllers and planning in¬ 
tuitive and safe interactions are still very young fields, 
however, they are the key to solving the long-term 
physical interaction problem. Furthermore, the current 
application of assistance robots in real-world problems 
will bring further novel research questions. The recent 
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use of these robots clearly underlines that the pro¬ 
gramming models and paradigms of interaction and 
soft manipulation are very different from classical in¬ 
dustrial robot programming. In particular, they go be¬ 


yond simple pick and place models toward models of 
force based programming, an interesting research ques¬ 
tion currently being investigated by various researchers 
worldwide. 
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70. Human-Robot Augmentation 


Massimo Bergamasco, Hugh Herr 


The development of robotic systems capable of 
sharing with humans the load of heavy tasks has 
been one of the primary objectives in robotics 
research. At present, in order to fulfil such an 
objective, a strong interest in the robotics commu¬ 
nity is collected by the so-called wearable robots, 
a class of robotics systems that are worn and di¬ 
rectly controlled by the human operator. Wearable 
robots, together with powered orthoses that ex¬ 
ploit robotic components and control strategies, 
can represent an immediate resource also for al¬ 
lowing humans to restore manipulation and/or 
walking functionalities. 

The present chapter deals with wearable 
robotics systems capable of providing different 
levels of functional and/or operational augmen¬ 
tation to the human beings for specific functions 
or tasks. Prostheses, powered orthoses, and ex¬ 
oskeletons are described for upper limb, lower 
limb, and whole body structures. State-of-the- 
art devices together with their functionalities and 
main components are presented for each class of 
wearable system. Critical design issues and open 
research aspects are reported. 
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70.1 Concept and Definitions 

Since early developments, one of the primary objec¬ 
tives of technology has been that of relieving humans 
from heavy tasks [70.1]. The concept of human-robot 
augmentation refers to the use of robotics systems to 
increase human functionalities in different operational 
contexts [70.2]. In general, the augmentation of hu¬ 
mans’ performances through the use of robotic systems 
refers solely to the increase in mechanical capabili¬ 
ties involving motion control and generation of scaled 
levels of forces. Although complex operations involve 
and exploit perceptual capabilities, in this chapter the 
concept of human-robot augmentation is not referred 
to perceptual functionalities only. Examples of work¬ 
ing conditions in which human-robot augmentation can 
play an essential role for the completion of the task can 
be considered, e.g., operations for debris removal after 
an earthquake in which human operators, wearing such 
systems, can move and operate in a more flexible and 
accurate way with respect to machines like cranes or 
excavators. 

Other scenarios are represented by tasks in which 
the number of repetitions, instead of level of loads to 
be operated, is high and becomes the critical factor for 
decreasing fatigue, thus maintaining long-term levels of 
humans’ safety. 

By definition, wearable robots follow the human op¬ 
erator’s movements and in this way can inherently and 
fully exploit human’s dexterity, thus assuring a higher 
level of flexibility in addressing a specific task with 
respect to usual loading machines. A first and fun¬ 
damental characteristic, and at the same time a strict 
design constraint, of wearable robots is the condition 
of possessing a physical structure which is always in 
contact with the human operator’s body during opera¬ 
tion. As a consequence, wearable robots design must 
take into account both biomechanics and human motor 
control aspects. 

Research on wearable robots has received a strong 
contribution from the development of haptic interfaces, 
i. e., robotic systems that are able to generate forces 
on the human body (force feedback) while maintain¬ 
ing a high level of naturalness of user’s movements. 
The development of haptic technologies in the last two 
decades has focused on the importance of designing 
robotic systems able to generate forces at the level of 
the hand or other parts of the human body. As a conse¬ 
quence, an increased attention of the scientific robotic 
community toward wearable robots has been recorded. 
In fact, most of the recent developments on robotic 


exoskeletons derive from their application in neurore¬ 
habilitation [70.3], where the robotic system is used to 
kinetically assist the patient’s limb during the perfor¬ 
mance of specific exercises. 

The augmentation of human performances can be 
achieved with different types of robotic systems, e.g. 
from teleoperated robots to self-standing robotic sys¬ 
tems amplifying human actions. Examples of robotic 
systems directly operated by the human operator are 
the Cobot system [70.4] and the intelligent autonomous 
system (IAS) [70.5]. However, in the framework of the 
present review work, the analysis is performed by con¬ 
sidering only robotic systems that can be worn by the 
user. 

At present, few definitions of wearable robotic sys¬ 
tems have been considered. Herr makes a first classifi¬ 
cation between serial and parallel system [70.6]. 

Serial systems are considered those that increase 
the length and reachability of a limb, while paral¬ 
lel structures contribute to the effective augmenta¬ 
tion of torques and forces generated by the human 
limb. Serial systems perform their action by exploit¬ 
ing elastic components integrated in their structure, thus 
not requiring a specific computer-based control sys¬ 
tem. Other classes of wearable robotic systems can 
be considered the so-called hybrid wearable systems, 
i. e., parallel robotic structures worn by the human 
operator in which the command for their motion is 
achieved by FES (functional electric stimulation). In 
this way, the control of the robotic structure is lim¬ 
ited and does not allow a complete force control by the 
user [70.7], Hybrid systems are not considered in this 
chapter. 

Wearable robotics systems for human augmentation 
can be divided into three different categories: (i) pros- 
theses, for the functional replacement of human limbs; 
(ii) powered orthoses, whose function is to actively op¬ 
erate in parallel with unhealthy human joints; and (iii) 
robotic exoskeletons that operate in parallel with human 
limbs [70.8]. 

The design of the above three types of robotic 
systems shows different approaches whether they are 
referred to upper, lower limbs, or to the whole body, 
respectively. In the following, the analysis of wearable 
robotic systems is given by considering present achieve¬ 
ments in terms of devices, robotic components utilized 
for the actuation, control systems architectures, and de¬ 
vised application domains in which they are presently 
utilized. 
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70.2 Upper Limb Wearable Systems 

Upper limb exoskeletons (ULEs) are wearable robots 
that are placed in parallel and work in concert with 
the human arm. Wearer’s motions that fall within 
ULEs coverage are shoulder elevation/depression and 
retraction/protraction which are provided by the hu¬ 
man’s sterno-clavicular articulation; shoulder flex¬ 
ion/extension, abduction/adduction and medial/lateral 
rotation which are provided by the human’s gleno¬ 
humeral articulation; elbow flexion/extension which 
is provided by the human’s humero-ulnar articu¬ 
lation; forearm prono/supination which is provided 
by the human’s radio-ulnar articulation; wrist flex¬ 
ion/extension and radial/ulnar deviation which are 
provided by the human’s radio-carpal, mid-carpal, 
and intra-carpal articulations; and finger abduc¬ 
tion/adduction and flexion/extension which are pro¬ 
vided by the human’s metacarpo-phalangeal and inter- 
phalangeal joints. 

Proposed usages of ULEs are as rehabilitation de¬ 
vices, motion assists, human power/force amplifiers, 
and haptic interfaces. Depending on the specific appli¬ 
cation, the available ULEs differ in location of place¬ 
ment, number of degrees of freedom (DOFs) and their 
subdivision between active and passive DOFs, type of 
kinematic architecture and implementation, type of ac¬ 
tuators and power transmission methods, types of sen- 
sorization, and number and place of connection points 
between ULE and wearer. 

70.2.1 Location of Placement 

With regard to location of placement, a number of sys¬ 
tems have been developed which differ from the type of 
segments they can cover; for instance shoulder-elbow- 
wrist-finger complex [70.10,11], shoulder-elbow- 
finger complex [70.9] (Fig. 70.1), shoulder-elbow- 



Fig. 70.1 Upper limb exoskeletons for the shoulder- 
elbow-wrist-finger complex (PERCRO-II, L-Exos, af¬ 
ter [70.9]) 


wrist complex [70.12-16] (Fig. 70.2), shoulder-elbow- 
forearm complex [70.17-20] (Fig. 70.3), shoulder- 
elbow complex [70.21-27] (Fig. 70.4), elbow-wrist 
complex [70.28,29], wrist-finger complex [70.30], and 
the hand (fingers only) [70.31-33]. 

70.2.2 Degrees of Freedom 

With regard to DOFs, developed systems largely dif¬ 
fer depending on the specific location of placement. 
In particular, exoskeletons with up to 9-DOFs have 
been developed for the shoulder-elbow-wrist-finger 
complex [70.16], whereas exoskeletons with up to 18- 
DOFs have been developed for the wrist-finger com¬ 
plex [70.30]. 

Focusing on ULEs covering at least the shoulder- 
elbow-forearm complex, the developed systems typ¬ 
ically feature [70.9-13, 15-20] the following: three 
active DOFs for shoulder flexion/extension, abduc¬ 
tion/adduction and medial/lateral rotation; one active 
DOF for elbow flexion/extension; and one DOF, ei¬ 
ther active or passive, for forearm prono/supination. 



Fig.70.2a-c Upper limb exoskeletons for the shoulder- 
elbow-wrist complex: (a) RUPERT (after [70.14]); 
(b) ARMIN (after [70.15]); (c) SUEFUL-7 (after [70.16]) 
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Fig.70.3a-d Upper limb exoskeletons for the shoulder- 
elbow-forearm complex: (a) PERCRO-I (after [70.34]); 
(b) MGA (after [70.19]); (c) PERCRO-III (after [70.9]), 
(d) RehabExos (after [70.20]) 

Some of these systems are also provided with a 
wrist having two active DOFs for flexion/extension 
and radial/ulnar deviation [70.10,12,13,16]; an ac¬ 
tive [70.11,15,19,23] or passive [70.26] DOF for 
shoulder elevation/depression (some systems also ex¬ 
ist in which shoulder elevation/depression is coupled to 
shoulder abduction/adduction [70.15,16,35]); and an 
active [70.23] or passive [70.11,26] DOF for shoulder 
retraction/protraction. Although the availability of er¬ 
gonomic evaluation data is still limited, the inclusion of 
shoulder elevation/depression is deemed necessary for 
large-range-of-motion ULEs with arm elevation angles 
larger than 60° [70.15]. 

70.2.3 Kinematic Architecture 
and Implementation 

Despite a few ULEs [70.10,12,26], the majority of 
the developed systems feature a kinematic architec¬ 
ture that is isomorphic to that of the wearer’s upper 
limb (which means that the rotation axes of ULE 
joints are coaxial to those of the wearer’s articulations). 
This choice usually reduces the chances of wearer- 
exoskeleton interference and enables the wearer to be 
connected to the ULE at different points over mul¬ 
tiple links. With regard to implementation, different 
solutions have been proposed depending on the con¬ 
sidered ULE articulation. In particular, three shoulder 
rotations have been implemented in the following dif¬ 
ferent ways: (1) via three standard revolute pairs with 
either nonintersecting axes [70.10,26,26,36] or with 
axes intersecting away from the center of the wearer’s 
gleno-humeral articulation [70.12], which leads to sys¬ 


tem designs that are not isomorphic with the wearer’s 
limb kinematics; (2) via three standard revolute pairs 
with axes intersecting at the center of the wearer’s 
gleno-humeral articulation [70.19,23], which usually 
leads to ULEs with either limited range of motion 
or excessive encumbrance; (3) as in (2) but with the 
last revolute joint being a circular guide [70.9, 11,13, 
16,20,37], which enables for a more compact design 
that is, however, affected by a limited singularity-free 
workspace; (4) as in (2) but with the second revo¬ 
lute joint being a circular guide [70.38], which pro¬ 
vides both a very compact design and a rather wide 
singularity-free workspace. Regarding the additional 
DOFs at the shoulder, both elevation/depression and 
retraction/protraction motions have been implemented 
via either revolute [70.19,23] or prismatic [70.11, 15, 
26] pairs. Elbow flexion/extension has been imple¬ 
mented almost univocally via a standard revolute pair. 
As for the wrist, flexion/extension and radial/ulnar de¬ 
viation have been implemented by using two revolute 
joints with either intersecting axes [70.10,12,13] or 
slightly off-set axes [70.16]. In fact, in the human 
wrist, the radial/ulnar deviation axis passes distally 
with respect to the flexion/extension axis by approxi¬ 
mately 5 mm. 

70.2.4 Actuation and Power Transmission 
Methods 

Actuation methods have also been quite variegate. 
Besides passive systems employing springs (or elas¬ 
tic bands) [70.22] and hydraulic disk brakes [70.26], 
ULEs have been equipped with hydraulic mo¬ 
tors [70.10], pneumatic cylinders [70.24], pneumatic 
muscles [70. 12, 14], and electric motors. Hydraulic mo¬ 
tors provide very large power-to-weight ratio, high 
stiffness, and direct-drive operation; however, they fea¬ 
ture low energetic efficiency and require bulky, high- 
pressure fluid components (power supply, valves, and 
tubing). Pneumatic cylinders and muscles provide large 
power-to-weight ratio and direct-drive operation; how¬ 
ever they feature low energetic-efficiency, low stiff¬ 
ness, slow and nonlinear response, and require bulky 
and noisy fluid components (power supply, valves and 
tubing). In addition, pneumatic muscles can generate 
forces in one direction only, and thus they need to 
be used in an agonist-antagonist arrangement [70.12]. 
Electric motors are very efficient, easy to control, and 
have good power-to-weight ratio; however, they work 
better at high speeds. In the existing ULEs, electric mo¬ 
tors are rarely used in direct-drive operation, but rather 
they are combined with either standard gearboxes (with 
spur [70.16] or planetary [70.9] gear-heads) or har¬ 
monic drives [70.19, 20, 27], 
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Fig.70.4a-e Upper limb 
exoskeletons for the 
shoulder-elbow complex: 

(a) WREX (after [70.22]); 

(b) MEDARM (after [70.23]); 

(c) BONES (after [70.24]); 

(d) ABLE (after [70.25]); 

(e) DAMPACE (after [70.26]) 


In the existing ULEs, hydraulic motors and pneu¬ 
matic muscles have been located directly across single 
exoskeleton joints [70.10,12,14], whereas pneumatic 
cylinders have been used in an in-parallel arrangement 
to activate multiple joints [70.24]. Instead, electric mo¬ 
tors have been placed either at the joints [70.19,20] or 
in a remote location [70.9, 13,23] (usually in the ULE 
base frame). This second choice has been made primar¬ 
ily in order to reduce the weight and inertia of moving 
exoskeleton parts, which enables the use of smaller mo¬ 
tors and minimizes the disturbances on the wearer’s 
natural motions. 

ULEs with remotely placed actuators require ap¬ 
propriate transmission systems to transfer the power 
from the actuators to the joints. Except from spe¬ 
cific cases [70.15,23,35] in which transmission link¬ 
ages and timing belts have been used, the common 


power transmission solution for ULEs has consisted 
in cable-drive systems, either closed-ended [70.9, 13] 
or open-ended [70.23]. Closed-ended transmissions re¬ 
quire long cables, whereas open-ended transmissions 
require more motors than the number of actuated DOFs. 
Both open- and closed-ended cable-drive systems are 
lightweight, highly efficient, and back-drivable; thus 
they are very good solutions to transmit mechanical 
power over long distances. In addition, cable drives 
enable the easy implementation of speed-reduction 
stages, which should be placed as close as possible 
to the joint in order to maximize overall transmis¬ 
sion stiffness [70.9], Moreover, cable-drive transmis¬ 
sions spanning multiple joints make it possible to 
distribute loads across several cables, which can be 
used to minimize the torque requirements of the actu¬ 
ators [70.23]. 
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70.2.5 Type of Sensorization 

Apart from few passive systems only conceived as 
weight compensation devices [70.39], all the developed 
ULEs are equipped with position sensors (potentiome¬ 
ters or encoders), which are placed at the motors and/or 
at the joints so as to make it possible to estimate/control 
the overall system configuration. Besides, the consid¬ 
ered ULEs largely differ from the point of view of 
force/torque sensorization. Thanks to the intrinsic low 
friction and high back-drivability, exoskeletons with 
cable-drives have been designed to operate without the 
need of force/torque sensors [70.9, 13, 23]. All the other 
ULEs are instead equipped with this kind of sensoriza¬ 
tion, which can be either concentrated at the points of 
connection with the wearer [70.11, 15,16, 19] or dis¬ 
tributed at every ULE joint [70.20]. The use of torque 
sensors distributed at every ULE joint enables for mul¬ 
ticontact force control at multiple points distributed 
over multiple links and, additionally, makes it possi¬ 
ble to design very robust force-feedback controllers that 
are rather insensitive to exoskeleton link variable in¬ 
ertia, transmission compliance, friction losses, actuator 
torque ripples, and gear teeth wedging actions [70.20]. 
The use of multi-DOF force/torque sensors concen¬ 
trated at the points of wearer-ULE interaction makes 
the use of the exoskeleton less flexible and the con¬ 
troller less robust, but enables for a more accurate 
estimation and control of the endpoint forces that are 
exchanged with the wearer. In general, ULEs equipped 
with force/torque sensorization provide better haptic 
feedback than those relying on cable-drive transmis¬ 
sions only. 

70.2.6 Hand Exoskeletons 

The functionality of hand exoskeletons (HE) consists in 
exerting programmable forces on human fingers. Hand 
exoskeletons can be classified in two functional cate¬ 
gories depending on the capability of exerting forces 
on a single phalanx (usually the distal phalanx cor¬ 
respondent to the fingertip) or on multiple phalanges 
(Fig. 70.5). 

Multiphalanx hand exoskeletons (MPHE) are de¬ 
vices able to exert different forces on at least two of 
the phalanges of the same finger: usually the force 
can be applied on a fixed direction normal to the pha¬ 
lanx axis and belonging to the medial plane of the 
finger. 

Single-phalanx hand exoskeletons (SPHE) are able 
to exert forces on one phalanx of the finger, usually 
the distal one only; some devices are able to generate 
a force only along a fixed direction but more commonly 
they can exert forces with any desired orientation. 


a) b) 



Fig. 70.5 (a) Multiphalanx and (b) single-phalanx scheme 
for hand exoskeletons 


Another taxonomy of hand exoskeletons considers 
two other categories: 

• Anthropomorphic devices: the kinematics of the HE 

is morphologically similar to the human fingers 

• Nonanthropomorphic devices: the kinematics is 

morphologically different from the human fingers. 

The scientific literature in the last 30 years reports 
many works dealing with different HE designs mainly 
addressing three application fields: (1) teleoperation 
and virtual environments in which the HE are utilized 
as master controllers providing force feedback or for 
generating illusory forces to the hand of the human op¬ 
erator; (2) neurorehabilitation, in which HE are used 
for restoring hand functionalities after strokes or other 
neural impairments; (3) space applications in which HE 
are employed as active devices for extending grasping 
forces of the astronauts in extra vehicular activities. 

The first known example of HE was introduced 
in the field of teleoperation by Zarudiansky back in 
1981 [70.40]. The inventor has registered a patent 
for a tele-manipulation system equipped with a mas¬ 
ter device able to provide force feedback on different 
phalanges of the human hand. At the Jet Propulsion 
Laboratory Pasadena (JPL) in 1988, Jau [70.41] built 
a complete teleoperated master/slave system compre¬ 
hending a hand exoskeleton device for four fingers. 
In the same years Burdea et al. developed a pneu¬ 
matic actuated hand force feedback device [70.42], 
called Rutgers Master, able to exert a single force 
to the distal phalanx along a direction dependent on 
the kinematics of the device and on the postion of 
the finger. Bergamasco [70.37] developed a four-finger 
MPHE. A nonanthropomorphic device was realized by 
Koyama et al. at Keyo University; this device is a three- 
finger exoskeleton for thumb, index, and middle finger 
with passive clutches actuation [70.43]. Such a device 
is anchored at the user’s wrist and is able to exert 
forces in every direction belonging to the sagittal plane 
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of the fingers. Frisoli et al. [70.9] developed another 
nonanthropomorphic HE, called Pure Form Hand-Exos 
for virtual environment (VE) applications that features 
a very complex steel cable transmission studied for an 
optimal use of motor torque. Nakagciwara et al. [70.44] 
built a device integrating the concept of encountered 
haptics: the device possesses 1-DOF for each finger 
and it is able to track the finger without any con¬ 
tact (even at the fingertip) through a noncontact sensor 
placed in correspondence of the nail. When contact is 
detected in the remote/virtual environment, a plate is 
moved against the user’s fingertip. Another example 
of anthropomorphic hand exoskeleton for VE applica¬ 
tions has been developed by Fontana et al. [70.32] with 
the aim of maximizing force accuracy and resolution 
(Fig. 70.6). 

Other examples of HEs have been developed for 
medical applications. In this field, HEs are employed 
for rehabilitation procedures or as power-assisting de¬ 
vices (orthoses). In rehabilitation procedures a HE is 
commonly used for exerting forces (or trajectories) and, 
at the same time, measuring velocities and positions 
(or force) of the fingers. Gomez et al. [70.45] have 
integrated the Rutgers Master in a VE rehabilitation 
system. Wege et al. [70.46] developed a 4-DOF HE with 
a modular structure, actuated by Bowden cables. Mulas 
et al. [70.47] realize a HE that makes use of electromyo¬ 
graphy (EMG) signals for predicting the movements of 
the user finger. Ito et al. [70.48] built a complex HE 
for rehabilitation purposes with 4-DOF for the index, 
medium, and ring fingers and 5-DOF for the thumb, 
able to exert controlled torque on each phalanx. 

HEs can implement the functionality of supporting 
people with permanent disabilities to perform com¬ 
mon activities for daily living (ADL). Such devices, 
called hand orthoses or assistive HE, are controlled as 
power extenders allowing the user to exert only a frac¬ 
tion of the force required to accomplish the task. The 
first orthosis for the finger was developed by Brown 
et al. [70.49]. The realized device was a HE composed 
by three rings attached on the phalanges that were ac¬ 
tuated with pulling cables. Other examples of hand 
orthoses are described in [70.4, 39]. Another field where 
HEs have proved their usefulness is in assisting astro¬ 
nauts in performing EVA (extra-vehicular activities). 
Due to the pressure difference, the suits that the astro¬ 
nauts wear during EVA require high force to be bended 
and represent a major constraint especially for manip¬ 
ulative tasks. For this reason, custom-designed HEs 



Fig.70.6a,b Hand exoskeleton (after [70.32]) (a) without 
and (b) with tracking system 


are used as an active tool to overcome the stiffness 
of the gloves. Shields et al. [70.50] developed the first 
HE for space applications. Such a device was an an¬ 
thropomorphic HE with three active fingers; the thumb 
was excluded and the little finger was connected to¬ 
gether with the ring finger. The HE has two DOFs for 
each finger and exploits a special mechanisms for the 
implementation of the finger flexion joints. The an¬ 
thropomorphic kinematics is obtained through remote 
center of rotation joints. In 2001, Yamada et al. [70.51] 
developed, for the same space applications, another 
simplified HE called Skil Mate in which only the first 
flexion DOF of each finger is actuated. The HE de¬ 
veloped by Wang et al. [70.52] is still utilized for the 
simulation and evaluation of different space gloves. 
The device features a kinematics implementation that 
is close to the HE developed by Shields but in this case 
the device has three DOFs coupled in a way that only 
one actuator is needed for each finger. Also in this case, 
flexion joints are implemented through remote center of 
rotations allowing the HE joints to closely correspond 
with the natural joints (metacarpophalangeal MCP, in- 
terphalangeal IP, distal D) of the human hand. 

In general, it can be observed that HEs realized 
for VE and teleoperation have several mechanical fea¬ 
tures that are not present in the devices conceived for 
rehabilitation, ADL or EVA applications. VE and tele¬ 
operation application require in fact the simulation of 
fine haptic interaction and generally more demanding 
performances are needed in terms of mechanical fea¬ 
tures. In VE and teloperation, the design of HE requires 
to minimize friction, inertia, and weight. On the other 
side, in other applications, a higher level of friction can 
be tolerated, allowing the use of Bowden cables for 
delocalizing the motors, or a simplified kinematic struc¬ 
ture can be acceptable allowing exerting forces with less 
accuracy. 
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70.3 Lower Limb Wearable Systems 

Lower extremity autonomous wearable robotics is 
a general classification covering actuated leg prosthe- 
ses, orthoses, and exoskeletons. 

70.3.1 Lower Extremity Autonomous 
Wearable Robotics 

These robots are defined as active mechanical devices 
that are worn on a wearer’s legs, and work in con¬ 
cert with the wearer’s movements, either as permanent 
assistive devices for persons suffering from leg pathol¬ 
ogy, or augmentative devices for persons having normal 
physiological leg function. In general, the classifica¬ 
tion active leg orthosis is typically used to describe 
a device that is used to increase the capacity of a per¬ 
son suffering from leg pathology to ambulate, whereas 
the classification leg exoskeleton is used to describe 
a device that augments the locomotory performance of 
an able-bodied wearer. However, the classification leg 
exoskeleton has also been used to describe certain as¬ 
sistive devices that span in parallel across the entirety 
of the human leg [70.6]. Autonomous wearable leg 
robots are used as assistive interventions in the treat¬ 
ment of leg disabilities caused by amputation, stroke, 
cerebral palsy, and multiple sclerosis. In the realm of 
human augmentation, leg exoskeletons have been con¬ 
structed for load-carrying augmentation, to improve the 
economy of walking or running, or to decrease muscu¬ 
loskeletal stress [70.6]. 

Since robots within this classification are worn 
by the wearer, it is critical that their design resem¬ 
ble biological leg function, both morphologically and 
neurologically. For this discussion on lower extremity 
wearable robotics, we use the term bionic to refer to 
an electromechanical structure designed to emulate, or 
extend, normal physiological function. A long-standing 
goal in robotic science is to apply neuromechanical 
principles of human movement to the development of 
highly functional wearable leg robotics [70.53]. Critical 
to this effort is the development of actuator technolo¬ 
gies that behave like muscle, device architectures that 
resemble the body’s own musculoskeletal leg design, 
and control methodologies that exploit principles of bi¬ 
ological leg movement. In the next section, we present 
a computational model of human walking that uni¬ 
fies muscle and joint biomechanics with whole-body 
metabolism for level-ground walking at self-selected 
speed. The model underscores critical design princi¬ 
ples for wearable leg robotics, including the impor¬ 
tance of tendon-like compliance and reflexive control 
on locomotory performance - discussed in detail in 
Sect. 70.3.6. These design principles are discussed in 


detail and then, by way of example, used to analyze 
the design of a bionic transtibial prosthesis. The chapter 
concludes with a discussion on critical areas for future 
research. 

70.3.2 Neuromechanical Basis 

for Wearable-Robotic Leg Design 

By understanding the underlying mechanisms that gov¬ 
ern individual muscle-tendon behaviors in human loco¬ 
motory function, roboticists can better design wearable 
robots that seamlessly integrate dynamically with the 
biological leg. To this end, in this section we present 
a computational model of human walking [70.54] that 
unifies muscle and joint biomechanics with whole-body 
metabolism for level-ground walking at self-selected 
speed. The model, shown in Fig. 70.7, comprises ankle, 
knee, and hip joints connected by rigid bodies repre¬ 
senting the human trunk and two, three-segment legs. In 
the model, muscle-tendon units that dorsiflex the ankle, 
and flex and extend the knee, are assumed to act as lin¬ 
ear springs upon neural activation; each muscle-tendon 
is modeled as a tendon spring in series with an isomet¬ 
ric force source, or a controllable clutch. To provide 
the mechanical power for step-to-step gait transitions, 
a Hill-type soleus muscle is modeled to actively plantar 
flex the ankle throughout mid to terminal stance using 
muscle state and force as reflex feedback signals. Fi¬ 
nally, to stabilize the trunk during stance, and to protract 
and retract each leg throughout the swing phase, two 
mono-articular Hill-type muscles actuate the model’s 
hip joint. State transitions of a finite machine were fa¬ 
cilitated by the walking model, and its interactions with 
a modeled ground surface, and each muscle was en¬ 
gaged or activated according to state [70.54]. Following 
a forward dynamic optimization procedure described 
in [70.54], the walking model is shown to predict joint 
biomechanics, as well as whole-body metabolism, as 
shown in Fig. 70.8, supporting the idea that the prepon¬ 
derance of leg muscle-tendons operates isometrically, 
affording the relatively high metabolic walking econ¬ 
omy of humans. 

70.3.3 Design Principles 

for Wearable Leg Robotics 

Neuromechanical models of human locomotory func¬ 
tion such as the Endo and Herr model [70.54] pro¬ 
vide critical insights into wearable robotic leg design. 
Biophysical models that describe the morphology and 
neural control of human limbs can motivate robotic de¬ 
signs that are quiet, low mass, economical, and stable; 
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-Wr Tendon spring 

- 0 - Pin joint 

—= Isometric muscle 

-CD- Contractile muscle 


Fig. 70.7 The musculoskeletal walking model of Endo 
and Herr [70.54]. Only three contractile muscles act 
about the model's ankle and hip joints capable of per¬ 
forming nonconservative positive work, namely the ankle 
plantar flexor, representing the soleus muscle, and two 
mono-articular, hip flexor/extensor muscles. All remaining 
muscle-tendon units of the leg are modeled as an isomet¬ 
ric muscle in series with a compliant linear tendon spring. 
The hip joint also includes a unidirectional, linear torsional 
spring representing the dominant ligaments that tend to flex 
that joint 

robots that move like, and feel like, their biological 
counterparts. In the subsequent sections, we discuss key 
design features critical for a robotic prosthetic, orthotic, 
or exoskeletal leg to behave like a healthy, normal bi¬ 
ological leg. Each design principle is first discussed, 
and then applied to a leg prosthesis for a transtibial 
amputee. 

70.3.4 Series and Parallel-Elastic Actuation 
for Power Amplification 

Perhaps the most paramount design feature of a wear¬ 
able robotic leg is a biomimetic actuator architecture 
designed to achieve biologic levels of powered plantar 
flexion at the ankle during terminal stance. The biologi¬ 
cal calf muscles generate nearly 80% of the mechanical 
work required to complete each gait cycle during mid 
to late stance plantar flexion, and typically performs 
greater positive than negative ankle work during each 
stance period of level-ground walking [70.36,56-61]. 
Ankle-powered plantar flexion in the trailing leg is crit¬ 
ical for powering the transition from one single support 
phase to the next single support phase, minimizing im¬ 
pact of the leading leg [70.56,58], For this purpose, 
in the walking model of [70.54] shown in Fig. 70.7, 
a Hill-type soleus muscle, or ankle plantar flexor, is 
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Fig. 70.8 Biomechanical and energetic predictions from 
the walking model shown in Fig. 70.7. Plotted is the maxi¬ 
mum cross-correlation coefficient R, summed across three 
joint angles and normalized by 3, versus the metabolic 
cost of transport (COT), or the metabolic energy required 
to transport unit body weight unit distance. R = 1 indi¬ 
cates perfect agreement with biological ankle, knee and 
hip angle data throughout one gait cycle, whereas R = 0 
indicates no agreement. Each closed circle is one forward 
dynamic model solution that can walk for at least 20 s 
without falling down. The dashed line and shaded area rep¬ 
resent mean ± one standard deviation for human metabolic 


COT data from the literature [70.55], The open diamond is 
the model’s optimal solution with a maximal R value, and 
energetically within one standard deviation of the average 
human walking metabolism 


modeled to actively plantar flex the model’s ankle from 
mid to late stance. The muscle is in series with a tendon 
spring, representing the Achilles tendon. Further, the 
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bi-articular gastrocnemius muscle, or the ankle-knee 
posterior in Fig. 70.7, spanning the ankle and knee, is 
modeled as an isometric force source, or clutch, in se¬ 
ries with a tendon spring. 

Such an architecture is critical for powering ankle 
plantar flexion while still achieving efficient, low-mass 
robotic actuation. Since the ankle-knee posterior mus¬ 
cle in Fig. 70.7 is shown to transfer little mechanical 
energy between the knee and ankle throughout the 
walking gait cycle [70.54], it is reasonable to model 
this muscle using mono-articular actuators about the 
knee and ankle. In such a framework, the effect of 
the ankle-knee posterior on these joints is represented 
as effective monoarticular torque-producing elements. 
Such an architecture is shown in Fig. 70.9a showing the 
schematic of a bionic transtibial prosthesis. Here the 
ankle is modeled as a series-elastic actuator (SEA) in 
parallel with a unidirectional parallel spring. The SEA 
represents the ankle-plantar flexor in Fig. 70.7, and 
the unidirectional spring represents the torque contri¬ 
bution of the ankle-knee posterior muscle on the ankle 
joint. 

In Fig. 70.9b, the results of a compliant transmis¬ 
sion optimization are presented, showing the effects of 
ankle total reduction ratio, series stiffness, and parallel 
stiffness on the ankle prosthesis COT. The prosthe¬ 
sis COT is equal to the amount of electrical energy 
consumed by the prosthesis throughout one gait cycle, 
divided by the product of half body weight and the dis¬ 
tance traveled in one gait cycle. The half body weight in 
the denominator assumes that each transtibial prosthe- 

Fig.70.9a,b Shown in (a) is a bionic transtibial prosthesis 
represented as an electromechanical schematic including 
ankle motor, ankle transmission ratio R, ankle series spring 
k s , and ankle unidirectional parallel spring k p . The paral¬ 
lel spring is unidirectional, engaging only at ankle angles 
less than 90° but completely disengaged for angles greater 
than 90°. In addition, spanning the knee joint are ele¬ 
ments of a series-elastic clutch mechanism. Plotted in (b) 
are the results of a compliant ankle transmission optimiza¬ 
tion showing the effects of ankle reduction ratio, series 
stiffness k s and normalized unidirectional parallel stiff¬ 
ness Kp on ankle prosthesis cost of transport, C,„. Parallel 
stiffness, normalized by the human-controlled dorsiflex- 
ion stiffness of 630 Nm rad -1 is noted as Kp on the upper 
portion of each graph. The total reduction ratio is equal 
to the SEA moment arm r multiplied by transmission ra¬ 
tio R. The compliant ankle assembly shown in (a) was 
constrained to track biological ankle torque for a human 
subject with a mass of 78kg walking at 1.25ms —1 with 
a stride distance of 1.4 m. Further details of the compliant 
transmission optimization can be found in [70.62]. ► 


sis has its own power supply. Without parallel stiffness 
(Kp = 0), the energy consumption of the prosthesis is 
minimal at a total reduction ratio greater than 400, and 
a series spring stiffness greater than 400 kNm -1 . As 
parallel spring stiffness increases (Kp > 0), the pros¬ 
thesis COT can attain a lower minimum value while 
also requiring a smaller total reduction ratio and series 
spring stiffness. Using a lower reduction ratio allows 
the system to have a larger bandwidth, a faster inter¬ 
mittent response, and a lower acoustic noise output. 
Further, a lower series stiffness improves the prosthe¬ 
sis COT as shown in Fig. 70.9a, shock tolerance upon 
impact loading at heel-strike [70.62], and motor power 
amplification at the ankle joint output [70.63]. For fur¬ 
ther details on compliant ankle transmission design, 
see [70.62,63]. In the design of bionic ankle-foot wear¬ 
able robots, both motor-series and parallel elasticity are 
of paramount importance. 
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70.3.5 Series-Elastic Actuation 

for Minimizing Motor Work 

It has been hypothesized that the preponderance of leg 
muscle-tendons generates force at low muscle fascicle 
speed, allowing economical force generation in level- 
ground walking [70.54]. In the walking model shown 
in Fig. 70.7, six of the nine leg muscles act in a purely 
isometric manner, where a tuned, series-tendon com¬ 
pliance enables full energy absorption and delivery, 
eliminating muscle work and lowering metabolic de¬ 
mands in walking. Specifically, all muscles spanning 
the knee, as well as the ankle dorsiflexor, are assumed 
to act isometrically upon neural activation when ambu¬ 
lating across a level-ground surface. 

For wearable robotic actuation in level-ground 
walking, such behaviors can be achieved using a series- 
elastic clutch mechanism with a tuned series stiffness. 
However, for a more diverse set of locomotory tasks 
such as ascending and descending slopes and steps, 
the ideal actuator architecture is a series-elastic ac¬ 
tuator with a tuned series-spring stiffness where the 
actuator is designed to lock or clutch at lower power 
consumption. As an example, for level-ground ambu¬ 
lation using an electric actuator, a clutch acting on 
the motor shaft can be used to lock the shaft during 
those times in the gait cycle where a spring response 
is sought, eliminating the high power consumption re¬ 
quired if the motor itself were to clutch the shaft. For 
other movement tasks outside of level-ground walk¬ 
ing, the motor-shaft clutch could then be disengaged, 
and the motor could apply the necessary torques and 
impedances required for the task using standard series- 
elastic actuation strategies [70.64], Such an architecture 
allows wearable robotic legs to move economically 
across level-ground surfaces, while still allowing a great 
deal of versatility for irregular-ground ambulation, sit- 
to-stand maneuvers, and general disturbance rejections. 

This design principle of using a tuned series stiff¬ 
ness to minimize the need for actuator work has been 
used in the design of a transtibial prosthesis in [70.65] 
(Fig. 70.9a). Since the gastrocnemius was modeled 
in [70.54] as an isometric force source in series with 
a tendon spring, in the study of [70.65] a series-elastic 
clutch was mounted at the knee brace to provide the 
knee flexion action of the gastrocnemius during level- 
ground walking. The mechanism consisted of a toothed 
clutch at the free end of a coil spring (spring stiffness of 
66 500 N/m) that acted as a knee flexor on a polycentric 
knee brace. The brace was integrated into a prosthetic 
socket connected to the ankle-foot prosthesis, as is 
shown in Fig. 70.9a. When the clutch was engaged via 
solenoid action, the free end of the spring locked with 
respect to the socket and the spring stretched as the 


knee straightened. The force developed by the spring 
produced a flexion torque at the knee joint. Conversely, 
when the clutch disengaged no torque was applied at 
the knee joint. The spring acted on the knee joint with 
a moment arm that varied between 0.02 and 0.03 m 
as a function of knee angle. This moment-arm func¬ 
tion was designed so that the apparent knee stiffness 
matched that of the biological knee during level-ground 
walking [70.66]. 

70.3.6 Neuromuscular-Based 
Reflexive Control 

We present a sensing and control scheme for produc¬ 
ing biomimetic positions, torques and impedances at 
the hip, knee, and ankle joints of a powered leg prosthe¬ 
sis, orthosis, or exoskeleton during walking and running 
gaits. In this paradigm, sensory data are collected using 
intrinsic and/or extrinsic sensors. Here intrinsic sens¬ 
ing refers to information collected from sensors located 
on the wearable robotic device, and extrinsic sensing 
refers to all information collected from sensors located 
externally to the wearable device. As an example, in 
the case of a leg prosthesis, a surface electrode for 
the measurement of the electromyographic signal from 
residual limb muscles would be an extrinsic sensor, and 
an inertial measurement unit located on the device itself 
would be classified as an intrinsic sensor. Intrinsic sen¬ 
sors measure device positions, motions, forces, torques, 
pressures, and temperatures, whereas extrinsic sensors 
might comprise such mechanical and temperature sen¬ 
sors positioned external to the wearable device, as well 
as neural sensors for the determination of user motor 
intent. 

In the neuromuscular-based control framework, 
such sensory data are passed to a neuromuscular model 
of human locomotion, such as the Endo and Herr 
model [70.54] presented in Fig. 70.7, which is used to 
compute appropriate joint dynamics for the device to 
provide to the user. The model-based control scheme, 
depicted in Fig. 70.10, relies on data collected from 
both intrinsic and extrinsic mechanical sensors, and 
extrinsic neural sensors used to infer the motor in¬ 
tent of the user, volitionally and/or nonvolitionally. The 
neuromuscular model used to compute desired joint dy¬ 
namics may include muscles modeled in a variety of 
ways. These muscle models (muscle model block in 
Fig. 70.10) may include, for example, a bilinear muscle 
model or a Hill-type muscle model [70.67]. The mea¬ 
sured states of the robotic joints are used to determine 
the internal state (length, velocity) of each of the virtual 
muscle-tendon units of the neuromuscular model using 
morphological information of the muscle moment arms 
about each modeled joint. This geometrical transforma- 
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Fig. 70.10 Block diagram of a neu¬ 
romuscular model-based control 
architecture for a bionic wearable 
robotic leg 


tion occurs within the muscle geometry block shown in 
Fig. 70.10. 

The impedance and force of each virtual muscle are 
additionally governed by the muscle activation, which 
may be determined from a local reflex loop, an ex¬ 
ternal source, or a combination thereof. In the reflex 
case, a feedback loop is implemented where virtual 
muscle force and state are used to produce muscle 
stimulation, which is then filtered to produce muscle ac¬ 
tivation (activation dynamics block in Fig. 70.10). This 
feedback-based control scheme is designed to emulate 
the force feedback and stretch reflex of an intact human 
muscle. This reflexive feedback loop can be a linear or 
nonlinear function of virtual muscle force, length, and 
velocity. For example, the reflexive feedback loop can 
be nonlinear, comprising a threshold prestim parameter, 
as well as force and state gains and exponents, or 

u(t ) =x + y F [F(t — At F )Y 

+ yi[I(t- At,)f +y v [v(t—At v )f* , 

( 70 . 1 ) 

where jc is the prestim parameter, y t are force and state 
gains, z, are force and state exponents, and f,- are corre¬ 
sponding delays. 

In the case where muscle activation is determined 
solely by an external source, a neural sensor might be 
used to provide some estimate of motor intent, which 
is then input to the activation dynamics block where 
a muscle activation is estimated as an input to the 
muscle model. Typically, such measurements of motor 
intent would comprise one or more peripheral neural 
sensors from implants interfacing with nerves and/or 


muscles, but in the most general case, such motor intent 
commands may additionally be measured from central 
brain implants. 

In the combination case, the framework of 
Fig. 70.10 describes a procedure where the reflexive pa¬ 
rameters are modulated by the controller either within 
a single gait cycle, and/or from gait cycle to gait cycle in 
an updating manner, based upon detected variations in 
gait speed and terrain. Extrinsic efferent neural signals 
from muscles and/or peripheral nerves might be used to 
modulate reflexive parameters, such as force and state 
gains and exponents. For example, measured calf mus¬ 
cle EMG in a transtibial amputee has been used to 
modulate the gain of a positive torque feedback during 
the stance period of a walking gait cycle, providing the 
amputee direct volitional control over powered plan¬ 
tar flexion during terminal stance in walking [70.68], 
Clearly, one day it may be possible to stimulate through 
a nerve implant to reflect intrinsic/extrinsic mechanical 
sensory data as an afferent feedback signal to allow the 
user of the wearable robot to better modulate efferent 
neural motor commands for a desired wearable robotic 
dynamical response. 

Once the force of each virtual muscle spanning 
a joint is determined (using the implemented muscle 
model), each muscle force is multiplied by its bio¬ 
logically realistic muscle moment arm and then all 
muscle torque contributions are summed around the 
joint to produce a net torque and impedance estimate. 
The model estimates are then sent to the controller 
(Fig. 70.10) as the desired net torque and impedance 
for each robotic joint. The controller tracks these de¬ 
sired values at each joint to produce human-like joint 
forces and impedances. In the case where the human 
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user’s motor intent is to control device joint position, 
the controller would integrate the desired joint torque 
to achieve a joint position estimate, and then modulate 
device joint position to achieve that desired position. 

By way of example, we present a neuromuscular- 
based control of a transtibial prosthesis (Fig. 70.11). 
For this example, the model-based control framework 
of Fig. 70.10 is used to control the prosthetic appara¬ 
tus. In the study of [70.65], ankle and knee joint states 
of the apparatus were measured and used to provide 
real-time input to a neuromuscular model simulated 
by an on-board microcontroller. The resulting torque 
command from the neuromuscular model was used to 
produce ankle torque while a knee controller adjusted 
the torque produced by the knee brace. This configu¬ 
ration, shown in Fig. 70.11, was set up to enable the 
prosthetic apparatus to behave as if it were a human 
lower leg with reflex-controlled muscles acting at the 
ankle (Fig. 70.12). 

70.3.7 Leg Exoskeletons 

Research has lead to the developement of leg exoskele¬ 
tons with the prime intention to allow otherwise healthy 
individuals to perform difficult tasks more easily or 


enable them to perform tasks that are otherwise im¬ 
possible using purely human strength or skill [70.69]. 
Early developments in leg exoskeletons consisted of 
long bow/leaf springs operating in parallel to the legs 
and were intended to augment running and jump¬ 
ing [70.70]. Each leg spring was engaged during the 
foot contact to effectively transfer the body’s weight 
to the ground and to reduce the forces borne by the 
stance leg. During the aerial phase, the parallel leg 
spring was designed to disengage in order to allow 
the biological leg to freely flex and to enable the 
foot to clear the ground. In 1963, Zaroodny published 
a technical report detailing his work on a powered 
orthopedic supplement intended to augment the load¬ 
carrying abilities of an able-bodied wearer such as 
a soldier [70.71], Other early implementations of ex¬ 
oskeletons, including legs, were attempted by General 
Electric Research with the Hardiman project [70.72], 
by Moore at the Los Alamos National Laboratory, and 
Rosheim [70.73]. However, a major impetus for the re¬ 
cent work in performance augmenting exoskeletons has 
come from the DARPA (Defense Advanced Research 
Projects Agency) program Exoskeletons for Human 
Performance Augmentation (EHPA) [70.74], the out¬ 
comes of which were different working exoskeletons. 
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Fig. 70.11 Labeled schematic and control architecture for the prosthetic apparatus. Rotary elements in the ankle-foot 
prosthesis are shown as linear equivalents in the model schematic to improve clarity. In the control schematic, the parallel 
spring contribution to prosthesis ankle torque, r p , was subtracted from the desired ankle torque command from the 
neuromuscular model, tj, to obtain the desired SEA torque, r ( /. A motor current command 7 mo t was obtained by dividing 
the desired SEA torque by the motor torque constant, K t . The knee clutch was engaged via the solenoid depending on 
knee state as obtained from the knee potentiometer. For further details on the mechatronic and control design, see [70.65] 
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Fig. 70.12 (a) Comparison of prosthesis ankle and knee angles and torques during the clinical trials (measured) with 
those from a height- and weight-matched subject with intact limbs (biological). The biological values are the thick solid 
lines (with shaded errors) in each plot while the dashed lines are the values measured on the prosthesis. In the ankle torque 
plot the commanded torque is shown as a thinner solid line, again with shaded error bars. The knee torque plot compares 
the torque provided by the clutch-spring mechanism to that provided by the natural gastrocnemius in simulation. The 
vertical line indicates toe-off in each plot, (b) Energy output of the ankle across gait speed. Shown are biological data, 
net work as commanded by the ankle-foot prosthesis during clinical trials, and measured net work during the clinical 
trials. For further details on experimental methodology and results, see [70.65] 


In this framework, the Berkely Exoskeleton (BLEEX) 
was the first example of an energetically autonomous 
system [70.75]. BLEEX features 3-DOF at the hip, 1 
at the knee, and 3 at the ankle. Of these, four are ac¬ 
tuated: hip flexion/extension, hip abduction/adduction, 
knee flexion/extension, and ankle inversion/eversion. 
Of the unactuated joints, the ankle inversion/eversion 
and hip rotation joints are spring loaded, and the an¬ 
kle rotation joint is free-spinning [70.76]. The BLEEX 
exoskeleton [70.77] was designed with linear hydraulic 
actuators since they were the smallest actuation option 
available based on their high specific power (ratio of 
actuator power to actuator weight) [70.78,79]. How¬ 
ever, a further study determined that electric motor 
actuation significantly decreased power consumption 
during level walking in comparison to hydraulic actu¬ 
ators [70.80, 81]. The weight of the implementation of 
the electrically actuated joint, however, was approxi¬ 
mately twice that of their hydraulically actuated joint 
(4.1 versus 2.1 kg). Significant effort was made in de¬ 
veloping a hybrid hydraulic-electric portable power 
supply [70.82]. In terms of performance, users wear¬ 
ing BLEEX can reportedly support a load of up to 
75 kg while walking at 0.9 m/s and can walk at speeds 
of up to 1.3 m/s without the load. A second genera¬ 
tion of the Berkeley exoskeleton is currently in testing. 
A laboratory spin-off company called Berkeley Bion¬ 


ics (Berkeley, CA) has been created in order to market 
exoskeleton technology. 

A quasi-passive exoskeleton concept has been ad¬ 
vanced in the Biomechatronics Group at the Mas¬ 
sachusetts Institute of Technology (MIT) Media Lab¬ 
oratory that seeks to exploit the passive dynamics of 
human walking in order to create lighter and more 
efficient exoskeleton devices. The MIT exoskeleton em¬ 
ploys a quasi-passive design that does not use any actu¬ 
ators for adding power at the joints. Instead, the design 
relies completely on the controlled release of energy 
stored in springs during the (negative power) phases 
of the walking gait [70.83]. The 3-DOF hip employs 
a spring loaded joint in the flexion/extension direction 
that stores energy during extension that is released dur¬ 
ing flexion. The hip abduction/adduction direction is 
also spring loaded, but only to counter the moment 
induced by the backpack load. Additionally, a cam 
mechanism was incorporated into the hip to compensate 
for the relative change in length between the thigh of 
the exoskeleton and the user due to the joint offset dur¬ 
ing abduction/adduction. Additionally, spring loaded 
hip rotation and ankle rotation joints were included to 
allow nonsagittal plane limb movements. The knee of 
the MIT exoskeleton consists of a magneto-rheological 
variable damper that is controlled to dissipate energy 
at appropriate levels throughout the gait cycle. For the 

























Human-Robot Augmentation I 70.4 Whole Body Wearable Systems 1889 



Fig. 70.13 MIT exoskeleton during metabolic testing 


ankle, separate springs for dorsi and plantar flexion 
are implemented in order to capture different behav¬ 
iors during these two stages of motion, and store/release 
the optimum amount of energy. Without a payload, the 
exoskeleton weighs 11.7 kg and requires only 2 W of 
electrical power during loaded walking. This power 
is used mainly to control the variable damper at the 
knee. The first studies on the metabolic cost associ¬ 
ated with walking under the aid of an exoskeleton have 
been reported by using the MIT exoskeleton [70.84] 
(Fig. 70.13). 


70.4 Whole Body Wearable Systems 

Despite the considerable interest and impressive ad¬ 
vancements in the field, so far only few researchers 
have attempted the development of full-body EHPA 
(FB-EHPA), in particular fully actuated systems for 
heavy-duty applications, and the available scientific 
literature is quite limited [70.8,83,85,91-99]. This 
section reviews the general design issues concerning 
electro-mechanical design of FB-EHPA. The general 
design issues are analyzed considering and presenting 
a series of specific designs. 

70.4.1 General Issues 

Kinematic Architecture 

The kinematic scheme of FB-EHPA is a fundamental 
aspect that strongly affects the basic functionalities and 
capabilities of such machines. Just as other wearable 


Further experimental work with the MIT quasi¬ 
passive exoskeleton showed a significant reduction in 
metabolic cost of walking versus the same exoskeleton 
without the springs at the hip and ankle and the variable 
damper at the knee, demonstrating the utility of quasi¬ 
passive elements. 

Other examples of leg exoskeletons comprehend 
the hybrid assistive leg developed at the University 
of Tsukuba by Sankai [70.85] that is targeted for 
both performance-augmenting and rehabilitative pur¬ 
poses. At the Kanagawa Institute of Technology in 
Japan, researchers have developed an exoskeleton for 
the purpose of assisting nurses during patient trans¬ 
fer [70.86]. Yobotics, Inc. (Cincinnati, OH) [70.87] 
developed a simple exoskeleton for adding power at 
the knee to assist in stair climbing and squatting during 
load-carrying tasks [70.78]. Researchers in the Depart¬ 
ments of Mechanical Engineering and Physical Therapy 
at the University of Delaware have developed a passive 
leg orthosis that is designed to reduce the forces of grav¬ 
ity on the patient during walking, thus easing the effort 
required for locomotion [70.88]. Caldwell has devel¬ 
oped a 10-DOF lower limb exoskeleton device [70.81]: 
actuation is provided to the flexion/extension directions 
of the hip, knee, and eankle, and abduction/adduction 
of the hip via pneumatic muscle actuators. Recently, 
few leg exoskeleton systems have been commercial¬ 
ized worldwide by different companies: the Re Walk 
lower limb exoskeleton developed by Argo Medical 
Technologies, Israel [70.89] for elderly/paraplegic as¬ 
sistance, and the system called REX developed by Rex 
Bionics, New Zealand [70.90]. 


robots, the kinematic design is strongly influenced by 
the close proximity of robot linkages and human limbs. 
Kinematics of an FB-EHPA must be designed in order 
to be compatible with: 

• The execution of common human movements: 
walking, squatting, lifting, turning, etc. 

• Requirements for the execution of specific tasks: 
raising weights, handling specifically shaped ob¬ 
jects, running, etc. 

• Encumbrance and spatial limitation 

• Possible collisions with the operator body and self¬ 
interferences. 

Common solutions for the existing FB-EHPA are 
based on anthropomorphic kinematics since this kind 
of solution guarantees: 
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• The best workspace matching with reduced encum¬ 
brances 

• The most simple strategy to avoid self/body colli¬ 
sions 

• The execution of common human movements 

• Mass distribution that is compatible with human ex¬ 
ecuted tasks. 

A full anthropomorphic design that allows replicat¬ 
ing all the human DOFs could be desirable; however, 
the complexity of such a system would be far above 
what is technically affordable. In this sense, a drastic 
simplification is required to reduce complexity without 
affecting the basic functional requirements of the ma¬ 
chine. This type of simplification is assumed in many 
of the available FB-EHPA. 

70.4.2 Specific Design 

According to a historical perspective, the first techni¬ 
cal concepts of FB-EHPA were proposed in 1956 by 
Lent, who proposed an inflatable space suit with pow¬ 
ered joints to assist the wearer during the flexion of suit 
extremities stiffened by the internal pressure [70.92], 
Lately, in 1966, Mizen proposed, although without real¬ 
izing it, the first man amplifier for civilian and military 
applications [70.93]. 

The first practical investigation was instead at¬ 
tempted by General Electric between 1965 and 1971, 
by developing the Hardiman [70.94] that is briefly 
described in the following section. Researches on 
FB-EHPA were resumed in 1996, when Snyder and 
Kazerooni proposed a novel under-actuated material 
handling system with 6 passive joints (2 for the legs and 
4 for the arms) and 12 electrically actuated joints (6 for 
the legs and 6 for the arms) under force-feedback con¬ 
trol [70.95]. System arms and legs were implemented 
and tested separately. 

Following this renewed interest, from 2002 to 
2009, three different under-actuated FB-EHPA have 
been developed in Japan for medium-duty applica¬ 
tions, such as nursing care [70.83,97], disabled or 
elderly assistance [70.85], and agriculture [70.100]. 
The best performing and most renowned system is the 
hybrid assistive limb (HAL) [70.85]. Still in Japan, 
in 2009, Panasonic-Activelink presented the POWER 
LOADER, a heavy-duty FB-EHPA for material han¬ 
dling [70.98]. In 2010, Raytheon-Sarcos unveiled the 
XOS2 [70.99, 101], which is one of the most advanced 
and performing medium-duty hydraulically actuated 
FB-EHPA for military/logistics applications. One of the 
latest FB-EHPA is called Body Extender (BE) [70.102] 
and has been realized at the PERCRO Laboratory, 


Scuola Superiore Sant’Anna (Italy). The BE is a full 
electrical machine developed for heavyweight material 
handling. In 2012, the French company RB3D has pre¬ 
sented HERCULE, an electrically actuated FB-EHPA 
with legs similar to those of HAL, but with differ¬ 
ent arm kinematics, and designed for lifting 20 kg 
with each arm at full horizontal extension. HERCULE 
has been developed for the French Ministry of De¬ 
fense and detailed specifications are not available at 
present. 

In this section, an overview of the basic features of 
some of the most relevant and advanced existing sys¬ 
tems is provided. 

Hardiman 

Hardiman I is the first prototype of FB-EHPA and 
has been developed between 1965 and 1971 by Gen¬ 
eral Electric Research (Schenectady, NY). The project 
aimed at developing a high-power machine for handling 
heavyweight materials in unstructured environments. 
The project was jointly supported by Engineering 
Psychology Programs-ONR, Naval Air Systems Com¬ 
mand, and the Army Mobility Equipment Research and 
Development Center (Fort Belvoir). 

The project results have been reported by Fick and 
Makinson in [70.94]. Hardiman I has a structure based 
on a 30-DOF kinematics including: 

• 2-DOF for each end-effectors: two flexion move¬ 
ments of a two-phalanges finger 

• 7-DOF for each arm: wrist flexion, forearm rotation, 
elbow, upper arm rotation, shoulder flexion, back 
flexion and arm abduction/adduction 

• 4-DOF for each leg: hip abduction/adduction, hip 
flexion, knee flexion and ankle flexion 

• 2-DOF for each foot: ankle inversion and ankle ro¬ 
tation. 

The actuation system employs hydraulic actuators 
that require an external hydraulic circuit at 206 bar 
(3000 psi) with a power of 18.6 kW (25 hp). The global 
weight of the system is 680 kg (1500 lbs) and the 
exoskeleton is rated for a lift capability of 6800 N 
(1500 lbs). The control of the hand is completely hy¬ 
dromechanical (implemented with no electronic com¬ 
ponents), while the arms, legs, and feet are controlled 
through electrohydraulic systems. Hardiman I did not 
succeed to be tested in its full functionalities. In par¬ 
ticular, the developers reported several problems in the 
implementation of the controller for the lower limbs, 
mainly also due to the lack of reliable electronic com¬ 
ponents and reduced computation power characterizing 
the technology at that time. 
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Fig.70.14a,b Picture of the Body Extender (BE): (a) sys¬ 
tem worn by a user; (b) rear view (after [70.103]) 


HAL: Hybrid Assistive Limb 
HAL is a series of full-body exoskeleton systems, the 
development started in 1996 and the current version, 
HAL-5, is the first commercial version. HAL has been 
developed by Sankai [70.85] and his research group of 
the University of Tsukuba and commercialized by the 
company Cyberdyne, Inc. (Tsukuba, Japan) [70.103]. 

It is realized in two versions, the first one is for 
lower limbs only and the second includes the upper 
limbs. The exoskeleton is developed with the aim to 
expand, augment, and support human physical capabil¬ 
ity in the field of medical welfare, heavy work support, 
and entertainment. The kinematic structure features 6 
passive joints (2 for the legs and 4 for the arms) and 
8 electrically actuated joints (6 for the legs and 6 for 
the arms). HAL is commanded via a specially designed 
controller which takes into account the wearer’s mus¬ 
cular activity through the employment of bioelectrical 
signals from EMG sensors that are disposed on wearer’s 
skin. HAL weighs 23 kg in the full-body version, in¬ 
cluding batteries for an autonomy of 2h and 40 min. 
Although the lifting capabilities are not specified, it is 
reported in [70.97] that laboratory tests have been con¬ 
ducted with a payload of 15 kg with each arm at full 
horizontal extension; however, it requires locking of the 
passive joints and an additional wrist support. HAL is 
currently being experimentally tested in clinics and the 
research group is currently working on improving the 
system with new control algorithms. 

POWER LOADER 

POWER LOADER is a full-body exoskeleton devel¬ 
oped by Activelink Ltd, a spin-off of Panasonic Ltd, 
Japan. The system development was started in 2003 
and the first working prototype reported in literature 
has been realized in 2010. POWER LOADER has been 


developed for rescue operation in natural disaster sce¬ 
narios such as big earthquakes. The mechanical design 
is based on an anthropomorphic design only for the 
arms. The legs have an anthropomorphic structure. The 
knee is back rotated [70.98] in order to leave more free 
space around the user legs. The actuation is achieved 
through electric motors and drivers. 

PERCR0 Body Extender 

One of the latest full-body exoskeleton has been real¬ 
ized at the PERCRO Laboratory of Scuola Superiore 
Sant’Anna, Italy (Fig. 70.14). The developed FB-EHPA 
has been named as the body extender (BE) [70.104]. 
The BE comprises two identical legs with 6-DOF each, 
whose kinematics is isomorphic to the wearer’s kine¬ 
matics, and two identical arms with 5-DOF each. All 
22-DOF are powered by modular highly efficient ac¬ 
tuation units; this solution allows to reduce the power 
consumption, as well as to minimize the costs and ef¬ 
forts related to its production and maintenance. The BE 
weighs 160 kg, can provide a maximum continuous us¬ 
able power of 16.5 kW, is capable of lifting 50 kg with 
each arm at full horizontal extension, and is designed 
for transporting loads up to 100 kg at a walking speed 
of 0.5 m/s. Potential application of this system is in 
earthquake disaster relief operations, where search and 
rescue teams need to walk across tight spaces over de¬ 
bris and to remove nibble in an attempt to find, release, 
and transport survivors. The system has been integrated 
and fully tested in 2010 and it is currently being im¬ 
proved in terms of energy efficiency and stability on 
uneven terrains. 

Commercial Systems 

(Uncovered by Scientific Literature) 

In the past years, two advanced commercial systems 
have been unveiled. The first one is XOS II, which is 
the second version of a full-body exoskeleton devel¬ 
oped by a US company, Sarcos-Raytheon [70.99, 101, 
105], The first version of the system was completed 
in 2008 and the second version has been presented in 
2010. The XOS II has been developed aiming at lo¬ 
gistic applications in the military field. It is based on 
an anthropomorphic kinematics equipped with 24-DOF 
actuated with hydraulic systems. It weighs 95 kg and 
has a lifting capability of 230 N. The company is cur¬ 
rently working on improving the energy efficiency for 
the realization of an autonomous (untethered) version 
of the system. The second device is called HERCULES 
and has been presented in 2012 by RB3D (France). 
It is an electric-driven system that has a lot of sim¬ 
ilarities with the HAL-5 exoskeleton. The details of 
HERCULES have not yet been disclosed [70.106]. 
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70.5 Control of Human-Robot Augmentation Systems 


From a control point of view, the functionality of 
a power augmentation system should simultaneously 
provide (a) force amplification and (b) support to nat¬ 
ural locomotion. 

Hence, the control of exoskeleton for human power 
augmentation (EHPA) can be split into two major sub¬ 
systems: lower- and upper-body designs. The first sub¬ 
system cooperates with a human gait, while sustaining 
much of the body loads, keeping overall equilibrium 
and following the human motion with transparent and 
natural reflection forces. The second subsystem en¬ 
hances human manipulation abilities through the mul¬ 
tiplication of the arm forces during grasping operations 
of heavy loads. 

Both components have to respect the following 
requisites: 


2. Transparency : The system should be highly re¬ 
versible (through mechanical design or sensor aug¬ 
mentation) in order to allow a correct perception of 
the overall interaction with the environment and the 
proper generation of the control actions. 

3. Regular force amplification : The gain ratio of the 
human force amplification should be kept as regular 
as possible in all interaction conditions, including 
a large bandwidth of interaction, different postures, 
and different environment interactions. 

4. Fast response : The bandwidth and the satura¬ 
tion values of the position/velocity/force controllers 
should be as close as possible to their respective 
ranges used in natural motions and interaction. 

70.5.1 A Historical Perspective 


1. Stability : The stability margin of the (linearized) 
control system should be large enough to cope with 
the environment and operator uncertainties, and to 
avoid uncompensated oscillations entering into res¬ 
onance with the human postural control. 
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Fig. 70.15 The control of the force amplifier joint (after [70.39]). 
The upper scheme represents the mechanical equivalent circuit for 
the control of the elbow joint, and the above equation represents 
the associated force balance equation. The lower scheme is a block 
diagram that represents how the components are arranged together 
and how the feedback is used to control the joint actuation 


Yagn [70.70], first submitted a series of patents for fa¬ 
cilitating human walking and running. At that time, 
the mechanism was limited to a series of springs and 
clutches, which altered the feet compliance during 
walking and used the potential energy of the springs to 
have a more efficient walk. 

The use of active orthoses, through hydraulic servo¬ 
valves was proposed by Filippi in 1937 [70.107]. The 
proposed system was a knee orthosis whose motion in 
the sagittal plane is powered by ahydraulic piston. At 
that time, there was not much advancement on elec¬ 
tric control of hydraulic systems and the joint control 
was supposed being regulated mechanically. A crank 
located on the hip moved a cam that, on its own, winded 
a torsional spring on the knee. The other extremity of 
the spring was used to regulate the piston flux. 

The first documented attempt to develop a pow¬ 
ered suit that amplifies human force was performed by 
Clark et al. in the early 1960s [70.39]. At that time, they 
proposed to design a man amplifier using hydraulic am¬ 
plification. In Fig. 70.15, it is represented by the device 
interaction model and the basic control loop they pro¬ 
posed. They have limited the investigation to one single 
joint of movement. 

In Clark et al.’s work, each joint was regulated with 
an independent position controller that was actuated by 
mechanical pistons placed closed to the human body. 
The control was based on an admittance scheme that 
translates user commands into joint velocities. This 
scheme had no opportunity to show the reflection of 
contact forces. 

The design and control of whole-body active ex¬ 
oskeletons started in 1965 with a joint Naval Army 
program who supported General Electrics for the de¬ 
sign of a whole-body EHPA (Hardiman I). The power 
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augmentation target was fixed to an amplification fac¬ 
tor (robot versus human) 25 : 1 having a nominal total 
force about 10000N. 

The proposed device had 28 independent DOFs plus 
two grasping tools with one additional DOF each. 

The control concept design of Hardiman was very 
ambitious. In the first attempt, the designer developed 
a set of mechanical servo-valves (as shown for the mas¬ 
ter hand controller in Fig. 70.16) placed in contact with 
the user’s skin which responded to local pressure by de¬ 
flecting the flux of hydraulic circuits [70.96]. 

The approach worked well only for simplified kine¬ 
matics of the mechanical hand, while the dynamic 
interference between the serial kinematics of the arm 
link required the use of electrohydraulic circuits and 
that of a computer-developed dynamical model. 

The project development stopped due to major limi¬ 
tations in the design and control of the lower leg. During 
the development, the authors only have tried unilateral 
electrohydraulic servos to track the motion of the user 
legs. The major development issue was related to the 
control of the leg for the body balance. The developers 
devised that the leg functionality changes between the 
stancing and the swinging phase, but the use of switches 
to detect the current leg phase demonstrated to be in¬ 
effective to validly cope with equilibrium and stability 
issues. Other issues were instead related to the power 
supply line (45 kW needed by the prototype) and to the 
kinematics differences between the master and the slave 
arm, which makes some motion unnatural, risky, and 
sometimes unstable for the driving user. 

Few years later, Vukobratovic et al. [70.108] started 
to develop an exoskeleton to assist injured people 
(mostly paraplegic) to recover basic walking abilities. 
That work shares several features that are being imple- 



Fig. 70.16 The master hand controllers. A set of tinklers 
were moved according the grasping force of the user and 
drive the hydraulic circuit of the slave hand with a design 
amplification factor of 25:1 (after [70.96]) 


mented in present power extenders. Vukobratovic et al. 
were able to define the following relevant features of 
control and stabilization loops for walking that are still 
valid today: 

• A model reduction approach 

• The method of prescribed synergies 

• The use of zero moment point (ZMP) to derive the 
dynamic equation of motion 

• The use of left-right symmetries and nominal gait 
cycle 

• The design of a model-based analog motion com¬ 
pensator that provides to create motion synergies 
through a set of servomotors. 

Motion synergies were computed using a nominal 
gait trajectory (step period = 1, step size coefficient = 
1) in a reduced phase space as a reference to derive 
in advance the weight, the inertia, and the friction 
compensation torques that have to be exerted at each 
joint. 

Figure 70.17 describes the high-level functional 
architecture of the controller. The adaptation for differ¬ 
ent gait conditions (from 1 to 5 s) was modulated on 
the precomputed trajectories through a couple of co¬ 
ordinated sequencers (time-base generators, TBG) that 
convert the absolute time into the loop phase required 
to determine compensation components. 

The overall feedback also includes two additional 
feedback components, which use force sensors in the 
shoes to compute torque perturbation around the ZMP. 
These disturbances appear when the exoskeleton rotates 
around the leg in the stance phase or when a change of 
acceleration (step length/period) alters the gait from the 
nominal cycle. 

In both cases, the author derived, through lineariza¬ 
tion of the dynamic model along the nominal cycle, 
a couple of static gains that map the sense forces into 
corrective torques that are added to the nominal hip 
torques. 

Vukobratovic et al.’s exoskeleton worked well only 
for limit cycles which were close to the nominal gait 
trajectory it was designed for. Both Vukobratovic’s 
exoskeleton and the experience derived form the Hardi¬ 
man project showed that the control technology avail¬ 
able at that time was premature to cope with the nonlin¬ 
ear, dynamic coupling effects that appears in walking 
and manipulation. 

A renewed effort in research was performed by 
researchers at Berkeley University who in the mid¬ 
dle of the 1980s promoted projects, in general called 
the human extenders, to develop arm exoskeletons that 
augment the arm force. Early extender projects were 


Part G | 70.5 



Part G | 70.5 


1894 Part G 


Robots and Humans 


TBGL 


Variable gaint I 
speed 


Analog 
(precomputed) 
Synergy 
reference 
Generator 
Ankle left 

Analog 

(precomputed) 

Synergy 

reference 

Generator 

Knee left 

Analog 
(precomputed) 
Synergy 
reference 
Generator hip 
Left 


Ankle motor 

Knee motor I 

Hip motor 

servo 

servo 

servo 


Force 

- Feedback from !- 
shoes sensors I 


Analog 
(precomputed) 
Synergy 
reference 
Generator hip 
Right 


Hip motor 


TBGR 


Analog 
(precomputed) 
Synergy 
reference 
Generator 
Knee right 


Knee motor 


Analog 
(precomputed) 
Synergy 
reference 
Generator 
Ankle right 


Hip motor 


Fig. 70.17 The control structure of the 1974 Vokubratovic et al.’s exoskeletons (after [70.108]). The active device was 
powered with six pneumatic actuators whose references and feedback are determined through three different components: 
servo feedback, prescribed synergies, and force feedback for sagittal and lateral stabilization 



Fig. 70.18 The basic extender controller (after [70.109]). The scheme represents the extender controller as proposed 
by Kazerooni in 1989. The controller is based on an internal velocity controller. A given force amplifier gain F ga 
determines the forces that are compensated dynamically with respect to the force shown to the operator 


based on the combination of a robust velocity con¬ 
troller in feedback with a force sensor that detects user 
inputs. 

Figure 70.18 shows the working principle of the 
amplification loop. With respect to previous attempts, 
here the use of the digital controller helped to im¬ 
plement more sophisticated feedback controllers that 
cope with the multi-input multioutput system model. 
The overall stability was proven through two methods: 
(1) a linear analysis, performed through the multi- 
variable Nyquist criterion on the linearized model (2) 
and an application of the small gain theorem that 
provides sufficient conditions for stability in the case 


of the nonlinear model interacting with the unstruc¬ 
tured environment. However, the stability conditions 
derived in [70.2] through the small gain theorem where 
highly restrictive and required the amplification gain 
to be less than 1. Subsequent author’s work intro¬ 
duced different criteria (than fully unstructured in¬ 
teraction), such as typical operator and environment 
impedance ranges, and stability margins. Under such 
conditions [70.110] and using the /z-synthesis, the au¬ 
thor demonstrated robust stability under higher force 
amplification gains. 

An example of the amplification goal achievement 
with this technique and a 3 DOF arm was reported 
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in [70. 11 1] as the maneuver bandwidth of 2 Hz, the ver¬ 
tical force amplification ratio 7:1, and the horizontal 
force amplification ratio 5:1. 

70.5.2 Power Augmentation 

through Proprioceptive Sensors 

This type of power augmentation systems use only sen¬ 
sors that are completely internal to the exoskeleton 
structures. These sensors have no direct contact with 
the human operator (e.g., motor or joint torque/position 
sensors). 

One benefit of this approach is the reduced design 
complexity through the use of more conventional torque 
sensors and the improved stability of the reading even 
in the case of multiple contacts between the user and 
the exoskeleton structure. 

As a side effect, there is no direct measurement 
of the force effectively felt by the user. The interac¬ 
tion controller decides the exoskeleton motion policy 
based on the estimation of the interaction force with the 
user. 

This approach is used in the control of the Berkeley 
Lower Extremity Exoskeleton [70.5], and most of the 
intelligent assisting devices (IADs) [70.1 12-1 14]. 

Given the Lagrange formulation of the exoskeleton, 
the basic of this approach is to let the control providing 
a predefined percentage of the force/torque required for 
the motion 

M (q)q + C (q. q)q + G(</) + G l (q) = 
e+I]JcF c , 

< C ( 70 . 2 ) 

Q = jJ(0)F m , 

q=MO)0 , 

where q is the joint coordinates vector, M, C, and G, 
are the exoskeleton mass, Coriolis, and gravity ma¬ 
trices, respectively, Gl is the manipulated load, Q is 
the motor-torques vector reduced to the joints J C ,F C 
refers to the user contact forces and the respective Ja- 
cobians computed in the points of contact, and J t and 
F m represent the transmission Jacobian and the motor 
force/torque vector, respectively, and 6 is the motor- 
coordinates vector. 

In order to properly work, this approach also sup¬ 
poses to have an almost accurate model of the mechan¬ 
ical design (M. C, G, J c , J t ) and the load to manipulate 
(Gl)- In these conditions, the candidate control law is 

F m = (Ji r r 1 (G(< ? ) + (l-/r 1 )GL 

+ (\-a~ 1 )(M(q)q + C(q,q)q)) . ( 70 . 3 ) 


By inserting (70.4) in (70.2), we have 

a~ 1 (M(q)q + C(q,q)q) +P^Gtiq) 

= JcF c + s(q) 

c 

■ e(?) = (M( 9 )-M( 9 ))jj+(C( ? .«) 

-C(q,q))q+(G(q)-G(q)) 

+ (Gl(< 7 )-Gl(< 7 )), 

( 70 . 4 ) 

where the model error is expected to converge toward 
zero. 

Equation (70.4) shows how the contact forces per¬ 
ceived by the user can be reduced by a static (/J) and/or 
a dynamic (a) amplification factor. As a matter of fact, 
if the simplification operates correctly (e —> 0), the pi¬ 
lot will only feel on his own body a percentage of the 
carried load (y3 1 ) plus a percentage of the exoskeleton 
dynamic (a 1 ). 

In the case of a walking exoskeleton, the above con¬ 
trol model is switched according to the stancing foot, or 
in the case of double stancing (the weight is distributed 
on both feet), a simplified zero-moment-point (ZMP) 
rule is applied [70.1 15] 

' tHTR*TR = «TlATL , 

, 'WTR + m TL = m L , ( 70 5 ) 

-*TR + -*TL 

-^- = x s > 

2 S 

where jcjr and ajl are the stancing position of the right 
and left feet, respectively, x g is the projection of the cen¬ 
ter of mass on the walking plane, is the overall load 
mass, and mi r and ;«tl are the mass components to be 
distributed on the right and left control models, respec¬ 
tively. 

Equation (70.5) is used as a means to separate 
the closed-loop kinematics into two (quasi) symmetri¬ 
cal equations, similar to (70.2), that only refer to the 
left- and right-leg dynamics. The load is consequently 
distributed among the two legs, thus generating two de¬ 
coupled control models, one for each leg. 


Discussion 70.1 

Experiments on such a kind of controller were per¬ 
formed at UC Berkeley. In [70.75], the system proved 
to be efficient to compensate completely the weight 
(/3 —1 = 0) with a good dynamic scale power amplifi¬ 
cation factor (a = 10), and that the resulting system is 
usable for walking at an average speed of 1.3 m/s with 
a transported payload of 70 kg (including the exoskele¬ 
ton weight). 
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Fig. 70.19 The basic EMG controller (after [70. 116]). The myo-electric signals were processed to derive a control signal 
through a sequence of (T) high-pass filter; (2) signal rectification (absolute value); (3) low-pass filtering; (4) normalization 
versus a nominal setup isometric contraction. The sign of the contraction is detected as a combination of pure agonistic- 
antagonistic contractions 


The proposed design, however, suffers from a num¬ 
ber of drawbacks that limit this control architecture 
being used in a more flexible interaction environment: 

• The measurement of interaction torques made at 
the joint level makes it indistinguishable, whose 
interaction caused the reading, be it a voluntary 
movement provided by the user or an unvoluntary 
interaction provided by the environment. In both 
cases, the control amplifies these effects regardless 
of the fact that the original stimuli were provided by 
the user. 

• The sensor and the mechanical model assumed in 
the discussed work only deal with a rigid me¬ 
chanical design model whose structures risk to be 
cumbersome and dangerous when the load to be 
carried/manipulated is high. Flexible designs may 
manipulate higher loads at the cost of more com¬ 
plex control design [70.38]. 

• The robustness of the control is demanded to the 
accuracy of the model and the design of the com¬ 
pensation gains. When no gain is programmed 
(a = /I = 1), the feedback is zeroed and the user 
perceives all the passive effects of the dynamics. In¬ 
creasing these gains lowers the stability margin in 
a way that is dependent on the quality of the model 
approximation. 


70.5.3 EMG Force Control 

and Power Augmentation 

Notwithstanding the fact that EMG-based arms were al¬ 
ready developed in the early 1980s, power-augmented 
systems based on EMG signals only appeared at the 
beginning of the 2000s. In 2001, Rosen et al. [70.116] 
proposed a powered exoskeleton system based on the 
myosignal analysis (as shown in Fig 70.19). 

In order to associate a map between the myo¬ 
electric signals and the amount of exerted force, Rosen 
first detected the level of activation, then he applied 
a basic muscle contraction/elongation model as de¬ 
scribed by Hill [70.117] during his investigation on 
tetanic contraction and by Winter [70.60] for antagonis¬ 
tic muscle analysis. According to Hill’s model, the force 
correlates the combined effects of muscle activation, 
muscle elongation, and muscle speed to a well-known 
state equation 

y= »Fofa + ») | (;06| 

(vFo + avo) 

where F is the force exerted by the muscle, v is con¬ 
traction velocity of the muscle, Fq is the maximum 
isometric force generated by the muscle, Vo is the 
muscle maximum velocity, and a is the coefficient of 
shortening heat. 
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Such a model has been applied to all involved mus¬ 
cles Mf = Mf(U emg , 6 m , 8 m ), to determine the moment 
Mf induced on the elbow joint angle 6 m , by the muscle 
fibers /. In addition to the muscle bio-signal, the con¬ 
trol closed loop used two more data from sensors: (1) 
a measurement of the external load and (2) the overall 
momentum applied by the operator on the exoskeleton. 

The underlying basic idea is that the muscle signal 
operates as an external loop that provides a reference 
for an internal moment controller. The three gains in 
the controller K\ . K^. K u define, respectively, the inter¬ 
nal loop force gain K t \ versus K L , and the overall EMG 
signal tracking bandwidth. 

In a similar effort to the Rosen elbow extender, 
Kawaie t al. [70.118] proposeda 1 -DOF-actuated wear¬ 
able device that lowers the internal backbone strain 
during pick-up tasks in order to prevent from lumbago. 
The device senses the average back angle (measured 
through a potentiometer) and the EMG strain on the 
thigh muscles. The motor control for the device is deter¬ 
mined through a state machine based on a typical load 
pick-up and release procedure. The threshold values 
for switching between phases were determined through 
a test with a nominal load. 

In [70.119] and [70.82], the EMG data are em¬ 
ployed to control the HAL-3, walking assistant, and 
exoskeleton. The control algorithm is a combination of 
a finite-state machine with a torque control. Two force 
sensors placed in soles of the shoes serve to determine 
the floor-reaction forces and to switch among phases. 
Each phase is then used to determine in which mode the 
user knee is moving (active, passive or free). The joint 
motors are consequently controlled using Table 70.1 

In different phases of motion, the joint control 
adopts a computed torque approach plus two compo¬ 
nents proportional to the myo-signals of flexion and 
extension 


joints = XflEMGfl — k'cxEMGex 
+ (I — M)0 + (D — B)0 
+ C(0,0) + K(0 o-0), ( 70 . 7 ) 

Table 70.1 The HAL dual-phase model. The phase 1 re¬ 
lates to the swinging leg and phase 2 relates to the stancing 
leg. In each phase, the table indicates the type of control to 
be implemented on the relative exoskeleton joint 



Phase 1 

Phase 2 

Joint 

Direction 

Mode 

Direction 

Mode 

Hip 

Flexion 

Active 

Extension 

Active 


Flexion 

Flexion 

Knee 

4 

Free 

4 

Passive 


Extension 

Extension 


where I and D are the inertia and viscous coeffi¬ 
cients, respectively, as estimated through a recursive 
least squares auto regressive estimator (ARX) identifi¬ 
cation, C includes all Coriolis and gravity effects, M , B 
and K represent the target inertia, impedance to show to 
the operator. The term r = XflEMGfl — K cx EMG CX rep¬ 
resents the estimated muscle torque whose conversion 
factors were determined through experimental tests. 

The measure assistance level is close to 60% dur¬ 
ing flexion activities and to 85% during extension. In 
its most recent version, the HAL-5 adds contributes 
from accelerometers and gyroscope mounted on the 
backpack for better posture estimation [70.120]. HAL- 
5 extends the principle of operation of the lower body 
control to the user arm, by allowing the user to hold and 
carry an additional weight of 40 kg. HAL-5 extends the 
control with a greater number of phases (swing, land¬ 
ing, and support) and provides different control gains to 
optimize each phase [70.121]. 

The above EMG controllers only work fine when 
the developers are able to find two antagonistic muscles 
whose flexions and extensions can be directly related to 
the overall force exerted on a user joint (e.g., the bicepts 
and tricepts for the elbow joint). If this is not the case, 
Kiguchi et al. showed out how fuzzy controllers may 
estimate joint torques from several torques. In [70.122], 
they showed how to decode shoulder torques by com¬ 
bining the EMG signals recorded from eight different 
muscle sources. Such a fuzzy network has been coupled 
in the control of a 3-DOF mobile exoskeleton [70.123] 
to assist motion in highly impaired people. 

70.5.4 Power Augmentation 

through Exteroceptive Sensors 

High-force power augmentation systems require a dif¬ 
ferent approach with respect to the proprioceptive and 
the EMG sensing approaches. In such devices, the flex¬ 
ibility of the structure and the high torques required 
at joints prevent the use of indirect measures of the 
force/torque exchanged between the device and the 
user. Moreover, the Hardiman approach, which inte¬ 
grated a master exoskeleton and an outfitting slave 
exoskeleton as a unique teleoperation system proved 
not to be functional [70.94], 

Two systems have proved so far to reach this tar¬ 
get in an operational and fully functional scenario: the 
Sarcos/Raytheon XOS Exoskeleton for the arms and 
legs [70.105] and the BE developed by PERCRO at 
Scuola Superiore Sant’Anna. Both of them are based 
on a similar working principle: the user body is directly 
attached to the exoskeleton in a predefined numbers 
of points, e.g. the two feet, the hands’ handles, and 
a shoulder harness for the BE; and additional interac- 
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tion at the pelvis for the XOS. The interaction forces 
exchanged between the exoskeleton and the user are 
measured at these points and used to simultaneously 
control equilibrium, force amplification, and motion 
control. 

On the other side, the two systems differ for a num¬ 
ber of technical details: the XOS design is light, ad¬ 
herent to the user body and based on rotary-hydraulic 
actuators, the BE instead mimics the weight distribu¬ 
tion of the human body, the exoskeleton does not touch 
the user body except for the predefined contact points 
and can automatically adapt to different human per¬ 
centiles, and finally, all the actuation structure is based 
on a unique high-torque electrical motorization. 

Very few data are documented about the operation 
of the XOS system and much of the information avail¬ 
able is contained within its related patents, instead the 
kinematics and the control of the BE are described in 
several documents [70.104,124]. 

70.5.5 Control Schemes 

The BE control is organized as a decentralized control, 
where each joint is supervised by a separate electronics. 
Figure 70.20 shows the PERECRO-BE device. All the 
electronics are organized in four clusters which respec¬ 
tively map to the single human limbs. The interconnec¬ 
tion between joint controllers is achieved through four 
separate EtherCAT-buses. 

All connections converge into a central electronics 
placed in the trunk backpack, which manages high-level 
control of walking, stability, and manipulation. 

A unique type of motorization as well as a unique 
type of low-level controller is implemented in each 
joint. Figure 70.21 highlights the interaction between 
the internal control loops and the higher control level. 
The internal controllers operate at a very high rate 
(2.5 kHz) and provide to estimate the joint velocities. 
At limb extremities, these controllers also acquire the 
information recorded by the embedded force sensors. 
The modularity of the architecture and the structure of 
the information do allow a high scalability of the sys¬ 
tem and a regular scheduling of the local and higher 
controllers. 

The two components highlighted in Fig. 70.21 as 
force control and feed forward control are managed at 
the central level at 1 kHz basic control rate. The first 
generates, using the sensor force information, a veloc¬ 
ity command that is forwarded to each limb cluster, 
whereas the second combines force information with 
posture and velocity information to define feedforward 
commands that can be directly added to compensate 
weight, loads or other gravity/inertial effects on the ex¬ 
oskeleton joints. 



Fig. 70.20 The PERCRO-Body Extender has been already 
defined (BE). The BE is a whole-body exoskeleton struc¬ 
ture designed to lift and manipulate 50 kg load per arm in 
the worst case conditions. The device has five attachment 
points with the user body: the two boots, two handles em¬ 
bedded in the grippers and the trunk corset. In addition to 
the internal sensors (position, force, and gyroscopes), force 
sensors embedded in the attachment connection help to de¬ 
termine the proper motion commands 

Given its flexible structure, the central controller 
can operate on the joint controllers in different oper¬ 
ational modalities; among these are: admittance con¬ 
troller, impedance controller, and feedback lineariza¬ 
tion controller. 

Admittance Controller 

Different control strategies have been implemented to 
move both the upper arm and the lower body part 
of the exoskeleton. Four basic schemes of admittance 
controller are implemented to manage the exoskeleton 
during walking and manipulation activities: 

• Arm admittance 

• Swinging leg 

• Stancing leg 

• Double stancing leg. 

The basic scheme is the arm-admittance controller. 
In this approach, the local force sensed at the inter- 
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Fig. 70.21 The basic BE control 
structure. The internal velocity con¬ 
trollers are placed in the exoskeleton 
joints and locally perform a velocity 
servo loop (proportional derivative) 
which is enhanced by a direct feedfor¬ 
ward torque. Both the inner-loop gain 
parameters and the external feedfor¬ 
ward torque can be commanded from 
the higher level central control unit. 
This structure is repeated identically 
for all the exoskeleton joints 


face is projected to the trunk reference system through 
the appropriate coordinate transformation. Its value is 
amplified through a gain factor and smoothed through 
a first-order low-pass filter that remove noisy effects. 
This reference signal is then back-propagated to the 
joint by using the inverse of the limb Jacobian. 

9iA=jl^^F h , ( 70 . 8 ) 

s+Pf 

where F h is the handle force/torque vector as read by 
the sensor, Rj r is the rotation matrix that maps to the ap¬ 
propriate trunk reference system, Kl v is a force/velocity 
gain, pf is the smoothing frequency, and Jl is the ap¬ 
propriate limb Jacobian that describes the motion of the 
limb with respect to the trunk. 

In addition to the above velocity commands, the 
control system also computes in real time the gravita¬ 
tional effects produced by the exoskeleton joints and 
provide to compensate them through the feedforward 
input (tff)of the joint velocity controllers. 


r ff = G(<7)Rl' 


0 

0 

9.81 


( 70 . 9 ) 


force to the operator. Hence, until the outer loop band¬ 
width is sufficiently smaller than the internal velocity 
servo, the two loops do not interfere [70.125], 

The arm-admittance (70.8) scheme, also applies 
to floating legs which swing during walking. In such 
a case, the controller uses the forces measured by the 
booths which have been biased using offset values that 
ensure a stable contact among the user’s foot and the 
exoskeleton booth. 

During the stancing phase, the information provided 
by the force sensors in the booth is useless. Hence the 
force sensor between the corset and the trunk backpack 
(backpack force) is used as a trajectory generator for 
the leg motion. The reference trajectories are generated 
with a unique, dual step, procedure which do not change 
being the exoskeleton stancing on one or two legs. 

In the first step, the backpack force has been deter¬ 
mined. This force is filtered and amplified in the same 
way as for the handle admittance control. This opera¬ 
tion generates a velocity vector (vu - tor) that applies to 
the backpack. In the second step, the central controller 
maps the same velocity vector to leg motion using a ve¬ 
locity map relationship that uses the direct kinematic 
between the trunk and the booths’ soles. The kinematic 
vector, r\(q), that describes the booth position with re¬ 
spect to the backpack force sensor is 


where G is the gravity contribution matrix (DOF x 3) 
that maps forces generated by the gravity to joints, R^ r 
is the rotation matrix that describes the arm orientation 
with respect to a world frame having the gravity vector 
aligned with the z-axis. 

The motion behavior generated to the programmed 
feedback policy can be examined through a two-step 
analysis. The internal position controller can be made 
asymptotically stable through standard robust control 
techniques. While the external admittance loop, which 
involves the human operator, exploits an intrinsically 
passive behavior, through the display of a pure viscous 


j v { = Vb + &> b Ar\(q) , 
| co f = <w b . 


( 70 . 10 ) 


The approach maps the force felt by the corset back¬ 
pack to the proper leg motion regardless the knowledge 
of the ZMP position. 

Feedback Linearization Controller 
The major limitation of the admittance control is the 
lack of transparency. The user who wears the power 
augmentation exoskeleton only feels forces that are pro¬ 
portional to his speed, and the feedforward component 
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Fig. 70.22 The second-order admittance controller with feedback linearization. In this control scheme, the control input provided 
by the pilot generates a reference acceleration information for the limb. The feedback linearization provides to compensate the 
inertial/centrifugal/gravity effects of the exoskeleton and to map user input in proper acceleration information at the contact 
point. Moreover, the appropriate force amplification factor programmed at the sensor provide to establish an equilibrium among 
the environment interaction and the user commands 


is only used to enlarge the control bandwidth and re¬ 
duce the nonlinearity due to different postures. 

However when the user touches any object, none of 
the contact force interferes with the exoskeleton motion 
creating in such a way an infinite force gain. While from 
the one side this is highly beneficial for joint stability, as 
a result it compromises the ability of the user to operate 
fine, scale force manipulation, tasks. 

In order to recover this functionality, the BE im¬ 
plements on the same internal control structure two 
alternative control policies: 

• A second-order admittance controller 

• An admittance adaptive controller. 

The second-order admittance controller collects 
user inputs to determine torque commands to the low- 
level joint controllers. This type of controller has been 
tested only on the upper body of the exoskeleton for the 
two relevant issues: 

• The arms require greater manipulation capabilities 
and a better transparency 

• The limited transparency of the leg motion reduces 
the interference between arm dynamics and trunk 
motion thus improving the overall stability. 

The principle of operation for power augmentation 
is as follows (the related numerical equations have been 
reported in Fig. 70.22): 

First, a force input is collected at each user handle. 
We suppose this force as being driven the interaction of 


a virtual mass with the given inertia properties (m p , 7 p ) 
and a separate weight property (»; pg ). 

Hence the resulting handle (end-effector) veloc¬ 
ity (Ve) will be determined as a direct integration of the 
user force as applied on this virtual mass, and the joint 
velocities as direct differentiation of the motion Jaco¬ 
bian (box admittance control 70.22). 

The feedback linearization control here compre¬ 
hends four components: 

• The B 4 B.V + Cabci + GabSt component, which takes 
into account the tracking error y, and the rotated 
gravity vector g T 

• The effective user force F s , considered as being 
equal to the sensor read, and already applied on the 
exoskeleton structure 

• The weight and inertial effects produced on the ex¬ 
oskeleton by the carried load (m a , 7 a ) moved at the 
prescribed velocity (V e ) 

• And the additional inertial and Coriolis effects pro¬ 
vided by the moving trunk (Pab). 

As for the first-order admittance controller, the sta¬ 
bility of the rigid model here can be ensured through 
the analysis of the linearized model. Combining the 
required equations, it can be derived that the robust¬ 
ness of the system is only affected by the amount of 
the inertia in the virtual mass properties (»; p ,7 p ). This 
property is unrelated from the power augmentation fac¬ 
tor (w a /w p ). 

This is the reason why it is wise to decouple among 
the virtual mass (?n p , 7 p ) and the weight used for gravity 
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Fig. 70.23 Power augmentation profile. The power augmentation coefficient ( k ) is defined as the ratio of the carried load 
versus the perceived load. In the BE approach this ratio is represented as a piecewise linear approximation (left drawing). 
The quality of the control, tested on 1-DOF of the BE, has integrated with the adaptive policy and is shown on the right 



Fig. 70.24 The admittance adaptive 
controller with feedback linearization. 
A standard parameter estimator 
procedure has been applied to the 
robot structure in order to achieve 
online identification of the carried 
load. To simplify the computation and 
improve accuracy of estimation, the 
procedure assumes that the grasped 
object has regular mass properties 
and can be described with only two 
parameters as shown in Fig. 70.22 


compensation (fft pg ). The last not affecting the overall 
control stability. 

Finally, the power augmentation factor has been de¬ 
fined as a piecewise linear approximation of the m a 
versus m p ratio which saturates for lower values of m p 
in order to preserve overall stability (Fig. 70.23 left). 

In order to work appropriately, the second-order ad¬ 
mittance model requires an approximate estimation of 
the carried load (Fig. 70.23 right). It is supposed that 
this information to be subjected to the following condi¬ 
tions: the grasp is stable during manipulation, and the 
load does not change frequently. 


Under these conditions, it is possible to use the 
motor currents as torque sensors that estimate the car¬ 
ried load. The principle of operation is reported in 
Fig. 70.24. Under the hypothesis that the load is stable 
and not frequently changing, the estimator can operate 
in lower frequencies. 

Several tests to check the quality of estimation and 
the real-time adaptation to load change have been per¬ 
formed. In Fig. 70.24, the specific case of three different 
loads are reported. Attached to the video material of 
the present chapter the mass load/unload operation have 
been shown. 
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70.6 Conclusions and Further Developments 


The presented review outlines some historical and state- 
of-the-art developments of wearable robotic systems for 
human-robot augmentation. Although EHPA systems 
are receiving an almost increasing interest from the 
robotics community, both from the research and from 
the commercial point of view, at present, the robotics 
technologies associated to their development are mainly 
derived from existing and well-established components. 
The lack of appropriate robotics components purposely 
developed for utilization on a wearable robotic system 
is mainly due, as testified by the existing research pro¬ 
totypes and commercial products, to a design approach 
that reflects the same methodologies pursued since the 
beginning for the design of industrial robots. 

The design of a robotics system that must be worn 
by the human operator during a continuous operation 
also in unstructured environments should necessarily 
hamper design criteria, such as safety, fatigue, trans¬ 
parency, etc., that are only sometimes minimally con¬ 
sidered for industrial robots. 

From the above analysis of different typologies of 
EHPA systems, it comes out that the availability of ade¬ 
quate actuation components is fundamental to fulfill the 
basic functional requirements in terms of total weight, 
dynamics behavior, and control aspects. However, what 
is needed for the future research in this extraordinary 


interesting and challenging held of research in robotics 
deals primarily with a new radical design philosophy 
centered on biomechanical aspects of humans behav¬ 
ior. In particular, a novel methodology considering as 
primary criteria the issue of energy consumption, not 
only for the wearable robotic system itself but also in 
terms of the complex resulting system human operator- 
wearable robot seems a fundamental prerequisite for 
future developments. 

As indicated in the previous sections, so far true 
advantages of exploiting an EHPA system in different 
real-application scenarios are still to be demonstrated. 
In fact, so far, research developments have been pri¬ 
marily devoted to the physical design and realization 
of wearable robots, without addressing the successive 
phase of validation and performance assessment in 
a true real environment. This aspect should enter as 
a prominent component for future developments of the 
research on EHPA. 

The held of wearable robots is rapidly expand¬ 
ing and the associated challenges in terms of design 
methodologies and ad-hoc robotics technologies just 
began to be understood in their full meaning. The mo¬ 
tivations for future developments in this held should 
always hrmly maintain human operator’s needs as the 
central focus of the research. 
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Arm light exoskeleton (ALEx) 

available from http://handbookofrobotics.org/view-chapter/70/videodetails/146 
Arm-Exos 

available from http://handbookofrobotics.org/view-chapter/70/videodetails/148 
Body extender transversal joint 

available from http://handbookofrobotics.org/view-chapter/70/videodetails/149 
Hand-exoskeletons 

available from http://handbookofrobotics.org/view-chapter/70/videodetails/150 
Collaborative control of the Body Extender 

available from http://handbookofrobotics.org/view-chapter/70/videodetails/151 
Body Extender - A fully powered whole-body exoskeleton 
available from http://handbookofrobotics.org/view-chapter/70/videodetails/152 
L-Exos for upper-limb motor rehabilitation 

available from http://handbookofrobotics.org/view-chapter/70/videodetails/180 
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71. Cognitive Human-Robot Interaction 


Bilge Mutlu, Nicholas Roy, Selma Sabanovic 


A key research challenge in robotics is to design 
robotic systems with the cognitive capabilities 
necessary to support human-robot interaction. 
These systems will need to have appropriate rep¬ 
resentations of the world; the task at hand; the 
capabilities, expectations, and actions of their 
human counterparts; and how their own actions 
might affect the world, their task, and their human 
partners. Cognitive human-robot interaction is 
a research area that considers human(s), robot(s), 
and their joint actions as a cognitive system and 
seeks to create models, algorithms, and design 
guidelines to enable the design of such systems. 
Core research activities in this area include the 
development of representations and actions that 
allow robots to participate in joint activities with 
people; a deeper understanding of human expec¬ 
tations and cognitive responses to robot actions; 
and, models of joint activity for human-robot 
interaction. This chapter surveys these research 
activities by drawing on research questions and 
advances from a wide range of fields including 
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computer science, cognitive science, linguistics, 
and robotics. 


When people interact with each other, they draw on 
mental models of themselves, of their interaction part¬ 
ners, of the immediate context of the interaction, and 
of their broader physical, social, and cultural context. 
These models help them predict the actions of their 
interaction partners and make decisions about their 
own actions. To effectively interact with people, robots 
need similar models that help them determine their 
own actions and predict the actions of their users. 
Cognitive human-robot interaction (HRI) is a research 
area that seeks to improve interactions between robots 
and their users by developing cognitive models for 


robots and understanding human mental models of 
robots. 

A central tenet of cognitive HRI is that humans, 
robots, and the context of their interaction form a com¬ 
plex cognitive system situated in the real world. A key 
research activity in the held involves the development 
of frameworks to represent this system [71.1-3]. This 
activity is informed primarily by research in cognitive 
science that develops frameworks to represent human 
cognitive systems. These frameworks include physical 
symbol systems [71.4], situated actions [71.5,6], and 
those that combine symbolic and situated perspectives, 
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Models of people 


Mental models of the robot, 
social cognition, & interpersonal 
differences in perceptions 


Models of robots 


Models of the human, 
the task context, & the broader 
context of the interaction 


Fig. 71.1 A visual summary of the 
research activities in cognitive HRI 
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such as activity theory [71.7] and distributed cogni¬ 
tion [71.8]. While the discussion on what framework 
best represents HRI as a cognitive system is ongoing, 
research in cognitive HRI involves the development 
of both symbolic and situated representations. More 
specifically, research activities in this area include (also 
illustrated in Fig. 71.1): 

1. Human models of interaction: Building an under¬ 
standing of people’s mental models of robots, how 
people perceive robots and interpret their actions 
and behaviors, and how these perceptions and inter¬ 
pretations change across contexts and user groups. 


71.1 Human Models of Interaction 

Robots are expected to increasingly enter everyday en¬ 
vironments - outside of factories and laboratories - 
including homes [71.9, 10], offices [71.11], and class¬ 
rooms [71.12, 13]. In these contexts, robots will need 
to coexist and collaborate with a wide variety of users, 
such as children and the elderly, many of whom will not 
be technically trained. Accordingly, there is a growing 
research emphasis in cognitive HRI on identifying the 
mental models people use to make sense of emerging 
robotic technologies and investigating people’s reac¬ 
tions to the appearance and behaviors of robots. This 
research aims not only to improve the ease of use of 
robots by designing them to fit human mental models, 
but also to gain new insights about human cognition and 
behavior. With the latter goal in mind, researchers also 
use robots to embody specific theories of human cogni¬ 
tion that are then evaluated through HRI studies. 

71.1.1 Mental Models of Robots 

Research in human-computer interaction has shown 
that people’s attitudes and behaviors toward digital 
technologies often follow the social rules established in 


2. Robot models of interaction : The development of 
models that enable robots to map aspects of the 
interaction into the physical world and develop 
cognitive capabilities through interaction with the 
social and physical environment. 

3. Models of HRI: Creating models and mechanisms 
that guide human-robot communication and collab¬ 
oration, action planning, and model learning. 

This chapter surveys existing efforts in these re¬ 
search areas, drawing on research questions and ad¬ 
vances from a wide range of fields including robotics, 
cognitive science, and linguistics. 


human-human interaction [71.14]. It is reasonable to 
expect that people will similarly interpret the interac¬ 
tive behaviors of robots in social ways. Cognitive HRI 
researchers continue to investigate the extent to and 
conditions in which this maxim applies to HRI as they 
use their understanding of human social cognition to de¬ 
velop robotic platforms adapted to users’ expectations 
and behaviors. In the process of evaluating such robotic 
platforms, researchers explore people’s mental models 
of robots and identify areas in which users’ expecta¬ 
tions and understandings of robots may not be born out 
by the robot’s appearance or behavior in ways detri¬ 
mental to the HRI experience. Knowing which mental 
models people are using to interpret robot behavior not 
only helps roboticists to understand HRI more deeply, 
but also helps them in designing appropriate behaviors 
for the robot. 

Models Ascribed to Robots 
Extensive research by Turkle et al. [71.15,16] exam¬ 
ines how people, including children and older adults, 
make sense of their novel interactions with social robots 
such as Kismet, Cog, PARO, Furby, and My Real Baby. 
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These studies show that people apply a variety of mental 
models relating to animacy, sociality, affect, and con¬ 
sciousness to explain their experiences and emerging re¬ 
lationships with robots. Some research participants ap¬ 
proached robots in a scientific-exploratory mode, inter¬ 
preting a robot’s actions in an emotionally detached and 
mechanistic manner. Others took a relational-animistic 
approach, investing in the interactions emotionally and 
treating robots as if they were living beings, such as ba¬ 
bies or pets. The ways in which participants described 
the robots verbally did not always fit with the way in 
which they interacted with them - a person who says the 
robot is only a mechanical thing may still act toward 
the robot in a nurturing manner, such as soothing a cry¬ 
ing My Real Baby [71.16, p. 118]. This corresponds to 
previous findings in human-computer interaction (HCI) 
which suggest that people mindlessly apply social char¬ 
acteristics to computers [71.17]. Field studies with the 
seal-like robot PARO have shown that robots can also 
act as evocative objects that spark reflections on pre¬ 
vious relationships and events (e.g., with a grandchild, 
spouse, or pet), which users then use to make sense of 
their interactions with the robots [71.18, 19]. 

In addition to identifying the mental models peo¬ 
ple use to interpret their experiences with robots, re¬ 
searchers study the effects from deliberately incorpo¬ 
rating specific social schemas into robot design. Anthro¬ 
pomorphism, or the attribution of human characteristics 
to nonhuman (e.g., animal or artifact) behavior, is an 
interpretive schema that has been of particular interest 
to HRI researchers. Some scholars, such as Nass and 
Moon [71.17], critique anthropomorphic explanations 
as false and misleading. Others, including Duffy [71.20] 
and Kiesler et al. [71.21], suggest that the deliberate use 
of anthropomorphism can benefit social robot design 
by taking advantage of people’s propensity to interpret 
events and other agents socially to make robot behav¬ 
iors more understandable to users. This interpretation 
raises the question of which characteristics of the robot 
or the interaction are instrumental in inciting people to 
anthropomorphize robots and has inspired researchers 
to study a variety of socio-cultural cues, behaviors, and 
task contexts. Kiesler et al. [71.21] showed that people 
anthropomorphize a physically embodied robot more 
readily than an on-screen agent, and people behave in 
a more engaged and socially appropriate manner while 
interacting with the co-present robot. People also an¬ 
thropomorphize robots they interact with directly more 
than they do with robots in general, and with robots that 
follow social conventions (e.g., polite robots) more than 
those that do not [71.22]. The personal characteristics, 
such as personality, of the human interaction partner 
can also affect their mental models of robots. For exam¬ 
ple, users with low emotional stability and extraversion 


scores were found to prefer mechanical-looking robots 
to human-like ones [71.23]. 

As might be expected, a robot’s human-like appear¬ 
ance can have a positive effect on people’s propensity to 
anthropomorphize [71.25]. Obversely, too high a level 
of human-likeness may place the robot in the uncanny 
valley [71.26]. The uncanny valley refers to a dip in the 
hypothetical nonlinear graph describing the relation¬ 
ship between a robot’s human-likeness and a human’s 
emotional response to it, suggesting that a robot with 
a very high degree of human-likeness coupled with 
some remaining nonhuman qualities will make users 
uncomfortable. This hypothesized effect essentially de¬ 
scribes what happens when a person’s mental model of 
the robot as human is not born out by its interactive ca¬ 
pabilities. Various cognitive aspects of this hypothesis 
have been studied, suggesting that the construct is mul¬ 
tidimensional [71.27,28] rather than two-dimensional 
(2-D), as depicted by Mori [71.26]. Furthermore, re¬ 
search suggests that the mismatch between different di¬ 
mensions, rather than any quality alone, can cause a dis¬ 
sonance that leads to people’s discomfort with robots. 
MacDorman et al. [71.24] show that incongruencies 
between a robot’s appearance and movement can dimin¬ 
ish anthropomorphic attributions (Fig. 71.2). A similar 
result was found by Saygin et al. [71.29], who used 
functional magnetic resonance imaging (fMRI) to show 
that the human action perception system made distinc¬ 
tive responses to the mismatch between the level of 
human-likeness of a robot’s appearance and motion but 
not to appearance or motion alone. Mismatches in the 
human- or robot-like qualities of an on-screen robot’s 
voice and appearance were also shown to heighten peo¬ 
ple’s sense of the character’s eeriness - people found 



Fig. 71.2 An extended notion of the uncanny valley which includes 
appearance and behavior as significant variables (after [71.24]) 
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both a robot with a human voice and a human with 
a robot voice to be creepy [71 .30]. 

Mental Models in Robot Design 
Researchers may deliberately include specific anthro¬ 
pomorphic schemas to promote user behaviors that aid 
robots in performing their tasks. One common example 
is the use of the baby schema - a soft round ap¬ 
pearance, large eyes, and proto-verbal utterances - in 
Kismet [71.31] (Fig. 71.4), Muu [71.32] (Fig. 71.3), 
and Infanoid [71.33] to encourage people to anthropo¬ 
morphize robots. This schema is also useful in that it 
can incite people to behave in a nurturing manner to¬ 
ward robots in the interest of scaffolding the robots’ 
learning in a way similar to infant-parent interactions. 
A robot’s perceived gender can also have an effect on 
people’s mental models of the robot’s knowledge of cer¬ 
tain topics; for example, in one study a female robot 
was expected to be more knowledgeable about dating 
than a male robot [71.34]. While certain mental models 
become operational as soon as a person starts interact¬ 
ing with a robot (e.g., gender, age, human-likeness), 
people can adapt their mental models of a robot’s ca¬ 
pabilities when given additional information about the 
robot’s personal characteristics, such as the robot’s 
country of origin or the language it speaks [71.35], 
Goetz et al. [71.36] showed that matching a robot’s per¬ 
sonality to the task it is supposed to perform can have 
a significant effect on its efficacy: people were more re¬ 
sponsive to a robot that had a serious, rather than an 
entertaining, demeanor when its job was to motivate 
them to exercise. Also focusing on task models in HRI, 
Lee et al. [71.37] showed how people use their exist¬ 
ing utilitarian and relational models of service to set 
expectations for their interactions with a service robot. 
These models also affected the preferred ways in which 
the robot should make up for any mistakes it makes 
in service - people with a utilitarian mental model of 
service preferred to receive compensation, while those 
with a relational model responded well to an apology. 

As interactive robots are developed and used all 
over the world, researchers have also started explor¬ 
ing how cultural models [71.38] affect people’s per¬ 
ceptions of and interactions with robots. Social and 
behavioral norms are culturally variable, so we can 
expect users’ understanding and adoption of socially 
interactive robots to differ accordingly. Cross-cultural 
research in F1RI largely supports this expectation. Ev¬ 
ers et al. [71.39] showed that users from China and 
the US respond differently to robots. Further research 
by Wang et al. [71.40] suggests that specific cultural 
models regarding communication norms, particularly 
explicit and implicit modes of communicating informa¬ 
tion and intent to interaction partners, affect people’s 



Fig. 71.3 Muu’s big eyes and soft round body are designed 
according to the baby schema. Using two robots that can 
interact with each other instead of one suggests a relational 
understanding of agency (courtesy of Sabanovic) 



Fig. 71.4 Kismet’s big round eyes and infant-like vocaliza¬ 
tions are another example of the baby schema (courtesy of 
Sabanovic) 


perceptions of a robot’s trustworthiness and its in-group 
membership. Researchers have also shown that roboti¬ 
cists themselves use cultural models unintentionally in 
their work, including particular models of emotional 
display [71.41], and cultural models reflecting histor¬ 
ical, theological, and popular perceptions of robotic 
technology [71.42]. Research on cultural models in HRI 
not only points to the importance of reflexively in¬ 
cluding such models in robot design, but also allows 
researchers to do systematic research on culturally situ¬ 
ated cognition using robots as stimuli. 

Research on mental models applied to interactive 
robots has not only shown that people use their ex¬ 
isting mental models to make sense of these novel 
artifacts, but also that we may need new ontological 
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categories to accommodate emerging mental models of 
these entities [71.16,43]. Kahn's et al. [71.44] studies 
of children’s moral interpretations of interactions with 
an AIBO robot showed that their mental models of the 
robots included rationalizations and behaviors related 
to both inanimate and animate objects. Turkle [71.45] 
suggests that interactive robots co-opting relational 
feelings and responses normally reserved for animals 
and humans call into question the authenticity of rela¬ 
tionships. Further, Turkle [71.45] suggests that a more 
sophisticated new notion of autonomous yet inanimate 
artifacts has become necessary. Both researchers have 
suggested that interactive robots might comprise a new 
ontological category, and that we also need to be con¬ 
scious of the ways in which interactions with these 
artifacts affect our mental models of animate beings. 

71.1.2 Social Cognition 

The development of robots that can interact naturally 
with humans calls for the detailed study of social activ¬ 
ity and the cognitive models that underly such activities. 
Scassellati [71.46] argues that robots can help us study 
the limits of human social cognition because they are 
not alive, yet they can behave in socially appropriate (or 
inappropriate) and evocative ways. Robots that incor¬ 
porate social cues such as gaze, proximity, and facial 
expressions, push our Darwinian buttons [71.16, p. 8] 
and effectively coerce us into interacting with them so¬ 
cially. Studying which cues have these effects is an 
opportunity to learn more about human social cognition 
and improve robot design. 

Researchers studying the social aspects of cogni¬ 
tive HRI are identifying the minimal cues robots need 
to evoke social responses from people, including those 
related to robotic embodiment, gaze, proxemic cues, 
and interaction rhythms. Current research is also fo¬ 
cused on applying and evaluating different models of 
cognition in the context of HRI. Robots can be un¬ 
precedented experimental tools for the study of social 
cognition. They can be used to provide stimuli in ex¬ 
periments and field studies, since their actions and 
behaviors can be carefully controlled, finely tuned and 
varied, and repeated exactly and indefinitely, which is 
often challenging even for well-trained human con¬ 
federates [71.47,48]. Furthermore, robots do not have 
difficulty acting unnaturally (e.g., not reacting to other 
person’s cues) or violating social norms (e.g., being 
rude) when needed, a source of potential stress in hu¬ 
man researchers [71.49]. 

Minimal and Human-Like Cues in HRI 
One approach to studying social cognition has been to 
try to isolate the minimal set of cues that evoke social 


responses and perceptions from human interaction part¬ 
ners. The creators of Muu followed a minimal design 
strategy [71.32], using cartoons and children’s draw¬ 
ings to develop a robot that can be communicatively 
engaging to people without relying on overt human¬ 
likeness. Kozima et al.’s [71.50] Keepon was designed 
to include characteristics common to living beings, such 
as lateral symmetry and two eyes, which are assumed 
to be important for social interaction (Fig. 71.5). The 
robot also performs fundamental social behaviors, such 
as joint attention, eye contact, and emotional expres¬ 
sion through bodily posture and movement and using 
only four degrees of freedom (Fig. 71.6). These mini¬ 
mal cues have been shown to be sufficient for engaging 
children in short-term interaction in the lab and long¬ 
term interaction in more natural environments, such as 
a classroom [71.50]. 

Studies with minimalist robots have also under¬ 
scored the effect of social context in people’s in¬ 
terpretations of robots. Field studies with Keepon in 
an elementary school showed that children incorpo- 



Fig. 71.5 Keepon is a simple robot used to investigate cues 
such as joint attention, emotive expression, and rhythmic- 
ity in HRI (courtesy of Sabanovic) 



Fig. 71.6 Keepon uses four degrees of freedom to express 
emotive and attentional cues (after [71.50]) 
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Fig. 71.7 An android robot fabricated by Kokoro Ltd. 
(courtesy of Sabanovic) 



Integration of science and engineering 


Fig. 71.8 Androids can be used to investigate human cognition an¬ 
alytically as well as synthetically (after [71.51]) 

rated the simple robot into a wide variety of inter¬ 
action contexts (e.g., playing house with Keepon as 


a baby or pet, or treating Keepon as another stu¬ 
dent in the classroom) due not only to its interpretive 
flexibility, but to the richness of the social environ¬ 
ment. This inspired children to engage with the robot 
over long periods of time, sometimes years, whereas 
they became bored after 10—15 min when interacting 
with Keepon in the laboratory. The above mentioned 
Muu’s design was inspired by ecological models of 
cognition [71 .52, 53] suggesting that a robot is in¬ 
herently incomplete as a communicative device - it 
needs a human interaction partner to imbue its ac¬ 
tions with meaning. Muu therefore relies on the context 
and the presence of other interactive agents (including 
people, other Muu, and objects such as blocks dis¬ 
played in Fig. 71.3 for triadic interaction) to enable 
people to make sense of its actions and relationally 
ascribe social agency to the robot. The Social Trash- 
can project [71.54] similarly explored how minimal 
social cues, including contingent motion and approach¬ 
ing people, can be used to display the robot’s intentions 
to children and get their assistance in trash collection. 
Yamaji et al. [71.54] also showed that robots moving 
together as a group - relationally - were more success¬ 
ful in attracting the children’s attention than that moved 
individually. 

An alternative approach to the study of social cog¬ 
nition through HRI focuses on human-like realism in 
appearance and behavior and is proposed by Ishig- 
uro [71.47] and MacDorman and Ishiguro [71.48], 
They claim that androids - robots that bear a close and 
sometimes uncanny resemblance to humans (Fig. 71.7, 
for example) - are unprecedented test beds for the study 
of social cognition. Used as stand-ins for humans in this 
android science, robots have a twofold function as ex¬ 
perimental tools for evaluating hypotheses about human 
perception, cognition, and interaction, and as a test¬ 
ing ground for various cognitive models (Fig. 71.8). 
Using an android platform, Ishiguro [71.55] showed 
the importance of micro movements as a cue that in¬ 
cites people to attribute human-likeness to a robot in 
short (1—2 s) interactions. Another topic of continu¬ 
ing investigation is the possibility of simulating the 
personal presence of a remote actor in the local environ¬ 
ment using an android platform [71.56,57], Shimada 
et al. [71.58] showed that people evaluated an android 
as more likable when it mimics them in a way similar 
to the chameleon effect that occurs when two people in¬ 
teract. MacDorman and Ishiguro [71 .48] suggested that 
such androids can be used in research relating to a num¬ 
ber of current topics of interest in cognitive science, 
including the mind-body problem, nature versus nur¬ 
ture, rationality and emotion in human reasoning, and 
the relationship between social interaction and internal 
cognitive mechanisms. 
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Embodied Social Cues 

Embodiment separates robots from other interactive 
digital technologies and has been investigated through 
studies comparing how people interpret and act toward 
robots that are physically co-present with them and on¬ 
screen robots or social agents. Wainer etal.’s [71.59] 
comparison of people’s interactions with an embodied 
robot and a simulated robot, and a co- and tele-present 
found that people were more engaged with, behaved 
more appropriately to, and anthropomorphized a co¬ 
located robot more than a tele-present robot. People 
interacting with an embodied robot have also shown 
tendencies to issue more commands than those in¬ 
teracting with a simulated robot [71.60]. The social 
effect of embodiment in HRI was further confirmed by 
Bainbridge’s et al. [71.61] study showing that people 
are more likely to comply with requests made by an 
immediately present robot rather than requests made 
by a remote robot communicating with them through 
a television screen. These converging results strongly 
suggest that a robot’s embodied presence has a signifi¬ 
cant cognitive effect on people’s social responses to the 
robot. The embodied nature of robots also enables the 
study and use of various other social cues, including 
proxemic behaviors, gaze, and interaction rhythms, in 
HRI. A more detailed review of embodied social cues 
is provided by Mutlu [71.62]. 

Proxemic behaviors [71.63], the study of which is 
enabled by the embodied nature of robots, not only 
have a significant effect on people’s perceptions of 
and behaviors toward robots, but have also been used 
as a measure of people’s perceptions of robots as so¬ 
cial agents. Takayama and Pantofaru [71.64] found 
that prior experience with pets and robots decreased 
the distance at which people felt comfortable around 
robots. Individual traits such as gender and personality 
also affect people’s preferences regarding the distance 
at which they are comfortable with a robot approach¬ 
ing them [71.64, 65]. Proxemic behavior can be related 
to other social cues in complex ways. For example, 
Mumm’s and Mutlu’s [71.66] study showed that people 
will compensate for the intense gaze in their direction 
of a robot they do not like by moving away from the 
robot (Fig. 71.9). While most studies of proxemic sys¬ 
tems have been done in the laboratory, recent work is 
also investigating more natural interactions between hu¬ 
mans and robots in open environments [71.67]. 

Gaze is an important cue in human-human inter¬ 
action and is also one of the most studied nonverbal 
social cues in HRI People use many 

such seemingly unintentional, unconscious, and auto¬ 
matic nonverbal cues as clues regarding the mental 
states and intentions of other actors, including robots. 
Gaze has been shown to be useful for communicat- 



Gaze follow Gaze avoid 


Fig. 71.9 In the study by Mumm and Mutlu [71.66], partic¬ 
ipants maintain a greater distance with the unlikable robot 
when the robot follows them with its gaze than when its 
gaze avoids the participant, while their proxemic behavior 
is not affected by a likable robot’s gaze (courtesy of Mutlu) 

ing intent, modulating interaction, and even affecting 
participants’ experience and memory of the interac¬ 
tion. Researchers have shown that gaze can be used 
to engage users [71.69,70] and to assign them par¬ 
ticular roles in and manage the interaction [71.71]. 
A robot’s gaze behavior can affect the human inter¬ 
action partner’s gaze and speech, their comprehension 
of the robot’s speech [71.72], and people’s memory 
of a story narrated by the robot and perceptions of 
the robotic storyteller [71.73]. Researchers studying the 
temporal aspects of gaze in HRI found that the tim¬ 
ing of gaze behavior provides cues to human intentions 
while teaching the robot the names of objects, sug¬ 
gesting that properly timed gaze behaviors can have 
a positive effect on collaborative tasks between a hu¬ 
man and a robot [71.68] (Fig. 71.10). Yu et al. [71.74] 
have developed a data-driven approach to analyzing hu¬ 
man gaze in the context of HRI, which can be used 



Fig. 71.10 Yu s et al. [71.68] HRI studies provide data for 
developing models of the temporal aspects of interaction 
(courtesy of Yu) 
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to develop detailed micro-behavioral gaze models that 
can guide robot behaviors as well as be used to un¬ 
derstand human intentions and behaviors in the course 
of collaborative activities. A recent study by Admoni 
et al. [71.75], contrary to the assumption that anthro¬ 
pomorphic robots engage us automatically in much the 
same way we are engaged by people, shows that robot 
gaze is not necessarily treated by people in the same 
automatic way that human gaze is treated; we do not, 
therefore, necessarily perceive robots as social in an au¬ 
tomatic and mindless way. 

Interaction rhythms - nonverbal and largely uncon¬ 
scious temporal coordination between partners in an 
interaction - enable the exchange of information, antici¬ 
pation of the interaction partner’s actions, and even pos¬ 
itive evaluations of interaction among humans as a fun¬ 
damental subscript of all human interaction [71 .76-78]. 
The rhythmicity of interaction is therefore also a cru¬ 
cial factor in HRI, both in terms of developing robots 
that can perceive and respond to people’s rhythmicity, 
and of understanding how people react to the temporal 
aspects of robot behaviors. Michalowski et al. [71.79] 
used a dancing robot to explore the rhythmic proper¬ 
ties of social interaction and showed that children were 
more likely to interact with a robot that was synchro¬ 
nized to background music rather than one that was not, 
and that the children’s own rhythmic behavior was in¬ 
fluenced by the robot’s rhythmicity. In further research, 
Michalowski et al. [71.80] suggest that rhythmic inter¬ 
action can be used as a form of play between children 
and robots, and that following the robot’s lead in rhyth¬ 
mic entrainment with music causes children to attend 
more closely to musical rhythm. Avrunin et al. [71.81] 
found that simple changes in a robot’s rhythmic danc¬ 
ing behavior, such as variation of motions, flaws in the 
robot’s synchrony with music, and coordination of be¬ 
havior changes with musical dynamics, increased peo¬ 
ple’s perceptions of the robot’s lifelikeness. Hoffman 
and Breazeal [71.82] used the temporal patterns of in¬ 
teraction - its rhythms - to develop robotic systems that 
can anticipate a human partner’s actions in collaborative 
tasks, such as AUR, a robotic desk lamp, and Shimon, 
a marimba playing robot [71.83] (loMihlilfia). Along 
with improving HRI, the use of robots in studying 
the rhythmic properties of interaction provides a new 


71.2 Robot Models of Interaction 

Simon [71.91] suggests that the study of human be¬ 
havior can be approached through synthesis as well as 
analysis and designs computer simulations as a tech¬ 
nique for understanding and predicting the behavior of 


tool for cognitive science research on these subtle, fine 
grained, and unconscious social cues. 

Cognitive Development in HRI 
A further topic of focus in cognitive HRI has been 
the study of social and cognitive development through 
studies of typically developing and autistic children’s 
interactions with robots. Multiple studies in educational 
contexts have focused on understanding how children 
ascribe social agency to robots [71.12,13]. Kozima 
et al. [71.50] found that children of different ages dis¬ 
play varying modes of interaction with the robot, which 
suggest different levels of comprehension of its on¬ 
tological status - 0-year-old interacted with Keepon 
as a moving thing, 1-2-year-old interacted with the 
robot as an autonomous system, and children over 2 
years of age treated the robot as a social agent. Dedk 
et al. [71.84] studied the mechanism of joint atten¬ 
tion in HRI to explore the importance of contingency 
and find out which perceptual features infants use to 
achieve shared attention by modeling these in a robot. 
Researchers also use robots to study social deficit 
disorders, particularly autism. Converging results on 
research using robots to study social deficit disorders 
show that autistic children respond to robots in a social 
manner that they do not display with people [71 .85-87], 
inspiring researchers to perform studies with children in 
the context of HRI. One aim of such research is to try 
to understand which aspects of a robot’s behavior en¬ 
able autistic children to participate in social interaction, 
which may clarify some of the reasons for their diffi¬ 
culties when interacting with humans. HRI researchers 
have also applied robots to various therapeutic scenar¬ 
ios with autistic children in an effort to provide parents 
and therapists with a tool to improve communication 
and understand the children better [71.88,89]. Studies 
by Kozima et al. [71.90] in which the robot Keepon 
interacts with children with autism suggest that such 
minimally designed robots can be used to motivate 
autistic children to share their mental states with others, 
such as therapists or parents. This work poses a promis¬ 
ing possibility for learning more about social deficits 
and development disorders such as autism, as well as 
providing tools for diagnosis and therapy using robotic 
technologies. 


natural, social, and cognitive systems. In the spirit of Si¬ 
mon’s synthetic approach to the study of human cogni¬ 
tion, robotics researchers have been engineering robots 
as tools for developing and testing a variety of cog- 
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nitive, behavioral, and developmental models [71.47, 
92, 93]. This approach assumes that cognitive models 
are validated when the implementation of a particular 
model on a robot produces behavior similar to that pro¬ 
duced by humans in the same situation; if this does not 
occur, it is a sign that there may be something wrong 
with the model or the way it was implemented in the 
robot [71.46]. Cognitive HRI research involves the de¬ 
velopment of robotic platforms based on findings from 
cognitive science and using such platforms to extend 
knowledge about human cognitive processes. 

71.2.1 Developmental Models 

Robots are particularly appropriate for exploring the¬ 
ories of embodied and social cognition, which em¬ 
phasize the centrality of the agent’s interactions with 
its environment and other agents in that environment 
to cognitive functioning. In the process of synthesiz¬ 
ing a robotic system, the researcher is drawn to focus 
on the dependency of cognition on noncognitive pro¬ 
cesses, including the social and physical environment 
in which cognition takes place. Robots such as Cog 
and Kismet [71.31] have been used to simulate and 
validate different theories of cognition, perception, and 
behavior. Cog was used to implement and test cog¬ 
nitive models relating to reaching behavior, rhythmic 
motor skills, visual search and attention, and social skill 
acquisition (e.g., joint attention and theory of mind). 
In the process they were able to validate, extend, and 
show the limitations of cognitive, behavioral, and de¬ 
velopmental theories. In later projects, researchers have 
developed models inspired by human cognition and be¬ 
havior such as social referencing [71.94], perception 
and action loops [71.95], anticipatory actions in collab¬ 
orative teamwork [71.96], and others. 

Robotics researchers apply the idea that the de¬ 
velopment of intelligence is embedded in social and 
cultural environment to the construction of robotic 
artifacts. For example, Breazeal [71.31] applied theo¬ 
ries relating to infant social development, psychology, 
ethology, and evolution to design the robot Kismet, 
which used infant-like social cues to engage a hu¬ 
man participant in interactions that would scaffold the 
robot’s learning, as in the case of infant-parent in¬ 
teractions. Researchers have also developed a variety 
of robotic systems that exhibit cognitive traits such 
as imitation [71.97,98], joint attention [71.99-101], 
and rhythmic synchrony [71.50,102]. The Infanoid 
project [71.33] also used a synthetic approach in which 
development was understood through studying how the 
robot learns. Situated and embodied models have been 
applied to robot learning, particularly through imita¬ 
tion. For example, Bakker and Kuniyoshi [71.103] pro¬ 


pose imitation as an interaction and learning paradigm 
in contrast to robot programming or robot learning. 
Further, they argue that robot programming is too hard 
and tedious to specify complex behaviors in sufficient 
detail and specify how they might be adapted to novel 
situations. 

71.2.2 Robot Spatial Cognition 

Systems dedicated to modeling spatial language 
and interaction, including the theories by Jackend- 
off [71.104], Landau and Jackendoff [71.105], and 
Talmy [71.106], have been produced for many years. 
Several previous works have been computational in¬ 
stantiations of the ideas presented in these theories, in 
particular the implementation and testing of spatial se¬ 
mantics models. Regier [71.107] built a system that 
assigns labels, such as through to a movie showing a fig¬ 
ure moving relative to a landmark object. Kelleher and 
Costello [71.108] and Regier and Carlson [71.109] built 
models for the meanings of static spatial prepositions, 
such as in front of and above. 

Many authors have proposed formalisms for en¬ 
abling systems to reason about the semantics of natural 
language use in the context of giving directions. For ex¬ 
ample, Bugmann et al. [71.110] identified a set of 15 
primitive procedures associated with clauses in a cor¬ 
pus of spoken natural language directions. Levit and 
Roy [71.111] designed navigational informational units 
that break down instructions into components. MacMa- 
hon et al. [71.112] represented a clause in a set of 
directions as a compound action consisting of a simple 
action (move, turn, verify, and declare-goal), plus a set 
of pre- and post-conditions. Many of these previous rep¬ 
resentations are expressive but difficult to automatically 
extract from text. Some authors avoid this problem by 
using human annotations [71. 111,1 12] or by specifying 
the robot’s behavior in a controlled language [71.113]. 
Matuszek et al. [71.114] created a system that follows 
directions using a machine translation approach. Sim¬ 
ilarly, Vogel and Jurafsky [71.115] used reinforcement 
learning to automatically learn a model for understand¬ 
ing route instructions. 

71.2.3 Symbol Grounding 

Mapping language from the human partner to aspects 
of the external world - locations, objects, or actions 
the robot should take - described by the language was 
referred to as an instance of the symbol grounding prob¬ 
lem [71.116]. There are three different ways people 
have approached the symbol grounding problem, which 
is more general than spatial cognition, in robotics. Start¬ 
ing with Winograd [71.117], many have created symbol 
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systems that map between some language and the exter¬ 
nal world by manually connecting each term onto a pre¬ 
specified action space and set of environmental fea¬ 
tures [71.110, 112,113,118-121]. This class of systems 
takes advantage of the structure of linguistic interaction, 
but the systems usually do not involve learning, have lit¬ 
tle perceptual feedback, and have a fixed action space. 
A second approach involves learning the meaning of 
words in the sensorimotor space (e.g., joint angles and 
images) of the robot [71.122-124]. By treating human 
interaction terms as sensory input, these systems must 
learn directly from complex features extracted by per¬ 
ceptual systems, resulting in a limited set of commands 
that can be robustly understood. A third approach is to 
use learning to convert from an interaction onto aspects 
of the environment. These approaches may only use lin¬ 
guistic features [71.125,126], spatial features [71.107] 
or linguistic, spatial, and semantic features [71.114, 
115,127-129]. These approaches learn the meaning of 
spatial prepositions (e.g., above [71.107]), verbs of ma¬ 
nipulation (e.g., push and shove [71.130]), and verbs of 
motion (e.g., follow and meet [71.131]) and landmarks 
(e.g., the doors [71.129]). 

Recent progress in probabilistic relational models, 
such as the generalized grounding graph (G 3 ), has 
addressed these issues by exploiting the structure of 


spatial discourse, breaking down a natural language 
command into component clauses and connecting each 
word to a physical interpretation [71.131,132]. The 
grounding graph takes full advantage of the hierarchical 
and compositional structure of natural language com¬ 
mands and is able to ground landmarks, such as the 
computers, by exploiting object co-occurrence statistics 
between unknown noun phrases and known perceptual 
features, spatial relations, such as past in the path of 
an agent relative to an object, and motion verbs, such 
as follow, meet, avoid, and go in the path of a single 
agent or multiple agents. Once trained, the G3 model 
can ground spatial discourse in a semantic map of the 
environment; the map can be given a priori or created 
on the fly as the robot explores the environment. The 
G3 model is dynamically instantiated as a hierarchical 
probabilistic graphical model that connects each ele¬ 
ment in a natural language command to an object, place, 
path, or event in the environment. Its structure is cre¬ 
ated according to the compositional and hierarchical 
structure of the command, learning the mapping from 
language onto a continuous robot plan. The G3 model 
is trained on a corpus of natural language commands 
paired with groundings, and learns meanings for words 
and phrases in the corpus, including complex verbs, 
such as put and take. 


71.3 Models of Human-Robot Interaction 


Robotic technologies that interact with people - 
whether they afford closed-loop teleoperation or col¬ 
laborate autonomously as peers - need to interpret, 
make decisions about, and respond to their environ¬ 
ment, particularly the physical world, the task that they 
are expected to support, and the actions, goals, and 
intentions of the other agents - including people. To 
achieve these goals, robots need models that accurately 
represent the physical and cognitive characteristics of 
their environment. These models might outline such 
characteristics as narrowly as control-action relation¬ 
ships in the context of teleoperation or as comprehen¬ 
sively as human-robot joint activity in the context of 
peer-to-peer collaboration. Cognitive HRI considers the 


robotic system to be a part of a distributed cognitive 
system and therefore seeks primarily to develop cog¬ 
nitively inspired models [71.2]. These models might 
draw on knowledge about human cognition to improve 
the usability of robotic system, mimic human deci¬ 
sion making or behavior mechanisms, or represent the 
complete human-robot cognitive system, offering cog¬ 
nitive representations for different paradigms of HRI 
(Fig. 71.11). 

71.3.1 Dialog-Based Models 

Research on human robot interaction across different 
interaction paradigms from teleoperation [71.133, 134] 


Cognitive models Models of human-robot 

of robot control joint activity 
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Fig. 71.11 Different paradigms of HRI (after [71.2]) 
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to peer-to-peer interaction [71.135] has highlighted 
the need for establishing common ground [71.136] 
for effective HRI. In the context of teleoperation, 
Burke et al. [71.133] found that a lack of appropriate 
shared representations among human team members 
and the robot resulted in discrepancies in understanding 
among team members and breakdowns in perceiving 
and interpreting data provided by the robot. Stubbs 
et al. [71.134] observed such lack of common ground 
between operators and the robot across varying levels 
of autonomy. In the context of peer-to-peer interac¬ 
tion, Kiesler [71.135] argues that participants in an 
encounter seek to minimize their collective effort to 
reach mutual understanding and that the effort needed 
to establish this understanding between a robot and 
its users might determine the outcomes and success of 
HRI. These examples have motivated a large body of 
research in developing dialog-based models for estab¬ 
lishing common ground in human-robot joint activity. 

An example of the application of a dialog-based 
model to a task domain that traditionally involved su¬ 
pervisory control is Fong's et al. [71.136] collaborative 
control system. In this system, the human and the 
robot collaborated as partners to perform tasks such 
as navigation, collaborative exploration, and multirobot 
teleoperation and achieve shared goals within these 
tasks. The interaction between the robot and its hu¬ 
man counterpart involved engaging in dialog to share 
information and control at key points in the task. For in¬ 
stance, when the robot encountered an obstacle, it asked 
the user, Can I drive through <image>? along with an 
image of the obstacle. In asking these questions, the 
robot drew on specific attributes of the user, such as re¬ 
sponse accuracy, expertise, availability, efficiency, and 
preferences to determine whether or not it should direct 
specific questions to its user. 

A number of proposed models and systems take the 
dialog-based interaction paradigm further to involve the 
robot and its human counterpart jointly addressing the 
domain task and dialog itself as joint action [71.137, 
138]. In this peer-to-peer setup, either party selects 
goals to address and strategies to be used to address 
them and either party performs any part of the task. 
The model proposed by Foster et al. [71.137] includes 
a semantic interpretation module and a central decision¬ 
making module which draw on resources, such as a his¬ 
tory of the ongoing discourse between the robot and its 
user, a world model, a domain planner, and a represen¬ 
tation of the plan that is currently being executed, in 
order to generate action and communication behaviors. 

The model proposed by Li et al. [71.138] draws on 
joint intention theory [71.1 39], considering the joint ac¬ 
tivity to involve a common persistent goal of achieving 
conversational grounding, and explicitly uses elements 


of grounding in representing conversational contribu¬ 
tions. These contributions involve a presentation and 
an acceptance phase. For example, when an agent asks 
a question and the other agent answers, the question 
becomes the presentation and the answer becomes the 
acceptance, forming a grounded exchange. The model 
considers exchanges that involve a presentation without 
an acceptance to be ungrounded. Discourse contribu¬ 
tions take place at two layers: intention and conver¬ 
sation. At the intention layer, the system plans com¬ 
munication intentions based on analyses of previous 
discourse and the robot’s control system. These inten¬ 
tions can be self- or other-motivated for each agent. 
The conversation layer involves the articulation of com¬ 
munication intentions through verbal and nonverbal 
behaviors. The two layers form an interaction unit (IU) 
in the model. The model determines whether an IU is 
presentation or acceptance and whether it is grounded 
or ungrounded by assessing whether it satisfies joint in¬ 
tentions of the agents. Figure 71.12 illustrates how an 
other-motivated exchange is assessed by the model to 
determine whether the exchange is a presentation or an 
acceptance. 

Models of Situated Hu man-Robot Dialog 
The models and systems described above consider task- 
based and communicative exchanges in HRI as a dialog 
and extend models of spoken dialog to accommodate 
requirements that are specific to HRI, such as task- 
management, mixed-initiative dialog management, and 
physically situated referencing. Research in cognitive 
HRI has also explored the development of dialog sys¬ 
tems that explicitly integrate these mechanisms into 
dialog modeling and the development of specific mod¬ 
els and mechanisms for these requirements. 

An example of dialog systems that are specifi¬ 
cally developed for situated human-robot dialog is 
the pattern-based mixed-initiative (PaMini) HRI frame¬ 
work [71.140]. This framework extends spoken dialog 
systems with two key components: a task-state protocol 
and interaction patterns. The task-state protocol com¬ 
ponent explicitly defines tasks that either the robot’s 
perceptual or control subsystems can perform. A task 
is defined as an execution state and preconditions for 
execution. The task-state protocol specifies task states 
and transitions among them to support coordination. 
The interaction patterns component provides high-level 
representations of recurring dialog structures such as 
a clarification. A comparison of most commonly used 
spoken dialog systems and the PaMini framework in the 
context of a human-robot situated learning scenario is 
provided by Peltason and Wrede [71.141]. 

Another example is the Robot Behavior Toolkit de¬ 
veloped by Huang and Mutlu [71.3], which supports 
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Fig. 71.12 The model evaluates an 
exchange provided by the interaction 
partner to determine its presentation 
or acceptance status and determine an 
appropriate action (after [71.138]) 


situated human-robot dialog by integrating nonverbal 
cues for task-based referential communication and con¬ 
versation into the robot’s speech. This system uses 
a repository of specifications of situated communica¬ 
tion cues based on models of human interactions and 
an activity model (described in more detail below) that 
specifies the joint human-robot activity including the 
agents, task context, shared task goals, and expected 


<behaviors> 

<channel type='gaze'> 

<action endTime='214.5' startTime='0' target='unspecified'/> 
<action endTime=' 1160' startTime='214.5' target='the green 
object with one peg'/> 

<actoin endTime='2735.4' startTime='1160' target='unspecified'/> 
<action endTime='3597' startTime='2735.4' target='the red box'/> 
<action endTime='4308' startTime='3597' target='unspecified'/> 
<action endTime='4963' startTime='4308' target='listener'/> 
</channel> 

<channel type= speech> 

Could you help me put the green object with one peg into the red 
box, please? 

</channel> 

</behaviors> 
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Fig. 71.13 The robot behavior toolkit uses specifications from 
a repository and a model of the joint activity to integrate effective 
multimodal task-related dialog behaviors (after [71.3]) 


task outcomes to integrate the situated communication 
cues that are expected to support these outcomes into 
the robot’s speech. Figure 71.13 displays an example 
behavior generated by the Toolkit in a collaborative ma¬ 
nipulation task. An evaluation of their system showed 
that interactions in which the robot displayed these sit¬ 
uated communication cues as directed by the system 
more effectively supported desired task outcomes com¬ 
pared with baseline interactions (l<sz>»'ii')4«IH:at 

Research in cognitive HRI has also explored the de¬ 
velopment of models for specific communication and 
coordination mechanisms in situated interaction, such 
as perspective-taking, spatial referencing, reference res¬ 
olution, and joint attention d<s&Wii>)j'li'kMt 

Perspective-Taking. A core process in situated in¬ 
teraction toward establishing common ground is 
perspective-taking [71.142]. Research in social cogni¬ 
tion has shown that the ability to take another’s perspec¬ 
tive and share common ground significantly improves 
collaborative performance in human teams [71.143]. 
Research in HRI has also explored how robots might 
employ this core mechanism to establish common 
ground with their users in situated interactions and 
has proposed several models that supported perspective 
taking. 

Trafton et al. [71.144] studied interactions among 
astronauts in a naturalistic collaborative assembly task 
and found that a quarter of the utterances in the data 
involved taking the perspective of another and that 
participants frequently switched among egocentric, ex- 
ocentric, addressee-centered, and object-centered per¬ 
spectives. Based on their results, they developed a cog- 
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nitive model of perspective taking that allowed the 
robot to maintain multiple perspectives - or alternative 
worlds - at once and explore propositions about these 
worlds, such as the perspective of an interaction partner. 
This exploration allowed the robot to make inferences 
about the perspective of its partner by simulating this 
alternative world and act on the world from this per¬ 
spective. The following sequences of actions illustrate 
the simulations that the robot might carry out based 
on the command go to the cone (adapted from Trafton 
et al. [71.2]). Underlined text describes components of 
the system implementation: 

Simulate current real world (i. e., perceive it) 
Perception specialist notices the existence and 
location of person, cone\, conei, and obstacle 
Language specialist hears Coyote, go to the 
cone and infers that there is an object, C, that 
is a cone and that the person wants it to go to 
Identity hypothesis specialist infers that C can 
be identical to cone 1 or cone 2 
C = cone 1 , C = cone 2 

Identity constraint specialist notices a contra¬ 
diction 

This contradiction triggers the counterfactual 
simulation strategy 

Simulate the world where C = cone 1 

Because in this world person has referred to 
cone 1 , the perspective-simulation strategy is 
triggered 

Simulate the world where C = cone 1 and 
robot = person 

The spatial reasoning perspective indicates 
that cone\ does not exist in this world be¬ 
cause person cannot see it 
Thus, C cone 1 

Simulate the world where C = cone 2 

Because in this world person has referred to 
cone 2 , the perspective-simulation strategy is 
triggered 

Simulate the world where C = cone 2 and 
Robot = Person 

Because cone 2 is visible in this world, there 
is no contradiction in this world 
Infer that C = cone 2 (i- e., the cone refers to 
cone 2 ) 

Following a counteifactual simulation strategy pro¬ 
vides the robot with the ability to make inferences about 
situated actions across alternative scenarios with alter¬ 
native physical (e.g., whether or not an object is present) 
and cognitive (e.g., whether or not the object is visible 
to the human counterpart) characteristics and determine 
appropriate next actions, such as carrying out a request 
or seeking clarification from its human counterpart. 


Figure 71.14 illustrates four alternative scenarios with 
different physical and cognitive properties explored by 
Trafton et al. [71.144], In each scenario, the robot as¬ 
sesses these properties to determine its next actions, as 
illustrated below. 


Algorithm 71.1 

function: ScenariofnCones = 1 => cone a ) 
if cone a = visible m b ot A cone a = visible human then 
Go to cone a 

end if 

function: ScenariofnCones = 2 => cone a , conef) 
if cone a , cone/, = visible m \, ot A cone a = visible^ um an 

then 

Go to cone a 

end if 

function: (Scenario (nCones = 1 =t> cone a ) 
if cone a ,coneb ^ visible m \, ol A cone a = visible human 

then 

Check hidden location 

end if 

function: Scenario(nCone? = 2 => cone a , conet, ) 
if cone a , conet, = visible m \, ol A cone a , 
conet, = visible human 

then 

Request clarification 

end if 


Berlin et al. [71. 145] developed a similar model that 
enabled the robot to understand its environment from 
the perspective of an interaction partner by maintaining 

a) b) 




c) d) 




Fig.71.14a-d The alternative scenarios considered by the 
system in which the robot and its human counterpart 
are in a room with several objects and possible occlu¬ 
sions from the perspectives of the robot or the human 
(after [71.144]) 
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separate and potentially different sets of beliefs in its 
belief system for itself and for its interaction partner. To 
construct a model of the beliefs of its interaction part¬ 
ner, the robot employed the same mechanisms it used 
to model its own beliefs but transformed the data it per¬ 
ceived from the world to match the reference frame of 
its interaction partner. These two sets of beliefs were 
maintained separately so that the robot can compare dif¬ 
ferences between its beliefs and its interaction partner’s 
beliefs and plan actions in order to establish common 
ground or identify discrepancies in its learning in the 
context of task learning. Figure 71.15 illustrates paral¬ 
lel beliefs maintained by the robot in a button-pressing 
task. 

Spatial Referencing. Moratz et al. [71.146] proposed 
a cognitive model of spatial reference that represented 
different kinds of spatial reference systems and allowed 
the robot to interpret instructions from an interaction 
partner. This model mapped the locations of all objects 
as projections on a plan view, considering the robot’s 
point of view as origin and the location of the object 
that will be used as relatum to determine the reference 
axis. This axis enabled the robot to interpret directions 
such as left of, right of, in front of, and to the back in 
relation to the relatum, providing the robot the ability 
to interpret natural language references to objects in the 
environment. 

Reference Resolution. Ros et al. [71.147] extended 
these approaches to develop a model that enabled the 
robot to clarify references made by its interaction part¬ 
ner. This model employed several mechanisms includ¬ 
ing visual perspective taking, spatial perspective taking, 
symbolic location descriptors, and feature descriptors 
to determine whether it needed any clarification on 
its interaction partner’s references. The visual perspec¬ 
tive taking mechanism allowed the robot to determine 
whether or not objects in the environment were in its in¬ 
teraction partner’s focus of attention (FOA), in its part¬ 


ner’s field of view (FOV), or out of its partner’s field of 
view (OOF). The spatial perspective taking mechanism 
maintained egocentric and addressee-centered perspec¬ 
tives to determine ambiguities in object references. The 
system also included symbolic location descriptions 
such as is in, is on, and is next to to determine spatial 
relationships between objects and the environment. Fi¬ 
nally, the robot used feature descriptors such as color 
and shape to identify ambiguities in the references of 
its interaction partner. Once the robot determined the 
need clarification in its partner’s references, it used an 
ontology-based clarification algorithm to ask questions 
to its partner about the object of reference. 

Joint Attention. Another key mechanism in situated 
interaction is joint attention - the ability to use non¬ 
verbal cues, such as gaze and pointing, to establish 
common ground on what referents in the environment 
are under consideration in the dialogue [71.149]. Scas- 
sellati [71.99] proposed a task-based decomposition of 
joint attention skills, including mutual gaze, gaze fol¬ 
lowing, imperative pointing, and declarative pointing, 
and implemented these skills in a robot as stages for es¬ 
tablishing joint attention with a human counterpart. The 
mutual gaze skill provided the robot with the ability to 
recognize and maintain eye contact with its interaction 
partner. At the gaze following stage, the robot followed 
the eyes of its partner to direct its attention to the object 
of its partner’s attention. Imperative pointing involved 
pointing at an object that is out of reach in order to re¬ 
quest the object. Finally, the declarative pointing stage 
involved extending an arm and index finger to draw 
attention to an object that is out of reach without neces¬ 
sarily requesting the object. 

Connection Events. Rich etal. [71.150] argued that 
mechanisms such as joint attention serve as connec¬ 
tion events in situated dialog and establish and maintain 
engagement among interaction partners. From data on 
human interactions, they identified a set of key connec- 
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Fig. 71.15 In the sys¬ 
tem proposed by Berlin 
etal. [71.145], the robot 
maintains a parallel beliefs 
for itself and for its human 
counterpart for the task and 
updates its beliefs based on 
sensory input and those of 
the user based on the user’s 
awareness (after [71.145]) 
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Fig. 71.16 The policy com¬ 
ponents, the BML realizer, 
and the generated connec¬ 
tion events for the robot in 
the engagement generator 
module (after [71.148]) 


tion events, including mutual gaze, directed gaze, adja¬ 
cency pairs, and backchannels, and developed a system 
that recognized these events in human counterparts and 
generated them for a robot (see Rich et al. [71.150] for 
details on the recognizer and Holroyd et al. [71. 148] for 
details on generation). The recognizer module included 
dedicated recognizers for each type of connection event 
and an estimator for engagement levels for the robot’s 
human counterpart, while the generation module in¬ 
cluded four policy components and a behavior mark-up 
language (BML) realizer for generating robot behaviors 
toward establishing and maintaining engagement. The 
components of this engagement generator are illustrated 
in Fig. 71.16. 

71.3.2 Simulation-Theoretic Models 

Research in cognitive HRI has also been inspired by 
neurocognitive mechanisms in developing models of 
human-robot joint activity, building particularly on 
simulation theory, which suggests that people (and 
primates) represent other people’s mental states by 
adopting their perspective, specifically by tracking or 
matching their states with resonant states of their 
own [71.151]. This simulation-theoretic approach led 
to several models of robot behavior and human-robot 
joint action that involve the robot imitating or simu¬ 
lating the behaviors of its interaction partner in order 
to learn from or make inferences about its partner’s 
goals. 

As an example of this approach, Bicho et al. [71. 1 52] 
proposed a model for action preparation and decision¬ 
making in cooperative human-robot tasks that is in¬ 
spired by the finding that action observation elicits an 
automatic activation of motor representations associ¬ 
ated with the execution of the observed action. This 
motor-resonance mechanism allows people to internally 
simulate action consequences using their own motor 
repertoire and predict the consequences of action of oth¬ 
ers. In the proposed model, a perception-action linkage 
enables efficient coordination of actions and decisions 
between the agents in a human-robot joint action task. 


The model integrates a mapping between observed ac¬ 
tions and complementary actions in memory, while 
taking into account the inferred goals of the actions of 
the interaction partner, contextual cues, and shared task 
knowledge. 

Building on simulation theory, Gray et al. [71.153] 
proposed a similar system in which the robot parses 
user actions and matches the user’s movements to 
movements in its own repertoire toward making infer¬ 
ences about the user’s goals and perform a task-level 
simulation (Fig. 71.17). This simulation allows the 
robot to determine the preconditions of the schemas 
that represent the task and track its human partner’s 
progress over the course of the task in order to antic¬ 
ipate its partner’s needs and offer relevant help accord¬ 
ingly. The simulation also provided the robot with the 
ability to make inferences on the beliefs of its part¬ 
ner and simulate its partner’s perspective in a fashion 
similar to the perspective-taking mechanisms proposed 
by Trafton et al. [71.144] and Berlin et al. [71.145] 

Aspects of the simulation-theoretic approach ex¬ 
plicitly taken in these examples can also be seen in other 
control architectures developed for HRI. Nicolescu and 
Mataric [71.154] proposed a control architecture that 
unifies perception and action to achieve action-based 



Fig. 71.17 The mapping of perceived human actions onto the 
robot’s body in order to make comparisons and task-level inferences 
(after [71.153]) 
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interaction. In this architecture, behaviors are built from 
perceptual and active components. Perceptual compo¬ 
nents allow the robot to link its observations and actions 
and thus to learn to perform a task from the experiences 
it gains from its interactions with people. Active com¬ 
ponents enable task-based behaviors that also serve as 
implicit communication rather than explicit behaviors 
such as speech and gestures. Behavior representation 
in the architecture captures two types of behaviors: 
abstract and primitive. Abstract behaviors are explicit 
specifications of the behaviors’ activation conditions 
(preconditions), goals in the form of abstracted en¬ 
vironmental states, and effects (postconditions), while 
primitive behaviors are those that the robot performs 
to achieve these effects. By linking perceptions and ac¬ 
tions, the robot learns what actions of its own might 
achieve the same observed effects. 

71.3.3 Intention- and Activity-Based 
Models 

The models and systems described above are concerned 
primarily with establishing and maintaining common 
ground and coordinating actions in task-based interac¬ 
tions using dialog- and simulation-theoretic approaches 
with limited consideration of the broader context of 
these interactions as complex activities involving mul¬ 
tiple agents with common goals and commitments to 
these goals. A number of models and systems sought 
to address this limitation, building on models and the¬ 
ories of human joint activity such as joint intention 
theory [71.139] and activity theory [71.7]. 

Building on joint intention theory, Breazeal 
et al. [71.155] proposed a model of human-robot col¬ 
laboration that involved dynamically meshing subplans 
into joint activity toward achieving common goals of 
the human-robot team. In this model, task and goal 
representations have a goal-centric view, employing 
an action-tuple data structure that captures precondi¬ 
tions, executables, until-conditions, and goals. Tasks 
are represented in a hierarchical structure of actions and 
recursively defined subtasks. Goals are also represented 
hierarchically as overall intent rather than a chain of 
low-level goals. The implemented joint intention model 
dynamically assigns tasks to members of the human- 
robot team. These intentions are derived based on the 
robot’s actions and abilities, the actions of the human 
partner, the robot’s understanding of the common goal 
of the team, and its assessment of the current task state. 
At every stage of the interaction, the robot negotiates 
who should complete the task. Action at these points 
might look like turn-taking or simultaneous action (the 
robot and the human working on different parts of the 
task). 


Alami et al. [71.156,157] similarly built on joint 
intention theory to propose a human-robot decision 
framework in which team members are committed to 
a joint persistent goal and follow cooperation schemes 
to contribute toward achieving this goal. The framework 
involves a goal planner called the agenda for the robot 
and human collaborators to pursue, a proxy representa¬ 
tion of the human in the robot called Interaction Agents 
(IAA), task delegates that monitor and control the task 
commitment of the human or the robot for each active, 
inactive, or suspended goal, and a robot supervision ker¬ 
nel that monitors and controls robot activities. For each 
new active goal, the Robot Supervision Kernel creates 
a Task Delegate, selects or elaborates a plan, and allo¬ 
cates the roles of each team member. 

Fong et al. [71.1] proposed a similar system called 
the HRI operating system (HRI/OS) to support human- 
robot teamwork. The system involves a task manager, 
resource manager, interaction manager, spatial reason¬ 
ing agent, context manager, human and robot agents, 
and an open agent architecture (OAA) facilitator. The 
task manager decomposes the overall goal of the sys¬ 
tem into high-level tasks and assigns to humans or 
robots for execution. The manager relies on the agents 
to complete the low-level steps of the tasks. It com¬ 
municates with the Resource Manager to find an agent 
capable of performing the work. Resource manager pro¬ 
cesses all agent requests, prioritizing the list of agents 
to be consulted when a task needs to be performed. 
Interaction manager coordinates dialog-based commu¬ 
nication between agents. Context manager keeps track 
of everything that occurs while the system is running 
including task status and execution, agent activities, 
agent dialogue, etc. Spatial reasoning agent (SRA) is 
used to resolve spatial ambiguities in human-robot di¬ 
alog through mechanisms such as perspective taking 
and frames of reference, resolving ambiguities among 
as ego-, addressee-, object-, and exo-centric references. 
To do this, SRA transforms the spatial dialog into a ge¬ 
ometric reference and perform a mental simulation of 
the interaction to explore how ambiguities might be 
resolved through multiple references. Finally, the OS 
includes a software representation of the human - a hu¬ 
man proxy agent that represents user capabilities and 
accepts task assignments in the way that robot agents 
do. These proxies represent task capabilities, includ¬ 
ing domains of expertise, and provide health monitoring 
feedback. 

Huang and Mutlu [71.3] built on an alternative 
model of human activity - activity theory [71.7] - to 
develop a model of human-robot joint activity. Their 
model builds on five key constructs from activity the¬ 
ory including consciousness, object-orientedness, hi¬ 
erarchical structure, internationalization and external- 
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ization, and mediation. The consciousness construct 
pertains to attention, intention, memory, reasoning, and 
speech and includes specific representations for atten¬ 
tion and intention. The object-orientedness construct 
describes material artifacts, plans of action, or common 
ideas to be shared by the members of the joint activ¬ 
ity. Following the hierarchical structure construct, the 
model organizes joint activity into three layers: activity, 
action, and operation. An activity consists of a series 
of actions that share the same goal, and each action 
has a defined goal and a chain of operations that are 
regular routines performed under a set of conditions. 
Internalization and externalization describes cognitive 
processes; internalization involves transforming exter¬ 
nal actions or perceptions into mental processes, while 
externalization is the process of manifesting mental 
processes in external actions. Finally, the mediation 
construct defines several external and internal tools, 
such as physical artifacts that might be used in an ac¬ 
tivity and cultural knowledge or social experience that 
an individual might have acquired, as mediators of 
human-robot joint activity. These constructs and their 
corresponding system elements allow the construction 
of and planning for joint human-robot activities. For 
each activity, a motive governs actions. Each action, 
by achieving its corresponding goal, helps to fulfill the 
motive of the activity. Each action may have several op¬ 
erations that are constrained by a set of conditions and 
that can be executed only when all the conditions are 
met. Actions have predefined outcomes, which spec¬ 
ify the orientation of an action. Figure 71.18 shows the 
XML (extensible markup language) representation of 
a model of a collaborative manipulation task. 

71.3.4 Models for Action Planning 

The models described above primarily enable commu¬ 
nication and coordination between humans and robots 
toward planning and carrying out joint tasks. In order to 
successfully contribute to these tasks, robots also need 
models for planning their actions in a dynamic physical 
and cognitive environment. Research in cognitive HRI 
seeks to develop models for action planning that help 
robots estimate the actions that they have to take in or¬ 
der to achieve task goals and learn the parameters of 
the tasks space. The paragraphs below review research 
in two common approaches to building such models: 
decision-theoretic models and model learning. 

Decision-Theoretic Models 
One of the simplest approaches to control and decision¬ 
making in HRI is to define the interaction as a decision- 
theoretic planning problem, such as a Markov decision 
process (MDP). Formally, an MDP consists of the n- 


tuple {5, A, T, R , y}. The set S is a set of states, which 
in the HRI setting typically correspond to the combina¬ 
tion of state variables, such as the robot state and the 
desired outcome of the interaction. For example, if the 
interaction model allows a human partner to instruct the 
robot to move to different locations in the environment, 
one state variable may correspond to different current 
locations of the robot and another state variable may 
correspond to the goal states intended by the human 
partner. The full state space S is given by the combina¬ 
tion of possible values for the different state variables. 

The action set A represents actions that the robot 
may take. The actions may include asking a question, 
performing some physical movement, or even doing 
nothing. Each action has a cost R depending on the 
current state, which rewards the robot for performing 
useful actions, and penalizes the robot for taking ac¬ 
tions that either make no immediate progress toward the 
specified goal (typically a small penalty) or completely 
unhelpful (a large penalty). 

Lastly, the transition function T provides a notion 
of the dynamics of the environment in terms of how the 
state changes as robot takes actions, and especially how 
a human partner’s state variables may change as the 
robot takes actions. The transition function T(/|j, a) 
places a probability distribution over the states to which 
the user in state may transit if the robot takes action a. 
The MDP formulation is very appealing, because there 
exist efficient techniques for solving for interaction 
policies. Once the policy is computed, the interaction 
can be managed simply by querying the policy for the 

<Activity id= 1> 

<Mofzve>clear(table)</Moftve> 

<Description>Clear objects on table</Description> 
<Participants>Self, Userl</Participants> 

<Action id= 1> 

<Outcome>Task</Outcome> 

<Goa/>disappear(object)</Goa/> 

<Description> 

Instruct Userl to categorize object 
</Description> 

<Operation type= utterance > 

<Condi tzon>present(User 1 )</ Condition> 

<Condition> 

known(the blue object with two pegs) 

</Conditicm> 

<Condition> known) the blue box )</Condition 

<Info tum='end"> 

Could you help me put the blue object with 
two pegs into the blue box, please? 

</Info> 

</Operation> 

Fig. 71.18 The XML representation of the activity-theory- 
based model for a collaborative manipulation task (af¬ 
ter [71.3]) 
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appropriate action in response to the current state of the 
robot and the human partner. 

A limitation of the MDP approach is that some of 
the state variables may not be directly observable, in 
particular the state variables corresponding to human 
intentional states, such as intended goal locations of 
the robot. The values of the state variables must be 
inferred from observations, such as speech acts per¬ 
formed by the human partner, which are inherently 
noisy. For example, the system may hear the words cof¬ 
fee machine when the user asks the robot to go to copy 
machine. While speech recognition errors may be mit¬ 
igated to some extent by asking the user to use only 
acoustically distinct keywords when speaking to the 
system, a system that does not model the likelihood of 
recognition errors and act accordingly will be brittle; 
a robust system must be able to infer user intent under 
uncertainty. 

The observations are rarely sufficient to uniquely 
determine the current state, but more commonly are 
used to compute a belief or probability distribution over 
dialog states. If the agent takes some action a and hears 
observation o from an initial belief h. it can easily up¬ 
date its belief using Bayes rule 

, a ,o n = n{o\s\a) E.esTVMlfrU) 

JLass^i°\ <J ^ a )T.sSS T ^ (J \ s ' a ) b i s ) 

( 71 . 1 ) 

This probability distribution will evolve as the di¬ 
alog manager asks clarification questions and receives 
responses. In Fig. 71.19, we show a cartoon of a simple 
dialog model. Initially, we model the user as being in 
a start state. Then, at some point in time, the user speaks 
to the robot to indicate that he or she wants it to perform 
a task. We denote this step by the set of vertical stack of 
nodes in the center of the model. Each node represents 
a different task. The dialog manager must now inter¬ 
act with the user to determine what is wanted. Once the 
task is successfully completed, the user transitions to 
the right-most end node, in which he or she again does 
not desire anything from the robot. We note that it can 
be easily augmented to handle more complex scenarios. 
For example, by including the time of day as part of the 
state, we can model the fact that the user may usually 
wish to go to certain locations in the morning and other 
locations in the afternoon. 

Intuitively, we can see how the belief can be used 
to select an appropriate action. For example, if the di¬ 
alog manager believes that the user may wish to go to 
either the coffee machine or the copy machine (but not 
the printer), then it may ask the user for clarification 
before commanding the wheelchair to one of the loca¬ 
tions. More formally, we call the mapping from beliefs 


to actions a policy. We represent this mapping using the 
concept of a value function V(b). The value of a belief 
is defined to be the expected long-term reward the dia¬ 
log manager will receive if it starts a user interaction 
in belief b. The optimal value function is piecewise- 
linear and convex, so we represent V with the vectors 
V,-; V(b) = max,- V t ■ b. The optimal value function sat¬ 
isfies the Bellman equation [71.158] 

V(b) = max Q(b, a) , 

aSA 

Q(b.a) =R(b,a) + y^£2(o\b,a)V(bl) , ( 71 . 2 ) 

oSO 

where Q(b,a ) represents the expected reward for start¬ 
ing in belief b, performing action a, and then acting 
optimally. The belief b° is b after a Bayesian update of b 
using (71.1), and Z2(o\b, a), the probability of seeing o 
after performing a in belief b (%2 ?es Z2(o\s,a)b(s)). 

There are also non-Bayesian approaches for acting 
in uncertain environments. Many interaction systems 
provide the dialog manager with a set of rules to follow 
given particular outputs from a speech recognition sys¬ 
tem. The drawback to rule-based systems is that they 
often have difficulty managing the many uncertainties 
that stem from noisy speech recognition or linguis¬ 
tic ambiguities. The ability to manage the trade-off 
between gathering additional information and servic¬ 
ing a user’s request have made partially observable 
Markov decision process (POMDP) planners partic¬ 
ularly useful in dialog management; applications in¬ 
clude a Nursebot robot, designed to interact with the 
elderly in nursing homes [71.159], a vision-based sys¬ 
tem that aids Alzheimer’s patients with basic tasks 
such as hand-washing [71.160], an automated tele- 



Fig. 71.19 A toy example of a dialog POMDP. The nodes 
in the graph are different states of the dialog (i. e., user re¬ 
quests). Solid lines indicate likely transitions; we assume 
that the user is unlikely to change their request before their 
original request is fulfilled. The system automatically re¬ 
sets once we reach the end state 
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phone operator [71.161], and a tourist information 
kiosk [71.162], 

Beyond the initial formulations of cognitive HRI as 
a decision-theoretic problem, there have been a number 
of algorithmic improvements that increase the domains 
of applicability of this approach. For example, the con¬ 
ventional MDP and POMDP algorithms have typically 
assumed that each observation and action takes approx¬ 
imately the same amount of time, which can lead to an 
implicit bias toward longer actions. Representing time 
explicitly leads to computational intractability, but Broz 
et al. [71.163] demonstrated that the similar states that 
vary only by the time-index can be aggregated, lead¬ 
ing to reduced-order models that can be solved very 
efficiently. Similarly, Doshi and Roy [71.164] showed 
that symmetries in human intentional states could be 
exploited to dramatically reduce the size of the plan¬ 
ning problem, also leading to very efficient solutions. 
Most recently, again in the non-Bayesian line, Wilcox 
et al. have shown that the temporal dynamics of task- 
based HRI can be formulated as a scheduling prob¬ 
lem [71.165]. 

Model Learning 

The behavior of the dialog manager derived from solv¬ 
ing (7 1 .2) depends critically on accurate choices of the 
transition probabilities, observation probabilities, and 
the reward. For example, the observation parameters 
affect how the system associates particular keywords 
with particular requests. Similarly, the reward function 
affects how aggressive the dialog manager will be in 
assuming that it understands a user’s request, given lim¬ 
ited and noisy information. An incorrect specification 
of the dialog model may lead to behavior that is either 
overly optimistic or conservative, depending on how ac¬ 
curately the model captures the user’s expectations on 
the interaction. 

A common approach in other domains is to collect 
data using a fixed policy, typically referred to as system 
identification. In HRI, this is easiest to perform using 
so-called Wizard of Oz studies where a human experi¬ 
menter executes the policy unseen to generate data or 
evaluate a policy. Prommer et al. [71.166] showed that 
Wizard-of-Oz studies could be used effectively not only 
to learn model parameters for an MDP dialog model, 
but also to learn an effective policy. 

At the same time, learning all the parameters re¬ 
quired to specify a rich dialog model can require 
a prohibitively large amount of data. While the model 
parameters may be difficult to specify exactly, either by 
hand or from data, we can often provide the dialog man¬ 
ager with an initial estimate of the model parameters 
that will generate a reasonable policy that can be exe¬ 
cuted while the model is improved. For example, even 


though we may not be able to attach an exact numerical 
value to driving a wheelchair user to the wrong location, 
we can at least specify that this behavior is undesirable. 
Similarly, we can specify that the exact numerical value 
is initially uncertain. As data about model parameters 
accumulate, the parameter estimates should converge to 
the correct underlying model with a corresponding re¬ 
duction in uncertainty. 

Figure 71.20a depicts the conventional model, 
where the arrows in the graph show which parts of the 
model affect each other from time t to t + 1. Although 
the variables below the hidden line in Fig. 71.20a are 
not directly observed by the dialog manager, the pa¬ 
rameters defining the model (i. e., the parameters in 
the function giving the next state) are fixed and known 




Fig. 71.20 (a) The standard POMDP model, (b) The ex¬ 
tended POMDP model. In both cases, the arrows show 
which parts of the model are affected by each other from 
time t to t + 1. Not drawn are the dependencies from time 
t- 1-1 onward, such as the user state and user model’s effect 
on the recognized keyword at time t + 1 
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a priori. For instance, the reward at time t is a function 
of the state at the previous time and the action chosen 
by the dialog manager. 

If the model parameters are not known a priori 
because the model is uncertain - for example, how 
much reward is received by the agent given the previ¬ 
ous state and the action selected - then the concept of 
the belief can be extended to also include the agent’s 
uncertainty over possible models. In this new represen¬ 
tation, which we call the model-uncertainty POMDP, 
both the user’s request and the model parameters are 
hidden. Figure 71.20b shows this extended model, in 
which the reward at time t is still a function of the 
state at the previous time and the action chosen by 
the dialog manager, but the parameters Eire not known 
a priori and are therefore hidden model variables that 
must be estimated along with the user state. The sys¬ 
tem designer can encode their knowledge of the system 
in the dialog manager’s initial belief over what dialog 
models it believes are likely - a Bayesian prior over 
models - and let the agent improve upon this belief with 
experience. 

Poupart et al. treated the unknown MDP parameters 
as hidden state in a larger POMDP and derived an ana¬ 
lytic solution (based on [71.167]) for a policy that will 
trade optimally between learning the MDP and max¬ 
imizing reward. Unfortunately, these techniques did 
not extend tractably to the model-uncertainty POMDP, 
which is continuous in both the POMDP parameters 
(like the MDP) and the belief state (unlike the MDP). 
Doshi and Roy [71.168, 169] provided an approximate, 
Bayes risk action selection criterion that allows the 
dialog manager to function in this complex space of 
dialog models. This approach was applied to the intel¬ 
ligent wheelchair assistant shown in Fig. 71.21. Their 
goal was to design an adaptable HRI system, or dialog 
manager, that allows both the user of the wheelchair 
and a CEiregiver to give natural instructions to the 
wheelchair, as well as ask the wheelchair computer for 
general information that may be relevant to the user’s 
daily life. 

In contrast to the Bayesian approach, Cakmak and 
Thomaz [71.170] pursued an active learning approach 
and identified three types of queries that a robot could 
generate while learning a new task (l<£aJKIEB3EJH). 
While this result does not provide a comparison to an 
approach embedded in an ongoing dialogue, their re¬ 
sults do provide guidelines for model designers. 

71.3.5 Cognitive Models of Robot Control 

A final line of research in cognitive HRI seeks to 
achieve greater task efficiency in human-robot teams, 
thus addressing common problems between opera¬ 


tors and robots such as those identified by Burke 
et al. [71.133] and Stubbs et al. [71.134], by developing 
models and control interfaces that exploit mechanisms 
of human cognition such as working memory and men¬ 
tal models [71.171,172]. This research includes for¬ 
malisms such as neglect time, the amount of time that 
an operator can neglect a robot before the robot’s per¬ 
formance drops below a certain threshold [71.171], and 
fan out, a measure of how many robots an operator can 
effectively manage in a human-robot team [71.172]. 
Such formalisms inform the development of guide¬ 
lines for designing effective control mechanisms such 
as the following principles proposed by Goodrich and 
Olsen [71.171]: 

1. Implicitly switch interfaces and autonomy modes. 
Context determines the mode of use. For instance, 
the user starts using a joystick and the interac¬ 
tion modality automatically switches, rather than 
the user explicitly selecting a modality. 

2. Let the robot use natural human cues. The robot 
uses the cues to provide feedback and present in¬ 
formation that the human uses to provide the com¬ 
mands or present information to the robot. 

3. Manipulate the world instead of the robot. Control 
interfaces integrate knowledge about the task and 
the world to minimize low-level control of the robot 
and maintaining of a mental model of the robot’s 
functioning. 

4. Manipulate the relationship between the robot 
and world. Control interfaces provide real-world 



Fig. 71.21 Our dialog manager allows for more natural 
human communication with a robotic wheelchair (af¬ 
ter [71.168,169]) 
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representations for control to minimize low-level 
control. 

5. Let people manipulate presented information. Inter¬ 
faces present information in a way that represents 
the real world and allows users to provide input di¬ 
rectly into the representation rather than translating 
information readings to a different modality or rep¬ 
resentation. 


71.4 Conclusion and Further Reading 

This chapter presented an overview of research in 
cognitive human-robot interaction, the area of re¬ 
search concerned with modeling human, robot, or joint 
human-robot cognitive processes in the context of HRI. 
This research seeks to gain a better understanding 
of people’s interaction with robots and build robotic 
systems with the necessary cognitive mechanisms to 
communicate and collaborate with their human coun¬ 
terparts. Three key themes fall within this research 
area. The first theme seeks to build a better understand¬ 
ing of human cognition in HRI; specifically, people’s 
mental models of robots as ontological entities, social 
cognition of robot behaviors, and the use of robots as 
experimental platforms to study cognitive development 
in humans. The second theme includes research that 
seeks to build models for simulating human cognition 
in robots, gaining cognitive capabilities through imi¬ 
tation and interaction with the physical environment, 
and mapping aspects of interaction, such as commands 
from or references by human counterparts to objects in 
the environment. The final theme seeks to build mod¬ 
els that support human-robot joint activity, including 
dialog-, simulation-theoretic-, joint-intention-, activity- 
and action-planning-based models that enable robots 
to reason about the physical and cognitive properties 
of the environment and the actions of their human 
counterparts and to plan actions toward achieving com¬ 
municative or collaborative goals. The common thread 
among these three themes of research is the consid¬ 
eration of humans and robots as part of a cognitive 

Video-References 


6. Externalize memory. Different types of information 
are integrated into a single representation to reduce 
the working memory load for the user. 

7. Help people manage attention. The robot provides 
appropriate indicators to capture the attention of the 
operator. 

8. Learn. Control mechanisms adapt system activity to 
the user’s mental models. 


system in which cognitive processes - natural or de¬ 
signed - shape how humans and robots communicate 
and collaborate. 

As an interdisciplinary area of research, cognitive 
human-robot interaction receives contributions from 
a diverse set of research fields including robotics, cogni¬ 
tive science, social psychology, communication studies, 
and science and technology studies. Further reading on 
the topic is also available in a diverse set of venues such 
as: 

• The Proceedings of the ACM/IEEE International 
Conference on Human-Robot Interaction {HRI) 

• The Proceedings of the Annual Meeting of the Cog¬ 
nitive Science Society (CogSci) 

• International Conference on Epigenetic Robotics 
(EpiRob) 

• The Proceedings of the AAAI Conference on Artifi¬ 
cial Intelligence 

• The Proceedings of the IEEE International Sympo¬ 
sium on Robots and Human Interactive Communi¬ 
cation (RO-MAN) 

• The Proceedings of the Robotics: Science and Sys¬ 
tems (RSS) Conference 

• Sun , [71.173] 

• Journal of Human-Robot Interaction 

• Interaction Studies: Social Behaviour and Commu¬ 
nication in Biological and Artificial Systems. lohn 
Benjamins 

• International Journal of Social Robotics. Sage. 








Gaze and gesture cues for robots 

available from http://handbookofrobotics.org/view-chapter/71/videodetails/128 
Robotic secrets revealed, Episode 1 

available from http://handbookofrobotics.org/view-chapter/71/videodetails/129 
Robotic secrets revealed, Episode 2: The trouble begins 
available from http://handbookofrobotics.org/view-chapter/71/videodetails/130 
Human-robot jazz improvization 

available from http://handbookofrobotics.org/view-chapter/71/videodetails/236 
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l<s>> 'imwa Design ing robot learners that ask good questions 

available from http://handbookofrobotics.org/view-chapter/71/videodetails/237 
Active keyframe-based learning from demonstration 

available from http://handbookofrobotics.org/view-chapter/71/videodetails/238 
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This chapter surveys some of the principal research 
trends in Social Robotics and its application to 
human-robot interaction (HRI). Social (or Socia¬ 
ble) robots are designed to interact with people in 
a natural, interpersonal manner - often to achieve 
positive outcomes in diverse applications such as 
education, health, quality of life, entertainment, 
communication, and tasks requiring collaborative 
teamwork. The long-term goal of creating social 
robots that are competent and capable partners 72.5 
for people is quite a challenging task. They will 
need to be able to communicate naturally with 
people using both verbal and nonverbal signals. 

They will need to engage us not only on a cognitive 
level, but on an emotional level as well in order to 
provide effective social and task-related support 
to people. They will need a wide range of social- 
cognitive skills and a theory of other minds to 
understand human behavior, and to be intuitively 72 ' 6 
understood by people. A deep understanding of 
human intelligence and behavior across multi¬ 
ple dimensions (i. e., cognitive, affective, physical, 
social, etc.) is necessary in order to design robots 
that can successfully play a beneficial role in the 
daily lives of people. This requires a multidisci¬ 
plinary approach where the design of social robot 72 7 
technologies and methodologies are informed by 
robotics, artificial intelligence, psychology, neuro¬ 
science, human factors, design, anthropology, and 
more. 
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72.1 Overview 

The way a person interacts with a social robot (or so¬ 
ciable robot) is quite different from interacting with the 
majority of autonomous mobile robots today. Modem 
autonomous robots are generally viewed as tools that 
human specialists use to perform hazardous tasks in re¬ 
mote environments (i. e., sweeping minefields, inspect¬ 
ing oil wells, mapping mines, etc.). In dramatic con¬ 
trast, social (or sociable) robots are designed to engage 
people in an interpersonal manner, often as partners, in 
order to achieve positive outcomes in domains such as 
education, therapy, or health, or task-related goals in ar¬ 
eas such as coordinated teamwork for manufacturing, 
search and rescue, domestic chores, and more. 

The development of socially intelligent and socially 
skillful robots drives research to develop autonomous or 
semiautonomous robots that are natural and intuitive for 
the general public to interact with, communicate with, 
work with as partners, and teach new capabilities. Daut- 
enhahn's work is among the earliest in thinking about 
robots with interpersonal social intelligence where 
relationships between specific individuals are impor¬ 
tant [72.1,2], These early works pose the question: 

What are the common social mechanisms of com¬ 
munication and understanding that can produce 
efficient, enjoyable, natural and meaningful inter¬ 
actions between humans and robots ? 

72.2 Social Robot Embodiment 

72.2.1 Anthropomorphic Design 

Social robots are designed to interact with people in 
human-centric terms and to operate in human envi¬ 
ronments alongside people. Many social robots are 
humanoid or animal-like in form, although this does 
not have to be the case. A unifying characteristic is 
that that social robots engage people in an interper¬ 
sonal manner, communicating and coordinating their 
behavior with humans through verbal, nonverbal, or af¬ 
fective modalities. People have a strong tendency to 
anthropomorphize social robots [72.12] and to reason 
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72.9.3 Research Direction . 1959 

72.10 Conclusion . 1959 

72.11 Further Reading . 1960 

Video-References . 1960 

References . 1961 


Promisingly, there have been initial and ongoing 
strides in all of these areas ([72.3-11], etc.). In addi¬ 
tion, this domain motivates new questions for robotics 
researchers, such as how to design for a successful 
long-term relationship where the robot remains ap¬ 
pealing and provides consistent benefit to people over 
weeks, months, and even years. The benefit that social 
robots provide people extends far beyond strict task¬ 
performing utility to include educational (Chap. 79), 
health and therapeutic (Chap. 64), domestic (Chap. 65), 
social and emotional goals (e.g., entertainment, com¬ 
panionship, communication, etc.), and more. 

We begin this chapter with a brief overview of 
a wide assortment of embodiments of socially interac¬ 
tive robots that have been developed around the world 
(Sect. 72.2). We follow with selected topics that high¬ 
light some of the representative research themes: social- 
emotional intelligence and emotion-based interaction 
(Sect. 72.3), and social-cognitive skills (Sect. 72.4). 
Human social responses to robots(Sect. 72.5), ver¬ 
bal and non-verbal communication (Sect. 72.6), long¬ 
term interaction (Sect. 72.7), touch-based interac¬ 
tion (Sect. 72.8), and teamwork with robot partners 
(Sect. 72.9). We rely on examples from our own re¬ 
search programs to illustrate these trends, while making 
reference to other excellent works performed in other 
research labs. 


about their behavior in terms of having their own men¬ 
tal states (e.g., thoughts, intents, beliefs, desires, etc.). 
Hence, anthropomorphic design principles, spanning 
from the physical appearance of robots, to how they 
move and behave, and how they interact with peo¬ 
ple, are often employed to facilitate interaction and 
acceptance. 

72.2.2 Design Space 

The design space of social robotics is quite large. It 
is important to note that a more human-like design 
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Fig.72.1a-c Examples of socially interactive humanoid robots: (a) Humanoid robots developed at Waseda University 
from left to right: a flutist robot WF-4RII (after [72.13]), WABIAN-2 (after [72.14]), and WE-4RII (after [72.15]); 
(b) Robovie developed at ATR (Advanced Telecommunications Research Institute, Kyoto) is able to gesture with its 
arms and give a hug; (c) Nexi and Maddox, developed at MIT, are mobile and dexterous social robots used to study 
collaborative human-robot teamwork (after [72.16]) 


does not necessarily correlate with a better design. One 
needs to balance the robot design with the task, user, 
and context [72.17], As can be seen in the following ex¬ 
amples, social robots exploit many different modalities 
to communicate and express social-emotional behavior. 
These include whole-body motion, proxemics (i. e., in¬ 
terpersonal distance), gestures, facial expressions, gaze 
behavior, head orientation, linguistic or emotive vocal¬ 
ization, touch-based communication, and an assortment 
of display technologies. 

For social robots to close the communication loop 
and coordinate their behavior with people, they must 
also be able to perceive, interpret, and respond appro¬ 
priately to verbal and nonverbal cues from humans. 
laaMTihltiHf shows dog-inspired robot behaviour to fa- 
ciliate intention reading by people. Given the richness 
of human behavior and the complexity of human en¬ 
vironments, many social robots are among the most 
sophisticated, articulate, behaviorally rich, and intelli¬ 
gent robots today. 

As shown in Fig. 72.1, a number of socially interac¬ 
tive humanoid robots have been developed (Chap. 65) 
that can participate in whole body social interaction 


with people such as dancing [72.22], walking hand- 
in-hand [72.23,24], playing a musical duet [72.13], or 
transferring skills to unskilled persons [72.25], or col¬ 
laborating as a team with people in search and retrieve 
tasks [72.16]. Their arms and hands are designed to ex¬ 
hibit human-like gestures such as pointing, shrugging 
shoulders, shaking hands, or giving a hug [72.26- 
28]. Some of them are designed with mechanical 
faces to communicate with humans via facial expres¬ 
sions [72.18,20,29], 

Whereas many of these humanoids have a mechan¬ 
ical appearance, android robots are designed to have 
a very human-like appearance with skin, teeth, hair, 
and clothes (Fig. 72.2). A design challenge of android 
robots is to avoid the uncanny valley where the appear¬ 
ance and movement of the robot resemble more of an 
animate corpse than a living human. Designs that fall 
within the uncanny valley elicit a strong negative reac¬ 
tion from people [72.37], In contrast to trying to look 
as human-like as possible, there are also more doll-like 
robots that are intentionally designed to have simplified 
facial cues and predictable movements to be suitable for 
therapeutic contexts [72.21]. 



Fig.72.2a-d Some examples of androids: (a) One of the earliest face robots developed at the Science University of 
Tokyo (after [72.18]); (b) Geminoid developed at ATR (after [72.19]); (c) ROMAN developed at the University of 
Kaiserslautern (after [72.20]); (d) KASPAR developed at the University of Hertfordshire is a child like robot used during 
therapeutic interventions to help children with autism (after [72.21]) 
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Fig.72.3a-d Examples of social robots inspired by animals with anthropomorphic qualities: (a) AIBO, the robotic dog 
developed by Sony (after [72.30]), (b) Paro, the therapeutic seal robot developed at AIST (after [72.31]), (c) Mel, the 
conversational robotic penguin developed at MERL (after [72.32]), and (d) Leonardo developed at the MIT Media 
Lab (after [72.33]) 



Fig.72.4a-d Examples of social robots that are neither humanoid nor zoomorphic but capture key social attributes: (a): 
Kismet (after [72.3]): (b) Keepon (after [72.34]); (c) Pearl (after [72.35]); (d) Valerie (after [72.36]) 


There are a number of more creature-like social 
robots that take their aesthetic and behavioral inspira¬ 
tion from animals (Fig. 72.3). Given that people pet 
and stroke companion animals, touch-based commu¬ 
nication has been explored in several of these more 
animal-inspired robots. Sony’s entertainment robot dog, 
AIBO [72.30,38], is a well-known commercial exam¬ 
ple. Other robots in this category have a more or¬ 
ganic appearance, such as the therapeutic companion 
robot seal, Paro [72.31]. Researchers have chosen to 
design robots with a more fanciful appearance, meld¬ 
ing anthropomorphic with animal-like qualities such as 
Leonardo ([72.32, 33, 39], etc.). 

Many social robots are not overtly humanoid or 
zoomorphic, but still capture key social attributes 
(Fig. 72.4). For instance, one of the best-known and 
pioneering social robots Kismet [72.3] developed at 
the MIT Artificial Intelligence Lab. Kismet had a very 
expressive mechanical face with anthropomorphic fea¬ 


tures like large blue eyes. Another example is the 
dancing robot Keepon developed by NiCT (Japan). This 
small yellow robot has a simplistic face and uses a clas¬ 
sic animation technique called squash and stretch for 
expression of the body [72.34]. 

Many mobile social robots have been fitted with 
faces to enhance social interaction (Fig. 72.4). Some 
examples are the eldercare robot. Pearl [72.35], and 
the robotic receptionist Valerie with a graphical face 
on a LCD (liquid crystal display) screen [72.36], both 
developed at Carnegie Mellon University. Other exam¬ 
ples are commercial robots like PaPeRo developed by 
NEC [72.40]. Still, some social robots have no overt 
social features like faces or eyes, but rely purely on 
language-based communication. Issues of proxemics on 
mobile social robots have also been explored such as 
how a robot should approach a person [72.41], follow 
a person [72.42], or maintain appropriate interpersonal 
distance [72.43]. 


72.3 Social Robots and Social-Emotional Intelligence 


Humans are fundamentally emotional beings. Conse¬ 
quently, human communication and social interaction 
often includes affective or emotive factors. To sup¬ 
port the emotional side of human behavior, researchers 
are exploring affective interaction and communication 


between people and robots. To participate in emotion- 
based interaction, robots must be able to recognize 
and interpret affective signals from humans, they must 
possess their own internal models of emotions (often in¬ 
spired by psychological theories), and they must be able 
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to communicate this affective state to others. In gen¬ 
eral, emotional displays can inform the interpretations 
about an individual’s internal states (agreement or dis¬ 
agreement about a belief, valuing a particular outcome, 
an action tendency to fight, etc.) and therefore help to 
predict future actions. Emotional displays can evoke 
emotional responses in others (e.g., displays of distress 
can elicit feelings if empathy and motive another to pro¬ 
vide social support). Given that social interactions can 
serve such a broad range of functions, prominent sci¬ 
entists have argued that emotions evolved because they 
provide an adaptive advantage to social species where 
individual relationships matter [72.44], 

A growing number of socio-emotional robots have 
been designed to realize such functions to facilitate 
human-robot interaction. Some of these robots have 
been designed with emotional responses or emotion- 
inspired decision making systems in order to entertain 
AIBO [72.30], QRIO [72.45,46], conversationally en¬ 
gage with WE-II [72.47], or bond with people [72.48, 
49]. Some have investigated the social-communicative 
aspects of emotions in coordinating behavior and influ¬ 
encing others, e.g., FEELIX [72.50], Kismet [72.3, 51]. 
Others have explored the functional role of emotion- 
inspired processing in order to make robots more in¬ 
telligent, better able to learn, and better adapted to 
performing tasks in complex environments [72.52]. 
More recently, researchers have investigated the role 
of affect in the context of robots that work with 
people and perform tasks such as search and res¬ 
cue/retrieve [72.16,53,54]. Finally, others model emo¬ 
tions to make robots better able to handle human 
emotional states, and to motivate people toward more 
effective interactions in a range of application domains 
such as education [72.55], coaching [72.56], or thera¬ 
peutic systems [72.31]. 

72.3.1 Theories of Emotion 

The robot’s computational model of emotion deter¬ 
mines the robot’s affective responses. This can depend 
on a myriad of interrelated physical, cognitive, and af¬ 
fective factors that continuously modulate and bias one 
another. These factors arise from the robot’s interac¬ 
tions with the external environment as well as its own 
internal state (i. e., the current emotional state, the cog¬ 
nitive state, goals, motives, physical states, etc.). The 
emotional model defines the relationship between these 
factors and mechanisms that result in the observable 
behavior of the robot. Many of these computational 
models of emotion are inspired by theories of hu¬ 
man emotions. These theoretical models offer insightful 
constraints that help researchers to derive coherent 
computational models. 


A number of theoretical perspectives have been 
particular influential in the development of computa¬ 
tional models of emotion. Appraisal theory empha¬ 
sizes a causal connection between cognition and emo¬ 
tion [72.57]. In appraisal theory, emotion is evoked 
from patterns of judgments (called appraisal variables) 
that characterize the personal significance of events 
(e.g., events and the individual’s beliefs, or desires and 
intentions). An active area of development is in under¬ 
standing the relationship between appraisal variables 
and specific emotion labels, or specific behavioral (i. e., 
facial expressions) and cognitive responses (i. e., cop¬ 
ing strategies) [72.58]. This kind of model lends itself to 
more symbolic AI (artificial intelligence) implementa¬ 
tions, with if-then rules [72.59,60]. In contract, dimen¬ 
sional theories posit that emotion and other affective 
phenomena are not discrete entities, but rather exist on 
a continuum of a continuous dimensional space [72.61]. 
Smith and Scott [72.62] proposed a three-dimensional 
(3-D) PAD model where P corresponds to pleasure 
(valence), A corresponds to arousal (intensity), and D 
corresponds to dominance (coping potential) [72.62]. 
Reference [72.63] mapped these appraisal dimensions 
to intensity varying expressions that can be computed 
as a weighted blend of basis postures corresponding to 
the main axes [72.63]. The core affect (the emotional 
state of the individual at any given time) is represented 
as a point in the 3-D space that is pushed around by 
eliciting events. Dimensional models are often used 
for generating the behavior of animated characters, as 
the dimensional space lends itself nicely to animation 
blending. 

72.3.2 Example: A Synthesis 

of Emotion Theories in Action 

Kismet is the first autonomous robot explicitly de¬ 
signed to explore socio-emotive face-to-face interac¬ 
tions with people [72.3,51]. Research with Kismet 
focused on exploring the origins of social interaction 
and communication in people, namely that which oc¬ 
curs between caregiver and infant, though extensive 
computational modeling guided by insights from mod¬ 
els of emotion [72.64]. Kismet was designed to be 
dependent on people to help it satisfy its goals and 
motivations. 

Internally, Kismet’s models of emotion interacted 
intimately with its cognitive systems to make affective 
appraisals about its interactions with the surrounding 
environment and people. These appraisals character¬ 
ized the robot’s interaction with its environment (e.g.. 
Is an object too close to the robot so that it might do 
damage?. Is that person speaking to me in a prais¬ 
ing tone of voice? etc.). These appraisals were tagged 
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with somatic markers [72.65] that characterized how 
these appraisals mapped to the robot’s internal mea¬ 
sures of arousal (A), valence (V), and stance ( S ), an 
action tendency to approach or avoid). The affective 
appraisals directly influenced the robot’s behavior se¬ 
lection and goal arbitration processes. The somatic 
markers influenced the robot’s core affective state - 
a continuously adjusted point within a three dimen¬ 
sional [A,V,S] space. The core affect mapped to the 
robot’s emotive expressions conveyed through vocal 
quality, facial expressions, and body posture [72.66]. 
Rather than simply triggering a set of discrete emotive 
expressions, Kismet’s facial and postural expressions 
were continuously computed following a componen- 
tial approach, as a weighted blend of basis postures of 
the face and body along the [A,V,S] axes. This sub¬ 
tle variability in expressive behavior was important for 
enabling the robot to mutually regulate affective states 
with a person [72.67,68]. Finally, the core affect and 
affective appraisals contributed to elicit adaptive behav¬ 
ioral responses (e.g., orient, search, avoid, etc.) as part 
of the robot’s emotive responses (loosely inspired by 
the idea of emotion circuits [72.69]). Through a process 
of behavioral homeostasis [72.70], these emotive re¬ 
sponses served to influence how people interacted with 
the robot in order to restore the robot’s internal drives, 
goals, and affective states [72.67,68]. I4E63KZH5EM 
illustrates Kismet’s ability to recognize, express and in¬ 
teract with people using emotive cues. Ultimately, the 
purpose of this dance was to keep the robot in a zone 
of proximal development conceptualized by Vygotzky to 
be optimal for learning to propel the robot down a de¬ 
velopmental path [72.71]. 

72.3.3 Emotional Empathy 

For humans, the dynamic coupling of like minds 
through the actions of similar bodies is critical for ac¬ 
quiring human-like intuitions about the internal states 
of others. Dautenhahn [72.2] is one of the earliest 
works to explore empathic mechanisms of understand¬ 
ing others in social robot-robot interaction. 

It is likely that emotional empathy in humans is 
learned, beginning in infancy. Various experiments with 
human adults have shown a dual affect-body connec¬ 
tion whereby posing one’s face into a specific emotive 
facial expression actually elicits the feeling associated 
with that emotion [72.72,73], Hence, imitating the fa- 



Fig.72.5a,b Kismet and a young woman mirroring af¬ 
fect. Facial expression and affective tone of voice are 
tightly correlated: (a) mirroring interest/arousal; (b) mir¬ 
roring negative affect 

cial expressions of others could cause an infant to feel 
what the other is feeling, thereby allowing the infant 
to learn the association of observed emotive expres¬ 
sions of others with the infant’s own internal affective 
states. Other time-locked multimodal cues may facili¬ 
tate learning this mapping, such as affective speech that 
accompanies emotive facial expressions during social 
encounters between caregivers and infants. Using a sim¬ 
ilar approach, Breazeal et al. [72.74] posit that a robot 
could leam the affective meaning of emotive expres¬ 
sions signaled through another person’s facial expres¬ 
sions, body language, and synchronized multimodal 
cues such as vocal prosody [72.68,75] (Fig. 72.5). 
These time-locked multimodal states occur because of 
the similarity in bodies and body-affect mappings, and 
they enable the robot to leam to associate its internal 
affective state with the corresponding observed expres¬ 
sion. In later work, they implemented a model of social 
referencing by which the robot, Leonardo, that com¬ 
bines models of empathic association with models of 
shared attention (Sect. 72.4.1). 
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72.4 Socio-Cognitive Skills 

Socially intelligent robots, however, must understand 
and interact with animate entities (i. e., people, animals, 
and other social robots) whose behavior is governed by 
having a mind and body. In other words, social robots 
need the ability to recognize, understand, and predict 
human behavior in terms of the underlying mental 
states such as beliefs, intents, desires, feelings, etc. Psy¬ 
chology calls this ability Theory of Mind (also known 
as mindreading, mental perception, social common- 
sense, folk psychology, social understanding, among 
others). 

This section reviews research in implementing mod¬ 
els of human socio-cognitive skills and abilities on 
robots. Social robots shall need a diverse repertoire of 
such skills to realize their full potential in daily human 
life - to communicate, cooperate, and learn from people 
in a human-centric and human compatible manner. 

For instance, social robots will need to be aware of 
people’s goals and intentions so that they can appro¬ 
priately adjust their behavior to help us as our goals 
and needs change. They will need to be able to flexibly 
draw their attention to what we currently find of interest 
so that their behavior can be coordinated and informa¬ 
tion can be focused about the same thing. They need to 
realize that perceiving a given situation from different 
perspectives impacts what we know and believe to be 
true about it. This will enable them to bring important 
information to our attention that is not easily accessi¬ 
ble to us when we need it. Social robots will need to be 
deeply aware of our emotions, feelings, and attitudes to 
be able to prioritize what is the most important thing to 
do for us according to what pleases us or to what we 
find to be most urgent, relevant, or significant. 

Furthermore, the behavior of social robots will need 
to adhere to people’s expectations. Namely, people will 
apply their theory of mind to understand the robot in 
terms of these mental states as well. 

72.4.1 Shared Attention 

Scassellati [72.76] was one of the earliest works to pose 
the question of how to endow robots with a theory of 
other minds. Inspired by the theoretical viewpoints pro¬ 
posed from the study of autism (believed to be a deficit 
of theory of mind), Scassellati implemented a hybrid 
model of those models proposed by Leslie [72.77] and 
Baron-Cohen [72.78] where shared attention is viewed 
to be a critical (and missing) precursor to the theory 
of mind competence. This hybrid model was imple¬ 
mented on the humanoid robot, Cog. The robot was 
able to exhibit an assortment of social-cognitive skills 
such as joint attention, distinguishing an entity in the 


environment as either being animate or inanimate, and 
imitating only entities deemed to be animate [72.79]. 

Several researchers have explored models of joint 
reference, guided by insights provided by develop¬ 
mental psychology and autism research [72.76, 79-81]. 
Normal human infants first demonstrate the ability to 
share attention with others at 9 to 12 months of age, 
such as following the adult’s gaze or pointing gestures 
to the object being referred [72.78, 82], In these works, 
joint attention is a learned process. For instance, the 
robot learns the visual motor mapping from the human’s 
attentional cue (often using head pose as a popular in¬ 
dicator of what the human is currently looking at) to 
the motor commands necessary to have the robot look 
at the same thing. This process is often bootstrapped by 
having the human look to where the robot initiates its 
gaze. In Fasel et al. [72.80], the robot learns a model 
of joint attention because it discovers that the human’s 
gaze is a reliable indicator of where there is something 
interesting to look. 

Thomaz et al. [72.83] explore attention-monitoring 
behavior of a robot in a social referencing interaction. 
In the developmental psychology literature, the ability 
for babies to actively monitor that others are looking 
at the same thing is a strong indicator of shared at¬ 
tention [72.84]. Social referencing is considered to be 
an early demonstration of shared attention because the 
baby looks back and forth between the novel object 
and the adult’s emotive reaction toward that object to 
learn the association between the two. To implement 
shared attention, the robot’s attentional state is modeled 
with two related but distinct foci: the current atten¬ 
tional focus (what is being looked at right now) and 
the referential focus (the current topic of shared focus, 
i. e., what communication, activities, etc. are about). 
Furthermore, the robot maintains a model for its own 
attentional state and a model for the attentional state 
of the human. The robot uses the heuristic of look¬ 
ing time upon a shared object to infer the referent of 
the interaction. Once the referent has been identified, 
the robot monitors the attention of the human in order 
to associate their emotional reaction about that object 
to the intended target. Video l-s&lt'lMf'ktUl illustrates 
Leonardo’s ability to interact and learn from a person 
via social referencing. 

72.4.2 Mental Perspective Taking 

This section explores this empathetic, self-as-simulator 
approach further to address more general challenges in 
endowing robots with mental perspective-taking abil¬ 
ities. These approaches are inspired and informed by 
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theories championed by neuroscience and embodied 
cognition called Simulation Theory. 

Simulation Theory 

Simulation theory holds that certain parts of the brain 
have dual use; they are used to not only generate our 
own behavior and mental states, but also simulate the 
introceptive states of the other person [72.85]. In other 
words, we engage in a process of perspective taking and 
mental simulation. 

For instance, Gallese and Goldman [72.86] pro¬ 
posed that a class of neurons discovered in monkeys 
(called mirror neurons ) is a possible neurological 
mechanism underlying both imitative abilities and 
Simulation Theory-type prediction of the behavior of 
others and their mental states. Further, Meltzoff and 
Decety [72.87] posit that imitation is the critical link in 
the story that connects the function of mirror neurons 
to the development of adult mind-reading skills. From 
the field of embodied cognition, Barsalou et al. [72.88] 
presents additional evidence from various social em¬ 
bodiment phenomena that when observing an action, 
people activate some part of their own representation of 
that action as well as other cognitive states that relate 
to that action. 

Mirror Systems for Recognizing Actions 
Inspired by these theories and findings, Johnson 
and Demiris [72.89] employ a simulation of visual 
perception to recreate the visual egocentric sensory 
space and corresponding egocentric behavioral space 
of the observed agent to increase the accuracy of action 
recognition. This approach is based on their HAMMER 
architecture (hierarchical attentive multiple models for 
execution and recognition) that takes a mirror neuron 
inspired approach to action recognition and imitation 
by directly involving the observer’s motor system in 
the action recognition process. Specifically, during 
observation of another’s actions, all of the observer’s 
inverse models (akin to motor programs) are executed 
in parallel via simulation using forward models, and 
then compared to the observed action. The one that 
matches best is selected as being the recognized action. 
Perceptual perspective taking is needed to provide 
meaningful data for comparison. The simulated actions 
used by the observer during recognition must be 
generated as though from the point of view of the 
other person. They demonstrate this approach in an 
experiment where a robot attributes perceptions and 
recognizes the actions of a second robot [72.89]. 

Mental Perspective-Taking 
for Inferring Beliefs and Goals 
Gray et al. [72.90] have implemented computational 
models of simulation-theoretic mechanisms throughout 


several systems within Leonardo’s cognitive architec¬ 
ture to enable the robot to infer beliefs and goal states 
of a human collaborator. 

The robot reuses its belief-construction systems 
from the visual perspective of the human to predict the 
beliefs the human is likely to hold to be true given what 
he or she can visually observe. This enables the robot to 
recognize and reason about the beliefs held by a person, 
even when they diverge from the robot’s own beliefs of 
the same situation. 

In psychology, the ability to appreciate the diver¬ 
gent beliefs of another is classically demonstrated by 
the famous false belief task. In this task, subjects are 
told a story with pictorial aides that typically proceeds 
as follows: two children, Sally and Anne, are playing 
together in a room. Sally places a toy in one of two 
containers. Sally then leaves the room, and while she 
is gone, tricky Anne moves the toy into the other con¬ 
tainer. Sally returns. At this point the human subject is 
asked Where will Sally look for the toyl 

The robot, Leonardo, has demonstrated its abil¬ 
ity to pass these sorts of false belief tasks where it 
observes two humans playing the roles of Sally and 
Anne [72.91]. Within the robot’s goal-directed behavior 
system (where schemas relate preconditions and ac¬ 
tions with desired outcomes) motor information is used 
along with perceptual and other contextual clues (such 
as hierarchically structured task knowledge) to infer the 
human’s goals and how he or she might be trying to 
achieve them (i. e., plan recognition). 

72.4.3 Perspective Taking 
in Collaboration 
and Teamwork 

By using a simulation-theoretic methodology, mental 
inferences made across different cognitive systems can 
interact in interesting and useful ways to support col¬ 
laborative behavior where a robot offers its human 
teammate appropriate assistance. 

Using Visual Perspective Taking to Resolve 
Ambiguous Referents 

Trafton et al. [72.92] have developed and implemented 
visual and spatial perspective taking abilities based 
on mental simulation to support human-robot inter¬ 
action and collaboration. Their cognitive architecture, 
Polyscheme, is designed to model how humans integrate 
multiple representational methods, reasoning, and plan¬ 
ning methods to keep track of the world, including rich 
facilities for representing counterfactual worlds. It thus 
supports simulations of other people’s visual perspec¬ 
tive to reason about interactions and the world from this 
alternate point of view. 
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They have demonstrated these skills in a number of 
experiments, such as demonstrating the robot’s ability 
to learn how to play hide and seek with a person where 
the robot learns what makes a good hiding place with 
respect to being completely occluded from the human 
seeker’s point of view [72.93]. They have also demon¬ 
strated the usefulness of this system for a robot that 
solves a series of perspective-taking problems using 
the same frames of references and spatial reasoning 
abilities that astronauts do to facilitate collaborative 
problem solving - such as repairing a vehicle with an¬ 
other person that has a different vantage point [72.94]. 
For instance, the robot can handle egocentric requests 
(i. e., hand me the cone to my right), addressee-centric 
requests (i. e., hand me the cone to your right), or 
object-centered requests (i. e., hand me the cone in 
front of the box). 

In Trafton et al. [72.92], a human interacts with the 
robot using a multimodal interface that supports speech 
and gesture. The robot’s perspective taking skills are 
used to resolve ambiguous referents that can arise when 
a person asks a robot perform an action in relation to 
an object (i. e., asking the robot to hand me the wrench 
when there are multiple wrenches to choose from). In 
particular, a visual occlusion in the workspace might 
hide another candidate wrench from the person’s view¬ 
point but not from the robot’s viewpoint (Fig. 72.6). 
The robot can infer which is the intended object by tak¬ 
ing the visual perspective of the human and applying 
principles of joint salience and least effort. If there still 
remains an ambiguity, the robot can act to resolve it by 
asking which one ? 

Providing Informational 
or Instrumental Support 

Gray et al. have demonstrated the ability for the 
Leonardo robot to successfully infer its human partner’s 
beliefs, desires, and intentions from real-time behav¬ 
ior during collaborative tasks. The shared workspace 
can have either visual occlusions [72.90] or can change 
dynamically where not all participants know of these 
changes [72.91]. The robot can integrate these mental 
state inferences to decide how to best help the person 
such as offering instrumental support (acting on the en¬ 
vironment to help the human complete their goal) or 
provide informational support (giving relevant informa¬ 
tion the person needs to successfully achieve his or her 
goal). 

Consider the following scenario: a helpful robot is 
introduced to two people, Sally and Anne. All three 
watch as Anne hides chips in a box to the left of the 
robot and cookies in a box to the right. Sally leaves the 
room, at which point Anne plays a trick on Sally by 
swapping the contents of the boxes and then locks both 


boxes with a combination lock. Anne leaves, and Sally 
soon returns craving the chips she saw placed in one 
of the boxes. Sally remembers seeing the chips placed 
in the left box and attempts to open it by working the 
combination lock. The robot has matching chips and 
cookies that it can give out. What should the robot do 
to assist Sally? 

Mindreading skills play an important role in this 
plan recognition scenario where the robot must ob¬ 
serve Sally in real time to infer Sally’s misconception of 
where the chips are (Anne switched the location when 
Sally was out of the room), to infer what her desire is 
based on her behavior (Sally never explicitly said she 
wants the chips), and to recognize that Sally’s plan for 
how to get the chips is actually invalid (she is trying to 
open the wrong box). The robot has true knowledge of 
the situation, and must then reason about how to best 
help Sally get the object of her desire. 

Gray et al. [72.91] (|43>M2EI5E3) combines these 
three kinds of mental inferences to demonstrate in¬ 
tention recognition with divergent beliefs for collab¬ 
orative robots. Specifically, for the case of informa- 



Fig. 72.6 Robonaut using visual perspective taking to dis¬ 
ambiguate the intended referent when asked to hand me 
the wrench. The human can only see one wrench, but the 
robot can see both. The robot correctly hands the wrench 
that both can see 
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tional support, Leonardo relates its own beliefs about 
the state of the shared workspace to those of the 
human based on the visual perspective of each. If 
a visual occlusion exists or an event occurs that pre¬ 
vents the human from knowing important information 
about the workspace, the robot knows to direct the 
human’s attention to bring that information into com¬ 


mon ground. For instance, Leonardo points to the 
box that actually holds the chips. For the case of 
instrumental support, Leonardo helps the person by 
directly giving the person a matching bag of chips. 
Gray and Breazeal [72.95] explore the robot reason¬ 
ing and taking explicit action to deceive a human 
competitor. 


72.5 Human Social Responses to Social Robots 


72.5.1 Social Judgments 

Anthropomorphic design is important for social robots 
given that the appearance, interface, and function of 
a technology or product impact how people perceive it, 
interact with it, and engage with it over time [72.12], 
Robots and other technologies with humanlike design 
cues elicit social responses from people [72.96-99]. For 
instance, people have been found to respond more pos¬ 
itively to artifacts that exhibit humanlike cues such as 
emotive expressions over those with a purely functional 
design, although user preferences were task and con¬ 
text dependent [72.100, 101]. Adding humanlike cues 
to a technological artifact can foster people’s social 
connection to it, aid people in learning how to use it, 
and enhance liking, engagement, and the desire to col¬ 
laborate [72.102-104], Others have found that people 
tend to hold richer mental models of as anthropomor¬ 
phic robots than mechanistic ones [72.103, 105]. Others 
have explored a number of anthropomorphic design fea¬ 
tures such personality, backstory, use of humor, and 
even the notions of self (e.g., referring to itself as 
/), deception, politeness, and moral regard [72.106- 
108]. However, it is important that the appearance and 
interface of the robot’s design match its capabilities 
and the users’ expectations or negative effects can re¬ 
sult [72.109]. 

72.5.2 Physical Versus Virtual Embodiment 

When considering the role and advantages of social 
embodiment, one might ask whether there is a differ¬ 
ence between physical and virtual counterparts. Indeed, 
physical social embodiment offers a number of ad¬ 
vantages over purely graphical representations. First, 
while many social interactions involve exchanging only 
visual and auditory cues, robots support communica¬ 
tion and collaboration through physical contact as well. 
Robots support the joint manipulation of artifacts and 
the sharing of physical space with people. Both are im¬ 
portant for all sorts of collaborative activities such as 
assembly and manufacturing [72.110, 111] search and 


rescue [72.16,53], domestic assistance [72.103], and 
more. Further touch is not only an interesting and im¬ 
portant communication modality with noted heath and 
therapeutic benefits, but it can also influence the social 
judgments people make of robots such as how caring 
or persuasive it is perceived to be [72.103, 104]. The 
modality of touch will be discussed further in section 
social touch [72.31,39]. 

In addition, a growing number of studies that di¬ 
rectly compare virtual to physical agents report that 
people show more trust, compliance, and enjoyment 
with physical robots [72.112-115]. Beyond user pref¬ 
erence of physical over virtual agents, a number of 
studies have also shown improved human performance 
and outcomes on a wide variety of tasks ranging from 
games [72.112] to educational contexts [72.115], as¬ 
sistive tasks [72.114], health-related activities [72.116], 
and Wizard-of-Oz user studies [72.1 13]. 

Finally, because virtual agents are a representational 
form, interpretation and mapping may prove too chal¬ 
lenging for individuals with cognitive or social deficits. 
Hence, robots may prove advantageous as therapeutic 
interventions for children with autism who can show 
little or no interest in forms on video monitors or tele¬ 
visions [72.117]. Finally, social robots tend to support 
face-to-face group dynamics [72.118] - while screens 
tend to capture eyeballs at the risk of diminishing 
face-to-face interaction [72.119]. This has important 
considerations for the design of learning technologies 
for young children, in particular, where supporting the 
participation of the parent in a social way is of great 
benefit to how children learn. 

72.5.3 Social Stimulus 

to Learn About People 

The impact of physically embodied social robots on 
human social responses has opened new applications 
for robots as an interesting tool to help scientists learn 
about human social behavior and judgments. People 
make a variety of social judgments through the dy¬ 
namic exchange of nonverbal cues such as postural 
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shifts, subtle head gestures, arm gestures, facial cues, 
tone of voice, etc. Indeed, ample evidence indicates that 
humans regularly use specific cues, often without con¬ 
scious awareness, to infer the motivations of others with 
some level of accuracy [72.120,121]. If people employ 
such cues without conscious awareness, then it becomes 
difficult to use people as experimental confederates 
where such cues must be carefully controlled. Thus, in 
one paradigm, social robots can be used as a highly 
controllable social stimulus in the place of a confed¬ 
erate in human participant studies. This is particularly 
useful when trying to understand how human nonverbal 
cues influence people’s social judgments, such as how 
trustworthy another is perceived to be from a brief en¬ 
counter [72.122]. 

72.5.4 Social Rapport 

One important skill for social robots is the ability to 
build and maintain social rapport with its user. Social 
rapport exists in the interaction between individuals. 
It creates a powerful interpersonal influence and re¬ 
sponsiveness based on mutual attentiveness, positivity, 
and mutual responsiveness [72.123-126]. In joint ac¬ 
tivities, the ability to establish a good rapport often 
results in improved outcomes. For instance, students 
learn better when they have a good rapport with their 
teachers [72.127], patients are more adherent and have 
better health outcomes when they have a good rapport 
with their doctors [72.124], teams make better deci¬ 
sions and work more effectively when they share a good 
rapport with each other [72.128]. The quality of rap¬ 
port between people is influenced by the exchange of 
nonverbal behaviors between individuals [72.129, 130]. 
Although the specifics of how people build a strong 
rapport with one another are still a topic of scientific 
inquiry, the more in-sync and open the participants’ 
nonverbal cues are in relation to one another, the more 
positive rapport results. For instance, appropriate mir¬ 
roring or synchrony of body posture, head movements, 
facial expression, and vocal prosody can all contribute 
to positive rapport. Open cues signal a receptiveness to 
interact, e.g., making appropriate eye contact, leaning 
toward another, and arm gestures that tend to not oc¬ 
clude the body or face. 

Hence, when considering how social robots can ef¬ 
fectively collaborate with people, their ability to build 
and maintain good social rapport with people (at least 
perceived rapport) is important. Thus, it is not just 
a robot’s ability to perform such nonverbal cues that ul¬ 
timately matters, but the robot’s ability to coordinate 
them with those of people in real-time. For instance, re¬ 
searchers have explored the coordination of an agent’s 


nonverbal cues with people to improve rapport [72.56], 
engagement [72.32], trustworthiness [72.122], and de¬ 
ictic cues [72.131], 

72.5.5 Social Support 

Improved rapport also facilitates the ability of social 
robots to provide people with effective forms of social 
support. Social support is recognized to play an effec¬ 
tive role in helping people to attain personal goals and 
improved outcomes in broad domains such as educa¬ 
tion, mental health, physical health, aging, coping, and 
more. It is conceptualized as the perception and ac¬ 
tuality that one is cared for, has assistance available 
from others, and that one is part of a supportive so¬ 
cial network [72.123]. These supportive sources can be 
emotional, instrumental, informational, or companion¬ 
ship. Emotional support, for instance, conveys to an 
individual know that they he or she is valued; it includes 
offering empathy, concern, nurturance, encouragement, 
and acceptance to name a few [72. 124, 125]. Instrumen¬ 
tal support concerns the concrete, utilitarian ways to 
provide assistance such as financial assistance, mate¬ 
rial goods, or services. Informational support includes 
offering useful advice, guidance, or information to help 
others problem-solve. Finally, companionship support 
gives someone a sense of social belonging and having 
another to participate in shared social activities. These 
forms of social support can come from people, profes¬ 
sionals, pets, and even social robots. 

The ability to provide a user with social support is 
one of the effective ways that social robots can help 
people through social means. Robots can provide this 
assistance through direct interaction with its user, or 
by helping to mediate the provision of social support 
from people (e.g., connecting people). A wide variety 
of social robots today are being developed to inter¬ 
act with people as tutors [72.132], learning compan¬ 
ions [72.118, 133], coaches [72.116], domestic helpers 
for the elderly [72.134], therapeutic aids [72.21,31], 
and more. Through dialog, nonverbal cues, expressive 
displays, and physical actions, these robots assist peo¬ 
ple by providing information, monitoring performance, 
offering feedback, incentivizing and sustaining moti¬ 
vation, giving encouragement, offering companionship, 
performing physical tasks, etc. 

As such, social robots have broad applicability in 
many domains where it is a technology that can extend 
and augment the social support provided by people. 
This is particularly relevant for societal challenges, e.g., 
eldercare, health and chronic disease management, and 
education, where social support is recognized as being 
critical for positive outcomes, but where there is a rec- 
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ognized shortage of trained professionals to meet the 
demand. Further, whereas frequent meetings with hu¬ 
man professionals is cost prohibitive, social robots have 
the potential to be used to fill the gaps in a cost-effective 


way. Importantly, social robots are not being designed 
to replace or obviate human professionals, but rather to 
serve an effective tool that supports human networks in 
a scalable and cost-effective way. 


72.6 Social Robots and Communication Skills 


Communication implies an exchange of information 
through natural language. Thus, one might consider that 
very good ASR (automatic spoken-language recogni¬ 
tion) is the primary function required to fulfill the com¬ 
munication skills for robots. However, social robots 
are expected to engage in casual communication with 
people in as natural a way as people communicate 
themselves. In such casual communication, informa¬ 
tion is often exchanged nonverbally as well as verbally. 
Thus, robots need to be well equipped to recognize 
people’s nonverbal cues and to express nonverbal cues 
via their nonverbal behavior. The required computation 
even takes into account good perceptual and cognitive 
capability of surrounding environments in addition to 
the targeted person. This section introduces the history 
of research in social robots and communication skills, 
and provides insights for future challenges. 

72.6.1 Verbal/Nonverbal Communication 

Historically, even first-generation humanoid robots de¬ 
veloped in the 1970s had primitive capabilities for 
communicating using natural language. For instance, 
WABOT and WABOT-2 had a conversation capabil¬ 
ity in natural language, which is based on the model 
as simple combinations of speech input/output map¬ 
pings [72.135,136]. 

Furthermore, early pioneers had noticed the impor¬ 
tance of nonverbal information in human conversation. 
Nonverbal behavior was classified into three roles: 

1. Regulators: expressions such as gaze, poses, and 
vocalizations that are used to regulate/control con¬ 
versational turn-taking. 

2. State displays: indication of internal state including 
affect, cognitive, or conversational states that im¬ 
prove interface transparency. 

3. Illustrators: gestures that supplement information 
for the utterance. These include deictic gestures 
(pointing) and iconic gestures. 

In this scope, nonverbal information is considered 
as supplemental. It is natural language that communi¬ 
cates the primary information in turn-based exchanges. 
We provide examples below. 


Regulatory Cues 

Even some of the earliest social robots displayed non¬ 
verbal information to regulate interactions with people. 
Hadaly 2 was the first robot to use mutual gaze as a non¬ 
verbal cue to regulate conversation [72.137, 138]. The 
mutual gaze is approximated using face recognition to 
determine when the human’s face was facing the robot; 
when a mutual gaze occurred, Hadaly 2 expressed 
readiness to commence conversation by blinking its 
eyes. People’s gaze toward a robot is also considered 
cues to inform whether he/she is engaging in the con¬ 
versation with the robot [72.139]. 

Other examples are Kismet [72.3, 140, 141] and 
Leonardo [72.33, 142], which had the capability for 
nonverbal cues called envelope displays to regulate 
the exchange of speaking turns. Backchannelling cues 
were found to reduce stress and cognitive load during 
complex human-robot teaming task (Fig. 72.7). See 
l<E&M’iiinn-w for how Kismet uses envelope displays 
to regulate speaking turns with people. The regulatory 
role for a speaking turn is demonstrated in multi-party 
interaction too. People tend to make eye contact and 
raise their eyebrows when they are ready to relinquish 
their speaking turn and tend to break their gaze and 
blink when starting their speaking turn. Recognition of 
these cues was implemented for smoothing and syn- 



Fig. 72.7 Backchannelling cues were found to reduce 
stress and cognitive load during complex human-robot 
teaming task. Teams where the robots engaged in 
backchannel cues to human requests also tended to find 
more items in a search and retrieve task (after [72.16]), 

I^MMSEa 
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chronizing the exchange of speaking turns. Gaze is also 
known to convey the speaker’s intention who may take 
the next turn. Mutlu et al. have successfully replicated 
it in human-robot interaction [72.143,144]. Kirchner 
and Alempijevic revealed that gaze is also effective in 
communicating who should receive an item provided by 
a robot [72.145]. In opposite way, gaze was also used 
as a cue to interpret whether one would wish to take 
turn [72.146]. 

Paralinguistic information is also processed. People 
frequently provide short acknowledgment utterances 
(e.g., uh-huh, um-hmm, huh, etc.) as the robot explains 
something. These responses are either acknowledg¬ 
ments, or ask-backs. It is very difficult to distinguish 
these two kinds of utterances from only the linguistic 
information as represented by the transcription of the 
utterance. Fujie et al. demonstrated a method to distin¬ 
guish the utterance as either an acknowledgment or an 
ask-back from the prosody of utterance [72.47]. 

State-Display Cues 

The facial expression and gaze were used to indi¬ 
cate the conversational or cognitive state of a robot. 
Such state-display cues make the robot’s internal state 
more transparent to a user and thus enables him/her to 
better understand the robot’s state. For instance, RO- 
BITA used the tightness of its facial expression to 
indicate readiness to engage in conversation; a tight 
face was used to express conversational readiness, 
while a loose face communicated a lack of readiness 
to engage [72.147]. Emotional state and attention tar¬ 
get can also be displayed nonverbally (Sects. 72.3 
and 72.4). 

State of listening and level of understanding are 
also displayed nonverbally. Human listeners use back- 
channel responses, such as head nods to convey the 
fact that he/she is successfully following the con¬ 
versation. Imitating human behavior, robots indicate 
their state of listening using head nods [72.148], fa¬ 
cial expressions [72.5], and bodily motion ([72.149] 
and l<s>®ii!l£liilil). Such state-display cues are used in 
a human-like telepresence robots to indicate operators’ 
state of being engaging to the conversation [72.150, 
151], 

Another back-channel signal is an expression of 
confusion by the listener (verbal or nonverbal). This 
flags the speaker to stop and try to repair the broken 
communication. Robots such as Leonardo and ROBITA 
use facial displays of confusion when speech recog¬ 
nition fails in order to intuitively communicate to the 
human that he or she should repeat their last utterance. 

There were robots that process humans’ back chan¬ 
nel feedback [72.32] A sophisticated head nod recog¬ 
nition system was developed whereby the robot, Mel, 


could successfully distinguish small feedback nods 
from other kinds of head nods such as those that 
communicate agreement. Mel used this information to 
determine its own nodding behavior in order to be an 
appropriate response for the human. In a series of hu¬ 
man subject studies, Sidner et al. found these nonverbal 
cues to enhance the social engagement of the robot to 
people [72.32]. 

Illustrator Cues 

Deictic gestures have often been implemented in robots 
for pointing to an object, such as using index-finger 
pointing [72.152,153], gaze [72.143], and the com¬ 
bination of the two [72.74, 154-157], Other types of 
gestures, such as iconic gestures [72.156] and region 
pointing ([72.158] and were also suc¬ 

cessfully used in robots. The effect of gestures has 
been successfully demonstrated. For instance, in a di¬ 
rection giving scenario, even though turn-by-turn di¬ 
rection is verbally given thus could be comprehend 
without gesture, supplemental pointing gesture im¬ 
proved listeners’ comprehension about the given direc¬ 
tion [72.159]. 

A number of robots are able to recognize deictic 
gestures of a person conveyed either through pointing 
gestures or head poses. For example, Leonardo is able 
to infer the object referent in an interaction by consider¬ 
ing a number of factors including pointing gesture, head 
pose, and speech. Sugiyama et al. associated verbal spa¬ 
tial deixis and pointing gestures to better recognize 
the pointed target ([72.160] and las&MihlilHJI). Brooks 
and Breazeal [72.153] developed a deictic recognition 
system that enabled a robot to infer the correct ob¬ 
ject referent from correlated speech and deictic gesture. 
Interestingly, it was found that the accuracy of the hu¬ 
man’s pointing gesture is surprisingly poor. As a result, 
the deictic recognition system relies on coordinated 
speech and gesture information, with spatial knowledge 
provided by a three-dimensional (3-D) spatial database 
constructed by the robot using real-time vision, and 
a deictic spatial reasoning system. This system was 
successfully demonstrated on the dexterous humanoid 
Robonaut developed at National Aeronautics and Space 
Administration (NASA) Johnson Space Center (JSC) 
(Lig. 72.8b) where the human points to and labels a set 
of four bolts on a wheel to be fastened in order by the 
robot. 

72.6.2 Mechanisms for Human-Robot 
Communication 

Mechanisms of turn-based communication are well 
studied. In linguistics, it is considered that conversa¬ 
tion is formed as the repetition of turn-taking [72.161]. 
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Fig.72.8a-c Examples of conversational robots: (a) RO- 
BITA performing group conversation; (b) Robonaut in¬ 
terpreting the pointing gestures of a human to determine 
which nut to fasten on the wheel; (c) Leonardo uses gaze 
and joint attention to ground the human’s pointing gesture 
for the desired referent 


There is one speaker who takes floor and speaks 
while listeners listen to him/her [72.162]. When the 
speaker finishes speaking, then the floor is taken by 
one of the listeners. There could be bystanders who 
also listen to, but do not intend to take turn. This 
turn-based model is the typical model used in dia¬ 
log modeling [72.163]. That is, a dialog management 
system identifies who owns floor, recognizes words 
in utterances the speaker spoke, and generate utter¬ 
ance when the system takes floor, often implemented 
with series of rules. It is successfully extended for 
human-robot communication. For instance, Nakano 
et al. developed rule-based architecture in which the 
planner deals with robot’s task-based actions as well 
as dialog management [72.164]. Scheutz et al. devel¬ 
oped software architecture, named DIARC. It uses 
a rule-based planner that receives inputs from all per¬ 
ception modules, and addresses effect and goal-directed 
actions in addition to natural language dialog manage¬ 
ment [72.165]. In such an approach, the robot commu¬ 
nicates through utterances accompanied with nonverbal 
cues [72.166]. Some systems deal with multi-party di¬ 
alog, in which robot’s gaze-cue (Fig. 72.8a) is used 
to regulate who is the addressee, the active listener 


who is expected to take the next turn [72.143,167- 
171], 

Researchers have also been well aware of the im¬ 
portance of the time-sensitive nature of communication. 
For instance, interruption in the middle of a speaker’s 
turn has also recently been taken into account in di¬ 
alog management systems [72.164,172]. Chao and 
Thomaz, proposed more elaborated model, in which 
time-synchrony in verbal and nonverbal cues, e.g., gaze 
and gesture, are addressed using a Petri-net-based rep¬ 
resentation [72.173]. Empirical studies have revealed 
what is good synchrony and timing. For instance, Ya¬ 
mamoto and Watanabe , have studied synchrony within 
a robot’s utterance and motion, and revealed that peo¬ 
ple prefer the robot whose utterance is slightly delayed 
from the start of its motion [72. 174]. This would mirror 
what humans do everyday, as it is reported that hu¬ 
mans’ gestures are performed slightly ahead than their 
utterances [72.175]. Shiwa et al. revealed that people 
prefer small delay in the robot’s response, and con¬ 
versational filler such as etto would be useful to buy 
time when the robot’s response is delayed ([72.176] and 

However, while above studies are under the assump¬ 
tion that information is communicated through turn- 
based dialog, recent studies have revealed more dy¬ 
namic cases of human-robot communication, in which 
the way robots communicate information is some¬ 
times out from the turn-based dialog paradigm. For 
instance, during the moment a robot and a person are 
going to initiate interaction both of them communi¬ 
cate their intention that they would like to meet and 
talk. A couple of studies investigated a way for a robot 
to express its intention to welcome the initiation of 
interaction [72.177]. When the target user is seated, 
Dautenhahn et al. revealed that they prefer the robot 
approach from side ([72.178] and l<s»n«JLiJ£M). Fig¬ 
ure 72.9 shows a scene in which the robot approached 
to pedestrians. Satake et al. revealed that such inter¬ 
action failed if the robot failed to communicate its 
intention to talk. As the robot was operated in noisy 
shopping mall, when it used only verbal utterance, 
the robot was simply ignored. Instead, they found that 
the robot needs to communicate its intention nonver¬ 
bally. It needs to approach from a frontal direction and 
needs to be responsive in adjusting its body orientation 
toward the targeted person; such nonverbal behavior 
made the robot more success in initiating conversation 
with pedestrians [72.179]. 

Pedestrians also communicate their intention non¬ 
verbally; for instance, when he/she does not wish to 
talk to the robot, they avoid approaching to the robot. 
Thus, a robot could use the proximity information 
to estimate people’s willingness in initiating interac- 
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Fig. 72.9 A robot that approaches 
pedestrians 


tion. Michalowski et al. classified spatial zones around 
a robot, and let the robot talk to the person if he/she 
approached to the robot and entered to social dis¬ 
tance [72.177]. There are some empirical studies about 
proxemics that revealed that people often prefer to stay 
at social distance when they talk to a robot [72. ISO- 
182]. 

Joint attention (as introduced in earlier Sect. 72.4) 
can be silent. Figure 72.10 and show 

a scene where a person share his attention target non¬ 
verbally with the robot. The person is curious about the 
computer and stands in front of it; then, the robot moves 
to the location convenient to explain the object. When 
the person moves to the other exhibit, the robot follows 
him and explains the one in front of him. Here, it is 
their standing position that communicates the target of 
attention [72.183, 184]. 

These examples show the cases where communi¬ 
cation is not necessarily turn-based. This is because 
information can be exchanged nonverbally in such ca¬ 
sual communication. Unlike the speech channel that 
needs to be typically occupied by only one speaker 
(speaking person), nonverbal signals can be mutu¬ 
ally exchanged at the same time. For instance, when 
a pedestrian and a robot meet, their positions contin¬ 
uously change while they walk which communicates 
whether they would like to initiate conversation. As so- 



Fig. 72.10 A person and robot implicitly share the target of 
attention via spatial formation 


cial robots aim to operate in people’s daily environment, 
such a casual and nonverbal exchange of information 
is not a trivial part of human-robot communication. 
The research for fully unveiling required mechanism 
for such continuous exchange of social signals is still 
premature. 

On the other hand, some studies started to highlight 
the needs of a mechanism that connects communica¬ 
tion and background knowledge. It is revealed that the 
model of common ground makes daily communication 
effective [72.185]. For example, a direction-giving sce¬ 
nario would be a case where environmental knowledge 
is useful. For instance, when a robot provides directions 
(Fig. 72.11 and l<st«IW Itt l), it could be more com¬ 
prehensive if the robot is aware of a visible landmark so 
that it could say please turn at the book store instead of 
saying please turn at the third corner [72.186]. Thus, 
a good model of environment, e.g., [72.187], would 
greatly improve a robot’s capability in communicating 
about a route. Moreover, if a robot has a model of users’ 
memory of location, it could provide destination-based 
direction, such as a cafe is nearby the book store you 
just visited [72.188], which is much easier to compre¬ 
hend than complex turn-by-turn destination like to go to 
a cafe, please turn right at first corner, turn left at third 
corner, and... 



Fig. 72.11 A robot that provides directions (after [72.189]) 
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72.6.3 Challenges 

To summarize, turn-based conversation has been stud¬ 
ied to a certain degree. However, social robots often 
engage in casual communication in which robots would 
need to deal with social signals and background knowl¬ 
edge. Here, we denote a couple of key challenges. 

Revealing Repertory of Communication Skills 
Previous studies started to reveal different forms of 
communication where various perceptual and cogni¬ 
tive capabilities are required. Early studies have re¬ 
vealed the importance of social signal processing, like 
recognizing and expressing gaze and facial expres¬ 
sions. Studies about initiation of conversation have 
revealed the mutual real-time exchange of informa¬ 
tion via their positions. Studies about giving direction 


demonstrated further needs of associating language 
and environmental knowledge, e.g., a cognitive map, 
how people perceive environments, and remembering 
landmarks. 

To what extent do we need to cover repertories? We 
believe that there could be many elements we will find. 
Many of them might be interrelated with perception and 
cognition. Such a case would improve our understand¬ 
ing about what communication skills truly are. Thus, 
one of the important challenges is here. 

Architecture for Communication Skills 
Along with finding out the repertory of communica¬ 
tion skills, the other important challenge is to deal with 
the integration, i. e., the architecture of communication 
skill. Moving from a turn-based structure to a dynamic 
structure is the real challenge. 


72.7 Long-Term Interaction with Robot Companions 


A number of social robots have been introduced as com¬ 
panion robots. Companion robots may have the sole 
purpose of providing companionship, e.g., in studies 
using toy robots such as the Pleo [72.190], but often 
they combine two aspects: being useful, i. e., being able 
to carry out certain tasks for the user, and carrying 
those tasks out in a manner that is socially accept¬ 
able [72.191]. The latter notion has been used, e.g., 
in several European projects [72.192-195]. The notion 
of a companion often entails repeated and long-term 
interactions. This poses particular challenges not only 
for the design of the robot, the interaction design, the 
choices of tasks/settings/scenarios, but also on the how 
to ensure satisfactory and successful interaction with 
the robot. 

72.7.1 Robot Companions 

Recently more and more researchers are moving toward 
studying application areas that develop such companion 
robots, e.g., the use of robots in assistive and rehabili¬ 
tation robotics. The use of robots to assist elderly users 
in their homes, with a view to extend the period they 
can stay and live independently in their own homes, 
is currently being studied extensively by different re¬ 
search groups worldwide. Several companies are also 
marketing their robots as such assistive companions, 
e.g., the Wakamaru robot (Mitsubishi Heavy Industries 
Ltd) that was introduced in 2005, or the Human Support 
Robot (Toyota) revealed in 2012. In addition, numerous 
research prototypes exist, e.g., Cody (Georgia Tech), 
Herb (CMU), Care-O-bot 3 (Fraunhofer), Hector (Com- 
panionAble project). 


A number of tele-presence robots have been de¬ 
veloped, but this chapter will focus on autonomous or 
partially tele-operated companion robots. Note, Cody 
targets the domain of patient hygiene care that is related 
to issues discussed in the section on tactile human- 
robot interaction. Regardless of whether human-robot 
interaction with companions include tactile interaction, 
or is a hands-off approach (Chapt. 73), developing such 
system involves a careful study of the roles and func¬ 
tions of such robots in the application domain, and of 
users’ perception of and attitudes toward such a system. 

Figure 72.12 and I^MMSEEl show the Care- 
o-bot robot (Fraunhofer) used in the ACCOMPANY 
project on home assistance for elderly people. The robot 
is shown in the University of Hertfordshire’s Robot 
House, a domestic setting for experiments into robot 
home assistance where participants regularly visit for 
human-robot interaction sessions. Such living lab set¬ 
tings are increasingly being used to design, experiment 
and evaluate innovative systems (e.g., the European 
Network of Living Labs, [72.196]). Providing such en¬ 
vironments facilitates progress toward complex robotic 
systems that can be deployed in real-life environments. 

72.7.2 Engagement and Long-Term 
Relationships 

The concept of a robot companion entails repeated, 
long-term interactions, which may also afford the de¬ 
velopment of relationships between robots and people. 
Relationships with robots can take many shapes and 
forms, and the formation of relationships will be in¬ 
fluenced by many factors. For example, the specific 
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role of the robot (Butler? Friend? Tutor? Assistant? 
Tool? etc., [72.197]) will matter, as well as the robots’ 
specific embodiment and behavioral and expressive 
repertoire that is often used to present them as re¬ 
lational artifacts [72.198,199] that are designed to 
build and maintain social-emotional relationships with 
users. 

Many applications with social robots do only in¬ 
volve short-term interactions, e.g., robots meant to 
function as a museum guide [72.200] and l<s>*li!liI05Iil, 
a receptionist, etc. Here, the novelty effect can of¬ 
ten be exploited, i. e., the general interest that many 
people have in new robots. However, after repeated 
interactions this effect can wear off and people can 
lose interest in a robot. Thus, a main research chal¬ 
lenge is how to keep people engaged in the interaction 
and keep them motivated to interact with the robot. 
Establishing a useful and enjoyable relationship with 
a robot is not an easy task. And what exactly is long¬ 
term ? Tanaka et al. [72.22] mentions at 10-hour barrier 
(total interaction time), Sung et al. [72.201] pose the 
goal of long-term interaction over more than 3 months 
as a main challenge. Hiittenrauch and Severinson- 
Eklundh [72.202] performed a user study with the 
service robot CERO meant to assist one user over three 
months in an office environment. Kanda et al. [72.189] 
and |43>JEEE!ElEI investigated a partially tele-operated 
robot over a 25-day period. The nature of these interac¬ 
tions depends on whether studies take place in a school, 
a home, a public place, etc. It will also depend on 
the frequency and duration of the interaction, e.g., how 
many interaction episodes per time interval does an in¬ 
dividual have with the robot? 

There is also the issue of quantity and quality of 
interaction that may be realized very differently in 
different application areas of social robots. More and 
more studies are bringing robots into the wild [72.203]. 
l-gBia'il'Jt'm shows how preschool age children in¬ 
teracting with a storytelling learning companion robot 
during repeated encounters over a 2 month period. Per¬ 
sonalization of the robot stories to the children led 
to improved vocabulary learning. Field studies are of¬ 
ten said to be preferable, and more ecologically valid 
than data collected in the laboratory. However, practi¬ 
cal, technical, and methodological constraints may limit 
the length and nature of the field studies. Also, bringing 
a robot in the wild does not necessarily make the inter¬ 
actions with people more natural, and the more messy 
conditions pose hard methodological problems and can 
often interfere with controlled data collection and sta¬ 
tistical data analysis [72.204]. A field study by Heylen 
et al. [72.204] placed a simple dialogue Nabaztag (rab¬ 
bit) robot in a few people’s homes over 10 days, and it 
illuminates a number of those real-life problems with 


field studies which impacts on the validity of the re¬ 
sults that can be gained and on the user experience. 
The latter will then ultimately decide on whether people 
will consider a long-term companion robot amusing or 
a nuisance [72.204]. A taxonomy to characterize stud¬ 
ies in the wild for child-robot interactions is provided 
in Salter et al. [72.205]. Systematic analyses of dif¬ 
ferent experimental conditions and tasks, settings, etc., 
whether in the laboratory or in field studies, will help 
the planning, design and comparison of different long¬ 
term studies. 

Note, often the expectations of users are not met 
by the robot’s design and abilities, and nor does it 
fit into daily activities of users [72.190]. The ad¬ 
vancement of robot technology and knowledge of 
HRI will enable more and more field studies, in 
schools [72.206], nurseries [72.22], private homes 
[72.190], care homes [72.31], etc. 

In many of the above long-term studies, commer¬ 
cially available robots have been used. The reasons for 
this are not only availability, but also robustness and re¬ 
liability, as well as safety. The use research prototypes 
in field studies should always entail securing Institu¬ 
tional Review Board (IRB) approval to meet ethical 
guidelines when human participants are involved. Even 
prototypes that are generally safe to use, may still have 
cables sticking out, etc. Hence often it is a necessity to 
have researchers present at all times during these stud¬ 
ies. Research prototypes are also by their very nature 
more prone to break down, and again this requires the 
constant attention of researchers. The design of research 
prototypes is also often not ideally suited for a field 
study, e.g., wheels when being used on cluttered sur¬ 
faces, etc. Interesting insights on those practical but 
nevertheless very important points can be found, e.g., 
in Hiittenrauch et al. [72.207] where a modified Peo- 
plebot (Mobile Robotics) robot was brought into eight 
different homes for approximately 1 hour each in order 
to study spatial management in HRI situations. Note, 
even when leaving a robot alone with a person or fam¬ 
ily in their own home, people may still be acutely aware 
of the experimental nature of the interaction [72.204]. 

Establishing and maintaining relationships between 
users and robots in long-term interactions needs care¬ 
ful consideration, and many insights can also be gained 
from related long-term studies with virtual agents, e.g., 
as exercise advisors [72.208, 209]. 

72.7.3 Robot Home Companions 

for Supporting Elderly Users 

Several HRI studies in the home focus on the use of 
a companion robot for the general population. Koay 
et al. [72.210] report on a long-term, 5-week study 
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in a domestic environment with a partially remote- 
controlled robot involving several different scenar¬ 
ios (e.g., a collaborative task, sharing physical space, 
recording and revealing personal information, interrupt¬ 
ing a person to serve them, and seeking assistance from 
a person using combinations of physical and verbal 
cues). The results highlight concerns and expectations 
that people have in such scenarios that are relevant for 
the domestic use of robots. Note, the robot used in this 
study was partially remotely controlled, which allowed 
the investigation of complex scenarios. 

Due to demographic changes, a lot of interest ex¬ 
ists worldwide into developing robot technology for the 
care of elderly people that would allow elderly people to 
remain in their own homes for longer, or to provide as¬ 
sistance in sheldered or otherwise specifically designed 
environments. A robot may assist an elderly person in 
the home in different ways, e.g., provide physical assis¬ 
tance (standing up, walking, fetching and carry objects, 
etc.), social assistance (e.g., engaging the user in in¬ 
teractions with other people) and cognitive assistance 
(e.g., reminder functions). 

In HRI, there is prior work investigating robot’s 
ability to remind users of future or prior activities. 
Autominder using the Nursebot represents an initial 
attempt to make reminders more intelligent and dy¬ 
namic [72.211]. Intelligence and dynamics were also 
key aspects of the system proposed by the Robo- 
care project [72.212]. More recently, the KSERA 
project [72.213] focused on the ability of the robot 
to draw on information not readily available to the 
user as well as its ability to persuade in order to 
safeguard the health of the user. Other examples for as¬ 
sistance and reminders for elderly people with dementia 
through multimodal interactions and smart-house in¬ 
tegration include the EU FP7 CompanionAble project 
and the MOBISERV project [72.214]. The project, Flo¬ 
rence [72.215], introduces a commercially available 
robot as an autonomous lifestyle device for ambient 
assisted living; it provided multiple services to users 
including an agenda reminder application that allowed 
the elderly to share information with caregivers, etc. 
The European projects SRS [72.216] and ACCOM¬ 
PANY [72.217] both use the care-o-bot 3 (Fraunhofer) 
(Fig. 72.12) as their target robotic platform. While 
SRS involves the use of the robot in a remote-control 
scenario, the ACCOMPANY project develops fully au¬ 
tonomous behaviors for an empathic and assistive home 
companion. 

Several projects combine a robotic companion with 
an intelligent smart/ambient environment as part a vari¬ 
ety of ambient assisted living (AAF) solutions to assist 
with older adults, or people with disabilities, to live in¬ 
dependently by providing support in activities of daily 



Fig. 72.12 Care-O-bot 3 robot (courtesy of Fraunhofer) 


living (ADFs). Such topics are currently being stud¬ 
ied world-wide. See, e.g., the quality of life technology 
(QOLT) [72.218] Centre of Carnegie Mellon University 
or the Centre for Affective Solutions for Ambient Liv¬ 
ing Awareness (CASALA) [72.219]. 

72.7.4 Example: Long-Term Interactions 
with a Robotic Weight Loss Coach 

A number of recent studies investigate the use of 
robotic companion as health coaches or advisors. 
Kidd and Breazeal developed Autom [72.116,220] 
a robot specifically designed for long¬ 
term interaction with people in the role of a robotic 
weight loss coach. Autom has a clear function and role, 
but also needs to interact with users in a socially accept¬ 
able and comfortable manner, so that users intuitively 
understand the robot, are willing to engage with the 
robot, and listen to it. This robot is also an example how 
academic research in long-term HRI can lead to new in¬ 
novations and commercialization. The main purpose of 
the robot is to help the user to lose weight, trying to per¬ 
suade the user to change his or her behavior [72.221], 

A key ingredient supporting this functionality is 
the creation of a relationship between the user and 
the robot. Autom is thus an example of a robot that 
presents itself as a relational artifact and encourages 
people to develop social-emotional relationships with 
them [72.198,199]. Autom uses a psychologically in¬ 
spired relationship model, with the robot’s role as 
a caregiver. It engages the use in a dialog modeled 
after patient-care professional dialog to provide social 
support (Sects. 72.5.4 and 72.5.5) in order to build 
a working alliance as well as to be helpful, persuasive, 
positive, and supportive. For a health care coaching 
robot, where behavior change is a central objective, 
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such a relationships is very important. People who need 
to lose weight typically do not lack the intelligence 
or knowledge necessary to understand that weight loss 
would improve their health and general well-being. The 
motivational factor is often a crucial point, so a weight 
loss robot’s role on providing social support is differ¬ 
ent from that of a kiosk robot only needs to provide 
information. 

A long-term study with 45 participants an Au- 
tom prototype robot over 6 weeks was the first study 
designed to create behavior change in people with 
a weight management goal (Fig. 72.13). The results 
were very encouraging: participants developed a close 
relationship with the robot, and they tracked their calo¬ 
rie consumption and exercise much longer when using 
the robot, compared to other methods (a computer run¬ 
ning the same dialog or a paper log for manual entry). 
Since these factors are indicators of longer term weight 
loss success, the study provided evidence for the effec¬ 
tiveness of sociable robots for long-term HRI [72.1 16]. 

72.7.5 Challenges 

Many of the projects studying the use of robot com¬ 
panions (e.g., supporting elderly users, providing assis¬ 
tance in therapy, or helping in office environments) are 
still at an initial stage. Future results from extensive, 
long-term, multi-site and even cross-cultural evaluation 
studies are still needed in order to illuminate the us¬ 
ability, usefulness, and acceptability of such systems. 
It is particularly advantageous if the same research 
platform is used in different projects so that direct com¬ 
parisons and the sharing of research developments are 
possible. 

The challenges of supporting engagement in long¬ 
term studies are beginning to emerge. New sensors 
and interfaces, such as brain-computer interfaces, could 
provide richer information on people’s emotional, at- 
tentional engagement states when interacting with 
a robot. Robots are providing richer social cues to ex¬ 
plore the whole spectrum of human-human interaction 
modalities, e.g., using gaze, gesture, proxemics, dia¬ 
logue, contingency, etc. [72.177,222-226]. Measuring 
engagement in real time and allowing a robot to respond 
to it is a key challenge. In a user study with 37 partic¬ 
ipants and a robot that could perceive user engagement 
Sidner et al. [72.32] found that engagement gestures 
were perceived more positively than a robot without 
such gestures. Rich et al. [72.139] proposed an initial 
computational model for a robot to model engagement, 
based on the recognition of different events involving 
gesture and speech. 

A clear framework of what engagement means in 
HRI in general, and for long-term HRI interactions 



Fig.72.13a,b Autom, a weight management and exercise coach, 
(a) The Autom prototype robot developed at MIT; (b) the commer¬ 
cial version of the Autom robot 


in particular, is needed and will needs to be vali¬ 
dated in extensive testing. Several other proof of con¬ 
cept studies of robots that adapt to user engagement 
and interaction show encouraging results. Frangois 
et al. [72.227] evaluated a robot that adapts to user 
interaction in a therapeutic context based on tactile 
information (how children touched the robot), while 
Szafir and Mutlu [72.228] demonstrated how a robot 
can adapt to the engagement of the user, utilizing 
techniques from brain-computer interfaces in order to 
assess engagement. Future applications of this research 
into adaptive robots target educational settings where 
a teacher robot can automatically adapt to the pupils’ 
engagement, or therapeutic/rehabilitation applications 
where the robot can automatically adapt to the pa¬ 
tient’s needs and preferences in light of therapeutic 
objectives. 

Future research needs to illuminate how best to de¬ 
sign interaction and embodiment of social robots that 
are successful in long-term interactions with people. 
Since adding social interaction skills to a robot is costly, 
the identification of a set of robot characteristics (ap¬ 
pearance, behavior, cognition) that are necessary and 
sufficient to create meaningful, acceptable, and efficient 
long-term interaction with a robot would be benefi¬ 
cial. Here, it is important to provide frameworks that 
connect to empirical methodologies, cf. the discus¬ 
sion of a conceptual and methodology framework for 
robot believability in Rose et al. [72.229]. Note, while 
predominantly humanoid or zoomorphic robots are cur¬ 
rently being used in long-term HRI experiments, even 
mechanically looking robots such as the Roomba robot 
may invite people to develop a social-emotional rela¬ 
tionship with its users [72.201, 230]. 

Using robots in long-term repeated interactions with 
people also involves a number of ethical issues, as it 
has already been highlighted also for embodied conver¬ 
sational agents and virtual characters [72.208]. While 
in some applications the robot may have a useful 
function if portrayed as a care-receiver [72.133], in 
most applications the companion robot should provide 
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care to its users [72.191]. As pointed out by Turkle 
et al. [72.198,199], robots that present themselves as 
relational artifacts may influence people’s understand¬ 
ing and expectations of the nature of social relationships 
and friendship. Ethical issues are particularly impor¬ 
tant when users are vulnerable people such as adults 
with special needs, elderly people, or children. Many 
researchers have commented on these issues [72.231, 
232], 

Practical issues can also become crucial to the ac¬ 
ceptance of a robot for long-term interaction, as it 
has been highlighted in a long-term study with Pleo - 
a commercially available robot, but which requires 
maintenance that does not fit easily into people’s rou¬ 
tines [72.190]. Fernaeus et al. also bring up the impor¬ 
tant issue of interaction design that needs to support 


long-term interaction, an issue that was also brought up 
in user feedback from the participants in their long-term 
study. Novel approaches to how to design human-robot 
interaction may be required to facilitate long-term use 
of companion robots [72.190], and in real-world set¬ 
tings attention must be paid to how the robot technology 
is being introduced to people and how people and robots 
must adapt to each other [72.230]. 

Recently, the blending of virtual and robotic charac¬ 
ters area a growing area of research whereby seamless 
transitions from a virtual to and from a robotic char¬ 
acter [72.233], or migration between different robot 
and other digital embodiments [72.234] pose interesting 
challenges for future long-term companions. Indeed, it 
may change the nature of what we usually perceive as 
a companion robot [72.148, 233,235-242]. 


72.8 Tactile Interaction with Social Robots 


In robotics and artificial intelligence research, tactile 
interaction has long been exploited primarily as a ne¬ 
cessity to enable, e.g., collision detection, grasping and 
object manipulation, particularly in combination with 
vision and other sensor modalities, and led to adding 
touch sensors to a robot’s gripper or hand [72.243]. 

72.8.1 Touch for Social Interaction 

However, the importance of human-robot tactile inter¬ 
action has been highlighted recently in a number of 
research projects, inspired by evidence from human- 
human interaction and child development. The recent 
interest in the sense of touch goes beyond the necessity 
of touch for interactions with the physical environment, 
but focuses on the important role of tactile interac¬ 
tion in interactions with the social environment. For 
instance, Siegel et al. [72.244] found that social touch, 
such as shaking hands, can impact how persuasive 
a robot is perceived to be [72.244]. In other work, the 
same kind of touch can have a different impact on 
people’s response to the robot depending on the con¬ 
text [72.245]. 

Indeed, humans are born as tactile creatures. Phys¬ 
ical touch is one of the most basic forms of human 
communication. In human development, touch plays 
a crucial role in developing cognitive, social, and emo¬ 
tional skills, as well as establishing and maintaining at¬ 
tachment and social relationships. Deprivation of touch 
in early child development can have devastating effects 
on a child’s development [72.246]. A comprehensive 
survey on communicative functions of touch in humans 
and other animals can be found in Hertenstein [72.247], 


72.8.2 Touch Sensors and Mechanisms 
Used for HRI 

Recently, more and more social robots are being 
equipped with tactile skin, thus allowing the robot to re¬ 
act according to the person touching the robot. Recent 
trends, e.g., in the European project Roboskin [72.248] 
tend toward covering the whole, or most of the robot’s 
body, see e.g., Schmitz et al. [72.249] for an example us¬ 
ing modular capacitive sensors to cover the humanoid 
robot iCub. Related work by Dahiya et al. [72.250] 
surveys a variety of different technological approaches 
toward tactile sensors and mechanisms of tactile sen¬ 
sor for robots. They show how one may take inspi¬ 
ration from biological tactile sensors in humans and 
derive design hints for robotic tactile sensing. Stiehl 
et al. [72.251] designed a tactile system inspired by 
the human skin as well as somatic processing [72.251], 
and was later developed to recognize social and affec¬ 
tive communicative intent of how a human touches the 
robot [72.252,253]. The new compliant skin technol¬ 
ogy developed in the Roboskin project has two primary 
functions: (a) to allow a robot to operate safely and 
efficiently, and (b) to use tactile sensors for communi¬ 
cation, interaction, and cooperation with people. 

The field of tactile human-robot interaction is in¬ 
deed a growing area of research, and a recent survey 
discusses tactile HRI from the perspectives of the types 
of interactions that may occur between a robot and 
a human, and the types of sensors that allow to de¬ 
tect these interactions in various robotic systems such 
as the Robovie series of robots, RI-MAN, the Hug- 
gable robot [72.253], or Paro [72.254], Note, equipping 
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a robot with tactile sensors may add its functional¬ 
ity, but in some cases the sense of touch is crucial 
for the key functionality of the robot (e.g., in the case 
of Paro [72.251] with a therapeutic/care function, or 
in the case of the Huggable to improve social rela¬ 
tionships [72.255]). Many toy robots built primarily 
for human-robot interaction have touch sensors for 
petting and stroking, e.g., the AIBO or more recent 
Pleo robots. Indeed, the use of tactile HRI to sup¬ 
port human-human communication over the distance 
is a promising area of research [72.255-258]. Tactile 
feedback can also improve teaching a robot by demon¬ 
stration [72.259]. 

In order to realize robots’ capability of a sense of 
touch, there are difficult sensor processing and percep¬ 
tion problems to be addressed. One problem is that 
we need to identify the geometrical relationship be¬ 
tween a number of sensors embedded in a whole body 
of a robot and body parts being touched when sen¬ 
sors are activated. For better perception of various 
touching, one can embed sensors into or under a soft 
skin. For instance, Robovie II-F has 274 sensor ele¬ 
ments (Piezofilms) embedded in soft silicone rubber 
(Fig. 72.14). Tactile action activates multiple sensors 
at the same time, thus nearby sensors are all activated 
when touch action occurs to one place of body. For 
this problem, Noda et al. [72.260] took a bottom-up 
approach using observed signal patterns. Their method 
works in a self-organizing manner to identify mapping 
between signal patterns and touched location. 

The second problem is the semantic relationship be¬ 
tween signal pattern and people’s communicative inten¬ 
tion of touch action. For instance, Tajika et al. [72.261], 
applied clustering method for signal pattern to re¬ 
trieve hierarchical structure of haptic actions. For 
instance, tickling-chest (Fig. 72.15) is found to be 
similar to stroking-chest, grouped together as lightly 
touching-chest action. Knight et al. [72.262] developed 
an algorithm based on identification of typical touch 
types, such as tickle, pet, and poke. Yohanan and 
MacLean [72.263] categorized humans’ intent to com¬ 
municate affect state and mapped touch action to each 
category, for example, comforting intent is mapped with 
actions like stroke and pat. Stiehl and Breazeal [72.264] 
trained a neural net to classify touch according to af¬ 
fect and communicative intent by recognizing types of 
socio-affective touch (e.g., pleasant or unpleasant ways 
of touching or teasing based on the way a person tickles, 
touches, pats, slaps, rubs, squeezes, etc.) 

Frangois et al. [72.227] describe an algorithm for 
pattern recognition in HRI, the cascaded information 
bottleneck method and apply it to real-time autonomous 
recognition of human-robot tactile interaction styles. 
This method uses an information theoretic approach 


Q 50x50 mm 



Fig. 72.14 An example of layouts of piezofilm sensors embedded 
in a soft skin {left) 



Fig. 72.15 A child tickling the chest of a robot 


and enables to progressively extract relevant informa¬ 
tion from time series. An evaluation with real inter¬ 
action data obtained with a Sony AIBO robot shows 
that the algorithm is capable of classifying interaction 
styles (frequency and gentleness of the interaction), 
with a good accuracy and a very acceptable delay. The 
cascaded information bottleneck method was later suc¬ 
cessfully applied to create a socially adaptive robot that 
can recognize and adapt to children’s play styles in real 
time [72.265]. The robot rewards well-balanced inter¬ 
action styles and encourages children to engage in the 
interaction. The potential impact of such an adaptive 
robot in robot-assisted play for children with autism is 
evaluated through a study conducted with seven chil¬ 
dren with autism in a school. A statistical analysis of 
the results shows the positive impact of such an adaptive 
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robot on the children’s play styles and on their engage¬ 
ment in the interaction with the robot. 

72.8.3 Example: Teaching Children 
with Autism About Tactile 
Interaction 

Using robots for the therapy and education for children 
with autism was first proposed by Kerstin Dauten- 
hahn [72.266,267] and it has recently attracted a lot 
of attention in the research community. A number of 
such approaches focus on robot-assisted play, since 
play has a crucial role in a child’s development. Dur¬ 
ing play, children can learn about themselves and their 
environments as well as develop cognitive, social, and 
perceptual skills. Autism, or better autistic spectrum 
disorders (ASD), is a life-long development disorder 
with key impairments in communication and social in¬ 
teraction and imagination (DSM IV, 1995) [72.268, 
269]. Robots allow for a simplified, predictable, and 
reliable environment, where the complexity of interac¬ 
tion can be controlled and gradually increased [72.270]. 
Children can play dyadic games with robots, or triadic 
games involving other children or adults. In the latter 
case, scenarios emphasize the role of the robot as a so¬ 
cial mediat or [72.271]. 

Robot mediated playing and learning activities, if 
successful, have the potential to enable children with 
cognitive disabilities to learn and acquire basic so¬ 
cial skills thus getting support to develop/enhance their 
individual potential especially in the areas of commu¬ 
nication and interaction [72.270,272]. A number of 
different social robots have been studied in this do¬ 
main, including zoomporphic robots, humanoid robots, 
or mechanically looking robots, e.g., Labo-1 [72.273], 
NAO [72.274], Probo [72.275], Robota [72.276, 277], 
Keepon [72.278], Aibo [72.279], Tito [72.280], KAS- 
PAR [72.21], and others. While we find encouraging 
results from case-study evaluations, further long-term 
and clinical studies are required [72.281]. 

Diehl et al. [72.281] in their review on the clinical 
use of robots for individuals with autism suggested that 
the use of interactive robots is a promising develop¬ 
ment in light of the research showing that individuals 
with autism exhibit strengths in understanding the phys¬ 
ical world and relative weaknesses in understanding the 
social world. Children with autism often find it very 
difficult to appropriately interact with their social en¬ 
vironment. In interactions between children and their 
peers, teachers, and family, usually touch has an impor¬ 
tant communicative and emotional role. 

However, children with ASD may be hyposensitive 
or hypersensitive to touch so that the children may crave 
or avoid touch. As part of the above-mentioned Eu¬ 


ropean Roboskin, a number of case study evaluations 
have been performed investigating how a humanoid 
robot equipped with touch sensors can teach chil¬ 
dren with autism about appropriate tactile interaction 
and the associated emotional responses. The studies 
used the child-sized, minimally expressive robot KAS- 
PAR [72.21]; [72.282]), a low-cost robot specifically 
designed for interaction - many of its features lend 
themselves to children with autism, e.g., the human¬ 
like but minimally expressive shape and form of the 
robot. The robot has 8 DOFs (degrees of freedom) 
in the head and face and 6 DOF in each arm, as 
well as one DOF in the torso. A comprehensive set 
of play scenarios for robot-assisted play for children 
with various special needs, and tactile play scenarios 
for children with autism have been developed [72.283- 

285] , 

KASPAR can play a variety of games with children, 
either operating completely autonomously [72.285, 

286] and kaXH-SaEBM, or being partially remotely con¬ 
trolled by the children or an adult as part of the play 
scenario. Case studies on tactile child-robot interaction 
have shown encouraging results ( Robins et al. [72.283, 

287] ). In the research on tactile interactions KASPAR 
was equipped with patches of tactile sensors, and it pro¬ 
duced some responses autonomously, e.g., a ticking of 
the chest resulted in laughing, or hitting in the face re¬ 
sults in the robot turning away from the child, covering 
its face with its hands and saying ouch, that hurts. If 
the child touches areas that are not covered by skin 
patches, or if the recognition of different types of touch 
is not reliable enough [72.288], then the experimenter 
can trigger the robot’s reaction. 

Figure 72.16 shows a child with autism first hitting 
the robot’s leg and then exploring the robot’s reaction. 
Since the legs cannot detect forceful touch, the experi¬ 
menter triggers the robot’s reactions. Note, this hybrid 
approach of autonomous behaviors combined with a re¬ 
mote control allows the robot to be perceptually more 
advanced than current state-of-the-art robotics sens¬ 
ing technology allows. Robots used with children with 
autism need to be predictable, and so a mistake in the 
robot’s sensing abilities can be compensated for by 
the adult present (experimenter, teacher or caregiver). 
A person who knows the child very well will also be 
able to trigger certain useful robot behaviors at certain 
movements in time which are not observable directly 
from the child or the context and can only be inferred 
from detailed knowledge about the child, his/her needs 
and preferences, and therapeutic goals and objectives 
for this particular child. 

Note, for a robot to detect and respond to tactile in¬ 
teractions of children with autism is not only beneficial 
to teach about appropriate tactile interaction, but it can 
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also provide a basis for adapting to a child’s individual 
interaction style [72.265]. 

Tactile interaction is often not the sole focus 
of interaction, it can be embedded in multimodal 
play scenarios with the therapeutic objective to in¬ 
crease children’s social skills through play. Frangois 
et al. [72.279] present a long-term study where six 
children diagnosed with autism interact with an au¬ 
tonomous zoomorphic robot (Sony Aibo) over 10 ses¬ 
sions. The study is inspired by nondirective play ther¬ 
apy to encourage children’s proactivity and initiative¬ 
taking. The behavior of each child is analyzed in detail 
according to three dimensions (play, reasoning, af¬ 
fect). Unique trajectories for children’s progressions 
along these three dimensions were observed, resulting 
in unique profiles. The work highlights methodological 
issues in the domain of robot-assisted therapy, and also 
points out and formalizes different potential roles of the 
experimenter in the sessions, who may be a passive or 
an active participant [72.289]. A regulation process is 
introduced whereby the experimenter can regulate the 
interaction under specific conditions in order to: 

1. Prevent or discourage repetitive behaviors 

2. Help the child engage in play 


3. Give a better pace to the game if it has already been 
experienced by the child 

4. Bootstrap a higher level of play, and 

5. Ask questions related to reasoning or affect. 

72.8.4 Challenges and Opportunities 

While many studies assume that tactile HRI will re¬ 
sult in a more enjoyable, meaningful, and efficient 
interaction with a robot, many issues are still un¬ 
clear and need to be investigated further. For example, 
a video-based study, where participants watched videos 
of interactions rather than interacting with the robot 
themselves, showed the impact of a robot’s level of 
autonomy, and suggests that touch behaviors are con¬ 
sidered more appropriate for proactive as compared 
to reactive robots [72.290]. Much further research is 
needed to find out when and how tactile interaction 
can enhance a person’s experience of interaction and 
benefit the overall performance of the human-robot tri¬ 
ads. One can also expect that individual differences will 
play a role in which types of tactile interaction with 
a robot are more appropriate, depending also on the 
tasks involved and the overall context and setting. The 
perception of people in terms of the robot’s roles and its 



Fig. 72.16 (a) Child hits the 
robot’s leg and then explores 
the robot’s reaction. (b) The 
child sees the robot looking 
sad, so he tickles its tummy 
(left) to make it happy (right). 
(c) The minimally expressive 
humanoid robot KASPAR 
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relationship with its users is also likely to impact these 
issues. 

In recent years a number of projects world¬ 
wide have tried to use robots for therapy of chil¬ 
dren with autism. Different modes of robot control 
and autonomy need to be investigated, from fully 
autonomous systems [72.285], to Wizard-of-Oz con¬ 
trolled robots [72.291], to using a hybrid approach 
where remote control is an integral part of the interac¬ 


72.9 Social Robots and Teamwork 

Verbal and nonverbal communication play a very im¬ 
portant role in coordinating joint action during collab¬ 
orative tasks. Sharing information through communica¬ 
tion acts is critical given that each teammate often has 
only partial knowledge relevant to solving the problem, 
different capabilities, and possibly diverging beliefs 
about the state of the task. For instance, all teammates 
need to establish and maintain a set of mutual beliefs 
regarding the current state of the task, the respective 
roles and capabilities of each member, and the respon¬ 
sibilities of each teammate [72.33, 153]. This is called 
common ground [72.162]. 

72.9.1 Human-Robot Teamwork 
and Collaboration 

Dialog certainly plays an important role in establishing 
common ground. Each conversant is committed to the 
shared goal of establishing and maintaining a state of 
mutual belief with the other. To succeed, the speaker 
composes a description that is adequate for the pur¬ 
pose of being understood by the listener, and the listener 
shares the goal of understanding the speaker. This com¬ 
munication act serves to achieve robust team behavior 
despite adverse conditions, including breaks in com¬ 
munication and other difficulties in achieving the team 
goals. 

Humans also use nonverbal skills such as visual 
perspective taking and shared attention to establish 
common ground with others. They orient their own 
gaze and direct the gaze of their teammate through 
deictic cues such as pointing gestures in order to es¬ 
tablish common ground. Given the visual perspective 
taking, shared attention, and the use of deictic cues to 
direct attention are core psychological processes that 
people use to coordinate joint action about objects and 
events in the world, robot teammates must be able to 
display and interpret these behaviors and cues when 
working with humans in a manner that adheres to hu¬ 
man expectations. 


tion and triggers autonomous behaviors [72.287,289]. 
Realistically, for therapeutic tools to be used widely 
outside the laboratory and the experimental setting, the 
technology needs to be highly robust, reliable, easy to 
operate by nonresearchers as well as cost-effective. 

A number of technological, methodological, and 
design challenges still need to be tackled, but tactile in¬ 
teractions with social robots open up a number of new 
research avenues as well as exciting applications. 


Breazeal et al. [72.142] investigated the impact 
grounding using nonverbal social cues and behavior on 
task performance by a human-robot team. In a human 
subject experiment, participants guided Leonardo to 
perform a physical task using speech and gesture. The 
robot communicates either implicitly through behav¬ 
ior (such as gaze and facial expressions) or explicitly 
through nonverbal social cues (i. e., explicit pointing 
gestures). The robot’s explicit grounding acts include 
visually attending to the human’s actions to acknowl¬ 
edge their contributions, issuing a short nod to acknowl¬ 
edge the success and completion of the task or subtask, 
visually attending to the person’s attention directing 
cues such as to where the human looks or points, look¬ 
ing back to the human once the robot operates on an 
artifact to make sure its contribution is acknowledged, 
and pointing to artifacts in the workspace to direct the 
human’s attention toward them. Both self-reporting via 
questionnaire and behavioral analysis of video support 
the hypothesis that implicit nonverbal communication 
positively impacts human-robot task performance with 
respect to understandability of the robot, efficiency of 
task performance, and robustness to errors that arise 
from miscommunication [72.142]. 

Common ground is grown along with partners that 
work together over time. In a simple example, if two 
persons repeatedly work in a sequence of collabora¬ 
tive manufacturing tasks, one will be able to easily 
predict what the other person will do next and thus 
proactively help each other. For instance, if one always 
needs a spanner to be passed at certain moment in the 
task, another person will probably take anticipatory ac¬ 
tion passing the spanner before being asked. Hoffman 
and Breazeal developed adaptive system that learn such 
task structure which enabled a robot’s anticipatory ac¬ 
tion [72.292]. They further revealed the importance of 
perceptual simulation [72.293]. 

The importance of such common ground and cogni¬ 
tive similarity is demonstrated in partnership in casual 
social interaction too. For instance, Morales et al. re- 
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vealed that a robot efficiently performed side-by-side 
walking with a human partner when it anticipated 
where a human partner would walk [72.186]. Such an¬ 
ticipatory computing was enabled by a capability of 
computing a preferred walking course in a similar way 
as humans would do. There are robots that explicitly 
learn common knowledge, like one that learns names 
of places [72.294], 

72.9.2 Robots as Social Mediators 

Researchers have started to explore the use of 
robots as a social mediator. One approach is to use 
a human-like presence of robots as social stimuli. It 
is known that presence of people facilitate others. 
For instance, people perform simple math-calculation 
faster when being watched by someone else. This is 
known to be social facilitation effect [72.295]. Riether 
et al. [72.296] reported that people’s performance on 
easy math-calculation is improved because of pres¬ 
ence of robots [72.296]. Takano et al. [72.297], put an 
android robot as a bystander where patients meet a med¬ 
ical doctor in a hospital, and found that it moderated 
clients’ anxiety and let them believe that the doctor pays 
more attention to them [72.297]. 

Another approach is to use a robot as an active coor¬ 
dinator in humans’ social settings. When there are many 
people, a coordinator could make their activity more 
efficient. Such a role can also successfully replicated 
by a robot. Consider the situation where an interactive 
robot is placed in the middle of a group of kids, and 
they start to push each other away when they want to 
play with the robot differently from each other. Shomi 
et al. [72.298] , developed a technique to identify when 
crowd of people around a robot is disordered, and let 
the robot perform attention-controlling behavior so that 
children play together with the robot in coordinated 
way [72.299] (Fig. 72.17). In other work, robots have 



Fig. 72.17 The robot interacted with a crowd of people 
with coordinating their attention 


been used to facilitate elderly people’s group conversa¬ 
tion. Matsuyama et al. [72.300], developed a robot to 
participate in a quiz game, conducted as a recreation 
activity in elderly-care facility, and provide inspiring 
answer to facilitate other elderly people to continue the 
game [72.300]. In these works, robots actively model 
the social situation and intervene in people’s activities 
based on its understanding of the situation. 

72.9.3 Research Direction 


Human activities are often social, involving multiple 
people who often have different skills, desires, and 
goals. Although the research is only in an early stage 
we have started to model such social situations while re¬ 
vealing potential roles of robots in such social settings. 
Along with further advance of relevant technologies, 
e.g., manipulation capability, navigation capability, and 
language capabilities, there should be many potential 
uses to be unveiled. However, it is most likely that the 
underlying theoretical work still needs a lot more work 
as well. 


72.10 Conclusion 

In this chapter, we have presented some of the principal 
research trends in social robotics and human-robot in¬ 
teraction. We have relied heavily on examples from our 
own research to illustrate these trends, and have used 
excellent examples drawn from other research groups 
around the world. 

From this overview, we have shown that one of 
the most important goals of social robotics as applied 
HRI is the creation of robots that are human-compatible 
and human-centered in their design. Their differences 
from human abilities should complement and enhance 


our strengths and support how people help one an¬ 
other. Their similarities to human abilities, such as 
computationally implementing human cognitive, af¬ 
fective, or multimodal communication models make 
them more intuitive for people to understand and in¬ 
teract with. Further, such robots are also being used as 
a scientific tool to help us to understand ourselves bet¬ 
ter. With this broadening understanding, social robots 
are being designed to offer increasingly sophisticated 
levels of social, affective, cognitive, and task-based sup¬ 
port for people, opening new applications for robots 
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in education, health, therapy, communication, domes¬ 
tic tasks, physical tasks requiring coordination and 
teamwork, and more. As the held advances, social 
robots are being applied to increasingly sophisticated 
tasks, in increasingly complex human environments. 


72.11 Further Reading 

For further reading, we recommend the following con¬ 
ference proceedings, journals, books, articles: 

• Annual conference proceedings: 

- Proceedings of the ACM/IEEE International 
Conference on Human-Robot Interaction (HRI) 

- Proceedings of the IEEE International Sympo¬ 
sium on Robot and Human Interactive Commu¬ 
nication (ROMAN) 

- AAAI Symposium Series 

- AISB Symposium Series 

• Journals: 

- Journal of Human-Robot Interaction (http:// 
humanrobotinteraction.org/journal/) 

- Interaction Studies-Social Behaviour and Com¬ 
munication in Biological and Artificial Systems 
published by John Benjamins Publishing Com¬ 
pany 

- International Journal of Social Robotics by 
Springer 

- IEEE Transactions on Autonomous Mental De¬ 
velopment (TAMD) 

- IEEE Transactions on Human-Machine Sys¬ 
tems 

- IEEE Transactions on Affective Computing 

- Paladyn, Journal of Behavioral Robotics, de 
Gruyter 

- PLoS ONE 

Video-References 


for longer deployment periods. We expect that in the 
coming decades, many other researchers, especially 
young researchers, will actively contribute to the transi¬ 
tion from today’s robots into capable robot partners of 
tomorrow. 


Reviews and overviews can be found in several 
books and articles: 

• Books: 

- C. Breazeal: Designing Sociable Robots (MIT 
Press, Cambridge 2002) 

- R. W. Picard: Affective Computing (MIT Press, 
Cambridge 1997) 

- J.-M. Fellous, M. Arbib (Eds.): Who Needs 
Emotions: The Brain Meets the Robot (Oxford, 
Oxford Univ. Press 2005) 

- K. Dautenhahn, J. Saunders (Eds.): New Fron¬ 
tiers in Human-Robot Interaction, Advances in 
Interaction Studies, (John Benjamins Publish¬ 
ing, Amsterdam 2011) 

- T. Kanda, H. Ishiguro (Eds.): Human-Robot In¬ 
teraction in Social Robotics (CRC Press, Boca 
Raton 2012) 

• Review Articles: 

- T. Fong, I. Nourbakshsh, K. Dautenhahn: A sur¬ 
vey of social robots, Robotics and Autonomous 
Systems 42, 143-166(2003) 

- M.A. Goodrich, A.C. Schultz: Human-robot in¬ 
teraction: A survey, Foundations and Trends 
in Human-Computer Interaction 1(3), 203-275 
(2007) 
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Home assistance companion robot in the Robot House 

available from http://handbookofrobotics.org/view-chapter/72/videocletails/218 
Visual communicative non-verbal behaviours of the Sunflower Robot 
available from http://handbookofrobotics.org/view-chapter/72/videodetails/219 
Playing triadic games with KASPAR 

available from http://handbookofrobotics.org/view-chapter/72/videodetails/220 
Explaining a typical session with Sunflower as a home companion in the Robot House 
available from http://handbookofrobotics.org/view-chapter/72/videodetails/221 
A robot that forms a good spatial formation 

available from http://handbookofrobotics.org/view-chapter/72/videodetails/257 
A robot that approaches pedestrians 

available from http://handbookofrobotics.org/view-chapter/72/videodetails/258 
A robot that provides a direction based on the model of the environment 
available from http://handbookofrobotics.org/view-chapter/72/videodetails/259 
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Human-robot teaming in a search and retrieve task 

available from http://handbookofrobotics.org/view-chapter/72/videodetails/555 
Social referencing behavior 

available from http://handbookofrobotics.org/view-chapter/72/videodetails/556 
Overview of Kismet's expressive behavior 

available from http://handbookofrobotics.org/view-chapter/72/videodetails/557 
Overview of Autom: A robotic health coach for weight management 
available from http://handbookofrobotics.org/view-chapter/72/videodetails/558 
Non-verbal envelope displays to support turn-taking behavior 
available from http://handbookofrobotics.org/view-chapter/72/videodetails/559 
Learning how to be a learning companion for children 

available from http://handbookofrobotics.org/view-chapter/72/videodetails/560 
Social learning applied to task execution 

available from http://handbookofrobotics.org/view-chapter/72/videodetails/562 

Mental state inference to support human-robot collaboration 

available from http://handbookofrobotics.org/view-chapter/72/videodetails/563 

A learning companion robot to foster pre-K vocabulary learning 

available from http://handbookofrobotics.org/view-chapter/72/videodetails/564 

Influence of response time 

available from http://handbookofrobotics.org/view-chapter/72/videodetails/806 
A scene of deictic interaction 

available from http://handbookofrobotics.org/view-chapter/72/videodetails/807 
An example of a social robot in a museum 

available from http://handbookofrobotics.org/view-chapter/72/videodetails/808 
An example of repeated long-term interaction 

available from http://handbookofrobotics.org/view-chapter/72/videodetails/809 
A robot that exhibits its listening attitude with its motion 
available from http://handbookofrobotics.org/view-chapter/72/videodetails/810 
Region pointing gesture 

available from http://handbookofrobotics.org/view-chapter/72/videodetails/811 
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73. Socially Assistive Robotics 


Maja J. Mataric, Brian Scassellati 


This chapter reviews the critical societal issues 
that have motivated research into socially assistive 
robotics (SAR) (Sect. 73.2) and describes the reason 
why physical robots rather than virtual agents are 
essential to this effort (Sect. 73.3). It highlights the 
major research issues within this area (Sects. 73.4- 
73.7), describes the primary application domains 
and populations where SAR research has shown an 
impact (Sects. 73.8-73.11), and closes with some 
of the ethical and safety issues that SAR research 
necessitates (Sect. 73.12). 
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73.1 Overview 

As robots become increasingly integrated into human 
environments, different forms of interaction are spawn¬ 
ing new research topics and subfields. Human-robot 
interaction (HRI) can be broadly classified into two cat¬ 
egories: physical and social/emotional. Physical HRI 
(Chap. 69) involves the research areas of manipula¬ 
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tion and haptics, among others, and is used in medical 
and rehabilitation robotics (Chaps. 63 and 64). In con¬ 
trast, social/emotional interaction involves verbal and 
nonverbal expression and communication, and thus the 
research areas of assistive robotics, social robotics, and 
SAR. 
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Robots and Humans 


Robots have long been used to provide assistance to 
individual users through physical interaction, typically 
by supporting direct physical rehabilitation (Chap. 64) 
or by providing a service such as retrieving items or 
cleaning floors (Chap. 55). Socially assistive robotics 
(SAR) is a comparatively new field of robotics that fo¬ 
cuses on developing robots capable of assisting users 
through social rather than physical interaction. Just as 
a good coach or teacher can provide motivation, guid¬ 
ance, and support without making physical contact with 
a student, socially assistive robots attempt to provide 
the appropriate emotional, cognitive, and social cues to 
encourage development, learning, or therapy for an indi¬ 
vidual. SAR focuses on the challenges of providing mo¬ 
tivation, coaching, training, and rehabilitation through 
nonphysical interaction; such systems have been val¬ 
idated in hands-off stroke rehabilitation, social skill 
training of children with autism, and eldercare, among 
others. In contrast, most of rehabilitation robotics fo¬ 
cuses on physical interaction with the patient; such sys¬ 
tems have been validated in hands-on stroke rehabil¬ 
itation. Social robotics (Chap. 72) focuses on endow¬ 
ing robots with the ability to behave in socially aware 
and engaging ways; such systems have been validated 


in museums, movies, classrooms, and informal settings. 
Service robotics can be seen as the overarching field that 
encompasses much of the work above. 

A socially assistive robot is a system that em¬ 
ploys hands-off interaction strategies, including the 
use of speech, facial expressions, and communica¬ 
tive gestures, to provide assistance in accordance with 
a particular assistive context. SAR systems equipped 
with motivational, social, pedagogical, and therapeu¬ 
tic capabilities have the potential to improve access to 
personalized care, training and rehabilitation to large 
populations, including individuals post-stroke, elderly, 
and children with social and developmental disabilities, 
in order to enhance their quality of life. An effective 
socially assistive robot must understand and interact 
with its environment, exhibit social behaviour, focus its 
attention and communication on the user, sustain en¬ 
gagement with the user, and achieve specific assistive 
goals. The robot must do all of this in a way that is 
safe, ethical and effective for the potentially vulnerable 
user. Socially assistive robots have been shown to have 
promise as therapeutic tool for children, the elderly, 
stroke patients, and other special-needs populations re¬ 
quiring personalized care. 


73.2 The Need for Socially Assistive Robotics 


The need for socially assistive robots is driven not 
only by rapidly growing populations that need addi¬ 
tional assistance (including the elderly, individuals with 
cognitive disabilities, and individuals who require non¬ 
physical support during rehabilitation) but also by the 
recognition that social robots can have a substantial 
positive impact on personal challenges such as main¬ 
taining a healthy diet, supporting an active lifestyle, 
maintaining therapeutic programs, and supporting indi¬ 
vidualized education [73.1]. 

One of the most rapidly expanding populations 
requiring daily assistance is the elderly. The United Na¬ 
tions estimates that the population of those aged 60 and 
older will double in the next thirty years, with the el¬ 
derly outnumbering children for the first time in history 
by 2045 [73.2]. Space and staff shortages at nursing 
homes and other care facilities are already an issue to¬ 
day. As the elderly population continues to grow, a great 
deal of attention and research will be dedicated to as¬ 
sistive systems aimed at promoting ageing-in-place, 
facilitating living independently in one’s own home as 
long as possible. While many of these users will require 
some degree of physical assistance (such as standing up 
or carrying heavy groceries), many more are expected 
to require social and cognitive support to reduce isola¬ 


tion, maintain active and healthy lifestyles, and to assist 
in daily therapy routines. 

Individuals with cognitive disabilities and devel¬ 
opmental and social disorders constitute another key 
growing population that can benefit from SAR in the 
contexts of special education, therapy, and training. 
Autism spectrum disorders (ASDs) have been the fo¬ 
cus of much of the research in this area due to their 
prevalence, rapid growth of diagnoses, inherently so¬ 
cial nature, and the critical need for long term, intensive 
treatment. Current research suggests that 1 in every 60 
children in the United States will be diagnosed with 
ASD; the rate of diagnosis has increased sixfold be¬ 
tween 1994 and 2003 [73.3]. The cause of the increase 
is not yet known. However, early intervention is critical 
to a positive long-term outcome, and many individu¬ 
als need high levels of support and care throughout 
their lives [73.4—8]. Research into applying robots as 
therapy tools for ASD has focused on how to support 
repetitive, but necessary, social and behavioral thera¬ 
pies [73.9]. 

Motivation is recognized as the most signifi¬ 
cant challenge in physical rehabilitation and train¬ 
ing [73.10]. SAR technology has the potential to 
provide novel means for monitoring, motivating, and 
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coaching. Post-stroke rehabilitation is one of the largest 
potential application domains, since stroke is a domi¬ 
nant cause of severe disability in the growing ageing 
population. In the United States alone, over 700000 
people suffer a new stroke each year, with the majority 
sustaining some permanent loss of movement [73.11]. 
Stroke patients frequently have difficulty with everyday 
functional movements and activities; the loss of func¬ 
tion can be decreased through rehabilitation therapy 
during the critical post-stroke period. Such rehabil¬ 
itation therapy involves carefully designed repetitive 
exercises, which can be passive and active. In passive 
exercises, the therapist (or a robot) actively helps the 
patient to repeatedly move the stroke-affected limb as 
prescribed. In active exercises, the patient does the work 
him/herself, with no physical assistance. 

Finally, there is growing interest in using socially as¬ 
sistive robots to support an active and healthy lifestyle. 
These applications are again motivated by rapidly grow¬ 
ing prevalence of obesity and a lack of effective in¬ 
terventions that are not labor-intensive. Childhood and 
adolescent obesity has been related to a host of adverse 


proximal and distal health outcomes, including high 
cholesterol and triglycerides, hypertension [73.12], in¬ 
sulin resistance [73.13], type 2 diabetes [73.14], nonal¬ 
coholic fatty liver disease [73. 15], as well as breast, col¬ 
orectal, and other cancers [73.16]. Furthermore, child¬ 
hood obesity sustains adverse consequences over the 
entire life span. In one review, all of the 25 longi¬ 
tudinal studies surveyed showed that overweight and 
obese youth were at higher risk of becoming overweight 
adults [73.17]. Per capita medical spending for an obese 
person is roughly 42% higher than for a person of nor¬ 
mal weight [73.18] and outranks the health costs of both 
smoking and drinking alcohol [73.19]. Understanding, 
preventing, and treating childhood obesity is therefore 
a top public health priority [73.20]. Education about 
health impacts of obesity in addition to coaching and 
motivation for a healthy lifestyle are known to be ef¬ 
fective means of lowering obesity rates [73.21]. SAR 
has the potential to play an important role in this con¬ 
text as an affordable and accessible means of providing 
personalized coaching, motivation, and training for ex¬ 
ercise and healthy eating habits [73.22]. 


73.3 Advantages of Embodied Robots over Virtual Agents 


Compared to virtual agents, robotic agents are typi¬ 
cally more expensive, more difficult to replicate and 
distribute, more difficult to maintain and support, re¬ 
quire a longer development time, and more fragile and 
susceptible to damage and system failure. In physically 
assistive tasks, virtual agent alternatives are not possi¬ 
ble; manipulation of physical objects requires a phys¬ 
ical presence. For socially assistive tasks, the added 
costs of a physical robot must be justified in compar¬ 
ison with a virtual agent solution. 

73.3.1 Embodiment and Social Influence 

One primary reason for using a physical robot in a so¬ 
cially assistive task is that robots can often exert more 
social influence over the user compared to a virtual 
agent. Socially assistive tasks often require the user 
to engage in tasks (such as therapy or homework) 
that are challenging and that the user might need en¬ 
couragement or guidance to maintain. Multiple studies 
have demonstrated that physically embodied and collo¬ 
cated robots are more capable of generating the social 
presence to convince users to take these challenging 
steps. A physically present robot yields significantly 
more compliance to its requests, even when those re¬ 
quests are unexpected or unconventional, than a video 
representation of the same robot [73.23]. Kidd and 


Breazeal [73.24] developed a table-top robot to serve 
as a daily weight-loss advisor; it engaged users through 
a touch-screen interface, tracked user progress and 
the state of the user-robot relationship over time, and 
was tested in a six-week field study with participants 
at home. The study compared the robot interactions 
against alternative weight loss methods, including us¬ 
ing a standalone computer and maintaining paper logs, 
and found that participants exercised and tracked their 
caloric intake for nearly twice as long when interacting 
with the robot than with the other methods. 

Interactions with physical robots are also more en¬ 
gaging and credible than virtual agents in cooperative 
task scenarios. For example, following a cooperative 
block-stacking task with a talking agent, participants 
found an agent more engaging, enjoyable, informa¬ 
tive, and credible if it were a physically embodied 
robot, than if it were a virtually embodied animated 
character [73.24]. Other work has shown that after 
playing a cooperative game, people most enjoyed, and 
attributed the most watchfulness and helpfulness to 
a physically embodied robot than to a virtual embodi¬ 
ment (a graphical simulation) of the same robot [73.26]. 
Bartneck [73.27] conducted a study comparing the ef¬ 
fectiveness of an emotionally expressive robot, eMuu, 
with its screen character version in engaging users in 
a simple negotiation task, and found that participants 
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Fig.73.1a-c The physical presence 
of a robot tutor (c) has been shown 
to increase learning gains during 
a puzzle solving task substantially 
more than when the same tutoring 
lessons are delivered by a video of the 
same robot (b) or from a disembodied 
voice (a) (after [73.25]) 


exerted more effort and received higher task scores 
when interacting with the physical eMuu than with the 
simulated eMuu. Jung and Lee [73.28] also demon¬ 
strated the positive effects of physical embodiment in 
relation to interactions with both a Sony Aibo robot 
and an anthropomorphic dancing robot, April. Another 
study compared a physical co-present robot, to a re¬ 
motely located one shown on the video screen, to 
a simulation of the same robot, all in the same interac¬ 
tion context; the study participants rated the co-present 
robot as more helpful, watchful, and appealing than 
remote and simulated counterparts [73.29]. These find¬ 
ings suggest that physical embodiment affords greater 
social attribution or enjoyment of an agent, than does 
virtual embodiment. 

73.3.2 Embodiment and Learning 

Physical robots have also been shown to enhance 
learning and impact later behavioral choice more sub¬ 
stantially than virtual agents. Leyzberg et al. [73.25] 
showed that even when delivering the same lessons on 
solving cognitive puzzles, users improved their perfor¬ 
mance during a 20 min session almost twice as much 
when those lessons were delivered by a physical robot 
rather than by an on-screen video of the same robot. 
Kiesler et al. [73.30] used task performance measures 
to find that participants who received health advice 
from a physically present robot were more likely to 
choose a healthy snack than participants who received 
the same information in robot-video or on-screen agent 
conditions. Kidd and Breazeal [73.22] demonstrated 
that users were more likely to continue sessions with 
a robot weight-loss coach than with a virtual coach over 
a period of six weeks (Fig. 73.1). 

73.3.3 Embodiment 

and Special Populations 

While most of the investigations of the impact of phys¬ 
ical embodiment on socially assistive tasks have been 
conducted with typical adults, embodiment effects have 
also been demonstrated on many of the particular pop¬ 
ulations that SAR projects typically target. Tapus and 


Mataric [73.31] found that individuals suffering from 
cognitive impairment and/or Alzheimer’s disease re¬ 
ported being more engaged with a robot treatment than 
a similar on-screen agent treatment. Numerous studies 
have demonstrated that children and adolescents with 
ASD engage readily with robots even when those chil¬ 
dren are unlikely to engage with animated characters on 
a screen [73.9, 32], 

This impact of embodiment on preschool-age chil¬ 
dren has been of particular interest to educators and 
developmental psychologists. Although preschoolers 
learn from watching television and on-screen agents, 
many studies have focused on how learning and behav¬ 
ior shaping are more robust and generalize more readily 
when embedded in real time interactions with a physi¬ 
cal agent: 

1. Preschool children are multimodal learners, learn¬ 
ing and retaining information better if there are 
overlapping modalities of experience [73.33-35]. 

2. Infants and preschool children off-load cognitive 
functions to the 3-D (three-dimensional) spatial 
structure of the world, for example, remembering 
the direction of attention rather than the content and 
then looking back to a location to re-perceive the 
content [73.36-38]. 

3. Children learn by physically manipulating and 
grouping objects [73.39, 40]. 

4. Preschool children learn from gross (whole body) 
movements; they do not have the fine motor skills to 
work in a limited spatial arena (such as a computer 
screen) and do not sustain attention when their gross 
motor activity is limited [73.41], 

These facts suggest that learning and develop¬ 
ment will be best achieved in physical environments 
with physically realized teachers. Robotics may offer 
a mechanism to augment and support teacher-child and 
parent-child interactions. 

Embodiment studies that have targeted the elderly 
population include the work of Heerink et al. [73.42], 
who investigated the acceptance of assistive social 
agents by older adults. The robot used in their evalu¬ 
ation was a table-top robot (the iCat), and was either 
controlled via a human operator during interaction with 
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elderly users (Wizard of Oz study), or interacted with 
users through a touch-screen interface. The interactions 
consisted primarily of short informational or utility in¬ 
teractions (e.g., medication/agenda reminders, weather 
forecast, companionship). Fasola and Mataric [73.43] 
also investigated the role of embodiment in a SAR 
exercise coach system, with a between-subjects study 
with participants split into two groups: physical robot 
coach versus virtual robot coach. Participants engaged 
in four exercise sessions over a two-week period, with 
each session lasting 20 min. The results of the between- 
subjects data analysis showed participants rated the 
physical robot significantly more positively than the 
virtual robot across all of the subjective measures, in¬ 
cluding enjoyableness and usefulness of the interaction, 
and also in terms of helpfulness, intelligence, social 
presence, and companionship of the robot coach. Di¬ 
rect comparison among the conditions also favored the 
physical robot significantly. 

73.3.4 Evaluation of SAR Systems 

One of the unique features of SAR is its evaluation 
process. All robotics requires the implementation of 
a physical system and its evaluation in the physical 
world, ideally outside of the laboratory setting, as close 


to the intended final use as possible. In reality, however, 
the majority of research robots are evaluated in labora¬ 
tory settings, because the alternative is prohibitive due 
to cost or access or both. 

HRI puts the human in the loop and therefore 
requires evaluation with human users. Since human 
behavior is complex, it is not readily simulated. Con¬ 
sequently, SAR, which involves HRI, also requires 
evaluation with human users. However, to provide truly 
relevant and realistic evaluation for SAR systems, such 
systems must be tested with the intended populations, 
since testing with healthy adults is not appropriate for 
systems aimed at stroke patients, testing with adults is 
not appropriate for systems aimed at children, and so 
on. 

Consequently, for a SAR approach to be truly eval¬ 
uated, it needs to be tested with members of the in¬ 
tended end-user population, the more rigorously the 
better, according to the principles and best practices 
of human subject evaluations, as well as consistent 
with all the ethical rules associated with such proce¬ 
dures (Sect. 73.1 1). As a result, SAR development and 
evaluation can be time-consuming and requires multi¬ 
disciplinary expertise that goes beyond robotics into 
social science and, as relevant, health science or edu¬ 
cation, among other disciplines. 


73.4 Motivation, Autonomy, and Companionship 


SAR systems equipped with motivational, social, and 
therapeutic capabilities have the potential to enhance 
user skills and quality of life. Motivation is a fun¬ 
damental tool in establishing adherence to a training, 
rehabilitation, or therapy regimen and in promoting be¬ 
havior change; therefore methods aimed at improving 
motivation are likely to be useful in the SAR contexts. 
There are two forms of motivation: intrinsic motivation, 
which comes from within a person, and extrinsic moti¬ 
vation, which comes from sources external to a person. 
Intrinsically motivated activities are those characterized 
by enjoyment [73.44]. Extrinsic motivation, though ef¬ 
fective for short-term task compliance, has been shown 
to be less effective than intrinsic motivation for long¬ 
term task compliance and behavior change [73.45]. 
Intrinsic motivation, however, can be and often is af¬ 
fected by external factors, thereby offering a promising 
avenue for future SAR research. In a task scenario, for 
example, the instructor (or a socially assistive robot) 
can impact the user’s intrinsic motivation through ver¬ 
bal feedback. Praise is considered a form of positive 
feedback and has the potential to increase the user’s 
intrinsic motivation for performing the task, while crit¬ 
icism, a form of negative feedback, tends to negatively 


impact the user’s intrinsic motivation [73.46,47], The 
effect of positive feedback is closely tied to the user’s 
own perceived competence at the task; once the user be¬ 
lieves he is competent at the task, then additional praise 
no longer affects his intrinsic motivation. 

Verbal feedback provided to the user by the instruc¬ 
tor plays an important role in task-based motivation, but 
the task itself and how it is presented perhaps plays an 
even more significant role. Csikszentmihalyi' s research 
suggests that [73.44]: 

when one engages in an optimally challenging 
activity with respect to one’s capacities there is 
a maximal probability for task-involved enjoyment 
or flow. 

Therefore, an instructor that oversees user performance 
in a task scenario should also be aware of the user’s 
current affective state or try to infer that state from task 
performance. The task must be continually adjusted to 
meet the appropriate needs of the user in order to in¬ 
crease or maintain intrinsic motivation to perform the 
task. 

Another task characteristic with the potential to in¬ 
fluence user enjoyment is the incorporation of direct 
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user input. Studies have shown that tasks that support 
user autonomy and self-determination lead to increased 
intrinsic motivation, self-esteem, creativity, and other 
related variables among the participants [73.48], all of 
which are essential for achieving task adherence and 
long term behavior change. Self-determination repre¬ 
sented in the task such as choice of activity [73.49], 


choice of difficulty level [73.50], and choice of re¬ 
wards [73.51] have been shown to increase or be less 
detrimental to intrinsic motivation when compared to 
similar task conditions that do not involve choice. The 
positive impact of self-determination in task scenarios 
has also been supported by studies investigating behav¬ 
ior change [73.52,53]. 


73.5 Influence and the Dynamics of Assistive Interaction 


While social robotics has approached the issues of 
modeling the dynamic interplay of social interactions 
(Chap. 69), assistive interactions are unique among 
these social engagements as they must simultaneously 
support the needs of the engagement itself (by main¬ 
taining interest, novelty, and the conventions of social 
behavior) as well as guide the interaction toward the 
long-term behavioral or educational goals of the sys¬ 
tem. These two goals can often be in conflict; at times 
a good teacher (or coach, or socially assistive robot) 
must sacrifice some of the enjoyment of the interac¬ 
tion, or bend a social rule, in order to promote an 
educational goal. Huang and Mutlu [73.54] recently 
found some evidence that a robot could make some 
of these tradeoffs by changing its nonverbal behavior, 
focusing either on improving task performance (recall 
in a storytelling task) or on improving social engage¬ 
ment by varying the type of gaze behavior used by the 
robot. 

Because the socially assistive robot must take an ac¬ 
tive part in shaping this interaction toward particular 
goals, the nature of the representations and the way in 
which this influence can be applied to shape the dynam¬ 
ics of the assistive interaction represents a core research 
question for SAR. Researchers address questions such 
as: 

• What abstractions allow a robot to model the unique 
dynamics of a social interaction with a user in order 
to shape the user’s behavior? 

• Can we define a set of computational primitives that 
describe social roles, concepts, or properties in such 
interactions? 

• Can this primitive set provide a basis of representa¬ 
tions that allows robots to capture in real time the 
nuances of interaction, to shape and have their be¬ 
havior shaped by that interaction, while allowing 


generalization and comparisons across users, time 

scales, and contexts? 

Much of the research in this area is based upon 
foundational psychological properties of human-hu¬ 
man social interaction including agency, intentionality, 
causality, social roles, and goal attribution. These foun¬ 
dational elements discriminate between agents and ob¬ 
jects, delineate which elements of the world can move 
with goal-directed purpose, and provide the primitive 
structure for describing cause and effect. If we are to 
build socially assistive robots, these systems must see 
the world in the same way that we humans do, in the 
same categories of animate and inanimate, and attribute 
cause and blame in the same way that we do. 

Current computational models represent these fea¬ 
tures explicitly, with users overtly detailing their goals 
and intentions in an operationalized way [73.55]. In 
contrast, people are able to detect and analyze each 
other’s agency and intentions on line, in a way that is of¬ 
ten implicit, automatic, effortless, and even irresistible. 
To be successful, a socially assistive robot must un¬ 
derstand, model, and engage in the dynamics of social 
interaction, so that computational systems can detect, 
analyze, and manipulate agency, intention, and other so¬ 
cial primitives in dynamic environments in a way that 
is akin to how people do the same for social perception 
and behavior. These models must operate over a wide 
range of time scales and across input modalities (visual, 
auditory, tactile, etc.) and must be capable of interpret¬ 
ing and responding to social overtures in real time and 
with only minimal latency. Multiple studies have exam¬ 
ined particular foundational aspects of the dynamics of 
interaction [73.56-58] as well as how persuasion might 
occur within those dynamics [73.59], but a complete 
model of the dynamics of social interaction remains one 
of the key research areas addressed within SAR. 


73.6 Personalization and Adaptation to Specific Needs and Abilities 

A second core research area for SAR concerns the per- and abilities of the user. While existing studies in so- 
sonalization of the robot to the ever changing needs cial robotics often focus on a single matching process 
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by which the behavior of the robot is changed to opti¬ 
mize the properties of the interaction at a single moment 
in time, adaptation and personalization in assistive tasks 
should ideally treat the goal as an ever-changing target. 
As the assistive interaction is often sustained over sub¬ 
stantially longer periods of time than other social robot 
interactions (lasting perhaps months or years), SAR 
systems should assume that the preferences, needs, and 
capabilities of the user change over the course of the 
interaction. Research in this area focuses on questions 
such as: How can we design a robot that adapts to each 
user’s individual social, physical, and cognitive differ¬ 
ences? How can a robot utilize patterns of interactions 
observed with other users (and possibly other robots) to 
bootstrap its own attempts at interaction? How do nov¬ 
elty and continuing personalization in interaction create 
and support trust, learning, and adaptation for improved 
outcomes for its user? 

Socially assistive robots could greatly benefit from 
adapting to each individual user’s unique social, cogni¬ 
tive, and physical abilities. Computationally, the chal¬ 
lenge in constructing such a system stems from the 
individual differences among users and adaptation must 
occur at multiple levels of abstraction and time scales. 
At a given moment, robot responses that one user might 
find frustrating might be optimally engaging to another, 


or even to the same user under different conditions or 
at different times. Their responses should also change 
over time as the user becomes tired, frustrated, or bored. 
Responses must also vary from session to session or 
day to day so that interactions remain engaging and 
interesting. 

Very little research to date addresses learning 
through personalized social interaction [73.60], espe¬ 
cially with populations like children, the elderly, or 
individuals with social and cognitive deficits. Var¬ 
ious learning approaches for HRI have been pro¬ 
posed [73.61], but to date, none have included the user’s 
profile, preferences, and/or personality. The robot’s ca¬ 
pability to adapt to an individual user’s needs and 
abilities must be bounded by the social and cultural 
context and while filling the role of a supportive part¬ 
ner or peer. The majority of current machine learn¬ 
ing methods are focused on learning specific policies 
or parameters for tasks or behaviors. While there is 
a plethora of available learning methods, and some are 
particularly relevant to SAR, such as user state and ac¬ 
tivity modeling [73.62,63], learning by imitation and 
from demonstration [73.64-68], and learning interac¬ 
tion models from crowd-sourced interaction [73.69], 
no work to date has focused on adapting to individual 
differences. 


73.7 Creating Long-Term Engagement and Behaviour Change 


The ultimate objective for any socially assistive robot is 
to alter the long-term behavior of the user in line with 
the behavioral, therapeutic, or educational goals that the 
robot was designed to support. For many of the appli¬ 
cation domains covered by SAR systems, this change 
might require months or years of interaction. A critical 
research issue for SAR is the development of systems 
that can maintain these long-term social interactions, 
monitor the behavioral changes of the user over these 
long-term interactions, and promote activities that sup¬ 
port selective changes in behavior. 

Research questions related to this area include: How 
can a robot continue to be interesting and engaging 
to a user over an extended time period? What social 
behaviors and attributes are needed to establish and sus¬ 
tain trust, rapport, and comfort with the user over time 
to build a successful relationship that continues to pro¬ 
vide value? How can a robot continue to adapt to and 
autonomously guide its user through successive mile¬ 
stones in order to achieve desired learning or behavior 
change outcomes over longitudinal time scales? What 
strategy should a robot employ to motivate and main¬ 
tain lessons or activities, and how should this strategy 
evolve over time? How can the robot recognize when 


its goals related to the behavior change of the user have 
been achieved, or when intermediary goals have been 
achieved? 

To better fill the social role of supportive partners, 
socially assistive robots should adapt their behavior and 
actively planning the trajectory of interactions to guide 
users toward learning or behavioral goals. The ability 
to adapt and plan should be demonstrated at multi¬ 
ple time-scales and abstraction levels so as to enable 
long-term engagement over days, weeks, months, and 
eventually years. Such capability is particularly impor¬ 
tant for special populations who may have different 
responses to techniques and methods that are often em¬ 
ployed with typical adults. Almost no work to date 
has focused on the computational challenges involved 
in creating social interactions that persist over multi¬ 
ple weeks, months, or years. Early efforts in the area 
of life-long learning still operate on short time-scales 
(e.g., two or three cases/tasks) compared to the weeks 
and months that will be necessary for SAR systems. 
This particular goal presents a myriad of interesting 
and challenging applications of complex computational 
methods such as machine learning. Socially assistive 
robots capable of long-term adaptation must constantly 
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sample new interaction and teaching strategies while 
exploiting previously learned preferences of each indi¬ 
vidual user and between users, all in an online setting. 
Extending engagement, social support, and guidance 
to long-term interaction requires more than scaling 


up existing methods and measures. A major scien¬ 
tific goal for SAR research is to develop new insights 
that enable long-term adaptive support and guidance 
to the user in order to sustain a productive assistive 
relationship. 


73.8 SAR for Autism Spectrum Disorder (ASD) Therapy 


Some of the earliest and most extensive investigations 
of SAR have focused on the diagnosis and treatment 
of autism spectrum disorders. Driven by pressing needs 
from a rapidly expanding population and remarkable 
early results, a diverse set of robotics research groups 
have examined the responses of individuals with autism 
spectrum disorders to robots [73.9, 32], 

73.8.1 Summary 

of ASD-Related Challenges 

Autism is a pervasive developmental disorder marked 
by severe deficits in social functioning such as diffi¬ 
culties recognizing body language, making eye contact, 
and understanding the emotions of other people [73.4], 
It is typically associated with a range of cognitive and 
attentional deficits [73.70,71] as well as tendency to 
engage in preservative and self-stimulatory behaviors. 
Expression of particular behaviors is highly hetero¬ 
geneous and associated with notable intra-individual 
variation in the level of functioning in specific domains. 
That is, while some children have exceptional cognitive 
skills, others struggle with major challenges in these 
areas. 

ASD has received a heightened interest from re¬ 
searchers in recent years due to, in part, an unprece¬ 
dented increase in the incidence rate. ASD is estimated 
to affect one in every 60 children in the United States, 
a rate that is roughly eight times higher than the best 
estimates from 20 years ago [73.72], ASD is a behav- 
iorally specified disorder [73.5]; there is no blood test, 
no genetic screening, and no functional imaging test 
that can diagnose ASD. Diagnosis relies on the clini¬ 
cian’s intuitive feel for the child’s social skills includ¬ 
ing eye-to-eye gaze, facial expression, body postures, 
and gestures. These observational judgments are then 
quantified according to standardized protocols [73.6, 
7,73, 74] that are both imprecise and subjective. The 
broad disagreement of clinicians on individual diag¬ 
noses creates difficulties both for selecting appropriate 
treatment for individuals and for reporting the results of 
population-based studies [73.8, 75]. 

ASD is a lifelong disorder; there is no known cure. 
However, early, sustained, and intensive intervention 


using a variety of behavioral techniques have been 
demonstrated to produce substantive gains in many 
individuals with ASD [73.5]. These therapeutic ap¬ 
proaches often focus on traditional behavioral therapies 
in which a skilled therapist guides an individual with 
ASD through a series of repetitive exercises aimed at 
improving daily living and/or functional social skills. 
These techniques require a tremendous amount of la¬ 
bor, and attempts to use technological innovations to 
alleviate some of this burden have been a popular re¬ 
search topic. For example, pet-assisted therapy [73.76, 
77], computer-assisted therapy [73.78-80] and virtual 
reality-based approaches [73.81, 82] have all been stud¬ 
ied. While these have shown some success, there has 
been limited investigation of the parameters of the con¬ 
ditions necessary to generalize the benefits to interac¬ 
tions with human partners. Among nonhuman partners, 
robots that provide instruction, support, and assistance 
through social interaction have been seen as a potential 
mechanism for supporting therapy and daily living for 
individuals with ASD [73.1, 83]. 

Robots promise unique practical advantages over 
other nonhuman interactive partners. First, they can 
potentially provide identical delivery of stimuli, estab¬ 
lishing a uniquely high level of control in diagnosis 
and assessment [73.83]. Second, they do not require 
the months or years of training that animal assistants 
may need, as robots can be designed to allow flexible 
customization. Third, they offer a potential for tac¬ 
tile interface and the immediacy of embodied agency, 
which computer programs and virtual realities cannot 
provide. In cases in which human or animal therapeutic 
aids may be unavailable or are prohibitively expensive, 
and where software or virtual reality therapeutic tools 
cannot provide sufficient embodiment, robots may pro¬ 
vide an especially useful addition to therapy. 

73.8.2 User Study Insights 

Research in this area has shown exciting, but often 
preliminary, benefits to individuals with ASD, includ¬ 
ing increased engagement in tasks, increased levels of 
attention, and novel social behaviors such as joint atten¬ 
tion and spontaneous imitation, when robots are part of 
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Fig.73.2a,b Both highly anthropomorphic robots ((a) Kaspar from 
the University of Hertfordshire) and highly mechanistic robots 
((b) Bubblebot from the University of Southern California) have 
shown increased social responses of children with ASD during in¬ 
teractions with robots 


the interaction [73.83-89]. Much of the early work in 
this domain was exploratory, performed on small sam¬ 
ple sizes (typically fewer than 5 individuals), lacked 
quantitative measurements of success, failed to include 
a control group, and had minimal diagnostic and inclu¬ 
sionary criteria. Given the heterogeneity of ASD, these 
exploratory studies were frequently dismissed by the 
clinical research community. These exploratory studies 
however were instrumental in laying the groundwork 
for the more extensive and detailed studies that are be¬ 
ing performed today [73.9]. 

More recent studies have focused on how robots 
can enhance human-to-human social behavior in chil¬ 
dren with ASD and have included statistically valid 
sample sizes, carefully designed control conditions, and 
extensive diagnostics to accurately describe the popu¬ 
lation being studied. For example, Kim et al. [73.90] 
recently demonstrated a robot that engaged children 
in a vocal prosody training task. Children with ASD 
were more likely to use appropriate inflection in their 
voices when engaging the robot than when interacting 
with a familiar clinician. Furthermore, some children in 
this study demonstrated increased frequency of social 
gaze behaviors to an adult (including social referencing) 
immediately following interaction with a social robot. 
Feil-Seifer and Mataric [73.91] recently demonstrated 
that children with ASD demonstrated an increased so¬ 
cial response to a robot that was socially responsive 
than to one that was not. The study included a com¬ 
parable toy as a control and its results included novel 
insights about child-robot interactions for verbal and 
nonverbal children with ASD (Fig. 73.2,73.3). 

73.8.3 Research Directions 

While these HRI studies offer the hope of a novel ther¬ 
apeutic approach, they have to date neither explained 
why robots provoke these responses in children nor 
demonstrated that a long-term behavioral change in 
children can be generated that generalizes to human- 
human interactions. A number of possible hypotheses 
could explain the increase in social behavior observed 


during and after interactions between an individual 
with ASD and a robot. Perhaps robots offer a more 
predictable interaction, or a more selective and simpli¬ 
fied set of social cues to be interpreted, or are more 
overt in their behavioral responses and easier to engage 
in reciprocal interactions. There is anecdotal evidence 
that some of these may not be a satisfactory explana¬ 
tion [73.83,90], but a robust explanation of why these 
effects are so notable in interactions with individuals 
with ASD has yet to be described. 

Constructing autonomous systems is particularly 
challenging for this population. The diversity of abili¬ 
ties and deficits within the ASD population results in 
vast differences in the rate of interaction, the mode of 
interaction (e.g., verbal versus nonverbal), the social 
skills to be included in the training, and even the criteria 
for what constitutes a successful application of a social 
skill [73.32], While a few studies [73.92] have begun to 
incorporate completely autonomous systems, most are 
still utilizing wizard-of-Oz techniques to deal with the 
variations across users. 

Perhaps the most clinically important research di¬ 
rection is to establish that HRIs provide a long¬ 
term, measurable change in human-human interactions. 
Nearly all of the studies carried out to date (with the 
exception of [73.87]), focus on a small number of short 
interactions (typically spanning a combined time of less 



Fig.73.3a-c Kim et al. [73.89, 90] demonstrated increases in child-adult interactions following a short (5 min) interaction 
with a robot. These images come from video captured before (a), during (b), and after (c) an interaction with a Pleo robot, 
which focused on prosodic production 
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than an hour). As behavioral change even with an ex¬ 
pert clinician can require hundreds of hours of training, 
our research studies must extend to more lasting in¬ 
teractions that span weeks and months. Furthermore, 
most studies to date focus on the development of social 
behaviors directed at the robot. These robot-directed 
behaviors have minimal clinical value unless they can 
be shown to generalize to human-human interactions. 


73.9 SAR Supporting Rehabilitation 

SAR is especially well suited for use in the process 
of rehabilitation, which involves regaining function lost 
due to illness or trauma. Where rehabilitation robotics 
focuses on hands-on HRI, in contrast, SAR focuses on 
providing motivation and coaching in order to facilitate 
the rehabilitation process. Since motivation has long 
been recognized as a major factor in rehabilitation, SAR 
has the potential to play a key role in this health context. 
SAR systems for supporting rehabilitation have been 
developed and tested with stroke patients [73.93,94] 
and Alzheimer’s patients [73.95], and are being consid¬ 
ered for use in traumatic brain injury rehabilitation. 

73.9.1 Summary 

of Rehabilitation Challenges 

Stroke is the leading cause of serious, long-term dis¬ 
ability in the US and the third leading cause of 
death [73.96]. In the US alone, over 800000 people 
suffer a stroke each year, and nearly 400000 survive 
with some form of neurologic disability [73.97]. The 
number of stroke survivors with disability is expected 
to double by the year 2025 [73.98]. Over 80% of first¬ 
time strokes involve loss of function of the upper limb 
that significantly impacts the independence and health 
of the stroke survivor [73.99-101]. The residual disabil¬ 
ity after stroke is substantial, with over 65% of patients 
unable to incorporate the affected hand effectively into 
daily activities. Functional recovery of the arm and 
hand has generally been resistant to the traditional ap¬ 
proaches of stroke rehabilitation compared with that for 
the lower extremities program [73.99, 102]. 

Stroke often leaves individuals unable to perform 
movements with the affected limb even though the 
limb is not completely paralyzed. This loss of function, 
termed learned disuse, can improve with rehabilita¬ 
tion therapy [73.103, 104]. Clinical studies using motor 
training have found improvement in functional upper 
limb performance in patients more than one year post 
stroke and cortical reorganization and recruitment of 
adjacent brain areas associated with intensive use of 


(One exception is a the studies by Kim et al. [73.89, 
90] which demonstrated that children with ASD show 
a significant improvement in face-to-face interactions 
with an adult immediately following an interaction with 
a robot tutor). To provide therapeutic value to an indi¬ 
vidual with ASD, interactive robots must provide either 
a measurable benefit to their ability to engage with other 
people. 


the affected upper limb have been documented sev¬ 
eral years after the initial stroke injury [73.105-107]. 
Repetitive, intense, task-specific practice is effective 
in rehabilitation [73.108-111]. Indeed, practice is fun¬ 
damental to motor learning - it is the single most 
important variable for skill acquisition. However, such 
practice requires trained coaching and motivation. 

73.9.2 Contact Versus SAR (Noncontact) 
Robotics for Rehabilitation 

SAR creates a bridge between rehabilitation robotics, 
which has historically been contact-based [73.112, 
113], with noncontact functionalities, such as com¬ 
panion robotics [73.114,115]. The noncontact ap¬ 
proach defines a key niche for SAR that involves 
an autonomous physically embodied robot providing 
contact-free monitoring, coaching, and encouragement, 
while also providing detailed assessments of client 
progress to key stakeholders including and importantly 
the user. Unlike rehabilitation robotics, where the appli¬ 
cation and perception of forces are the main focus, and 
which are consequently currently limited to in-clinic 
settings, work with noncontact robots is instead directed 
at the interaction modalities and motivation strategies 
for achieving specific goals in the home and commu¬ 
nity settings. 

One of the most important elements of any reha¬ 
bilitation program is the practice of carefully directed, 
well-focused and repetitive exercises, which can be 
passive or active. In passive exercises (also known as 
hands-on rehabilitation), the patient is helped by a hu¬ 
man (or robotic) therapist to appropriately exercise the 
affected limb(s). In contrast, in active exercises, the pa¬ 
tient performs the exercises with no physical assistance. 
The majority of existing work in rehabilitation robotics 
focuses on hands-on robotic systems for passive ex¬ 
ercise, focusing on attempting to recover upper-limb 
function primarily through robotic manipulation of the 
affected-limb. Burgare t al. [73.1 16] developed a robot- 
assisted arm therapy workstation, in which patients 
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can exercise their upper limbs and evaluate their per¬ 
formance. A similar device, which also depends on 
hands-on robotic technology, was developed by Krebs 
et al. [73.117]. Other related systems have been inves¬ 
tigated [73.118] and [73.119], Because human-robot 
contact involves complex issues of safety, such hands- 
on robotics methods remain areas of active ongoing 
research, with many outstanding challenges. Currently, 
contact rehabilitation robots are not portable; the logis¬ 
tics of contact robot therapy are further limited by cost, 
availability of adequate amount of practice, expertise 
necessary to program and execute trials, and liability. 
Risk of injury to the patient is of concern when move¬ 
ment of a limb with sensorimotor loss is imposed by 
a robot [73.120, 121]. Finally, intense functional exer¬ 
cise is more likely to sustain the behavior (upper limb 
functional use) in the long term in contrast to the decline 
in long-term performance with contact robot systems 
after therapy has ceased [73.120-122]. 

The SAR system acts as a coach, providing encour¬ 
agement, and direction, and ensuring proper adherence 
with the therapy regimen. Given the size of the stroke- 
affected population and the time-intensive personalized 
requirements, such coaching is not possible with hu¬ 
man therapists, creating a niche for technological means 
of filling the gap. By providing the opportunity for 
time-extended monitoring and encouragement of reha¬ 
bilitation exercises in any setting (at the clinic or at 
home), SAR systems complement human care [73.1 23— 
126], 

73.9.3 User Study Insights 

This section reviews some of the insights from 
SAR studies performed with stroke patients. One 
study [73.93, 127] focused on how different robot be¬ 
haviors may affect the patient’s willingness to adhere 
to the rehabilitation program. The study evaluated dif¬ 
ferent voices, movements, and levels of patience on 
the part of the robot, and correlated those with patient 
adherence. The robot was equipped with navigation 
capabilities, so it could find and follow the patient, ma¬ 
neuver itself to an appropriate position for monitoring 
the patient, and leave when it was not wanted. The robot 
acquired real-time information about the patient’s use 
of the stroke-affected limb from inertial measurement 
unit (IMU)-based sensors worn by the patient. Wear¬ 
able sensors present an important alternative to machine 
vision, which may not be acceptable to all users due 
to privacy concerns. The robot used information pro¬ 
vided by the wearable sensors to encourage the patient 
to continue using the limb, or to use the limb more or in 
a different way, as appropriate. The robot was well re¬ 
ceived by the patients. Some continued to perform the 


activity beyond the end of the experiment, providing 
further evidence of improved adherence well beyond 
any novelty effects. As expected, there were signifi¬ 
cant personality differences among the patients; some 
were highly adherent but appeared un-engaged by the 
robot, while others were highly engaged and even en¬ 
tertained, but preferred to interact with the robot rather 
than perform the coached exercises. This leads toward 
interesting research questions of how to define adaptive 
robot-assisted rehabilitation protocols that will serve 
the diverse user population as well as the time-extended 
and evolving needs of every individual user. 

Modeling personality in order to customize HRI has 
a particular relevance to stroke rehabilitation because 
pre-stroke personality has a great influence on post¬ 
stroke recovery [73.96]; individuals classified as extro¬ 
verted before the stroke mobilize their strength more 
easily to recover than do introverted subjects [73.128]. 
Further, work in human-computer interaction (F1CI) 
has demonstrated the similarity-attraction principle, 
which posits that individuals are more attracted to oth¬ 
ers manifesting the same personality as theirs [73.129- 
131]. Tapus and Mataric [73.132] performed a SAR 
study to test related effects. In a series of experiments 
with a simple mobile robot equipped with a camera and 
a microphone, participants were asked to perform four 
tasks (designed as functional activities) similar to those 
used during standard stroke rehabilitation. The partici¬ 
pants completed a personality assessment questionnaire 
before the experiment; the resulting personality based 
specifically on the extroversion-introversion dimension 
was used to determine the robot’s personality. The be¬ 
havior control architecture was based on the model of 
reciprocal influences on behavior [73.133]. The robot 
expressed its personality through several means: (1) 
proxemics (social use of space; extroverted personali¬ 
ties used smaller personal distances) [73.134]; (2) speed 
and amount of movement (extroverted personalities 
moved more and faster); and (3) verbal content (extro¬ 
verted personalities talked more assertively ( You have 
done only x movements, I’m sure you can do more!), 
using a challenge-based style compared to a nurture- 
based style (7 know it’s hard, but remember it’s for 
your own good.). The experiment compared person¬ 
ality-matched versus personality-mismatched (random) 
conditions. The results showed that the robot’s person¬ 
ality was fundamental to the quality of the interaction. 
Two statistically significant results were found: (1) 
participants consistently performed better on the task 
when interacting with the personality-matched robot; 
(2) participants reported preferring the personality- 
matched robot. Tapus et al. [73.94] then expanded on 
this work by enabling the SAR system to adapt its per¬ 
sonality based on the past interactions with the user. 
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Fig. 73 . 4 a,b Seated exercise coaching with a socially assistive 
humanoid robot on wheels, (a) Bandit, from the University of 
Southern California, and (b) upper extremity rehabilitation manip- 
ulandum 


Formulated as policy gradient reinforcement learning 
(PGRL), the adaptation allowed for optimizing the 
interaction parameters: interaction distance/proxemics, 
speed, and verbal content (what the robot says and 
how it says it). The learning algorithm was initial¬ 
ized with parameter values that were in the vicinity 
of what was thought to be acceptable for both ex¬ 
troverted and introverted individuals, based on the 
user-robot personality matching study described ear¬ 
lier. Gradual adaptation around a solution rather than 
random probing of the solution space is important; if 
the robot’s initial personality is far off the mark, or 
if it changes drastically, users are dissatisfied with the 
interaction. The results provided evidence for the effec¬ 
tiveness of robot behavior adaptation to user personality 
and performance: users tended to perform more or 
longer trials under the personality matched and therapy 
style-matched conditions (nurturing styles were corre¬ 
lated with the introversion, and challenging styles with 
extroversion). 

Another study with stroke patients compared 
matched humanoid robot and simulated agent imple¬ 
mentation of the SAR coach. The study participants 
were seated at a desk, and were asked by the robot 
to move magazines from the lower to the higher rack 
and to de-shelf as many magazines as possible in 
6 min. The robot monitored the participant’s motions 
using a revised version of the motion sensors described 
above [73.135,136]. The participants could use the but¬ 
ton on a WiiMote to stop the interaction at any time. The 
IMU data and shelf state (using a scale) were used to 
classify the movement into states of progress (e.g., rest¬ 
ing, picking up magazine, lifting arm, shelving book, 
lowering arm). In addition to the verbalizations, the 
robot used beat gestures whenever it spoke. This con¬ 
sisted of moving the head, arms, and hands in a beat 
with amplitude and frequency governed by the charac¬ 
teristics of the phrase that was being uttered. Baseline 


movement data were assessed using standard stroke 
movement tests and task improvement was measured 
after repeated sessions. The experimental design in¬ 
volved all participants experimenting with both the 
robot and the computer. Two-thirds of the participants 
preferred the physical robot, with one third having no 
preference. 

73.9.4 Research Directions 

SAR for rehabilitation presents multi-faceted research 
challenges. 

Personalizing the interaction to adapt to the user’s 
changing state over multiple timescales remains a ma¬ 
jor challenge, since rehabilitation aims for long-term 
improvement, but must contend with mood, fatigue, 
and performance variations within each session, as well 
as short term trends that may plateau or even reverse. 
Personalizing the interaction thus requires new ways 
of integrating robot learning, user modelling, and mo¬ 
tivation strategies, among many other SAR research 
challenges. Creating robotic systems capable of adapt¬ 
ing their behavior in order to provide an engaging and 
motivating customized protocol is a challenging target, 
especially when working with vulnerable user popu¬ 
lations. In the SAR context, behavior adaptation must 
address both short-term changes that represent individ¬ 
ual differences and long-term changes that allow the 
interaction to continue to be engaging over a period of 
months and even years. Various learning approaches for 
HRI have been proposed in the literature [73.61, 65] but 
this remains a major challenge in robotics in general 
and in SAR in particular. 

Modeling the user and predicting user behavior is 
a challenge common to HRI and HCI, as well as re¬ 
search in signal processing and related fields. However, 
SAR also brings up the issue of modelling the identity 
of the robot character. Specifically, studies in SAR are 
beginning to show that having not only a personality 
but also a back story for the robot facilitates user en¬ 
gagement. 

Research into relational agents [73.137] has been 
applied to short-term interactions in medical settings, 
such as hospital checkout procedures. Some of the 
insights from that field, including politeness theory, 
empathy, etc., are being explored in SAR for use in 
long-term interactions involved in rehabilitation. 

Finally, one of the much needed new directions of 
research is the exploration of bringing together contact 
and noncontact rehabilitation robotics in relevant do¬ 
mains, in order to enhance the impact of both robotics 
subfields. 
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73.10 SAR and Eldercare 

73.10.1 Summary of Eldercare Challenges 

The growing population of aging adults is increasing the 
demand for healthcare services worldwide. By the year 
2050, the number of people over the age of 85 will in¬ 
crease threefold [73.138], while the shortfall of nurses 
and caregivers is already an issue [73.139-141], Regu¬ 
lar physical exercise has been shown to be effective at 
maintaining and improving overall health [73.1 42- 145]. 
Physical fitness is associated with higher functioning in 
the executive control processes [73.146] and is corre¬ 
lated with less atrophy of frontal cortex regions [73. 147] 
and with improved reaction times [73.148]. Social in¬ 
teraction, and specifically high perceived interpersonal 
social support, has also been shown to have a pos¬ 
itive impact on general mental and physical wellbe¬ 
ing [73.149], in addition to reducing the likelihood of 
depression [73.150-153]. Thus, the availability of phys¬ 
ical exercise therapy, social interaction, and companion¬ 
ship will be critical for the growing elderly population. 

The probability for acquired motor, cognitive, 
and perceptual disabilities dramatically increases with 
age [73.154]. Over 50 million Americans have a dis¬ 
ability that affects one or more of their major life activi¬ 
ties [73.155, 156] and only 30% are employed [73.157]. 
The economic cost of the associated loss of function¬ 
ing is high; e.g., the cumulative total of stroke-affected 
Americans is over 4 million, and the estimated annual 
burden from stroke-related disability is $53.6 billion, of 
which $20.6 billion is in indirect costs due to lost pro¬ 
ductivity and income [73.96, 158]. Given projections 
that the number of people older than 65 years is antici¬ 
pated to double between 1997 and 2025 [73.156] and 
that 30% of the population will be over 65 [73.159, 
160], we will soon face new major technical and so¬ 
cial challenges to attain, prolong, and preserve quality 
of life. 

A especially vulnerable and sizeable population 
consists of individuals who are aging with disabili¬ 
ties that were acquired at any stage of life, from birth 
or early in life to stroke or other disorders later in 
life [73.161]. The number of middle aged and older 
adults living with disabilities will grow significantly as 
the U.S. population ages rapidly [73.162]. People who 
have managed their lives very successfully for years 
may find that they are experiencing the effects of aging 
earlier than others or are developing secondary health 
conditions as they reach their 30s or 40s [73.163, 164]. 
As persons with disabilities age, progressive declines 
in health and medical status can challenge the adaptive 
resources required to maintain functional independence 
and quality of life [73.165,166]. 


Motor and cognitive dysfunctions that accompany 
chronic illness and disability in late life are major 
precursors for people who age into disability. Cognitive 
impairments are associated with increased disability 
in aging populations [73.167]; 19% of women and 
13% of men aged 65—69 live with a disability in 
mobility [73.168]. The respective percentages in¬ 
crease to 83% and 63% for women and men aged 
90—95 [73.168]. These dysfunctions greatly compound 
the disruptions in overall lifestyle and occupational sta¬ 
tus experienced by individuals aging with disabilities 
primarily associated with cognitive or motor deficits. 
Thus, there is a critical need to develop effective and 
affordable assistive technologies to improve the motor 
and cognitive functioning, especially in individuals 
aging with disabilities. This creates an important niche 
for SAR. 

73.10.2 User Study Insights 

The literature that addresses assistive robotics in¬ 
tended for and evaluated by the elderly is limited but 
growing. 

Researchers have investigated the use of robots to 
help address the social and emotional needs of the 
elderly, including reducing depression and increasing 
social interaction with peers. Wada et al. [73.114] stud¬ 
ied the psychological effects of a seal robot, Paro, used 
to engage seniors at a day service center. The study 
found that Paro, always accompanied by a human han¬ 
dler, was able to consistently improve the moods of 
elderly participants who spent time petting and engag¬ 
ing with it over the course of a 6-week period. Kidd 
et al. [73.1 15] used Paro in another study that found it 
to be useful as a catalyst for social interaction. They 
observed that seniors who participated with the robot 
in a group were more likely to interact socially with 
each other when the robot was present and powered on, 
rather than when it was powered off or absent. 

Social agents that aim to assist individuals in health- 
related tasks such as physical exercise have also been 
developed in the HCI community. Bickmore and Pi¬ 
card [73.137] developed a computer-based virtual re¬ 
lational agent that served as a daily exercise advisor 
by engaging the user in conversation and providing 
educational information about walking for exercise, 
asking about the user’s daily activity levels, tracking 
user progress over time while giving feedback, and en¬ 
gaging the user in relational dialogue. 

Fasola and Mat arid [73.169] conducted a study to 
investigate the role of praise and relational discourse 
(politeness, humor, empathy, etc.) in a SAR exercise 
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Fig.73.5a-c Paro robot (a) from the University of Tokyo, Autom robot (b) used in an MIT weight-loss study, and 
Healthbot robot (c) from the University of Auckland used in a large-scale elderly care study 


system designed for older adults. The study compared 
a relational robot versus a nonrelational robot. The re¬ 
lational robot employed praise and relational discourse, 
while the nonrelational robot guided the exercise in¬ 
teraction without such praise or discourse. Results 
showed participants rated the relational robot signifi¬ 
cantly more positively than the nonrelational robot in 
terms of enjoyableness and usefulness of the interac¬ 
tion, and as a companion and exercise coach. Fasola 
and Mcitaric [73.169] also conducted a user study to in¬ 
vestigate the role of choice and user autonomy within 


the same SAR exercise system. The results of the study 
showed no clear preference for one condition over the 
other, as the user enjoyment level of the interaction was 
reported to be equally high for choice and no-choice 
conditions. This suggests that a hybrid approach that 
includes both user and robot decision making, person¬ 
alized and tuned automatically for each user, might 
ultimately be the best solution for achieving a fluid and 
enjoyable task interaction for all users. This once again 
underscores the need for personalized methods of HRI 
and user modeling and adaptation. 


73.11 SAR for Alzheimer's Dementia and Cognitive Rehabilitation 


Dementia is a progressive disabling neurological condi¬ 
tion that can be a symptom of several diseases, includ¬ 
ing Alzheimer’s, which makes up about half of all cases 
of dementia. The latest estimate is that 26.6 million peo¬ 
ple were suffering from Alzheimer’s disease worldwide 
in 2006, and it will rise to 100 million by 2050 - 1 
in 85 of the total population [73.170]. More than 40% 
of those cases will be in late-stage Alzheimer’s, requir¬ 
ing a high level of attention equivalent to nursing home 
care. Individuals suffering from Alzheimer’s experience 
a decrease in verbal and cognitive skills along with 
the deterioration of strength and flexibility that comes 
with aging. Rehabilitation therapies for individuals with 
dementia often focus on both cognitive and motor exer¬ 
cises to in order to preserve patients’ autonomy for as 
long as possible. 

Therapy for individuals with dementia must be per¬ 
vasive and ongoing in their everyday lives. Mental 
and physical exercise involving art, music, and playing 
with/tending to animals have been used to preserve cog¬ 
nitive function in older adults who have begun experi¬ 
ences symptoms of dementia. However, these therapies 
can only be offered by well-trained personnel. In par¬ 
ticular, animal therapy can involve safety risks for both 
the animal and the patient. The goal of SAR in this ther¬ 


apy context is to provide a means of providing therapy 
that is safe, easy to use, and does not require special 
training. 

73.11.1 User Study Insights 

In a study by Tamura et al. [73.171], an entertain¬ 
ment robot was used as occupational therapy instead 
of animal-assisted therapy to avoid any danger or in¬ 
jury to the patient and maintain cleanliness. The study 
compared the effectiveness of a robot animal, AIBO, 
with a toy. AIBO responded to spoken commands and 
was demonstrated to severely demented elderly people 
living in a geriatric home. The most frequent reactions 
to AIBO consisted of looking at, communicating with, 
and caring for the robot. The patients recognized that 
AIBO was a robot but, once the robot was addressed 
by the experimenters, the patients perceived it as either 
a dog or a baby. The presentation resulted in positive 
outcomes for the severe dementia patients, including 
increased communication between the patients and the 
robot. 

Tapus et al. [73.172] performed a long-term (6- 
month) study of socially assistive robots in the context 
of cognitive training for Alzheimer’s patients to test if 
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elderly individuals with Alzheimer’s impairments can 
maintain attention to music with the help of a SAR sys¬ 
tem. A small group of four participants over 70 years 
old, suffering from mild to severe cognitive impair¬ 
ment and/or Alzheimer’s disease, took part in the study 
and received per- and post-study cognitive assessments. 
Each interacted in one-on-one twice-weekly sessions 
with the robot, in which the robot presented a music 
recognition game with three different levels of chal¬ 
lenge: difficult (no hints), medium (hints given), and 
easy (name of song given). The challenge level was ad¬ 
justed to each participant based on performance. The 
results obtained over 6 months of robot interaction (ex¬ 
cluding the 2 months of learning) suggest that elderly 
people suffering of dementia and/or Alzheimer’s can 
sustain attention to music across a long period of time 
and even improve on the music recognition task when 
coached by a socially assistive robot; the SAR system 
was able to adapt the challenge level of the game it 
was presenting to the user in order to encourage task 
improvement and attention training. Importantly, the 
participants demonstrated fondness for the robot and in¬ 
cluded it into their daily narratives (e.g., My buddy is 
coming to visit, I don’t want to miss it.). 

The multiyear Healthbots project at the Univer¬ 
sity of Auckland is examining the role of telepres¬ 
ence and humanoid robots in assisted living communi¬ 


ties [73.173]. This large study included three different 
types of robots, ranging from large telepresence to 
small humanoid, which were deployed as companions 
and nursing assistants in a retirement village of around 
650 residents. Several insights were gained from the 
long-term deployment, including the preferences of 
both the elderly users and staff members. The most 
promising avenue for robotic assistance was a robot that 
was sent into residents’ rooms to remind them to take 
their medications or fetch them for appointments. 

73.11.2 Research Directions 

As with all areas of SAR, the user’s willingness to en¬ 
gage with a socially assistive robot to accept advice, 
interact, and ultimately alter behavior depends on the 
robot’s ability to obtain the user’s trust and sustain the 
user’s interest. User interfaces and input devices that are 
easy and intuitive for a range of users, including those 
with special needs, need to be developed. Social inter¬ 
action is inherently bidirectional and thus involves both 
multimodal perception and communication, including 
verbal and nonverbal means. Toward that end, auto¬ 
mated behavior detection and classification as well as 
activity recognition, including user intent, task-specific 
attention, and failure recognition, are critical enabling 
components being developed. 


73.12 Ethical and Safety Considerations 


SAR, with its focus on developing human-robot re¬ 
lationships and influencing human behavior, naturally 
brings about several distinct types of ethical considera¬ 
tions. 

Safety of the user is the first and most obvious 
consideration in any area of HRI. Because SAR sys¬ 
tems do not involve physical contact initiated by either 
people or robots, they present a minimum of physical 
safety issues. However, while physical safety is one of 
the largest concerns in other areas of robotics research 
(Chap. 30), in SAR, emotional and other types of non¬ 
physical safety must be considered. 

As SAR research strives to achieve user engagement 
and attachment, some critics have argued that such rela¬ 
tionships should be reserved for humans only [73.174]. 
Evaluations of specific SAR systems have triggered eth¬ 
ical dilemmas [73.174,175], Turkle [73.174] demon¬ 
strated that some participants interacting with robots 
can correctly identify the robot’s intended emotional 
abilities and operational capabilities. These participants 
could also correctly distinguish equivalent capabilities 
in a person, pet, or other relational artifact. However, it 


was also demonstrated that some users formed attach¬ 
ments and emotional bonds with robots they were inter¬ 
acting with. These attachments led to misconceptions 
about the robots’ emotional capabilities. For example, 
one user felt that the robot would miss him when he was 
gone, something that the robot was not capable of do¬ 
ing. Sharkey and Sharkey [73.175] argue strongly that 
such attachments in children could lead to malformed 
development and emotional problems. Some such ar¬ 
guments are based on the notion that human care is 
replaced by robot care, which is not consistent with the 
premise of most of SAR research. Most SAR research 
clearly state that their aim is to augment rather than 
replace human care, as discussed in Sect. 73.7 of this 
chapter, in the context of SAR for autism spectrum dis¬ 
orders. 

Some more pragmatic ethical considerations take 
into account the inevitable progress of technology, 
which will render any particular system obsolete well 
within a user’s lifetime, therefore undermining attach¬ 
ment and likely making long-term system operation 
impossible. Since long-term robot system operation is 
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a goal of the majority of robotics research, this is not 
yet a pressing ethical concern, but is one that is well 
recognized as important. 

As one example of ethics applied to SAR in 
particular, Feil-Seifer and Mataric [73.176] outlines 
the ethical issues of SAR around the core principles 
of ethics applied to human subjects, namely benef¬ 
icence, nonmaleficence, autonomy and justice. The 
first two principles encompass the SAR issues of re¬ 
lationships, authority and attachment, perception and 
personifications of the robot, and replacement of hu- 

Refe rences 


man care/changes to human-human interaction. The 
third principle, autonomy, spans the issues of privacy, 
choice, and intentional user deception. Finally, jus¬ 
tice spans the complex issues of cost/benefit analysis, 
and locus of responsibility in the case of failure or 
harm. 

As robots get closer to people, ethical issues be¬ 
come more pressing. It is a good sign that so early in 
the development of SAR those issues are being actively 
discussed and considered, toward the development of 
most ethically informed methods and systems. 
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This chapter surveys the main approaches devel¬ 
oped to date to endow robots with the ability 
to learn from human guidance. The field is best 
known as robot programming by demonstration, 
robot learning from/by demonstration, appren¬ 
ticeship learning and imitation learning. We start 
with a brief historical overview of the field. We 
then summarize the various approaches taken 
to solve four main questions: when, what, who 
and when to imitate. We emphasize the im¬ 
portance of choosing well the interface and the 
channels used to convey the demonstrations, 
with an eye on interfaces providing force control 
and force feedback. We then review algorith¬ 
mic approaches to model skills individually and 
as a compound and algorithms that combine 
learning from human guidance with reinforce¬ 
ment learning. We close with a look on the use 
of language to guide teaching and a list of open 
issues. 
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74.1 Learning of Robots 

Robot learning from humans relates to situations in 
which the robot learns from interacting with a human. 
This must be contrasted to the vast body of work on 
robot learning where the robot learns on its own , that is, 
through trial and error and without external guidance. 
In this chapter, we cover works that combine reinforce¬ 
ment learning (RL) with techniques that use human 
guidance, e.g., to bootstrap the search in RL. However, 
we exclude from this survey all works that use purely 
reinforcement learning, even though one could argue 
that providing a reward is one form of human guid¬ 
ance. We consider that providing a reward function is 
akin to providing an objective function and hence re¬ 
fer the reader to the companion chapter on Machine 


Learning for robotics. We also exclude works where 
the robot learns implicitly from being in presence of 
a human, while the human is not actively coaching the 
robot, as these works are covered in the companion 
chapter on Social Robotics. We hence focus our survey 
to all works where the human is actively teaching the 
robot, by providing demonstrations of how to perform 
the task. 

Various terminologies have been used to refer to this 
body of work. These include programming by demon¬ 
stration (PbD), learning from human demonstration 
(LfD), imitation learning, and apprenticeship learning. 
All of these refer to a general paradigm for enabling 
robots to autonomously perform new tasks from ob- 
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serving and learning, therefore, from the observation of 
humans performing these tasks. 

74.1.1 Principle 


view of the main approaches to solving LfD and con¬ 
clude with an outlook on open issues. 

74.1.2 Brief History 


Rather than requiring users to analytically decompose 
and manually program a desired behavior, work in LfD- 
PbD takes the view that an appropriate robot controller 
can be derived from observations of a human’s own per¬ 
formance thereof. The aim is for robot capabilities to be 
more easily extended and adapted to novel situations, 
even by users without programming ability: 

The main principle of robot learning from demon¬ 
stration is that end-users can teach robots new tasks 
without programming. 

Consider a household robot capable of performing 
manipulation tasks. One task that an end-user may de¬ 
sire the robot to perform is to prepare a meal, such 
as preparing an orange juice for breakfast (Fig. 74.1 
and l<Bf ). Doing so may involve multiple sub¬ 
tasks, such as juicing the orange, throwing the rest of 
the orange in the trash, and pouring the liquid into a cup. 
Further, every time this meal is prepared, the robot will 
need to adapt its motion to the fact that the location and 
type object (cup, juicer) may change. 

In a traditional programming scenario, a human pro¬ 
grammer would have to code a robot controller that is 
capable of responding to any situation the robot may 
face. The overall task may need to be broken down into 
tens or hundreds of smaller steps, and each one of these 
steps should be tested for robustness prior to the robot 
leaving the factory. If and when failures occur in the 
field, highly-skilled technicians would need to be dis¬ 
patched to update the system for the new circumstances. 
Instead, LfD allows the end-user to program the robot 
simply by showing it how to perform the task - no cod¬ 
ing is required. Then, when failures occur, the end-user 
only needs to provide more demonstrations, rather than 
calling for professional help. LfD hence seeks to endow 
robots with the ability to learn what it means to per¬ 
form a task by generalizing from several observations 
(Fig. 74.1 and l<s>M3HSEB). 

LfD is not a record and play technique. LfD implies 
learning, henceforth, generalization. 

Next, we give a brief historical overview of the way 
the field evolved over the years. This is followed, in 
Sect. 74.2, by an introduction to the issues at the core 
of LfD. In Sect. 74.3, we discuss the crucial role that 
the interface used for LfD plays in the success of the 
teaching, emphasizing how the choice of interface de¬ 
termines the type of information that can be conveyed 
to the robot. Finally, in Sect. 74.4, we give a generic 


Robot learning from demonstration started in the 1980s. 
Then, and still to a large extent now, robots had to be 
explicitly and tediously hand programmed for each task 
they had to perform. PbD sought to minimize, or even 
eliminate, this difficult step. 

The rationale for moving from purely prepro¬ 
grammed robots to very flexible user-based interfaces 
for training the robot to perform a task is threefold. First 
and foremost, PbD is a powerful mechanism for reduc¬ 
ing the complexity of search spaces for learning. When 
observing either good or bad examples, one can reduce 
the search for a possible solution, by either starting the 
search from the observed good solution (local optima), 
or conversely, by eliminating from the search space 
what is known as a bad solution. Imitation learning is, 
thus, a powerful tool for enhancing and accelerating 
learning in both animals and artifacts. 



Fig. 74.1 (a) The teacher does several demonstrations of 
the task of juicing an orange, by changing the location of 
each item to allow the robot to generalize correctly. That is, 
the robot should be able to infer, by comparing the demon¬ 
strations, that only the relative locations matter, as opposed 
to the exact locations as recorded from a global coordinate 
system. (b) The robot can then reproduce the task even 
when the objects are located in positions not seen in the 
demonstrations 
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Second, imitation learning offers an implicit means 
of training a machine, such that explicit and tedious pro¬ 
gramming of a task by a human user can be minimized 
or eliminated. Imitation learning is thus a natural means 
of interacting with a machine that would be accessible 
to lay people. 

Third, studying and modeling the coupling of per¬ 
ception and action, which is at the core of imitation 
learning, helps us to understand the mechanisms by 
which the self-organization of perception and action 
could arise during development. The reciprocal inter¬ 
action of perception and action could explain how 
competence in motor control can be grounded in the 
rich structure of perceptual variables, and vice versa, 
how the processes of perception can develop as means 
to create successful actions. 

PbD promises were thus multiple. On the one hand, 
one hoped that it would make the learning faster, in con¬ 
trast to trial-and-error methods trying to learn the skill 
tabula rasa. On the other hand, one expected that being 
user-friendly, the methods would enhance the applica¬ 
tion of robots in human daily environments. 

At the beginning of the 1980s, LfD, known then as 
programming by demonstration (PbD), started attract¬ 
ing attention in manufacturing robotics. PbD appeared 
as a promising route to automate the tedious manual 
programming of robots, reducing the costs involved in 
the development and maintenance of robots in the fac¬ 
tory. 

As a first approach in PbD, symbolic reasoning 
was commonly adopted in robotics [74.1-5], with 
processes referred to as teach-in, guiding, or play¬ 
back methods. In these works, PbD was performed 
through manual (teleoperated) control. The position of 
the end-effector and the forces applied on the object 
manipulated were stored throughout the demonstra¬ 
tions together with the positions and orientations of 
the obstacles and of the target. This sensorimotor in¬ 
formation was then segmented into discrete subgoals 
(key points along the trajectory) and into appropri¬ 
ate pre-defined actions to attain these subgoals. Ac¬ 
tions were commonly chosen to be simple point-to- 
point movements that industrial robots employed at 
this time. Examples of subgoals would be, e.g.. the 
robot’s gripper orientation and position in relation to the 
goal [74.3]. Consequently, the demonstrated task was 
segmented into a sequence of state-action-state transi¬ 
tions. 

To take into account the variability of human mo¬ 
tion and the noise inherent to the sensors capturing the 
movements, it appeared necessary to develop a method 
that would consolidate all demonstrated movements. 
For this purpose, the state-action-state sequence was 
converted into symbolic if-then rules, describing the 


states and the actions according to symbolic relation¬ 
ships, such as in contact, close-to, move-to, grasp- 
object, move-above, etc. Appropriate numerical defi¬ 
nitions of these symbols (i. e., when would an object 
be considered as close-to or far-from ) were given as 
prior knowledge to the system. A complete demonstra¬ 
tion was thus encoded in a graph-based representation, 
where each state constituted a graph node and each 
action a directed link between two nodes. Symbolic 
reasoning could then unify different graphical repre¬ 
sentations for the same task by merging and deleting 
nodes [74.2], 

Munch et al. [74.6] suggested the use of machine 
learning (ML) techniques to recognize elementary op¬ 
erators (EOs), thus defining a discrete set of basic motor 
skills, with industrial robotics applications in mind. In 
this early work, the authors already established several 
key issues of PbD in robotics. These include questions 
such as how to generalize a task, how to reproduce 
a skill in a completely novel situation, how to evaluate 
a reproduction attempt, and how to better define the role 
of the user during learning. Munch et al. [74.6] admit¬ 
ted that generalizing over a sequence of discrete actions 
was only one part of the problem since the controller 
of the robot also required the learning of continuous 
trajectories to control the actuators. They proposed to 
overcome the missing parts of the learning process by 
leveraging them to the user, who took an active role in 
the teaching process. 

These early works highlighted the importance of 
providing a set of examples that are usable by the robot: 
(1) by constraining the demonstrations to modalities 
that the robot can understand; and (2) by providing 
a sufficient number of examples to achieve a desired 
generality. They noted the importance of providing an 
adaptive controller to reproduce the task in new sit¬ 
uations, that is, how to adjust an already acquired 
program. The evaluation of a reproduction attempt was 
also leveraged to the user by letting him/her provide 
additional examples of the skill in the regions of the 
learning space that had not been covered yet. In this 
way, the teacher/expert could control the generalization 
capabilities of the robot. 

With the increasing development of mobile and 
humanoid robots, the field went on adopting an inter¬ 
disciplinary approach, taking into account evidence of 
specific neural mechanisms for visuomotor imitation 
in primates [74.7-9] and of developmental stages of 
imitation capacities in children [74.10,11]. The latter 
promotes the introduction of socially driven behavior 
in the robot to sustain interaction and improve teach¬ 
ing [74.12,13] and of an interactive teaching process, 
in which the robot takes a more active role and may 
ask the user for additional sources of information, when 
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needed [74.14,15]. Eventually, the notion of robot pro¬ 
gramming by demonstration was replaced by the more 
biological labeling of imitation learning. In essence, 
a large part of current works in PbD follow a conceptual 
approach very similar to that followed by these prior 
works. 

Recent progress affected mostly the interfaces 
at the basis of the teaching. Traditional ways of 
guiding/teleoperating the robot have been progres¬ 
sively replaced by more user-friendly interfaces, such 
as vision [74.16,17], speech command [74.18], data 
gloves [74.19], the laser range finder [74.20] or kines¬ 
thetic teaching (i. e., by manually guiding the robot’s 
arms through the motion) [74.21-23]. 

The held progressively moved from simply copying 
the demonstrated movements to generalizing across sets 
of demonstrations. As machine learning progressed, 
PbD started incorporating more of those tools to tackle 
both the perception issue, i. e., how to generalize across 


demonstrations, and the production issue, i. e., how to 
generalize the movement to new situations. Initially, 
tools such as artificial neural networks (ANNs) [74.24, 
25], radial-basis function networks (RBFs) [74.26], 
and fuzzy logic [74.27] were quite popular. These 
have lately been replaced by hidden Markov models 
(HMMs) [74.28-33] and various non-linear regression 
techniques [74.21,34,35], as we will discuss in more 
detail in Sect. 74.4. 

New learning challenges were, thus, set forth. 
Robots were expected to show a high degree of flex¬ 
ibility and versatility both in their learning system and 
in their control system in order to be able to interact nat¬ 
urally with human users and demonstrate similar skills 
(e.g., by moving in the same rooms and manipulating 
the same tools as humans). Robots were more and more 
expected to act human-like to enhance the interaction 
and so that their behavior would be more predictable 
and, hence, more acceptable. 


74.2 Key Issues When Learning from Human Demonstrations 


As mentioned in the beginning, learning from demon¬ 
stration (LfD) has at core to develop algorithms that are 
generic in their representation of the skills and in the 
way they generate the skills. 

The field has identified a number of key problems 
that need to be solved for ensuring such a generic ap¬ 
proach to transferring skills across various agents and 
situations [74.36,37]. These have been formulated as 
a set of generic questions, namely what to imitate, how 
to imitate, when to imitate, and who to imitate. These 
questions were formulated in response to the large body 
of diverse work in robotics LfD [74.18, 26, 38-41] that 
could not easily be unified under a small number of co¬ 
herent operating principles. The above four questions 
and their solutions aim at being generic in the sense of 
making no assumptions on the type of skills that may 
be transmitted. 

74.2.1 When and Whom to Imitate 

Whom and when to imitate has been largely unex¬ 
plored so far, and hence to date, only the first two 
questions have really been addressed. Figure 74.2 and 
illustrate how these two problems can be 
solved in a principled manner through statistical obser¬ 
vation of the demonstrations. 

How to Determine the Evaluation Metric 
What to imitate relates to the problem of determining 
which aspects of the demonstration should be imi¬ 


tated. For a given task, certain observable or affectable 
properties may be irrelevant and safely ignored. For 
instance, if the demonstrator always approaches a lo¬ 
cation from the north, is it necessary for the robot to 
do the same? The answer to this question strongly in- 


Observation of multiple Reproduction of a generalized 
demonstrations motion in a different situation 

Fig. 74.2 (a) A robot leams how to make a chess move 
(namely moving the queen forward) by generalizing across 
different demonstrations of the task performed in slightly 
different situations (different starting positions of the 
hand). The robot records the trajectories of its joints and 
leams to extract invariant features ( what-to-imitate ), i. e., 
that the task constraints are reduced to a subpart of the mo¬ 
tion located in a plane defined by the three chess pieces, 
(b) The robot reproduces the skill in a new context (for 
a different initial position of the chess piece) by find¬ 
ing an appropriate controller that satisfies both the task 
constraints and constraints relative to its body limitation 
( how-to-imitate problem) (after [74.21]) 
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fluences whether or not a derived robot controller is 
a successful imitation - a robot that approaches from 
the south is appropriately trained if direction is not 
important, but needs further education if it is. This is¬ 
sue is related to questions of signal versus noise and 
is answered by determining the metric by which the 
resulting behavior is evaluated. Different ways can be 
taken to address this issue. The simplest approach is to 
take a statistical perspective and deem as relevant the 
parts (dimension, region of input space) of the data that 
are consistently measured across all demonstration in¬ 
stances [74.21], If the dimension of the data is too high, 
such an approach may require too many demonstrations 
to gather enough statistics. An alternative is then to have 
the teacher help the robot determine what is relevant by 
pointing out the parts of the task that are most impor¬ 
tant. 

In summary, what to imitate removes consideration 
of details that, while perceptible/performable, do not 
matter for the task. It participates in determining the 
metric by which the reproduction of the robot can be 
measured. In continuous control tasks, what to imitate 
relates to the problem of defining automatically the fea¬ 
ture space for learning, as well the constraints and the 
cost function. In discrete control tasks, such as those 
treated by reinforcement learning and symbolic reason¬ 
ing, what to imitate relates to the problem of how to 
define the state and action space and of how to automat¬ 
ically learn the pre/post conditions in an autonomous 
decision system. 

74.2.2 How to Imitate and How to Solve 
the Correspondence Problem 

How to imitate consists in determining how the robot 
will actually perform the learned behaviors to maximize 
the metric found when solving the what to imitate prob¬ 
lem. Often, a robot cannot act exactly the same way as 
a human does, due to differences in physical embodi¬ 
ment. For example, if the demonstrator uses a foot to 
move an object, is it acceptable for a wheeled robot to 
bump it, or should it use a gripper instead? If the met¬ 
ric does not have appendage-specific terms, it may not 
matter. 

This issue is closely related to that of the corre¬ 
spondence problem [74.36]. Robots and humans, while 
inhabiting the same space and interacting with the same 
objects, and perhaps even superficially similar, still 
perceive and interact with the world in fundamentally 
different ways. To evaluate the similarity between hu¬ 
man behavior and that of robots, we must first deal with 
the fact that humans and robots may occupy different 
state spaces, of perhaps different dimensions. We iden¬ 
tify two different ways in which states of demonstrator 


and imitator can be said to correspond, and give brief 
examples: 

• Perceptual equivalence: Due to differences between 
human and robot sensory capabilities, the same 
scene may appear to be very different. For in¬ 
stance, while a human may identify humans and 



Fig. 74.3 (a,b) Perceptual equivalence (adapted 
from [74.42]). (c) Physical equivalence. The humanoid 
robot has the same arrangement of principal articulations 
as the human demonstrator, but different limb lengths 
and joint angle limits. The industrial robot has a different 
number and arrangement of articulations, which makes the 
mapping problem more challenging (illustration created 
with the V-REP simulator [74.43]). (d,e) Offline full-body 
motion transfer by taking into account the kinematic 
and dynamic disparity between the human and the hu¬ 
manoid [74.44]. See also and 

for example of mapping of full body motion from human 
to humanoids 
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gestures from color and intensity, a robot may use 
depth measurements to observe the same scene 
(Fig. 74.3a). Another point of comparison is tactile 
sensing. Most tactile sensors allow robots to per¬ 
ceive contact, but do not offer information about 
temperature, in contrast to the human skin. More¬ 
over, the low resolution of the robots’ tactile sensors 
does not allow robots to discriminate across the va¬ 
riety of existing textures, while human skin does. 
As the same data may, therefore, not be available 
to both humans and robots, successfully teach¬ 
ing a robot may require a good understanding of 
the robot’s sensors and their limitations. LfD ex¬ 
plores the limits of these perceptual equivalences, 
by building interfaces that either automatically cor¬ 
rect or make explicit these differences. 

• Physical equivalence'. Due to differences between 
human and robot embodiments, humans and robots 
may perform different actions to accomplish the 
same physical effect. For instance, even when per¬ 
forming the same task (soccer), humans and robots 
may interact with the environment in different ways 
(Fig. 74.3b). Humans run and kick, while robots 


74.3 Interfaces for Demonstration 

The interface used to provide demonstration plays a key 
role in the way the information is gathered and transmit¬ 
ted. We distinguish three major trends: 

1. One may directly record human motions. If one 
is interested solely in the kinematic of the mo¬ 
tion, one may use any of the various existing 
motion tracking systems, whether these are based 
on vision, exoskeleton, or other types of wearable 
motion sensors. The left-hand side of Fig. 74.4b 
and I'WiM'ii'H'Hl show an example of full body 
motion tracking during walking using vision. The 
motion of the human body is first extracted from 
the background using a model of human body. This 
model is subsequently mapped to an avatar and 
then to the humanoid robot DB at ATR, Kyoto, 
Japan. 

These external means of tracking human motion 
return precise measurement of the angular displace¬ 
ment of the limbs and joints. They have been used in 
various works for LfD of full body motion [74.33, 
47— 49]. These methods are advantageous in that 
they allow the human to move freely. However, they 
require solutions to the correspondence problem, 
i. e., the problem of how to transfer motion from hu¬ 
man to robot when both differ in the kinematic and 


roll and bump. Solving this discrepancy in motor 
capabilities is akin to solving the how to imitate 
problem to achieve the same effect. LfD develops 
way to solve this problem. Typically, the robot may 
compute a path (in Cartesian space) for its end- 
effector that is close to the path followed by the 
human hand, while relying on inverse kinematics 
to find the appropriate joint displacements. In the 
football example above, this would require the robot 
to determine a path for its center of mass which 
corresponds to the path followed by the human’s 
right foot when projected on the ground. Clearly, 
this equivalence is very task dependent. Recent so¬ 
lutions to this problem for hand motion and body 
motion can be found in [74.45, 46]. 

We can think of perceptual equivalence as dealing 
with the manner in which the agents perceive the world. 
Perceptual equivalence requires to make sure that the 
information necessary to perform the task is available 
to both humans and robots. Physical equivalence deals 
with the manner in which agents affect and interact with 
the world, so that the task is performable by both agents. 


dynamics of their body or, in other words, if the con¬ 
figuration space is of different dimension and size. 
This is typically done when mapping the motion of 
the joints that are tracked visually to a model of the 
human body that matches closely that of the robot. 
Such mapping would be particularly difficult to per¬ 
form when the walking machine (e.g., a hexapod) 
differs importantly from the human body. The prob¬ 
lem of mapping actions across two dissimilar bodies 
was already evoked earlier on and refers to the cor¬ 
respondence problem. 

2. Second, there are techniques such as kinesthetic 
teaching, where the robot is physically guided 
through the task by the human. This approach sim¬ 
plifies the correspondence problem by letting the 
user demonstrate the skill in the robot’s environ¬ 
ment with the robot’s own capabilities. It also pro¬ 
vides a natural teaching interface to correct a skill 
reproduced by the robot. Recent advances in skin 
technology offer the possibility to teach robots how 
to exploit tactile contact on an object (Fig. 74.4 
middle and |<S>»M£1I£1). By exploiting the com¬ 
pliance of the iCub robot’s fingers, the teacher can 
teach the robot how to adapt the posture of the fin¬ 
gers in response to a change in tactile sensing as 
measured at the robot’s finger tips [74.50]). 
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One main drawback of kinesthetic teaching is that 
the human must often use more degrees of free¬ 
dom to move the robot than the number of degrees 
of freedom moved on the robot. This is visible in 
Fig. 74.4. To move the fingers of one hand of the 
robot, the teacher must use both hands. This lim¬ 
its the type of tasks that can be taught through 
kinesthetic teaching. Typically tasks that would re¬ 
quire moving both hands simultaneously could not 
be taught this way. One could either proceed incre¬ 
mentally, teaching first the task for the right hand 
and then, while the robot replays the motion with 
its right hand, teach the motion of the left hand. 
However, this may prove to be cumbersome. The 
use of external trackers as reviewed above are more 
amenable to teaching coordinated motion between 
several limbs. 

3. Third, there are immersive teleoperation scenarios, 
where a human operator is limited to using the 
robot’s own sensors and effectors to perform the 
task. Teleoperation may be done using simple joy¬ 
sticks or other remote control devices, including 
haptic devices (Fig. 74.4 bottom and 
The later have the advantage that they can allow the 
teacher to teach tasks that require precise control 
of forces, while joysticks would only provide kine¬ 
matic information (position, speed). 

Teleoperation is advantageous compared to exter¬ 
nal motion tracking systems, as this solves the 
correspondence problem entirely, since the system 
directly records the perception and action from 
the robot’s configuration space. It is also advanta¬ 
geous compared to kinesthetic training, as it allows 
training the robots from a distance and is, hence, 
particularly suited for teaching navigation and lo¬ 
comotion patterns. The teacher no longer needs to 
share the same space with the robot. Teleopera¬ 
tion is, usually, used to transmit the kinematics of 
motion. For instance, in [74.51], the acrobatic tra¬ 
jectories of a helicopter are learned by recording the 
motion of the helicopter when teleoperated by an 
expert pilot. In [74.52], a robot dog is taught to play 
soccer by a human guiding it via a joystick. How¬ 
ever, in recent work, teleoperation has been used 
successfully to teach a humanoid robot balancing 
techniques [74.53]. Learning to react to perturba¬ 
tions is done through a haptic interface attached 
to the torso of the demonstrator, which measures 
the interaction forces when the human is pushed 
around. The kinematics of motion of the demon¬ 
strator are directly transmitted to the robot through 
teleoperation and are combined with haptic infor¬ 
mation to train a model of motion conditioned on 
perceived forces. 


The disadvantage of teleoperation techniques is 
that the teacher often needs training to learn to 
use the remote control device. Teleoperation us¬ 
ing a simple joystick allows guiding only a subset 
of degrees of freedom. To control for all degrees 
of freedom, very complex, exoskeleton type of de¬ 
vices must be used, which can be cumbersome. 
Moreover, teleoperation prevents the teacher from 
observing all sensorial information required to per¬ 
form the task. For instance, teleoperation, even 
when using haptic device, poorly renders the con¬ 
tacts perceived at the robot’s end-effector. To pal¬ 
liate to this, one may provide the teacher with 
visualization interfaces to simulate the interaction 
forces. 

4. Lastly, one can use explicit information, such as that 
conveyed by speech, to provide additional advice 
and comments to the demonstration [74.18,57,58] 
and Speech is a very natural means 

of communication among humans and, hence, is 
viewed as an easy way to allow the end-user to com¬ 
municate with robots. However, it necessitates that 
vocabulary that is understandable to the robot and 



Fig. 74.4 (a) Demonstration by visual tracking of gestures (af¬ 
ter [74.54], and l<s>MMSEM). (b) Demonstra¬ 

tion by kinesthetic teaching (after [74.55] and loiMIiilHILLM). ( c ) 
Demonstration by teleoperation (after [74.56] and 
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grounded in the actions and perceptions of the robot 
be defined beforehand. While this restricts teaching 
to discrete state-action pairs, it is particularly useful 
for symbolic reasoning. 


Each teaching interface has its pros and cons. It 
is thus interesting to investigate how these interfaces 
could be used in conjunction to exploit complementary 
information provided by each modality [74.50]. 


74.4 Algorithms to Learn from Humans 


Current approaches to encoding skills through LfD can 
be broadly divided into two trends: a low-level repre¬ 
sentation of the skill, taking the form of a non-linear 
mapping between sensory and motor information, and, 
a high-level representation of the skill that decom¬ 
poses the skill into a sequence of action-perception 
units. 

While the majority of work in LfD uses solely 
the demonstrations for learning, a growing number of 
works develops methods by which LfD can be com¬ 
bined with other learning techniques. One group of 
work investigates how to combine imitation learning 
with reinforcement learning, a method by which the 
robot learns through trial and error to maximize a given 
reward. Other works take inspiration in the way humans 
teach each other and introduce interactive and bidirec¬ 
tional teaching scenarios whereby the robot becomes an 
active partner during the teaching phase. We briefly re¬ 
view the main principles underlying each of these areas 
below: 

74.4.1 Learning Individual Motions 

Individual motions/actions (e.g., juicing an orange, 
trashing it, and pouring liquid into the cup in the exam¬ 
ple shown in 74.1) could be taught separately instead of 
simultaneously, as shown in this previous example. The 
human teacher would then provide one or more exam¬ 
ples of each submotion. If learning proceeds from the 


observation of a single instance of the motion/action, 
one calls this one-shot learning [74.60]. Examples of 
learning locomotion patterns can be found in [74.61]. 
To make sure that this is not akin to simple record 
and play, the controller is provided with prior knowl¬ 
edge in the form of primitive motion patterns. Learning 
then consists of instantiating the parameters modulating 
these motion patterns. 

Teaching can also proceed in batch mode af¬ 
ter recording several demonstrations, or incremen¬ 
tally by adding recursively more information trial by 
trial [74.12,50,62]. When learning in batch mode, 
learning considers all examples and draws inference 
by comparing the individual demonstrations. Infer¬ 
ence is usually based on a statistical analysis, where 
the demonstration signals are modeled via a proba¬ 
bility density function, exploiting various non-linear 
regression techniques stemming from machine learn¬ 
ing. Popular methods these days include Gaussian pro¬ 
cesses, Gaussian mixture Models, and support vector 
machines. 

Choosing properly the variables to encode a par¬ 
ticular movement is crucial, as it already implies part 
of the solution to the problem of defining what is 
important to imitate. Work in LfD encodes human 
movements in either joint space, task space, or torque 
space [74.63-65]. The encoding may be specific to 
cyclic motion [74.22], discrete motion [74.21], or to 
a combination of both [74.61], 



Fig. 74.5 Probabilistic encoding of 
motion in a subspace of reduced 
dimensionality (after [74.59] and 
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Encoding often encompasses the use of dimension¬ 
ality reduction techniques that project the recorded 
signals into a latent space of motion of reduced dimen¬ 
sionality. These techniques may either perform a local 
linear transformations [74.66-68] or exploit global non¬ 
linear methods [74.59, 69, 70] (Fig. 74.5). Additionally, 
task-specific rating functions [74.71] and simulation- 
based optimization [74.72] are investigated to identify 
relevant learning features. 

Teaching Force-Control Tasks 
While most LfD to date work focused on learning 
the kinematics of motions by recording the posi¬ 
tion of the end-effector and/or the position of the 
robot’s joints, more recently, some works have investi¬ 
gated transmission of force-based signals through hu¬ 
man demonstration [74.56,73-76]. See 
and for examples of kinesthetic teaching 

of compliant motion. Transmitting information about 
force is difficult for humans and for robots alike. Force 
can be sensed only when performing the task our¬ 
selves. Current efforts, hence, seek to develop methods 
by which one may embody the robot. This allows hu¬ 
man and robot to simultaneously perceive the forces 
applied when performing the task. A new exciting line 
of research, hence, leverages on recent advances in the 
design of haptic devices and tactile sensing, and on the 
development of torque and variable impedance actuated 
systems to teach force-control tasks through human 
demonstration. 

74.4.2 Learning Compound Actions 

Learning complex tasks, composed of a combination 
and juxtaposition of individual motions, is the ultimate 
goal of LfD. There are two major ways to proceed to 
learning of such complex tasks: 

1. One may first learn models of all individual mo¬ 
tions, using demonstrations of each of these actions 
individually. In a second stage, one may learn the 
right sequence and combination of these actions 
by observing a human performing the whole task. 
This approach, however, assumes that one can list 
all necessary individual actions, so-called primitive 
actions. To date, there does not exist a database of 
such primitive actions and one may wonder whether 
the variability of human motion may really be re¬ 
duced to a finite list of possible motions. A common 
approach is to first learn models of all of the individ¬ 
ual motions, using demonstrations of each of these 
actions individually [74.77,78], and then learn the 
right sequencing/combination in a second stage ei¬ 
ther by observing a human performing the whole 


task [74.79, 80] or through reinforcement learn¬ 
ing [74.81]. However, this approach assumes that 
there is a known set of all necessary primitive ac¬ 
tions. For specific tasks this may be true, but to date 
there does not exist a database of general purpose 
primitive actions, and it is unclear whether the vari¬ 
ability of human motion may really be reduced to 
a finite list. 

2. The alternative is to observe the human perform¬ 
ing the complete task and to automatically segment 
the task to extract the primitive actions, which may 
then become task-dependent, see e.g., [74.82,83]. 
This has the advantage of learning, in one swipe, 
both the primitive actions and the way they should 
be combined. One issue that arises is that the num¬ 
ber of primitive tasks is often unknown, and there 
could be multiple possible segmentations that must 
be considered [74.52]. 

Other examples include learning how to sequence 
known behaviors to enable complex navigation tasks 
through the imitation of a more knowledgeable robots 
or humans [74.9, 84, 85] and learning how to sequence 
primitive motions for full body motion in humanoid 
robots [74.25, 33, 86]. 

A large body of these works uses a symbolic repre¬ 
sentation of both the learning and the encoding of the 
task [74.6, 30, 85, 87-91]. This symbolic way of encod¬ 
ing skills may take several forms. One common way is 
to segment and encode the task according to sequences 
of predefined actions, described symbolically. Encod¬ 
ing and regenerating the sequences of these actions can, 
however, be done using classical machine learning tech¬ 
niques, such as HMM, [74.30], 

Often, these actions are encoded in a hierarchical 
manner. In [74.85], a graph-based approach is used 
to generalize an object moving skill, using a wheeled 
mobile robot. In this model, each node in the graph 
represents a complete behavior and generalization takes 
place at the level of the topological representation of the 
graph. The latter is updated incrementally. 

References [74.88, 89] follow a similar hierarchical 
and incremental approach to encode various house¬ 
hold tasks (such as setting the table and putting dishes 
in a dishwasher) (Fig. 74.6 and There, 

learning consists in identifying a sequence of prede¬ 
fined, elementary actions, which is further combined 
into a hierarchical task network. By analyzing multi¬ 
ple demonstrations, the ordering of elementary actions 
is learned, resulting in a precedence graph. The prece¬ 
dence graph defines a partial ordering on the set of 
learned elementary actions, which can be exploited to 
execute elementary actions in parallel, extracting sym¬ 
bolic rules that manage the way each object must be 
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handled. In [74.92], the approach was extended to learn¬ 
ing subsymbolic goal and constraint descriptions for 
each elementary action. In the execution phase, the 
robot applies motion planning to generate a motion to 
reach the goals while obeying the constraints. The re¬ 
sulting task description mimics the strategy that humans 
follow when performing the task. Based on the subsym¬ 
bolic goal and constraint descriptions, the robot can rea¬ 
son to adapt the strategy to changes in object location, 
obstacle occurrence, and varying start configurations. 

The approaches reviewed above assume a deter¬ 
ministic world, where actions unfold uniquely from 
perception of the current state of the world. However, 
robots operating in real environments will observe the 
world using imperfect sensors and the effects of their 
actions may be stochastic. To account for the stochastic- 
ity of the robot’s perceptions and actions, Schmidt-Rohr 
et al. [74.93] use a model of the task with partially ob¬ 
servable Markov decision processes (POMDP). At run 
time, an optimal (in a maximum likelihood sense) deci¬ 
sion is then taken. 

Reference [74.90] exploits also a hierarchical ap¬ 
proach to encoding a skill in terms of pre-defined 
behaviors. The skill consists in moving through a maze 
where a wheeled robot must avoid several kinds of 
obstacles and reach a set of specific subgoals. The par¬ 
ticularity of this approach lies in the use of symbolic 
representations of the skill, which are applied to explore 
the role of the teacher in guiding incremental learning 
of the robot. 


Finally, [74.91] took a symbolic approach to en¬ 
coding human motions as sets of pre-defined postures, 
positions, or configurations, considering different lev¬ 
els of granularity for the symbolic representation of the 
motion. This a priori knowledge is then used to explore 
the correspondence problem through several simulated 
setups, including motion in joint space of arm links and 
displacements of objects on a two-dimensional (2-D) 
plane. 

The main advantage of these symbolic approaches 
is that high-level skills (consisting of sequences of 
symbolic cues) can be learned efficiently through an 
interactive process. However, because of the symbolic 
nature of their encoding, the methods rely on a large 
amount of prior knowledge to predefine the important 
cues and to segment those efficiently. 

74.4.3 Incremental Teaching Methods 

The statistical approach described previously is an in¬ 
teresting way to extract autonomously the important 
features of the task, and, thus to avoid putting too much 
prior knowledge in the system. However, it requires 
a large number of demonstrations to draw statistically 
valid inference. It is not reasonable to assume that 
a layuser will perform many demonstrations of the same 
task. Hence, for LfD to be amenable to lay users, learn¬ 
ing should require as few demonstrations as possible. 
Ideally, one would like the robot to be bootstrapped 
with some initial knowledge, so that the robot can start 
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right away to perform the task, and human training 
would be used solely to help the robot gradually im¬ 
prove its performance. 

Incremental learning approaches that gradually re¬ 
fine task knowledge as more examples become avail¬ 
able pave the way towards LfD systems suitable for 
such continuous and long-life robot learning. Fig¬ 
ure 74.7 and l<S3>Mllil£lLLl shows an example of such 
incremental teaching of a simple skill. 

These incremental learning methods use various 
forms of deixis, as well as verbal and non-verbal inter¬ 
actions, to guide the robot’s attention to the important 
parts of the demonstration or to particular mistakes pro¬ 
duced by the robot during the reproduction of the task. 
Such incremental and guided learning is often referred 
to as scaffolding or molding of the robot’s knowledge, 
and is key to teaching robots tasks of increasing com¬ 
plexity [74.90,94], 

Research on the use of incremental learning tech¬ 
niques for robot LfD has contributed to the development 
of methods for learning complex tasks within the house¬ 
hold domain from as few demonstrations as possible. 
Moreover, it has contributed to the development and 
application of machine learning that allow a continu¬ 
ous and incremental refinement of the task model. Such 
systems have sometimes been referred to as background 
knowledge-based or EM deductive LfD-sy stems, as pre¬ 
sented in [74.95,96]. They usually require very few or 
even only a single user demonstration to generate ex¬ 
ecutable task descriptions. The main objective of this 
line of research is to build a meta-representation of the 
knowledge that the robot has acquired on the task and 
to apply reasoning methods on this knowledge database 
(Fig. 74.6). In this scenario, reasoning involves recog¬ 
nizing, learning, and representing repetitive tasks. 

Pardowitz et al. [74.97] discuss how different forms 
of knowledge can be balanced in an incremental learn¬ 
ing system. The system relies on building task prece¬ 
dence graphs. Task precedence graphs encode hypothe¬ 
ses that the system makes on the sequential structure 
of a task. Learning of the task precedence graphs al¬ 
lows the system to schedule its operations most flexibly, 


while still meeting the goals of the task ([74.98] for 
details). Task precedence graphs are directed, acyclic 
graphs that contain a temporal precedence relation that 
can be learned incrementally. Incremental learning of 
task precedence graphs leads to a more general and flex¬ 
ible representation of the task knowledge (Fig. 74.6 and 
i^EfiBansiEa). 

74.4.4 Combining Learning from Humans 
with Other Learning Techniques 

To recall, a main argument for the development of 
LfD methods was that they would speed up learning by 
providing examples of good solutions. This assumption, 
however, is realistic only if the context for the reproduc¬ 
tion is sufficiently similar to that of the demonstration. 
We saw previously that the use of dynamical systems- 
based representation at the trajectory level allows the 
robot to depart to some extent from a learned trajectory 
to reach the target, even when both the object and the 
hand of the robot have moved from the location shown 
during the demonstration. There are, however, situa¬ 
tions in which such an approach would fail, such as, for 
instance, when placing a large obstacle in the robot’s 
pathway (Fig. 74.8). Besides, robots and humans may 
differ significantly in their kinematics and dynamics of 
motion and, although there are varieties of ways to by¬ 
pass the so-called correspondence problem, relearning 
a new model may still be required in special cases. 

To allow the robot to relearn to perform a task in any 
new situation, it appeared important to combine LfD 
methods with other motor learning techniques. Rein¬ 
forcement learning (RL) appeared particularly suitable 
for type of problem. Indeed, imitation learning is lim¬ 
iting in that it requires the robot to learn only from 
what has been demonstrated. Reinforcement learning, 
in contrast, allows the robot to discover new control 
policies through free exploration of the state-action 
space. Approaches that combine imitation learning and 
reinforcement learning aim at exploiting the strength 
of both algorithms to overcome their respective draw¬ 
backs. Demonstrations are used to guide the exploration 



Fig. 74.7 An incremental learning strategy where a manipulation skill is first demonstrated through the use of a data 
glove. After a first reproduction trial, the skill is refined through kinesthetic teaching, by exploiting the tactile capabilities 
of the iCub humanoid robot (after [74.50]) 
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in reinforcement learning (RL). This, hence, reduces 
the time it takes for RL algorithms to find an adequate 
control policy, while allowing the robot to depart from 
the demonstrated behavior. Figures 74.8 show two ex¬ 
amples of techniques that use reinforcement learning 
in conjunction with LfD to improve the robot’s perfor¬ 
mance beyond that of a demonstrator. 

Early work on LfD using RL started in the 1990s 
with learning to swing up and control an inverse pen¬ 
dulum [74.100] and learning industrial tasks like peg- 
in-hole with a robot arm [74.26]. More recent efforts 
include [74.101-103], who tackled robust control of the 
upper body of humanoid robots in various manipulation 
tasks, learning an archery skill [74.104], and learning 
how to hit a snooker ball [74.105]. 

Demonstrations can be used in different ways to 
bootstrap RL. They may be used as initial roll-outs 
from which an initial estimate of the policy is com¬ 
puted [74.106-108], or to generate an initial set of 
primitives [74.81,103,107]. In the latter case, RL is 
then used to learn how to select across these primitives. 
Demonstrations can also be used to limit the search 
space covered by RL [74.101, 109], or to estimate the 
reward function [74.110,111]. Finally, RL and imita¬ 
tion learning can be used in conjunction at run time, 
by letting the demonstrator take over part of the control 
during one trial [74.112]. 

Another way to enable the robot to learn a control 
strategy through a combination of self-experimentation 
and learning from watching others is to evolve popu¬ 
lation of agents that mimic each other. Such an evo¬ 
lutionary approach using genetic algorithms has been 
investigated by a number of authors, e.g., for learn¬ 
ing of manipulation skills [74.113], navigation strate¬ 
gies [74. 1 14], or sharing a common vocabulary to name 
sensoriperception and actions [74.115]. 

Variants on Reinforcement Learning 
While most of the works that combine imitation learn¬ 
ing with reinforcement learning assume the reward to 
be known, inverse reinforcement learning (IRL) offers 
a framework to determine automatically the reward and 
the optimal control policy [74. 116]. When using human 
demonstrations to guide learning, IRL solves jointly 


the what to imitate and how to imitate problems. Other 
approaches to estimating the reward or cost function au¬ 
tomatically have been proposed, see, for instance, the 
maximum margin planning technique [74.1 17] and the 
automatic extraction of constraints [74.118]. 

Underlying all IRL works is the assumption of 
a consistent reward function. When demonstrations are 
provided by multiple experts, this assumes that all ex¬ 
perts optimize the same objectives. This is constraining 
and does not exploit the variability of ways in which 
humans may solve the same task. Recent IRL works 
consider multiple experts and identify multiple different 
reward functions [74.119, 120]. This allows the robots 
to learn multiple (albeit suboptimal) ways to perform 
the same task. The hope is that this multiplicity of 
policies will make the controller more robust, offering 
alternative ways to complete the task, when the context 
no longer allows the robot to perform the task in the 
optimal way. 

The vast majority of work on LfD relies on suc¬ 
cessful demonstrations of the desired task by the hu¬ 
man. It hence assumes that all the demonstrations are 
good demonstrations and discards those that are poor 
proxy of what would be deemed as a good demonstra¬ 
tion. Recent work has also investigated the possibility 
that demonstrations may instead be failed attempts at 
performing the task [74.121, 122]. Learning then pro¬ 
ceeds from observing solely incorrect demonstrations 
(loJEEHiEEl and |4S>M3EB5Eai). Note that demonstra¬ 
tions are never completely incorrect. Learning from 
failed demonstration then attempts to discover which 
parts of the demonstrations were correct and which 
were incorrect, so as to improve solely the incorrect 
parts. In this context, LfD addresses the questions of 
what to and what not to imitate. It offers an interesting 
alternative to approaches that combine imitation learn¬ 
ing and reinforcement learning, in that no reward needs 
to be explicitly determined. 

74.4.5 Learning from Humans, a Form 
of Human-Robot Interaction 

Another perspective adopted by LfD to make the trans¬ 
fer of skill more efficient is to focus on the interaction 


Fig. 74.8 Illustration of the use of 
reinforcement learning in policy 
parameter space to refine a skill 
initially learned from demonstration 
(after [74.99] and 
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aspect of the transfer process. As this transfer problem 
is complex and involves a combination of social mech¬ 
anisms, several insights from human-robot interaction 
(HRI) were explored to make efficient use of the teach¬ 
ing capabilities of the human user, [74.123-125] for 
surveys. Next, we briefly survey some of these works. 

The development of algorithms for detecting social 
cues given implicitly or explicitly by the teacher dur¬ 
ing training and the integration of those as part of other 
generic mechanisms for LfD has become the focus of 
a large body of work in LfD. Such social cues can 
be viewed as a way to introduce priors in a statistical 
learning system, and, by so doing, to speed up learning. 
Indeed, several hints can be used to transfer a skill not 
only by demonstrating the task multiple times but also 
by highlighting the important components of the skill. 
This can be achieved by various means, using different 
modalities. 

A large body of work explored the use of point¬ 
ing and gazing (Fig. 74.9 left and l<s>M3EE!lB3i) as 
a way of conveying the intention of the user [74.79, 
126-132]. Vocal deixis, using a standard speech recog¬ 
nition engine, has also been explored widely [74.79, 
133]. In [74.88], the user makes vocal comments to 
highlight the steps of the teaching that are deemed as 
being the most important. In [74.134,135], only the 
prosody of the speech pattern is looked at, rather than 
the exact content of the speech, as a way to infer some 
information on the user’s communicative intent. 

In [74.136], these social cues are learned through 
an imitative game, whereby the user imitates the robot. 
This allows the robot to build a user-specific model of 
these social pointers, and, hence be more robust to de¬ 
tecting those. 


Recent lines of research in interactive LfD seeks 
to give a more active role to the teacher in a bidi¬ 
rectional teaching process [74.15,137,138]. Robots 
become more active partners and can indicate which 
portion of the demonstration was unclear. Teachers may 
in turn refine the robot’s knowledge by providing com¬ 
plementary information where the robot is performing 
poorly. This supplementary information may consist of 
additional rounds of demonstrations of the complete 
task [74.139], or may be limited to subparts of the 
task [74.140,141], The information can be conveyed 
through specific task’s features, such as a list of way- 
points [74.142]. The robot is then left free to interpolate 
a trajectory using these key points. 

The design of such incremental teaching methods 
calls for machine learning techniques that enable the 
incorporation of new data in a robust manner. It also 
opens the door to the design of other human-robot inter¬ 
facing systems, including the use of speech, which leads 
to meaningful dialogs between humans and robots. An 
example of such bidirectional teaching is given on the 
right-hand side of Fig. 74.10. The robot asks for help 
during or after teaching, verifying that its understanding 
of the task is correct [74.14]. This teaching interaction 
is tailored to let the user become an active participant 
in the learning process (and not only a model of expert 
behavior). 

By taking inspiration from the human tutelage 
paradigm, [74.15] shows that a socially guided ap¬ 
proach can improve both the human-robot interaction 
and the machine learning process by taking into ac¬ 
count human benevolence. That work highlights the 
role of the teacher in organizing the skill into manage¬ 
able steps and maintaining an accurate mental model 



Fig. 74.9 Illustration of the use of social cues to speed up the imitation learning process. Here, gazing and pointing 
information are used to select probabilistically the objects relevant for the manipulation skill (l<sa>MMJ!li!lS) 



Fig. 74.10 Example of an active teaching scenario. The robot asks for help during or after teaching, verifying that its 
understanding of the task is correct (after [74.14]) (|43>)BlL!ll!il!M) 
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of the learner’s understanding. Reference [74.138] use 
a similar teaching paradigm and extend the concept 
to the learning of continuous motion trajectories and 
of actions on objects, and propose experiments where 
a humanoid robot learns new manipulation skills by 
first observing a human demonstrator (through motion 
sensors) and then gradually refining its skill through 
teacher support. In this application, the user provides 
scaffolds to the robot for the reproduction of the skill by 
moving kinesthetically a subset of the motors. Through 
the supervision of the user who progressively disman¬ 
tles the scaffolds after each reproduction attempt, the 
robot can finally reproduce the skill on its own. Ref¬ 
erence [74.143] highlights the importance of an active 
participation of the teacher not only to demonstrate 
a model of expert behavior but also to refine the ac¬ 
quired motion through spoken feedback. 

Reference [74.90] provides experiments where 
a wheeled robot is teleoperated through a screen inter¬ 
face to simulate a molding process, that is, by letting 
the robot experience sensory information when explor¬ 
ing its environment through the teacher’s support. Their 
model uses a memory-based approach in which the user 
provides labels for the different components of the task 
to teach hierarchically high-level behaviors. 

Finally, a core idea of the HRI approach to LfD is 
that imitation is goal directed, that is, actions are meant 
to fulfill a specific purpose and to convey the inten¬ 
tion of the actor [74.144]. While a longstanding trend 
in LfD approached the problem from the standpoint of 


trajectory following [74.84, 145, 146] and joint motion 
replication, [74.147-150], recent works, inspired by the 
above rationale, start from the assumption that imitation 
is not just about observing and replicating the motion, 
but rather about understanding the goals of a given ac¬ 
tion (see the above survey of approaches to determining 
automatically the reward or what to imitate). 

Determining the way humans learn to both ex¬ 
tract the goals of a set of observed actions and give 
these goals a hierarchy of preference is fundamental 
to our understanding of the underlying decisional pro¬ 
cess to imitation. While we have surveyed recent work 
in that area, it is important to recall other approaches 
to tackling these issues that have previously followed 
a probabilistic approach to explain the derivation and 
sequential application of goals and apply this to enable 
learning of manipulatory tasks requiring sequencing of 
subsets of goals [74.97, 145,151, 152], 

Understanding the goal of the task is still only half 
of the picture, as there may be several ways of achiev¬ 
ing the goal of the task. Moreover, what is feasible 
(or optimal) for the demonstrator may not necessarily 
be appropriate for the imitator [74.36]. Thus, different 
models, modes and communication channels, should be 
used in conjunction to find a solution that is optimal 
both from the point of view of the imitator and that 
achieves what the demonstrator seeks to teach the robot. 

This concludes our survey. As the reader can see, 
the issues of what and how to imitate are tightly con¬ 
nected and to a large extent remain only partly solved. 


74.5 Conclusions and Open Issues in Robot LfD 


Research in LfD or programming by demonstration 
(PbD) is progressing rapidly, pushing back limits and 
posing new questions all the time. As such, any list 
of limitations and open questions is bound to be in¬ 
complete and out of date. However, there are a few 
long-standing limitations and open questions that bear 
further attention. 

Generally, work in LfD assumes a fixed, given form 
for the robot’s control policy, and learns appropriate pa¬ 
rameters. To date, there are several different forms of 
policies in common usage, and there is no clear cor¬ 
rect (or dominant) technique. Furthermore, it is possible 
that a system could be provided with multiple possible 
representations of controllers and select which is most 
appropriate. 

The combination of reinforcement learning and im¬ 
itation learning has been shown to be effective in 
addressing the acquisition of skills that require fine tun¬ 
ing of the robot’s dynamics. Likewise, more interactive 


learning techniques have proven successful in allowing 
for collaborative improvement of the learnt policy by 
switching between human-guided and robot-initiated 
learning. However, there do not yet exist protocols to 
determine when it is best to switch between the various 
learning modes available. The answer may, in fact, be 
task dependent. 

In work to date, teaching is usually done by a single 
teacher, or teachers with an explicit concept of the task 
to teach. More work needs to be done to address issues 
related to conflicting demonstrations across teachers 
with different styles. Similarly, teachers are usually hu¬ 
man beings, but could instead be an arbitrary expert 
agent. This agent could be a more knowledgeable robot 
or a computer simulation. Finally, another relatively lit¬ 
tle explored question relates to the problem of how to 
transfer skills across multiple agents, including multi¬ 
ple robots (i. e., teaching is done from a teacher robot to 
various learner robots). Early work in this direction was 
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done in the 1990s [74.115, 153,154]. This work, how¬ 
ever, has so far been reduced to transfer of navigation or 
communication skills across swarms of simple mobile 
robots. 

Experiments in LfD have mostly focused on a sin¬ 
gle task (or set of closely related tasks), and each 
experiment starts with a tabula rasa. As learning of 


complex tasks progresses, means to store and reuse 
prior knowledge at a large scale will have to be devised. 
Learning stages, akin perhaps to those found in child 
development, may be required. There will need to be 
a formalism to allow the robot to select information, to 
reduce redundant information, select features, and store 
new data efficiently. 
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Demonstrations and reproduction of the task of juicing an orange 

available from http://handbookofrobotics.org/view-chapter/74/videodetails/29 

Demonstrations and reproduction of moving a chessman 

available from http://handbookofrobotics.org/view-chapter/74/videodetails/97 

Full-body motion transfer under kinematic/dynamic disparity 

available from http://handbookofrobotics.org/view-chapter/74/videodetails/98 

Demonstration by visual tracking of gestures 

available from http://handbookofrobotics.org/view-chapter/74/videodetails/99 
Demonstration by kinesthetic teaching 

available from http://handbookofrobotics.org/view-chapter/74/videodetails/100 
Demonstration by teleoperation of humanoid HRP-2 

available from http://handbookofrobotics.org/view-chapter/74/videodetails/101 
Probabilistic encoding of motion in a subspace of reduced dimensionality 
available from http://handbookofrobotics.org/view-chapter/74/videodetails/102 
Reproduction of dishwasher unloading task based on task precedence graph 
available from http://handbookofrobotics.org/view-chapter/74/videodetails/103 
Incremental learning of finger manipulation with tactile capability 
available from http://handbookofrobotics.org/view-chapter/74/videodetails/104 
Policy refinement after demonstration 

available from http://handbookofrobotics.org/view-chapter/74/videodetails/105 
Exploitation of social cues to speed up learning 

available from http://handbookofrobotics.org/view-chapter/74/videodetails/106 
Active teaching 

available from http://handbookofrobotics.org/view-chapter/74/videodetails/107 
Learning from failure I 

available from http://handbookofrobotics.org/view-chapter/74/videodetails/476 
Learning from failure II 

available from http://handbookofrobotics.org/view-chapter/74/videodetails/477 

Learning compliant motion from human demonstration 

available from http://handbookofrobotics.org/view-chapter/74/videodetails/478 

Learning compliant motion from human demonstration II 

available from http://handbookofrobotics.org/view-chapter/74/videodetails/479 
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75. Biologically Inspired Robotics 


Fumiya lida, Auke Jan Ijspeert 


Throughout the history of robotics research, nature 
has been providing numerous ideas and inspi¬ 
rations to robotics engineers. Small insect-like 
robots, for example, usually make use of reflexive 
behaviors to avoid obstacles during locomotion, 
whereas large bipedal robots are designed to 
control complex human-like leg for climbing up 
and down stairs. While providing an overview of 
bio-inspired robotics, this chapter particularly fo¬ 
cus on research which aims to employ robotics 
systems and technologies for our deeper un¬ 
derstanding of biological systems. Unlike most 
of the other robotics research where researchers 
attempt to develop robotic applications, these 
types of bio-inspired robots are generally devel¬ 
oped to test unsolved hypotheses in biological 
sciences. Through close collaborations between 
biologists and roboticists, bio-inspired robotics 
research contributes not only to elucidating chal¬ 
lenging questions in nature but also to developing 
novel technologies for robotics applications. In 
this chapter, we first provide a brief histori¬ 
cal background of this research area and then 
an overview of ongoing research methodologies. 

A few representative case studies will detail the 
successful instances in which robotics technolo¬ 
gies help identifying biological hypotheses. And 
finally we discuss challenges and perspectives in 
the field. 

Biologically inspired robotics (or bio-inspired 
robotics in short) is a very broad research area 
because almost all robotic systems are, in one 
way or the other, inspired from biological systems. 
Therefore, there is no clear distinction between 
bio-inspired robots and the others, and there is no 
commonly agreed definition [75.1]. For example, 
legged robots that walk, hop, and run are usually 
regarded as bio-inspired robots because many 


75.1 General Background . 2016 

75.1.1 Brief History 

and Conceptual Background . 2016 

75.2 Methodology . 2017 

75.2.1 Similarities and Differences 

in Robotics and Biology . 2017 

75.2.2 Modeling Biological Systems . 2018 

75.2.3 Research Methods and Tools . 2020 

75.3 Case Studies . 2021 

75.3.1 Bio-Inspired 

Legged Locomotion . 2021 

75.3.2 Reflexes and Central Pattern 

Generators . 2023 

75.3.3 Bio-Inspired Navigation . 2024 

75.4 Landscape of Bio-Inspired Robotics 

Research and Challenges . 2026 

75.4.1 Bio-Inspired Climbing . 2026 

75.4.2 Flapping Flight 

and Swimming Mechanisms . 2027 

75.4.3 Artificial Hands, Haptics, 

and Whiskers . 2027 

75.4.4 Self-Reconfigurable 

and Evolutionary Robotics . 2028 

75.4.5 Bio-Inspired Soft Robots . 2028 

75.4.6 Neuroprosthetics and Social 

Interactions . 2028 

75.5 Conclusion . 2028 

Video-References . 2028 

References . 2029 


biological systems rely on legged locomotion for 
their survival. On the other hand, many robotics 
researchers implement biological models of motion 
control and navigation onto wheeled platforms, 
which could also be regarded as bio-inspired 
robots [75.2], 
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75.1 General Background 

The broad spectrum of bio-inspired robotics research 
is reflected to a variety of synonyms used in different 
scientific communities. For example, biomimetics and 
bionics are usually used to represent the types of re¬ 
search in which researchers observe biological systems 
and extract design principles for robotic applications. 
The terms bio-robotics and bio-engineering are also 
used interchangeably to biomimetics and bionics, but 
they often refer to engineered solutions specifically 
for biomedical applications. Another approach, usu¬ 
ally classified as artificial life, biological cybernetics, 
or biophysics, investigates biological systems by using 
synthetic approaches. Here biological systems are typ¬ 
ically viewed as mechanical and chemical entities and 
characterized by mechanistic models that are often very 
similar to the bio-inspired robots. The definitions and 
types of research studies conducted in these research 
areas are often overlapping, and many research projects 
provide results across the different areas. 

While some of these research areas are also cov¬ 
ered in the other chapters of this handbook, one of 
the most prominent, and more importantly useful, as¬ 
pects of bio-inspired robotics lies in the contributions 
of robotics research for biological sciences. In contrast 
to the other robotics research, a significant number of 
bio-inspired robots were developed for the purpose of 
testing hypotheses concerning biological systems and 
for identifying the underlying mechanisms of biological 
systems that are very difficult to be clarified otherwise. 
In this chapter, we specifically focus on this aspect of 
bio-inspired robotics. 

75.1.1 Brief History 

and Conceptual Background 

There is a long history of the engineers’ desire to repli¬ 
cating biological systems into artificial ones, includ¬ 
ing the famous examples of Japanese Karakuri dolls 
and Swiss automata several centuries ago (Fig. 75.1). 
Similarly, scientific efforts to use robotic systems to 
understand biological systems have also a relatively 
long history, which goes back even before the modern 
robotics started. One of the most influential examples in 
the earlier ages can be represented by the works in the 
field of cybernetics in which a number of system theo¬ 
ries such as the concepts of feedback and feedforward 
were applied to understand phenomena in biological 
systems [75.3,4]. 

The rise of neuroscience in the mid-twentieth cen¬ 
tury influenced bio-inspired robotics research signifi¬ 
cantly. In the 1950s, one of the first bio-inspired robots, 
Tortoises, was built by Grey Walter, a neurophysi¬ 


ologist. The work was mainly driven by biological 
questions, such as how a simple neuron connectiv¬ 
ity can result in richness of sensory-motor behaviors, 
and a turtle-like robot consisting of analog electronics, 
sensors, and electric motors demonstrated reflexive be¬ 
haviors as well as basic motor learning in autonomous 
robots [75.5], Another physiologist, Valentino Brait- 
enberg, also explored the power of understanding by 
building approach, which led to the famous thought 
experiments of Braitenberg Vehicles, i. e., a series of 
imaginary mobile robots that are still often used to teach 
the relationship between neural connectivity and sen¬ 
sory-motor behaviors [75.6]. 

The invention of digital computers was also an¬ 
other historical milestone in bio-inspired robotics. The 
pioneers of digital computers such as John von Neu¬ 
mann and Alan Turing provided significant influences 
in the biological studies through their seminal works 
on self-replication and self-organization [75.7, 8], and 
the subsequent foundation of the field of Artificial 
Intelligence in the 1950s was driven based on the un¬ 
derstanding of human intelligence especially from the 
computational standpoint [75.9]. 

While the power of digital computers dominated 
bio-inspired robotics for a while, the study of insect-like 
reflexive behaviors became popular again in the 1980s 
with the emergence of behavior-based robotics. The 
intensive studies of this approach shed light on the mas¬ 
sively parallel nature of motion control processes, and 
demonstrated behaviors that cannot be fully explained 
by the conventional sense-think-act style control ap¬ 
proach [75.10,11). One of the main contributions of 
this line of studies lies in the fact that the diversity and 
flexibility of control architecture is essential in adaptive 
behaviors in real-world systems, and the complexity of 
behaviors is not necessarily originated in the complex¬ 
ity of controller but in physical system-environment 
interactions [75.12]. 

Since then the research on physical system-environ¬ 
ment interactions has been very popular in the interdis¬ 
ciplinary community of embodied cognitive science and 
artificial intelligence in which roboticists, biologists, 
computer scientists, and physicists work together to look 
into further details of underlying mechanisms of adap¬ 
tivity in biological systems. Here the body as a physical 
entity is not only considered as a necessary container 
of intelligent adaptive behaviors but it also plays a cen¬ 
tral role that induces self-organization of patterns and 
structures in adaptive intelligent behaviors [75.13-15]. 
In this context, robotics methodologies and technologies 
are used to investigate hypotheses about the roles of em¬ 
bodiment that cannot be easily tested in animals. 
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Fig.75.1a-e Examples of bio-inspired robots in the history, (a) Japanese Karakuri doll, (b) Swiss Automaton, (c) Grey 
Walter’s Tortoises Robot (after [75.5]), (d) Braitenberg Vehicle (after [75.6]), and (e) a cleaning robot Roomba as an 
example of behavior-based robotics application 


75.2 Methodology 

In order to provide contributions to both robotics and bi¬ 
ological sciences, bio-inspired robotics research usually 
follows a series of unique research processes which are 
not necessarily common to the other areas of robotics. 
This section first explains similarities and differences in 
robotics and biology research, and then introduces a set 
of important concepts, methodology, and research tools 
often used in the bio-inspired robotics. 

75.2.1 Similarities and Differences 
in Robotics and Biology 

Both robotics and biological research aim to obtain 
the basic understanding about autonomous and adap¬ 
tive behaviors of complex systems, thus the scientific 
methodologies of these fields are usually very simi¬ 
lar. For example, biologists usually start with finding 
an interesting problem in animals, that can be compa¬ 
rable to roboticists building robot prototypes of their 
interests as the first step of research. Second, the an¬ 
imals are carefully observed such that the underlying 
mechanisms can be efficiently and effectively analyzed, 
a step which roboticists also follow in the case of their 
robots. And finally, both biologists and roboticists de¬ 
velop hypothetical models to explain the mechanisms 
of their interests and test them through additional ex¬ 
periments and analyses. Here the modeling processes 
are also similar in both robotics and biology in a sense 
that they typically employ similar scientific methods, 
such as dynamics modeling tools, computational opti¬ 
mization techniques, and the other analysis methods of 
system engineering, for example. 

There are, however, a number of fundamental dif¬ 
ferences in biological and robotics research, which are 
mostly originated in the fact that biological systems are 
constructed by considerably different design principles 
in nature. Although it is very difficult to cover all dif¬ 
ferences, the following aspects of biological systems 


characterize some of the major discrepancies from to¬ 
day’s robotic systems. 

Multipurpose Systems in Unstructured 
and Uncertain Environment 
In contrast to most of the robotic systems that are de¬ 
signed to perform one type of task in a well-defined 
environment, all biological systems are intrinsically 
multipurpose systems that are designed for many tasks 
in undefined and uncertain environments. For exam¬ 
ple, animals need to process tasks such as regulating 
metabolism for self-sufficiency, protecting themselves 
from predators, mating, and reproducing. It is important 
for roboticists to know that (i) animals are not opti¬ 
mized for one of these tasks, but they are designed to 
conduct all of them, and (ii) they do so in a way that is 
not optimal in any sense, but just good enough to sur¬ 
vive and reproduce. Therefore, it is often not a good 
idea to blindly copy a part of animals’ designs and 
mechanisms into robotic systems because there could 
be an alternative optimal solution if the system has to 
do a single task in a well-defined environment. Robotics 
researchers are, however, able to learn many principles 
and mechanisms from biological systems, if they are in¬ 
terested in the systems that need to deal with many tasks 
in unstructured and uncertain environments. 

Massively Parallel, Modular, 
and Redundant Structures 
Biological systems are composed of highly redundant 
structures in their bodies [75.13]. For example, most of 
multicellular organisms have massively parallel mus¬ 
cle fibers constituting a muscle group, a skeletal joint 
controlled through multiple muscle groups, millions of 
nerve cells conducting parallel signal processing, and 
countless receptors sensing changes in the environment. 
If compared to our robotic systems today, biological 
systems have orders of magnitude larger numbers of 
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such sensory, actuation, and computational units that 
have to be carefully considered in bio-inspired robotics 
research. In addition, the processes necessary for auton¬ 
omy and adaptivity are generally highly decentralized 
in biological systems. The control of animals’ arms 
and legs, for example, involves mechanical interactions 
of musculoskeletal structures, reflexive sensory-motor 
pathways in spinal cord, and more complex motion 
control and planning in the higher centers of nervous 
systems, which are running in parallel. The redundant 
structures of biological systems also provide an impor¬ 
tant discrepancy between animals and today’s robots: 
most of the subsystems in a living organisms have great 
autonomy and adaptivity by themselves, as exemplified 
by the fact that individual cells in skins and bones as 
well as receptors and muscles have their own regulation 
mechanisms as metabolism and growth. 

Self-Organization and Dynamic Changes 
of Entire Organisms 

Another important discrepancy between animals and 
today’s robots lies in the fact that there is no human 
designers behind animals, and all components in an or¬ 
ganism have to be designed, assembled and repaired by 
itself. Consequently, every individual animal has to start 
its life smaller and gradually grow larger over time; 
every part of their bodies is continuously changing; 
sensory-motor control has to be continuously updated 
to reflect the changes in the bodies. It is particularly im¬ 
portant for robotics researchers to consider that there 
are different timescales in these dynamic processes in 
biological systems (Fig. 75.2 [75.16]). Some of the dif¬ 
ferent timescales can be represented by the continuous 
changes of body plans at the evolutionary processes, the 
changes of body sizes and muscle strength through on¬ 
togenetic timescale, and update of sensory-motor loops 
in here and now timescale. 

75.2.2 Modeling Biological Systems 

Modeling plays an essential role in biological sciences. 
Biological models are the representation of knowl¬ 
edge which is not only used to communicate scientific 
discoveries among researchers but also structuring re¬ 
search areas by labeling knowns and unknowns. To 
make research activities efficient and effective, there are 
many different ways to model biological systems such 
as descriptive/illustrative explanations, mathematical, 
physical, or chemical representations. From this per¬ 
spective, the researchers in bio-inspired robotics have 
been exploring whether robotic platforms can be used 
as a scientific tool to develop models of biological 
systems, which leads to an alternative approach to elu¬ 
cidating biological hypotheses (Fig. 75.3). Robots are 



Fig. 75.2 Elements and their interactions that influence be¬ 
haviors in biological systems. The model includes both 
neural and non-neural elements such as hormones (which 
constitute part of the extracellular biochemistry), bones, 
muscles, sensors, and their ontogenetic processes (af¬ 
ter [75.16]) 

useful because they are physical entities as compared to 
simulated ones. First of all, by building models that can 
be implemented in a physical system, the process en¬ 
sures whether the model in question is physically mean¬ 
ingful or not. In contrast, computationally simulated 
models always involve many approximations and sim¬ 
plifications which might affect the realism and therefore 
usefulness of the model (e.g., correctly modeling the 
hydrodynamics of a swimming fish is very hard in 
a simulation while it comes for free in a fish robot). Sec¬ 
ond, building robots directly contributes to the develop¬ 
ment of unconventional technological components. 

Having said that, the use of robots as a scientific 
tool for biology is challenging because of the con¬ 
siderable discrepancies between animals and robots 
as explained in the previous subsection. A superficial 
copy of biological systems into a robotic counterpart 
is not a good idea because robots usually functions 
based on a completely different set of mechanisms, 
and the developed robot would most likely not explain 
much about the underlying mechanisms of biological 
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Fig. 75.3 Overview of bio-inspired robotics. Both biology 
and robotics follow similar research methods and inter¬ 
act to each other through models and tools, while they 
contribute to different objectives, i. e., understanding of bi¬ 
ological systems or development of robotic applications 

systems. Instead, it is particularly important to carefully 
examine what biological hypothesis we are interested 
in and we are testing by building robotic platforms. As 
an extreme example, if one is interested in understand¬ 
ing navigation mechanisms of animals, it is usually 
better to start with wheeled robot platforms than legged 
or flying ones, because the latter platforms would 
introduce unnecessary complexity in the research to 
examine a given hypothesis. 

It is, however, not a trivial problem to find good 
biological hypotheses that can be applied in the bio¬ 
inspired robotics research. Biological systems are gen¬ 
erally very complex, and hypotheses are not clearly 
separable from one question to another. Behaviors of 
animals are, for example, a result from neuronal activi¬ 


ties of short and long terms, musculoskeletal dynamics, 
genetic and social interactions, and the mechanisms 
could also significantly vary in different individuals or 
species. 

In order to deal with such a complexity in na¬ 
ture, Full and Koditchek have proposed an insightful 
methodology [75.17]. As shown in Fig. 75.4, they pro¬ 
posed a two-level modeling process which could benefit 
bio-inspired robotics research greatly: in the first level, 
a model should be simplified as much as possible such 
that it can be generalized over many species or in a large 
scale without being bothered too much by the details 
of complex animal structures. These models are called 
templates. A good example of this kind is the so-called 
spring-mass model that characterizes running behav¬ 
iors of many different types of legged animals, even 
though it is an extremely simple model consisting of 
a point mass and a linear spring only. This approach 
enables researchers to examine the basic principles in 
nature, and despite its simplicity, the model can explain 
behavioral characteristics in a wide variety of animals 
even without considering detailed anatomical discrep¬ 
ancies between species. In the second level of modeling 
processes, templates should be enhanced by more de¬ 
tails, which are called anchors. The investigations of the 
anchors are generally intended for more specific ques¬ 
tions by adding redundancies of muscles or limbs or 
implementing more complex neuromuscular circuitry, 
for example. Through the template-anchor research ap¬ 
proach, a research area can be effectively structured, 
and, for bio-inspired robotics research in particular, 
the simplified models can benefit robotics engineers to 
replicate behaviors of biological systems in a concep¬ 
tual level. 

Independently from the simplicity, it is also im¬ 
portant to consider the purposes of models which 



Simple 

general 

prescriptive 


Elaborate 

representative 


Complex 

redundant 


Fig. 75.4 Modeling hierarchy ex¬ 
emplified by the legged locomotion 
models. Animals can be abstracted 
into more elaborate and representative 
models (anchors) and/or simpler and 
more general models (templates) 
(after [75.17]) 
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Fig. 75.5 Dimensions for describing models (after [75.18]) 


also influence how to structure bio-inspired robotics 
projects. After all, there is no correct model, but there 
are good or bad models with respect to the goal and 
hypothesis that are investigated. Webb has, for exam¬ 
ple, pointed out that there are seven major criteria 
with which we could evaluate models of bio-inspired 
robotics (Fig. 75.5 [75.18]): 

1. Relevance to biology: as not all robotics research 
contributes to biological studies, it is important to 
clarify the degree to which a target robotic system 
and model are relevant. 

2. Level: what are the base units of the model, and on 
what level of hierarchy in biological systems does 
the model attempt to represent? 

3. Generality: a model could be developed for eluci¬ 
dating a mechanism of a specific system of interest, 
or for a more general mechanism that can be applied 
to many others. Therefore, the generality criterion 
considers how many systems the model intends to 
represent? 

4. Abstraction: this criterion concerns the number and 
complexity of mechanisms included in the model. 

5. Structural accuracy: there are many ways of rep¬ 
resenting systems but a question is the degree to 
which the model explains the internal mechanism 
of the target systems. In an extreme case, a model 
can behave the same at the level of input/output re¬ 
lationship, but it does not necessarily mean that the 
internal mechanism is the same. 

6. Behavioral match: to what extent does the model 
behave like the target animal? 

7. Medium of model: what is the model built from? 
A model can be made of mechanical, electrical, 
hydraulic, etc. or alternatively iconic, analog, sym¬ 
bolic, for example. Properly identifying these ques¬ 
tions and the purpose of a robot is important. In¬ 
deed, there is a risk in bio-inspired robotics that 
a project is not useful to biology (i. e., it does 


not properly address a scientific question) nor to 
robotics (i. e., it does not perform better than a more 
conventional robot). 

75.2.3 Research Methods and Tools 

As it might have been already noticed, robotics tech¬ 
nologies are used for many different purposes in the 
context of bio-inspired robotics. Robots are often used 
to explore new research areas and questions; many 
platforms were built and examined for the purpose 
of testing biological hypotheses', some of the other 
platforms were developed specifically for application 
discovery based on the identified principles in biolog¬ 
ical studies; and more recently, robotics technologies 
were interfaced with or integrated into biological sys¬ 
tems to understand or enhance animals capabilities. 

Even though there are many successful contribu¬ 
tions of robotics technologies to biological studies, it is 
often not trivial to identify what are the good methods 
and tools for the given specific hypothesis or prob¬ 
lems. In particular, one of the most critical questions 
is probably whether it is necessary to build a phys¬ 
ical robotic platform or it is sufficient to investigate 
pseudo-robots in simulation. Building physical robots 
is usually very costly and requires significant amount 
of additional knowledge and know how; thus there are 
many case studies of bio-inspired robotics research con¬ 
ducted only in simulation in the past [75.19,20]. This 
type of research generally makes use of physics mod¬ 
els or physically realistic simulation environments that 
allow us to explore fairly complex artificial creatures. 
This approach is useful to explore a large parame¬ 
ter space which cannot be optimized in the real-world 
platforms. Also, the use of virtual creatures in the bio¬ 
inspired robotics research is also extremely important 
to investigate concepts and hypotheses that are not pos¬ 
sible to test technologically such as robots with point 
masses, frictionless joints, self-replication, and robotic 
hardware evolution that cannot be possible with our to¬ 
day’s technologies. 

However, building physical robots is a must in some 
cases. For example, there are often target concepts or 
hypotheses involved in physical processes that are dif¬ 
ficult to theoretically model, such as friction, impacts, 
thermodynamics, and hydro/aerodynamics. In addition, 
there has been an increasing interest in complex robotic 
systems that contain a considerable number of physical 
elements such as joints and actuators, simulation mod¬ 
els of which tend to be vulnerable against accumulated 
errors. Another aspect of robots used as a scientific tool 
is to explore hypotheses that cannot be tested in nature. 
This approach is often called synthetic methodology (an 
understanding-by-building approach [75.13]), meaning 
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that, by building and using artificial systems, we test bi¬ 
ological principles that cannot be tested in conventional 
biological methods. The most representative example 
is the Braitenberg Vehicles, which we introduced in 
Sect. 75.1.1. Every vehicle has a set of oversimplified 
neural connections of an animal which does not exist in 
nature, but these creatures can explain important princi¬ 
ples in biological systems. A similar approach was also 

75.3 Case Studies 

This section introduces three research areas that have 
been particularly active in bio-inspired robotics in the 
last few decades. Here we highlight the case studies in 
which robotics researches contributed to our further un¬ 
derstanding of biological systems, and we illustrate how 
the concepts and methods we introduced in the previous 
sections were applied to these studies. 

75.3.1 Bio-Inspired Legged Locomotion 

Legged locomotion has been one of the most popular 
research topics in bio-inspired robotics for a long time 
because it characterizes the salient differences between 
biological and artificial systems. Although legged lo¬ 
comotion is seemingly easy, useful, and efficient for 
biological systems to move around in complex en¬ 
vironments, it is surprisingly difficult to understand 
the underlying mechanisms hence challenging to im¬ 
plement into robotic systems. The legged locomo¬ 
tion research has a relatively long history that goes 
back to the foundation of biomechanics in the sev¬ 
enteenth century [75.24], and it became particularly 
popular when the modem robotics was established in 
the 1970s [75.25,26]. One of the main reasons that 
many robotics researchers were attracted by the issue 
of legged locomotion lies in the fact that it covers 
many of the discrepancies between biological and arti¬ 
ficial systems that we discussed in Sect. 75.2.1. More 
specifically, legged systems have to carry out many 
different tasks in unstructured environments such as es¬ 
tablishing sturdy footholds, avoiding obstacles, dealing 
with variations of payload and velocities, and chang¬ 
ing gait patterns, for example; Legged systems have 
to deal with massive parallel processes which are re¬ 
quired for coordinating many joints and muscles, as 
well as local reflexes and high-level decision and plan¬ 
ning; And adaptivity is essential for legged systems 
because they need to maintain mobility in different en¬ 
vironments as well as under significant changes of its 
body plan over growth processes, for example. While 
there are countless case studies in the past, this subsec¬ 
tion focuses on the four key challenges of bio-inspired 


extended to the studies of life as it could be [75.21, 
22], in which, by creating biological phenomena in 
simulation or robots, a broader and more universal 
understanding of living systems. This methodology is 
particularly effective in the studies related to evolution¬ 
ary biology, in which verifications of hypotheses are 
usually very challenging ([75.20,23]; see also the chap¬ 
ter of evolutionary robotics). 


legged locomotion, i. e., stability, gait, energetics, and 
actuation, because they have been recognized as the 
long-standing challenges in bio-inspired legged loco¬ 
motion. More comprehensive robotics research can be 
found in the section of legged locomotion. 

Stability and Gait in Legged Locomotion 
One of the main challenges in the study of legged 
locomotion is to uncover the mechanisms of motion sta¬ 
bilization against disturbances [75.27]. Biological sys¬ 
tems usually make use of a variety of mechanisms in¬ 
cluding mechanical self-stabilization [75.28, 29], spinal 
reflexes [75.30,31], central pattern generators [75.32], 
or sensory-motor control originated in the higher center 
of nervous system [75.33]. Because of the complex¬ 
ity of stabilization mechanisms in animals, bio-inspired 
robotics played an important role to systematically in¬ 
vestigate the issues of stabilization through modeling of 
locomotion dynamics and reproducing the behaviors in 
legged robots of various kinds in the past. 

Among others, bipedal walking was one of the most 
intensively studied topic areas which nicely illustrates 
how a synthetic approach could structure a research area 
of a complex biological problem. The backbone of this 
research area is the so-called inverted pendulum walking 
model which was originally investigated in biomechan¬ 
ics, and later used in the bio-inspired robotics research. 
The model considers the simplest physics representation 
of walking dynamics, i. e., a point mass is attached on 
a massless link, and simulates walking dynamics as the 
mass vaulting over the link [75.24]. This model can be, 
for example, physically implemented by the so-called 
rimless wheel that is the simplest robotic representa¬ 
tion of the model (Fig. 75.6a; [75.34,43]), and then, 
a slightly more complex configuration, the compass gait 
model, was proposed (Fig. 75.6b, [75.44]; l<s&Wil>]j'imil 
shows an experiment of a compass gait bipedal robot 
locomotion on rough terrains). In contrast to the rim¬ 
less wheel model that considers only stance leg dynam¬ 
ics during walking, the compass gait model has three 
masses with a passive hip joint, which allow us to in¬ 
vestigate swing leg dynamics in addition to the stance 
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Fig.75.6a-h Examples of bio-inspired legged robots, (a) A physical implementation of rimless wheel walking model 
(after [75.34]), (b) A physical implementation of compass gait walking model (after [75.35,36]), (c) passivity based 
bipedal walking robot (after [75.37]), (d) passive dynamic runner (after [75.38]), (e) energy efficient hopping robot 
(after [75.39]), (f) bipedal walking and running robot based on biarticlar springs (after [75.40]), (g) biped robot based on 
variable stiffness actuators (after [75.41]), and (h) hexapod robot based on spring mass dynamics (after [75.42]) 


leg [75.45]. An important implication of this line of re¬ 
search lies in the fact that, because of the simple formu¬ 
lations of complex dynamics, researchers were able to 
systematically investigate different aspects of the com¬ 
plex behaviors while keeping an overarching structure 
of research issues. These simple models were, for exam¬ 
ple, gradually and systematically enhance by integrating 
knee and ankle joints (Fig. 75.6c; [75.37,43]), foot seg¬ 
ments with variations of shapes, influences of mass dis¬ 
tributions [75.46], influence of lateral motions, as well 
as a variety of advanced motion control architectures to 
demonstrate actuated locomotion in more complex en¬ 
vironments [75.35, 36,47, 48]. 

Although walking dynamics is a highly interest¬ 
ing challenge, the basic locomotion stability in walking 
is not sufficient to understand legged locomotion in 
nature, but stability has to be also maintained in dif¬ 
ferent gait patterns, such as running because most 
of biological systems exhibit a rich variety of gaits. 
For this reason, running dynamics has also been stud¬ 
ied intensively by investigating a simple model, i. e., 
the so-called spring-loaded inverted pendulum (SLIP 
model; [75.28,49,50]). The model consists of a point 
mass and a massless linear spring, on top of which 
many variations were proposed such as running models 
with nonlinear spring, segmented legs [75.51], swing 
leg dynamics, upper torso, lateral balancing [75.52], 
and wheel-like configuration (Fig. 75. 6h [75.42]), for 
example. The SLIP model was also used to study walk¬ 


ing dynamics and gait transitions between walking and 
running (Fig. 75. 6f [75.40,53]; l-g&M'iMHif shows an 
example of a biped robot walking and running). It is 
important to note that many of these models and robots 
were developed for the hypotheses difficult to test in bi¬ 
ological systems. The biological legs are hardly linear 
springs but they consist of numerous active and nonlin¬ 
ear components. However, by examining these models 
and robots, we are able to learn the basic underlying 
principles such as the degree to which a spring-like 
behavior of legs could contribute to the stability of 
walking and running locomotion, for example. In addi¬ 
tion, such an abstraction of biological body structures 
is very practical for robotics research as we are able 
to design and construct robots based on the underly¬ 
ing principles without replicating complex anatomical 
structures consisting of organic components. 

Energy Efficiency and Bio-Inspired Actuation 
Another considerable challenge in the study of bio¬ 
inspired legged locomotion is the principles for energy 
efficiency. It has been known that the locomotion ef¬ 
ficiency of biological systems is known to be at least 
an order of magnitude better than most of the legged 
robots today but it is not fully understood why bio¬ 
logical locomotion is so efficient. The complexity of 
the energetic problem in legged locomotion is origi¬ 
nated in the many possible sources of energy dissipation 
such as frictional and damping losses in joints and mus- 
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cles, mechanical impact losses at foot touchdown to the 
ground, metabolic costs, and energy required for ac¬ 
celeration and deceleration of body parts. Because of 
such a complexity, the bio-inspired robotics research 
has been significantly contributing to this problem by 
building and analyzing, for example, purely mechanical 
locomotion systems [75.38,43], underactuated loco¬ 
motion control [75.35, 37], the use of passive spring 
and self-excited vibration [75.39,54], and exoskeleton 
devices [75.55]. All of these case studies were con¬ 
tributing to a comprehensive understanding of energy 
efficiency in biological locomotion [75.56], and some 
of the hypotheses have been analyzed and tested in bio¬ 
logical systems. 

In addition to the whole body dynamics, energy effi¬ 
cient locomotion has also been investigated at the level 
of actuation because the muscle-tendon systems play 
a major role in animals’ efficient locomotion. Inspired 
from the biological models of muscles, this research 
trend started from the so-called series elastic actuators, 
which is an actuator unit containing a mechanical spring 


being installed in series to an electric motor [75.62]. 
The implementation of mechanical spring in an electric 
motor explained the unique characteristics such as stor¬ 
age of kinetic energy to elastic energy, shock absorption 
to protect mechanical transmission, and force-based 
feedback control, all of which are favorable for both 
biological and artificial legged systems. More recently, 
many researchers have been attempting to enhance the 
actuation mechanisms with variable stiffness and damp¬ 
ing capabilities [75.41,63] that are also expected to 
provide valuable insights into the roles of muscle prop¬ 
erties in efficient legged locomotion. 

75.3.2 Reflexes and Central Pattern 
Generators 

Agility and adaptability of animals’ locomotion are not 
only originated in mechanical dynamics as explained 
in the previous subsection, but there are also highly 
complex control systems regulating animals’ motions. 
Usually animals control systems are labeled into four 




Fig.75.7a-f Robotic implementation of CPG models, (a) A salamander robot consisting of motorized modules, and (b) its control 
architecture. Every module has a servomotor which is controlled through a simulated CPG model, (c) Typical behavioral response 
of the CPG model (upper figure) with respect to a control signal (lower figure). One control input is sufficient to control smooth 
oscillations of multiple body segments because of coupled dynamics originated in the CPG model (after [75.57]). (d,e) A hexapod 
and biped robots that use CPG models to adapt to changes in the environment (after [75.58-60]). (f) A musculoskeletal humanoid 
robot that simulate developmental processes of human babies (after [75.61]) 
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components: the musculoskeletal system, reflexes, cen¬ 
tral pattern generators (CPGs), and modulation by 
higher control centers. Bio-inspired robotics has been 
investigating these components individually or in com¬ 
bination, and resulted in a number of demonstrations of 
surprisingly robust robot locomotion (Fig. 75.7). 

This research area has been historically investigated 
by the two distinguished approaches, i. e., the CPG- 
based approach and the reflex-based one. The former 
approach usually considers voluntary oscillations of 
neural circuits, that is, CPGs, and the output of these cir¬ 
cuits triggers locomotion cycles. Because these behav¬ 
iors are found in the spinal cord of vertebrate animals 
and in ganglions in invertebrates [75.64], this approach 
has been very popular in the bio-inspired robotics. In 
contrast, the reflex-based approach does not incorporate 
intrinsic oscillators and generate periodic behavior as 
a chain of reflex-mediated events. While conceptually 
different, most of the CPG-based approach considers re¬ 
flexes in the neural processes; thus these two approaches 
are often overlapping and reflex-based approaches can 
be seen as a subset of CPG-based approaches. 

One of the pioneering works in the reflex-based 
approach was based on the neural circuits underlying 
walking identified in stick insects [75.65,66]. In this 
study, it was identified that a series of reflexes for 
each leg use information related to leg postures and 
ground contact to generate movements. Coordination 
between legs (i. e., specific gaits) is obtained with di¬ 
rect neural couplings between individual legs circuits 
and also through mechanical couplings (e.g., the move¬ 
ment of one limb affecting the load on other limbs). 
This leads to a decentralized control mechanism, similar 
in spirit to those developed in behavior-based robotics, 
that has been validated on simulated and real hexapod 
robots [75.65, 67]. 

The reflex-based approach was also investigated in 
human walking, in which a series of neuromechanical 
models were developed to demonstrate how the combi¬ 
nation of muscle properties and low-level reflexes can 
lead to stable locomotion in simulated bipeds [75.68, 
69]. In particular, Geyer and Herr [75.69] present a sim¬ 
ulated bipedal walker which manages to walk in the 
sagittal plane and to be stable against slight slopes. This 
model captures principles of neuromuscular feedbacks 
and predicts muscle activation patterns observed in leg 
muscles. Similar reflex-based controllers (without mus¬ 
cle models but with simulated synaptic plasticity) have 
successfully been ported to real robots such as the Run- 
bot [75.60] and the dynamic walkers reported in [75.37], 

One of the first examples of the CPG approach has 
been done in simulation by Taga and colleagues [75.70, 
71]. They developed a series of 2-D models of biped 
locomotion that combine a simple musculoskeletal 


model with a CPG modeled as a system of cou¬ 
pled Matsuoka oscillators [75.72], The work showed 
how bidirectional couplings between the CPG and 
the musculoskeletal model could lead to entrainment 
(i. e., frequency locked regimes) between the two as 
well as to robust locomotion. Since then a large 
number of CPG-based controllers have been imple¬ 
mented in robots for different types of locomotion. 
Examples include hexapod and octopod robots [75.73— 
75] (see also Robot Roach swim¬ 

ming robots [75.76-78] (see also Salamandra Robotica 
quadruped robots [75.57, 79, 80], and 
biped robots [75.81-85]. 

As discussed in [75.64], a CPG-based approach has 
several interesting properties: 

i) Stable limit cycle behavior that provides robustness 
against perturbations. 

ii) Suitability for a distributed implementation. 

iii) Possibility to modulate gaits with a few control pa¬ 
rameters. 

iv) Integration of sensory feedback signals in order to 
obtain mutual entrainment between the CPG and the 
mechanical body. 

v) Suitable substrate for learning and optimization 
algorithms. These properties were particularly dif¬ 
ficult to investigate by using animals; hence the 
bio-inspired robotics has provided significant con¬ 
tributions to the nature of motion control in biolog¬ 
ical systems. 

75.3.3 Bio-Inspired Navigation 

Biological systems use a variety of cognitive pro¬ 
cesses for their navigation in complex environment: it 
is known, for example, that animals use both propri¬ 
oceptive and exteroceptive receptors for sensing envi¬ 
ronments; the obtained sensory information is passed to 
massively parallel processes distributed over many hi¬ 
erarchical levels in central nervous systems; animals’ 
perception of the world is coordinated with low-level 
sensory-motor processes; and in addition to these mech¬ 
anisms, long-term planning and learning processes are 
continuously running to achieve more advanced tasks 
such as goal-directed navigation. In order to tackle such 
a complex problem of animals’ navigation, bio-inspired 
robotics also provided a set of effective tools to apply 
the synthetic methodology some of which are outlined 
in this subsection. 

Sensor Morphology 
and Sensory-Motor Coordination 
A significant number of researchers in bio-inspired 
robotics have been working on relatively simple animals 
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such as insects because their central nervous systems 
are far more tractable than other animals. Despite the 
simplicity of their brains, insects are incredible naviga¬ 
tors being capable of avoiding obstacles while running 
or flying with enormous speed, recognizing landmarks 
in unstructured environment, and traveling long distance 
for foraging. Moreover, some of the social insects can 
even learn to go back to their own nests, and communi¬ 
cate with co-workers for efficient community manage¬ 
ment. Although biologists have been investigating these 
fascinating animals for centuries, there are still a num¬ 
ber of issues that are not fully clarified yet, and among 
others, robotics platforms were used to investigate the 
mechanisms in which physical system-environment in¬ 
teractions play central roles [75.86]. 

One of the most successful case studies in bio¬ 
inspired navigation research was on the mechanisms 
of sensory-motor coordination in flying insects such as 
flies and bees. These insects are known to rely on vi¬ 
sual sensory information, more specifically, optic flow 
to detect ego motions, stabilize body posture, measure 
distance to various objects and landing spots, and track 
traveling distance, for example. While the visual in¬ 
formation processing is usually regarded as computa¬ 
tionally expensive, many insects, which have very lim¬ 
ited computational resources, make use of this modality 
to achieve these behavioral functions. The underlying 
mechanisms are found in the hardware setups of an¬ 
imals, in which sensor morphology (i. e., how recep¬ 
tors are distributed) and low-level sensory information 
processing are exploited in the coordination of sensory- 
motor processes [75.87]. More specifically, the photore¬ 
ceptors of these insects are usually distributed to almost 
all directions which give rise to surprisingly informa¬ 
tive stimuli about the environment and ego motions, and 
low-level neural circuits are configured such that ex¬ 
tremely low-processing power is necessary. To test these 
hypotheses, a number of robotic platforms were devel¬ 
oped and tested previously, which showed the feasibility 
of these mechanisms such as the optic flow to detect 
nearby objects [75.88], visual odometry [75.89], flight 
altitude [75.90], and flight stabilization [75.91]. 

Technological advances are essential to gain ad¬ 
ditional insights into the complex sensory-motor pro¬ 
cesses in the animals. At the beginning of the investi¬ 
gations, many researchers developed omni-directional 
vision based on specifically shaped mirrors attached to 
regular cameras, while recently more advanced tech¬ 
nologies are being developed to flexibly adjust pho¬ 
toreceptors [75.92]. Neuromorphic engineering also 
provides an additional enabling technology to ex¬ 
plore physical foundations of biological nervous sys¬ 
tems [75.93,94]. Neuromorphic silicon retinas are, un¬ 
like conventional visual sensors, able to process sensory 


information extremely fast and computationally less de¬ 
manding owing to the event driven and asynchronous 
processing architectures, while keeping sensitivity very 
high (i. e., the receptors can be sensitive in very dark 
environments as well as in a very bright one, [75.95]). 
With the technological progress, we will be able to 
reproduce more precise landscape of the world from in¬ 
sects’ viewpoint for more comprehensive investigation 
of bio-inspired navigation. 

Goal-Directed Navigation 
Compared to reflexive behaviors, goal-directed ones are 
significantly more complex where much less is known 
even in biology. In nature, goal-directed behaviors such 
as navigation to a nest from a distant location require 
learning of routes and locations, short- and long-term 
memories, episodic memories, while flexibly adapting 
to unstructured and often dynamically changing envi¬ 
ronments. The underlying mechanisms are related to 
many different locations in the central nervous systems 
and they vary one species to another, thus the ongo¬ 
ing researches are essentially driven on the basis of the 
important findings in biology, rather than developing 
a unified and generalized framework. 

One of the representative case studies on bio¬ 
inspired goal-directed navigation was again conducted 
in relation to insect behaviors (Fig. 75.8): some social 
insects such as desert ant Cataglyphis are known to ex¬ 
hibit the so-called visual homing behaviors, in which 
the animals go back to their nests by using visual cues 
nearby [75.96]. These insects usually walk randomly 
when searching for food sources, while they go back 
straight to the nest by using visual cues. Although the 
neural basis of these behaviors is not yet identified, the 
biologists argue that an abstract model, the so-called 
average landmark vector method, explain the insects’ 
behavior fairly accurately. Here it is assumed that in¬ 
sects know the global orientation in the world, and 
every once in a while, they perceive the direction of for¬ 
aging behavior stored as vector information. Over time, 
the animals sum up the vectors so that they can keep 
track on the direction to the nest while randomly search 
for the food sources. The biologists have been explor¬ 
ing these behaviors for decades, and the accumulated 
knowledge and hypothetical models were implemented 
and tested in physical robot platforms. Unlike most of 
the simulation experiments, the implementation to the 
robots helped to test the hypothesis in the real-world 
desert environment [75.96], or physically implemented 
in an analog circuitry (see also multimedia material of 
Analog Robot navigation in |4S>M!I0$Ea; [75.97]). 

A significantly more challenging problem is to 
identify the mechanisms of goal-directed behaviors in 
more complex animals, especially in mammals. There 
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Fig.75.8a-d Visual homing of insect-inspired robots, (a) An result of navigation experiment of desert ant Cataglyphis. 
(b) A mobile robot Sahabot II that was developed to investigate navigation mechanisms of Cataglyphis. The robot 
is equipped with an omni-directional camera and a digital compass that can be used for the bio-inspired landmark 
navigation (after [75.96]). (c) A fully analog implementation of the visual homing algorithm, (d) Simulation results of 
visual homing algorithm (after [75.97]) 


is a large body of literature about this issue includ¬ 
ing cognitive science and brain science, but one of the 
most prominent contributions of robotic platforms in 
this research area was to explore neural dynamics dur¬ 
ing physical system-environment interactions. The re¬ 
search was originally motivated by a discovery in phys¬ 
iology such as the so-called place cells, i. e., a group 
of neurons in hippocampus exhibit unique behaviors 
whenever the animal is in a specific location in an en¬ 
vironment [75.98]. This hard evidence in neuroscience 
has been widely used to analyze how brains func¬ 
tion in the context of spatial cognition and navigation 


in general, including those investigating computational 
neuroscience and bio-inspired robotics. Essentially, it is 
still a challenge to explain the behaviors of place cells 
because they involve sensory-motor activities as well 
as temporal changes of neural activities (i. e., learning 
of sensory motor activities) thus a synthetic methodol¬ 
ogy is extremely helpful. So far it has been shown that 
the computational models of hippocampus were imple¬ 
mented onto some mobile robot platforms to replicate 
the behaviors of place cells in navigation tasks [75.99, 
100] as well as some more complex goal-oriented be¬ 
haviors and learning [75.101]. 


75.4 Landscape of Bio-Inspired Robotics Research and Challenges 


So far we introduced only a few representative and on¬ 
going case studies of bio-inspired robotics research, but 
there are many other active topic areas in the field. 
Although many of these studies are covered also in 
the other chapters of this handbook, this section pro¬ 
vides a brief overview of the relevant topics in which 
robotic technologies are being used as scientific tools 
for biology. 


75.4.1 Bio-Inspired Climbing 

When legged systems are reduced down to smaller 
scales, adhesion forces become more dominant than 
the gravitational, and for this reason, the small-sized 
animals in nature such as insects, amphibians, and 
lizards tend to climb terrains rather than walk on level 
grounds. While the governing physics in the climbing 
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Fig.75.9a-h Examples of recent bio-inspired robots, (a) A mobile robot that uses active whisker arrays for navigation 
(after [75.102]). (b) A climbing robot based on feet made of dry adhesives (after [75.103]). (c) Micro robot for flapping 
flight developed by the micro-fabrication techniques (after [75.104]). (d) A fish-like swimming robot that exploits soft 
continuum body structure (after [75.105]). (e) A worm-like robot that exhibit rolling-locomotion of soft body structure 
(after [75. 106]). (f) A humanoid robot that is equipped with soft skin for interactions with human partners (after [75. 107]). 
(g) A wheel-chair controlled by brain signals of the user (after [75.108]). (h) Self-reconfigurable robot that is capable of 
autonomously changing its own body structure (after [75.109]) 


locomotion is different from that of gravity-oriented 
legged locomotion, robotic platforms are also useful 
because the dynamics during the locomotion is sim¬ 
ilarly complex. One of the most representative case 
studies in this line of research was the use of dry 
adhesives in climbing robots that are inspired from 
geckos. Many research topics focused on the fabri¬ 
cation techniques of micro hair-like structures that 
can generate adhesive forces for a series of small¬ 
sized robots (Fig. 75.9b, [75.103, 110]). Similarly a few 
other approaches were also proposed to explore the 
different climbing strategies of animals including the 
use of material-dependent adhesion [75.111], rough- 
surface locomotion by using feet with micro spine 
structures [75.112,113], and climbing strategy based 
on force closure of relatively long legs [75.114]. There 
still exist many challenges in fabrication techniques 
of micro structures in order to replicate the sophisti¬ 
cated climbing mechanisms of animals, which requires 
continuing close collaborations between researchers in 
biology and robotics. 

75.4.2 Flapping Flight 

and Swimming Mechanisms 

Another complex, yet popular, dynamics used in ani¬ 
mal kingdom is fluid/aerodynamics that are typically 
observed in flying and swimming systems. Fluid/ 
aerodynamics are also dynamics difficult to model and 
simulate thus robotic platforms are intensively em¬ 


ployed for exploring underlying mechanisms [75.115, 

116] . As is the case of walking and running on land, 
mechanical dynamics also play an important role in 
flying and swimming locomotion and a number of 
underactuated robots were developed to understand 
the nature of locomotion in fluid Fig. 75. 9d [75.105, 

117] . As the microfabrication techniques evolved in 
the recent years, roboticists and biologists also started 
collaborating to investigate small-sized flying robots 
(Fig. 75.9c, [75.104]; Chap. 26). 

75.4.3 Artificial Hands, Haptics, 
and Whiskers 

Haptic perception is known to be one of the most 
important sensor modalities in biological systems, al¬ 
though the biological nature is far from a compre¬ 
hensive understanding because animals make use of 
complex sensory-motor interactions for the purpose of 
tactile sensing [75.1 18, 119]. Haptic sensing can be de¬ 
fined as sensing of mechanical environment through 
touch although there are many different variations in 
nature including tactile sensing through fingers and 
skins [75.119], active whisking (Fig. 75.9a) [75.102], or 
more specifically targeted sensing such as slippage de¬ 
tection [75.120]. Exploration on haptic technologies is 
also crucial in this research area as the biological tactile 
sensing involves an enormous number of mechanore- 
ceptors each of which has a large sensitivity range. 
Currently, a number of researchers are actively inves- 
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tigating technological solutions through haptic devices 
and soft and stretchable electronics for tactile sens¬ 
ing [75.121,122], 

75.4.4 Self-Reconfigurable 

and Evolutionary Robotics 

Animals’ adaptivity in nature relies significantly on 
their capability of changing their body sizes and struc¬ 
tures. Animals are, for example, able to start their 
lives smaller and gradually grow larger and more 
complex; they are able to self-repair or regenerate 
when encountering failures in body parts; and muscles 
and skins are able to strengthen themselves if neces¬ 
sary [75.123]. Synthetic methodology has also been 
employed to investigate these fascinating capabilities 
of biological systems by using, for example, modular 
robots Fig. 75. 9h [75.109,124,125], redundant body 
structures for snake-like motion control [75.126,127], 
and self-repair and self-assembly of structures [75.128, 
129]. Due to the technological limitations, many re¬ 
searchers take advantage of simulation-based methods 
to explore the ontogenetic and evolutionary processes 
to uncover the characteristics of optimization strategies 
in nature [75.19,20, 130]. 

75.4.5 Bio-Inspired Soft Robots 

Unlike the conventional robots that are usually made 
of rigid materials articulated into discrete pieces, ani¬ 
mals’ body structures mostly consist of soft, continuum. 


75.5 Conclusion 

This chapter introduced a class of bio-inspired robotics 
research that is specifically targeted to deepen our 
understanding of biological systems. Through the rep¬ 
resentative case studies, we explained how bio-inspired 
robotics research can be useful not only for develop¬ 
ing innovative robots, but also for exploring uncovered 
challenges in biology by employing the understanding- 
by-building approach. There are, however, a set of 
important concepts that need to be considered for suc¬ 
cessful collaborations between robotics and biology. 
Specifically: 

1. It is necessary to take the similarities and differ¬ 
ences in robots and animals into account. 

2. There are different goals and methods to develop 
models of biological systems. 

3. The use of robots as a scientific tool has both advan¬ 
tages and disadvantages. 


and elastic components such as muscles, tendons, skins, 
organs surrounded by smooth membrane [75.1]. Re¬ 
cently, there has been an increasing interest in the 
use of soft deformable materials in robotic systems 
to enhance capabilities of, for example, soft locomo¬ 
tion [75.106,131] (I^JBZEBilEa shows and example 
of soft robot locomotion), manipulation [75.132, 133], 
shape adaptation [75.134], and soft human-robot inter¬ 
actions [75.63]. Despite its demand in the robotics and 
biological studies, there are still a number of techno¬ 
logical challenges in this field such as soft actuation 
and sensing [75.135], simulation of soft deformable 
structures [75.136], and control of flexible continuum 
bodies [75.137], 

75.4.6 Neuroprosthetics and Social 
Interactions 

As we develop more technological components com¬ 
patible to biological systems, there are more possibili¬ 
ties to implant artificial devices into biological bodies. 
Although most of the case studies in this research 
area aim at bio-medical applications as exemplified by 
visual/auditory prosthetics, pain relief, and motor pros¬ 
thetics, there are also intensive investigations on the 
use of prosthetic devices to gain additional insights into 
the nature of motion control [75.108, 138] and percep¬ 
tion [75.139]. There is also an increasing interest in the 
use of robotic platforms in the studies of social inter¬ 
actions where robots are used to study communications 
with humans [75.107] and the other animals [75.140]. 


4. There are types of hypotheses in biology that bio¬ 
inspired robotics can be particularly beneficial for. 

And finally, we also introduced a concise landscape 
of trends and challenges in bio-inspired robotics. As 
mentioned earlier, the field of bio-inspired robotics is 
very broad, and the outline introduced in this chapter 
is by no means complete. For example, although this 
chapter only focused on the types of research which 
contribute to biological studies, there is a large body of 
the literature on bio-inspired robotic applications which 
are mostly ignored in this chapter. The interested read¬ 
ers should refer to the other chapters in this handbook as 
well as the other review articles in the field. Also, there 
are significantly more case studies available in literature 
which reported on the different species or the other as¬ 
pects of animals which were summarized in [75.15, 18, 
64], 
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Dynamic rolling locomotion of GoQBot 

available from http://handbookofrobotics.org/view-chapter/75/videodetails/109 
JenaWalker - Biped robot with biologically-inspired bi-articular springs 
available from http://handbookofrobotics.org/view-chapter/75/videodetails/110 
MIT Compass Gait Robot - Locomotion over rough terrain 
available from http://handbookofrobotics.org/view-chapter/75/videodetails/111 
RobotRoach with adaptive gait pattern variations 

available from http://handbookofrobotics.org/view-chapter/75/videodetails/112 
Salamandra Robotica II - Swimming to walking transition 
available from http://handbookofrobotics.org/view-chapter/75/videodetails/113 
Analog Robot 

available from http://handbookofrobotics.org/view-chapter/75/videodetails/242 
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76. Evolutionary Robotics 


Stefano Nolfi, Josh Bongard, Phil Husbands, Dario Floreano 


Evolutionary Robotics is a method for automati¬ 
cally generating artificial brains and morphologies 
of autonomous robots. This approach is useful 
both for investigating the design space of robotic 
applications and for testing scientific hypothe¬ 
ses of biological mechanisms and processes. In 
this chapter we provide an overview of methods 
and results of Evolutionary Robotics with robots of 
different shapes, dimensions, and operation fea¬ 
tures. We consider both simulated and physical 
robots with special consideration to the transfer 
between the two worlds. 
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76.5 Evolving Bodies . 2044 


Evolutionary robotics is a method for the automatic cre¬ 
ation of autonomous robots [76.1]. It is inspired by 
the Darwinian principle of selective reproduction of 
the fittest, captured by evolutionary algorithms [76.2], 
In evolutionary robotics, robots are considered as au¬ 
tonomous artificial organisms that develop their own 
control system and body configuration in close interac¬ 
tion with the environment without human intervention. 
Drawing inspiration from principles of biological self¬ 
organization, evolutionary robotics includes elements 


of evolutionary, neural, developmental, and morpho¬ 
logical systems. The idea that an evolutionary process 
could drive the generation of control systems dates back 
to at least the 1950s [76.3] with a more explicit form 
appearing in the mid 1980s with the ingenious thought 
experiments by neuroscientist Valentino Braitenberg 
on neurally driven vehicles [76.4]. In the early 1990s, 
the first generation of simulated artificial organisms 
with a genetic code describing the neural circuitry and 
morphology of a sensory motor system began evolv- 
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ing on computer screens [76.5-8]. At that time, real 
robots were still complicated and expensive machines 
that required specialized programming techniques and 
skillful manipulation. Towards the end of that period, 
a new generation of robots started to emerge that shared 
important characteristics with simple biological sys¬ 
tems: robustness, simplicity, small size, flexibility, and 
modularity [76.9, 10]. Above all, those robots were 


designed so that they could be programmed and manip¬ 
ulated by people without engineering training. Those 
technological achievements, together with the growing 
influence of biological inspiration in artificial intel¬ 
ligence [76.11], coincided with the first evolutionary 
experiments on real robots [76.12-14] 
and and the term evolutionary robotics 

was coined [76.15]. 


76.1 Method 

The major methodological steps in evolutionary 
robotics proceed as follows (Fig. 76.1). An initial 
population of different artificial chromosomes, each 
encoding the control system (and possibly the mor¬ 
phology) of a robot, is randomly created. Each of 



Fig. 76.1 Evolutionary experiments on a single robot. Each indi¬ 
vidual of the population is decoded into a corresponding neurocon¬ 
troller which reads sensory information and sends motor commands 
to the robot every 300 ms while its fitness is automatically evaluated 
and stored away for reproductive selection 


these chromosomes is then decoded into a correspond¬ 
ing controller, for example a neural network (NN), 
and downloaded into the processor of the robot. The 
robot is then let free to act (move, look around, ma¬ 
nipulate the environment) according to a genetically 
specified controller while its performance for a given 
task is automatically evaluated. Performance evaluation 
is done by a fitness function that measures how fast 
and straight the robot moves, how frequently it col¬ 
lides with obstacles, etc. This procedure is repeated 
for all chromosomes of the population. The fittest in¬ 
dividuals (those that have received more fitness points) 
are allowed to reproduce by generating copies of their 
chromosomes with the addition of random modifica¬ 
tions introduced by genetic operators (e.g., mutations 
and exchange of genetic material). The newly ob¬ 
tained population is then tested again on the same 
robot. This process is repeated for a number of gen¬ 
erations until an individual is born which satisfies the 
fitness function set by the user. The control system 
of evolved robots, encoded in an artificial genome, is 
therefore generated by a repeated process of selective 
reproduction, random mutation, and genetic recombi¬ 
nation, similarly to what happens in natural evolution 
(|<Et>MM£IIEI). 


76.2 First Steps 

In an early experiment on robot evolution without hu¬ 
man intervention, carried out at Ecole Polytechnique 
Federale de Lausanne (EPFL) [76.12], a small wheeled 
robot was evolved for navigation in a looping maze 
(Fig. 76.2). The Khepera robot has a diameter of 55 mm 
and two wheels with controllable velocities in both di¬ 
rections of rotation. It also has eight infrared sensors, 
six on one side and two on the other side, that can 
function either in active mode to measure distance from 
obstacles or in passive mode to measure the amount of 
(infrared) light in the environment. The robot was con¬ 


nected to a desktop computer through rotating contacts 
that provided both power supply and data exchange 
through a serial port (l<£aJfc!ii!l£EB). 

A simple genetic algorithm [76.16] was used to 
evolve the synaptic strengths of a neural network com¬ 
posed of eight sensory neurons and two motor neurons. 
Each sensory unit was clamped to one of the eight 
active infrared sensors whose value was updated ev¬ 
ery 300 ms. Each motor unit received weighted signals 
from the sensory units and from the other motor unit, 
plus a recurrent connection with itself with a 300 ms 












Evolutionary Robotics I 76.2 First Steps 2037 


delay. The net input of the motor units was offset by 
a modifiable threshold and passed through a logistic 
squashing function. The resulting outputs, in the range 
[0,1], were used to control the two motors so that 
an output of 1 generated maximum rotation speed in 
one direction, an output of 0 generated maximum ro¬ 
tation speed in the opposite direction, and an output 
of 0.5 did not generate any motion in the correspond¬ 
ing wheel. A population of 80 individuals, each coding 
the synaptic strengths and threshold values of the neural 
controllers, was initialized with all weights set to small 
random values centered around zero. Each individual 
was tested on the physical robot for 80 sensorimotor cy¬ 
cles (approximately 24 s) and evaluated at every cycle 
according to a fitness function with three components 
measured onboard the robot 

<p = V(l-VAu)(l-0 , (76.1) 

where V is the average rotation speed of the two wheels, 
A v is the absolute value of the algebraic difference be¬ 
tween the signed speed values of the wheels (positive 
is one direction, negative the other), and i is the nor¬ 
malized activation value of the infrared sensor with the 
highest activity. The first component is maximized by 
speed, the second by straight motion, and the third by 
distance from objects. 

During the first 100 generations, both average and 
best fitness values grew steadily, as shown in Fig. 76.3. 
A fitness value of 1.0 would correspond to a robot mov¬ 
ing straight at maximum speed in an open space and 
therefore was not attainable in the looping maze shown 
in Fig. 76.2, where some of the sensors were often ac¬ 
tive and where several turns were necessary to navigate. 
Fig. 76.4 shows the trajectory of the best individual of 
the last generation. 

Although the fitness function did not specify in 
what direction the robot should navigate (given that it 



Fig. 76.2 Bird’s-eye view of the desktop Khepera robot in 
the looping maze 


was perfectly circular and that the wheels could ro¬ 
tate in both directions), after a few generations all the 
best individuals moved in the direction corresponding 
to the side with the highest number of sensors. In¬ 
dividuals moving in the other direction had a higher 
probability of colliding into corners without detecting 
them and thus disappeared from the population. Fur¬ 
thermore, the cruising speed of the best evolved robots 
was approximately half of the maximum speed that 
could be technically achieved and did not increase even 
when the evolutionary experiment was continued up 
to 200 generations. Further analysis revealed that this 
self-limitation of the navigation speed had an adaptive 
function because, considering the sensory and motor 

Fitness 



Fig. 76.3 Average fitness of the population and fitness of the best 
individual at each generation (error bars show standard error over 
three runs from different initial populations) 



Fig. 76.4 Trajectory of the robot with the best neural con¬ 
troller of the last generation. Segments represent the axis 
between the two wheels. Data were recorded and plotted 
every 300 ms using an external laser positioning device 
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refresh rate together with the response profile of the dis¬ 
tance sensors, robots that traveled faster had a higher 
risk of colliding with walls before detecting them; they 
gradually disappeared from the population. 

Despite its simplicity, this experiments shows that 
evolution can discover solutions that match not only the 
computational requirements of the task to be solved, but 
also the morphological and mechanical properties of the 
robot in relation to its physical environment. 

76.2.1 Evolution of Neural Controllers 
for Walking 

Over the past 20 years or so, there has been a grow¬ 
ing body of work on evolving controllers for various 
kinds of walking robots - a nontrivial sensorimotor co¬ 
ordination task. Early work in this area concentrated 
on evolving dynamical network controllers for simple 
(abstract) simulated insects (often inspired by cock¬ 
roach studies) which were required to walk in simple 
environments [76.17,18]. Earlier, Beer et al. had in¬ 
troduced a neural architecture for locomotion based 
on studies of cockroaches [76.19], which is shown 
in Fig. 76.5. The promise of this work soon led to ver¬ 
sions of this methodology being used on real robots. 
Probably the first success in this direction was by Lewis 
et al. [76.14,20] who evolved a neural controller for 
a simple hexapod robot using coupled oscillators built 
from continuous-time, leaky-integrator, artificial neu¬ 
rons. All evaluations were done on the actual robot with 
each leg connected to its own pair of coupled neurons, 
leg swing being driven by one neuron and leg eleva¬ 
tion by the other. These pairs of neurons were cross 
connected, in a manner similar to that used by Beer 
and Gallagher [76.18] (Fig. 76.5), to allow coordina¬ 
tion between the legs. In order to speed up the process, 
they employed staged evolution where first an oscillator 
capable of moving a leg was evolved and then an archi¬ 
tecture based on these oscillators was further evolved 
to develop walking. The robot was able to execute an 
efficient tripod gait on flat surfaces. 

Gallagher et al. [76.21] described experiments 
where neural networks controlling locomotion in an 
artificial insect were evolved in simulation and then suc¬ 
cessfully downloaded onto a real hexapod robot. This 
machine was more complex than Lewis et al.’s, with 
a greater number of degrees of freedom per leg. In this 
approach, each leg was controlled by a fully connected 
network of five continuous-time, leaky-integrator neu¬ 
rons, each receiving a weighted sensory input from 
that leg’s angle sensor. Initially the architecture shown 
in Fig. 76.5 was used, with the connection weights and 
neuron time constants and biases under genetic control. 
This produced efficient tripod gaits for walking on flat 


surfaces. In order to produce a wider range of gaits op¬ 
erating at a number of speeds such that rougher terrain 
could be successfully negotiated, a different distributed 
architecture, more inspired by stick insect studies, was 
found to be more effective [76.22]. 

Galt et al. [76.23] used a genetic algorithm to de¬ 
rive the optimal gait parameters for a Robug III robot, 
an eight-legged, pneumatically powered walking and 
climbing robot. The individual genotypes represented 
parameters defining each leg’s support period and the 
timing relationships between leg movements. These 
parameters were used as inputs to a mechanistic finite- 
state machine pattern-generating algorithm that drove 
the locomotion. Such algorithms, which are often used 
in conventional walking machines, rely on relatively 
simple control dynamics and do not have the same 
potential for the kind of sophisticated multigait coor¬ 
dination that complex dynamical neural network archi- 



Fig. 76.5 Schematic diagram of a distributed neural net¬ 
work for the control of locomotion as used by Beer 
et al. [76.19], Excitatory connections are denoted by open 
triangles and inhibitory connections are denoted by filled 
circles. C, command neuron; P, pacemaker neuron; FT, 
foot motor neuron; FS and BS, forward swing and back¬ 
ward swing motor neurons; FAS and BAS, forward and 
backward angle sensors 
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tectures, such as those described in this section, have 
been shown to produce. However, controllers were suc¬ 
cessfully evolved for a wide range of environments and 
to cope with damage and systems failure (although an 
individual controller had to be tuned to each environ¬ 
ment; they were not able to self-adapt across a wide 
range of conditions). Gomi and Ide [76.24] evolved the 
gaits of an eight-legged robot (Fig. 76.6) using geno¬ 
types made of eight similarly organized sets of genes, 
each gene coding for leg motion characteristics such 
as the amount of delay after which the leg begins to 
move, the direction of the leg’s motion, the end po¬ 
sitions of both vertical and horizontal swings of the 
leg, and the vertical and horizontal angular speed of 
the leg. After a few dozen generations, where evalua¬ 
tion was on the robot, a mixture of tetrapod and wave 
gaits was obtained. Using the cellular encoding [76.25] 
developmental approach - which genetically encodes 
a grammar-tree program that controls the division of 
cells growing into a dynamical recurrent neural net¬ 
work of the kind used by Beer et al. - Gruau and 
Quatramaran [76.26] evolved a single-leg neural con¬ 
troller for the same eight-legged robot used by Gomi 
and Ide. This generated a smooth and fast quadru- 
pod locomotion gait (l<s>3E135E3l and |43>)KMi!£i&l). 
Kodjabachian and Meyer [76.27] extended this work 
to develop more sophisticated locomotion behaviors. 
Jakobi [76.28] successfully used his minimal simu¬ 
lation techniques (described in Sect. 76.3) to evolve 
controllers for the same eight-legged robot as Gruau. 
Evolution in simulation took less than 2h on what 
would today be regarded as a very slow computer, 
and was then successfully transferred to the real robot. 
Jakobi evolved modular controllers based on Beer’s 
continuous recurrent networks to control the robot as 
it engaged in walking about its environment, avoiding 
obstacles and seeking out goals depending on the sen¬ 
sory input. The robot could smoothly change gait, move 
backward and forward, and even turn on the spot. More 
recent work has used similar architectures to those ex¬ 
plored by the researchers mentioned above, to control 
more mechanically sophisticated robots such as the 
Sony Aibo [76.29]. 

Recently there has been successful work on evolv¬ 
ing coupled oscillator style neural controllers for 
the highly unstable dynamic problem of biped walk¬ 
ing. Red and Husbands [76.30] showed that accurate 
physics based simulations employing physics-engine 
software could be used to develop controllers able 
to generate successful bipedal gaits (l‘S3>Miiil£lf£I). 
Reil et al. have now significantly developed this tech¬ 
nology to exploits its commercial possibilities, in the 
animation and games industries, for the real-time con¬ 
trol of physically simulated three-dimensional (3-D) 



Fig. 76.6 The octopod robot built by Applied Al Systems 
Inc. 


humanoid characters engaged in a variety of motor be¬ 
haviors (refer to [76.31] for further details). Coupled 
neural oscillators have been evolved also to control the 
swimming pattern of articulated, snake-like, underwa¬ 
ter robots using physics-based simulations [76.32]. 

Vaughan has taken related work in another direc¬ 
tion. He has successfully applied evolutionary robotics 
techniques to evolve a simulation of a 3-D ten-degree- 
of-freedom bipedal robot. This machine demonstrates 
many of the properties of human locomotion. By using 
passive dynamics and compliant tendons, it conserves 
energy while walking on a flat surface. Its speed and 
gait can be dynamically adjusted and it is capable of 
adapting to discrepancies in both its environment and its 
body’s construction [76.33]. Parameters describing the 
body shape (leg segment lengths, hip width, etc.) and 
properties of a continuous dynamical neural network 
controller were under genetic control. The machine 
started out as a passive dynamic walker [76.34] on 
a slope, and then throughout the evolutionary process 
the slope was gradually lowered to a flat surface. The 
machine demonstrated resistance to disturbance while 
retaining passive dynamic features such as a passive 
swing leg. Wischmann and Passemann independently 
took a very similar approach [76.35]. Vaughan’s orig¬ 
inal machine did not have a torso, but he has also 
successfully applied the method to a simplified two- 
dimensional (2-D) machine with a torso above the hips. 
When pushed, this dynamically stable bipedal machine 
walks either forward or backwards just enough to re¬ 
lease the pressure placed on it. It is also able to adapt 
to external and internal perturbations as well as vari¬ 
ations in body size and mass [76.36]. These biped 
examples make use of the co-evolution of body mor¬ 
phology and neural controller, an idea also used in 
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earlier, more abstract, work on evolved bipedal locomo¬ 
tion by Endo et al. [76.37]. Although possible changes 
to the body morphology are quite tightly constrained, 
nonetheless this aspect was important. This theme is 
covered in more detail in the later section on Evolv¬ 
ing Bodies, which also describes recent examples of 
evolved walking behaviors in the context of body-brain 
co-evolution. 

McHale and Husbands [76.38, 39] have compared 
many forms of evolved neural controllers for bipedal 
and quadrupedal walking machines. Recurrent dynami¬ 
cal continuous time networks and GasNets (described 


76.3 Simulation and Reality 

Few of the experiments in the previous section were car¬ 
ried out entirely on physical robots because: 

1. Evolution may take a long time, especially if it is 
carried out on a single robot that incarnates the bod¬ 
ies of all the individuals of the evolving population. 

2. The physical robot can be damaged because popula¬ 
tions always contain a certain number of poorly per¬ 
forming individuals (for example, colliding against 
walls) by effect of random mutations. 

3. Restoring the environment to initial conditions be¬ 
tween trails of different individuals or populations 
(for example, replenishing the arena with objects) 
may not always be feasible without human interven¬ 
tion. 

4. Evolution of morphologies and evolution of robots 
that can grow during their lifetime is almost impos¬ 
sible with today’s technology without some level of 
human intervention. 

For those reasons, researchers often resort to evolu¬ 
tion in simulation and transfer the evolved controllers 
to the physical robot. However, it is well known that 
programs that work well in simulations may not func¬ 
tion properly in the real world because of differences 
in sensing, actuation, and in the dynamic interactions 
between robot and environment [76.41]. This reality 
gap is even more evident in adaptive approaches, such 
as evolutionary robotics, where the control system and 
morphology are gradually crafted through the repeated 
interactions between the robot and the environment. 
Therefore, robots will evolve to match the specifici¬ 
ties of the simulation, which differ from the real world. 
Although these issues clearly rule out any simulation 
based on grid worlds or pure kinematics, over the last 
10 years simulation techniques have dramatically im¬ 
proved and resulted in software libraries that model 
reasonably well dynamical properties such as friction, 


in Sect. 76.7.3) were shown to have advantages in 
most circumstances. The vast majority of the studies 
mentioned above were conducted for relatively be¬ 
nign environments. Notwithstanding this observation, 
we can conclude that the more complex dynamical 
neural network architectures, with their intricate dy¬ 
namics, generally produce a wider range of gaits and 
generate smoother, more adaptive locomotion than the 
more standard use of systems based on finite-state 
machines employing parameterized rules governing 
the timing and coordination of individual leg move¬ 
ments [76.40]. 


collision, mass, gravity, and inertia [76.42]. These soft¬ 
ware tools allow one to simulate articulated robots of 
variable morphology and their environment as fast as, 
or faster than, real time in a desktop computer. 

Nonetheless, even physics-based simulations in¬ 
clude small discrepancies that can accumulate over 
time and result in very different behavior from real¬ 
ity (for example, a robot may get stuck against a wall 
in simulation whereas it can get free in reality, or vice 
versa). Also, physics-based simulations cannot account 
for diversity of response profiles of the individual sen¬ 
sors, motors, and gears of a physical robot. Several 
methods can be used to cope with these problems and 
improve the quality of the transfer from simulation to 
reality. 

A widely used method consists of adding indepen¬ 
dent noise to the values of the sensors provided by the 
model and to the end position of the robot computed by 
the simulator [76.43]. Some software libraries allow the 
introduction of noise at several levels of the simulation. 
This solution prevents evolution from finding solutions 
that rely on the specificities of the simulation model. 
Another method consists in sampling the actual sensor 
values of the real robot positioned at several angles and 
distances from objects of different texture. Those val¬ 
ues are then stored in a look-up table and retrieved with 
the addition of noise according to the position of the 
robot in the environment [76.44]. This method proved 
to be very effective for generating controllers that trans¬ 
fer smoothly from simulation to reality. A drawback of 
this sampling method is that it does not scale up well to 
high-dimensional sensors (e.g., vision) or geometrically 
complicated objects. 

Another method, also known as minimal simula¬ 
tions, consists of modeling only those characteristics 
of the robot and environment that are relevant for the 
emergence of desired behaviors [76.45]. These char- 
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acteristics, which are referred to as base-set features, 
should be accurately modeled in simulation. Instead, all 
the other characteristics, which are referred to as im¬ 
plementation aspects, should be randomly varied across 
several trials of the same individual in order to ensure 
that evolving individuals do not rely on implementa¬ 
tion aspects, but rely on base-set features only. Base-set 
features must also be varied to some extent across tri¬ 
als in order to ensure some degree of robustness of 
the individual with respect to base-set features, but 
this variation should not be so large that reliably fit 
controllers fail to evolve at all. This method allows 
very fast evolution of complex robot-environment situ¬ 
ations, as in the example of the hexapod walk described 
in Sect. 76.2.1. A drawback of minimal simulations is 
that it is not always easy to tell in advance which are the 
base-set features that are relevant for the desired behav¬ 
ior. 

Yet another method consists of the coevolution of 
the robot (control and/or morphology) and of the simu¬ 
lator parameters that are most likely to differ from the 
real world and that may affect the quality of the trans¬ 
fer [76.46]. This method consists of coevolving two 
populations, one encoding the properties of the robot 
and one encoding the parameters of the simulator. Co¬ 
evolution happens in several passes through a two-stage 
process. In stage one, a randomly generated population 
of robots are evolved in the default simulator and the 
best individual is tested on the real robot while the time 
series of sensory values are recorded. In stage two, the 
population of simulators is evolved to reduce the differ¬ 
ence between the time series recorded on the real robot 
and the time series obtained by testing evolved robots 
within the simulator. The best evolved simulator is then 
used for stage one where a new randomly generated 
population is evolved and the best individual is tested 
on the real robot to generate the time series for stage 


two of simulator evolution. This two-stage coevolution 
is repeated several times until the error between simu¬ 
lated and real robot behavior is the smallest possible. 
It has been shown that approximately 20 passes of the 
two-stage process are sufficient to evolve a good con¬ 
trol system that could be transferred to an articulated 
robot. In that case, the real robot was used to test only 
20 individuals. 

A recent approach tackles the simulation to reality 
transfer problem by using a multi-objective formula¬ 
tion of ER in which two main objectives are optimized 
via a Pareto-based multi-objective evolutionary algo¬ 
rithm: (1) the fitness and (2) the transferability [76.47]. 
To evaluate the transferability a simulation-to-reality 
disparity measure was defined in terms of the differ¬ 
ence in behavior between simulation and reality for 
any given controller. This measure is approximated 
for each member of the population and the method 
has successfully been demonstrated for walking behav¬ 
iors [76.47]. 

Finally, another method consists of genetically en¬ 
coding and evolving the learning rules of the control 
system, rather than its parameters (e.g., connection 
strengths). The parameters of the decoded control sys¬ 
tem are always initialized to small random values at 
the beginning of an individual lifetime and must self- 
organize using the learning rules [76.48]. This method 
prevents evolution from finding a set of control param¬ 
eters that fit the specificities of the simulation model, 
and encourages emergence of control systems that 
remain adaptive to partially unknown environments. 
When such an evolved individual is transferred to the 
real robot, it will develop online its control parame¬ 
ters according to the genetically evolved learning rules 
and taking into account the specificities of the phys¬ 
ical world. This method is described in more detail 
in Sect. 76.7.2 on evolution of learning. 


76.4 Behavior as a Complex Adaptive System 


Behavior is a dynamical process resulting from nonlin¬ 
ear interactions (occurring at a fast time rate) between 
the agent’s control system, its body, and the environ¬ 
ment [76.49, 50]. At any time step, the environment and 
the agent-environment relation influence the body and 
the motor reaction of the agent, which in turn influences 
the environment and/or the agent-environmental rela¬ 
tion (Fig. 76.7). Sequences of these interactions lead 
to a dynamical process where the contributions of the 
different aspects (i. e., the robot’s control system, the 
robot’s body, and the environment) cannot be sepa¬ 
rated. This implies that even complete knowledge of 


the elements governing the interactions provides little 
insight into the behavior emerging from these interac¬ 
tions [76.51,52]. 

An interesting property of evolutionary robotics is 
that it can enable to synthesize robots displaying a cer¬ 
tain behavioral capacity without specifying the manner 
in which such capacity should be realized and/or the 
combination of elementary behaviors that should be 
produced and combined to achieve the desired overall 
capacity. This allows the evolving robots to discover 
and exploit behaviors that emerge from the interactions 
between the robot control system, the robot body and 
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the environment and/or from the interaction between 
previously developed behavioral capacities [76.52], 
The synthesis and exploitation of emergent properties, 
in turn, often allows evolving robots to discover solu¬ 
tions that rely on relatively parsimonious control policy 
and/or body structures. 

For an example of how evolving robots can solve 
an adaptive task on the basis of a simple control policy, 
thanks to the possibility to exploit properties emerging 



Fig. 76.7 Schematization of how: (1) behavior can emerge 
from several nonlinear interactions, occurring at fast time 
rates, between the agent’s control system, its body, and the 
environment, and (2) behavior can display a multi-level or¬ 
ganization in which the robot/environmental interactions 
and the interaction between lower-level behaviors give rise 
to higher level behaviors that later affect the interactions 
from which they originate 



Fig. 76.8 The environment and the robot. The environ¬ 
ment consists of an arena of 60 x 35 cm and contains 
a cylindrical objects placed at a randomly selected location 


from the agent/environmental interactions, let us con¬ 
sider the case of a Khepera robot placed in an arena 
surrounded by walls (Fig. 76.8) that should evolve an 
ability to forage by finding and remaining close to 
a food object (i. e., a cylindrical object) [76.53]. The 
robot is provided with eight infrared sensors and two 
motors controlling the desired speed of the two corre¬ 
sponding wheels. From the point of view of an external 
observer, solving this problem requires robots able to: 

1. Explore the environment until an obstacle is de¬ 
tected. 

2. Discriminate whether the obstacle detected is a wall 
or a cylindrical object. 

3. Approach or avoid the object depending on the ob¬ 
ject type. 

A detailed analysis of the sensory patterns expe¬ 
rienced by the robot indicated that the task of dis¬ 
criminating the two objects is far from trivial since 
the two classes of sensory patterns experienced by 
robots close to a wall and close to cylindrical ob¬ 
jects overlap significantly. Flowever, robots evolved 
for the ability to solve this task resorted to a strat¬ 
egy that does not require to explicitly discriminate 
of the two types of objects [76.53]. This solution 
consists in reacting to sensory states so 
that the robot/environmental dynamics converge into 
a limit cycle near the cylindrical object, in which the 
robot keep moving forth and back and left and right, 
and not near a wall (Fig. 76.9). 



Fig.76.9a,b Angular trajectories of an evolved robot close 
to a wall (a) and to a cylinder (b). The picture was obtained 
by placing the robot at a random position in the environ¬ 
ment, leaving it free to move for 500 cycles, and recording 
its relative movements with respect to the two types of ob¬ 
jects for distances smaller than 45 mm. For sake of clarity, 
arrows are used to indicate the relative direction, but not 
the amplitude of movements 
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The possibility to discover and rely on these forms 
of emergent behavior allows evolving robots to find 
computationally simple solutions to apparently com¬ 
plex problems. Indeed, the foraging task described 
above can be solved by a Khepera robot provided with 
a simple reactive controller (i. e., a feedforward neu¬ 
ral network with eight sensory neurons that encode the 
state of the corresponding infrared sensors directly con¬ 
nected to two motor neurons that set the desired speed 
of the two wheels). 

76.4.1 Behavior Recombination and Re-Use 

Evolving robots can recombine and re-use acquired el¬ 
ementary behavioral capacities to produce higher level 
behaviors. This has been demonstrated in a series of 
simulation experiments in which a population of hu¬ 
manoid robots provided with an articulated arm, a cam¬ 
era, and a touch sensor on the palm of the hand, have 
been evolved for the ability to execute two-words im¬ 
perative sentences constituted by the combination of 
three action and three object words (reach, touch, move, 
red-object, green-object, and blue-object) encoded by 
six corresponding binary sensors [76.54]. 

During the evolutionary process the robots were 
evaluated for the ability to comprehend seven out of 
the nine possible sentences (that can be generated by 
combining the three action and the three object words) 
by executing the seven corresponding behaviors. The 
robots were then post-evaluated also on the other two 
sentences not experienced during the evolutionary pro¬ 
cess. 

Some of the evolved robots were able to develop 
the required skills and to generalize their capacities to 
the two new sentences by executing the appropriate cor¬ 
responding behaviors (l-g&M'll'ifim). Differently from 
the other individuals, the robots able to generalize 
where characterized by a hierarchical organization in 
which the nine behaviors were produced by combining 
over time a set of elementary behaviors and in which the 
same elementary behaviors was re-used to produce dif¬ 
ferent high-level behaviors. More specifically the robots 
able to generalize displayed a reach-X behavior (that 
consisted in moving the arm toward a red, or green, or 
blue object), a touch behavior (that consisted in mov¬ 
ing the hand until the object is touched irrespectively 
from the color), and a move behavior (that consists in 
keep moving the hand also after the object has been 
touched irrespectively from the color) and combined 
these lower-level behaviors in a compositional man¬ 


ner to produce the nine required higher-level actions. 
This means that, for example, the same reach red-object 
behavior was used incombination with the touch or 
the push behavior to produce a touch the red-object 
and a push the red-object behavior. For other works 
discussing the emergence and the role of multi-level be¬ 
havioral organizations see [76.52]. 

76.4.2 Sensory-Motor Coordination 

By acting robots inevitably modify the robot-envi¬ 
ronmental relation and/or the environment and conse¬ 
quently the stimuli that they will experience next. By 
exploiting the possibility to actively influence the per¬ 
ceived stimuli through actions, robots can hnd adaptive 
solutions based on parsimonious control policies. Ar¬ 
tificial evolution constitutes an effective method for 
discovering such type of solutions that are often hard 
to imagine from the point of view of a human observer. 
Indeed, examples of clever use of sensory-motor coor¬ 
dination abounds in the evolutionary robotics literature. 

Let us consider, for instance, the case of a Khepera 
robot endowed with infrared and wheels speed sensors, 
that can forage by ramaining close to large cylindrical 
objects (food) while avoiding small cylindrical objects 
(dangers) [76.55]. From a passive perspective, that does 
not take into account the fat that the robot can self-select 
useful stimuli through action, the ability to discriminate 
between sensory stimuli experienced near small and 
large cylindrical objects requires a relatively complex 
control policy since the two classes of stimuli strongly 
overlap in the robot’s perceptual space. On the other 
hand, the exploitation of sensory-motor coordination 
can can allow the robots to simplify the discrimination 
problem. 

Indeed, evolving robots tend to converge on a rather 
simple solution that consists in circling around the 
cylindrical objects, as soon as an object is perceived, 
and in using the differential speed of the left and 
right wheels sensed during the execution of the object- 
circling behavior to decide to keep circling around the 
object (in the case of small differential speeds) or to 
abandon the object (in the case of large differential 
speeds). Indeed, the execution of the object-circling be¬ 
havior allows the robots to experience sensory stimuli 
on the wheel sensors that are well differentiated for 
small and large objects. This, in turn, allows them to 
solve the object discrimination problem with a rather 
simple but reliable control policy. For other examples 
see [76.53,56,57], 
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76.5 Evolving Bodies 

Most evolutionary robotics experiments - and most 
robotics experiments in general - assume that the body 
plan of the robot has already been designed; an opti¬ 
mization method is used to improve the control policy 
only (The term body plan is here used to denote all as¬ 
pects of a robot’s design other than its control policy. 
Such design considerations include the robot’s me¬ 
chanical layout and material properties as well as its 
sensor and motor distributions.). This emphasis be¬ 
lies an assumption within the held of robotics, which 
is that control policy design is non-intuitive and thus 
should be automated, while choosing an appropriate 
robot body plan is intuitive and thus can be manually 
designed. 

However, it has been shown that the careful de¬ 
sign of the robot’s body can have large and desirable 
impacts on its resulting behavior. For example, proper 
curvature on the underside of a biped robot’s feet 
(along with other settings) can allow it to walk down 
a declined plane with no control policy at all [76.58]. 
Or, that modifications to an anthropomorphic robot 
arm and hand can facilitate the evolution of active 
categorical perception [76.59] (Active categorical per¬ 
ception occurs when a robot or animal actively inter¬ 
acts with objects of interest, and the sensory stimu¬ 
lation resulting from this physical interaction allows 
for categorization of those objects.). These results fit 
with the view of embodied behavior, as outlined in 
Fig. 76.7: because behavior arises from the interac¬ 
tion between a robot’s body and its environment, al¬ 
terations to the robot’s body will alter the resulting 
behavior. 

This suggests that it is useful to automatically im¬ 
prove not just a robot’s control policy but also its body 
plan. Evolutionary algorithms are a uniquely well suited 
tool for this task because, unlike many learning meth¬ 
ods, they do not make assumptions about the structure 
of the system being optimized: the length of a robot’s 
leg - or the number of legs - can be evolved just as eas¬ 
ily as can the strength of a synaptic connection in an 
artificial neural network. 


76.5.1 Co-Evolving Body and Brains 

The Sussex group was the first to demonstrate the 
evolution of robot morphology: they evolved sensor 
placements on a physical robot [76.15], although the 
other aspects of the robot’s body plan remain fixed. 
A year later Sims [76.61] demonstrated an evolutionary 
algorithm that improved the structure and parameters 
of the robots’ body plans and control policies. Although 
Sim’s creatures were virtual and operated in a simulated 
environment, the robots exhibited a wide range of intu¬ 
itive and non-intuitive body plans that allowed them to 
swim, walk or compete over a limited resource [76.62], 

Funes and Pollack [76.63] demonstrated that it was 
possible to evolve three-dimensional forms in simu¬ 
lation, build them in reality, and have the physical 
structure act similarly to the originally-evolved simu¬ 
lated structure. This was followed by work from the 
same group in which robots evolved in simulation 
were manufactured as physical robots using 3-D print¬ 
ing technology [76.60] (Fig. 76.10). Although only the 
plastic frame of the robot was printed and the electron¬ 
ics and battery had to be manually added, this served 
as a demonstration that, in principle, robot design could 
be automated using evolutionary algorithms and robot 
manufacture could be automated using rapid prototyp¬ 
ing [76.64]. 

Since then, a number of research groups have 
evolved robot body plans and control policies simul¬ 
taneously for various purposes. Some researchers have 
adapted this approach for studying biological ques¬ 
tions. For example Long et al. [76.65] have evolved the 
stiffness of artificial tails attached to physical swim¬ 
ming robots: robots with tails of differing stiffness 
have differing abilities to swim fast or turn well. This 
provides a unique experimental tool for investigating 
how backbones originally evolved in early vertebrates. 
Clark et al. [76.66] also evolved the material properties 
of part of a robot fish, but focussed in this case on evolv¬ 
ing the stiffness and shape of its fins in simulation. This 
project had an engineering aim: the evolved fins were 



Fig.76.10a,b An example 
evolved robot from the GOLEM 
project [76.60], (a) The virtual robot, 
as originally evolved in the simulated 
environment. (b) The physical robot 
comprised of a 3-D-printed plastic 
frame and manually-added electronics 
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manufactured and tested on a physical robot fish and 
were found to aid desirable swimming behavior. 

Evolutionary algorithms can be used to explore the 
space of possible robot body plans, but they can also 
be used to explore metamorphosis, or how a robot’s 
body plan might change over its lifetime. In recent 
work Bongard [76.67] compared two approaches to 
evolving walking behaviors for upright legged robots 
(|43>M3liliikiU). In the first approach, upright legged 
robots were evolved until successful walking was dis¬ 
covered. In the second approach, locomotion was first 
evolved for legless, anguilliform robots that gradu¬ 
ally grew legs while moving. As evolution proceeded 
in this second approach, later generations of robots 
gradually lost this infant legless body plan and in¬ 
stead were born with the upright, legged body plan. It 
was found that walking evolved for the upright legged 
robots more rapidly in the second approach, and that the 
evolved controllers were more robust. The explanation 
for this result is that the legless robot provides a form 
of scaffolding that accelerates search: it is easier for 
evolution to generate locomotion for the legless robot 
because with the anguilliform body plan the robot can¬ 
not fall over (Scaffolding is the phenomenon in which 
a teacher introduces some aspect into the learner’s en¬ 
vironment that helps the learner to grasp a concept and 
then later refine the concept when the scaffold is re¬ 
moved [76.68]. The canonical example of scaffolding is 
training wheels for bicycles.). This locomotion strategy 
is then refined subsequently by evolution to success¬ 
fully control the upright and legged (and thus unstable) 
robot. 

The simultaneous evolution of robot body plans and 
control policies offers other avenues for investigating 
the relationship between body, brain and environment. 
In [76.69] it was found that more complexly-shaped 
robots were produced when evolved to walk over rough 
terrain than when evolved to walk over flat terrain 
(|<sz>Xil>H!U£fl). This was due to the fact that robots 
evolved in rough terrain evolved appendages and hooks 
to gain purchase between outcroppings and then pull or 
push themselves forward. 

Most recently Hiller and Lipson [76.70] have 
demonstrated the evolution of soft robots: this requires 
evolving not just the control policy of the robot and its 
physical shape, but also the material properties of each 
voxel comprising the machine. This allows for complex 
three-dimensional patterning of soft and rigid material 
throughout the robot, which can be exploited by evolu¬ 
tion to produce locomotion [76.71]. Soft robots are an 
ideal vehicle for demonstrating the power of evolution¬ 
ary algorithms: the design and control of such machines 
is highly nonintuitive, making manual design extremely 
difficult. 


76.5.2 Self-Modeling 

The evolution of robot body plans can be useful not 
just for robot design but also for increasing the adap¬ 
tivity of a physical robot once it has been designed 
and deployed. For example in [76.72] it was shown 
that a physical robot could be equipped with an on¬ 
board simulator that the robot could use to continuously 
evolve models of itself. These self-models reflect the 
mechanical construction of the robot. This method was 
found useful for robots that might sustain unanticipated 
damage such as the mechanical separation of a leg: the 
robot diagnoses the damage; it then evolves a simu¬ 
lated damaged robot that accurately reflects the physical 
damage; it evolves a compensatory control policy inter¬ 
nally using the simulated, damaged robot; and finally 
the physical robot uses the internally evolved compen¬ 
satory control policy to continue moving despite its 
injury. Figure 76.11 outlines this method. 


Self-model synthesis Exploratory action synthesis 



Fig.76.11a-f The machine begins by performing a random action 
and collects the resulting sensor-motor data (a). An evolutionary 
algorithm evolves a population of simulated robots that, when they 
move, produce similar sensor-motor data as the physical machine 
(b). Another algorithm searches for a new action for the physical 
robot to perform (c). The physical robot performs the new action, 
and re-evolves self-models to explain the result of the first action 
and the new action ((a)). After several cycles of this self-modeling, 
the best self-model is used to evolve new behaviors (d). Finally, the 
physical robot executes these newly-evolved behaviors (e,f) 
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A method for generalizing this to robot swarms 
was demonstrated in [76.73]. Each robot in the swarm 
maintained its own self-modeling engine, but would pe¬ 
riodically export its best self-model and control policy 
to others in the swarm. The result of this was that if one 
robot was damaged and recovered, a second robot that 
suffered similar damage recovered more rapidly. 

Finally, instead of modeling the self, a robot could 
create a model of another robot in its vicinity. Even bet- 

76.6 Seeing the Light 

Pioneering experiments on evolving visually guided be¬ 
haviors were performed at Sussex University [76.75] on 
a specially designed gantry robot (Fig. 76.12, see also 
I<bM '1 1 >»»*«■) Discrete -time dynamical recurrent neu¬ 
ral networks and visual sampling morphologies were 
concurrently evolved: the brain was developed in tan¬ 
dem with the visual sensor [76.13,76,77]. The robot 
was designed to allow real-world evolution by having 
off-board power and processing so that the robot could 
be run for long periods while being monitored by au¬ 
tomatic fitness evaluation functions. A charge-coupled 
device (CCD) camera points down towards a mirror an¬ 
gled at 45° as shown in Fig. 76.12. The mirror can 
rotate around an axis perpendicular to the camera’s im¬ 
age plane. The camera is suspended from the gantry, 
allowing motion in the X, Y, and Z dimensions. This ef¬ 
fectively provides an equivalent to a wheeled robot with 
a forward-facing camera when only the X and Y dimen¬ 
sions of translation are used. The additional dimension 
allows flying behaviors to be studied. 



Fig. 76.12 The gantry robot used in the visual discrimina¬ 
tion task. The camera inside the top box points down at the 
inclined mirror, which can be turned by the stepper motor 
beneath. The lower plastic disk is suspended from a joy¬ 
stick to detect collisions with obstacles 


ter, the robot could evolve a model of the other robot’s 
intentions and use this information to aid or thwart 
the other robot’s actions, as demonstrated in [76.74], 
Although the robots did not model each other’s body 
plans, this ability to model others in general is known 
as Theory of Mind. One could imagine how increas¬ 
ing levels of recursion of such embedded mind reading 
could provide continued evolutionary pressure toward 
increasingly intelligent machines. 


The apparatus was initially used in a manner sim¬ 
ilar to the real-world experiments on navigation in the 
looping maze with the miniature mobile robot described 
in Sect. 76.2. A number of visually guided navigation 
behaviors were successfully achieved, including navi¬ 
gating around obstacles, tracking moving targets, and 
discriminating between different objects [76.76]. The 
evolutionary process was incremental. The ability to 
distinguish between two different targets was evolved 
on top of the single target-finding behavior. The chro¬ 
mosome was of dynamic length so the neurocontroller 
was structurally further developed by evolution to 
achieve the new task (neurons and connections added). 
In the experiment illustrated in Figs. 76.12 and 76.13, 




Fig.76.13a,b The shape discrimination task, (a) The posi¬ 
tion of the robot in the arena, showing the target area in 
front of the triangle, (b) The robot camera’s field of view 
showing the visual patches selected by evolution for sen¬ 
sory input 
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starting from a random position and orientation, the 
robot had to move to the triangle rather than the rectan¬ 
gle. This had to be achieved irrespective of the relative 
positions of the shapes and under very noisy light¬ 
ing conditions. Recurrent neural network controllers 
were evolved in conjunction with visual sampling mor¬ 
phologies. Only genetically specified patches from the 
camera image were used (by being connected to in¬ 
put neurons according to the genetic specification). The 
rest of the image was thrown away. This resulted in 
extremely minimal systems using only two or three 
pixels of visual information, yet still able to perform 
the task reliably under highly variable lighting condi¬ 
tions [76.13, 76]. 

This was another example of staged, or incremental, 
evolution to obtain control systems capable of solv¬ 
ing problems that are either too complex or may profit 
from an evolutionary methodology that discovers, pre¬ 
serves, and builds upon subcomponents of the solution. 
For an evolutionary method that incorporate strate¬ 
gies to explicitly address this issue, interested readers 
may refer to [76.78]. However, staged evolution re¬ 
mains a poorly explored area of evolutionary robotics 
that deserves further study and a more principled ap¬ 
proach [76.79] in order to achieve increasingly complex 
robotic systems. 

76.6.1 Coevolution of Active Vision 
and Feature Selection 

Machine vision today can hardly compete with biolog¬ 
ical vision despite the enormous power of computers. 
One of the most remarkable - and often neglected - dif¬ 
ferences between machine vision and biological vision 
is that computers are often asked to process an entire 
image in one shot and produce an immediate answer 
whereas animals take time to explore the image over 
time, searching for features and dynamically integrat¬ 
ing information over time. 

Active vision is the sequential and interactive pro¬ 
cess of selecting and analyzing parts of a visual 
scene [76.80-82]. Feature selection instead is the de¬ 
velopment of sensitivity to relevant features in the 
visual scene to which the system selectively responds, 
e.g., [76.83]. Each of these processes has been inves¬ 
tigated and adopted in machine vision. However, the 
combination of active vision and feature selection is 
still largely unexplored. An intriguing hypothesis is 
that coevolution of active vision and feature selection 
could greatly simplify the computational complexity of 
vision-based behavior by facilitating each other’s task. 

This hypothesis was investigated in a series of ex¬ 
periments [76.84] on coevolution of active vision and 
feature selection for behavioral systems equipped with 


a primitive moving retina and a deliberately simple neu¬ 
ral architecture (Fig. 76.14). The neural architecture 
was composed of an artificial retina and two sets of 
output units. One set of output units determined the 
movement and zooming factor of the retina, and the 
other set of units determined the behavior of the system, 
such as the response of a pattern-recognition system, 
the control parameters of a robot, or the actions of a car 
driver. The neural network was embedded in a behav¬ 
ioral system and its input/output values were updated 
every 300 ms while its fitness was computed. Therefore, 
the synaptic weights of this network were responsi¬ 
ble for both the visual features on which the system 
based its behavior and for the motor actions necessary 
to search for those features. 

In a first set of experiments, the neural network was 
embedded in a simulated pan-tilt camera and asked to 
discriminate between triangles and squares of differ¬ 
ent size that could appear at any location of a screen 
(Fig. 76.15a), a perceptual task similar to that explored 
with the gantry robot described in Sect. 76.5. The visual 
system was free to explore the image for 60 s while con¬ 
tinuously reporting whether the current screen showed 
a triangle or a square. The fitness was proportional to 


d) System e) Vision 

behavior behavior 



Fig.76.14a-f The neural architecture of the active vision 
system is composed of: (a) a grid of visual neurons with 
nonoverlapping receptive fields whose activation is given 
by (b) the grey level of the corresponding pixels in the 
image; (c) a set of proprioceptive neurons that provide 
information about the movement of the vision system; 
(d) a set of output neurons that determine the behavior 
of the system (pattern recognition, car driving, robot nav¬ 
igation); (e) a set of output neurons that determine the 
behavior of the vision system; and (f) a set of evolvable 
synaptic connections. The number of neurons in each sub¬ 
system can vary according to the experimental settings 
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the amount of correct responses accumulated over the 
60 s for several screenshots containing various instances 
of the two shapes. Evolved systems were capable of cor¬ 
rectly identifying the type of shape with 100% accuracy 
after a few seconds despite the fact that this recognition 
problem is not linearly separable and that the neural net¬ 
work does not have hidden units, which in theory are 
necessary to solve nonlinearly separable tasks. Indeed, 
the same neural network presented with the same set of 
images and trained with supervised learning, but with¬ 
out the possibility to actively explore the scene, was not 
capable of solving the task. The evolved active vision 
system developed sensitivity to vertical edges, oriented 
edges and corners, and used its movement to search for 
these features in order to tell whether the shape was 
a triangle or a square. These features, which are also 
found in the early visual system of almost all animals, 
are invariant to size and location. 

In a second set of experiments, the neural network 
was embedded in a simulated car and was asked to drive 
over several mountain circuits (Fig. 76.15b). The simu¬ 
lator was a modified version of a car race video game. 
The neural network could move the retina across the 
scene seen through the windscreen at the driver’s seat 
and control the steering, acceleration, and braking of the 
car. The fitness was inversely proportional to the time 
taken to complete the circuits without exiting the road. 
Evolved networks completed all circuits with time laps 
competitive to those of well-trained students controlling 
the car with a joystick. The evolved network started by 
searching for the edge of the road and tracked its rela¬ 
tive position with respect to the edge of the windscreen 
in order to control steering and acceleration. This be¬ 
havior was supported by the development of sensitivity 
to oriented edges. 

In a third set of experiments, the neural network was 
embedded in a real mobile robot with a pan-tilt camera 

a) b) 



Fig. 76.15 (a) An evolved individual explores the screen searching 
for the shape and recognizes it by the presence of a vertical edge, 
(b) Search for the edge of the road at the beginning of a drive over 
a mountain road 


that was asked to navigate in a square arena with low 
walls located in an office (Fig. 76.16). The fitness was 
proportional to the amount of straight motion measured 
over two minutes. Robots that hit the walls because they 
watched people or other irrelevant features of the office 
had lower fitness than robots that could perform long 
straight paths and avoid walls of the arena. Evolved 
robots tended to fixate the edge between the floor and 
the walls of the arena, and turned away from the wall 
when the size of its retinal projection became larger 
than a threshold (l-ssfcMUEHiEB). This combination of 
sensitivity to oriented edges and looming is also found 
in the visual circuits of several insects and birds. 

In a further set of experiments [76.85], the visual 
pathway of the neural network was augmented by an 
intermediate set of neurons whose synaptic weights 
could be modified by Hebbian learning [76.86] while 
the robot moved in the environment. All the other 
synaptic weights were genetically encoded and evolved. 
The results showed that lifelong development of the 
receptive fields improved the performance of evolved 
robots and allowed robust transfer of evolved neural 
controllers from simulated to real robots, because the 
receptive fields developed sensitivity to features en¬ 
countered in the environment where they happen to 
be born (see also the section above on simulation and 
reality). Furthermore, the results showed that the de¬ 
velopment of visual receptive fields was significantly 
and consistently affected by active vision as compared 
to the development of receptive fields passively ex¬ 
posed to the same set of sample images. In other words, 
robots evolved with active vision developed sensitiv¬ 
ity to a smaller subset of features in the environment 
and actively tracked those features to maintain a stable 
behavior. 



Fig. 76.16 A mobile robot with a pan-tilt camera is asked 
to move within the walled arena in the office environment 
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76.7 Computational Neuroethology 

Evolutionary robotics is also used to investigate open 
questions in neuroscience and cognitive science [76.87- 
90] because it offers the vantage point of a behavioral 
system that interacts with its environment [76.91]. Al¬ 
though the results should be carefully considered when 
drawing analogies with biological organisms, evolu¬ 
tionary robotics can generate and test hypotheses that 
could be further investigated with mainstream neuro¬ 
science methods. 

For example, the active vision system with Hebbian 
plasticity described in the previous section was used to 
answer a question raised by Held and Hein [76.92] in 
the 1960s. The authors devised the apparatus shown 
in Fig. 76.17 where the free movements of a kitten 
( 1 active kitten) were transmitted to a second kitten that 
was carried in a gondola (passive kitten). The sec¬ 
ond kitten could move its head, but its feet did not 
touch the ground. Consequently, the two kitten re¬ 
ceived almost identical visual stimulation, but only 
one of them received that stimulation as a result of 
body self-movement. After a few days in that envi¬ 
ronment, only the active kitten displayed normal be¬ 
havior in several visually guided tasks. The authors 
suggested the hypothesis that proprioceptive motor in¬ 
formation resulting from generation of actions was nec¬ 
essary for the development of normal, visually guided 
behavior. 

The kitten experiments were replicated by cloning 
an evolved robot controller and randomly initializing 
the synaptic values of the adaptive visual pathways in 
both clones. One cloned robot was then left free to move 
in a square environment while the other cloned robot 
was forced to move along imposed trajectories, but was 
free to control its camera position, just like the pas¬ 
sive kitten [76.94]. The results indicated that the visual 
receptive fields and behaviors of passive robots differ 
significantly from those of active robots. Furthermore, 
passive robots that were later left free to move were no 
longer capable of properly avoiding walls. A thorough 
analysis of neural activation correlated with behavior 
of the robot and even transplantation of neurons across 
active and passive robots revealed that the poor perfor¬ 
mance was due to the fact that passive robots could not 
completely select the visual features they were exposed 
to. Consequently, passive robots developed sensitivity 
to features that were not functional to their normal be¬ 
havior and interfered with other dominant features in 
the visual field. Whether this explanation also hold for 
living animals remains to be further investigated, but at 
least these experiments indicated that motor feedback 
is not necessary to explain the pattern of pathological 
behavior observed in animals and robots. 


76.7.1 Emergence of Place Cells 

Fet us now consider the case of an animal exploring 
an environment and periodically returning to its nest 
to feed. It has been speculated that this type of situa¬ 
tion requires the formation of spatial representations of 
the environment that allow the animal to find its way 
home [76.95]. Different neural models with various de¬ 
grees of complexity and biological detail that could 
provide such functionality have been proposed [76.96, 
97]. 

Would a robot evolved under similar survival con¬ 
ditions develop a spatial representation of the envi¬ 
ronment and, if so, what type of representation would 
that be? These questions were explored using the 
same Khepera robot and evolutionary methodology de¬ 
scribed in Sect. 76.2 for reactive navigation in the 
looping maze. The environment was a square arena with 
a small patch on the floor in a comer where the robot 
could instantaneously recharge its (simulated) battery 
(Fig. 76.18). The environment was located in a dark 
room with a small light tower over the recharging sta¬ 
tion. 

The sensory system of the robot was composed of 
eight distance sensors, two ambient-light sensors (one 



Fig. 76.17 The original apparatus in [76.92], where the 
gross movements of a kitten moving almost freely were 
transmitted to a second kitten that was carried in a gon¬ 
dola. Both kittens were allowed to move their head. They 
received essentially the same visual stimulation because of 
the unvarying pattern on the walls and the center post of 
the apparatus (after [76.93], with permission) 
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on each side), one floor-color sensor, and a sensor for 
battery charge level. The battery lasted only 20 s and 
had a linear discharge. The evolutionary neural net¬ 
work included five fully connected internal neurons 
between sensory and motor neurons. The same fitness 
function described in Sect. 76.2 for navigation in the 
looping maze was used, except for the middle term 
which had been used to encourage straight navigation in 
the looping maze. The fitness value was computed every 
300 ms and accumulated over the life span of the in¬ 
dividual. Therefore, individuals who discovered where 
the charger was could live longer and accumulate more 
fitness by exploring the environment (individuals were 
killed if they survived longer than 60 s to limit the ex¬ 
perimentation time). 

The same physical robot evolved for 10 days and 
nights as both the fitness and life span of individu¬ 
als continued to increase (Fig. 76.19). After approx¬ 
imately 200 generations, the robot was capable of 
navigating around the environment, covering long tra¬ 
jectories while avoiding both walls and the recharging 
area (l<s>jHlitll!lllifl). When the battery was almost dis¬ 
charged it initiated a straight navigation towards the 
recharging area and exited immediately after battery 
recharge to resume navigation. Best evolved individuals 
always entered the recharging area one or two seconds 
before full discharge of the battery. That implies that 
robots must somehow calibrate the timing and trajec¬ 
tory of their homing behavior depending on where they 
happened to be in the environment. 

In order to understand how that behavior could pos¬ 
sibly be generated, a set of neuroethological measures 



Fig. 76.18 Bird’s eye view of the arena with the light tower 
over the recharging station and the Khepera robot 


were performed using a laser positioning device that 
provided exact position and orientation of the robot 
every 300 ms. By correlating the robot position and be¬ 
havior with the activation of the internal neurons in 
real time while the evolved individual freely moved 
in the environment, it was possible to see that some 
neurons specialized for reactive behaviors, such as 
obstacle avoidance, forward motion, and battery mon¬ 
itoring. Other neurons instead displayed more complex 
activation patterns. One of them revealed a pattern of 
activation levels that depended on whether the robot 
was oriented facing the light tower or facing the op¬ 
posite direction (Fig. 76.20). In the former case, the 
activation pattern reflected zones of the environment 
and paths typically followed by the robot during ex¬ 
ploration and homing. For example, the robot trajectory 
towards the recharging area never crossed the two gate 
walls visible in the activation maps around the recharg¬ 
ing station. When the robot faced the opposite direction, 
the same neuron displayed a gradient field orthogonally 
aligned with the recharging area. This gradient provides 
an indication of the distance from the recharging area. 
Interestingly, this pattern of activity is not significantly 
affected by the charge level of the battery. 

The functioning of this neuron reminds of the clas¬ 
sic findings on the hippocampus of the rat brain where 
some neurons (also known as place cells) selectively 
fire when the rat is in specific areas of the environ¬ 
ment [76.98]. Also, the orientation-specific pattern of 
neural activation measured on the evolved robot is rem- 


a) Fitness 




Fig. 76.19 (a) Average population fitness (continuous line) 
and fitness of the best individual (dotted line), (b) Life span 
of the best individuals measured as number of sensorimo¬ 
tor cycles, or actions. Individuals start with a full battery 
which lasts 50 actions (20 s), if not recharged. The maxi¬ 
mum life span is 150 actions 
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iniscent of the so-called head-direction neurons in the 
rat hippocampus, which are positioned nearby place 
cells, whose firing patterns depend on the rat head¬ 
ing direction with respect to an environmental land¬ 
mark [76.99]. Although the analogy between brains of 
evolved robots and of biological organisms should not 
be taken too literally, these results indicate that the two 
organisms converge towards a functionally similar neu¬ 
ral strategy, which may be more efficient to address 
this type of situation than a strategy that does not rely 
on representations (but only on reactive strategies such 
as random motion, light following, or dead reckon¬ 
ing). 

76.7.2 Spiking Neurons 

The great majority of biological neurons communicate 
using self-propagating electrical pulses called spikes, 
but from an information-theoretic perspective it is not 
yet clear how information is encoded in the spike 
train. Connectionist models [76.100], by far the most 
widespread, assume that what matters is the firing rate 
of a neuron, that is, the average quantity of spikes emit¬ 
ted by the neuron within a relatively long time window 
(for example, over 100 ms). Alternatively, what matters 
is the average number of spikes of a small population of 
neurons at a give point. In these models the real-valued 
output of an artificial neuron represents the firing rate, 
possibly normalized relatively to the maximum attain¬ 
able value. Pulsed models [76.101], instead, are based 
on the assumption that the firing time, that is, the pre¬ 
cise time of emission of a single spike, may convey 
important information [76. 102]. Spiking neuron models 
have slightly more complicated dynamics of synaptic 
and membrane integration. Depending on one’s theory 
of what really matters, connectionist or spiking models 
are used. 

However, designing circuits of spiking neurons that 
display a desired functionality is still a challenging task. 
The most successful results in the field of robotics ob¬ 
tained so far focused on the first stages of sensory pro¬ 
cessing and on relatively simple motor control [76. 103, 
104]. Despite these implementations, there are not yet 
methods for developing complex spiking circuits that 
could display minimally cognitive functions or learn be¬ 
havioral abilities through autonomous interaction with 
a physical environment. 

Artificial evolution represents a promising method¬ 
ology to generate networks of spiking circuits with 
desired functionalities expressed as behavioral criteria 
(fitness function). Evolved networks could then be ex¬ 
amined to detect what communication modality is used 
and how that correlates with observed behavior of the 
robot. 


Floreano and Mattiussi [76.105] evolved a fully 
connected network of spiking neurons for driving 
a vision-based robot in an arena painted with black 
stripes of variable size against a white background 
(Fig. 76.21). The Khepera robot used in these exper¬ 
iments was equipped with a vision turret composed 
of one linear array of grayscale photoreceptors span¬ 
ning a visual field of 36°. The output values of a bank 



Fig. 76.20 Activation levels (brightness proportional to activation) 
of an internal neuron plotted over the environment while the robot 
was positioned at various locations in each of the four conditions 
(facing recharging area or not, discharged battery or not). The 
recharging area is located at the top left corner of each map 



H i 

Fig. 76.21 A network of spiking neurons is evolved to 
drive the vision-based robot in the arena. The light be¬ 
low the rotating contacts allows continuous evolution also 
overnight 
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of local contrast detection filters were converted in 
spikes (the stronger the contrast, the larger the num¬ 
ber of spikes per second) sent to ten fully connected 
spiking neurons implemented according to the spike re¬ 
sponse model [76.106]. The spike series of a subset 
of these neurons was translated into motor commands 
(more spikes per second corresponded to faster rotation 
of the wheel). The fitness function was the amount of 
forward translation of the robot measured over 2 min. 
Consequently robots that turned in place or hit the 
walls had comparatively lower fitness than robots that 
could move straight and turn only to avoid walls. 
The genome of these robots was a bit string that en¬ 
coded only the sign of the neurons and the presence 
of synaptic connections. Existing connections were set 
to 1 and could not change during the lifetime of the 
robot. 

Evolution reliably discovered very robust spik¬ 
ing controllers in approximately 20 generations, ap¬ 
proximately 30 h of evolution on the real robot 
(l-saMUEHiEB). Evolved robots could avoid not only 
the walls, but any object positioned in front of them. De¬ 
tailed analysis of the best evolved controllers revealed 
that neurons did not exploit time differences between 
spikes, which one would have expected if optic flow 
was used to detect distance from walls. Instead, they 
simply used the number of incoming spikes (firing rate) 
as an indication of when to turn. When the robot per¬ 
ceived a lot of contrast it would go straight, but when 
the contrast decreased below a certain threshold (indi¬ 
cating that it approached an object), it started to turn 
away. This extremely efficient and simple result seems 
to be in contrast with theories of optic flow detection in 
insects and may be worth considering as an alternative 
hypothesis for vision-based behavior. 

Spiking neural networks turned out to be more 
evolvable than connectionist models (at least for this 
task). One possible explanation is that spiking neurons 
have subthreshold dynamics that, to some extent, can 
be affected by mutations without immediately affecting 
the output of the network. 

The robust results and compact genetic encoding 
encouraged the authors to use an even simpler model of 
spiking neuron so that the entire neural network could 
be mapped in less than 50 bytes of memory. The evo¬ 
lutionary algorithm was also reduced to a few lines of 
code and the entire system was implemented within 
a programmable intelligent computer (PIC) microcon¬ 
troller without the need for any external computer for 
data storage. The system was used for a sugar-cube 
robot (Fig. 76.22) that autonomously and reliably de¬ 
veloped the ability to navigate around a maze in less 
than an hour [76.107], Interestingly, evolved spiking 
controllers developed a pattern of connections where 


spiking neurons received connections from a small 
patch of neighboring sensors, but not from other sen¬ 
sors, and were connected only to neighboring spiking 
neurons. This pattern of connectivity is also observed 
in biological systems and encourages specialization of 
neurons to sensory features. 

76.7.3 GasNets 

This section describes another style of artificial neural 
network strongly inspired by those parts of contempo¬ 
rary neuroscience that emphasize the complex electro¬ 
chemical nature of real nervous systems. In particular, 
they make use of an analogue of volume signaling, 
whereby neurotransmitters freely diffuse into a rela¬ 
tively large volume around a nerve cell, potentially 
affecting many other neurons [76. 108, 109]. This exotic 
form of neural signaling does not sit easily with classi¬ 
cal pictures of brain mechanisms and is forcing a radical 
rethink of existing theory [76.110-113]. The class of 
artificial neural networks developed to explore artifi¬ 
cial volume signaling are known as GasNets [76.114]. 
These are essentially standard neural networks aug¬ 
mented by a chemical signaling system comprising 
a diffusing virtual gas which can modulate the response 
of other neurons. A number of GasNet variants, inspired 
by different aspects of real nervous systems, have been 
explored in an evolutionary robotics context as artifi¬ 
cial nervous systems for mobile autonomous robots. 
They have been shown to be significantly more evolv¬ 
able, in terms of speed to a good solution, than other 
forms of neural networks for a variety of robot tasks 
and behaviors [76.38, 114-116]. They are being inves¬ 
tigated as potentially useful engineering tools and as 
a way of gaining helpful insights into biological sys¬ 
tems [76.112, 117-119]. 

By analogy with biological neuronal networks, Gas¬ 
Nets incorporate two distinct signaling mechanisms, 
one electrical and one chemical. The underlying elec¬ 
trical network is a discrete-time-step recurrent neural 
network with a variable number of nodes. These nodes 



Fig. 76.22 The Alice sugar-cube robot equipped with the 
evolutionary spiking neural network implemented within 
its PIC microcontroller 
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are connected by either excitatory or inhibitory links 
(Fig. 76.23). 

In addition to this underlying network in which posi¬ 
tive and negative signals flow between units, an abstract 
process loosely analogous to the diffusion of gaseous 
modulators is at play. Some units can emit virtual gases 
which diffuse and are capable of modulating the behav¬ 
ior of other units by changing the profile of their out¬ 
put functions. The networks occupy a 2-D space; the 
diffusion processes mean that the relative positioning 
of nodes is crucial to the functioning of the network. 
Spatially, the gas concentration varies as an inverse ex¬ 
ponential of the distance from the emitting node with 
a spread governed by a parameter r with the concentra¬ 
tion set to zero for all distances greater than r. The total 
concentration of gas at a node is determined by sum¬ 
ming the contributions from all other emitting nodes. 

For mathematical convenience, in the original Gas- 
Net there are two gases, one whose modulatory effect 
is to increase the transfer function gain parameter and 
one whose effect is to decrease it. Thus the gas does 
not alter the electrical activity in the network directly 
but rather acts by continuously changing the mapping 
between input and output for individual nodes, either 
directly of by stimulating the production of further vir¬ 
tual gas. The general form of the diffusion is based on 
the properties of a (real) single-source neuron as mod¬ 
eled in detail by Philippides et al. [76.112, 117]. The 
modulation chosen is motivated by what is known of 
NO modulatory effects at synapses [76.120]. For full 
details see [76.114], 

Various extensions of the basic GasNet have been 
produced. Two in particular are strongly inspired by 
contemporary neuroscience. The plexus model is di¬ 
rectly inspired by a type of signaling seen in the 
mammalian cerebral cortex in which the NO signal 
is generated by the combined action of many fine 
NO-producing fibers, giving a targeted cloud which is 
distant from the neurons from which the fiber plexus 
emanates [76.118]. In the plexus GasNet, which mod¬ 
els this form of signaling at an abstract level, the spatial 
distribution of gas concentration has been modified to 
be uniform over the area of affect. The center of this 
gas diffusion cloud is under genetic control and can be 
distant from the controlling node (which, by analogy, is 
the source of the plexus) [76.116]. All other details of 
the models are identical to the original GasNet model, 
as described earlier. The receptor GasNet incorporates 
an aspect of biological neuronal networks that has no 
analog in the vast majority of artificial neural networks 
(ANNs): the role of receptor molecules. Although neu¬ 
roscience is a long way from a full understanding of 
receptor mechanisms, a number of powerful systems 
level ideas can be abstracted. 


Neuron 1 



Neuron 4 Neuron 5 


A GasNet. Neuron 3 is emitting gas, and modulating 

neuron 2 despite there being no synaptic connection. 

Fig. 76.23 A basic GasNet showing positive (solid) and 
negative ( dashed) electrical connections and a diffusing 
virtual gas creating a chemical gradient 

Details of the receptor variant are similar to the ba¬ 
sic GasNet except there is now only one virtual gas 
and each node in the network can have one of three 
discrete quantities (zero, medium, maximum) of a num¬ 
ber of possible receptors. The modulation the diffusing 
neurotransmitter affects at a neuron depends on which 
receptors are present. The strength of a modulation at 
a node is proportional to the product of the gas concen¬ 
tration at the node and the relevant receptor quantity. In 
the original GasNet, any node that was in the path of 
a diffusing transmitter would be modulated in a fixed 
way. The receptor model allows site-specific modu¬ 
lations, including no modulation (no receptors) and 
multiple modulations at a single site ([76.116] for fur¬ 
ther details). 

Although most of the GasNet variants described in 
this section have been successfully used in a number 
of robotic tasks, their evolvability and other properties 
were thoroughly compared on a version of the (gantry) 
robot visual discrimination task described in Sect. 76.5, 
|43>MHhl£iikl. All aspects of the networks were under 
genetic control: the number of nodes, the connectivity 
and, in the case of the GasNets, all parameters gov¬ 
erning volume signaling (including the position of the 
nodes and whether or not they were virtual gas emit¬ 
ters). The visual sampling morphology was also under 
evolutionary control. The original basic GasNet was 
found to be significantly more evolvable than a vari¬ 
ety of other styles of connectionist neural networks as 
well as a GasNet with the volume signaling disabled. 
Successful GasNet controllers for this task tended to be 
rather minimal, in terms of numbers of nodes and con¬ 
nections, while possessing complex dynamics [76.1 14]. 
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Later experiments comparing the basic GasNet with the 
plexus and receptor variants showed the latter two to be 
considerably more evolvable than the former, with the 
receptor GasNet being particularly successful [76.116]. 
These GasNet experiments demonstrated that the intri¬ 
cate network dynamics made possible by the artificial 
volume signaling mechanisms can be readily harnessed 
to generate adaptive behaviors in autonomous agents. 
They also throw up such questions as why GasNets 
are more evolvable than many other forms of ANN 
and why there is a difference in evolvability between 
GasNet variants. In order to gain insight into what fac¬ 
tors are most important in GasNet evolvability, several 
other varieties were studied, including non-spatial Gas- 
Nets where the diffusion process is replaced by explicit 
gas connections with complex dynamics and version 
with other forms of modulation and diffusion [76.119], 
Detailed comparative studies of these variants with 
each other, and with other forms of ANN, were per¬ 
formed using the visual discrimination task described 
above [76.1 16,1 19]. 

The comparative studies revealed that the rich dy¬ 
namics and additional timescales introduced by the 
gas played an important part in enhanced evolvabil¬ 
ity, but were not the whole story [76.116,119]. The 
particular form of modulation was also important - 
multiplicative or exponential modulation (in the form 
of changes to the transfer function) were found to be 
effective, but additive modulations were not. The for¬ 
mer kind of modulations may well confer evolutionary 
advantages by allowing nodes to be sensitive to differ¬ 
ent ranges of input (internal and sensory) in different 
contexts. The spatial embedding of the networks also 
appears to play a role in producing the most effective 
coupling between the two distinct signalling processes 
(electrical and chemical). By exploiting a loose, flexi¬ 
ble coupling between the two processes, it is possible 
to significantly reduce destructive interference between 
them, allowing one to be tuned against the other while 
searching for good solutions [76.1 15,116, 121]. Similar 
forces may be at play in spiking neural networks, where 
sub-threshold and spiking dynamics interact with each 
other, which although not yet compared to GasNets, 
were shown to be more evolvable than connectionist 
networks. Measurements of the degree of coupling in 
the GasNets variants versus speed of evolution sup¬ 
ported this view [76.116]; the receptor GasNet, for 

76.8 Evolution and Learning 

Evolution and learning (or phylogenetic and ontoge¬ 
netic adaptation) are two forms of biological adaptation 
that differ in space and time. Evolution is a process 


which the evolutionary search process has the most di¬ 
rect control over the degree of coupling between the sig¬ 
naling processes, and which has a bias towards a loose 
coupling, was by far the most evolvable [76.1 16]. 

Analysis of GasNet solutions often reveals high 
levels of degeneracy, with functionally equivalent sub¬ 
networks occurring in many different forms, some in¬ 
volving gas and some not [76.121]. Their genotype to 
phenotype mapping (where the phenotype is robot be¬ 
havior) is also highly degenerate with many different 
ways of achieving the same outcome (e.g., moving node 
positions, changing gas diffusion parameters or adding 
new connections can all have the same effect). This is 
especially true when variable length genotypes are used 
to efficiently sculpt solutions in a search space of vari¬ 
able dimensions. The levels of degeneracy are generally 
significantly higher than when using connectionist net¬ 
works. These properties partly explain the robustness 
and adaptability of GasNets in noisy environments and 
are another important factor in their evolvability (there 
are many paths to the same phenotypical outcome with 
reduced probabilities of lethal mutations) [76.1 19, 122]. 
In the most successful varieties of GasNet, multi-scale 
dynamics, modulation and spatial embedding act in 
concert to produce highly evolvable degenerate net¬ 
works. 

These and ongoing investigations indicate that ex¬ 
plicitly dealing with the electrochemical nature of ner¬ 
vous systems is likely to be an increasingly fruitful 
area of research, both for evolutionary robotics and for 
neuroscience, that will likely force us to broaden our 
notions of what behavior-generating mechanisms might 
look like. 

Because of its ability to explore whole classes of 
underspecified models, ER is being increasingly used 
to develop or explore neural models aimed at answer¬ 
ing specific questions in neuroscience [76.88-90] or 
to probe new theories about possible neural mecha¬ 
nisms [76.90]. One intriguing recent hypothesis is that 
one of the forms of plasticity on which the brain relies 
is itself a form of evolution via natural selection acting 
within neural tissue [76.123, 124]. The units of selec¬ 
tion in this case are activity and connection patterns 
which are copied between groups of neurons. Irrespec¬ 
tive of whether or not it occurs in nature (and it might), 
this kind of mechanism could be employed in a whole 
new kind of evolutionary robotics. 


of selective reproduction and substitution based on 
the existence of a population of individuals display¬ 
ing variability at the genetic level. Learning, instead. 
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is a set of modifications taking place within each sin¬ 
gle individual during its own life time. Evolution and 
learning operate on different time scales. Evolution is 
a form of adaptation capable of capturing relatively 
slow environmental changes that might encompass sev¬ 
eral generations (e.g., the perceptual characteristics of 
food sources for a given species). Learning, instead, 
allows an individual to adapt to environmental modifi¬ 
cations that are unpredictable at the generational level. 
Learning might include a variety of mechanisms that 
produce adaptive changes in an individual during its 
lifetime, such as physical development, neural matura¬ 
tion, variation of the connectivity between neurons, and 
synaptic plasticity. Finally, whereas evolution operates 
on the genotype, learning affects only the phenotype, 
and phenotypic modifications cannot directly modify 
the genotype. 

Researchers have combined evolutionary tech¬ 
niques and learning techniques (supervised or unsuper¬ 
vised learning algorithm such us reinforcement learning 
or Hebbian learning; for a review see [76.125]). These 
studies have been conducted with two different pur¬ 
poses: 

1. Identifying the potential advantage of combining 

these two methods from the point of view of devel¬ 
oping robust and effective robots. 

2. Understanding the role of the interaction between 

learning and evolution in nature. 

Within an evolutionary perspective, learning has 
several different adaptive functions. First, it might al¬ 
lows individuals to adapt to changes that occur too 
quickly to be tracked by evolution [76.126]. Secondly, 
learning might allows robots to use information ex¬ 
tracted during their interaction with environment to 
develop adaptive characters ontogenetically without 
necessarily discovering these characters through ge¬ 
netic variations and without encoding these characters 
in their genome. To understand the importance of this 
aspect, we should consider that evolutionary adaptation 
is based on an explicit but concise indication of how 
well an individual robot coped with its environment - 
the fitness value of a robot. Ontogenetic adaptation, on 
the contrary, is based on extremely rich information - 
the state of the sensors while the robot interacts with 
its environment. This huge amount of information en¬ 
codes very indirectly how well an individual is doing in 
different phases of its lifetime or how it should modify 
its behavior to increase its fitness. However, evolving 
robots that have acquired a predisposition to exploit 
this information to produce adaptive changes during 
their lifetime might be able to develop adaptive char¬ 
acteristics on the fly, thus leading to the possibility to 


produce complex phenotypes on the basis of parsimo¬ 
nious genotypes. Finally, learning can help and guide 
evolution. Although physical changes of the phenotype, 
such as strengthening of synapses during learning, can¬ 
not be written back into the genotype, Baldwin [76. 127] 
and Waddington [76.128] suggested that learning might 
indeed affect the evolutionary course in subtle but ef¬ 
fective ways. Baldwin’s argument was that learning 
accelerates evolution because suboptimal individuals 
can reproduce by acquiring during life necessary fea¬ 
tures for survival. However, variation occurring during 
successive generation might lead to the discovery of ge¬ 
netic traits that lead to the establishment of the same 
characteristics that were previously acquired thorough 
lifetime learning. This latter aspect of Baldwin’s ef¬ 
fect, namely indirect genetic assimilation of learned 
traits, has been later supported by scientific evidence 
and defined by Waddington [76.128] as a canalization 
effect. 

Learning however, also has costs such as: (1) a de¬ 
lay in the ability to acquire fitness (due to the need 
to develop fit behavior ontogenetically), and (2) in¬ 
creased unreliability due to the fact that the possibility 
to develop certain abilities ontogenetically is subjected 
to partially unpredictable characteristics of the robot- 
environment interaction [76.129]. In the next two sub¬ 
sections we describe two experiments that show some 
of the potential advantages of combining evolution and 
learning. 

76.8.1 Learning to Adapt to Fast 
Environmental Variations 

Consider the case of a Khepera robot that should ex¬ 
plore an arena surrounded by black or white walls 
to reach a target placed in a randomly selected loca¬ 
tion [76.126]. Evolving robots are provided with eight 
sensory neurons that encode the state of the four corre¬ 
sponding infrared sensors and two motor neurons that 
control the desired speed of the two wheels. Since the 
color of the walls change every generation and since the 
color significantly affects the intensity of the response 
of the infrared sensors, evolving robots should develop 
an ability to infer whether they are currently located in 
an environment with white or black walls and learn to 
modify their behavior during lifetime. That is, robots 
should avoid walls only when the infrared sensors are 
almost fully activated in the case of arenas with white 
walls, while they should avoid walls even when the in¬ 
frared sensors are slightly activated in the case of arenas 
with black walls. 

Robots were provided with a neural controller 
(Fig. 76.24) including four sensory neurons that en¬ 
coded the state of four corresponding infrared sensors; 
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two motors neurons that encoded the desired speed of 
the two wheels; and two teaching neurons that encoded 
the teaching values used to modify the connection 
weights from the sensory neurons to the motor neurons 
during the robots’ lifetime. This special architecture al¬ 
lows evolving robots to transform the sensory states 
experienced by the robots during their lifetime into 
teaching signals that might potentially lead to adaptive 
variations during lifetime. Analysis of evolved robots 
revealed that they developed two different behaviors 
that are adapted to the particular arena where they hap¬ 
pen to be born (surrounded by white or black walls). 
Evolving robots did not inherit an ability to behave ef¬ 
fectively, but rather a predisposition to learn to behave. 
This predisposition to learn involves several aspects 
such as a tendency to experience useful learning experi¬ 
ences, a tendency to acquire useful adaptive characters 
through learning, and a tendency to channel variations 
toward different directions in different environmental 
conditions [76.126]. 

76.8.2 Evolution of Learning 

In the previous example, the evolutionary neural net¬ 
work learned using a standard learning rule that was 
applied to all synaptic connections. Floreano and Mon- 
dada [76.130] explored the possibility of genetically 
encoding and evolving the learning rules associated to 
the different synaptic connections of a neural network 
embedded in a real robot. The main motivation of this 
line of work was to evolve robots capable of adapt¬ 
ing to a partially unknown environment, rather than 
robots adapted to the environment(s) seen during evo¬ 
lution. In order to prevent evolutionary tuning of the 
neural network to the specificities of the evolutionary 
environment (which would limit transfer to different 


Motors Teaching 



Fig. 76.24 A self-teaching network. The output of the two 
teaching neurons is used as a teaching value for the two 
motor neurons. The weights that connect the sensory neu¬ 
rons to the teaching neurons do not vary during the robots’ 
lifetime while the weights that connect the sensory neurons 
to the motor neurons are modified with an error-correction 
algorithm 


environments or transfer from simulation to reality), 
the synaptic weight values were not genetically en¬ 
coded. Instead, each synaptic connection in the network 
was described by three genes that defined its sign, its 
learning rule, and its learning rate (Fig. 76.25). Every 
time a genome was decoded into a neural network and 
downloaded onto the robot, the synaptic strengths were 
initialized to small random values and could change ac¬ 
cording to the genetically specified rules and rates while 
the robot interacted with the environment. Variations of 
this methodology included a more compact genetic en¬ 
coding where the learning properties were associated to 
a neuron instead of a synapse. All synapses afferent to 
a neuron used its genetically specified rules and rates. 
Genes could encode four types of Hebbian learning that 
were modeled upon neurophysiological data and were 
complementary to each other [76.131]. 

Experimental results in a nontrivial, multitask en¬ 
vironment (Fig. 76.26, I^spMTI'H'XU) indicated that 
this methodology has a number of significant advan¬ 
tages with respect to the evolution of synaptic strengths 
without learning [76.48]. Robots evolved faster and 
obtained better fitness values. Furthermore, evolved be¬ 
haviors were qualitatively different, notably in that they 
did not exploit minimal solutions tuned to the envi¬ 
ronment (such as turning only on one side, or turning 
in circles tuned to the dimensions of the evolutionary 
arena). Most important, these robots displayed remark¬ 
able adaptive properties after evolution. Best evolved 
individuals: (1) transferred perfectly from simulated 
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Fig. 76.25 Two methods for genetically encoding a synap¬ 
tic connection. Genetically determined synapses cannot 
change during the lifetime of the robot. Adaptive synapses 
instead are randomly initialized and can change during 
lifetime of the robot according to the learning rules and 
rates specified in the genome 
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to physical robots, (2) accomplished the task when 
the light and reflection properties of the environment 
were modified, (3) accomplished the task when key 
landmarks and target areas of the environment were 
displaced, and (4) transferred well across morpholog¬ 
ically different robotic platforms. In other words, these 
robots were selected for their ability to solve a par¬ 
tially unknown problem by adapting on the fly, rather 
than for being a solution to the problem seen during 
evolution. 

In further experiments where the genetic code for 
each synapse of the network included one gene whose 
value caused its remaining genes to be interpreted as 
connection strengths or learning rules and rates, 80% of 
the synapses made the choice of using learning, rein¬ 
forcing the fact that this genetic strategy has a compar¬ 
atively stronger adaptive power [76.131]. This method¬ 
ology could also be used to evolve the morphology of 
neural controllers were synapses are created at runtime 
and therefore their strengths cannot be genetically spec¬ 



Fig. 76.26 (a) A mobile robot Khepera equipped with a vision 
module can gain fitness points by staying on the grey area only 
when the light is on. The light is normally off, but it can be switched 
on if the robot passes over the black area positioned on the other 
side of the arena. The robot can detect ambient light and wall color, 
but not the color of the floor, (b) Behavior of an individual evolved 
in simulation with genetic encoding of learning rules 


ified [76.132]. Recently, the adaptive properties of this 
type of adaptive genetic encoding were confirmed also 
in the context of evolutionary spiking neurons for robot 
control [76.133]. 


76.9 Evolution of Social Behavior 

In the previous sections, we limited our analysis to indi¬ 
vidual behaviors, i. e., to the evolution of robots placed 
in an environment that does not include other robots. 
The evolutionary method, however, can also be applied 
to evolve social behaviors in which multiple robots 
situated in the same environment interact between 
themselves in cooperative or competitive manners. 

As we will see, competitive co-evolution is particu¬ 
larly interesting from the point of view of synthesizing 
progressively more complex capacities and from the 
point of view of developing solutions that are robust 
with respect to environmental variations. Cooperative 
evolution instead is particularly interesting for the pos¬ 
sibility to solve problems that cannot be handled by 
a single robot, because of physical constraints or limited 
behavioral capabilities [76.134] and to develop solu¬ 
tions that are robust. 

76.9.1 Coevolving Predator and Prey Robots 

Competitive coevolution, for example the coevolution 
of two populations of predator and prey robots that 
are evolved for the ability to catch prey and to es¬ 
cape predators, respectively, has two characteristics 
that are particularly interesting from an evolutionary 
robotics perspective. The first aspect is that the competi¬ 
tion between populations with different interests might 
spontaneously lead to a sort of incremental evolution¬ 
ary process where evolving individuals are faced with 


progressively more complex challenges (although this 
is not necessarily the case). Indeed, in initial genera¬ 
tions the task of the two populations is relatively simple 
because opponents have simple and poorly developed 
abilities on average. After a few generations, however, 
the abilities of the two populations increase and, con¬ 
sequently, the challenges for each population become 
more difficult. The second aspect consists of the fact 
that the environment varies across generations because 
it includes other coevolving individuals. This implies 
that coevolving individuals should be able to adapt to 
ever-changing environments and to develop behaviors 
that are robust with respect to environmental varia¬ 
tions [76.135]. 

The potential advantages of competitive coevolu¬ 
tion for evolutionary robotics have been demonstrated 
by a set of experiments conducted by Floreano and 
Nolfi [76.136,137] where two populations of robots 
were evolved for the ability to catch prey and escape 
predators, respectively (Fig. 76.27). 

The results indicated that both predator and prey 
robots tended to vary their behavior throughout gen¬ 
erations without converging on a stable strategy. The 
behavior displayed by individuals at each generation 
tended to be tightly adapted to the counter-strategy 
exhibited by the opponent of the same generation 
(|4E!«iL!i£I;3:|). This evolutionary dynamic however 
does not really lead to long-lasting progress because, 
after an initial evolutionary phase, the coevolutionary 
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Fig. 76.27 Experimental setup. The predator and prey 
robot (from left to right) are placed in an arena surrounded 
by walls and are allowed to interact for several trials start¬ 
ing at different randomly generated orientations. Predators 
are selected on the basis of the percentage of trials in which 
they are able to catch (i. e., to touch) the prey, and prey on 
the basis of the percentage of trials in which they were able 
to escape (i. e., to not be touched by) predators. Predators 
have a vision system, whereas the prey have only short- 
range distance sensors, but can go twice as fast as the 
predator. Collision between the robots is detected by a con¬ 
ductive belt at the base of the robots 



Fig. 76.28 An s-bot and a simulated swarm-bot consisting of four 
s-bots assembled in chain formation 


process led to a limit cycle dynamic where the same 
small set of behavioral strategies recycled over and over 
again along generations [76.137]. This limit cycle dy¬ 
namic can be explained by considering that prey robots 
tended to vary their behavior in order to disorient preda¬ 
tors as soon as predators become effective against the 
current behavioral strategies exhibited by prey robots. 

However, experiments [76.138] where robots were 
allowed to change their behavior on the fly on the ba¬ 
sis of unsupervised Hebbian learning rules showed that 


the evolutionary phase where coevolving robots were 
able to produce real progress was significantly longer, 
and evolved predators displayed an ability to effectively 
cope with prey exhibiting different behavioral strate¬ 
gies by adapting their behavior on the fly to the prey’s 
behavior. Prey instead tended to display behavior that 
changed in unpredictable ways. 

Further experiments showed that competitive co¬ 
evolution can solve problem that the evolution of a sin¬ 
gle population cannot. Nolfi and Floreano [76.137] 
demonstrated that the attempt to evolve predators robot 
for the ability to catch a fixed pre-evolved prey pro¬ 
duced lower performance with respect to control exper¬ 
iments where predators and prey were coevolved at the 
same time. 

76.9.2 Evolving Cooperative Behavior 

As testified by social insects, colonies of simple co¬ 
operating individuals can display remarkable capaci¬ 
ties and exhibit self-organising behaviors in which the 
spatio-temporal pattern observed at the system level 
emerge from numerous interactions among the indi¬ 
vidual robots. On the other hand, designing collec¬ 
tive robotic systems of this sort constitutes a difficult 
problem due to the indirect relationship between the 
desired group behavior and the characteristics of the 
individual robots. By evaluating the robotic system as 
a whole (i. e., by selecting the robots on the basis of 
the global behavior that emerge from a large number 
of robot/environmental and robot/robot interactions), 
Evolutionary Robotics provides a means for discover¬ 
ing effective behavioral solutions and simple and robust 
control policies [76.139]. 

Recent research showed that teams of evolved 
robots can: 

1. Develop robust and effective coordinated behav¬ 
ior [76.140,141] 

2. Collaborate by assuming complementary 

role [76.141, 142] (I^JEESEa) 

3. Display self-organizing properties [76.143] 

4. Develop and use communicative capabili¬ 
ties [76.144—146]. 

Moreover, some of the research carried in this 
area demonstrated how evolutionary robotics experi¬ 
ments can contribute to model biological phenomena, 
e.g., to identify the evolutionary conditions that enable 
the emergence of cooperative communicative behav¬ 
iors [76.146] or the mechanisms enabling the evolution 
of effective division of labour strategies [76.147] 
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Here we briefly review a series of experiments 
where swarm-bots [76.148], i. e., groups of autonomous 
robots capable of assembling by physically connect¬ 
ing together, were evolved for the ability to display 
coordinated motion Each individual 

robot consisted of a main platform (chassis) and tur¬ 
ret that could actively rotate with respect to each other 
(Fig. 76.28). The chassis included tracks with teethed 
wheels for navigation on both rough and flat terrain, 
and infrared sensors pointing to the ground. The tur¬ 
ret included a gripper, sixteen light-sensors distributed 
around the body, a loudspeaker, three microphones, 
and a traction sensor placed between the turret and 
the chassis to detect the direction and the intensity of 
the traction force that the turret exerts on the chassis. 
Swarm-bots were formed by several robots provided 
with identical neural controllers and assembled together 
so to form a single physical entity. 

By evolving the neural controllers of these Swarm- 
bots, Baldassare et al. [76.140] demonstrated how the 
robots can display a robust and effective coordinated 
capacities that allow the individuals to negotiate and 
converge on a coherent direction and to keep mov¬ 
ing along that direction by compensating the dis¬ 
alignments originating during motion. Such behav¬ 
ioral capacity was robust enough to allow a smooth 
transfer from simulation to reality and to allow the 
robots to to generalize their capacity to rugged ter¬ 
rains. In an extended experiments in which the s-bots 
were also equipped with infrared sensors, speakers 
and microphones, the evolved swarm-bots also showed 
a capacity to avoid dangers (e.g., holes) by coordi- 
nately changing direction as soon as one s-bot detected 
a hole [76.149]. 

Evolved swarm-bots generalized their coordinated 
motion capabilities also when they were tested in differ¬ 
ent conditions (e.g., when they were assembled in much 
more numerous groups and/or in different topologies, or 
when they had to also carry heavy objects by pushing 
and pulleding them in a coordinated manner). Finally, 
when placed in new environmental conditions (e.g., in 
environment with obstacle and walls), the swarm-bots 
spontaneously displayed new behavioral skills (related 
to acquired skills), such as the ability to cooperatively 
avoid obstacles, without any further adaptation. This 
ability to display new related behaviors, in new behav¬ 
ioral conditions, emerged as a result of the dynamical 
process originating from the interaction of the same 
robots with the new environmental conditions [76.142, 
150], 


76.9.3 Evolution of Communication 

Communication represents a key aspect in collective 
behaviors. Recent research in evolutionary robotics has 
demonstrated how sophisticated communication capa¬ 
bilities can emerge and evolve in population of robots 
selected for the ability to perform tasks requiring coor¬ 
dination and/or cooperation. 

The analysis of these experiments indicate that 
communicative interactions often originate as the re¬ 
sult of cues, that provide useful information to other 
robots, produced inadvertently during the execution of 
specific behaviors [76.146,151]. The presence of these 
cues create the basis for the development of an abil¬ 
ity to react to them in an adaptive way thus leading 
to the establishment of adaptive communicative inter¬ 
actions in which robots produce signals and react to 
detected signals adaptively. The establishment of these 
forms of communicative interactions then create the 
adaptive conditions for the co-evolution of signalling 
and response strategies [76.152] (|o>.MlliLi!iilH). 

The reliability and stability of the resulting com¬ 
munication system depend on the level of relatedness 
(i. e., genetic similarity) between robots and the level at 
which they were selected [76.146], Robots that are ge¬ 
netically higly related or that are selected on the basis of 
the behavior exhibited by the group evolve reliable sig¬ 
nals and stable communicative conventions. In contrast, 
when relatedness between robots is low and selection 
is acting at the level of the individuals, the evolution¬ 
ary process might lead to the emergence of instable, 
ineffective and in some case deceptive communication 
forms [76.153,154]. 

The evolution of communication is strongly inter¬ 
linked with the evolution of other behavioral capac¬ 
ities [76.155]. Indeed, after all, robots need to de¬ 
velop appropriate behaviors to access and/or generate 
the information to be communicated and/or to react 
to detected signals appropriately. The co-adaptation 
of behavioral and communicative skills might lead 
to prolonged innovation phases in which the devel¬ 
opment of behavioral capacities create the adaptive 
conditions for the development of communication ca¬ 
pacities and vice versa [76.151,152]. Moreover, the 
co-adaptation of behavioral and communication capac¬ 
ities tend to lead to highly contingent evolutionary 
processes in which the capacities possessed by the 
population at a certain evolutionary phase strongly in¬ 
fluence the outcome of the successive phases [76.152, 
156] (l<s«ii!l£lIM). 
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76.10 Evolutionary Hardware 


In recent years, technology advancements have allowed 
researchers to explore evolution of electronic circuits. 
In this section, we briefly summarize some foundational 
work in this direction. 

76.10.1 Evolvable Hardware 
Robot Controllers 

In most of the work discussed so far some form of ge¬ 
netically specified neural network, implemented in soft¬ 
ware, has been at the center of the robot control system. 
Work on a related approach of evolving control sys¬ 
tems directly onto hardware dates back to Thompson’s 
work in the mid 1990s [76.157]. In contrast to hard¬ 
ware controllers that are designed or programmed to 
follow a well-defined sequence of instructions, evolved 
hardware controllers are directly configured by evolu¬ 
tion and then allowed to behave in real time according 
to semiconductor physics. By removing standard elec¬ 
tronics design constraints, the physics can be exploited 
to produce highly nonstandard and often very efficient 
and minimal systems [76.158], 

Thompson [76.157] used artificial evolution to de¬ 
sign an onboard hardware controller for a two-wheeled 
autonomous mobile robot engaged in simple wall- 
avoidance behavior in an empty arena. Starting from 
a random orientation, and position near the wall, the 
robot had to move to the center of the arena and stay 



Fig.76.29a-d Wall-avoidance behavior of a robot with an evolved 
hardware controller in virtual reality (a-c) and the real world (d) 


there using limited sensory input (Fig. 76.29). The di¬ 
rect current (DC) motors driving the wheels were not 
allowed to run in reverse and the robot’s only sensors 
were a pair of time-of-flight sonars rigidly mounted on 
the robot, pointing left and right. 

Thompson’s approach made use of a so-called dy¬ 
namic state machine (DSM) - a kind of generalized 
read-only memory (ROM) implementation of a finite- 
state machine where the usual constraint of strict syn¬ 
chronization of input signals and state transitions are 
relaxed (in fact put under evolutionary control). The 
system had access to a global clock whose frequency 
was also under genetic control. Thus evolution deter¬ 
mined whether each signal was synchronized to the 
clock or allowed to flow asynchronously. This allowed 
the evolving DSM to be tightly coupled to the dynam¬ 
ics of interaction between the robot and environment 
and for evolution to explore a wide range of systems 
dynamics. The process took place within the robot in 
a kind of virtual reality in the sense that the real evolv¬ 
ing hardware controlled the real motors, but the wheels 
were just spinning in the air. The movements that the 
robot would have actually performed if the wheels had 
been supporting it were then simulated and the sonar 
echo signals that the robot was expected to receive were 
supplied in real time to the hardware DSM. Excellent 
performance was attained after 35 generations, with 
good transfer from the virtual environment to the real 
world (Fig. 76.29). 

Shortly after this research was performed, particu¬ 
lar types of field programmable gate arrays (FPGAs) 
which were appropriate for evolutionary applications 
became available. FPGAs are reconfigurable systems 
allowing the construction of circuits built from ba¬ 
sic logic elements. Thompson exploited their prop¬ 
erties to demonstrate evolution directly in the chip. 
By again relaxing standard constraints, such as syn¬ 
chronizing all elements with a central clock, he was 
able to develop very novel forms of functional cir¬ 
cuits, including a controller for a Khepera robot 
using infrared sensors to avoid obstacles [76.158, 
159], 

Following Thompson’s pioneering work, Keymeu- 
len et al. evolved a robot control system using 
a Boolean function approach implemented on gate- 
level evolvable hardware [76.160]. This system acted 
as a navigation system for a mobile robot capable of 
locating and reaching a colored ball while avoiding ob¬ 
stacles. The robot was equipped with infrared sensors 
and an vision system giving the direction and distance 
to the target. A programmable logic device (PLD) was 
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used to implement a Boolean function in its disjunc¬ 
tive form. This work demonstrated that such gate-level 
evolvable hardware was able to take advantage of the 
correlations in the input states and to exhibit useful 
generalization abilities, thus allowing the evolution of 
robust behavior in simulation followed by a good trans¬ 
fer into the real world. 

In a rather different approach, Ritter et al. used 
an FPGA implementation of an onboard evolution¬ 
ary algorithm to develop a controller for a hexapod 


76.11 Closing Remarks 

Evolutionary robotics is a young and integrated ap¬ 
proach to robot development without human inter¬ 
vention where machines change and adapt by cap¬ 
italizing on the interactions with their environment. 
Despite initial skepticism by mainstream and applied 
robotics practitioners and even by pioneers of this ap¬ 
proach [76.164], over the years the field has been 
constantly growing with new methods and approaches 
for evolving more complex, efficient, and sometimes 
surprising robotic systems. In some areas, such as mor¬ 
phology and self-assembly, evolutionary robotics is still 
the most widely used and powerful approach. 

Evolutionary robotics is not only a method for au¬ 
tomatic robot development inspired by biology, but 


robot [76.161], Roggen et al. devised a multicellu¬ 
lar reconfigurable circuit capable of evolution, self¬ 
repair, and adaptation [76.162], and used it as a sub¬ 
strate for evolving spiking controllers of a wheeled 
robot [76.163]. Although evolved hardware controllers 
are not widely used in evolutionary robotics, they still 
hold out the promise of some very useful properties, 
such as robustness to faults, which make them inter¬ 
esting for extreme condition applications such as space 
robotics. 


also a tool for investigating open questions in biology 
concerning evolutionary, developmental, and brain dy¬ 
namics. Its richness and fecundity make us believe that 
this approach will continue to grow and progress to¬ 
wards the creation of a new species of machines capable 
of self-evolution. 

To gain a pratical knowledge, interested read¬ 
ers might use software libraries such as frame¬ 
work for autonomous robotics simulation and analysis 
(FARSA) [76.165], an open-software tool that permit 
to carry on evolutionary robotics experiments based 
on a variety of robotic platforms and to replicate and 
vary some of the experiments described in this chap¬ 
ter [76.166]. 
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Visual navigation of mobile robot with pan-tilt camera 

available from http://handbookofrobotics.org/view-chapter/76/videodetails/36 

Visual navigation with collision avoidance 

available from http://handbookofrobotics.org/view-chapter/76/videodetails/37 
Coevolved predator and prey robots 

available from http://handbookofrobotics.org/view-chapter/76/videodetails/38 
Evolution of collision-free navigation 

available from http://handbookofrobotics.org/view-chapter/76/videodetails/39 
Online learning to adapt to fast environmental variations 
available from http://handbookofrobotics.org/view-chapter/76/videodetails/40 
iCub language comprehension 

available from http://handbookofrobotics.org/view-chapter/76/videodetails/41 
Resilent machines through continuous self-modeling 

available from http://handbookofrobotics.org/view-chapter/76/videodetails/114 

A swarm-bot of eight robots displaying coordinated motion 

available from http://handbookofrobotics.org/view-chapter/76/videodetails/115 

Discrimination of objects through sensory-motor coordination 

available from http://handbookofrobotics.org/view-chapter/76/videodetails/116 

Evolution of cooperative and communicative behaviors 

available from http://handbookofrobotics.org/view-chapter/76/videodetails/117 

Exploration and homing for battery recharge 

available from http://handbookofrobotics.org/view-chapter/76/videodetails/118 
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Introduction to evolutionary robotics at EPFL 

available from http://handbookofrobotics.org/view-chapter/76/videodetails/119 
Evolution of visually guided behavior on Sussex gantry robot 
available from http://handbookofrobotics.org/view-chapter/76/videodetails/371 
Evolved walking in an Octpod 

available from http://handbookofrobotics.org/view-chapter/76/videodetails/372 
Evolved homing walk on rough ground 

available from http://handbookofrobotics.org/view-chapter/76/videodetails/373 
Evolved bipedal walking 

available from http://handbookofrobotics.org/view-chapter/76/videodetails/374 
Evolved GasNet visualization 

available from http://handbookofrobotics.org/view-chapter/76/videodetails/375 
Evolved group coordination 

available from http://handbookofrobotics.org/view-chapter/76/videodetails/376 
Morphological change in an autonomous robot 

available from http://handbookofrobotics.org/view-chapter/76/videodetails/771 

More complex robots evolve in more complex environments 

available from http://handbookofrobotics.org/view-chapter/76/videodetails/772 
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77. Neurorobotics: From Vision to Action 


Patrick van der Smagt, Michael A. Arbib, Giorgio Metta 


The lay view of a robot is a mechanical human, 
and thus robotics has always been inspired by 
attempts to emulate biology. In this chapter, we 
extend this biological motivation from humans to 
animals more generally, but with a focus on the 
central nervous systems in its relationship to the 
bodies of these creatures. In particular, we in¬ 
vestigate the sensorimotor loop in the execution 
of sophisticated behavior. Some of these sections 
concentrate on cases where vision provides key 
sensory data. Neuroethology is the study of the 
brain mechanisms underlying animal behavior, 
and Sect. 77.2 exemplifies the lessons it has to 
offer robotics by looking at optic flow in bees, vi¬ 
sually guided behavior in frogs, and navigation in 
rats, turning then to the coordination of behaviors 
and the role of attention. Brains are composed of 
diverse subsystems, many of which are relevant to 
robotics, but we have chosen just two regions of 
the mammalian brain for detailed analysis. Sec¬ 
tion 77.3 presents the cerebellum. While we can 
plan and execute actions without a cerebellum, 
the actions are no longer graceful and become 
uncoordinated. We reveal how a cerebellum can 
provide a key ingredient in an adaptive control 
system, tuning parameters both within and be¬ 
tween motor schemas. Section 77.4 turns to the 
mirror system, which provides shared represen¬ 
tations which bridge between the execution of 
an action and the observation of that action when 
performed by others. We develop a neurobiological 
model of how learning may forge mirror neurons 
for hand movements, provide a Bayesian view of 
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a robot mirror system, and discuss what must be 
added to a mirror system to support robot im¬ 
itation. We conclude by emphasizing that, while 
neuroscience can inspire novel robotic designs, it is 
also the case that robots can be used as embodied 
test beds for the analysis of brain models. 
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77.1 Definitions and History 

Neurorobotics may be defined as: 

the design of computational structures for robots in¬ 
spired by the study of the nervous systems of humans 
and other animals. 

We note the success of (deep) artificial neural net¬ 
works - networks of simple computing elements whose 
connections change with experience - as providing 
a medium for parallel adaptive computation that has 
seen application in robot vision systems and controllers 
but here we emphasize neural networks derived from 
the study of specific neurobiological systems. Neuro¬ 
robotics has a twofold aim: creating better machines 
which employ the principles of natural neural compu¬ 
tation; and using the study of bio-inspired robots to 
improve understanding of the functioning of the brain. 
Chapter 75, Biologically Inspired Robots, complements 
our study of brain design with work on body design, the 
design of robotic control and actuator systems based on 
careful study of the relevant biology. 

77.1.1 History and Definitions 

Science has long been playing with technical replicas 
of biological behavior. As a famous example, Wal¬ 
ter [77.1] described two biologically inspired robots, 
the electromechanical tortoises Machina speculatrix 
and M. docilis (though each body has wheels, not 
legs). M. speculatrix has a steerable photoelectric cell, 
which makes it sensitive to light, and an electrical 
contact, which allows it to respond when it bumps 
into obstacles. The photoreceptor rotates until a light 
of moderate intensity is registered, at which time the 
organism orients itself towards the light and approaches 
it. However, very bright lights, material obstacles, and 
steep gradients are repellent to the tortoise. The latter 
stimuli convert the photoamplifier into an oscillator, 
which causes alternating movements of butting and 
withdrawal, so that the robot pushes small objects out 
of its way, goes around heavy ones, and avoids slopes. 
The tortoise has a hutch, which contains a bright 
light. When the machine’s batteries are charged, this 
bright light is repellent. When the batteries are low, the 
light becomes attractive to the machine and the light 
continues to exert an attraction until the tortoise enters 
the hutch, where the machine’s circuitry is temporarily 
turned off until the batteries are recharged, at which 
time the bright hutch light again exerts a negative 
tropism. The second robot, M. docilis was produced 
by grafting onto M. speculatrix a circuit designed to 


form conditioned reflexes. In one experiment, Walter 
connected this circuit to the obstacle-avoiding device in 
M. speculatrix. Training consisted of blowing a whistle 
just before bumping the shell. 

Although Walter’s controllers are simple and not 
based on neural analysis, they do illustrate an attempt 
to gain inspiration from seeking the simplest mecha¬ 
nisms that will yield an interesting class of biologically 
inspired robot behaviors, and then showing how differ¬ 
ent additional mechanisms yield a variety of enriched 
behaviors. Braitenberg' s book [77.2] is very much in 
this spirit and has entered the canon of neurorobotics. 
While their work provides a historical background for 
the studies surveyed here, we instead emphasize stud¬ 
ies inspired by the computational neuroscience of the 
mechanisms serving vision and action in the human 
and in animal brains. We seek lessons from linking 
behavior to the analysis of the internal workings of 
the brain (1) at the relatively high level of charac¬ 
terizing the functional roles of specific brain regions 
(or the functional units of analysis called schemas, 
Sect. 77.2.4), and the behaviors which emerge from 
the interactions between them, and (2) at the more 
detailed level of models of neural circuitry linked to 
the data of neuroanatomy and neurophysiology. There 
are lessons for neurorobotics to be learned from even 
finer-scale analysis of the biophysics of individual neu¬ 
rons and the neurochemistry of synaptic plasticity, 
but these are beyond the scope of this chapter (see 
Segev and London [77.3] and Fregnac [77.4], respec¬ 
tively, for entry points into the relevant computational 
neuroscience). 

The plan of this chapter is as follows. We will 
start with explaining how the higher-level cognitive 
functionality of vision-based planning and navigation 
is realized in biology, and how this relates to robotic 
systems (Sect. 77.2). We then (Sect. 77.3) explain verte¬ 
brate movement generation itself, and put forth a theory 
on what role the cerebellum plays in tuning and coor¬ 
dinating actions. This is followed by a section on the 
mirror system and its roles in action recognition and 
imitation (Sect. 77.4). The extroduction will then in¬ 
vite readers to explore the many other areas in which 
neurorobotics offers lessons from neuroscience to the 
development of novel robot designs. What follows, 
then, can be seen as a contribution to the continuing 
dialog between robot behavior and animal and human 
behavior in which particular emphasis is placed on the 
search for the neural underpinnings of vision, visually 
guided action, and cerebellar control. 
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77.2 The Case for Vision 

Before we turn to vertebrate brains for much of our 
inspiration for neurorobotics, we briefly sample the 
rich literature on insect-inspired research. Among the 
founding studies in computational neuroethology were 
a series of reports from the laboratory of Werner Re- 
ichardt in Tubingen, which linked the delicate anatomy 
of the fly’s brain to the extraction of visual data 
needed for flight control. More than 40 years ago, Re- 
ichardt [77.5] published a model of motion detection 
inspired by this work that has long been central to dis¬ 
cussions of visual motion in both the neuroscience and 
robotics literatures. Borst and Dickinson [77.6] provide 
a recent study of continuing biological research on vi¬ 
sual course control in flies. Such work has inspired 
a large number of robot studies, including those of van 
der Smagt and Groen [77.7], van der Smagt [77.8], Liu 
and Usseglio-Viretta [77.9], Ruffier et al. [77.10], and 
Reiser and Dickinson [77.11]. 

77.2.1 Optic Flow in Bees and Robots 

Here, however, we look in a little more detail at honey¬ 
bees. Srinivasan et al. [77.15] continued the tradition 
of studying image motion cues in insects by investi¬ 
gating how optic flow (the flow of pattern across the 
eye induced by motion relative to the environment) is 
exploited by honeybees to guide locomotion and nav¬ 
igation. They analyzed how bees perform a smooth 



Fig. 77.1 (a) Observation of the trajectories of honeybees 
flying in visually textured tunnels has provided insights 
into how bees use optic flow cues to regulate flight speed 
and estimate distance flown, and balance optic flow in 
the two eyes to fly safely through narrow gaps (images 
courtesy of Srinivasan et al. [77.12]). This information 
has been used to build autonomously navigating robots. 

(b) Schematic illustration of a honeybee brain, carrying 
about a million neurons within « 1mm 3 (after [77.13]). 

(c) A mobile robot guided by an optic flow algorithm based 
on the studies exemplified in [77.14] 


landing on a flat surface: image velocity is held con¬ 
stant as the surface is approached, thus automatically 
ensuring that flight speed is close to zero at touchdown. 
This obviates any need for explicit knowledge of flight 
speed or height above the ground. This landing strat¬ 
egy was then implemented in a robotic gantry to test 
its applicability to autonomous airborne vehicles. Bar¬ 
ron and Srinivasan [77.14] investigated the extent to 
which ground speed is affected by headwinds. Honey¬ 
bees were trained to enter a tunnel to forage at a sucrose 
feeder placed at its far end (Fig. 77. la). The bees used 
visual cues to maintain their ground speed by adjusting 
their airspeed to maintain a constant rate of optic flow, 
even against headwinds which were, at their strongest, 
50% of a bee’s maximum recorded forward velocity. 

Vladusich et al. [77.16] studied the effect of adding 
goal-defining landmarks. Bees were trained to forage 
in an optic-flow-rich tunnel with a landmark posi¬ 
tioned directly above the feeder. They searched much 
more accurately when both odometric and landmark 
cues were available than when only odometry was 
available. When the two cue sources were set in con¬ 
flict, by shifting the position of the landmark in the 
tunnel during tests, bees overwhelmingly used land¬ 
mark cues rather than odometry. This, together with 
other such experiments, suggests that bees can make 
use of odometric and landmark cues in a more flex¬ 
ible and dynamic way than previously envisaged. In 
earlier studies of bees flying down a tunnel, Srini¬ 
vasan and Zhang [77.17] placed different patterns on 
the left and right walls. They found that bees bal¬ 
ance the image velocities in the left and right visual 
fields. This strategy ensures that bees fly down the 
middle of the tunnel, without bumping into the side 
walls, enabling them to negotiate narrow passages or to 
fly between obstacles. This strategy has been applied 
to a corridor-following robot (Fig. 77.1c). By hold¬ 
ing constant the average image velocity as seen by the 
two eyes during flight, the bee avoids potential col¬ 
lisions, slowing down when it flies through a narrow 
passage. The movement-sensitive mechanisms underly¬ 
ing these various behaviors differ qualitatively as well 
as quantitatively, from those that mediate the optomo- 
tor response (e.g., turning to track a pattern of moving 
stripes) that had been the initial target of investigation 
of the Reichardt laboratory. The lesson for robot con¬ 
trol is that flight appears to be coordinated by a number 
of visuomotor systems acting in concert, and the same 
lesson can apply to a whole range of tasks that must 
convert vision to action. Of course, vision is but one 
of the sensory systems that play a vital role in insect 
behavior. Webb [77.18] uses her own work on robot 
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design inspired by the auditory control of behavior in 
crickets to anchor a far-ranging assessment of the ex¬ 
tent to which robotics can offer good models of animal 
behaviors. 

77.2.2 Visually Guided Behavior 
in Frogs and Robots 

Lettvin et al. [77.19] treated the frog’s visual system 
from an ethological perspective, analyzing circuitry in 
relation to the animal’s ecological niche to show that 
different cells in the retina and the visual midbrain 
region known as the tectum were specialized for de¬ 
tecting predators and prey. However, in much visually 
guided behavior, the animal does not respond to a sin¬ 
gle stimulus, but rather to some property of the overall 
configuration. We thus turn to the question what does 
the frog’s eye tell the frog?, stressing the embodied ner¬ 
vous system or, perhaps equivalently, an action-oriented 
view of perception. Consider, for example, the snap¬ 
ping behavior of frogs confronted with one or more 
fly-like stimuli. Ingle [77.20] found that it is only in 
a restricted region around the head of a frog that the 
presence of a fly-like stimulus elicits a snap, that is, the 
frog turns so that its midline is pointed at the stimu¬ 
lus and then lunges forward and captures the prey with 
its tongue. There is a larger zone in which the frog 
merely orients towards the target, and beyond that zone 
the stimulus elicits no response at all. When confronted 
with two flies within the snapping zone, either of which 
is vigorous enough that it could elicit a snapping re¬ 
sponse alone, the frog exhibits one of three reactions: 
it snaps at one of the flies, it does not snap at all, or 
it snaps in between at the average fly. Didday [77.21] 
offered a simple model of this choice behavior which 
may be considered as the prototype for a winner-take- 
all (WTA) model, which receives a variety of inputs and 
(under ideal circumstances) suppresses the representa¬ 
tion of all but one of them; the one that remains is the 
winner that will play the decisive role in further pro¬ 
cessing. This was the beginning of Rana computatrix 
(see Arbib [77.22, 23] for overviews). 

Studies on frog brains and behavior inspired the 
successful use of potential fields for robot navi¬ 
gation strategies. Data on the strategies used by 
frogs to capture prey while avoiding static obstacles 
(Collett [77.24]) grounded the model by Arbib and 
House [77.25], which linked systems for depth percep¬ 
tion to the creation of spatial maps of both prey and 
barriers. In one version of their model, they represented 
the map of prey by a potential field with long-range at¬ 
traction and the map of barriers by a potential field with 
short-range repulsion, and showed that summation of 
these fields yielded a field that could guide the frog’s de¬ 


tour around the barrier to catch its prey. Corbacho and 
Arbib [77.26] later explored a possible role for learn¬ 
ing in this behavior. Their model incorporated learning 
in the weights between the various potential fields to 
enable adaptation over trials as observed in the real an¬ 
imals. The success of the models indicated that frogs 
use reactive strategies to avoid obstacles while moving 
to a goal, rather than employing a planning or cog¬ 
nitive system. Other work, Cobas and Arbib [77.27], 
studied how the frog’s ability to catch prey and avoid 
obstacles was integrated with its ability to escape from 
predators. These models stressed the interaction of the 
tectum with a variety of other brain regions such as the 
pretectum (for detecting predators) and the tegmentum 
(for implementing motor commands for approach or 
avoidance). 

Arkin [77.28] showed how to combine a com¬ 
puter vision system with a frog-inspired potential field 
controller to create a control system for a mobile 
robot that could successfully navigate in a fairly struc¬ 
tured environment using camera input. The resultant 
system thus enriched other roughly contemporaneous 
applications of potential fields in path planning with 
obstacle avoidance for both manipulators and mobile 
robots (Khatib [77.29], Krogh and Thorpe [77.30]). 
The work on Rana computatrix proceeded at two lev¬ 
els - both biologically realistic neural networks and 
in terms of functional units called schemas, which 
compete and cooperate to determine behavior. Sec¬ 
tion 77.2.4 will show how more general behaviors can 
emerge from the competition and cooperation of per¬ 
ceptual and motor schemas, as well as more abstract 
coordinating schemas. Such ideas were, of course, de¬ 
veloped independently by a number of authors, and 
so entered the robotics literature by various routes, of 
which the best known may be the subsumption archi¬ 
tecture of Brooks [77.31] and the ideas of Braitenberg 
cited above, whereas Arkin’s work on behavior-based 
robotics [77.32] is, indeed, rooted in schema theory. 
Arkin et al. [77.33] present a recent example of the con¬ 
tinuing interaction between robotics and ethology, of¬ 
fering a novel method for creating high-fidelity models 
of animal behavior for use in robotic systems based on 
a behavioral systems approach (i. e., based on a schema- 
level model of animal behavior, rather than analysis of 
biological circuits in animal brains), and describe how 
an ethological model of a domestic dog can be imple¬ 
mented with AIBO, the Sony entertainment robot. 

77.2.3 Navigation in Rat and Robot 

The tectum, the midbrain visual system which deter¬ 
mines how the frog turns its whole body towards it prey 
or orients it for escape from predators (Sect.77.2.2), is 
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homologous with the superior colliculus of the mam¬ 
malian midbrain. The rat superior colliculus has been 
shown to be frog like, mediating approach and avoid¬ 
ance ( Dean et al. [77.34]), whereas the best-studied role 
of the superior colliculus of cat, monkey, and human is 
in the control of saccades, rapid eye movements to ac¬ 
quire a visual target. Moreover, the superior colliculus 
can integrate auditory and somatosensory information 
into its visual frame ( Stein and Meredith [77.35]), and 
this inspired Strosslin et al. [77.36] to use a biolog¬ 
ically inspired approach based on the properties of 
neurons in the superior colliculus to learn the rela¬ 
tion between visual and tactile information in control 
of a mobile robot platform. More generally, then, the 
comparative study of mammalian brains has yielded 
a rich variety of computational models of importance in 
neurorobotics. In this section, we further introduce the 
study of mammalian neurorobotics by looking at stud¬ 
ies of mechanisms of the rat brain for spatial navigation. 

The frog’s detour behavior is an example of what 
O’Keefe and Nadel [77.37] called the taxon (behavioral 
orientation) system (as in Braitenberg, [77.38] a taxis 
(plural taxes ) is an organism’s response to a stimulus by 
movement in a particular direction). They distinguished 
this from a system for map-based navigation and pro¬ 
posed that the latter resides in the hippocampus, though 
Guazzelli et al. [77.39] qualified this assertion, showing 
how the hippocampus may function as part of a cogni¬ 
tive map. The taxon versus map distinction is akin to the 
distinction between reactive and deliberative control in 
robotics ( Arkin et al. [77.33]). It will be useful to relate 
taxis to the notion of an ajfordance (Gibson [77.40]), 
a feature of an object or environment relevant to action, 
for example, in picking up an apple or a ball, the iden¬ 
tity of the object may be irrelevant, but the size of the 
object is crucial. Similarly, if we wish to push a toy car, 
recognizing the make of car copied in the toy is irrele¬ 
vant, whereas it is crucial to recognize the placement of 
the wheels to extract the direction in which the car can 
be readily pushed. Just as a rat may have basic taxes for 
approaching food or avoiding a bright light, say, so does 
it have a wider repertoire of affordances for possible 
actions associated with the immediate sensing of its en¬ 
vironment. Such affordances include go straight ahead 
for visual sighting of a corridor, hide for a dark hole, 
eat for food as sensed generically, drink similarly, and 
the various turns afforded by, e.g., the sight of the end 
of the corridor. It also makes rich use of olfactory cues. 
In the same way, a robot’s behavior will rely on a host 
of reactions to local conditions in fulfilling a plan, e.g., 
knowing that it must go to the end of a corridor it will 
nonetheless use local visual cues to avoid hitting obsta¬ 
cles or to determine through which angle to turn when 
reaching a bend in the corridor. 


Both normal and hippocampal-lesioned rats can 
learn to solve a simple T-maze (e.g., learning whether 
to turn left or right to find food) in the absence of 
any consistent environmental cues other than the T- 
shape of the maze. If anything, the lesioned animals 
learn this problem faster than normal ones. After the 
criterion was reached, probe trials with an eight-arm 
radial maze were interspersed with the usual T-trials. 
Animals from both groups consistently chose the side 
to which they were trained on the T-maze. However, 
many did not choose the 90° arm but preferred either 
the 45° or 135° arm, suggesting that the rats eventually 
solved the T-maze by learning to rotate within an ego¬ 
centric orientation system at the choice point through 
approximately 90°. This leads to the hypothesis of an 
orientation vector being stored in the animal’s brain 
but does not tell us where or how the orientation vec¬ 
tor is stored. One possible model would employ coarse 
coding in a linear array of cells, coding for turns from 
—180° to +180°. From the behavior, one might expect 
that only the cells close to the preferred behavioral di¬ 
rection are excited, and that learning marches this peak 
from the old to the new preferred direction. To unlearn 
—90°, say, the array must reduce the peak there, while 
at the same time building a new peak at the new di¬ 
rection of +90°. If the old peak has massp(t) and the 
new peak has massq(t), then as p(t) declines toward 0 
while q(t ) increases steadily from 0, the center of mass 
will progress from —90° to +90°, fitting the behavioral 
data. 

The determination of movement direction was mod¬ 
eled by rat-ification of the Arbib and House [77.25] 
model of frog detour behavior. There, prey was repre¬ 
sented by excitation coarsely coded across a population, 
while barriers were encoded by inhibition whose extent 
closely matched the retinotopic extent of each barrier. 
The sum of excitation was passed through a winner- 
takes-all circuit to yield the choice of movement direc¬ 
tion. As a result, the direction of the gap closest to the 
prey, rather than the direction of the prey itself, was of¬ 
ten chosen for the frog’s initial movement. The same 
model serves for behavioral orientation once we replace 
the direction of the prey (frog) by the direction of the 
orientation vector (rat), while the barriers correspond to 
the presence of walls rather than alley ways. 

To approach the issue of how a cognitive map 
can extend the capability of the affordance system, 
Guazzelli et al. [77.39] extended the Lieblich and Ar¬ 
bib [77.41] approach to building a cognitive map as 
a world graph, a set of nodes connected by a set of 
edges, where the nodes represent recognized places or 
situations, and the links represent ways of moving from 
one situation to another. A crucial notion is that a place 
encountered in different circumstances may be repre- 
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Fig. 77.2 The TAM-WG model has at its basis a system, TAM (the taxon affordance model), for exploiting affordances. 
This is elaborated by a system, WG (the world graph), which can use a cognitive map to plan paths to targets which 
are not currently visible. Note that the model processes two different kinds of sensory inputs. At the bottom right are 
those associated with, e.g., hypothalamic systems for feeding and drinking and that may provide both incentives and 
rewards for the animal's behavior, contributing both to behavioral choices, and to the reinforcement of certain patterns 
of behavior. The nucleus accumbens and caudoputamen mediate an actor-critic style of reinforcement learning based 
on the hypothalamic drive of the dopamine system. The sensory inputs at the top left are those that allow the animal to 
sense its relation with the external world, determining both where it is (the hippocampal place system), as well as the 
affordances for action (the parietal recognition of affordances can shape the premotor selection of an action). The TAM 
model focuses on the parietal-premotor reaction to immediate affordances; the WG model places action selection within 
the wider context of a cognitive map (after Guazzelli et al. [77.39]) 


sented by multiple nodes, but that these nodes may 
be merged when the similarity between these circum¬ 
stances is recognized. They model the process whereby 
the animal decides where to move next, on the ba¬ 
sis of its current drive state (hunger, thirst, fear, etc.). 
The emphasis is on spatial maps for guiding locomo¬ 
tion into regions not necessarily currently visible, rather 
than retinotopic representations of immediately visible 
space, and yields exploration and latent learning with¬ 
out the introduction of an explicit exploratory drive. 
The model shows: 

1. How a route, possibly of many steps, may be chosen 
that leads to the desired goal. 

2. How short cuts may be chosen. 

3. Through its account of node merging why, in open 
fields, place cell firing does not seem to depend on 
direction. 

The overall structure and general mode of opera¬ 
tion of the complete model is shown in Fig. 77.2, which 
gives a vivid sense of the lessons to be learned by study¬ 
ing not only specific systems of the mammalian brain 


but also their patterns of large-scale interaction. This 
model is but one of many inspired by the data on the 
role of the hippocampus and other regions in rat nav¬ 
igation. Here, we just mention as pointers the wider 
literature the papers by Girard et al. [77.42] and Meyer 
et al. [77.43], which are part of the Psikharpax project, 
which does for rats what Rana computatrix did for frogs 
and toads. 

77.2.4 Salience and Visual Attention 

Discussions of how an animal (or robot) grasps an ob¬ 
ject assume that the animal or robot is attending to the 
relevant object. Thus, whatever the subtlety of process¬ 
ing in the canonical and mirror systems for grasping, 
its success rests on the availability of a visual system 
coupled to an oculomotor control system that bring 
foveal vision to bear on objects to set the parameters 
needed for successful interaction. Indeed, the general 
point is that attention greatly reduces the processing 
load for animal and robot. The catch, of course, is that 
reducing the computing load is a Pyrrhic victory unless 
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the moving focus of attention captures those aspects 
of behavior that are relevant for the current task - or 
supports necessary priority interrupts. Indeed, direct¬ 
ing attention appropriately is a topic for which there 
is a great richness of both neurophysiological data and 
robotic application (see Deco and Rolls [77.44] and 
Choi et al. [77.45]). 

In their neuromorphic model of the bottom-up guid¬ 
ance of attention in primates, Itti and Koch [77.46] de¬ 
compose the input video stream into eight feature chan¬ 
nels at six spatial scales. After surround suppression, 
only a sparse number of locations remain active in each 
map, and all maps are combined into a unique saliency 
map. This map is scanned by the focus of attention in 
order of decreasing saliency through the interaction be¬ 
tween a winner-takes-all mechanism (which selects the 
most salient location) and an inhibition-of-return mech¬ 
anism (which transiently suppresses recently attended 
locations from the saliency map). Because it includes 
a detailed low-level vision front-end, the model has 
been applied not only to laboratory stimuli, but also to 
a wide variety of natural scenes, predicting a wealth of 
data from psychophysical experiments. 

When specific objects are searched for, low-level 
visual processing can be biased both by the gist (e.g., 
outdoor suburban scene) and also for the features of 
that object. This top-down modulation of bottom-up 
processing results in an ability to guide search towards 
targets of interest ( Wolfe [77.47]). Task affects eye 
movements ( Yarbus [77.48]), as do training and gen¬ 
eral expertise. Navalpakkam and Itti [77.49] propose 
a computational model which emphasizes four aspects 
that are important in biological vision: determining the 
task relevance of an entity, biasing attention for the 
low-level visual features of desired targets, recogniz¬ 
ing these targets using the same low-level features, and 
incrementally building a visual map of task relevance 

77.3 Vertebrate Motor Control 

The body of literature on primate motor control is, of 
course, vast, and gives a patchy view on the principles 
behind it. Getting a clear view of how limb and general 
body control functions is difficult; moreover, there are 
no clear proofs of whether any of the existing views on 
motor control are correct. 

But there exist a few observations of the human cen¬ 
tral and peripheral nervous systems from which clear 
conclusions can be drawn. The first observation is the 
presence of neural communication delays. How does 
the system know the position of limbs? There are two 
principled methods: (1) through proprioceptive signals, 
consisting of muscle spindles and Golgi tendon or- 


at every scene location. It attends to the most salient 
location in the scene, and attempts to recognize the 
attended object through hierarchical matching against 
object representations stored in long-term memory. It 
updates its working memory with the task relevance of 
the recognized entity and updates a topographic task- 
relevance map with the location and relevance of the 
recognized entity; for example, in one task the model 
forms a map of likely locations of cars from a video 
clip filmed while driving on a highway. Such work illus¬ 
trates the continuing interaction between models based 
on visual neurophysiology and human psychophysics 
with the tackling of practical robotic applications. 

Orabona et al. [77.50] implemented an extension 
of the Itti-Koch model on a humanoid robot with 
moving eyes, using log-polar vision as in Sandini 
and Tagliasco [77.51], and changing the feature con¬ 
struction pyramid by considering proto-object elements 
(blob-like structures rather than edges). The inhibition- 
of-return mechanism has to take into account a moving 
frame of reference, the resolution of the fovea is very 
different from that at the periphery of the visual field, 
and head and body movements need to be stabilized. 
The control of movement might thus have a relation¬ 
ship with the structure and development of the attention 
system. Rizzolatti et al. [77.52] proposed a role for 
the feedback projections from premotor cortex to the 
parietal lobe, assuming that they form a tuning signal 
that dynamically changes visual perception. In prac¬ 
tice, this can be seen as an implicit attention system 
that selects sensory information while the action is 
being prepared and subsequently executed ( Flanagan 
and Johansson [77.53], Flanagan et al. [77.54], and 
Mataric and Pomplun [77.55]). The early responses, 
before action onset, of many premotor and parietal neu¬ 
rons suggest a premotor mechanism of attention that 
deserves exploration in further work in neurorobotics. 


gans (GOs); and (2) through skin information. It is, 
however, not very likely that information from mus¬ 
cle spindles and GOs are accurate enough to code limb 
position. Tendon organs are sensitive to forces along 
in-series motor units and there is no physiological ev¬ 
idence that Golgi tendon organs signal muscle length 
(but, of course, force changes with muscle length, so 
during movement a correlation is found). There is an¬ 
other problem with respect to limb position, which is 
particularly clear for fingers: flexibilities and nonlinear 
relationships between finger position and muscle force, 
in combination with the imprecise receptors, makes the 
relationship between GO/spindle data and finger posi- 
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Table 77.1 Classification of sensory fibers from muscle (after [77.56]) 


Type 

Receptor 

Axon diameter [p,m] 

Transmission speed [m/s] 

Sensitive to 

la 

Primary spindle endings 

12-20 

60-100 

Muscle length and rate of change 

lb 

Golgi tendon organs 

12-20 

60-100 

Muscle tension 

n 

Secondary spindle endings 

6-12 

30-70 

Muscle length 

m 

Free nerve endings 

2-6 

10-30 

Skin signals 


tion too complex and variable to be a likely candidate 
to code finger position; after all, the sensors are in the 
forearm rather than in the fingers; and information on 
finger position is not available in muscle movement 
or tendon force. Furthermore, muscle spindle data is 
noisy [77.57]. It has conversely been shown [77.58, 
59] that the receptors in hairy skin code information 
that can be related to finger position; furthermore, sim¬ 
ilar data have been found for the knee joint [77.60]. 
Table 77.1 lists nerve transmission speeds for these 
signals. 

Since neurons are only to be found in the spinal 
cord and the brain, for hand skin, therefore, we can 
expect signal transfer delays to the spine of around 
30—100 ms. Round-trip muscle activation is, therefore, 
around 70ms for signals based on skin data [77.61], 
or around 25 ms for spindle-based signals (we have 
verified these delays by measuring hand skin-based 
reflexes by measuring the corresponding electromyog- 
raphy(EMG) signals, and found a round-trip delay of 
around 75 ms. Spindle-based feedback for the wrist was 
measured at around 25 ms. 

Of course, when a sensory signal has to be pro¬ 
cessed in the brain, the delays are correspondingly 
longer. At any rate: error-correcting feedback control 
has delays of several tens of milliseconds; feedback 
control based on such delays cannot lead to any accept¬ 
able accuracy with the movement speeds that humans 
typically display. This means that large portions of our 
movement, over time frames in the order of 100 ms or 
more, need to be controlled open loop. 

A second important observation is our generalizing 
capabilities. Consider the case of playing fast and ac¬ 
curate sports, e.g., table tennis. During play, we obtain 
sensory visual, haptic, and tactile sensory data, the re¬ 
sult of which must lead to an accurate movement of the 
bat in order to score a point. Even a player with little 
training is able to do this rather accurately: at ball flight 
times between 200 and 500 ms, the brain does not have 
much time to plan an accurate whole-body movement 
for each and every possible sensory state, but we are 
usually capable of returning the ball. Training helps, but 
we do not need to exhaustively learn many states in the 
very high-dimensional sensor space in which our obser¬ 
vations move. 

Generalization can only be done with reasonably 
accurate models of the sensor/motor behavior. How¬ 


ever, models of our motor system are difficult to obtain: 
variations such as including payloads, wearing heavy 
clothing, muscle fatigue, etc., do not influence our ac¬ 
curacy considerably. 

77.3.1 The Flat Hierarchy of Neurocontrol 

How is an open-loop movement generated? In this 
paper we concentrate on voluntary vertebrate motor 
control; the only reason for any animal to have a brain 
is to generate movement. Moreover, despite differences 
in brain structures, there is a large correspondence in 
movement patterns among the whole animal kingdom, 
irrespective of the presence of a cortical structure or 
a cerebellum. What parts of the brain are directly in¬ 
volved in movement? 

The major role of the cerebral cortex seems to 
be unsupervised learning to establish relationships be¬ 
tween sensory and action patterns [77.62]. The neocor¬ 
tex is only to be found in mammals; experiments with 
decorticated cats [77.63] clearly show that the cortex is 
not necessary to generate movement; rather, it is likely 
that the motor cortex models and weighs movements, to 
subsequently make decisions based thereon. 

The major role of the basal ganglia seems to be 
reinforcement learning to filter out unwanted move¬ 
ments [77.62]. They play a dominant role in movement 
generation or gating (filtering) of generated movement 
patterns. The effect of Parkinson’s disease (the in¬ 
ability to initiate movement) and Huntington’s disease 
(the inability to prevent unwanted movement) on the 
basal ganglia is well known and clearly indicates their 
function. 

The major role of the cerebellum seems to be su¬ 
pervised learning of motor patterns [77.62]. Moreover, 
decerebellation does not lead to complete movement 
loss. An individual with cerebellar lesions may be able 
to move the arm to successfully reach a target and to 
successfully adjust the hand to the size of an object. 
However, the action cannot be made swiftly and accu¬ 
rately, and the ability to coordinate the timing of the two 
subactions is lacking. The behavior will thus exhibit 
decomposition of movement - first the hand is moved 
until the thumb touches the object, and only then is the 
hand shaped appropriately to grasp the object [77.64]. 

Robot control usually favors a strict hierarchical 
approach. A typical robot works as follows. At the low- 
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est level, a very fast 100 (is) current control loop 
controls the rotation of the dc motor. On top of that, 
a torque controller (running typically at 1 kHz) controls 
the torque of all joints, and is in its turn controlled by 
an impedance or position controller. On top of that, typ¬ 
ically, a Cartesian path planner forms the slowest loop. 
An error in any of these elements will disable the robot. 

A look into evolutionary development of neural 
control thus makes immediately clear that a strict hi¬ 
erarchical approach is not viable in neural control. 
Although any of the above-mentioned brain regions is 
important in movement control, and similar structures 
can be found in any vertebrate, their dysfunction leads 
to movement degradation but not to movement loss (this 
is, of course, not true for the spinal cord, which (com¬ 
bines and) transmits the controls to the muscles). Also, 
the development of the neural system shows that an¬ 
imals were always capable of movement - irrespective 
of their brain structure. However, the cerebellum is usu¬ 
ally rightly focused upon when analyzing vertebrate 
movement. How do the parts of the brain collaborate 
towards smooth goal-directed movement? 

In placing the function of the cerebellum in the loop, 
a normal distinction is to consider the cerebellum as 
representing (a) a forward or direct model which rep¬ 
resents the path from motor command to motor output, 
or (b) an inverse model of motor function, i. e., going 
from a desired motor outcome to a set of motor com¬ 
mands likely to achieve it. As we have just suggested, 
the action plan unfolds as if it were feedforward or 
open loop when the actual parameters of the situation 
match the stored parameters, while a feedback com¬ 
ponent is employed to counteract disturbances (current 
feedback) and to learn from mistakes (learning from 
feedback). This is obtained by relying on a forward 
model that predicts the outcome of the action as it un¬ 
folds in real time. The accuracy of the forward model 
can be evaluated by comparing the output generated 
by the system with the signals derived from sensory 
feedback (Miall et al. [77.65]). Also, delays must be ac¬ 
counted for to address the different propagation times of 
the neural pathways carrying the predicted and actual 
outcome of the action. Note that the forward model in 
this case is relatively simple, predicting only the motor 
output in advance; since motor commands are generated 
internally it is easy to imagine a predictor for these sig¬ 
nals (known as an efference copy). The inverse model, 
on the other hand, is much more complicated since it 
maps sensory feedback (e.g., vision) back into motor 
terms. 

We suggest a much simpler approach to the ver¬ 
tebrate control system. However, let us first look into 
the functionality of the lower-level apparatus: muscle, 
spinal cord, and cerebellum. 


77.3.2 On Spinal Cord and Muscle 

The key element in movement generation is given by 
two building blocks: (a) our muscles, and (b) the spinal 
cord. Muscle behavior is strongly nonlinear; the exerted 
force decreases nonlinearly with velocity (Fig. 77.3) 
and varies nonlinearly with length (Fig. 77.4). 

Limb movement, however, is caused by a complex 
of muscles - for instance, the human arm uses a total 
of 19 muscle groups for planar motion of the elbow 
and shoulder alone (Nijhof and Kouwenhoven [77.67]) 
with altogether highly nonlinear dynamics. How can 
this large number of actuators be controlled without 
feedback error control? 

The concept is simple and was first well described 
by Bernstein [77.68]: skeletal muscles are always con¬ 
trolled in functional groups, leading to synergies of 
movement. Rather than activating muscles indepen- 



Fig. 77.3 The force/velocity and power/velocity relation¬ 
ship of muscle (after [77.56]) 


Force 



Fig. 77.4 The force/length relationship of muscle (af¬ 
ter [77.56]) 


2077 


Part G | 77.3 









Part G | 77.3 


2078 Part G 


Robots and Humans 


Cortico-nuclear 
microzone (cerebellar cortex) 






Fig. 77.5 (a) Major cells in the cerebellum, (b) Cells in the Marr-Albus model. The granule cells are state encoders, 
feeding system state, and sensor data into the PC. PC/PF synapses are adjusted using the Widrow-Hoff rule. The output 
of the PC are steering signals for the robotic system, (c) The APG model, using the same state encoder as in (b). 
(d) The MPFIM model. A single module corresponds to a group of Purkinje cells: predictor, controller, and responsibility 
estimator. The granule cells generate the necessary basis functions of the original information (after [77.66]) 


dently, a neural signal controls groups of muscles that 
perform (a part of) an action. Linear dimension reduc¬ 
tion methods [77.69] (e.g., principal component anal¬ 
ysis, (PCA), independent component analysis (ICA), 
or non-negative matrix factorization (NMF)) have been 
used to establish synergies in EMG data, and this can 
be used [77.70] to linearly combine single-finger move¬ 
ment to whole-hand movement in EMG space. So, we 
cannot control single muscles (i. e., coherent groups of 
muscle fibers) but rather control muscle groups, the lin¬ 
ear combination of which can be used to span a decent 
part of our voluntary movement. 

There are currently still open questions as to the na¬ 
ture of movement synergies: how much of the synergies 
are defined by the biomechanical structure of our mus¬ 
cles and tendons; how much of it is laid out in the spinal 
cord; and which part of it is learned in the higher move¬ 
ment control regions? 

77.3.3 Models of Cerebellar Control 

The cerebellum can be divided into two parts: the cor¬ 
tex and the deep nuclei. There are two systems of fibers 
bringing input to the both the cortex and nuclei: the 
mossy fibers and the climbing fibers. The only output 


from the cerebellar nucleus comes from cells called 
Purkinje cells, and they project only to the cerebellar 
nuclei, where their effect is inhibitory. This inhibition 
sculpts the output of the nuclei which (the effect varies 
from nucleus to nucleus) may act by modulating ac¬ 
tivity in the spinal cord, the mid-brain, or the cerebral 
cortex. We now turn to models that make explicit use of 
the cellular structure of the cerebellar cortex (see Ec- 
cles et al. [77.71] and Ito [77.72], and also Fig. 77.5a). 
The human cerebellum has 7—14 million Purkinje cells 
(PCs), each receiving about 200000 synapses. Mossy 
fibers (MFs) arise from the spinal cord and brainstem. 
They synapse onto granule cells and deep cerebellar nu¬ 
clei. Granule cells have axons which each project up 
to form a T, with the bars of the T forming the paral¬ 
lel fibers (PFs). Each PF synapses on about 200 PCs. 
The PCs, which are grouped into microzones, inhibit 
the deep nuclei. PCs with their target cells in cerebellar 
nuclei are grouped together in microcomplexes [77.72]. 
Microcomplexes are defined by a variety of criteria to 
serve as the units of analysis of cerebellar influence 
on specific types of motor activity. The climbing fibers 
(CFs) arise from the inferior olive (IO). Each PC re¬ 
ceives synapses from only one CF, but a CF makes 
about 300 excitatory synapses on each PC that it con- 
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tacts. This powerful input alone is enough to fire the PC, 
though most PC firing depends on subtle patterns of PF 
activity. The cerebellar cortex also contains a variety of 
inhibitory interneurons. The basket cell is activated by 
PF afferents and makes inhibitory synapses onto PCs. 
Golgi cells receive input from PFs, MFs, and CFs, and 
inhibit granule cells. 

The Marr-Albus Model 

In the Marr-Albus model (Marr [77.73] and Al¬ 
bus [77.74]) the cerebellum functions as a classifier of 
sensory and motor patterns received through the MFs. 
Only a small fraction of the parallel fibers (PF) are ac¬ 
tive when a Purkinje cell (PC) fires and thus influence 
the motor neurons. Both Marr and Albus hypothesized 
that the error signals for improving PC firing in re¬ 
sponse to PF, and thus MF input were provided by the 
climbing fibers (CF), since only one CF affects a given 
PC. However, Marr hypothesized that CF activity would 
strengthen the active PF/PC synapses using a Widrow- 
Hoff learning rule, whereas Albus hypothesized they 
would weaken them. This is an important example of 
a case where computational modeling inspired impor¬ 
tant experimentation. Eventually, Masao Ito was able 
to demonstrate that Albus was correct - the weakening 
of active synapses is now known to involve a pro¬ 
cess called long-term depression [77.72]. However, the 
rule with weakening of synapses is still known as the 
Marr-Albus model, and remains the reference model 
for studies of synaptic plasticity of the cerebellar cortex. 
However, both Marr and Albus viewed each PC as func¬ 
tioning as a perceptron whose job it was to control an 
elemental movement, contrasting with more plausible 
models in which PCs serve to modulate the involvement 
of microcomplexes (which include cells of the deep nu¬ 
clei) in motor pattern generators (e.g., the APG model 
described below). 

Since the development of the Marr-Albus model 
several cerebellar models have been introduced in 
which cerebellar plasticity plays a key role. Limiting 
our overview to computational models, we will de¬ 
scribe: 

1. The CMAC (cerebellar model articulation con¬ 
troller). 

2. The adjustable pattern generator (APG). 

3. The Schweighofer-Arbib model. 

4. The multiple paired forward-inverse models [77.75, 

76], 

The Cerebellar Model Articulation Controller 
One of the first well-known computational mod¬ 
els of the cerebellum is the CMAC (Albus [77.77]; 
Fig. 77.5b). The algorithm was based on Albus’ un¬ 


derstanding of the cerebellum, but it was not proposed 
as a biologically plausible model. The idea has its ori¬ 
gins in the BOXES approach, in which for n variables 
an /7-dimensional hypercube stores function values in 
a lookup table. BOXES suffers from the curse of di¬ 
mensionality: if each variable can be discretized into 
D different steps, the hypercube has to store D" func¬ 
tion values in memory. Albus assumed that the mossy 
fibers provided discretized function values. If the signal 
on a mossy fiber is in the receptive field of a par¬ 
ticular granule cell, it fires onto a parallel fiber. This 
mapping of inputs onto binary output variables is of¬ 
ten considered to be the generalization mechanism in 
CMAC. The learning signals are provided by the climb¬ 
ing fibers. 

Albus’ CMAC can be described in terms of a large 
set of overlapping, multidimensional receptive fields 
with finite boundaries. Every input vector falls within 
the range of some local receptive fields. The response 
of CMAC to a given input is determined by the aver¬ 
age of the responses of the receptive fields excited by 
that input. Similarly, the training for a given input vec¬ 
tor affects only the parameters of the excited receptive 
fields. 

The organization of the receptive fields of a typical 
Albus CMAC with a two-dimensional input space can 
be described as follows. The set of overlapping recep¬ 
tive fields is divided into C subsets, commonly referred 
to as layers. Any input vector excites one receptive 
field from each layer, for a total of C excited recep¬ 
tive fields for any input. The overlap of the receptive 
fields produces input generalization, while the offset of 
the adjacent layers of receptive fields produces input 
quantization. The ratio of the width of each receptive 
field (input generalization) to the offset between adja¬ 
cent layers of receptive fields (input quantization) must 
be equal to C for all dimensions of the input space. This 
organization of the receptive fields guarantees that only 
a fixed number, C, of receptive fields is excited by any 
input. 

If a receptive field is excited, its response equals 
the magnitude of a single adjustable weight specific 
to that receptive field. The CMAC output is the aver¬ 
age of the weights of the excited receptive fields. If 
nearby points in the input space excite the same re¬ 
ceptive fields, they produce the same output value. The 
output only changes when the input crosses one of the 
receptive field boundaries. The Albus CMAC thus pro¬ 
duces piecewise-constant outputs. Learning takes place 
as described above. 

CMAC neural networks have been applied in var¬ 
ious control situations Miller [77.78], starting from 
adaptation of PID (proportional-integral-derivative) 
control parameters for an industrial robot arm and 
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hand-eye systems up to biped walking (see, for in¬ 
stance, Sabourin and Bruneau [77.79]). 

The Adjustable Pattern Generator APG 
The APG model (Houk et al. [77.80]) got its name be¬ 
cause the model can generate a burst command with 
adjustable intensity and duration. The APG is based 
on the same understanding of the mossy fiber-granule 
cell-parallel fiber structure as CMAC, using the same 
state encoder, but has the crucial difference (Fig. 77.2c) 
that the role of the nuclei is crucial. In the APG model, 
each nucleus cell is connected to a motor cell in a feed¬ 
back circuit. Activity in the loop is then modulated by 
Purkinje cell inhibition, a modeling idea introduced by 
Arbib et al. [77.81], 

The learning algorithm determines which of the 
PF-PC synapses will be updated in order to improve 
movement generation performance. This is the tradi¬ 
tional credit assignment problem: which synapse (the 
structural credit assignment) must be updated based on 
a response issued when (temporal credit assignment). 
While the former is solved by the CFs, which are con¬ 
sidered binary signals, for the latter eligibility traces on 
the synapses are introduced, serving as memory for re¬ 
cent activity to determine which synapses are eligible 
for updates. The motivation for the eligibility signal is 
this: each firing of a PC cell will take some time to affect 
the animal’s movement, and a further delay will occur 
before the CF can signal an error in the movement in 
which the PC is involved. Thus the error signal should 
not affect those PF-PC synapses that are currently ac¬ 
tive, but should instead act upon those synapses that af¬ 
fected the activity whose error is now being registered. 

The Schweighofer-Arbib Model 
The Schweighofer-Arbib model was introduced in 
Schweighofer [77.82]. It does not use the CMAC state 
encoder but tries to copy the anatomy of the cerebel¬ 
lum. All the cells, fibers, and axons in Fig. 77.2a are 
included. Several assumptions are made: 

1. There are two types of mossy fibers, one type re¬ 
flecting the desired state of the controlled plant 
and another carrying information on the current 
state. A mossy fiber diverges into approximately 16 
branches. 

2. Granule cells have an average of four dendrites, 
each of which receive input from different mossy 
fibers through a synaptic structure called the 
glomerulus. 

3. Three Golgi cells synapse on a granule cell through 
the glomerulus and the strength of their influence 
depends on the simulated geometric distance be¬ 
tween the glomerulus and the Golgi cell. 


4. The climbing fiber connection on nuclear cells as 

well as deep nuclei is neglected. 

Learning in this model depends on directed error 
information given by the climbing fibers from the infe¬ 
rior olive (10). Here, long-term depression is performed 
when the 10 firing rate provides an error signal for 
an eligible synapse, while compensatory but slower in¬ 
creases in synaptic strength can occur when no error 
signal is present. Schweighofer applied the model to 
explain several acknowledged cerebellar system func¬ 
tions: 

1. Saccadic eye movements 

2. Two-link limb movement control Schweighofer 

et al. [77.83,84] 

3. Prism adaptation ( Arbib et al. [77.85]). 

Furthermore, control of a simulated human arm was 
demonstrated. 

Multiple Paired Forward-Inverse Models 
(MPFIM) 

Building on a long history of cerebellar modeling, 
Wolpert and Kawato [77.86] proposed a functional 
model of the cerebellum, which uses multiple coupled 
predictors and controllers that are trained for control, 
each being responsible for a small state-space region. 
The MPFIM model is based on the indirect/direct model 
approach by Kawato, and is also based on the micro¬ 
complex theory. We noted earlier that a microzone is 
a group of PCs, while a microcomplex combines the 
PCs of a microzone with their target cells in cerebel¬ 
lar nuclei. In MPFIM, a microzone consists of a set of 
modules controlling the same degree of freedom and is 
learned by only one particular climbing fiber. The mod¬ 
ules in this microzone compete to control this particular 
synergy. Inside such a module there are three types 
of PC, which perform the computations of a forward 
model, an inverse model, or a responsibility predic¬ 
tor, but all receiving the same input. A single internal 
model i is considered to be a controller that generates 
a motor command x and a predictor that predicts the 
current acceleration. Each predictor is a forward model 
of the controlled system, while each controller contains 
an inverse model of the system in a region of specializa¬ 
tion. The responsibility signal weights the contribution 
that this model will make to the overall output of the 
microzone. Indeed, MPFIM further assumes that each 
microzone contains n internal models of situations oc¬ 
curring in the control task. Model i generates motor 
command r,, and estimates its own responsibility r,. 
The feedforward motor command iff consists only of 
the output of the single models adjusted by the sum of 
responsibility signals: tff = Y r,-r,-/ Y r i- 
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The PCs are considered to be roughly linear. The 
MF inputs carry all necessary information including 
state information, efference copies of the last motor 
commands, as well as desired states. Granule cells, and 
eventually the inhibitory interneurons as well, nonlin- 
early transform the state information to provide a rich 
set of basis functions through the PFs. A climbing fiber 
carries a scalar error signal while each Purkinje cell en¬ 
codes a scalar output - responsibilities, predictions, and 
controller outputs are all one-dimensional values. MP- 
FIM has been introduced with different learning meth¬ 
ods: its first implementations were done using gradient 
descent methods; subsequently, expectation maximiza¬ 
tion (EM) batch-learning, and hidden Markov chain EM 
learning were applied. 

Comparison of the Models 
Summing up, we can categorize the cerebellar mod¬ 
els CMAC, APG, Schweighofer-Arbib, and MPFIM as 
follows: 

• State-encoder-driven models: This kind of model 
assumes that the granule cells are on-off types of 
entities that split up the state space. This kind of 
model is best suited for, e.g., simple function ap¬ 
proximation, and suffers strongly from the curse of 
dimensionality. 

• Cellular-level models: Obviously, the most realistic 
simulations would be at the cellular level. Unfor¬ 
tunately, modeling only a few Purkinje cells at 
realistic conditions is an immense computational 
challenge, and other relevant neurons are even less 
well understood. Still, from the biological point of 
view this kind of model is the most important since 
it allows obtaining insight into cerebellar function 
on cellular level. The first steps in this direction 
were taken by the Schweighofer-Arbib model. 

• Functional models: From the computer-science 
point of view, the most interesting models are based 
on functional understanding of the cells. In this 
case, we obtain only a basic insight of the functions 
of the parts and apply it as a crude approximation. 
This kind of approach is very promising and MP¬ 
FIM, with its emphasis on the use of responsibility 
signals to combine models appropriately, provides 
an interesting example of this approach. 

Proprioceptive feedback is used for adaptation of 
the motor programs as well as for updating the for¬ 
ward model stored in the cerebellum. However, the 
Schweighofer-Arbib model is based on the view that 
the cerebellum offers not so much a total forward model 
of the skeletomuscular system as a forward model of the 
difference between the crude model of the skeletomus¬ 


cular system available to the motor planning circuits of 
the cerebral cortex, and the more intricately parame¬ 
terized forward model of the skeletomuscular system 
needed to support fast, graceful movements with min¬ 
imal use of feedback. This hypothesis is reinforced 
by the fact that cerebellar lesions do not prohibit mo¬ 
tion but substantially reduce its quality, since the for¬ 
ward model of the skeletomuscular system is of lesser 
quality. 

77.3.4 Cerebellar Models and Robotics 

From the previous discussions, it is clear that a popular 
view is that the function of the cerebellum within the 
motor control loop is to represent a forward model of 
the skeletomuscular system; but how can these models 
be used in control? 

Our assumption is that the cerebellum stores motor 
primitive relationships, which can be recalled through 
a certain state (i. e., sensor plus cerebrum-directed goal) 
input. These motor primitives perform certain coordi¬ 
nated movements ( synergies ) to, e.g., intercept a ball 
with a tennis racket. A key property of the underly¬ 
ing spine-controlled musculoskeletal system, however, 
is that voluntary movement can be easily interpolated 
within the control realm of the spinal cord. With this 
we mean that the combination of two movement primi¬ 
tives that are nearby in the relevant sensor domain will 
lead to a good prediction. In one possible interpretation, 
the spinal cord-based control of our muscular system 
is approximately linear or linearized through internal 
models [77.87]. It allows the cerebellum to store or 
recall movements at any level of granularity, and get 
good enough results in unlearned areas. There are vari¬ 
ous papers which, in part, confirm this theory (e.g., Osu 
and Gomi show the linear relationship between muscle 
activation and joint stiffness [77.88] or Hoppner et al. 
between grip force and stiffness [77.89]). 

Does this understanding of the human control sys¬ 
tem help robotics? Biological control algorithms are 
certainly a result of slow feedback loops and the flex¬ 
ibility of the actuators. One may argue that, as robotic 
systems move towards their biological counterparts, the 
control approaches can or must do the same. There are 
many lines of research investigating the former part; 
Chaps. 11 and 75. It should be noted that the drive prin¬ 
ciple that is used to move the joints does not necessarily 
have a major impact on the outer control loop. Whether 
McKibben muscles, which are intrinsically flexible but 
bulky ( van der Smagt et al. [77.90]), low-dynamics 
polymer linear actuators, or direct-current (DC) motors 
with spindles and added elastic components are used 
does not affect the control approach at the cerebellar 
level, but rather at the motor control level (cf. the spinal 
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cord level). Of key importance, however, are the re¬ 
sulting dynamic properties of the system, which are, of 
course, influenced by its actuators. Linearity of the low- 
level control system, as we find in biology, is a goal to 


77.4 The Role of Mirror Systems 

Area F5 (frontal area 5) in the premotor cortex of the 
macaque contains, among others, neurons which fire 
when the monkey executes a specific manual action, 
e.g., one neuron might fire when the monkey performs 
a precision pinch, another when it executes a power 
grasp. (In discussing neurorobotics, it seems unneces¬ 
sary to explain in any detail the areas like F5, AIP (an¬ 
terior intraparietal sulcus), and STS (superior temporal 
sulcus) described here - they will function as labels for 
components of functional systems. To fill in the missing 
details see, e.g., Rizzolatti et al. [77.91,92].) 

77.4.1 Mirror Neurons and the Recognition 
of Hand Actions 

A subset of these neurons, the so-called mirror neurons, 
also discharge when the monkey observes meaningful 
hand movements made by the experimenter, which are 
similar to those whose execution is associated with the 
firing of the neuron. In contrast, the canonical neurons 
are those belonging to the complementary, anatomically 
segregated subset of grasp-related F5 neurons, which 
fire when the monkey performs a specific action and 
also when it sees an object as a possible target of such 
an action - but do not fire when the monkey sees an¬ 
other monkey or human perform the action. Finally, F5 
contains a large population of motor neurons that are ac¬ 
tive when the monkey grasps an object (either with the 
hand or mouth) but do not possess any visual response. 
F5 is clearly a motor area although the details of the 
muscular activation are abstracted out - F5 neurons can 
be effector-independent. In contrast, the primary motor 
cortex (FI) formulates the neural instructions for lower 
motor areas and motor neurons. 

Moreover, macaque mirror neurons encode transi¬ 
tive actions and do not fire when the monkey sees 
the hand movement unless it can also see the object 
or, more subtly, if the object is not visible but is ap¬ 
propriately located in working memory because it has 
recently been placed on a surface and has then been 
obscured by a screen behind which the experimenter 
is seen to be reaching ( Umilta et al. [77.93]). All mir¬ 
ror neurons show visual generalization. They fire when 
the instrument of the observed action (usually a hand) 
is large or small, far from or close to the monkey. 


strive for. Yet technical systems can benefit from ad¬ 
vanced modeling approaches, and equally good results 
can be obtained - yet at the cost of more complex sens¬ 
ing, computation, and less generalizability. 


They may also fire even when the action instrument has 
shapes as different as those of a human or monkey hand. 
Some neurons respond even when the object is grasped 
by the mouth. When naive monkeys first see small ob¬ 
jects grasped with a pair of pliers, mirror neurons do 
not respond, but after extensive training some precision 
pinch mirror neurons do show activity, also with this 
new grasp type [77.94]. 

Mirror neurons for grasping have also been found 
in parietal areas of the macaque brain and, recently, it 
was shown that parietal mirror neurons are sensitive 
to the context of the observed action being predictive 
of the outcome as a function of contextual cues - e.g., 
some grasp-related parietal mirror neurons may fire for 
a grasp that precedes eating the grasped object, while 
others fire for a grasp that precedes placing the object 
in a container (Fogassi et al. [77.95]). In practice, the 
parieto-frontal circuitry seems to encode action execu¬ 
tion and simultaneously action recognition by taking 
into account a large set of potential candidate actions, 
which are selected on the basis of a range of cues such 
as vision of the relation of the effector to the object 
and certain sounds (when relevant for the task). Further, 
feedback connections (frontal to parietal) are thought 
to be part of a stimulus selection process that refines 
the sensory processing by attending to stimuli relevant 
for the ongoing action ( Rizzolatti et al. [77.52] and re¬ 
call the discussion in Sect. 77.2.4). Recognition is then 
supported by the activation of the same circuitry in the 
absence of overt movement. 

We clarify these ideas by briefly presenting the 
FARS model of the canonical F5 neurons and the MNS 
model of the F5 mirror neurons. In each case, the F5 
neurons function effectively only because of the inter¬ 
action of F5 with a wide range of other regions. We have 
stressed (Sect. 77.2.3) the distinction between recogni¬ 
tion of the category of an object and recognition of its 
affordances. The parietal area AIP processes visual in¬ 
formation to extract affordances, in this case properties 
of the object relevant to grasping it ( Taira et al. [77.96]). 
AIP and F5 are reciprocally connected, with AIP being 
more visual and F5 more motoric. 

The Fagg-Arbib-Rizzolatti-Sakata (FARS) model 
(Fagg and Arbib [77.97] and Fig. 77.6) embeds F5 
canonical neurons in a larger system. The dorsal stream 
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(which passes through AIP) can only analyze the ob¬ 
ject as a set of possible affordances, whereas the ventral 
stream (via the inferotemporal cortex, IT) is able to 
recognize what the object is. The latter information is 
passed to the prefrontal cortex (PFC) which can then, on 
the basis of the current goals of the organism, bias the 
choice of affordances appropriate to the task at hand. 
Neuroanatomical data (as analyzed by Rizzolatti and 
Luppino [77.98]) suggest that PFC and IT may mod¬ 
ulate action selection at the level of the parietal cortex. 
Figure 77.6 gives a partial view of the FARS model up¬ 
dated to show this modified pathway. The affordance 
selected by AIP activates F5 neurons to command the 
appropriate grip once they receive a go signal from 
another region, F6, of the prefrontal cortex. F5 also 
accepts signals from other PFC areas to respond to 
working memory and instruction stimuli in choosing 
among the available affordances. Note that this same 
pathway could be implicated in tool use, bringing in 
semantic knowledge as well as perceptual attributes to 
guide the dorsal system ( Johnson-Frey [77.99]). 

With this, we turn to the mirror system. Since 
grasping a complex object requires careful attention to 
motion of, e.g., fingertips relative to the object, we hold 
that the primary evolutionary impetus for the mirror 
system was to facilitate feedback control of dexterous 
movement. We now show how parameters relevant to 
such feedback could be crucial in enabling the monkey 
to associate the visual appearance of what it is doing 
with the task at hand. The key side-effect will be that 
this feedback-serving self-recognition is so structured 
as to also support recognition of the action when per¬ 
formed by others - and it is this recognition of the 



Fig. 77.6 The original FARS diagram (after Fagg and Ar- 
bib [77.42]) is here modified to show PFC acting on AIP rather than 
F5. The idea is that the prefrontal cortex uses the IT identification 
of the object, in concert with task analysis and working memory, to 
help the AIP select the appropriate affordance from its menu 

actions of others that has created the greatest interest 
in mirror neurons and systems. 

The MNS model of Oztop and Arbib [77.101] pro¬ 
vides some insight into the anatomy while focusing 
on the learning capacities of mirror neurons. Here, the 
task is to determine whether the shape of the hand and 
its trajectory are on track to grasp an observed affor¬ 
dance of an object using a known action. The model 
is organized around the idea that the AIP -> F5 can onicai 
pathway emphasized in the FARS model (Fig. 77.6) is 



Fig. 77.7 The mirror neuron system 
(MNS) model (after Oztop and 
Arbib [77.100]). Note that this basic 
mirror system for grasping crucially 
links the visual process of the STS to 
the parietal regions (b) and premotor 
regions (F5), which have been shown 
to contain mirror neurons for manual 
actions 
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complemented by another pathway 7b -> FSmirror- As 
shown in Fig. 77.7 (middle diagonal), object features 
are processed by AIP to extract grasp affordances; these 
are sent on to the canonical neurons of F5 that choose 
a particular grasp. Recognizing the location of the ob¬ 
ject (top diagonal) provides parameters to the motor 
programming area F4 which computes the reach. The 
information about the reach and the grasp is taken by 
the motor cortex Ml (= FI) to control the hand and the 
arm. The rest of the figure provides components that 
can learn and apply key criteria for activating a mirror 
neuron, recognizing that the preshape of the observed 
hand corresponds to the grasp that the mirror neuron en¬ 
codes and is appropriate to the object, and that the hand 
is moving on an appropriate trajectory. Making crucial 
use of input from the superior temporal sulcus ( Perrett 
et al. [77.102] and Carey et al. [77.103]), schemas at the 
bottom left recognize the shape of the observed hand 
and how that hand is moving. Other schemas implement 
hand-object spatial relation analysis and check how ob¬ 
ject affordances relate to hand state. Together with F5 
canonical neurons, this last schema (in parietal area 7b) 
provides the input to the F5 mirror neurons. 

In the MNS model, the hand state was defined as 
a vector whose components represented the movement 
of the wrist relative to the location of the object and of 
the hand shape relative to the affordances of the object. 
Oztop and Arbib showed that an artificial neural net¬ 
work corresponding to PF and F5 mirror could be trained 
to recognize the grasp type from the hand state tra¬ 
jectory, with correct classification often being achieved 
well before the hand reached the object, using activity 
in the F5 canonical neurons that commands a grasp as 
training signal for recognizing it visually; this basically 
shows that there is a causal relationship. Crucially, this 
training prepares the F5 mirror neurons to respond to 
hand-object relational trajectories even when the hand 
is of the other rather than the self because the hand 
state is based on the view of movement of a hand rela¬ 
tive to the object, and thus only indirectly on the retinal 
input of seeing the hand and object, which can differ 
greatly between observation of self and other. Bonaiuto 
et al. [77.104] have developed MNS2, a new version of 
the MNS model to address data on audiovisual mirror 
neurons that respond to the sight and sound of actions 
with characteristic sounds such as paper tearing and nut 
cracking Kohler et al. [77.93], and on the response of 
mirror neurons when the target object was recently vis¬ 
ible but is currently hidden Umilta et al. [77.93]. Such 
learning models, and the data they address, make it clear 
that: 

mirror neurons are not restricted to recognition of 

an innate set of actions but can be recruited to rec¬ 


ognize and encode an expanding repertoire of novel 

actions. 

The discussion of this section avoided any refer¬ 
ence to imitation (Sect. 11 A3). On the other hand, even 
without considering imitation, mirror neurons provide 
a new perspective for tackling the problem of robotic 
perception by incorporating action (and motor informa¬ 
tion) into a plausible recognition process. The role of 
the fronto-parietal system in relating affordances, plans, 
and actions shows the crucial role of motor information 
and embodiment. We argue that this holds lessons for 
neurorobotics: the richness of the motor system should 
strongly influence what the robot can learn, proceeding 
autonomously via a process of exploration of the envi¬ 
ronment rather than overly relying on the intermediary 
of logic-like formalisms. When recognition exploits the 
ability to act, then the breadth of the action space be¬ 
comes crucially related to the precision, quality, and 
robustness of the robot’s perception. 

77.4.2 Computational Models 

Roboticists have been fascinated by the discovery of 
mirror neurons and the purported link to imitation that 
exists in the human nervous system, for they can help 
to teach robots new tasks with relative ease. The litera¬ 
ture on the topic extends from models of the monkey’s 
(nonimitative) action recognition system ( Oztop and Ar¬ 
bib [77. 101]) to models of the putative role of the mirror 
system in imitation ( Demiris and Johnson [77.105] and 
Arbib et al. [77.106]), and in real and virtual robots 
(Schaal et al. [77.107]). Oztop et al. [77.108] propose 
a taxonomy of the models of the mirror system for 
recognition and imitation, and it is interesting to note 
how different the computational approaches that have 
now been framed as mirror system models are, in¬ 
cluding recurrent neural networks with parametric bias 
(Tani et al. [77. 109]), behavior-based modular networks 
(Demiris and Johnson [77.105]), associative memory- 
based methods ( Kuniyoshi et al. [77.110]), and the use 
of multiple direct-inverse models as in the MOSAIC 
architecture ( Wolpert et al. [77.111]; cf. the multiple 
paired forward-inverse models of Sect. 77.3.2). 

Following [77. 1 12], we can cast much that is known 
about the mirror system into a controller-predictor 
model [77.65, 113] and analyze the resulting model as 
a Bayesian classifier. As shown by the FARS model, 
the decision to initiate a particular grasping action 
is attained by the convergence in area F5 of several 
factors, including contextual and object-related infor¬ 
mation; similarly many factors affect the recognition 
of an action. All this depends on learning both direct 
(from decision to executed action) and inverse models 



Neurorobotics: From Vision to Action I 77-4 The Role of Mirror Systems 2085 


(from observation of an action to activation of a mo¬ 
tor command that could yield it). Similar procedures 
are well known in the computational motor control lit¬ 
erature [77.114,115]. Learning of the affordances of 
objects with respect to grasping can also be achieved 
autonomously by learning from the consequences of 
applying many different actions to different parts of dif¬ 
ferent objects. 

However, how is the decision made to classify an 
observed behavior as an instance of one action or 
another? Many comparisons could be performed in par¬ 
allel with the models for one action to become predom¬ 
inantly activated. There are plausible implementations 
of this mechanism using a gating network [77.105, 

116]. A gating network learns to partition an input space 
into regions; for each region a different model can be 
applied or a set of models can be combined through an 
appropriate weight function. The design of the gating 
network can encourage collaboration between models 
(e.g., linear combination of models) or competition 
(choosing only one model rather than a combination). 
Reference [77.117] offers a similar approach to the es¬ 
timation of the mental states of the observed actor, 
using some additional circuitry involving the frontal 
cortex. 

On the other hand, if we take the Bayesian view of 
the predictor-controller formulation, then affordances 
are simply the priors in the action recognition process 
where the evidence is conveyed by the visual infor¬ 
mation of the hand, providing the data for finding the 
posterior probabilities as mirror neuron-like responses 
which automatically activate for the most probable ob¬ 
served action. Recall that the presence of a goal (at least 
in working memory) is needed to elicit mirror neuron 
responses in the macaque. We believe it is also partic¬ 
ularly important during the ontogenesis of the human 
mirror system. For example, [77.118] has shown that 
even at 9 months of age, infants recognized an action 
as being novel if it was directed toward a novel ob¬ 
ject rather than just having a different kinematics - 
showing that the goal is more fundamental than the 
enacted trajectory. Similarly, if one sees someone drink¬ 
ing from a coffee mug then one can hypothesize that 
a particular action (that one already knows in motor 
terms) is used to obtain that particular effect. The asso¬ 
ciation between the canonical response (object-action) 
and the mirror one (including vision) is made when 
the observed consequences (or goal) are recognized 
as similar in the two cases. Similarity can be evalu¬ 
ated following criteria ranging from kinematic to social 
consequences. 

In a similar experiment Lopes et al. [77.119] com¬ 
pared action recognition performance (a) when using 
the output of an inverse visuo-motor model and thus 


employing motor features to aid classification during 
the training phase, and (b) when only visual data were 
available for recognition. Overall, their interpretation of 
the results is that by mapping in motor space through 
inverse model mapping, they allow the classifier to 
choose features that are much better suited for perform¬ 
ing optimally with respect to the task of recognizing 
actions, which in turn facilitates generalization. The 
same is not true when recognition is performed purely 
in visual space using generic visual features, since 
a given action is viewed from different viewpoints. One 
may compare this to the viewpoint-invariant hand state 
adopted in the MNS model - which has the weakness 
of being built in rather than emerging from training. 

Along the same line, the work of Gijsberts 
et al. [77.120] included motorically-derived affordance 
information, which was recorded using a data-glove- 
based system and a set of cameras. In this case though, 
motor information was not much for action recogni¬ 
tion but rather used to simulate the response of F5’s 
canonical neurons by generating discrete grasping types 
from the time-varying set of postures recorded with 
the data glove. After training the original motor in¬ 
formation is removed and only reconstructed using an 
inverse model. Furthermore, this motoric information 
was combined with a simulation of the brain ventral 
pathway which extracts pictorial features from images 
(e.g., SIFT (scale-invariant feature transform), H-Max). 
The dorsal and ventral features were combined through 
a special kernel function in a simple least squares classi¬ 
fier, showing a significant improvement at recognizing 
objects in comparison to a purely visual classification. 
A machine learning framework to address the question 
of learning from multimodal signals (some of which can 
even be intermittent) is presented in [77.121]. 

We can speculate that this computational advantage 
(better recognition rates) makes the presence of mixed 
sensory and motor information compelling in the brain 
(i. e., the fronto-parietal system); this may not necessar¬ 
ily lead to mirror neurons although it seems plausible 
that any clear advantage of using information at best 
is eventually selected during evolution. These exper¬ 
iments, using robots, simulations, and computational 
arguments can thus explain the whys of certain brain 
structures and mechanisms. 

77.4.3 Mirror Neurons and Imitation 

Fitzpatrick&nd Metta [77.122] also addressed the ques¬ 
tion of what is further required for interpreting observed 
actions. Whereas in observing its own actions, the robot 
identifies them from the effects on the objects, later it 
could backtrack and derive the type of action needed 
to replicate a certain observed effect on a given object. 
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Therefore, imitation can be framed into the identifica¬ 
tion of a common goal between the observed action 
and various possible actions in the motor repertoire 
of the robot. In [77.122] the robot used the same vi¬ 
sual processing algorithms both in observing its own 
hand and the hand of a person (although they were 
different in appearance). One might argue that obser¬ 
vation alone can be used for learning, never relying 
on active exploration of objects and actions. This is 
possibly true to the extent that passive vision is re¬ 
liable and action is not required. The advantage of 
the active approach, at least for the robot, is that it 
allows controlling the amount of information imping¬ 
ing on the visual sensors by, for instance, controlling 
the speed and type of action. This strategy might be 
especially useful given the limitations of artificial per¬ 
ceptual systems. Thus, observations can be converted 
into interpreted actions. The action whose effects are 
closest to the observed consequences on the object 
(which we might translate into the goal of the action) 
is selected as the most plausible interpretation given 
the observation. Most importantly, the interpretation re¬ 
duces to the interpretation of the simple kinematics of 
the goal and consequences of the action rather than 
to understanding the complex kinematics of the human 
manipulator. The robot understands only to the extent 
it has learned to act. One might note that a more re¬ 
fined model should probably include visual cues from 
the appearance of the manipulator into the interpreta¬ 
tion process. Indeed, the hand state that was central 
to the Oztop-Arbib model was based on an object- 
centered view of the hand’s trajectory in a coordinate 
frame based on the object’s affordances. The last ques¬ 
tion to address is whether a robot can imitate the goal 
of the action. The step is indeed small, since most of 
the complexity is actually in interpreting observations. 
Imitation can be generated by replicating the latest ob¬ 
served human movement with respect to the object 
utilizing one of the many approximation methods for 
motion generation such as, e.g., a mixture of Gaus- 
sians [77.123], dynamic motion primitives [77.124], or 
reinforcement learning [77.125]. More generally, fol¬ 
lowing the work of School et al. [77.107] and Oztop 
et al. [77. 108] we can propose a set of schemas required 
to produce imitation: 

• Determining what to imitate, inferring the goal of 

the demonstrator 

• Establishing a metric for imitation (correspondence; 

see Nehaniv [77.126]) 

• Map between dissimilar bodies (mapping). 

• Imitating behavior formation. 

These are also discussed in greater detail by Ne¬ 
haniv and Dautenhahn [77.127]. In practice, computa¬ 


tional and robotic implementations have tackled these 
problems with different approaches and emphasizing 
different parts or specific subproblems of the whole, for 
example, in the work of Demiris and Hayes [77.128], 
the rehearsal of the various actions (akin to the afore¬ 
mentioned theory of motor perception) was used to 
generate hypotheses to be compared with the actual sen¬ 
sory input. It is then remarkable how more recently 
a modified approach of this paradigm has been used 
in comparison with real human transcranial magnetic 
stimulation (TMS) data. 

Ito et al. [77. 129] (not Masao Ito of cerebellar fame) 
took a dynamical systems approach using a recurrent 
neural network with parametric bias (RNNPB) to teach 
a humanoid robot to manipulate certain objects. In 
this approach the parametric bias (PB) encodes (tags) 
certain sensorimotor trajectories. Once learning is com¬ 
plete, the neural network can be used either to recall 
a given trajectory by setting the PB externally or pro¬ 
vide input for the sensory data only and observe the PB 
vector that would represent in that case the recognition 
of the situation on the basis of the sensory input only 
(no motor information available). It is relatively easy to 
interpret these two situations as the motor generation 
and the observation in a mirror neurons model. 

The problem of building useful mappings between 
dissimilar bodies (consider a human imitating a bird’s 
flapping wings) was tackled by Nehaniv and Daut¬ 
enhahn [77.127] where an algebraic framework for 
imitation is described and the correspondence problem 
formally addressed. Any system implementing imita¬ 
tion should clearly provide a mapping between either 
dissimilar bodies or even in the case of similar bodies 
when either the kinematics or dynamics is different de¬ 
pending on the context of the imitative action. 

Sauser and Billard [77.130] modeled the ideomotor 
principle, according to which observing the behavior of 
others influences our own performances. The ideomotor 
principle points directly to one of the core issues of the 
mirror system, that is, the fact that watching somebody 
else’s actions changes something in the activation of the 
observer, thus facilitating certain neural pathways. The 
work in question also gives a model implemented in 
terms of neural fields (see Sauser and Billard [77.130] 
for details) and tries to explain the imitative cortical 
pathways and the behavior formation. 

77.4.4 Mirror System and Speech 

Already in the 1960s Liberman et al. [77.131] started to 
discuss the possible links between production and per¬ 
ception in speech: in other words the contribution of 
articulation into the perception of utterances. Later he 
commented [77.132]: 
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A result in all cases is that there is not, first, a cog¬ 
nitive representation of the proximal pattern that 
is modality-general, followed by a translation to 
a particular distal property; rather, perception of 
the distal property is immediate, which is to say that 
the module has done all the hard work. 

Liberman argued that there is no such a thing 
as a modality-aspecific representation which then be¬ 
comes speech as an effect of a translation to a specific 
set of articulators (the vocal apparatus in this case), 
rather, he claimed that perception of speech is immedi¬ 
ate and effected by the same speech module (the same 
that generates speech); speech remains a motor fact. 
Lately, theories of the motor involvement in speech per¬ 
ception have gained credit because of the discovery 
of the mirror neurons. It has been postulated that the 
mirror system in humans controls jointly speech pro¬ 
duction and perception, whereby the actions in speech 
are the articulation of appropriate segments of the utter¬ 
ances [77.133], 

We recall this line of reasoning in the follow¬ 
ing [77.133], that is: 

• Mirror neurons (or a mirror system) exist in hu¬ 
mans [77.134]. 

• The human mirror system is identified in Broca’s 
area, a cytoarchitectonical homolog of area F5 in 
the macaque’s brain. 

• Speech articulation is coded/controlled in/by the ar¬ 
eas of the human mirror system (Broca’s) [77.135]. 

• The recognition of the intention of the speaker by 
the listener owing to a mirror mechanism leads to 
the first seed of true communication (via, e.g., oro¬ 
facial gestures) [77.133]. 

• The combinatorial properties of F5/Broca and the 
precise control of the effectors are needed to gener¬ 
ate speech (the evolutionarily older animal calls are 
too stereotyped to grant this flexibility that eventu¬ 
ally leads to speech proper) [77.136]. 

To establish that this is the case, however, more em¬ 
pirical evidence is required. Recently, two experiments 
improved the plausibility of the mirror neurons theory 
of speech perception. In a first TMS experiment, Fadiga 
and colleagues [77.137] established that MEPs (motor 
evoked potentials) in the tongue muscles directly corre¬ 
late with high specificity to the perception of particular 
sounds (these were rr and j ff in Italian). The listener was 
delivered TMS (single pulse) and the observed MEPs 
correlated in amplitude with the different use of the 
tongue muscles for the pronunciation of either the rr or 
ff sounds (rr in Italian requires a strong mobilization of 
the tongue). Albeit convincing, this experiment leaves 


open the question of specificity, since it can still be the 
case of a diffuse/generic activation of Broca’s area. 

A second experiment also by Fadiga et al. [77.137] 
was designed to set the issue. In this case, the TMS 
was delivered to the primary motor cortex with the aim 
of establishing a specific motor involvement into the 
perception of different sounds/phones. Two areas were 
individuated in the primary motor cortex as responsible 
to the lip and tongue movement, respectively (e.g., p/b 
sounds versus t/d). The data show a double dissociation 
pattern, that is, when the lip motor area is stimulated 
there is a decrease of the reaction times (RTs) of the 
subject in perceiving the p/b (labial sounds) and vice- 
versa an increase for the perception of the t/d (dental 
sounds). The opposite happens when the tongue motor 
area is stimulated. This experiment clearly relates a very 
specific (small) region of the primary motor cortex with 
the perception of certain specific (and related) sounds. 

Clearly, this is only part of the story; to complicate 
matters, for example, the semantic content of words 
related to actions (e.g., kick, pick, lick) activate both 
motor and pre-motor brain areas somatotopically. Ob¬ 
ject features, odors, etc., instead have been shown to 
generate responses in the corresponding cortices. For 
a review of these and other results, see [77.136]. 

Theories and models such as the perception for 
action control theory (PACT) [77.138] take a more 
moderate interpretation by including both a motor com¬ 
ponent and perceptual shaping, that is, the filtering 
of certain linguistic combinations because of purely 
perceptual characteristics (e.g., separation of vowel for¬ 
mants). In PACT, it is hypothesized that the motor 
system is activated more in adverse conditions, while 
it is perhaps under-threshold for normal speech under¬ 
standing in good signal-to-noise conditions. 

Indeed, we can recognize speech in a foreign ac¬ 
cent, and recognizing what is being said can then be 
decoiupled from being able to articulate how it is being 
said - but both possibilities are available. This has led 
to a new view of the integration of mirror systems with 
other systems [77.139] which downplays the motor the¬ 
ory of speech perception while preserving many other 
features of the mirror system hypothesis of Rizzolatti 
and Arbib [77.133]. 

Armed with these results Castellini and col¬ 
leagues [77.140] conducted a computational experi¬ 
ment that mimic some of the TMS results of D 'Ausilio 
et al. [77.141]. All processing employed a database 
of synchronized recordings of Italian speakers with 
acoustic, articulograph, camera, ultrasound, and elec- 
troglottograph data [77.142]. For the experiments, only 
the articulograph and electroglottograph signals were 
used together with speech sound. These identify the 
position of the tongue and teeth versus the lips in 
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Fig. 77.8 Conceptual schema of the classifiers used in the experi¬ 
ments 


Balanced error rate (%) 



Fig. 77.9 Baseline experiment comparing the performance 
of acoustic versus motor data (or jointly acoustic plus mo¬ 
tor data) in classifying b/p versus d/t 


real time (200 Hz) in addition to the activation of the 
vocal folds (voicing signal). The conceptual schema 
of all experiments and learning follows some previ¬ 
ous work as by Metta et al. [77.112], and which as 
is shown in Fig. 77.8. In particular, acoustic data are 
mapped into motoric features and these are used for 
classifying phones. Similarly to the PACT model, it 
was found useful to incorporate also a purely acoustic 
classifier. Acoustic features were the standard Mel cep- 
stral coefficients with similar parameters and frequency 


bands of conventional automatic speech recognition 
(ASR). 

The mapping from acoustic to motor data was per¬ 
formed using either an artificial neural network or 
support vector machine for regression with indistin¬ 
guishable results. The classifier was always a support 
vector machine with Gaussian kernel and parameters 
optimized through grid search. 

In order to compare it with the TMS experiments, 
phones were divided into two classes, the b/p and 
t/d, respectively, as representing the bi-labial and den¬ 
tal (movement of the tongue toward the teeth) phones. 
Fivefold cross-validation was employed on all results 
by either random splits of the data or by selecting data 
from various participants (e.g., training on 1—5 par¬ 
ticipants, testing on 1—5 participants). Gaussian white 
noise was added to the stimuli (at increasing levels from 
0 to 150%) to replicate the conditions of the TMS. 

Figure 77.9 shows these results. The baseline exper¬ 
iment shows an improved performance where either the 
real motoric or jointly motoric and acoustic features are 
used. The comparison of audio versus joint features is 
statistically significant (p < 0.01) and verifies the claim 
as no new information is added to the system when the 
reconstructed motor features are employed. Motor fea¬ 
tures are reconstructed by the audio-motor map (AMM) 
of Fig. 77.8 and replicates previous results obtained in 
the classification of hand gestures [77. 1 19] or handwrit¬ 
ing characters [77.143]. 

A second experiment from the same work of 
Castellini and colleagues [77.140] shows the behavior 
of the same system in various conditions of increased 
difficulty ranging from running classification on speak¬ 
ers not included in the training sets to co-articulation. 
Figure 77.10 shows a number of variants where N vs M 
indicates N speakers for training versus M speakers for 
testing given the size of the database (6 speakers). Ex- 


Balanced error rate (%) 



Fig. 77.10 Comparison across various 
conditions. In all cases apart from 
coart4vsl the use of motor features 
improve classification with statistical 
significance (p < 0.01) 
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Balanced error rate (%) 



Added noise (% of speech stdv) 


Fig. 77.11 Comparison of acoustic 
versus motor features under increasing 
level of added Gaussian white noise 
for the same classifier of the previous 
experiments 


periments with co-articulation were conducted also on 
five speakers, albeit the number of identifiable examples 
in the database was smaller. 

In a final experiment, the classifier was tested on 
acoustic data corrupted by Gaussian white noise. Re¬ 
sults show a consistent improvement with the motor 
information gain increasing with the increase of the 
noise level (up to 150% of the speech standard devia¬ 
tion): Fig. 77.11. 


More recently a full phone classifier was built using 
similar principles [77.144] together with a combina¬ 
tion of deep belief networks (DBNs) and more standard 
hidden Markov models (HMMs). The results show im¬ 
provement with respect to the state of the art, continuing 
the long tradition of neurorobotics and bringing mod¬ 
els very close to concrete applications on robots that 
bear resemblance to the exquisite human performance 
in speech recognition in noisy environments. 


77.5 Conclusion and Further Reading 

As the foregoing makes clear, robotics has much to 
learn from neuroscience and much to teach neuro¬ 
science. Neurorobotics can learn from the ways in 
which the brains and bodies of different creatures adapt 
to diverse ecological niches - as computational neu¬ 
roethology helps us understand how the brain of a crea¬ 
ture has evolved to serve action-oriented perception, 
and the attendant processes of learning, memory, plan¬ 
ning, and social interaction. 

We have sampled the design of just a few sub¬ 
systems (both functional and structural) in just a few 
animals - optic flow in the bee, approach, escape, and 
barrier avoidance in frogs and toads, and navigation 
in the rat, as well as the control of eye movements in 
visual attention, the role of the mammalian cerebel¬ 
lum in handling the nonlinearities and time delays of 
flexible motor systems, and the mirror systems of pri¬ 
mates in action recognition and of humans in imitation. 
There are many more creatures with lessons to offer the 
roboticist than we can sample here. 

Moreover, if we just confine attention to the brains 
of humans, this chapter has mentioned at least 7a, 


7b, AIP, lateral, medial and ventral intraparietal sulcus 
(LIP, MIP and VIP), area 46, basal ganglia, caudoputa- 
men, cerebellum, F2, F4, F5, hippocampus, hypotha¬ 
lamus, inferotemporal cortex, motor cortex, nucleus 
accumbens, parietal cortex, prefrontal cortex, premotor 
cortex, pre-SMA (F6), spinal cord, STS, and - and it is 
clear that there are many more details to be understood 
for each region, and many more regions whose interac¬ 
tions hold lessons for roboticists. We say this not to de¬ 
press the reader, but rather to encourage further explo¬ 
ration of the literature of computational neuroscience 
and to note that the exchange with neurorobotics pro¬ 
ceeds both ways: neuroscience can inspire novel robotic 
designs; conversely, robots can be used to test whether 
brain models still work when they make the transition 
from disembodied computer simulation to meeting the 
challenge of guiding the interactions of a physically em¬ 
bodied system with the complexities of its environment. 

Nonetheless, a thorough study of the spinal cord and 
its effect on muscle behavior is where a roboticist, who 
is interested in replicating some of the functionality of 
vertebrate movement, may want to start looking. 
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77.5.1 Further Reading 

• Arbib (2006) [77.145]: This volume provides 16 
articles on the mirror system, written by diverse 
experts. Of particular relevance to this chapter are 
articles on dynamical systems: brain, body and im¬ 
itation; attention and the minimal subscene; the 
development of grasping and the mirror system; 
and development of goal-directed imitation, ob¬ 
ject manipulation, and language in humans and 
robots. 

• Bell (1996) [77.146]: This somewhat older BBS 
special issue provides what was, back then, a rather 
definitive number of articles on the cerebellum, in¬ 
cluding an overview of models in a paper by Houk 
et al. 

• van der Smagt and Bullock (2002) [77.147]: This 
special issue is focused on the application of cere¬ 


bellar and other models to robotics tasks, and lists 
some successful and - between the lines - more un¬ 
successful applications thereof. 

• Gallese et al. (1996) [77.148]: This paper provides 
a detailed account of the neurophysiological evi¬ 
dence for mirror neurons. It is good reading to get 
the real data unbiased from further interpretation on 
the role of mirror neurons and it is complete and ac¬ 
curate. Although it is a technical paper it is easy to 
read also for a general audience. 

• Fadiga et al. 2002 [77.149]: This work extends the 
mirror system concept with an interesting perspec¬ 
tive on its role into language. This paper is interest¬ 
ing reading by providing evidence in humans (the 
other references above are about monkey experi¬ 
ments). In this case, it has been shown that speech 
listening facilitates the activation of tongue muscles 
which match the specific phoneme being listened to. 
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78. Perceptual Robotics 


Heinrich Biilthoff, Christian Wallraven, Martin A. Giese 


Robots that share their environment with humans 
need to be able to recognize and manipulate ob¬ 
jects and users, perform complex navigation tasks, 
and interpret and react to human emotional and 
communicative gestures. In all of these percep¬ 
tual capabilities, the human brain, however, is 
still far ahead of robotic systems. Hence, tak¬ 
ing clues from the way the human brain solves 
such complex perceptual tasks will help to de¬ 
sign better robots. Similarly, once a robot interacts 
with humans, its behaviors and reactions will be 
judged by humans - movements of the robot, 
for example, should be fluid and graceful, and 
it should not evoke an eerie feeling when inter¬ 
acting with a user. In this chapter, we present 
Perceptual Robotics as the field of robotics that 
takes inspiration from perception research and 
neuroscience to, first, build better perceptual ca¬ 
pabilities into robotic systems and, second, to 
validate the perceptual impact of robotic systems 
on the user. 
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The technical realization of perceptual functions is 
a central problem for many applications in robotics. 
Robots require perception to navigate in space and to lo¬ 
calize and recognize goal objects, e.g., for manipulation 
(Chaps. 7, 8, 32, 33, 36-38, 47, 67). Social interac¬ 
tive robots must be able to interpret gestures, actions, 
and even emotions (Chap. 69, 71, 72) in order to inter¬ 
act naturally with their users. One important approach 
for the programming of complex perceptual and be¬ 
havioral functions, for example, needed for humanoid 
robots is imitation learning (Chaps. 75, 77). Imitation 
learning requires the robot to perceive complex actions 


that are executed by the user and to subsequently map 
them into an efficient representation that is suitable for 
the synthesis of the corresponding motor behavior on 
the available platform. This chapter focuses on impor¬ 
tant principles of the representation of complex shapes 
and movements, which can be derived from biologi¬ 
cal perception systems, and more specifically the basic 
functionality of the primate visual cortex. Such prin¬ 
ciples have interesting implications for the design of 
technical systems in robotics and computer vision for 
the recognition of objects, shapes and faces, and for the 
recognition and synthesis of complex movements and 
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actions. The limited space of the chapter forced us to 
focus mainly on visual perception and related techni¬ 
cal applications. In the context of robotics many other 
aspects of perception are important, for example hap¬ 
tic perception (Chap. 41), auditory perception, sensory 
cue fusion (Chap. 35), and the interaction between the 
visual recognition of objects and actions and motor pro¬ 
grams, e.g., during grasping (treated in Chap. 38). 

In the following, we will first formulate several biolog¬ 
ical principles that are relevant for form and motion 
representations, specifically in the visual system. We 
will then, on the one hand, describe technical systems 
that implement these principles using neural mecha¬ 
nisms that are inspired by the basic architecture of the 
brain. On the other hand, we will discuss also imple¬ 
mentations that are inspired by biological principles on 
a more abstract level, and which exploit instead of neu¬ 
ral networks more efficient technical algorithms for the 
realization of biologically relevant functions. Many of 
these systems are derived in the held of computer vi¬ 
sion and are based on the advantages and limitations of 
modern digital computers in order to more efficiently 
realize biological principles of information processing. 

Our approach to establish relationships between 
biological perception and robotics systems at differ¬ 
ent levels reflects David Marr's classical distinction of 
multiple levels of description, originally developed for 
the analysis of vision systems [78.1]: Robotics systems 
can be inspired by biological system at the level of 
implementation, i. e., one can try to build robots con¬ 
taining neural mechanisms that imitate the function of 
neurons in central nervous systems of biological or¬ 
ganisms. This type of analogy between technical and 
biological systems coincides with the definition of Neu¬ 
rorobotics given in Chap. 77. A transfer of principles 
from biological perception systems to robots might also 
be accomplished at the more abstract levels of compu¬ 
tational problems and algorithms. The computational 
level is defined by the abstract theoretical formulation 
of computational problems that have to be solved by 
perception systems. Examples are the identification or 
classification of goal object, or the recognition of hu¬ 
man gestures. Marr’s level of algorithms specifies the 
computational methods for the solution of such prob¬ 
lems, independent of the underlying specific hardware 
or architecture. For example, an object might be rep¬ 
resented by modeling its full 3-D structure, e.g., using 
a parametric 3-D shape model, or it might be rep¬ 
resented in terms of two-dimensional example views. 
Example views, however, might be represented using 
neural networks, establishing an analogy with the hu¬ 
man brain at the level of implementation, or using more 
efficient computational methods, e.g., as support vec¬ 
tors of a classifier that has been trained with appropriate 


images of the object and distractor patterns. In both the 
cases, the robot system realizes mechanisms that are de¬ 
rived form perception in biological systems. 

Marr’s distinction of levels is only one way to in¬ 
troduce description levels for complex systems. Other 
approaches, particularly relevant for robotics, are, for 
example the subsumption architecture and behavior- 
based approaches (Chap. 13) that decompose robotics 
system into a system of simpler behavioral modules. 
Another examples are dynamical systems approaches 
to robotics [78.2—4] that are based on the biologi¬ 
cally motivated idea that behaviors can be mapped onto 
stable states of (nonlinear) dynamical systems or re¬ 
current neural networks. Individual behaviors result by 
self-organization over the whole system as collectively 
stable modes, which can be described and analyzed 
by the introduction of appropriate collective variables. 
Interestingly, such robotics-inspired approaches have 
been quite successful in modeling human navigation be¬ 
havior [78.5]. 

In the following, we will apply the term Perceptual 
Robotics to signify the design of robots based on prin¬ 
ciples that are derived from human perception on all 
three levels in the sense of Marr. This includes a real¬ 
ization in terms of specific neural circuits as well as the 
transfer of more abstract biologically inspired strategies 
for the solution of relevant computational problems. 
A direct interaction between robotics and perception 
research can be very fruitful for both disciplines. On 
the one hand, our current knowledge about the human 
perception and the underlying computational principles 
might help us to build more efficient robotics architec¬ 
tures that inherit properties from biological perception, 
e.g., very efficient and robust processing or complex 
dynamic flexibility. Such architectures will be a neces¬ 
sary pre-requisite for the creation of truly intelligent, 
cognitive robots (Chaps. 13, 71, 74, 75). On the other 
hand, perception science often uses robots as testbed 
for gaining a deeper understanding of computational 
processes, in particular, for testing the computational 
power of specific computational solutions under real- 
world conditions. How can a child, for example, learn 
how to handle new objects, and what allows us to 
learn the visual categorization of thousands of objects 
from just a few examples? Perceptual robot platforms, 
equipped with a variety of sensory inputs and operating 
in different types of artificially structured or real-world 
environments provide very helpful tools for the study of 
such questions. 

Finally, perceptual robotics not only means to take 
inspiration from perception to build more efficient 
robots, but it also encapsulates the perceptual valida¬ 
tion of robotic systems. As robots move into the human 
environment and are increasingly also interacting with 
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humans, it becomes important to evaluate and validate 
their effectiveness and efficacy with respect to human 
standards. Here, we do not refer to their social accep¬ 
tance, but rather to the way that robots are judged by 
humans in terms of their appearance, movements, and 
interactive capabilities. If a robot displays jerky move¬ 
ments, for example, it may still successfully grasp and 
manipulate an object, but it would be immediately no¬ 
ticeable to a human observer and potentially disturbing 
to interact with. This eeriness or weirdness was already 
anticipated in the early 1970s in a famous paper about 
the uncanny valley by a Japanese roboticist [78.6]. Mori 
anticipated that as robots become more human-like, hu¬ 
mans’ familiarity with the robot would increase until 
at some point (when the robot looks or acts almost 
human-like), they would suddenly feel highly unfamil¬ 
iar toward the robot. As the human likeness increases 
further, the robot would again be judged as familiar or 
appealing. 

More specifically, Mori also postulated that this un¬ 
canny valley would not only hold for the robot’s static 
appearance, but would in fact be increased for a moving 
or acting robot. With the increase in interest in develop¬ 


ing humanoid robots over the past decades, being aware 
of the perceptual judgments of such humanoids be¬ 
comes a critical component in their development. Since 
the evaluation of appearance and movements of a hu¬ 
manoid are driven by perceptual processes, it makes 
sense to also use protocols from perception research to 
evaluate and fine-tune their effectiveness. In such exper¬ 
iments, typically the robot’s performance is evaluated 
with respect to measures such as general user accep¬ 
tance, recognizability of expressions, smoothness of 
motions, ease of interaction, duration and quality of in¬ 
teraction, etc. It is important that the experiment should 
not only be about simply asking how good is the robot, 
but it should actually tests the robot in the intended task 
context or that whether it uses additional, indirect mea¬ 
sures of effectiveness. As a tutorial on designing and 
analyzing perceptual experiments and user studies is 
beyond the scope of this chapter, we refer the reader 
to introductory texts such as [78.7,8]. In this chapter, 
we will focus on two important topics related to hu¬ 
manoid perception in the context of perceptual robotics: 
facial animation and the perceptual processing of body 
movements. 


78.1 Perceptual Mechanisms of Object Representations 


Object recognition is a fundamental visual function 
that is critical for many applications in robotics. Ma¬ 
nipulation and grasping (Chaps. 36-38) require exact 
knowledge about the shape of the goal object that 
is often derived from visual sensors. Also the imita¬ 
tion of goal-directed movements (Chap. 77) requires 
knowledge about target objects. Finally, social and col¬ 
lective robots require robust recognition of other agents 
and objects which are taking part in the present ac¬ 
tion (Chaps. 71, 72). The importance of object and 
shape recognition for many other applied robot sys¬ 
tems, like construction and assembly robots or smart 
cars (Chap. 54) is immediately evident. 

78.1.1 Perceptual and Computational Basis 
of Object Representations 

The question of how humans learn, represent, and 
recognize objects under a wide variety of viewing con¬ 
ditions presents a great challenge to both neurophysiol¬ 
ogy and cognitive research. Frameworks for explaining 
the amazing robustness of human recognition processes 
and how humans represent objects can be broadly 
classified into two approaches: in the model-based rep¬ 
resentation, an image on the retina is analyzed to yield 
three-dimensional parts of an object based on geomet¬ 


ric primitives (cf. also Chap. 32). These primitives are 
then matched to an internal, three-dimensional model 
of the object (Fig. 78.1, bottom). Exemplar-based rep¬ 
resentation approaches assume that the internal storage 
consists of, typically two-dimensional, snapshot-like 
representations of objects, which are directly compared 
to the visual input via simple image transformations. 
In the following, we will briefly describe the basic 
properties of these two approaches as well as percep¬ 
tual evidence for their plausibility in explaining human 
recognition performance. 

Structural Description Models 
The basic idea of structural description models is that 
object recognition or categorization is based on a struc¬ 
tural representation, which is defined as a configura¬ 
tion of elementary object parts that are regarded as 
shape primitives [78.9]. Structural description models 
aim at supplying abstract and propositional descrip¬ 
tions of objects, while at the same time disregard¬ 
ing irrelevant spatial information. Therefore, structural 
description models typically predict that recognition 
performance is invariant regarding spatial transforma¬ 
tions. Biederman’s recognition-by-components (RBCs) 
or geon structural description (GSD) model can be 
regarded as the best developed example of the struc- 
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Fig. 78.1 Schematic drawing comparing exemplar-based with model-based representations. Object perception based on 
model-based representations assumes that the brain extracts 3-D parts from the visual image, which are then matched to 
an internally stored 3-D model of the teapot. Contrasting with this approach, object perception based on exemplar-based 
representations is accomplished by directly comparing stored templates or example images with the current picture of 
the teapot 


tural description model type [78.10]. According to 
this model, objects are represented as configurations 
of elementary three-dimensional primitive parts, called 
geons. These geons are derived from nonaccidental 
properties (NAPs) in the image, i. e., from properties 
which unlikely arise by chance, and are more or less in¬ 
variant over a wide range of views. For example, the 
properties straight vs. curved, symmetrical vs. asym¬ 
metrical, parallel vs. nonparallel are regarded as NAPs 
(NAPs were originally proposed within an image-based 
approach by Lowe [78.11]). According to the model, 
geons and their spatial configuration are combined into 
a structural representation, called GSD. The spatial re¬ 
lations between parts are described in a categorical way, 
using relations like above, below, etc. Like other struc¬ 
tural description models, Biederman’s model predicts 
invariance in relation to position and size and also in 
relation to orientation in depth, as long as no parts are 
occluded. 

The question has to be raised whether objects can 
be decomposed into geons at all. It was argued that 
Biederman’s RBC cannot be applied to a whole range 
of biological stimuli [78.12], or that biological shapes 
in general cannot be adequately described by struc¬ 


tural description models [78.13]. This problem extends 
also to artifact categories like shoe, hat or backpack, 
which seem to exceed the scope of the geon model. 
Therefore it has to be doubted that object parts are 
necessarily represented as geons, or as similar geomet¬ 
rical primitives (further problems of RBC in [78.14, 
15]). However, this does not mean that category rep¬ 
resentations do not have a part structure: in fact, it is 
not the notion of the part structure in object representa¬ 
tions by itself which is problematic, but the use of parts 
and relations as a basis to derive invariant recognition 
performance [78.15]. 

Exemplar-Based Models 

Over the last two decades, an increasing number of 
studies has demonstrated that recognition is not view- 
independent. Orientation-dependent recognition effects 
were found for novel objects [78.16,17], and also 
for common, familiar objects [78.18, 19]. Orientation- 
dependent recognition performance has been shown not 
to be limited to individual objects, such as faces [78.20, 
21], or to objects on the subordinate level of catego¬ 
rization [78.16, 22], but also was demonstrated for basic 
level recognition [78.19, 23]. 
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Moreover, recognition performance is not only in¬ 
fluenced by the orientation, but also by the size of the 
stimulus. Results are quite similar: reaction times (RTs) 
and error rates depend on the extent of transforma¬ 
tion that is necessary to align memory and stimulus 
representation. RTs increase in a monotonic way with 
increasing change of (perceived) size (for a review 
Ashbridge and Perrett [78.24]). Several studies even 
show a systematic relationship between the amount 
of translation and recognition performance: Increasing 
displacement between two sequentially presented stim¬ 
uli led to a deterioration of performance, both for novel 
objects [78.25] and familiar objects [78.26]. Overall, 
view-independent models are difficult to reconcile with 
these findings which indicate that recognition perfor¬ 
mance depends systematically on different spatial trans¬ 
formations. 

In the following, we will briefly review three 
types of exemplar-based models, which - by virtue 
of different computational mechanisms and pro¬ 
cesses (including alignment, interpolation, and pooling/ 
thresholding) - explain the transformation-dependent 
performance that was found in the psychophysical ex¬ 
periments. 

In the class of alignment models, Ullman' s [78.12, 
27] 3-D alignment model and Lowe's [78.11] SCERPO 
model are probably the best-known examples. Both 
models work by storing 3-D models of objects, which 
are aligned to images by perspective projection of cor¬ 
responding features (edges or feature points on the 
object). As an alternative to Ullman s [78.27] model 
that relies on 3-D object representations, Ullman and 
Basri [78.28] suggested an alignment model on the ba¬ 
sis of 2-D (two-dimensional) views. In this model, an 
internal object model is constructed by a linear com¬ 
bination of a small number of stored 2-D exemplar 
images. Thus, the alignment is not achieved by a spa¬ 
tial compensation process, but by linear combination 
of images. The intuition behind the linear combination 
approach can be explained in simple terms. Suppose 
that two views of the same three-dimensional object are 
stored, taken from somewhat different viewing direc¬ 
tions. An intermediate view can then be described as 
a weighted sum of the views that are already stored. 
In this case, the representation is based on the two- 
dimensional positions of corresponding features in each 
view. Making the set of views closer results in an ob¬ 
ject representation that is equivalent to storing a 3-D 
model. 

In the interpolation model, recognition is achieved 
by localization in a multidimensional representational 
space, which is spanned by stored views [78.29]. The 
interpolation model is based on the theory of approx¬ 
imation of multivariate functions and can be imple¬ 


mented with radial basis functions (RBFs). In this 
scheme, the whole viewing space of an object is approx¬ 
imated by the learned exemplar views through a limited 
number of series of so-called radial basis functions 
(such as Gaussian functions) each of which becomes ac¬ 
tivated within a limited region of the high-dimensional 
feature space. Object recognition then means to exam¬ 
ine whether a new point corresponding to the actual 
stimulus can be approximated by the existing tuned 
set of basis functions. Thus, recognition does not oc¬ 
cur by transformation or reconstruction of an internal 
image, but rather by interpolation or approximation 
of exemplars in a high-dimensional representational 
space. 

At the end of the 1990s - and as an extension to 
the interpolation models - recognition models based on 
pooling and thresholding were developed [78.30-33]. 
Recognition is explained on the basis of the behaviour 
of cells that are selectively tuned to specific image fea¬ 
tures (fragments or whole shapes) in a view-dependent 
(and size-dependent) way. A hierarchical pooling of the 
outputs of view-specific cells provides generalization 
over viewing conditions [78.30]. A similar proposal 
was made by Riesenhuber and Poggio [78.31]. The 
threshold model [78.34] also accounts for the sys¬ 
tematic relation between recognition latencies and the 
amount of rotation (and size-scaling). The speed of 
object recognition depends on the rate of accumula¬ 
tion of activity from neurons selective for the object, 
evoked by a particular viewing circumstance. For a fa¬ 
miliar object, more tuned cells will be activated in the 
views most frequently presented, so that a given level 
of evidence (threshold) can be achieved fast. When the 
object is seen in an unusual view, fewer cells will re¬ 
spond, and activity among the population of cells selec¬ 
tive for the object’s appearance will accumulate more 
slowly. Consequently, these threshold models explain 
orientation-dependence without the need to postulate 
transformation or interpolation processes. 

In a recent paper, an attempt has been made at 
view-dependent and view-independent approaches to 
object processing [78.35]. A careful study of the view- 
dependency of novel objects was designed by combin¬ 
ing structural properties (number of parts) with metric 
properties (thickness, size of parts) has found that both 
view-dependent and view-independent processing seem 
to be combined in object recognition. Thus, instead of 
taking the extreme standpoints of view-based versus 
view-invariant processing, one might envisage a visual 
processing framework in which features are selected 
according to the current task, where the optimality, effi¬ 
ciency and thus the dependency on viewing parameters 
of the features depend on the amount of visual experi¬ 
ence with this particular task. 
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Several computational models have been proposed 
that aim at modelling and explaining the dependence 
of human recognition performance on spatial transfor¬ 
mations in its complexity. All of these models rely on 
storing exemplars - in the simplest form just 2-D views 
of objects - and matching the retinal image to these 
stored examples by different computational methods. 
The later models of recognition take their inspira¬ 
tion from recent findings from physiological studies 
concerning the functional building blocks of human vi¬ 
sion in the brain. In the following, we will therefore 
briefly review the neural processing of visual informa¬ 
tion in the brain that underlies our ability to recognize 
objects. 

78.1.2 Neural Representations 
in Object Recognition 

Functionally, it has been shown that the flow of visual 
information in the brain can be divided into two major 
pathways: the dorsal pathway is believed to process 
motion and motor- or action-related visual information, 
whereas the ventral pathway usually is associated with 
the task of object recognition. The structure of the 
ventral pathway is hierarchically organized and consists 
of a series of interconnected stages that start from the 
retina, passing through the lateral geniculate nucleus 
(LGN) to the primary visual cortex (VI) and extrastriate 
visual areas V2, V4, and IT. The inferotemporal cortex 
(IT) provides input to the prefrontal cortex (PFC), 
which is believed to play an important role in identifi¬ 
cation and categorization of visual stimuli. Recordings 
in the parietal cortex [78.36] suggest, in addition, that 
specifically for grasping and object manipulation also 
dorsal regions might be centrally involved in the recog¬ 
nition of manipulable objects and their affordances 
(Chap. 77 for a more detailed discussion). 

The seminal work of Hubei and Wiesel [78.37] in 
the cat (and later also in the macaque) visual cortex first 
established the idea of a hierarchical organization of vi¬ 
sual processing. They found so-called simple cells in 
the early visual cortex (area VI) that responded best to 
bar-like stimuli at a particular orientation and position 
in the visual held. The response pattern of these cells 
could be modeled as a receptive held using Gabor-type 
functions. Later in the processing stream they found so- 
called complex cells which responded best to bar-like 
stimuli at a particular orientation nearly everywhere in 
the visual held - cells, which had become partially posi¬ 
tion invariant. This general idea of increasing invariance 
to stimulus properties with later stages of the process¬ 
ing stream has been verihed in further physiological 
studies. In general, it has been found that the receptive 
held of the neurons increases and that the complexity of 


the stimulus it responds also increases. One of the key 
studies about the functional role of IT regions has inves¬ 
tigated the responses of neurons to real-world objects in 
anesthetized monkeys ([78.38]; see also Tanaka [78.39] 
for a review). Although some neurons were found 
which responded maximally to simple bar-like stimuli, 
the majority of neurons in posterior inferotemporal cor¬ 
tex (PIT) preferred complex objects such as star shapes 
or circles with protruding elements. Interestingly, neu¬ 
rons were highly sensitive to minuscule changes to 
these objects such as the relative orientation or thick¬ 
ness of the elements. On the other hand, neurons were 
quite insensitive to stimulus variations such as size, 
contrast or retinal location. These findings were taken 
as evidence that one of the strategies for representing 
objects might be to use a number of moderately com¬ 
plex visual elements, whose pattern of co-activation 
encodes the visual appearance of the stimulus. In ad¬ 
dition, Wang et al. [78.38] found neurons in anterior 
inferotemporal cortex (AIT), which responded maxi¬ 
mally to images of whole objects such as faces or cars, 
indicating that already in IT object specific encodings 
might be present. Several other studies have also found 
neurons in this area which are tuned to faces, parts of 
faces, as well as body parts ([78.40] for a review). 

In another set of experiments, Logothetis 
et al. [78.41] found AIT neurons, which showed 
a strong view-based behavior for the same stimuli 
that were used in the study of Bidthoff and Edel- 
man [78.16], whereas they were invariant to size and 
location of the stimulus. Their findings provide strong 
evidence that a neural implementation of view-based 
object encoding is possible and indeed seems to be 
used for recognition. In addition to view-selectivity and 
size invariance, the investigated cells were also found 
to be maximally selective for the holistic stimulus 
rather than its constituent parts. This finding indicates 
that these cells might be encoding the pooled co¬ 
activation pattern of earlier PIT cells and thus form 
view-tuned units of recognition. It is important to stress 
in this context that an abstraction such as grandmother 
neurons, which specifically encode only one stimulus, 
does not seem plausible. Rather, the majority of neural 
responses in this and other experiments showed selec¬ 
tivity for a number of stimuli. A plausible explanation 
for this finding is that objects are encoded not by 
a single neuron but by a population code encompassing 
a number of neurons, which greatly increases the 
robustness of the representation [78.33]. 

The findings from this area of research can be 
summarized in a simple functional architecture: going 
from early stages to later stages of visual processing 
in a feedforward fashion, feature complexity increases 
from simple edge detectors toward view-tuned, com- 
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plex object cells and invariance to changes in the 
stimulus increases. This functional architecture is re¬ 
flected not only in the object recognition framework 
discussed previously, but it also provides the motivation 
for computational vision systems that have been devel¬ 
oped over the last few decades which will be discussed 
in the following. 

78.1.3 Object Recognition: 

Lessons from Computer Vision 

Computer vision started out as a subfield of artificial 
intelligence in the 1960s. Early work on scene under¬ 
standing by Roberts [78.42] showed how computers 
could parse worlds consisting of simple, geometric ob¬ 
jects such as cubes, pyramids, etc. The main thrust of 
computer vision systems in the following decades con¬ 
sisted of building algorithms for reconstructing a three- 
dimensional world from images - this development was 
further stimulated by Marr’s very influential theory of 
vision as 3-D reconstruction [78.1]. This theory was 
built on extracting geometric primitives from images 
that could be mathematically described as generalized 
cylinders. Although the mathematical rigor of such ap¬ 
proaches was very appealing, computational implemen¬ 
tations turned out to have strong limitations. Extracting 
robust features is a necessary prerequisite for building 
a 3-D reconstruction of the image, and finding these 
features proved to be hard under real-world conditions 
due to the enormous amount of variation in the image 
caused by changes in lighting, depth rotations, noise, 
occlusion, etc. 

Parallel to the paradigm shift in human psy¬ 
chophysics and physiology, exemplar-based computa¬ 
tional systems began to emerge, which for the first time 
showed good recognition performance under a larger 
range of viewing conditions. These recognition sys¬ 
tems were based on - sometimes surprisingly simple - 
histograms of pixel values [78.43], local feature de¬ 
tectors [78.44,45] or on a straightforward pixel repre¬ 
sentation of images using principle components analy¬ 
sis [78.46]. All of these recognition systems relied on 
a database of labeled example images, an algorithm 
for extracting features from these images, and a suit¬ 
able classification method for comparing sets of image 
features. 

Returning to the discussion of modeling human vi¬ 
sion, in the following we provide an exemplary review 
of three neuromorphic recognition systems that are 
based on a functionally plausible, exemplar-based ar¬ 
chitecture: these are SpikeNET [78.48], LeNet [78.49], 
and a framework by Serre et al. [78.50]. The first system 
is motivated by the finding that humans are amazingly 
fast at categorizing images as containing an animal or 


a face [78.51]. Typical response times for this task are 
so small (on the order of 100 ms) that the visual sig¬ 
nal has only time for one feedforward pass through 
the visual areas of the brain (Fig. 78.2) - any recur¬ 
rent feedback processing would necessarily delay the 
decision and therefore result in longer response times. 
Based on this finding, a neural network architecture 
was designed [78.48] that exploits the timing of neu¬ 
ronal responses (spikes) to encode visual signals using 
a who fires first - strategy. This is different from tra¬ 
ditional neural networks in that the timing is used 
rather than the firing strength. An object in this sys¬ 
tem will therefore be represented by an ensemble of 
neurons that represents a pattern of spike responses 
from earlier low-level, feature extraction neurons. In 
their implementation, these low-level neurons consist 
of standard Gabor-type receptive fields that are similar 
to the receptive fields found in the cat’s visual cor¬ 
tex [78.37]. This spike time encoding allows for very 
fast processing of visual stimuli and has been shown 
to provide robust recognition results. The network ar- 


Accuracy (%) 



Fig. 78.2 Recognition performance of four highly similar object 
(shown in the inset ) by an in-hand recognition system using active 
view selection (after [78.47]). The five methods compared in the 
plot contrast planned (blue) and unplanned (orange) exploration of 
the objects in the hand of the robot. The .v-axis is the number of iter¬ 
ations, and the y-axis is the recognition accuracy in percent. As time 
(or iteration number) proceeds, the planned approaches surpass ran¬ 
dom exploration significantly. In addition, employing proabilistic 
methods for recognition of the objects using a particle filter also 
provides a recognition improvement. Finally, the thick, solid blue 
line shows performance in a system which boosts the likelihood 
of an object given the current visual evidence in the particle filter 
framework - this approach fares best overall. These results show 
that active view selection enhances the robot’s ability to learn and 
recognize objects in real-world environments 
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chitecture LeNet [78.49] consists of a neural network 
that uses a hierarchy of layers of trainable convolutions 
and spatial subsampling, as well as nonlinear filtering 
to extract features of increasingly large receptive fields, 
increasing complexity, and increasing robustness. Us¬ 
ing extensive, supervised training of the full hierarchy, 
such a network provides a very efficient, sparse set of 
features for many visual recognition tasks. Finally, the 
network architecture by Serre et al. [78.50] uses a very 
similar hierarchical structure of layers in which feature 
complexity and invariance are successively increased 
by linear and nonlinear pooling - its lower level fea¬ 
ture detectors, however, are trained in an unsupervised 
fashion on a large database of natural images, yield¬ 
ing a large set of detectors that are optimally tuned 
to natural image statistics. Again, the performance of 
this model in recognition tasks has been shown to be 
very good - in addition, comparisons with physiolog¬ 
ical and psychophysical experiments have shown that 
this framework is also capable of modeling human re¬ 
sults from these experiments. 

Recent research has mainly focused on two top¬ 
ics: the automatic extraction of optimal visual features 
for efficient recognition and categorization, and the 
extension of the frameworks for providing invariance 
against changes in viewing conditions (such as rotations 
in depth, scaling, translation, illumination, and occlu¬ 
sion, for example, DiCarlo et al. [78.52], Rolls [78.53] 
for discussions of invariance in neuromorphic architec¬ 
tures). In a recent paper [78.54], these two issues have 
been addressed in a face recognition task conducted 
on a difficult database of faces taken in uncontrolled 
environments. The selection of optimal features was 
done by evaluating a large set of potential visual fea¬ 
ture combinations using GPU-accelerated algorithms. 
The issue of invariance was addressed by using a hier¬ 
archical, multilayer model in which each layer includes 
linear and nonlinear pooling operations that encode the 
input image. The combined system was benchmarked 
against other standard feature-extraction methods and 
a flat, nonhierarchical one-layer model. Both proper¬ 
ties resulted in increased recognition performance on 
the database outperforming other benchmarked state- 
of-the-art methods. In addition, the system also showed 
increased robustness against viewing variations, which 
included pose, position, scale, and background clutter. 

In summary, neuromorphic architectures have now 
reached a stage of maturity that can put them even 
ahead of sophisticated, state-of-the-art computer vision 
frameworks. The ability to learn and adapt the feature 
set to viewing conditions and the increased robustness 
to viewing conditions makes such architectures good 
candidates for building the visual learning and recog¬ 
nition system for a perceptual robot. 


78.1.4 Object Learning and Recognition 
for Perceptual Robotics 

In general, it can be said that the success of perceptu¬ 
ally inspired recognition systems can be seen as a strong 
indicator for the feasibility of a data-driven, exemplar- 
based approach to recognition. There are three issues, 
however, which so far have not been addressed in any of 
these vision systems and which will be important both 
for achieving human performance in generic recogni¬ 
tion tasks in a perceptual robotics application - as well 
as for a full understanding of the processes in human 
object recognition. 

First of all, all of the above-mentioned systems are 
feedforward - virtually no feedback, recurrent process¬ 
ing is implemented in their architecture, which makes 
them in a sense very similar to the simpler frog- or bee¬ 
like neural systems discussed in Chap. 77. Although 
there is evidence that humans solve some recogni¬ 
tion tasks using very little feedback (see, e.g., Thorpe 
et al. [78.51]; DiCarlo et al. [78.52]), it nevertheless is 
a crucial component of visual processing driving, for 
example, attentional focus, context awareness, as well 
as memory and reasoning processes - basically every¬ 
thing that makes up visual intelligence. Some visual 
attention models that are relevant for robotics systems 
are reviewed in Chap. 77. 

Secondly, a severe limitation of most of today’s ar¬ 
tificial recognition systems is that they solely focus on 
the static domain of object recognition. Visual input on 
the retina, however, consists of dynamic changes due to 
object- and self-motion, nonrigid deformations of ob¬ 
jects, articulated object motion as well as scene changes 
such as variations in lighting, occluding, and re- and 
disappearing objects - where at any given point in time 
several of these changes can be interacting. Several 
psychophysical experiments, indeed suggest an impor¬ 
tant role for dynamic information, both in learning and 
recognition of objects [78.55-58]. These results ask 
for an extension of current object recognition frame¬ 
works with a temporal component in order to arrive 
at truly spatiotemporal object representations. Com¬ 
bining methods from computer vision, psychophysics, 
and machine learning, Wallraven and Biilthoff [78.59, 
60], have developed a framework that fulfills this re¬ 
quirement and learns spatiotemporal, exemplar-based 
object representations from image sequences. More 
specifically, spatiotemporal characteristics of the visual 
input are integrated into a connected view-graph rep¬ 
resentation based on tracked local features. In order 
to provide robust classification performance, machine 
learning techniques are used to design efficient meth¬ 
ods for combining support vector classification schemes 
with these local feature representations [78.61]. In sev- 
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eral studies it was shown that the framework achieved 
excellent recognition results on both highly controlled 
databases as well as on real-world data. The integration 
of spatiotemporal information provides characteristic 
information about dynamic visual input via the connec¬ 
tion of views and the two-dimensional image motion of 
discriminative features. In addition to delivering good 
recognition performance, the framework was also able 
to model results from psychophysical experiments on 
face and object recognition. A similar model using 
a neuromorphic architecture integrating the temporal 
dimension was proposed by Kietzmann et al. [78.62], 

A third issue that - in our view - will be essen¬ 
tial for designing and implementing efficient perceptual 
robots consists of the multisensory nature of our per¬ 
ceptual system (see also the discussion of embodied 
robots in Chap. 13). As an example, there is a close 
coupling between the human visual and haptic system - 
touch can provide a wealth of complementary infor¬ 
mation about an object when it is manipulated, such 
as its texture, its shape, its position in space relative 
to our body, etc. In a series of psychophysical exper¬ 
iments [78.63], participants had to learn views of four 
simple, 3-D objects made of stacked toy-bricks either in 
the haptic modality (when they were blind-folded) or in 
the visual modality (without being able to touch them). 
Subsequently, they were tested both within the same 
modality as well as across modalities. Recognition re¬ 
sults showed that cross-modal recognition is possible 
well above chance. Not surprisingly, recognition of 
rotated objects in the within-modality condition was 
severely affected by rotation in both modalities. This 
shows that not only visual recognition is highly view- 
dependent but also that haptic recognition performance 
is directly affected by different viewing parameters. The 
results from this experiment thus support the view that 
haptic recognition is also mediated by exemplar-based 
processes. 

Taken together with the keyframe framework out¬ 
lined above, this cross-modal transfer might be an 
important reason for the excellent visual performance 


of human object recognition - after all, it is known 
that infants learn extensively by actively grasping and 
touching objects, which thus could provide a database 
of object representations for visual recognition [78.64]. 
Using this basic perceptual principle as a motiva¬ 
tion [78.60] have applied an extension of the keyframe 
framework in an online robotics scenario for efficient 
learning and recognition of multisensory object rep¬ 
resentations. More specifically, a framework was de¬ 
veloped to integrate both proprioceptive information 
originating from haptic sensors in the robot’s hands 
and visual information coming from the robot’s cam¬ 
eras. For this, the robot would perform an exploratory 
movement with an object in its hand (such as turning it 
and looking at it from all angles) and from the result¬ 
ing image sequence learn spatiotemporal, view-based 
representations using the keyframe framework. Each 
view of this representation, however, is also linked to 
the current proprioceptive state (i. e., the joint angles 
of the hand at that point in time) and therefore pro¬ 
vides an anchor into a hand-centered, three-dimensional 
space. In this way, a representation is generated that 
links perception and action. The proprioceptive infor¬ 
mation can then be used as an additional constraint for 
both learning of objects and recognition of objects and 
was shown to provide increased robustness compared 
to visual matching alone. The framework was also used 
as the basis for recent work in which a humanoid robot 
(the iCub) performed active in-hand object recognition, 
searching for the optimal view that allowed it to disam¬ 
biguate the object currently held from other, previously 
seen objects [78.47]. Again, linking the exploratory ac¬ 
tions (turning the hand) with the visual data resulted 
in a much faster and more reliable object recognition 
performance. Sample data comparing unplanned and 
planned recognition of difficult objects is shown in 
Fig. 78.2 (Ida iM’J I']*'«*■ ) 

Such approaches pave the way for a view of recog¬ 
nition as an active, multisensory process in which rich, 
extensible object representations are formed and im¬ 
proved over the life-time of the robot. 


78.2 Perceptual Mechanisms of Action Representation 


The recognition of complex movements and actions is 
fundamental for many applications in robotics, such 
as imitation learning by observation. Interactive robots 
need to analyze their users’ movements in order to re¬ 
spond in a natural way to their social and emotional 
behavior (Chap. 72). The following section reviews 
what is known about movement and action recognition 
in the brain and tries to highlight a few aspects that have 


or might be successfully transferred to biologically in¬ 
spired applications in robotics and computer vision. 

78.2.1 Recognition of Complex Movements 
and Actions in Primate Cortex 

The recognition of complex movements and actions is 
a fundamental problem for higher animals and specif- 
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ically for primates. While simple movement patterns 
are sufficient for eliciting stereotypical prey catch¬ 
ing behavior in simple vertebrates ([78.65]; see also 
Chap. 77), higher animals exploit more complex move¬ 
ment patters, e.g., for the recognition of conspecifics 
or predators, or for communication by facial move¬ 
ments, gestures, or body expressions. Human percep¬ 
tion of body motion patterns is very efficient, even for 
extremely impoverished stimuli. This has been demon¬ 
strated in classical experiments by Johansson [78.66], 
who showed that complex dynamic actions can be rec¬ 
ognized even from displays that consist only of a small 
number of dots moving like the joints of a human ac¬ 
tor. Subsequent research has demonstrated that humans 
can extract highly specific information from such point- 
light displays, e.g., the gender or the identity of people. 
To our knowledge, no technical system for motion 
recognition has been proposed so far that accomplishes 
a comparable level of robustness. While much more 
research in neuroscience has been dedicated to object 
recognition (Sect. 78.1.2), some studies have tried to 
uncover neural [78.67-70], and computational princi¬ 
ples [78.71-73] of visual movement recognition. Some 
of these principles have been transferred to the con¬ 
struction of systems in computer vision and robotics. 

Neurophysiological and brain-imaging studies indi¬ 
cate that the recognition of facial and body movements 
involves the ventral and the dorsal visual pathway. This 
implies that likely form and optic flow information are 
integrated during the processing of action stimuli in vi¬ 
sual cortex. The ventral pathway, which is specifically 
responsible for the processing of form information has 
been discussed already in Sect. 78.1.2. Like the ventral 
stream, also the dorsal pathway is hierarchically struc¬ 
tured, and the size of the receptive fields of the neurons 
increases along the hierarchy. Some cortical areas that 
are part of the dorsal pathway are listed in Fig. 78.3. 
The medial temporal area (MT) contains neurons that 
are selective for simple local motion and coherent mo¬ 
tion. On higher levels of the dorsal stream, e.g., in the 
superior temporal sulcus (STS), neurons that are se¬ 
lective for hand and body movements and for facial 
expressions have been found in monkeys [78.69], and 
similar structures are activated by these stimuli in the 
human brain. In addition, areas selective for human 
body shapes, such as the extrastriate body part area 
(EBA), likely to contribute to the recognition of ac¬ 
tions [78.74], where information of form and motion 
features seems to be integrated on higher processing 
levels [78.73]. 

For the recognition of goal-directed actions, such as 
reaching or grasping, in addition cortical structures be¬ 
yond the visual cortex, such as the parietal and premotor 
cortex, seem to play a critical role. The role of these 


structures for action recognition has been analyzed in 
particular in the context of the study of the mirror neu¬ 
ron system [78.75]. Mirror neurons are sensorimotor 
neurons that combine visual tuning during action ob¬ 
servation as well as selective motor tuning. Areas in 
parietal cortex, such as the anterior might be specifi¬ 
cally relevant for the recognition of action-related ob¬ 
jects and their relationship to moving effectors [78.36]. 
Research about the mirror neuron system has influenced 
the construction of a whole generation of biologically 
inspired robots (Chap. 77). The guiding hypothesis has 
been that the visual recognition and understanding of 
actions is accomplished by mapping of observed body 
movements onto motor representations that are relevant 
for the execution of the same type of action. 

78.2.2 Biological Principles with Relevance 
for Computer Vision and Robotics 

We discuss in the following two major principles that 
have been derived from the analysis of action recogni¬ 
tion in biological systems that have been transferred to 
technical applications in computer vision and robotics. 

A first principle that seems to be implemented in 
movement recognition in primate cortex is a hierar¬ 
chical architecture of feature detectors, which accom¬ 
plishes action recognition by the detection of temporal 
sequences of relevant motion and form features. Such 
detection does not necessarily require the reconstruc¬ 
tion of the three-dimensional facial or body shape, 
nor an exact simulation of the dynamics of the under¬ 
lying movements. Instead it can be accomplished by 
much simpler computational mechanisms. Like object 
recognition, the recognition of complex motion pat¬ 
terns is strongly orientation- and view-dependent. This 
property has been observed at the level of individual 
neurons in the STS, and for the activation of biologi¬ 
cal motion-selective areas in human cortex [78.69] as 
well as for action-selective neurons in higher areas 
such as the premotor cortex [78.76]. View- and orien¬ 
tation dependence seem compatible with an encoding 
of visually perceived movements in terms of poten¬ 
tially learned example views, or keyframes (snap shots), 
and of instantaneous optic flow patterns that are char¬ 
acteristic for actions [78.67,73]. While there might be 
some innate preferences for specific features [78.77], 
psychophysical and fMRI (functional magnetic reso¬ 
nance imaging) experiments suggest an important role 
of learning in visual movement recognition [78.78-80]. 
For example, subjects can learn easily to recognize indi¬ 
vidually - specific body and facial movements [78.81, 
82], Learning-based theoretical models, exploiting sim¬ 
ilar principles as neural object recognition models, 
account for a variety of experimental data on action 
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Fig. 78.3 Example-based neural model for the visual recognition of body movements that integrates the processing of 
form and motion features in the ventral and dorsal visual pathways (after Giese and Poggio [78.73]) 


recognition in biological systems [78.73,83,84], also 
supporting a central role of learning. 

As an example of such a learning-based archi- 
texture, Fig. 78.3 illustrates a hierarchical model for 
the recognition of complex body movements [78.73]. 
It consists of two hierarchical streams modeling the 
ventral and the dorsal visual pathways, which contain 
detectors for action-specific motion and form features. 
The form pathway of this model is similar to the 
object recognition models described in Sect. 78.1.2. 
The motion pathway of the model contains detectors 
for action-specific optic flow features with different 
complexity. Like for the described object recognition 
models, position, and scale invariance is accomplished 
by appropriate nonlinear pooling of the responses of de¬ 
tectors with different spatial and scale selectivity along 
the hierarchy. In addition, the model contains recurrent 
neural circuits that make the responses of the recogni¬ 
tion neurons selective for temporal order. In this way the 
model responds only to actions that are executed with 
the correct temporal order, and also with approximately 
correct speed. The underlying network dynamics can 
be interpreted as a neural implementation of a Markov 
model, where the present recognized pattern predicts 


possible future patterns (Chap. 68). A strong activity in 
the network emerges only when the stimulus sequence 
matches these predictions. 

Similar hierarchical neural architectures inspired by 
the visual cortex have been used in the context of 
mirror-neuron robot systems [78.85, 86]. In addition, 
recent work in computer vision shows that such bi¬ 
ologically inspired architectures can reach very high 
performance levels, comparable to state-of-the-art algo¬ 
rithms in computer vision [78.87-89]. 

A second principle of movement recognition, which 
has been discussed extensively as basis for the recog¬ 
nition of imitable actions, and as explanation of the 
function of the mirror neuron system [78.75] is the 
idea that action observation is based on an internal sim¬ 
ulation of the observed motor behavior. A variety of 
computational models for action recognition by internal 
simulation have been proposed in the neuroscience lit¬ 
erature, e.g., exploiting feedforward controllers [78.90], 
coupled forward and backward models [78.91], hier¬ 
archical Bayesian predictive models [78.92], or a free 
energy minimization framework [78.93]. (A further 
more extensive discussion about theoretical models for 
the mirror neuron system with relevance for robotics 


-a 

OJ 

3 . 

Cl 


-4 

00 

NJ 

















Part G | 78.2 


2106 Part G 


Robots and Humans 


can be found in Chaps. 68 and 77.) A main difficulty 
of the recognition of actions by internal simulation of 
associated motor behaviors is the accurate estimation 
of relevant geometrical quantities from image data, es¬ 
pecially when no special depth sensors or even online 
motion capture are available. Many of the underly¬ 
ing motor control models are formulated in joint angle 
space, and the robust recognition of joint angles from 
monocular videos is known to be a difficult computer 
vision problem, which so far is solvable only for highly 
restricted classes of movements with strong learned pri¬ 
ors, and at considerable computational cost [78.94, 95]. 
This raises the question about simpler computational 
approaches for the recognition of goal-directed actions, 
which explain biological data and might be interesting 
for technical applications. 

A recently developed model for the visual recogni¬ 
tion of goal-directed hand actions in cortex that follows 
these lines [78.96] is illustrated in Fig. 78.4. The under¬ 
lying architecture is an extension of the form pathway 
of the model shown in Fig. 78.3, by the addition 
of neural circuits that process the spatial and tempo¬ 
ral relationship between the observed effector (in this 
case the hand) and the recognized goal object (e.g., 
a grasped object). The model works, exploiting a purely 
exemplar-based approach (Sect. 78.1.1), without ex¬ 
plicit reconstruction of the three-dimensional structure 
of the object or the effector. The model comprises 
three modules: The first module (A, in Fig. 78.4) rec¬ 
ognizes shapes of the goal object and of the effector, 


implementing a shape recognition hierarchy similar to 
standard object recognition models as the ones de¬ 
scribed in Sect. 78.1.3. The analysis of the temporal 
deformation of the hand is based on the recognition 
of sequences of key shapes, like in the form path¬ 
way of the model in Fig. 78.3. Opposed to standard 
object recognition models, however, the highest level 
of this shape recognition hierarchy is not completely 
position-invariant. Rather, it retains coarse position se¬ 
lectivity by implementing multiple replica of the same 
shape detectors that are selective for different image po¬ 
sitions. This makes it possible to further analyze the 
spatiotemporal relationship of the recognized goal ob¬ 
jects and effector. This analysis is realized in the second 
module (B) whose core is formed by two-dimensional 
relative position maps. These are neural activity maps 
that represent the effector position as activity peak in 
a two-dimensional coordinate system that is centered 
on the object position in the image. These maps are 
computed by a gain fields [78.97] that multiply the 
output activities of shape selective neurons with selec¬ 
tivity for object and effector. The activity distribution 
in these neural map is analyzed by affordance neurons 
that are activated only when hand and object shape 
match and are in a spatial relationship that is suit¬ 
able for a successful grip. By appropriate pooling of 
the responses of motion energy detectors that receive 
input from the relative position map relative motion 
neurons can be constructed, whose activity character¬ 
izes the relative motion between the effector and object 



Fig. 78 . 4 a-c Physiologically inspired model for the recognition of goal-directed hand actions. The shape-recognition (a) rec¬ 
ognizes the shape of goal objects and the shapes of individual hand postures, retaining some course position information. The 
module (b) associates the information of hand and object by computing maps that represent the relative positions of hand and 
object in image coordinates. From these maps the spatial matching hand and object and their relative motion can be computed. 
The highest level module (c) contains model neurons that are selective for different types of goal-directed actions. Up to this 
level the model recognizes actions in a view-dependent manner, and only at the highest level (view-independent transitive actions 
neurons) the model accomplishes view independence by pooling the outputs from view-specific modules (courtesy of Fleischer 
et al. [78.96]) 
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(e.g., the hand approaching the object). The outputs 
of affordance and the relative motion neurons are in¬ 
tegrated in the third module (C), which contains only 
neurons that are selective for visually observed goal- 
directed actions. This integration of information is first 
accomplished in a purely view-specific manner. View 
independence is not established until the very last hier¬ 
archy level of the model that contains view-independent 
action-selective neurons. The idea of establishing view- 
dependence only very late in the cortical hierarchy 
seems counter-intuitive, and is at odds with several es¬ 
tablished computational models for action recognition. 
However, this dominance of exemplar-based represen¬ 
tations until very high levels of the cortical processing 
hierarchy has been observed in electrophysiological 
experiments studying mirror neurons in premotor cor¬ 


tex [78.76]. In this structure, which is traditionally 
associated with motor planning, the majority of mir¬ 
ror neurons is view-dependent and only a minority is 
view-independent. The discussed model can recognize 
hand actions from gray level videos. It could be aug¬ 
mented by integration of disparity or depth features, 
and by appropriate attentional control mechanisms that 
would make it more robust to cluttered scenes with 
multiple relevant objects. Whether similar architectures 
have advantages for the robust visual recognition of 
goal-directed actions in technical systems remains to 
be shown. Very recent work shows that such hierarchi¬ 
cal deep architectures, which consist of learned feature 
detectors, outperform classical technical solutions on 
actual computer vision benchmarks for action detec¬ 
tion [78.98], 


78.3 Perceptual Validation of Robotics 


Successful human-robot interaction is perhaps easiest 
when the robot offers interaction channels that are com¬ 
patible to that of human-human interaction [78.99]. 
The most important interaction channels in this case 
are verbal and nonverbal communication with the face. 
Importantly, in human-human interaction, nonverbal 
communication using facial expressions, for example, 
constitutes up to 30% of the communicative content. 
Facial expressions are not only used to convey some¬ 
one’s mood and emotion [78.100], but are also used in 
communicative contexts to signal understanding (a nod 
of the head), to modify what is being said (a raise 
of the eye-brows), and to control the conversational 
flow (a look of confusion may signal to the speaker 
to repeat what has been said). Hence, many humanoids 
have incorporated more or less sophisticated heads ca¬ 
pable of producing human-like facial expressions and 
movements. Traditionally, this has been achieved us¬ 
ing mechatronic implementations in which actuators 
drive facial features directly (e.g., as in the MDS robot 
by Lee and Breazeal [78.101]), or - in more com¬ 
plex implementations of android robots - mimic human 
muscle movements that are then used to deform artifi¬ 
cial skin [78.102-104]. Other systems have used LEDs 
for displaying simple, changeable facial features (e.g., 
as in the iCub platform [78.105]. 

78.3.1 Realistic Faces for Robots 

With such a great variety of robot systems also comes 
the need for an investigation of their perceptual eval¬ 
uation and their interaction capabilities ([78.106] for 
such a study in the context of facial animations in 


computer graphics). Indeed, one particular problem of 
android, human-like systems is that they easily could 
suffer from the uncanny valley effect as the actuators 
and/or the control framework cannot easily reproduce 
the smoothness of human facial expressions. A study 
of morphed images between a nonhuman robot face 
and a highly realistic android robot head, for exam¬ 
ple, clearly showed evidence of the uncanny valley 
effect [78.107] - a similar study for moving robot faces 
yielded more mixed results, but still showed that the 
most realistic robotic faces were clearly perceived as 
different from that of a human talking [78. 108]. One so¬ 
lution for this is to change the robot’s appearance such 
that it stays away from close human likeness; however, 
conveying the full breadth of human communicative 
signals with different facial features, or a different facial 
topology may also be problematic. A different solution 
consists of avoiding a mechanical solution and instead 
resorting to facial animation from computer graphics. 
One example of such a system was presented in De¬ 
launay et al. [78.109] in which facial animations are 
projected onto a rigid face mask. Since facial anima¬ 
tion techniques are in many ways much more advanced, 
such a system allows for a more realistic and flexi¬ 
ble interaction in human-robot interaction. Subtle cues 
such as eye-gaze, wrinkles, and other nonrigid facial 
deformations could be displayed via projection of an 
advanced facial animation engine. Initial perceptual ex¬ 
periments with such systems [78.109, 110] have yielded 
promising results. However, more studies need to be 
done to assess the properties of human-robot inter¬ 
actions in these and other implementations of facial 
displays. 
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78.3.2 Perceptual and Neural Processing 
of Body Movements of Robots 

Research in humanoid robotics finally aims at opti¬ 
mizing the perceived naturalness or human-likeness of 
generated robot movements, since this in the long run 
will increase the acceptance of humanoid robots in 
social contexts. However, the present humanoid plat¬ 
forms have typically substantial constraints that still 
prevent the realization of complex really human-like 
movements. This is even more the case for the real¬ 
ization of behaviors on bipedal robots, due to the dif¬ 
ficult problem to maintain dynamic balance (Chap. 67). 
Therefore, most body movements realized by present 
humanoid robots still differ in many aspects from hu¬ 
man movements. This makes the quantification of the 
degree of realism of such movements presently a less 
pressing topic than the field of computer graphics, 
where psychophysical studies for the validation of the 
realism and quality of computer animation methods 
are meanwhile a standard [78.112-114]. However, re¬ 
search in psychophysics and neuroscience has started 
to investigate the differences between the perceptual 
processing of human and robot movements, and inter¬ 
esting results have been obtained that localize corti¬ 
cal subsystems that might be essential to distinguish 
human and nonhuman robot movements. A typical 
question in these studies has been which critical prop¬ 
erties determine whether visual stimuli produce motor 
resonance, or an activation of action-selective neural 
structures. The results of such studies have not been 
completely consistent, since some studies found de¬ 
creased activation of action-selective networks for robot 
movements [78.115-118] while others found no such 
differences [78.119]. Primarily visual processing areas 
responded sometimes more for robot movements than 
for normal human [78.111,117]. 


Fig. 78.5 Stimuli used in experiment 
investigating the fMRI correlates of 
the observation of human and robot 
movements. The neural responses to 
the movements by a real human are 
compared with the ones induced by 
a human-looking robot (Android), 
and by a nonhumanlike robot (Robot) 
(after Saygin et al. [78.111]) 

The problem of such studies is that many factors 
might influence the perception and neural signals in 
action-selective areas, such as form, kinematics, and 
optic flow patterns. Typically, it is very difficult to 
control these parameters separately for real robots. In 
addition, the learning experience of observers with the 
specific robot might play an important role [78.120]. 
A recent study by Saygin et al. [78.111] tried to sep¬ 
arate at least the influences of the robot appearance 
(shape) and the motion kinematics by comparing the 
fMRI signals (using an adaptation paradigm) driven by 
three different stimuli (Fig. 78.5): a real person (that 
served as model for the building of the robot), the 
human-like looking robot (android), and the robot with¬ 
out skin and surface parts that made it look human-like 
(resulting in very similar motion as the full robot). In 
visual areas (e.g., the extrastriate body area) the hu¬ 
man and the human-like robot stimulus result in very 
similar activity. This is not true for the parietal cor¬ 
tex, which is part of the mirror neuron network. This 
region shows large differences between the human¬ 
like robot and the other two conditions, potentially 
reflecting an increase of neural processing resources 
that are required to cope with the contradiction be¬ 
tween the form and the kinematic information that is 
presented by this stimulus. Opposed to this, the not 
human-like robot makes it expected that the motion 
is also not human-like, potentially causing no such 
conflict. Future studies of similar type, controlling for 
the different information channels of action process¬ 
ing (Sect. 78.2.2) as well as for the predictibility of 
such stimuli dependent on previous learning, poten¬ 
tially combined with quantitative neural modeling, will 
be required to really understand how different factors 
are integrated in the neural processing of robot move¬ 
ments, and how this causes different levels of perceived 
human-likeness. 


78.4 Conclusion and Further Reading 

In this chapter, we have presented several principles the development of systems in robotics and computer 
derived from high-level cognitive processing in vi- vision. The recognition of shapes and complex move- 
sion in the human brain that have been fruitful for ments and actions is an important problem for many 
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applications in robotics. We have discussed a vari¬ 
ety of results from neuroscience that indicate that 
these brain functions are likely realized by example- 
based representations. We have discussed neural im¬ 
plementations of such representations which partially 
have been tested successfully in the context of tech¬ 
nical applications, and which are strongly inspired by 
the real cortical neural architecture. In addition, we 
have presented some new computational principles that 
seem to emerge from recent experimental results on 
the representation of goal-directed actions. Finally, we 
have discussed work that tries to use psychophysi¬ 
cal and neuroscience methods for the validation of 
the appearance and the movements of human-like 
robots, and for the investigation of underlying neural 
mechanisms. 

Example-based mechanisms for object and motion 
recognition account for the invariant recognition of 
complex patterns. However, they do not automatically 
extract the metric information about the object geom¬ 
etry, position and the spatial parameters of complex 
trajectories in world coordinates. For some tasks in 
robotics, like grasping, manipulation, or obstacle avoid¬ 
ance, such information is required (Chaps. 36-38, 47). 
For such tasks, example-based recognition must be 
fused with methods for the extraction of the relevant 
metric information. In robotics such information can 
be extracted by stereo vision or using special sensors, 


like laser range finders. In the brain the fusion between 
such spatial information and information about objects 
occurs likely in parietal areas, like the anterior interpari¬ 
etal area (AIP) [78.121]. However, it is unclear whether 
the information about objects is only represented in 
terms of 2-D example views. Instead, it seems likely 
that also some form of 3-D information is encoded, po¬ 
tentially in an example-based manner. Also haptic and 
visual information about object shape might be merged 
in higher brain areas, e.g., in parietal and fusiform ar¬ 
eas [78.122]. A further discussion about biologically 
inspired models for the extraction of action-relevant ge¬ 
ometrical information in the context of grasping and 
manipulation is given in Chaps. 32 and 77. 

The perceptual validation of the human-likeness 
and affective impact of humanoid robots likely will 
become increasingly important along with the further 
development of the technology that will increase the 
level of similarity between humanoid robots and hu¬ 
mans. Likewise, it seems increasingly important to use 
quantitative methods from perception science to inves¬ 
tigate the quality of the emotional and social interaction 
between robots and humans. We expect this to be a field 
where psychology can really contribute quantitative 
methods to engineering, reaching a level that goes be¬ 
yond a qualitative and subjective comparison of clemos 
which partially is still the standard in the field of hu¬ 
manoid robotics. 


Video-References 


IdaMJESIESl Active in-hand object recognition 

available from http://handbookofrobotics.org/view-chapter/78/videodetails/569 


References 


78.1 D. Marr: Vision (Freeman, San Francisco 1982) 

78.2 J.A.S. Kelso: Dynamic Patterns: The Self- 
Organization of Brain and Behaviour (MIT, 
Cambridge 1995) 

78.3 G. Schoner, M. Dose, C. Engels: Dynamics of be¬ 
havior: Theory and applications for autonomous 
robot architectures, Robotics Auton. Syst. 16 , 213— 
245 (1997) 

78.4 J. Tani, M. Ito: Self-organization of behavioral 
primitives as multiple attractor dynamics: A robot 
experiment, IEEE Trans. Syst. Man Cybern. A 33(4), 
481-488 (2003) 

78.5 W.H. Warren: The dynamics of perception and ac¬ 
tion, Psychol. Rev. 113 , 358-389 (2006) 

78.6 M. Mori: The uncanny valley, Energy 7(4), 33-35 
(1970), in Japanese 

78.7 D.W. Cunningham, C. Wallraven: Experimental 
Design: From User Studies to Psychophysics (CRC, 
Boca Raton 2011) 


78.8 A. Field, G. Hole: How to Design and Report Ex¬ 
periments (Sage, London 2011) 

78.9 D. Marr, H. Nishihara: Representation and recog¬ 
nition of the spatial organization of three- 
dimensional shapes, Proc. R. Soc. B 200, 269-294 
(1978) 

78.10 I. Biederman: Recognition-by-components: 
A theory of human image understanding, 
Psychol. Rev. 94 , 115-147 (1987) 

78.11 D. Lowe: Perceptual Organization and Visual 
Recognition (Kluwer, Boston 1985) 

78.12 S. Ullman: High-Level Vision. Object Recognition 
and Visual Cognition (MIT, Cambridge 1996) 

78.13 M.A. Kurbat: Structural description theories: Is 
RBC/JIM a general-purpose theory of human 
entry-level object recognition?, Perception 23, 
1339-1368 (1994) 

78.14 S. Edelman: Representation and Recognition in 
Vision (MIT, Cambridge 1999) 


Part G | 78 






Part G | 78 


2110 Part G 


Robots and Humans 


78.15 M. Graf, W. Schneider: Structural descriptions in 
HIT - A problematic commitment, Behav. Brain 
Sci. 24 , 483-484 (2001) 

78.16 H.H. Biilthoff, S. Edelman: Psychophysical support 
for a two-dimensional view interpolation theory 
of object recognition, Proc. Nati. Acad. Sci. USA 
89, 60-64 (1992) 

78.17 M.J. Tarr, S. Pinker: Mental orientation and 
orientation-dependence in shape recognition, 
Cogn. Psychol. 21 , 233-282 (1989) 

78.18 W.G. Hayward, M.J. Tarr: Testing conditions for 
viewpoint invariance in object recognition,!. Exp. 
Psychol. 23,1511-1521 (1997) 

78.19 S.E. Palmer, E. Rosch, P. Chase: Canonical perspec¬ 
tive and the perception of objects. In: Attention 
and Performance IX, ed. by J. Long, A. Baddeley 
(Erlbaum, Hillsdale 1981) pp. 135-151 

78.20 H. Hill, P.G. Schyns, S. Akamatsu: Information and 
viewpoint dependence in face recognition, Cog¬ 
nition 62 , 201-222 (1997) 

78.21 C. Wallraven, A. Schwaninger, S. Schuhmacher, 
H.H. Biilthoff: View-based recognition of faces in 
man and machine: Re-visiting inter-extra-ortho, 
Led. Notes Comput. Sci. 2525 , 651-660 (2002) 

78.22 M.J. Tarr: Rotating objects to recognize them: 
A case study on the role of viewpoint dependency 
in the recognition of three-dimensional objects, 
Psychon. Bull. Rev. 2, 55-82 (1995) 

78.23 R. Lawson, G.W. Humphreys: View-specific effects 
of depth rotation and foreshortening on the ini¬ 
tial recognition and priming of familiar objects, 
Percep. Psychophys. 60 , 1052-1066 (1998) 

78.24 E. Ashbridge, D.l. Perrett: Generalizing across ob¬ 
ject orientation and size. In: Perceptual Con¬ 
stancy. Why Things Look as They Do, ed. by 
V. Walsh, J. Kulikowski (Cambridge Univ. Press, 
Cambridge 1998) pp. 192-209 

78.25 M. Dill, S. Edelman: Imperfect invariance to ob¬ 
ject translation in the discrimination of complex 
shapes, Perception 30, 707-724 (2001) 

78.26 K.R. Cave, S. Pinker, L. Giorgi, C.E. Thomas, 
L.M. Heller, J.M. Wolfe, H. Lin: The representation 
of location in visual images, Cogn. Psychol. 26,1- 
32 (1994) 

78.27 S. Ullman: Aligning pictorial descriptions: An ap¬ 
proach to object recognition, Cognition 32 , 193— 
254 (1989) 

78.28 S. Ullman, R. Basri: Recognition by linear combi¬ 
nations of models, lEEETrans. Pattern Anal. Mach. 
Intell. 13 , 992-1006 (1991) 

78.29 T. Poggio, S. Edelman: A network that learns to 
recognize three-dimensional objects, Nature 343 , 
263-266 (1990) 

78.30 D. Perrett, W.M. Oram: Visual recognition based 
on temporal cortex cells: Viewer-centred process¬ 
ing of pattern configurations, Z. Naturforsch. C 53 , 
518-541 (1998) 

78.31 M. Riesenhuber, T. Poggio: Hierarchical models 
of object recognition in cortex, Nat. Neurosci. 2, 
1019-1025 (1999) 

78.32 E.T. Rolls, T. Milward: A model of invariant ob¬ 
ject recognition in the visual system: Learning 


rules, activation functions, lateral inhibition, and 
information-based performance measures, Neu¬ 
ral Comput. 2(11), 2547-2572 (2000) 

78.33 G. Wallis, H.H. Biilthoff: Learning to recognize ob¬ 
jects, Trends Cogn. Sci. 3, 22-31 (1999) 

78.34 D. Perrett, W.M. Oram, E. Ashbridge: Evidence 
accumulation in cell populations responsive to 
faces: An account of generalization of recogni¬ 
tion without mental transformations, Cognition 
67, 111-145 (1998) 

78.35 D.H. Foster, S.J. Gilson: Recognizing novel three- 
dimensional objects by summing signals from 
parts and views, Proc. R. Soc. B 269, 1939-1947 
( 2002 ) 

78.36 H. Sakata: The role of the parietal cortex in grasp¬ 
ing, Adv. Neurol. 93,121-139 (2003) 

78.37 D.H. Hubei, T.N. Wiesel: Receptive fields, binocu¬ 
lar interaction and functional architecture in the 
cat's visual cortex, J. Physiol. (Lond.) 160,106-154 
(1962) 

78.38 G. Wang, M. Tanifuji, l(. Tanaka: Functional archi¬ 
tecture in monkey inferotemporal cortex revealed 
by in vivo optical imaging, Neurosci. Res. 32, 33- 
46 (1998) 

78.39 K. Tanaka: Representation of visual feature ob¬ 
jects in the inferotemporal cortex, Neural Netw. 
9(8), 1459-1475 (19 9 6) 

78.40 K. Grili-Spector, R. Malach: The human visual cor¬ 
tex, Annu. Rev. Neurosci. 27, 649-677 (2004) 

78.41 N.K. Logothetis, J. Pauls, H.H. Biilthoff, T. Poggio: 
View-dependent object recognition by monkeys, 
Curr. Biol. 4, 401-414 (1994) 

78.42 L. Roberts: Machine perception of three- 

dimensional solids. In: Optical and 

Electro-Optical Information Processing, ed. 
by J.T. Tippet (MIT, Cambridge 1965) pp. 159-197 

78.43 M. Swain, D. Ballard: Color indexing, Int. J. Com¬ 
put. Vis. 7,11-32 (1991) 

78.44 C. Schmid, R. Mohr: Local greyvalue invariants for 
image retrieval, IEEE Trans. Pattern Mach. Intell. 
19, 530-535 (1997) 

78.45 D. Lowe: Distinctive image features from scale 
invariant keypoints, Int. J. Comput. Vis. 60(2), 90- 
110 (2004) 

78.46 M. Kirby, L. Sirovich: Applications of the 
Karhunen-Loeve procedure for the charac¬ 
terization of human faces, IEEE Trans. Pattern 
Mach. Intell. 12,103-108 (1990) 

78.47 B. Browatzki, V. Tikhanoff, G. Metta, H.H. Biilthoff, 
C. Wallraven: Active in-hand object recognition 
on a humanoid robot, lEEETrans. Robotics 30(5), 
1260-1269 (2014) 

78.48 A. Delorme, S. Thorpe: SpikeNET: An event-driven 
simulation package for modeling large networks 
of spiking neurons, Netw. Comput. Neural Syst. 14, 
613-627 (2003) 

78.49 Y. LeCun, F. Huang, L. Bottou: Learning methods 
for generic object recognition with invariance to 
pose and lighting, Proc. 2004 IEEE Comput. Soc. 
Conf. Comput. Vis. Pattern Recogn. (2004) 

78.50 T. Serre, L. Wolf, T. Poggio: Object recognition with 
features inspired by visual cortex, Proc. 2005 IEEE 



Perceptual Robotics 


References 2111 


Comput. Soc. Conf. Comput. Vis. Pattern Recogn. 
(2005) 

78.51 S. Thorpe, D. Fize, C. Marlot: Speed of process¬ 
ing in the human visual system, Nature 381(6582), 
520-522 (1996) 

78.52 J.J. DiCarlo, D. Zoccolan, N.C. Rust: How does 
the brain solve visual object recognition?, Neu¬ 
ron 73(3), 415-434 (2012) 

78.53 E.T. Rolls: Invariant visual object and face recog¬ 
nition: Neural and computational bases, and 
a model, VisNet. Front. Comput. Neurosci. 6(35), 
( 2012 ) 

78.54 N. Pinto, D. Cox: High-throughput-derived 
biologically-inspired features for unconstrained 
face recognition, Image Vis. Comput. 30(3), 
159-168 (2012) 

78.55 6. Wallis, H.H. Biilthoff: Effects of temporal asso¬ 
ciation on recognition memory, Proc. Natl. Acad. 
Sci. USA 98, 4800-4804 (2001) 

78.56 J.V. Stone: Object recognition using spatio- 
temporal signatures, Vis. Res. 38(7), 947- 951 
(1998) 

78.57 J.V. Stone: Object recognition: View-specificity 
and motion-specificity, Vis. Res. 39(24), 4032- 
4044 (1999) 

78.58 Q.C. Voung, M.J. Tarr: Rotation direction af¬ 
fects object recognition, Vis. Res. 44(14), 1717-1730 
(2004) 

78.59 C. Wallraven, H.H. Biilthoff: Automatic acquisition 
of exemplar-based representations for recogni¬ 
tion from image sequences, CVPR 2001 - Work¬ 
shop Models vs. Ex. (2001) 

78.60 C. Wallraven, H.H. Biilthoff: Object recognition 
in humans and machines. In: Object Recogni¬ 
tion, Attention and Action, ed. by N. Osaka, 
I. Rentschler, I. Biederman (Springer, Tokyo 2007) 
pp. 89-104 

78.61 C. Wallraven, B. Caputo, A.B.A. Graf: Recognition 
with local features: The kernel recipe, Proc. Int. 
Conf. Comput. Vis., Vol. 2 (2003) pp. 257-264 

78.62 T.C. Kietzmann, S. Lange, M. Riedmiller: Compu¬ 
tational object recognition: A biologically moti¬ 
vated approach, Biol. Cybern. 100, 59-79 (2009) 

78.63 F.N. Newell, M.O. Ernst, B.S. Tjan, H.H. Biilthoff: 
Viewpoint dependence in visual and haptic ob¬ 
ject recognition, Psychol. Sci. 12, 37-42 (2001) 

78.64 H. Lee, C. Wallraven: Exploiting object constancy: 
Effects of active exploration and shape morph¬ 
ing on similarity judgments of novel objects, Exp. 
Brain Res. 225(2), 277-289 (2012) 

78.65 J.-P. Ewert: Neural mechanisms of prey-catching 
and avoidance behavior in the toad Bufo bufo L, 
Brain Behav. Evol. 3, 36-56 (1970) 

78.66 G. Johansson: Visual perception of biological mo¬ 
tion and a model for its analysis, Percept. Psy- 
chophys. 14, 201-211 (1973) 

78.67 K. Verfaillie: Perceiving human locomotion: Prim¬ 
ing effects in direction discrimination, Brain 
Cogn. 44, 192-213 (2000) 

78.68 A.J. O'Toole, D.A. Roark, H.H. Abdi: Recognizing 
moving faces: A psychological and neural synthe¬ 
sis, Trends Cogn. Sci. 6, 261-266 (2002) 


78.69 D. Perrett, A. Puce: Electrophysiology and brain 
imaging of biological motion, Philos. Trans. R. 
Soc. B 358, 435-445 (2003) 

78.70 R. Blake, M. Shiffrar: Perception of human mo¬ 
tion, Annu. Rev. Psychol. 58, 47-73 (2007) 

78.71 D.D. Hoffman, B.E. Flinchbaugh: The interpreta¬ 
tion of biological motion, Biol. Cybern. 42, 195— 
204 (1982) 

78.72 J.A. Webb, J.K. Aggarwal: Structure from motion of 
rigid and jointed objects, Artif. Intell. 19 , 107-130 
(1982) 

78.73 M.A. Giese, T.T. Poggio: Neural mechanisms for 
the recognition of biological movements, Nat. 
Rev. Neurosci. 4, 179-192 (2003) 

78.74 J. Jastorff, G.A. Orban: Human functional mag¬ 
netic resonance imaging reveals separation and 
integration of shape and motion cues in bi¬ 
ological motion processing, J. Neurosci. 29(22), 
7315-7329 (2009) 

78.75 G. Rizzolatti, L. Craighero: The mirror-neuron sys¬ 
tem, Annu. Rev. Neurosci. 27,169-192 (2004) 

78.76 V. Caggiano, L. Fogassi, G. Rizzolatti, J.K. Pomper, 
P. Thier, M.A. Giese, A. Casile: View-based en¬ 
coding of actions in mirror neurons of area f5 in 
macaque premotor cortex, Curr. Biol. 21 ( 2 ), 144- 
148 (2011) 

78.77 F. Simion, E. Di Giorgio, I. Leo, L. Bardi: The pro¬ 
cessing of social stimuli in early infancy: From 
faces to biological motion perception, Prog. Brain 
Res. 189 , 173-193 (2011) 

78.78 E.D. Grossman, R. Blake, C.Y. Kim: Learning to see 
biological motion: Brain activity parallels behav¬ 
ior, J. Cogn. Neurosci. 16 , 1669-1679 (2004) 

78.79 J. Jastorff, Z. Kourtzi, M.A. Giese: Learning to dis¬ 
criminate complex movements: Biological versus 
artificial trajectories, J. Vis. 6, 791-804 (2006) 

78.80 J. Jastorff, Z. Kourtzi, M.A. Giese: Visual learn¬ 
ing shapes the processing of complex movement 
stimuli in the human brain, J. Neurosci. 29(44), 
14026-14038 (2009) 

78.81 H. Hill, F.E. Pollick: Exaggerating temporal differ¬ 
ences enhances recognition of individuals from 
point light displays, Psychol. Sci. 11 , 223-228 
( 2000 ) 

78.82 B. Knappmeyer, I.M. Thornton, H.H. Biilthoff: The 
use of facial motion and facial form during the 
processing of identity, Vis. Res. 43, 1921-1936 
(2003) 

78.83 J. Lee, W. Wong: A stochastic model of coherent 
motion detection, Biol. Cybern. 91 , 306-314 (2004) 

78.84 J. Lange, M. Lappe: A model of biological motion 
perception from configural form cues, J. Neurosci. 
26(11), 2894-2906 (2006) 

78.85 G. Tessitore, F. Donnarumma, R. Prevete: An 
action-tuned neural network architecture for 
hand pose estimation, Proc. Int. Conf. Fuzzy Com¬ 
put. Int. Conf. Neural Comput. Valencia (2010) 
pp. 358-363 

78.86 G. Metta, G. Sandini, L. Natale, L. Craighero, 
L. Fadiga: Understanding mirror neurons - A bio- 
robotic approach, Interact. Stud. 7, 197-232 
(2006) 


Part G | 78 



Part G | 78 


2112 Part G 


Robots and Humans 


78.87 H. Jhuang, T. Serre, L. Wolf, T. Poggio: A biologi¬ 
cally inspired system for action recognition, IEEE 
Int. Conf. Comput. Vis. (ICCV) (2007) pp. 1-18 

78.88 M.J. Escobar, G.S. Masson, T. Vieville, P. Korn- 
probst: Action recognition using a bio-inspired 
feedforward spiking network, Int. J. Comput. Vis. 
82(3), 284-301 (2009) 

78.89 H. Jhuang, E. Garrote, J. Mutch, T. Poggio, 
A. Steele, T. Serre: Automated home-cage behav¬ 
ioral phenotyping of mice, Nat. Commun. 1(86), 
1-9 (2010) 

78.90 D.M. Wolpert, K. Doya, M. Kawato: A unifying 
computational framework for motor control and 
social interaction, Philos. Trans. R. Soc. B 358, 
593-602 (2003) 

78.91 Y. Demiris, M. Johnson: Distributed, predictive 
perception of actions: A biologically inspired 
robotics architecture for imitation and learning, 
Connect. Sci. 15(4), 231-243 (2003) 

78.92 J.M. Kilner, K.J. Friston, C.D. Frith: The mirror- 
neuron system: A Bayesian perspective, Neurore¬ 
port 18, 619-623 (2007) 

78.93 K. Friston, J. Mattout, J. Kilner: Action un¬ 
derstanding and active inference, Biol. Cybern. 
104(1/2), 137-160 (2011) 

78.94 R. Li, T.P. Tian, S. Sclaroff, M.H. Yang: 3D human 
motion tracking with a coordinated mixture of 
factor analyzers, Int. J. Comput. Vis. 87, 170-190 
( 2010 ) 

78.95 D.R. Weinland, R. Ronfard, E. Boyer: A survey of 
vision-based methods for action representation. 
Segmentation and recognition, Comput. Vis. Im¬ 
age Underst. 115(2), 224-241 (2011) 

78.96 F. Fleischer, V. Caggiano, P. Thier, M.A. Giese: 
Physiologically inspired model for the visual 
recognition of transitive hand actions, J. Neurosci. 
33, 6563-6580 (2013) 

78.97 E. Salinas, L.F. Abbott: Transfer of coded informa¬ 
tion from sensory to motor networks, J. Neurosci. 
15 , 6461-6474 (1995) 

78.98 A. Karpathy, G. Toderici, S. Shetty, T. Leung, 
R. Sukthankar, L. Fei-Fei: Large-scale video clas¬ 
sification with convolutional neural networks, 
Proc. 2014 IEEE Conf. Computer Vision and Pat¬ 
tern Recognition, New York (2014) pp. 1725— 
1732 

78.99 C. Breazeal: Designing Sociable Robots (MIT Press, 
Cambridge 2002) 

78.100 M. Nusseck, D.W. Cunningham, C. Wallraven, 
H.H. BLilthoff: The contribution of different facial 
regions to the recognition of conversational ex¬ 
pressions, J. Vis. 8(8), 1 (2008) 

78.101 J.K. Lee, C. Breazeal: Human social response to¬ 
ward humanoid robot's head and facial features, 
Proc. CHI 2010 (2010) pp. 4237-4242 

78.102 D. Hanson: Exploring the aesthetic range for hu¬ 
manoid robots, CogSci-2006 Workshop: Toward 
Soc. Mech. Android Sci. (2006) 

78.103 H. Ishiguro: Understanding humans by building 
androids, Proc. SIGDIAL Conf. (2010) 

78.104 P. Jaeckel, N. Campbell, C. Melhuish: Facial be¬ 
haviour mapping - From video footage to a 


robot head, Robotics Auton. Syst. 56(12), 1042— 
1049 (2008) 

.105 G. Metta, G. Sandini, D. Vernon, L. Nataie, F. Nori: 
The iCub humanoid robot: An open platform for 
research in embodied cognition, Proc. 8th Work¬ 
shop Perform. Metr. Intell. Syst. (2008) pp. 50-56 

.106 C. Wallraven, M. Breidt, D.W. Cunningham, 
H.H. Bulthoff: Evaluating the perceptual realism 
of animated facial expressions, ACM Trans. Appl. 
Percept. 4(4), 1-20 (2008) 

.107 K.F. MacDorman: Subjective ratings of robot video 
clips for human likeness, familiarity, and eeri¬ 
ness: An exploration of the uncanny valley, 
ICCS/CogSci-2006 Symp. Toward Soc. Mech. An¬ 
droid Sci. (2006) pp. 26-29 

.108 C. Ho, K.F. MacDorman, Z.A.D. Pramono: Human 
emotion andthe uncanny valley: AGLM, MDS, and 
isomap analysis of robot video ratings, Proc. HRI 
2008 (2008) pp. 169-176 

.109 F. Delaunay, J. de Greeff, T. Belpaeme: Towards 
retro-projected robot faces: An alternative to 
mechatronic and android faces, Proc. 18th IEEE 
Int. Symp. Robot Human Interact. Commun. RO¬ 
MAN (2009) pp. 306-311 

.110 T. Kuratate, M. Riley, B. Pierce, G. Cheng: Gender 
identification bias induced with texture images 
on a life size retro-projected face screen, Proc. 
21st IEEE Int. Symp. Robot Human Interact. Com¬ 
mun. R0-MAN 2012 (2012) pp. 43-48 

.111 A.P. Saygin, T. Chaminade, H. Ishiguro, J. Driver, 
C. Frith: The thing that should not be: Predic¬ 
tive coding and the uncanny valley in perceiving 
human and humanoid robot actions, Soc. Cogn. 
Affect. Neurosci. 7(4), 413-422 (2012) 

.112 P.S.A. Reitsma, N.S. Pollard: Perceptual metrics 
for character animation: Sensitivity to errors in 
ballistic motion, ACM SIGGRAPH 2003 Papers (SIG- 
GRAPH '03) (ACM, New York 2003) pp. 537-542 

.113 T. Ezzat, G. Geiger, T. Poggio: Trainable videore- 
alistic speech animation, Proc. 29th Annu. Conf. 
Comput. Gr. Interact. Techn. (SIGGRAPH '02) (ACM, 
New York 2002) pp. 388-398 

.114 J. Wang, B. Bodenheimer: Synthesis and eval¬ 
uation of linear motion transitions, ACM Trans. 
Graph. 27(1), Article 1 (2008) 

.115 M. Candidi, C. Urgesi, S. lonta, S.M. Aglioti: Virtual 
lesion of ventral premotor cortex impairs visual 
perception of biomechanically possible but not 
impossible actions, Soc. Neurosci. 3(3/4), 388-400 
(2008) 

.116 T. Chaminade, J. Hodgins, M. Kawato: Anthro¬ 
pomorphism influences perception of computer- 
animated characters' actions, Soc. Cogn. Affect. 
Neurosci. 2(3), 206-216 (2007) 

.117 T. Chaminade, M. Zecca, S.J. Blakemore, A. Takan- 
ishi, C.D. Frith, S. Micera, P. Dario, G. Rizzolatti, 
V. Gallese, M.A. Umilta: Brain response to a hu¬ 
manoid robot in areas implicated in the percep¬ 
tion of human emotional gestures, PLoS ONE 5(7), 
ell577 (2010) 

.118 Y.F. Tai, C. Scherfler, D.J. Brooks, N. Sawamoto, 
U. Castiello: The human premotor cortex is 'mir- 


78, 

78, 

78, 

78, 

78, 

78, 

78, 

78, 

78, 

78, 

78, 

78, 

78, 

78, 



Perceptual Robotics I References 2113 


ror' only for biological actions, Curr. Biol. 14, 
117-120 (2004) 

78.119 L.M. Oberman, J.P. McCleery, V.S. Ramachan- 
dran, J.A. Pineda: EE6 evidence for mirror 
neuron activity during the observation 
of human and robot actions: Toward an 
analysis of the human qualities of inter¬ 
active robots, Neurocomput. 70, 2194-2203 

(2007) 


78.120 C. Press, H. Gillmeister, C. Heyes: Sensorimotor ex¬ 
perience enhances automatic imitation of robotic 
action, Proc. Biol. Sci. 274(1625), 2509-2514 (2007) 

78.121 C.L. Colby: Action-oriented spatial reference 
frames in cortex, Neuron 20, 15-24 (1998) 

78.122 A.R. Kilgour, R. Kitada, P. Servos, T.W. James, 
S.J. Lederman: Haptic face identification acti¬ 
vates ventral occipital and temporal areas: An 
fMRI study, Brain Cogn. 59, 246-257 (2005) 


Part G | 78 



2197 


Index 


2- D flow 596 

3- D computer aided design model 
788 

3-D point cloud data 1189 
3-D printer 539 
3-D query point 749 
3-D reconstruction 626 
3-tiered (3T) 337 
6 DOF localization 987 

A 


a three layer architecture for 
navigating through intricate 
situations (ATLANTIS) 280 
AAAI Robot Challenge 288 
abstraction layer 1116 
acceleration-resolved control 197, 
207 

accelerometer 1014 
access and economics 1577 
accuracy analysis 431 
ACM-R snake robots 464 
Acorn RISC machine architecture 
(ARM) 577 
acoustic 

-baseline 1212,1216 

- communication 577 

- imaging 574 

- modem 576, 578 

- positioning system 575 

- sensor 574 

acoustic Doppler current profiler 
(ADCP) 574 

Acrobot surgical robot 1532, 1533, 
1536 

acrylonitrile-butadiene-styrene 

567 

activation behavior-based 305, 
307-309 

activities for daily living 1552, 
1718, 1781 

activity theory 1754 
actuating system 1213 
actuation 1010 

- architecture 451 

- net 452 


actuator 539 

- dynamics 605 

- effort sensor 677 

- electromagnetic 78 

- for manipulation 647 

- hydraulic 80 

- placement 449 

- pneumatic 80 

- vibrotactile 1014 

- with mechanically adjustable series 
compliance (AMASC) 487 

AdaBoost 785 
added mass 1207 
additive manufacturing 539 
adjacency pair 1752 
adjustable pattern generator (APG) 
1896 

admissible configuration 855 
admittance 1007 

- control 193, 199, 200 

- matrix 861 

adversarial environments 1282 
aerial 

- based precision agriculture 1382 

- robot 623, 625 

- robot control 1234 

- robot guidance 1234 

- robotic emerging application 625 

- robotics 593, 1225 

- vehicle 1485 
aerodynamic 

- center (AC) 607 

- drag 613 

- efficiency 606 

- flight mechanics 595 

- force 597, 606 

- mechanism 617 

- moment 606 

Aesop surgical robot 1530 
affordance 

- detection 971 

- learning 967 
agent remote 281 
agile 

- manufacturing 1301 

- robot development network 
(aRDnet) 292 


agricultural robotics 1365 
agriculture 626, 1366 
agriculture and forestry (A&F) 

1365 

aircraft dynamics 603 
airfoil 596 
airplane 606 

- static performance 606 

- static stability 606 
algorithmic singularity 229 
altimeter 576 

ambient assisted living (AAL) 1781 
amplitude shift keying (ASK) 578 
analytic Jacobian 219 
anatomically correct testbed (ACT) 
1649 

android 1745 

- robot 1767 

angle of attack (AOA) 606, 1230 

- matrix 1230 

angular momentum 1127,1143 

animacy 1742 

animal detection 1513 

annealed particle filter (APF) 988 

anthropomorphism 448, 1743, 1766 

antipersonnel 647, 1413 

anti-tank mine 1414 

anti-vehicle 1414 

arable farming 1369 

arbitration 279 

arc length 468 

architecture 279, 300-302, 1255 

- behavior-based 280, 310, 311 

- component 282 

- design 290 

- implementation 291 

- layered control 280 

- subsumption 280 

- three-layer 301, 302 
Arduino 1939 

area correlation stereo 744 
arm guide 1561 
articulated body 54 
articulated object 791, 993 
articulated robot 555 
articulate-type modular robot 515 
artifact 717 


Index 




Index 


2198 Index 


artificial 

- bacterial flagella (ABF) 650 

- constraint 205 
-evolution 1871,1879 
-hand 1851 

- intelligence (AI) 2, 8, 282, 1064, 
1494, 1646, 1952 

- intelligence system (AIS) 282 

- muscle 486 

- muscle system 482 

- neural network (ANN) 789, 

1823, 1872 

artificial intelligence (AI) 319, 
1768 

ask-back 1776 
assembled state 860 
assembly 847 

- incidence matrix (AIM) 514 

- line 1306 

- motion 859 
-process 1309 
assistive 

- interaction dynamic 1804 

- rehabilitation robot 1568 

- robot 1567 

- robot service manipulator (ARM) 
1569 

- technology 1006 
atmosphere 596 
attention 

- focus 1771 

- monitoring behavior 1770 

- shared 1786 
attitude 

- and heading reference system 
573, 699 

- observer 1246 
attitude control 

- free-floating robot 1334,1347 

- reaction wheel 1347 
attractor field 1673 
augmentation 1575 
augmented Jacobian 229 
autism 1747, 1933 
automated 

- brick alignment 1397 

- brick laying 1395 

- bus rapid transit (ABRT) 1520 

- guided vehicle (AGV) 1303 

- highway 1520 

- highway system (AHS) 1502 

- reasoning 320 

- reinforcement 1397 


- synthesis of multirobot task 
solutions through software 
reconfiguration (ASyMTRe) 

1275 

-vehicle 1518 

- welding robot 1397 
automatic 

- constructions building system 
(ACBS) 1404 

- speech recognition (ASR) 1905 

- spoken-language recognition 
(ASR) 1775 

- target recognition 1483, 1489 
autonomous 620 

- aquatic vehicle (AUV) 1036 

- combat flying vehicle (ACFV) 
1967 

- digging 1439 

- dozer 1439 

- excavation 1438 

- exploration 993 

- flight 621 

- flying vehicle (AFV) 1967 

- guided vehicle (AGV) 403 

- mental development (AMD) 

1649 

- orchard vehicle 1373 

- robot architecture (AuRA) 280, 
302 

- robotics research 1618 

- space transport robotic operations 
(ASTRO) 1335 

- subtrack control 1198 

- tramming 1445 

- underwater vehicle (AUV) 565, 
566, 1204, 1466, 1485, 1967 

- vehicle 1417, 1427, 1445 
autonomous system 1450 

- in mining 1450 
autopilot 610 

auxiliary surgical support robots 
1530 

average landmark vector 1850 
average-reward criterion 362 
aware home 1612 

B 


baby schema 1743 
back-channel 1752 
- response 1776 
back-drivability 415, 1004, 1007 
backstepping control 258 


backward Euler algorithm 183 

bag-of-word 780 

balancer 435 

ballast 568 

bang-bang-control 144 

baseball playing robot 1603 

basic level object class 779 

BASIC stamp 1939 

basis behavior 1269 

basis of object representation 1915 

basket cell 1895 

batch estimation 95 

bathymetric survey 1435 

battery 569 

Battlebots 1935 

Baumgarte stabilization 60 

Bayes rule 325, 820 

Bayesian 

- belief 983 

- filter 985 

- framework 983 

- inference 985 

- network 324 

- prior 985 

- probability 324 

- recursion 985 

- state estimation 983 
Bayesian dynamic network 325 
beam 

- bending 260 

- sonar pattern 712 

- torsion 260 
bearing estimation 722 
bee and robot 1889 
behavior 279 

- adaptive 1873 

- control 284 

- homeostasis 1769 

- language 284 

- mark-up language (BML) 1752 

- primitive (BP) 308 
behavior-based 303 

- activation 302, 313 

- basis behavior 304 

- coordination 303 

- fuzzy system 310 

- history of behavior use 308 

- interaction dynamics 301, 303, 
305-307 

- learning 306 

- misconception 303 

- multi-robot system 305-307, 310 

- planning 304 

- principles 302 




Index 2199 


- reasoning 304 

- reinforcement learning 306 

- representation 304 

- robotics 279 

- system 299 

- vision system 311 
belief 1753 

- roughness 985 
-set 1751 

- space 990 

- space search tree 990 

- state 988 
-system 1751 
Bellman 

- equation 326 

- principle of optimality 363 
bend propagation 536 
Bernoulli equation 597 
best practice in robotics 293 
best-first-planner 1657 
bi-articular muscle 534 
bigram model 1673 
bilateral control 1024, 1335 
bi-manu-track 1561 
bin-picking 1307 
bio-inspired 

- actuation 1847 
-climbing 1850 
-robotics 1842 
-soft robot 1851 
biological 

-leg 1847 
-model 1844 

- principle 525 

- society 1267 

- system 499, 1844 
biomimetic 731 

- gait 526 

- robot 524 

bionic handling assistant 467 
biped robot 1846,1860 
blind bulldozing 1280 
blind-spot detection 1513 
Bode 

- diagram 248 

- plot 457 

body extender 1726 
body pose 785 
body-affect mapping 1770 
boosting 782 
botball 1934 
boundary layer 171, 597 
bounding sphere 1011 
box pushing 1280 


brachistochrone OC problem 501 
bracing strategy 260 
brain imaging 1563 
brain-computer interface 1563 
brickwork 

- component 1395 

- plant 1396 

- positioning 1397 
broad particle 987 
broadcast 283 

- of local eligibility (BLE) 310, 
1277 

Brockett’s theorem 1145,1169 
browser 1049 

Brunovsky canonical form 169 
brush tire model 558 
brushless DC electric motor 78 
buckling 533 

- gripper 536 

building component recycling 1404 
bulldozing resistance 1353 
bundle adjustment 770 
buoyancy 567, 1209 

- center of 1209 

bus rapid transit 1520 

c 


cable-driven parallel robot 240, 428 
Canadian Scientific Submersile 
Facility (CSSF) 1204 
CAN-bus 514 

capacitive pressure sensing array 
678 

capstan 1009, 1010 
capture 

- and berting 1334 
-point (CP) 1134 

car body painting 1328 
carbon fiber 537, 1328 

- reinforced plastic 567 

- reinforced prepreg 536 
carbon nanotube 485, 652 
cardinal direction calculus 331 
car-like robot 553 

carrot heading 1194 
Cartesian 

- impedance control 494 

- manipulator 69 

- space stiffness matrix 397 

- stiffness 495 

- stiffness control 494 


cascaded control 
-architecture 611 

- structure 495, 622 
caster wheel 549, 550 
caterpillar 534, 1436, 1446 
ceiling 

- panel placement robot 1400 

- vision SLAM 1609 
cellular robotic system 513 
Center for Robot-Assisted Search 

and Rescue 1468 
center of 

- buoyancy 1209 

- compliance 195 

- gravity 606, 1195, 1206, 1650 

- mass 267, 414, 419, 995, 1127, 
1556, 1656 

- pressure (COP) 1128 

- rotation 893 

- stiffness 195 

center of rotation (COR) 883 
central pattern generation 528, 

1143 

centrifugal clutch assembly 1311 
centroid 

- contact 1015 

- moment pivot 414 
ceramic matrix composite 485 
cerebellar 

- control 1895 
-model 1898 

- model articulation controller 370, 
1896 

cerebellum 1887 
cerebral palsy 1552 
chain 

- kinematic topology 388 
-reconfiguration 517 
chained-form system 1106,1163 
charge-coupled detector 93 
charge-coupled device 637, 674, 

1376, 1867 

Chasles’ theorem 17 
chattering 170 
Chebychev-Griibler-Kutzbach 
formula 429 
chemical spill 1460 
chemical vapor deposition 639 
Christoffel symbol 46 
circuit switching 1046 
CKbot 516 
clamping 1683 
class detection 786 
classical mobility formula 429 


Index 




Index 


2200 Index 


classification of customers 1594 
classifier 782 
Claytronics project 1272 
cleaning 

- robot 1402, 1592 

- technology 1595 
client/server 278, 1047 
cliff sensor 1607 
climb factor 606 
climbing fiber 1895 
clinical 

- testing 1562 

- use 1564 
closed 

- chain 427 

- world assumption 327 
closed kinematic 

- chain 145 

- loop 57, 114 
closed-loop 122 

- dynamic 499 

- execution and recovery 282 

- inverse kinematics 226 
cluster bomb 1414 
clustering 101,781 
clutching 1029 
co-activation stiffness 490 
coarse-acquisition 701 
C-obstacle 857 
cockroach inspired robot 525 
codebook 780 
coefficient 597 

- of restitution 1350 
cognition 299 

- embodied 1747 

- social 1747 
cognitive 

- aid 1572 

- development 1746 

- human-robot interaction 1741 

- model 1744 

- model of robot control 1757 

- system 1741 
cognizant failure 285 
collaboration and teamwork 1772 
collaborative control 1749 
collaborative interaction 1681 
collective 1254, 1268 

- robot construction 1280 

- transport 1280 
collision 1011 

- avoidance 1046 

- avoidance technology 1443 

- detection 1010, 1046, 1694 


- isolation 1694 
-mitigation 1512 

- modeling 1683 
-reaction 1694 
-warning 1512 

color camera with depth 998 
command shaping 265 
commercial off-the-shelf 832 
common 

- gateway interface 1047 

- ground 1749, 1786 

- object request broker architecture 
279, 1056 

- objects in context (COCO) 791 
communication 622, 1257 

- access for land mobiles 1508 

- for control 1264 

- for perception 1264 

- graph 1263 

- intelligence 1488 

- interface 518 
-protocol 518 

- skill 1775 

- system 1214 

- theory 1674 
compass 1212 

- gait walking 1846 
competing behavior 285 
competition 1617 

- and cooperation 1876 

- challenge 1619 

- scenario 1619 
complementarity condition 1130 
complementarity problem 887 
complementary 

metal-oxide-semiconductor 674, 
715, 1539, 1607 
compliance 

- active 195 

- center of 195 

- control 193, 252 

- frame 193, 205 

- in grasping 954 

- matrix 195, 204 
compliant 

- contact 880 

- environment 204 

- fin 532 

- grasping 956 

- hand 955 

- humanoid platform 1689 
-material 531 
compliant motion 

- active 862 


- execution 865 

- planning 862 
component 
-modeling 1211 

- production 1395 
composite material 567 
composite-rigid-body algorithm 39, 

55 

compression 260 

- criterion 1685 
computational 

- adequacy 321 

- fluid dynamics 602,1211 

- model 1901 

- neuroethology 1869 
computed tomography 1528, 1537 
computed-torque 255 

- control 170, 173 
computer ethic 1957 
computer graphics 1654 
computer integrated construction 

1400 

computer numerical control 513, 
1302, 1305, 1394 
computer-aided 

- design 90, 788, 984, 1006, 1307, 
1318, 1392, 1527, 1671 

- drafting 1053 

- engineering 388 
-manufacturing 1313,1395,1527 

- remote driving 1339 

- tomography 1965 
computer-aided design (CAD) 

- model 1012 

computer-integrated surgery 1528 
computional brain 416 
concentric tube manipulators 467 
conceptual design 408 
concrete 

- distribution 1398 

- finishing 1400 

- leveling robot 1402 

- production 1395 

- pumping car 1469 

- spraying robot 1399 
concurrent execution 278 

- mapping and localization 1216 

- reactive plan 337 
condition number 124, 126, 221 
conditional factor graph 965 
conditional random field 99, 965 
conditioning kinematic 1007 
confidence measure 125 



Index 2201 


configuration 859 
-space 71,136,858,1127 

- space manipulation 854 

- space obstacle 857 

- space topology 856 
connection 

- event 1752 

- system 1405 
connectivity graph 47 
conservative congruence 

transformation 435 
constant curvature model 471 
constellation model 783 
constrained 

- modes of a joint 21 

- motion 192 
constraint 

- (selection) matrix 885 

- artificial 205 

- holonomic 25 

- Jacobian 202 

- kinematic 192, 201 

- natural 204 

- nonholonomic 25 

- satisfaction problem (CSP) 329 

- task Jacobian 229 
construction 1392 

- automation 1389 

- elementary process 1391 

- industry 1390 
-machine 1394 
-on-site 1399 
-phase 1391 
-process 1391 
-project 1390 
-robot 1390 

- robotics 1392 

- robotics categories of 1394 

- site 1403 

- stakeholder 1391 
contact 1015 

- angle of track 1196 
-centroid 1015 

- compliant 880 

- configuration 860 

- display 1015 

- dynamics 1349 

- estimation 994 

- formation 863 

- interface 879, 889 

- kinematics 880 

- location sensor 678 

- manifold 988 

- mode 882 


- model 879 

- sensor 1607 

- stiffness 895 

- vibration 1014 

- virtual manipulator 206 

- wrench sum 1658 
contact state 

- graph 863 

- identification 864 

- principal contact 860 
contextual inquiry 1943 
continuous 

- activity scheduling, planning, 
execution and replanning 282 

- hidden Markov model 1700 

- operating reference station 704 

- path 1303 

- time 1013 

- variable transmission 485 
continuous-transmission frequency 

modulation 709, 723 

- application 725 

- range discrimination and resolution 
724 

- sonar 723 

- transmission coding 723 
continuum 

-Jacobian 471 

- kinematics 470 

- limbed robot 468 

- manipulation 858 

- manipulator 466, 858 

- robot modeling 463 
contract net protocol 1277 
contraction mapping 252 
control 957, 1214, 1446 

- acceleration-resolved 195 

- adaptive 173, 258 

- admittance 193 

- basis 853 

- biomimetic 1143 

- command interpreter 834 

- compliance 193 

- damping 193 

- decentralized 163 
-feedback 1158 

- for perception 1265 
-force 1007,1146 

- impedance 193 

- in mobile manipulation 960 

- interaction 192 

- law 499 

- motion 160, 1317 

- multiobjective 852 


- null-space projection 852 

- of nonholonomic systems 1163 

- operational space 850 

- optimal 239, 1137 

- path-tracking 1446 

- position and force 851 

- redundant manipulator 851 

- robot hand 459 
-robust 1146 

- shared 1028 

- stiffness 193 

- supervisory 1027 

- synthesis 1270 

- system 143 

- task 1828 

- task-level 850 

- unit 647 

- volume analysis 596 
controllability 

- capturability 1133 

- capture point 1133 

- small-space 1142 
controlled 

- floor-by-floor deconstruction 
1404 

- oscillatory dynamics 498 
controller 

- area network 411,576,841 

- optimization 1675 
controlling robot 1188 
convergence tracking error 176 
convolution 789 

- neural network 783, 789 
cooperative 1254 

- adaptive cruise control 1505 

- intelligent real-time control 
architecture 282 

-interaction 1681 

- manipulation 1280 

- manipulation control 869 

- manipulation planning 870 

- manipulator 933 

- multiarm system 943 

- multirobot observation of multiple 
moving target 1281 

- task space 939 

- vehicle infrastructure system 
1508 

coordinate 

- absolute frame 939 

- generalized 25 

- measurement machine 982 

- spatial transform 41 


Index 



Index 


2202 Index 


coordination 1258 

- behavior-based 303 

- decentralized 1260 

Coriolis and centrifugal force 160 

correspondence 788 

cost 

- function 497 

- of transport 1720 

- pressure 1594 
Coulomb 1011 
-friction 462,1129 
counter-rotating thruster 572 
counterweight 435 
coupling 

- momentum 1348 

- stiffness 195 
-virtual 1011,1012 
coverage 1266, 1369 

- configuration protocol 1268 

- strategy 1602 
CPG-based controller 1848 
Cramer-Rao lower bound 1264 
crane automation 1379 
crank-rocker 529 

credit assignment 1278 

- problem 1897 
crop 

- production cycle 1366 

- yield estimation 1372 
cross-sensor acoustic interference 

576 

crowd-sourcing 790 
crush depth 567 
C-space 136 
cue 1744 

- nonverbal 1746 
cultural model 1744 
curvature 469 

- velocity method 1112 
curvilinear abscissa 1161 
customer expectation 1595 
cutaneous (tactile) 1004 
cyber physical system 1304 
cyclicity 232 
cylindrical 

- algebraic decomposition 150 

- joint 22 

- shaped pressure vessel 567 

D 


da Vinci surgical robot 1530, 
1538-1540 


damaged building 1482 
damped least squares 125 
damper 485 
damping 1009 

- control 193 

- factor 224 

- ratio 199 
DARPA 

-challenge 1631 

- grand challenge 1619,1633 

- Robotics Challenge 1687 

- Software for Distributed Robotics 
1262 

- Urban Challenge 1633 
data 

- association 101 

- communication 1426 

- distribution service (DDS) 283 

- gathering 993 

- processing problem 1493 
datcom 1208 

Davis-Putnam algorithm 323 
DC motor 79, 163 
deactivation and decommissioning 
1421 

dead reckoning 1450 
decentralized data fusion 836 
decision-theoretic model 1754 
decontamination 1469 
decoupled architecture 389 
decoupled robot 434 
decoupling control 254 
dedicated short-range 

communication 1507, 1521 
deep 

- belief network 1905 

- reactive ion etching 642, 649 

- sea remotely operated vehicle 
571 

Defense Sciences Office 1573 
deformable 

- object 994 

- part model 780 
-surface 1188 

- terrain 556 

- tire 557 
degree 

- of freedom (DOF) 23, 39, 69, 

115, 195, 218, 292, 387, 409, 427, 
487, 513, 530, 582, 621, 642, 680, 
699, 756, 844, 886, 905, 958, 982, 
985, 1007, 1029, 1093, 1162, 1197, 
1208, 1332, 1376, 1440, 1561, 
1689, 1784 


degree of 

- freedom (DOF) 465, 621 
-mobility 551 

- steerability 552 

degree of freedom (DOF) 161,854, 
1303, 1649, 1669, 1715 
deictic gesture 1776 
deliberative system 300 
delicate object 995 
delta robot 388, 428, 1306 
demining 1415-1418 
Dempster-Shafer theory 830 
Denavit-Hartenberg 49, 109, 112, 
388, 514, 926, 1312 

- convention 26 

- parameter 112, 388 
Denning ring 727 

dense wave division multiplex 577 
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epistemological adequacy 321 
equation of motion 160 

- impulse 47 

- joint-space 45 

- Lagrangian 46 

- operational-space 45 

- rigid body 43 

- spatial 43 


error 

- convergence 186 

- correction 861 
escapement cam 532 
essential matrix 768 
estimated inertia 174 
estimation 89 

- linear model 95 

- process 94 
ethemet 1046 

ethical issues 1577, 1782 
ethnography 1944 
Euler 

- angle 13, 984, 1205 

- equation 597 

European Conference on Educational 
Robotics 1936 
European Octopus 468 
European robotic arm 1333 
event calculus 324 
evidential reasoning 830 
evolution 
-artificial 1871 

- Hebbian learning 1869 

- learning rule 1875-1877 
-of learning 1862 

- robotics software platform 580 
evolutionary 

-algorithm 1857,1871,1880 

- biology 1846 
-robotics 1851,1862 
evolvable hardware 1879 
evolving morphologies for 

human-robot symbiotic interaction 
1565 

excavator 1421 
execution 

- monitoring 287 

- support language 279 
exoskeleton 1004 

- based therapy robot 1553 

- for human performance 
augmentation 1724 

expandable polystyrene 568 
expanding role 1203 
expectation maximization 94,751, 
781, 1069, 1674, 1898 
exploratory procedure 1005, 1006 
explosive 

- disposal 1025 

- ordnance disposal 1411,1412 

- vapor detector 1416 


extensible 

- hyper text markup language 1056 

- markup language 283, 1056, 
1452, 1754 

external force 936 
external velocity manipulability 
ellipsoid 942 
exteroception 89 
extrapolated center of mass 1134 
extrastriate body part area 1921 
extravehicular activity 1027, 1332, 
1718 

extreme locomotion 524 

F 


fabrication method 538 
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- sensitivity norm 420 
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- sensor 576 
HeartLander 1539 
Heathkit Hero-1 1936 

heavy payload manipulator 1400 
helical joint 22 
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- fuel/battery 570 

- position/force control 944 

- robot 556 

- system 301, 310 

- system manipulation planning 
854 

Hybrid III dummy 1689 
hydrodynamic 1207 

- damping 1208 

- modeling 1210 
hyper-redundant 

- mechanism 464 
-robot 218 

- structure 468 

I 


ICP algorithm 747 
identifiability 122 
identifier 996 
ideomotor principle 1903 
illumination scenario 1511 
illustrator cue 1776 
image 

- acquisition 624 


- gradient 786 

- pyramid 782 
image-based visual servo 797 

- control 796 

- cylindrical coordinates 801 

- interaction matrix 798 

- stability analysis 800 

- stereo cameras 801 
ImageNet 789 
imitation 1901 
iMobot 516 
impact 47 

- dynamics 1349 

- scenario 1681 
impedance 488 

- control 193, 198, 239, 258, 946, 
1335,1351 

- controller 495 
-matching 1351 
implicit shape model 784 
importance 

- density 827 

- sampling 98, 985 
improvised explosive device 1414, 

1968 

incremental 

- evolution 1876 

- teaching method 1829 
independent 

- component analysis 1895 

- joint control 163 

- likelihood pool 820 
indexed time table 281 
indirectly actuated state 491 
industrial 

- description 1390 

- manipulator 584 

- robotics 1301 

- standard architecture 576 
inertia 1207 

- articulated-body 54 

- estimation 995 

- matrix 160 

- navigation system 573 

- operational-space 56 

- spatial 42 
inertial 

- measurement unit 92,412,571, 
670,698, 756,813, 1179, 1195, 
1212, 1226, 1246, 1449, 1507, 
1611, 1809 

- navigation 1506 

- navigation system 571, 705, 1443 

- parameter estimation 117 


- sensor 572 

- space 1343 
inference 320, 787, 985 

- Bayesian 325 
inferior olive 1895 
inferotemporal cortex 1899, 1917 
inferring belief 1771 

infinite impulse response 1490 
information 

- and communication technology 
1396, 1941 

- ethics 1957 

- filter 826 
informational 

- support 1772 

information-gathering grasp 991 
infrared 648, 676, 746, 835, 1485, 
1593, 1938 

- detector 1415 
inherent instability 621 
inherently cooperative 1278 
initialization 787 

injury measure 1685 
inner loop 

- control 170 
innovation 626, 824 
input-output-to-state stability 166 
input-to-state stability 166 
insect-inspired robot 1849 
insertion task 861 

in-site actuation 451 
in-situ resource utilization 1452 
instability 1012 
instantaneous 

- allocation 1276 

- center of curvature 694 

- center of rotation 551 

- forward kinematics 31 

- inverse kinematics 32 
instrumental 

- support 1772 
-variable 131 

instrumented logical sensor system 
833 

integral control action 1167 
integrated 

- chip 291 

- circuit 641 

- development environment 1940 

- factory 1390 

- proximity model 984 

- robotized 1393 

- robotized construction site 1389 

- services digital network 1047 







Index 2209 


integrating planning and execution 
288 

integration 620, 1283 
intellectual property right 1612 
intelligent 

- assisting device 1730 

- autonomous system 1714 

- multimode transit system 1520 

- wheelchair system 1570 
intentional 1254 
intention-based model 1753 
interaction 954 

- agent 1753 

- control 192 

- framework 1750 

- modality 1757 

- rhythm 1746 

- unit 1750 
interaction dynamics 

- behavior-based 305-307 
interaction matrix 796 

- approximation 798 

- direct estimation 803 

- image-based visual servo 798 

- pose-based visual servo 804 
interactive perception 970 
interaural 

- amplitude difference 722 

- time difference 722 
interest operator 780 
interface 848 

- contact 889 

- definition language 283 
-design 514 

- for demonstration 1825 

- haptic 1003, 1426 
interference rejection 726 
interferometric fiber-optic gyro 572 
interferometric SAR 1488 
interior finishing robot 1400 
intermediate haptic interaction point 

1011 

internal 

- dynamics 247 

- force 937 

- friction angle 1353 

- velocity manipulation ellipsoid 
943 

International Conference on 
Advanced Robotics 3 
international space station 1027, 
1332 

international submarine engineering 
583 


International Symposium of Robotics 
Research 3 

International Symposium on Micro 
Mechatronics and Human Science 
1936 

internet 1049 

- communication 1034 

- communications engine 283, 580 

- engineering task force 1508 

- protocol 1046, 1508 
interoperability 1392, 1405 
interpenetration 1012 
interphalangeal 1718 
interprocess communication 279, 

289, 579 

intersection operation 429 
interval 

- algebra 328 

- analysis 430 

- calculus 829 

- programming 580 
intrinsic tactile 455 
invariance 202,783 
inverse 

- instantaneous kinematics 32 

- model 348, 349 

- reinforcement learning 1831 

- socially assistive robotics 1488 
inverse differential kinematics 

- least-squares solution 222 

- weighted damped least-squares 
solution 234 

inverse dynamics 51, 197, 208, 

245, 265 

- control 167, 357 

inverse kinematics 29, 430, 857, 
1317 

- algorithm 225 

- computation 1669 

ionic polymer-metal composite 
485, 532 

isomorphic configuration 516 
isotropic configuration 221 
Istituto Italiano di Tecnologia 1689 
i-swarm project in Karlsruhe 1261 
iterated extended Kalman filter 98 
iterative closest point algorithm 94, 
747, 1092, 1449 

iterative linear quadratic regulator 
502 

iteratively reweighted least square 
100 


J 


jackknife effect 1163 
Jacobian 31,47,71,162,851 

- analytic 219 

- condition number 393 

- dynamically-consistent inverse 
47 

- ellipsoid 393 

- extended 228 

- geometric 219 

- loop 59 

-matrix 113,935,1010 

- task 219 

- time-derivative 396 
jamming 888 

Japanese experiment module remote 
manipulator system 1334 
jigsaw positioning system 1442 
jogging companion 1603 
Johnson Space Center 1776 
joint 

- action 1749, 1785 

- activity 1753 

- architecture for unmanned systems 
283 

- attention 1752 

- coordinate frame 48 

- directors of laboratories 831 

- drive gain 121 

- elasticity 241 

- intention theory 1750 

- model 48 

- parameter 26 

- persistent goal 1753 

- probability distribution 820 

- reference 1770 
-torque 119,1671 

- torque feedback 252 

- torque sensor 193, 254, 454 
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jumping microrobot 533 

just noticeable difference 1005 
just-in-time 1397 

K 


Kalman filter 97, 129, 625, 823, 
1070, 1188, 1610 
- unscented 1445 
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Karel the robot 1932 
key-poses 1654 
Khepera 1858, 1871, 1937 
Kilobot 516 
Kinect 1669 
kinematic 427, 1010 

- chain topology 388 

- conditioning 1007 

- constraint 192, 549 

- contact 880 

- coupling 1029 

- dissimilarity 1036 

- duality 889 

- equation 71 

-inverse 29,430,857,1317 

- loop 39, 48 

- redundancy 217 

- singularity 220 

- structure 993 
-tree 39,51,121,784 
kinematic algorithms 

- inverse 225 
kinematic calibration 111 

- closed loop 113 
-index 116 

- open loop 113 

- sensor index 116 
kinesthetic 1003 

- display 1014 
kinetic energy 43 

- of motor 242 

- recovery system 485 
kinodynamic planning 143 
k-rnean 781 

^-means clustering 101 
fc-nearest neighbor 749 
knowledge 

- assertional 323 

- base update 324 

- representation 320 

- representation (KR) 320 

- terminological 323 
Koryu robot 464 
Kullback-Leibler divergence 

L 


Lagrange 

- dynamics 160 

- equation 243 

- formulation 46 
-multiplier 202,1011 


Lagrangian 241 
-dynamics 1127 
-formulation 515 
laminar damper module 487 
landmine 1415, 1417, 1418 
LANdroid 1472 
lane 

-changing 1516 
-keeping 1515 

- tracking 1509 
Laplacian of Gaussian 745 
LaSalle’s theorem 165, 250 
laser 

- distance sensor 1607 

- imaging radar 1488 

- measurement system 742 

- radar 740 
-ranging 1341 

- scanner 92 
laser-based SLAM 1610 
latent 

- Dirichlet allocation 781 

- support vector machine 787 
lateral 

- geniculate nucleus 1917 

- intraparietal sulcus 1906 
-slip 1188 

lattice reconfiguration 517 
lawn mowing 1602 
layer 789 
layered 

- architecture 1257 

- molding 536 

lead lanthanum zirconate titanate 
484 

lead zirconate titanate 484, 636 
leader-follower 943 
leading edge vortex 617 
leaf belief 991 

learning 301, 787, 964, 1278, 1802 

- and adaptation 1702 

- applied to ground robots 1486 
-architecture 351 

- behavior-based 302 

990 ^ by demonstration 308 

- compound action 1828 

_ - control 184 

- control for assembly 861 

- feature 967 

- from demonstration 1824 

- from history 308 

- from human demonstration 1821 

- in mobile manipulation 968 


- individual motion 1827 

- reinforced 306 
learning from humans 1831 

- algorithm 1827 

- correspondence problem 1825 

- evaluation metric 1824 

- history 1822 

- imitate 1824 

- key issues 1824 

least squares estimation 110 
leg-arm hybrid robot 419 
legged crawling 525 
legged locomotion 959 
-bio-inspired 1846 
-model 1844 

- stability and gait 1846 
legged robot 1471 
leg-wheel hybrid robot 418 
Leonardo Da Vinci 1205 
level of biomimicry 525 
Lewellyn’s absolute stability 

criterion 1013 
LIDAR sensor 1198 
Lie algebra rank condition 1104 
Lie-group-algebraic method 429 
life-cycle-costing 1304 
life-like robot 525 
lift force 1209 
lifting line method 599 
lift-to-drag 1485 
light detection and ranging 670, 
740, 1189, 1443 

light-emitting diode 741, 1521, 
1607, 1939 

lighter-than-air system 593 
light-weight 

- communications and marshalling 
579 

- manipulator 240 
-robot 240,481,1684 

- structure 1687 
likelihood function 820 
Likert 1944 

limb prosthetic device 1574 
limbed system 408 
limit cycle 1132,1147 

- walking 413 
linear 

- complementarity problem 1131 

- constraint satisfaction program 
887 

- control design 256 

- elasticity 241 

- impedance 493 
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-program 915,1146 

- quadratic Gaussian 807 

- quadratic regulator 271, 496, 611, 
1140 

- temporal logic 328 

- variable differential transformer 
117 

linear inverted pendulum model 
414 

linearization feedback 168, 253 
linearizing coordinates 254 
line-of-sight 1492 
link 1939 

- equation 243 

- parameter 26 
linkage model 

- virtual 945 

liquid-crystal display 289, 1768 
lithium 

- polymer 624 

- primary battery 569 
livestock 

- breeding 1380 
-exploitation 1381 
-harvesting 1381 
-nurturing 1380 
Llewellyn criterion 1032 
load 

- capacity 68 

- distribution 941 

- haul-dump 1435 
-parameter 119 

- sharing coefficient 941 
local 

- and small-time controllability 
1103 

- area network 514, 577, 1046, 
1329, 1492 

- autonomy 1027 

- learning method 356 

- minimum 866 

- potential function 867 

- product-of-exponential 514 

- regression 356 

- steering method 1105 
locality constrained linear coding 

782 

localization 1216, 1265, 1445 

- map-based 1445 

- sound source 311 
locomotion 470 

- mechanism 1598 

- performance 526 


logic 

- based reasoning 321 

- description 321 

- first order predicate 321 

- propositional 322 
logical sensor system 833 
logistic regression 966 
long-baseline 575 

- system 1216 

longitudinal stability margin 419 
long-range cruising AUV 569 
long-term 

- behavior 1805 

- depression 1896 

- interaction 1766, 1779 

- social interaction 1805 
lookahead 991 

loop 

- closure 430 

- closure constraint 58 

- haptic 1004 
loss function 788 
low 

- Earth orbit 1334 

- level control 1702 

- power consumption 1687 

- uncertainty 985 
low-cost sensor 1594 
lower 

- extremity nonanthropomorphic 
robot 1566 

- extremity powered exoskeleton 
1565 

- pair 21 
Lunokhod 1336 
Lyapunov function 249-251 

M 


Mach model 595 
machine 

- interface 1491 

- learning 1823 

- safety standard 1314 

- tool 439 

- vision 1446, 1867 
macro arm 1342 
macro-micro manipulator 1334 
magazining 1395 

magnetic resonance imaging 1528, 
1537, 1685 

magnetoencephalography 1563 
magnetorheological 486, 1426 


magnetostrictive 484 
Mahalanobis distance 753, 984 
maintenance robot 1402 
manage attention 1757 
maneuverability 606 
manifold 
-learning 791 

- particle filter 988 
manipulability 1008, 1345 

- dynamic 1008 

- measure 221 

manipulation 449, 470, 680, 847, 
848, 953, 981, 1217 

- admissible configuration 855 

- cooperative 1280 

- grasp configuration 856 

- grasp planning 869 

- inverse kinematics 857 

- multiple part 870 

- multiple robot 870 

- nonprehensile 870 

- pick-and-place task 854 

- planning 145 

- stable part configuration 856 

- task specification 856 

- three DOF manipulator example 
856 

- transit path 855 

- workspace goal 858 
manipulation aid 1568 

- mobile autonomous system 1569 

- wheelchair manipulator 1569 
manipulator 

- constraint 865 

- cooperative 933 

- Jacobian 495 

- stiffness 396 
manmade 

- disaster 1463 

- event 1467 
man-packable UAV 1465 
manual 

- control 1024 

- flight 621 
ManuBuild 1392 
manufacturing 1006, 1302 

- and positioning 1398 

- process 1302 
map 

- digital 1507 

- geometric 1067 

- grid 1067 
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mapping 756, 1216, 1608 

- moving object 970 

- passive 175 
marine robotics 1203 
Markov 

- decision process 324, 361, 994, 
1754 

- decision process (MDP) 1270 

- process 983 

- random field 772 
Marr-Albus model 1895 

Mars environmental survey 1340 
Mars exploration rover 1341 
Mars Pathfinder 1340 
Mars rover sample return 1339 
Mary 

- phase shift keying 578 

- quadrature amplitude modulation 
578 

masonry wall erecting robot 1399 
mass modal 263 
Massachusetts Institute of 
Technology 1126 
mass-spring system 994 
master-slave 

- manipulator 1419, 1425 

- system 1024 

material handling robotics 1403 
maximally stable extremal region 
781 

maximum a posteriori 96, 785, 
1491 

maximum likelihood estimate 95, 
721, 1265 
max-pooling 783 
M-blocks 516 
McKibben muscle 1898 
mean time between failures 85, 
1304 

mean-shift 784 
measurement 

- and signatures intelligence 1488 

- model 983, 984 
mechanical 488 

- adjustable compliance and 
controllable equilibrium position 
actuator 487 

- impedance adjuster 487 
-interface 517 

- motion capture 1668 
mechanism 1776 

- isotropy 1008 

- passive 1424 

- synthesis 429 


- toothless gear 532 
-used for HRI 1783 
mechanization 1390 
mechanoreception 674, 1016 
mechatronic robot 1687 
medial intraparietal sulcus 1906 
medial temporal area 1921 
medical 

- image segmentation 1534 

- robot 998, 1528 

- simulation 1006 
memory 1757 

- alloy 484 

mental inference 1773 
mental model 1741,1757 

- robot 1742 

mental perspective-taking 1771 
mesencephalic locomotor 528 
mesencephalic locomotor region 
528 

M-estimator 100 
metal 

- corrosion 567 

- matrix composite 485, 567 
metrics 1275 

micro 

- aerial vehicle 536 

- arm 1342 
-robot 1850 
microcomplex 1895 
microelectromechanical system 

(MEMS) 81, 265, 404, 483, 533, 
572, 632, 675, 696, 716, 814, 

1016, 1226, 1274, 1449, 1539 
microfabrication 638 
micromechanical flying insect 537 
micromouse competition 1618 
microrobot technology 525 
Microsoft robotics developers studio 
580 

microspine array 530 
microsurgery robot 1538 
microsystem technology 647 
microzone 1895 
middle-size league 1624 
middleware 283, 579 

- for robot 580 
millibot 518 

millimeter/centimeter scale crawler 
526 

mindreading 1770, 1772 
mine crawler robot 1482 
mine-permissible robot 1482 
minima-free potential function 867 


minimally invasive surgery 1531 
minimum 

- description length 784 

- mean-square error 96 

- shift keying 578 
mining 1433, 1447, 1452 

- coal 1447 

- disaster 1467 

- dragline 1441 

- ocean floor 1452 

- open-pit 1441 

- robotics 1433 

- shovel 1440 

- stages of 1434 

- surface 1434 

- system 1450 

- underground 1434 
mirror 

- image motion enabler 1556 

- image movement enhancer 1561 
-neuron 1771,1899 

- neuron system 1900 
-system 1771,1887,1903 
misconception 

- behavior-based 303 
mission 625 

- control system 1214 

- oriented operating suite 580 
mixed model 350 

mixture model 787 
mobile 

- base system 1333 

- beacon 1462 

- detection assessment and response 
system 1491 

- domestic robotic 1592 

- manipulation 951 

- manipulator 959 

- navigation 1486 

- platform 958 

- repeater 1472 
mobile robot 1253,1310 

- competition 1460 

- nonholonomic 1159 

- telerobotics 1036 
mobility 58, 548 

- and manipulation 852 

- degree of 551 

- equation 116 

- restriction 552 
mobility aid 
-exoskeleton 1571 
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- walking assistance system 1571 

- wheelchair navigation system 
1570 

modal 

- mass 263 

- stiffness 264 

- vectors 398 
mode 986 

- contact 882 

- identification and recovery 281 

- shape 470 
model 

- based controller 471 

- based method 346 

- behavioral 1747 

- checking 323 

- cognitive 1747, 1749 

- contact 879 

- developmental 1747 

- dialog-based 1749 

- driven engineering 293 
-environmental 91 
-learning 1756 

- of emotion 1768 
-physical 1749 

- predictive control 349, 622, 1137, 
1236 

- property 1210 

- reference adaptive control 176, 
349 

-uncertainty 369,1192 
model learning 348 

- architecture 351 

- method 354 
modeling 606, 1205 

- actuator 487 

- biological system 1844 

- deformable terrain 1188 

- first-order form closure 914 

- of locomotion 472 
-rubble 1189 

- soft robot 490 

- tracked vehicle 1195 
modular 

-block 1395 
-manipulator 513 

- mining system 1436 

- robot configuration 514 
modularity 1395 
modulation friction 1016 
modulation-based range sensor 741 
molecular biology 1006 

molten carbonate fuel cell 569 


moment 606 

- labeling 885 

- representation 597 
momentum 

- conservation 596, 1344 

- spatial 41, 42 

- theory 600 
monitor 288 
monitoring system 1576 
monolithic fabrication 538 
Monte Carlo method 364, 986 
moon buggy 1336 
Moore-Penrose inverse 169 
mossy fiber 1895 

motion 

- capture 1669 

- category 1673 

- constraint 849 

- effect 728 

- generation 960 

- human-like style 1667 

- instability 621 

- primitive 1673 

- requirement 961 

- transmission 449 

- whole-body 1144 

motion control 160,1157,1317 
-performance 1188 
motion execution 

- compliant 865 

motion planning 470, 961, 1141 

- compliant 862 

- feedback 866 

- feedback motion planning 866 

- problem 434 

motion-oriented operating system 
1214 

motivated behavioral architecture 
311 
motor 

- equation 243 

- evoked potential 1904 

- feedback 249 
-neuron 1671 
mouse 1004 

move value estimation for robot 
teams 310 

movement therapy 1558 
moving observation 

- of a comer 729 

- of a plane 729 

- of an edge 730 
moving plate 400 
moving plate (MP) 389 


moving target indicator 1487 
multiappendage robotic system 403 
multiarm system 943 
multicellular organism 1843 
multifingered manipulation 938 
multifunctional satellite 
augmentation system 703 
multihypothesis tracking 1511 
multijoint 
-bending 531 

- model 164 

multilateral telerobotics 1037 
multilevel surface map 757 
multimode transit 1520 
multipass effect of wheels 1188 
multiphalanx hand exoskeleton 
1717 
multiple 

- beam scanning LIDAR 741 

- input-multiple-output 173, 254, 
610, 1237 

- kernel learning 783 

- layered composite 538 

- master multiple-slave 1037 

- master single-slave 1037 

- material 536 

- model switching adaptive estimator 
271 

- operator multiple robot 1044 

- operator single robot 1044 

- paired forward-inverse model 
1897 

-reflection 717 

- resource host architecture 1492 

- underwater vehicle 1219 

- view registration 753 
multipulse sonar 726 
multipurpose system 1843 
multiresolution mapping 970 
multirobot 1254, 1281 

- coordination 281 

- path planning 1281 
-system 304,514 

- system behavior-based 305-307 

- task 1276 

- task allocation 1276 
multisensor 

- data fusion 819 

- environment modeling 835 

- fusion architecture 831 

- gripper 1334 

- system control dynamic 835 
multistep planning 990 
multitarget observation 1281 
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multitask 1276 
multitracked vehicle 1197 
muscle 

-length 1671 

- like module 485 

- spindle 1005 
-tension 1671 

musculoskeletal model 1670,1671 
musculoskeletal walking model 
1720 

mutual belief 1786 

N 


naive Bayes 781 

nanoelectromechanical system 633 
nanorobotic manipulator 651 
NASA/NBS standard reference 
model 281 

national livestock identification 
scheme 1380 

national qualifying event 1633 
national robotics initiative 1567 
nationwide different GPS system 
704 
natural 

- constraint 205 

- disaster 1463 

- frequency 199, 260, 398 

- human cue 1757 

- machine motion initiative 503 

- muscle 482 

- orifice transluminal surgery 1531 
navigation 756, 1445, 1849 

- and obstacle avoidance help 1570 

- function 867 

- in rat and robot 1890 

- underground 1445 

nearness diagram navigation 1115 
nearness diagram navigation (ND) 
1112 

needle placement robot 1537 

negative information 985 

negotiation 1276 

Nerd Herd 1255 

Nereus 576 

network 

- data distribution service 283 

- mobility 1508 

- partitioning 1267 

- real-time kinematic 704 

- server 1048 

- topology control 1267 


network data distribution service 
283 

networked 1253 

- infomechanical systems 1265 

- mobile robot 1258 

- robot 1258, 1259, 1472 
neural interface 1574 

neural network 789, 1858, 1879 

- GasNet 1861, 1872 

- neural oscillator 1860 

- recurrent neural network 1866, 
1867 

- spiking neural network 1871 
neurobiological system 1888 
neurocontrol 1894 
neuroethology 1887 
neuromorphic engineering 1849 
neuron 789 
neuroprosthetics 1852 
neurorobotics 1888 

neutral buoyancy 1335 
Newton-Euler 

- equation 44, 117 
-formulation 515 
Newton-Raphson 431 

- algorithm 431 
IV-gram model 1673 

nickel metal hydride battery 412 
NIST response robot evaluation 
exercises 1477 
nonaccidental property 1916 
nonacoustic sensor 572 
noncontact therapy robot 1553 
nonholonomic 1344 

- constraint 1102 

- mobile robot 1159 

- planning 143 
nonholonomy 1128 
nonlinear 

- dynamic inversion 611 

- feedback 254 

- force function 488 

- mechanical system 488 

- optimal control 177 

- optimization convergence 1347 

- programming problem 1346 
nonmaxima suppression 787 
nonminimum-phase 248 
nonnegative matrix factorization 

1895 

nonneural element 1844 
nonparametric method 985 
nonprehensile manipulation 879, 
896 


non-RT thread (real-time thread) 

581 

nonslip condition 549 
nontangential proper part 331 
nonuniform rational B-spline 1012 
nonverbal 

- communication 1775 

- cue 1752 
norm 

- communication 1744 

- social 1744 

normal distributions transform 970 
normal vector 1012 
normalized energy stability margin 
(NESM) 1197 
normalized ESM 1189 
nuclear 

- decommissioning 1421 

- magnetic resonance 1965 

- operation 1025 
-radiation 1412,1421 
null-space 

-projection 851,852 

- velocity 221 
numerical control 1313 
nursery and greenhouse (N&G) 

1377 

nurturing 1380 
Nyquist criteria 1013 

0 


Oberon 1212 
object 

- anchoring 321 

- class detection 786 

- class recognition 779 

- detection 786 

- identity resolution 339 

- learning 1919 

- localization 982 
-part 791 

- recognition 1917 
-reorientation 991 

- representation 1915, 1921 

- request broker 283 

- shape matching 996 

- transportation 1280 
objectness 789 
Oblix/Mogura mechanism 464 
observability index 111,126 
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observation 

- class ROV 571 

- update 821 
obstacle 758 

- region 136 
obstacle avoidance 470 

- path modification 867 

- problem 1111 

obstacle restriction method 1113 
obstacle restriction method (ORM) 
1111 

occupancy grid map 993 
occupancy map 970 
oceanic engineering 1203 
octarm manipulator 467 
octopod 439 
octopus-arm 466 
odometry 756, 1337, 1450 
offline programming 1308 
off-policy method 362 
off-read locomotion 1338 
off-road vehicle 1190 
offsite robotics 1395 
olive 1895 
omnidirectional 

- camera 92 

- robot with steerable wheel 555 
omnimobile robot 554 

on board unit 1508 
on real-time communication 1471 
onboard autonomous science 
investigation system 282 
onboard integration 623 
onboard power source 569 
one-step lookahead 989 
one-way bearing 533 
online 

- gravity compensation 251 

- programming 1317 
onsite robotics 1399 
on-the-fly target classification 726 
ontological category 1744 
ontology 323 

- based unified robot knowledge 
334 

open 

- agent architecture 1754 

- dynamics engine 580 

- platform for robotic service 580 

- roboethics initiative 1972 
open robot 

- control architecture 580 


- control software 292, 580 

- controller computer aided design 
279, 282 

operating 

- point 392 

- system 1046 
operational 

- frame 850 

- point 850 

- therapy robot 1552 
operational space trajectory 180 
operational-space 45 

- control 162, 358 

- inertia matrix 39, 56 
operator 

- control unit 284 

- model 348 

optic flow 1871,1889 
optical 

- coherence tomography 1530 

- microscope 637 

- microscope (OM) 652 

- motion capture 1668 

- quadrature encoder 1008 

- underwater communication 579 
optimal 

- arbitrary time-delay 266 

- control 500 

- coverage path planning 1369 

- design 436 

- task assignment 1437 
optimization 1008 
optokinetic response 1659 
orbital replacement unit 1333, 1335 
orbital replacement unit (ORU) 

1333 

orchard 1367 
ordinary differential 
-equation 488,1137,1208 
-inclusion 1128 
orientation 13 

- absolute 939 
-quaternion 1198 

- relative 939 
oriented gradient 786 
ornithopter 617 
orthopaedic surgical robot 1536 
Oswald 

-coefficient 1231 

- efficiency 599 
out of field 1752 
outer 

- loop control 168 

- poles 170 


overfitting 789 
over-the-horizon 1492 
Oxford intelligent machine 1555 

P 


packaging line 1305 
packet switching 1046 
paddle 

- haptic 1009 
painting 1312, 1402 
-robot 1312 

palm tree spraying 1374 
pan-tilt unit 289, 1440 
pantograph 409 
parallel 
-fiber 1895 

- force/position control 193,211 

- kinematic machine 400, 1303 

- manipulator 427 

- manipulator calibration 115 

- mechanism 12 

- robot 73, 122, 427, 1303 

- tracking and mapping 1092 
parameter 

- drift 177 

- estimate 173 

- inertial estimation 117 
parameterization of track 1195 
parametric 

- bias 1903 

- design 430 

- force 607 

- model 607 

parking assistance 1515 
part-based model 780 
partially observable Markov decision 
process 326, 963, 989, 1756, 
1829 

partial-order planning 338 
particle 986 

- filter 988, 1610 

PASCAL visual object class 789 
passenger protection 1518 
passive 

- action recognition 1257 

- compliance 860 

- dynamic walker 1860 

- dynamic walking 412,1126, 

1147 

- gripper 534 

- mapping 175 

- mechanism 1424 
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- set-position modulation 1034 

- stiffness 435 

- suspension dynamics 559 
passivity 175, 210, 248, 1013, 1031 

- based control 169 

- controller 1034 

- observer 1034 

patchwork of primitives 992, 993 
path 179 

- consistency 329 

- deformation 1118 
-following 1159,1162,1193 

- following algorithm 1194 

- planning 179 
Pathfinder 390 
pattern generator 1848 
pattern-based mixed-initiative 1750 
payload 620 

- delivery 624, 1249 

- sensor 1486 
pedestrian detection 1511 
peg-in-hole 205 

- problem 888 

- task 848 

PEIS Ecology project 334 
people detection 784 
perception 89, 299, 311, 969, 1262 

- active 1266 

- for action control theory 1904 

- for off-road robotics 1486 

- process 90 

- via-manipulation 998 
periodic motion 

- control 497 

- tracking 497 
peripheral 1302 

- component interconnect 576, 644 
peristaltic waves 473 
persistency of excitation 175 
personal roving presence 1045 
perspective taking 1750, 1772 

- spatial 1752 

- visual 1752 

Petri net transducer 280 
Phantom haptic device 1007 
phase shift keying 578 
Phoenix 1208 
phosphoric acid fuel cell 569 
physical 

- damper 487 

- embodiment 1773 

- human-robot interaction 1642, 
1679 


- interaction 481 

- vapor deposition 639 
physically collocation 240 
piano mover’s problem 136 
pick-and-place manipulation 854 
pictorial structure 784 
piecewise polynomial 180 
piezoelectric 715, 1016 

- actuator 484 
pin in hole 1339 
pinhole camera model 93 
pipe clamp 887 

place cell 1869, 1870 
placing object 953 
plan 

- execution interchange language 
279 

- partial-order 338 
planar joint 24 
plan-based control 335 
planetary exploration 1452 
planner 288 

planning 287,300-302 

- and control 847, 962 

- behavior-based 304, 305, 307, 
311 

- closed kinematic chain 870 

- configuration space obstacle 857 

- feedback 866 

- for hybrid system 854 
-horizon 991 

- manipulation 854 

- movable obstacles 870 

- nonholonomic 143 

- self-collision 857 

- trajectory 144 

- under uncertainty 862 
plant probing 1375 

pleasure arousal dominance 1768 
Pliicker coordinates 20, 40 
plug-and-play 512 
pneumatic 483 

- actuator 483 

- artificial muscle 483 

- continuum manipulator 467 

- McKibben muscle 467 

- network 539 
Poincare map 1126,1147 
point 

- algebra 328 

- contact 890 

- estimation 95 
-measurement 113 

- of interest 1307,1697 


point cloud 993,1196 

- library 1092 
point feature 749 

- histogram 750 
point-contact 904 
pointing 

- declarative 1752 

- imperative 1752 
point-to-point (PTP) 283 
pole placement 255 
policy gradient 

- reinforcement learning 1810 

- theorem 365 
policy iteration 326 
polygonal mesh 984, 992, 1011 
polymer 484 

- electrolyte fuel cell 569 

- matrix composite 485 

- MEMS 534 
polymeric actuator 485 
poly-mesh 992 
polynomial trajectory 180 
poly oxy methylene 567 
polypod 

- robot 464 

- system 1271 
polytope 396 

polyvinyl chloride 567, 1937 
polyvinylidene fluoride 484, 636, 
675, 715 

Pontryagin’s minimum principle 
500 

pool cleaning 

- robot 1600 
pop-up book 

- inspired design 538 

- MEMS 538 
port-based approach 1035 
pose 

- estimation 754 

- normalization 790 

- tracking 985 
pose-based visual servo 803 

- control 796 

- interaction matrix 804 

- stability analysis 804 
poselet 784 
position 

- absolute 939 

- control of subtrack 1199 

- force architecture 1031 

- localization 1506 

- observer 1247 

- relative 939 
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- sensing device 678 

- velocity 164 

positional dilution of precision 702 
positioning accuracy 1308 
position-position architecture 1030 
position-sensitive-device 1607 
positive photoresist 639 
positron emission tomography 
1528, 1965 
possible 
-failure 1218 

- injury 1685 
posterior 

- distribution 983 

- inferotemporal cortex 1918 
pot handling 1377 
potential 

- damping 1208 

- field 1890 

- field method 1112 

- function 285, 866 
power 

- consumption 1406 

- data grapple fixture 1333 

- loading 594 

- scaling 1030 

- source 647 

- supply 620 

- system 569 

power-law equation 890, 895 
power-to-weight ratio 1009 
precise positioning system 700 
precision 

- forestry 1378 
-irrigation 1372 
predator and prey 1876 
prediction step 821 
predictive graphic display 1334 
prefabrication 1389 
prefrontal cortex 1899,1917 
preimage 862 
prespective-n-point 765 
pressure 

- distribution 891 

- hull 566 

- sensor 573 

- sinkage equation 1190 

- vessel 566 
pretectum 1890 
principal 

- behavior-based 302 
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-mapping 1189 
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- mechanism 1598 
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- range sensor 740 
-ranging 718 

- sensor 1375 
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tire model 559 
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- algorithm 1489 
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trigonometric trajectory 181 
trim condition 605 

tripod gait 526 
truck spotting 1440 
truss grasper 534 
Tustin algorithm 183 
twist 881 

- end-effector 392 
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ultrasonic 1016 
ultrasound imaging 1537 
ultraviolet 638 
ultrawide band 622 
umbilical cable 571 
uncanny valley 1743, 1767 
uncertain environment 1843 
uncertainty 324,437 
unconditional stability 1032 
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- manipulator system 1217 
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- resource locator 1050 
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-testbed 1191 

- vehicle slippage 1193 
wheel traction 1351 
-characteristics 1191 
-performance 1193 
wheeled mechanism 1424 
wheeled mobile robot 547, 1036, 

1157, 1187 
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- demographic 1553 

- graph 1892 

- wide web 1046 
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